Skip to main content
Start a Conversation

Unsolved

This post is more than 5 years old

DavidHampson

1.1K Posts

8548

June 5th, 2009 09:00

Savesets not expiring - the retention time is invalid

Here is another strange one - I have one client which has a month browse/retention which has savesets that go back to February that have not expired - no dependency here and generally they are the only saveset on the tapes meaning they are holding up tape reuse - here is some mminfo data:

mminfo -q "client=s-cpp-cmfe01.net1.cec.eu.int,name=C:\\" -ot -r volume,savetime,ssretent,level,clretent,ssflags
...2B1517 02/02/09 03/02/09 full 03/02/09 vF
2B1538 02/03/09 03/03/09 full 03/03/09 vF
2B1541 02/03/09 03/03/09 full 03/03/09 vF
2B1535 02/04/09 03/06/09 full 03/04/09 vrEF
2B1624 02/05/09 03/06/09 full 03/05/09 vrEF
2B1587 02/06/09 03/07/09 full 03/06/09 vrEF
2B1571 02/06/09 03/07/09 full 03/06/09 CvrEF
2B1571 02/06/09 03/07/09 full 03/06/09 CvrEF
2B0011 02/07/09 03/08/09 full 03/07/09 vrF
2B0956 02/07/09 03/08/09 full 03/07/09 vrF

A93674 02/07/09 03/08/09 full 03/08/09 vrF

2B0257 02/08/09 03/09/09 full 03/08/09 vrF
A93582 02/08/09 03/09/09 full 03/09/09 vrF
2B0210 02/09/09 03/11/09 full 03/09/09 vrF
A92929 02/09/09 03/11/09 full 03/11/09 vrF
2B0202 02/09/09 03/11/09 full 03/09/09 vrF
A91516 02/09/09 03/11/09 full 03/11/09 vrF
2B0506 02/10/09 01/19/38 full 01/19/38 vKF
A91526 02/10/09 01/19/38 full 03/11/09 vKF
2B0217 02/10/09 03/12/09 full 03/10/09 vrF
A91077 02/10/09 03/12/09 full 03/12/09 vrF
2B1163 02/12/09 02/12/09 full 03/12/09 vrF
2B0685 02/23/09 03/24/09 full 03/23/09 vrF
A94073 02/23/09 03/24/09 full 03/24/09 vrF
T00602 02/24/09 05/01/09 full 05/01/09 vF
etc (from here the same all the way down)

I looked at a specific piece of media and ran nsrim -V volume to see if it marked the tape as recyclable - some of the messages which may be important are:

-----------------------
s-net1luxdc88:G:\, 6 browsable cycle(s)10066:nsrim: Failed updating continuation save set ssid `3707290228' as purged
39077:nsrim: error, Save set ssid `3707290228' the retention time is invalid

10067:nsrim: Failed updating continuation save set ssid `3707290228' as eligible
39077:nsrim: error, Save set ssid `3707290228' the retention time is invalid

9335:nsrck: WARNING: no valid save times for client 's-net1luxdc88' - cross check not performed

A93674: 308 GB used, 7 save sets, full, 1 recoverable save sets, 6 recyclable save sets
-----------------------
Note the saveset 3707290228 is not on this tape though the saveset name is and has been marked as recyclable; also all savesets for s-net1luxdc88 on this media are recyclable. These messages may therefore be red herrings?


Next I ran nsrck -L6 s-cpp-cmfe01.net1.cec.eu.int which went through as expected:

nsrck: checking index for 's-cpp-cmfe01.net1.cec.eu.int'
nsrck: /nsrsrv30/nsr/index/s-cpp-cmfe01.net1.cec.eu.int contains 3485803 records occupying 457 MB
nsrck: Completed checking 1 client(s)

And another nsrim with no effect:

A93674: 308 GB used, 7 save sets, full, 1 recoverable save sets, 6 recyclable save sets

The saveset in question here was taken 7 February with month browse and retention and with successful full backups taken afterwards so what could be preventing this saveset expiring?

DavidHampson

1.1K Posts

June 5th, 2009 09:00

Another thing, there is another saveset here which has also not expired (and spans two tapes):

ssid clone id volume ssflags retent clretent
3264077411 1234034275 2B0011 vrF 03/08/09 03/07/09
3264077411 1234034275 2B0956 vrF 03/08/09 03/07/09
3264077411 1234098790 A93674 vrF 03/08/09 03/08/09

Anonymous

5 Practitioner

 • 

274.2K Posts

June 5th, 2009 13:00

Make sure you don't have media database corruption. In the daemon.log or in daemon.raw ( newer NetWorker versions) you should not see a text named "WISS". This error is seen, when you start the services. If you see this text, then this indicates the media database is corrupted. If it's corrupted, it needs to be fixed first.( scavange the media database)

on the NetWorker server run the command "nsrim -X" this will correct any issues in the media database.

make sure the pool doesn't have a longer retention policy. The Newer versions of NW server have a new option where the pools can have their own retention policy.

DavidHampson

1.1K Posts

June 8th, 2009 08:00

The last time I saw a WISS error was Networker 6.22! There does not appear to be any indication of media database errors, I've already ran nsrim -X and nsrck -L6 against this client. I'll see if I can come up with any further information...

DavidHampson

1.1K Posts

June 8th, 2009 08:00

There is no retention policy set on the pool.

DavidHampson

1.1K Posts

June 8th, 2009 08:00

I also tried recovering the indexes (successfully) then running nsrim -V against one of the effected tapes - however saveset still comes up as being browsable.

I'm going to see what happens if I stage the data onto a different tape....

DavidHampson

1.1K Posts

June 9th, 2009 01:00

Staged to another tape, run nsrim -V on that tape, still not expired. Very confusing:

7 root@bkpsrv30:->nsrstage -b T10KBKP030 -m -S 2072461207/1233600407
6365:nsrstage: Space can only be recovered from adv_file and file type devices.
8 root@bkpsrv30:->mminfo -q ssid=2072461207
volume client date size level name
094872 s-cpp-cmfe01.net1.cec.eu.int 02/02/09 8458 MB full C:\
9 root@bkpsrv30:->nsrim -V 094872
...
Cross check completed successfully

->mminfo -q ssid=2072461207 -r ssflags,ssretent
ssflags retent
vF 03/02/09

DavidHampson

1.1K Posts

June 9th, 2009 09:00

Now this is interesting - here is a random affected saveset:

mminfo -q "client=s-cpp-cmfe01.net1.cec.eu.int,name=C:\\" -ot -r volume,savetime,ssretent,level,clretent,ssflags -r ssid,cloneid,ssflags,savetime,ssretent,clretent

094872 02/02/09 03/02/09 full 03/02/09 vF 2072461207 1244477590 vF 02/02/09 03/02/09 03/02/09

I reset the retention period of the clone as follows:

nsrmm -e 03/02/09 -S 2072461207/1244477590

And now we see this:

094872 02/02/09 03/02/09 full 04/08/45 vF 2072461207 1244477590 vF 02/02/09 03/02/09 04/08/45


I had a similar issue going from 7.3.3 to 7.4.3 at my old company and I came up with a workaround for it. I'm going to check on my laptop when I get home if I have a record of it and also I have a call with EMC to check my old cases!

ble1

14.3K Posts

June 10th, 2009 03:00

What version do you use now?

DavidHampson

1.1K Posts

June 10th, 2009 04:00

This is Networker 7.4.4 - the savesets affected were written with the same version of Networker too and not subjected to any upgrade.

Today I've tried removing some of the earliest savesets (successfully) - both expired and should be expired; this had no effect on any existing backup after nsrim.

EMC's response has been to do a media scavenge; since this requires the server to be brought down we will not action this for a while...

ble1

14.3K Posts

June 10th, 2009 11:00

I don't remember the details, but in 7.4.4.3 some of the issues with clretent have been resolved - can you try that patch? Or even 7.4.4.4....

DavidHampson

1.1K Posts

June 11th, 2009 01:00

I'll take a look at the release notes and see if we can justify an upgrade; I have a call logged with EMC so I will see if they can also offer advice on this.

DavidHampson

1.1K Posts

June 22nd, 2009 02:00

I've returned to looking at this one - some more interesting information. So looking at one of these tapes:

nsrim -V A92944

returns:

Cross check completed successfully

A92944: 384 GB used, 233 save sets, full, 9 browsable save sets, 224 recyclable save sets
62173:nsrim: nsrim has finished at Mon Jun 22 11:47:56 2009.

mminfo -q volume=A92944 -r client,name,savetime,level,ssbrowse,ssretent,clretent,ssflags,clflags|grep -v clretent|wc -l

224
mminfo -q volume=A92944 -r client,name,savetime,level,ssbrowse,ssretent,clretent,ssflags,clflags|grep vrEF|wc -l

223
mminfo -q volume=A92944 -r client,name,savetime,level,ssbrowse,ssretent,clretent,ssflags,clflags|grep -v vrEF

client name date lvl browse retent clretent ssflags clflg
s-cc-rwp01 E:\ 02/23/09 full 03/23/09 03/23/09 03/23/09 vrEiF E

So the "9 browsable savesets" which seem to be stopping this tape recycling do not get listed as being in the media database.

DavidHampson

1.1K Posts

June 22nd, 2009 03:00

So what I am going to look at now is deleting the tape and scanning in then running the same queries again...

DavidHampson

1.1K Posts

June 24th, 2009 02:00

Scanning the tape back in discovered an interesting result; the savesets returned by mminfo matched the recyclable tapes but the nsrim -V volume seemed to be referencing ana additional 9 browsable savesets that were not listed in the mminfo output.

I'm now running a media scavenge on this box and seeing what results this gives.

DavidHampson

1.1K Posts

June 24th, 2009 02:00

Ran the media scavenge successfully; looked at tape A92944 again - ran nsrim -V then checked the savesets:

Cross check completed successfully

A92944: 45 GB used, 67 save sets, read-only, 9 browsable save sets, 58 recyclable save sets


124 root@bkpsrv30:->mminfo -q volume=A92944|wc -l
59
125 root@bkpsrv30:-


We still appear to be referencing the phantom savesets....?

View More

View All

No Events found!

Top