8 Krypton

Savesets not expiring - the retention time is invalid

Here is another strange one - I have one client which has a month browse/retention which has savesets that go back to February that have not expired - no dependency here and generally they are the only saveset on the tapes meaning they are holding up tape reuse - here is some mminfo data:

mminfo -q "client=s-cpp-cmfe01.net1.cec.eu.int,name=C:\\" -ot -r volume,savetime,ssretent,level,clretent,ssflags
...2B1517 02/02/09 03/02/09 full 03/02/09 vF
2B1538 02/03/09 03/03/09 full 03/03/09 vF
2B1541 02/03/09 03/03/09 full 03/03/09 vF
2B1535 02/04/09 03/06/09 full 03/04/09 vrEF
2B1624 02/05/09 03/06/09 full 03/05/09 vrEF
2B1587 02/06/09 03/07/09 full 03/06/09 vrEF
2B1571 02/06/09 03/07/09 full 03/06/09 CvrEF
2B1571 02/06/09 03/07/09 full 03/06/09 CvrEF
2B0011 02/07/09 03/08/09 full 03/07/09 vrF
2B0956 02/07/09 03/08/09 full 03/07/09 vrF

A93674 02/07/09 03/08/09 full 03/08/09 vrF

2B0257 02/08/09 03/09/09 full 03/08/09 vrF
A93582 02/08/09 03/09/09 full 03/09/09 vrF
2B0210 02/09/09 03/11/09 full 03/09/09 vrF
A92929 02/09/09 03/11/09 full 03/11/09 vrF
2B0202 02/09/09 03/11/09 full 03/09/09 vrF
A91516 02/09/09 03/11/09 full 03/11/09 vrF
2B0506 02/10/09 01/19/38 full 01/19/38 vKF
A91526 02/10/09 01/19/38 full 03/11/09 vKF
2B0217 02/10/09 03/12/09 full 03/10/09 vrF
A91077 02/10/09 03/12/09 full 03/12/09 vrF
2B1163 02/12/09 02/12/09 full 03/12/09 vrF
2B0685 02/23/09 03/24/09 full 03/23/09 vrF
A94073 02/23/09 03/24/09 full 03/24/09 vrF
T00602 02/24/09 05/01/09 full 05/01/09 vF
etc (from here the same all the way down)

I looked at a specific piece of media and ran nsrim -V volume to see if it marked the tape as recyclable - some of the messages which may be important are:

-----------------------
s-net1luxdc88:G:\, 6 browsable cycle(s)10066:nsrim: Failed updating continuation save set ssid `3707290228' as purged
39077:nsrim: error, Save set ssid `3707290228' the retention time is invalid

10067:nsrim: Failed updating continuation save set ssid `3707290228' as eligible
39077:nsrim: error, Save set ssid `3707290228' the retention time is invalid

9335:nsrck: WARNING: no valid save times for client 's-net1luxdc88' - cross check not performed

A93674: 308 GB used, 7 save sets, full, 1 recoverable save sets, 6 recyclable save sets
-----------------------
Note the saveset 3707290228 is not on this tape though the saveset name is and has been marked as recyclable; also all savesets for s-net1luxdc88 on this media are recyclable. These messages may therefore be red herrings?


Next I ran nsrck -L6 s-cpp-cmfe01.net1.cec.eu.int which went through as expected:

nsrck: checking index for 's-cpp-cmfe01.net1.cec.eu.int'
nsrck: /nsrsrv30/nsr/index/s-cpp-cmfe01.net1.cec.eu.int contains 3485803 records occupying 457 MB
nsrck: Completed checking 1 client(s)

And another nsrim with no effect:

A93674: 308 GB used, 7 save sets, full, 1 recoverable save sets, 6 recyclable save sets

The saveset in question here was taken 7 February with month browse and retention and with successful full backups taken afterwards so what could be preventing this saveset expiring?
Tags (1)
37 Replies
8 Krypton

Re: Savesets not expiring - the retention time is invalid

Another thing, there is another saveset here which has also not expired (and spans two tapes):

ssid clone id volume ssflags retent clretent
3264077411 1234034275 2B0011 vrF 03/08/09 03/07/09
3264077411 1234034275 2B0956 vrF 03/08/09 03/07/09
3264077411 1234098790 A93674 vrF 03/08/09 03/08/09
0 Kudos
rkanapat1
6 Indium

Re: Savesets not expiring - the retention time is invalid

Make sure you don't have media database corruption. In the daemon.log or in daemon.raw ( newer NetWorker versions) you should not see a text named "WISS". This error is seen, when you start the services. If you see this text, then this indicates the media database is corrupted. If it's corrupted, it needs to be fixed first.( scavange the media database)

on the NetWorker server run the command "nsrim -X" this will correct any issues in the media database.

make sure the pool doesn't have a longer retention policy. The Newer versions of NW server have a new option where the pools can have their own retention policy.
0 Kudos
8 Krypton

Re: Savesets not expiring - the retention time is invalid

The last time I saw a WISS error was Networker 6.22! There does not appear to be any indication of media database errors, I've already ran nsrim -X and nsrck -L6 against this client. I'll see if I can come up with any further information...
0 Kudos
8 Krypton

Re: Savesets not expiring - the retention time is invalid

There is no retention policy set on the pool.
0 Kudos
8 Krypton

Re: Savesets not expiring - the retention time is invalid

I also tried recovering the indexes (successfully) then running nsrim -V against one of the effected tapes - however saveset still comes up as being browsable.

I'm going to see what happens if I stage the data onto a different tape....
0 Kudos
8 Krypton

Re: Savesets not expiring - the retention time is invalid

Staged to another tape, run nsrim -V on that tape, still not expired. Very confusing:

7 root@bkpsrv30:->nsrstage -b T10KBKP030 -m -S 2072461207/1233600407
6365:nsrstage: Space can only be recovered from adv_file and file type devices.
8 root@bkpsrv30:->mminfo -q ssid=2072461207
volume client date size level name
094872 s-cpp-cmfe01.net1.cec.eu.int 02/02/09 8458 MB full C:\
9 root@bkpsrv30:->nsrim -V 094872
...
Cross check completed successfully

->mminfo -q ssid=2072461207 -r ssflags,ssretent
ssflags retent
vF 03/02/09
0 Kudos
8 Krypton

Re: Savesets not expiring - the retention time is invalid

Now this is interesting - here is a random affected saveset:

mminfo -q "client=s-cpp-cmfe01.net1.cec.eu.int,name=C:\\" -ot -r volume,savetime,ssretent,level,clretent,ssflags -r ssid,cloneid,ssflags,savetime,ssretent,clretent

094872 02/02/09 03/02/09 full 03/02/09 vF 2072461207 1244477590 vF 02/02/09 03/02/09 03/02/09

I reset the retention period of the clone as follows:

nsrmm -e 03/02/09 -S 2072461207/1244477590

And now we see this:

094872 02/02/09 03/02/09 full 04/08/45 vF 2072461207 1244477590 vF 02/02/09 03/02/09 04/08/45


I had a similar issue going from 7.3.3 to 7.4.3 at my old company and I came up with a workaround for it. I'm going to check on my laptop when I get home if I have a record of it and also I have a call with EMC to check my old cases!
0 Kudos
8 Krypton

Re: Savesets not expiring - the retention time is invalid

What version do you use now?
0 Kudos
Highlighted
8 Krypton

Re: Savesets not expiring - the retention time is invalid

This is Networker 7.4.4 - the savesets affected were written with the same version of Networker too and not subjected to any upgrade.

Today I've tried removing some of the earliest savesets (successfully) - both expired and should be expired; this had no effect on any existing backup after nsrim.

EMC's response has been to do a media scavenge; since this requires the server to be brought down we will not action this for a while...
0 Kudos