Unsolved

This post is more than 5 years old

2 Intern

 • 

1.1K Posts

9245

June 5th, 2009 09:00

Savesets not expiring - the retention time is invalid

Here is another strange one - I have one client which has a month browse/retention which has savesets that go back to February that have not expired - no dependency here and generally they are the only saveset on the tapes meaning they are holding up tape reuse - here is some mminfo data:

mminfo -q "client=s-cpp-cmfe01.net1.cec.eu.int,name=C:\\" -ot -r volume,savetime,ssretent,level,clretent,ssflags
...2B1517 02/02/09 03/02/09 full 03/02/09 vF
2B1538 02/03/09 03/03/09 full 03/03/09 vF
2B1541 02/03/09 03/03/09 full 03/03/09 vF
2B1535 02/04/09 03/06/09 full 03/04/09 vrEF
2B1624 02/05/09 03/06/09 full 03/05/09 vrEF
2B1587 02/06/09 03/07/09 full 03/06/09 vrEF
2B1571 02/06/09 03/07/09 full 03/06/09 CvrEF
2B1571 02/06/09 03/07/09 full 03/06/09 CvrEF
2B0011 02/07/09 03/08/09 full 03/07/09 vrF
2B0956 02/07/09 03/08/09 full 03/07/09 vrF

A93674 02/07/09 03/08/09 full 03/08/09 vrF

2B0257 02/08/09 03/09/09 full 03/08/09 vrF
A93582 02/08/09 03/09/09 full 03/09/09 vrF
2B0210 02/09/09 03/11/09 full 03/09/09 vrF
A92929 02/09/09 03/11/09 full 03/11/09 vrF
2B0202 02/09/09 03/11/09 full 03/09/09 vrF
A91516 02/09/09 03/11/09 full 03/11/09 vrF
2B0506 02/10/09 01/19/38 full 01/19/38 vKF
A91526 02/10/09 01/19/38 full 03/11/09 vKF
2B0217 02/10/09 03/12/09 full 03/10/09 vrF
A91077 02/10/09 03/12/09 full 03/12/09 vrF
2B1163 02/12/09 02/12/09 full 03/12/09 vrF
2B0685 02/23/09 03/24/09 full 03/23/09 vrF
A94073 02/23/09 03/24/09 full 03/24/09 vrF
T00602 02/24/09 05/01/09 full 05/01/09 vF
etc (from here the same all the way down)

I looked at a specific piece of media and ran nsrim -V volume to see if it marked the tape as recyclable - some of the messages which may be important are:

-----------------------
s-net1luxdc88:G:\, 6 browsable cycle(s)10066:nsrim: Failed updating continuation save set ssid `3707290228' as purged
39077:nsrim: error, Save set ssid `3707290228' the retention time is invalid

10067:nsrim: Failed updating continuation save set ssid `3707290228' as eligible
39077:nsrim: error, Save set ssid `3707290228' the retention time is invalid

9335:nsrck: WARNING: no valid save times for client 's-net1luxdc88' - cross check not performed

A93674: 308 GB used, 7 save sets, full, 1 recoverable save sets, 6 recyclable save sets
-----------------------
Note the saveset 3707290228 is not on this tape though the saveset name is and has been marked as recyclable; also all savesets for s-net1luxdc88 on this media are recyclable. These messages may therefore be red herrings?


Next I ran nsrck -L6 s-cpp-cmfe01.net1.cec.eu.int which went through as expected:

nsrck: checking index for 's-cpp-cmfe01.net1.cec.eu.int'
nsrck: /nsrsrv30/nsr/index/s-cpp-cmfe01.net1.cec.eu.int contains 3485803 records occupying 457 MB
nsrck: Completed checking 1 client(s)

And another nsrim with no effect:

A93674: 308 GB used, 7 save sets, full, 1 recoverable save sets, 6 recyclable save sets

The saveset in question here was taken 7 February with month browse and retention and with successful full backups taken afterwards so what could be preventing this saveset expiring?

2 Intern

 • 

1.1K Posts

June 5th, 2009 09:00

Another thing, there is another saveset here which has also not expired (and spans two tapes):

ssid clone id volume ssflags retent clretent
3264077411 1234034275 2B0011 vrF 03/08/09 03/07/09
3264077411 1234034275 2B0956 vrF 03/08/09 03/07/09
3264077411 1234098790 A93674 vrF 03/08/09 03/08/09

2 Intern

 • 

1.1K Posts

June 8th, 2009 08:00

The last time I saw a WISS error was Networker 6.22! There does not appear to be any indication of media database errors, I've already ran nsrim -X and nsrck -L6 against this client. I'll see if I can come up with any further information...

2 Intern

 • 

1.1K Posts

June 8th, 2009 08:00

There is no retention policy set on the pool.

2 Intern

 • 

1.1K Posts

June 8th, 2009 08:00

I also tried recovering the indexes (successfully) then running nsrim -V against one of the effected tapes - however saveset still comes up as being browsable.

I'm going to see what happens if I stage the data onto a different tape....

2 Intern

 • 

1.1K Posts

June 9th, 2009 01:00

Staged to another tape, run nsrim -V on that tape, still not expired. Very confusing:

7 root@bkpsrv30:->nsrstage -b T10KBKP030 -m -S 2072461207/1233600407
6365:nsrstage: Space can only be recovered from adv_file and file type devices.
8 root@bkpsrv30:->mminfo -q ssid=2072461207
volume client date size level name
094872 s-cpp-cmfe01.net1.cec.eu.int 02/02/09 8458 MB full C:\
9 root@bkpsrv30:->nsrim -V 094872
...
Cross check completed successfully

->mminfo -q ssid=2072461207 -r ssflags,ssretent
ssflags retent
vF 03/02/09

2 Intern

 • 

1.1K Posts

June 9th, 2009 09:00

Now this is interesting - here is a random affected saveset:

mminfo -q "client=s-cpp-cmfe01.net1.cec.eu.int,name=C:\\" -ot -r volume,savetime,ssretent,level,clretent,ssflags -r ssid,cloneid,ssflags,savetime,ssretent,clretent

094872 02/02/09 03/02/09 full 03/02/09 vF 2072461207 1244477590 vF 02/02/09 03/02/09 03/02/09

I reset the retention period of the clone as follows:

nsrmm -e 03/02/09 -S 2072461207/1244477590

And now we see this:

094872 02/02/09 03/02/09 full 04/08/45 vF 2072461207 1244477590 vF 02/02/09 03/02/09 04/08/45


I had a similar issue going from 7.3.3 to 7.4.3 at my old company and I came up with a workaround for it. I'm going to check on my laptop when I get home if I have a record of it and also I have a call with EMC to check my old cases!

6 Operator

 • 

14.4K Posts

 • 

56.2K Points

June 10th, 2009 03:00

What version do you use now?

2 Intern

 • 

1.1K Posts

June 10th, 2009 04:00

This is Networker 7.4.4 - the savesets affected were written with the same version of Networker too and not subjected to any upgrade.

Today I've tried removing some of the earliest savesets (successfully) - both expired and should be expired; this had no effect on any existing backup after nsrim.

EMC's response has been to do a media scavenge; since this requires the server to be brought down we will not action this for a while...

6 Operator

 • 

14.4K Posts

 • 

56.2K Points

June 10th, 2009 11:00

I don't remember the details, but in 7.4.4.3 some of the issues with clretent have been resolved - can you try that patch? Or even 7.4.4.4....

2 Intern

 • 

1.1K Posts

June 11th, 2009 01:00

I'll take a look at the release notes and see if we can justify an upgrade; I have a call logged with EMC so I will see if they can also offer advice on this.

2 Intern

 • 

1.1K Posts

June 22nd, 2009 02:00

I've returned to looking at this one - some more interesting information. So looking at one of these tapes:

nsrim -V A92944

returns:

Cross check completed successfully

A92944: 384 GB used, 233 save sets, full, 9 browsable save sets, 224 recyclable save sets
62173:nsrim: nsrim has finished at Mon Jun 22 11:47:56 2009.

mminfo -q volume=A92944 -r client,name,savetime,level,ssbrowse,ssretent,clretent,ssflags,clflags|grep -v clretent|wc -l

224
mminfo -q volume=A92944 -r client,name,savetime,level,ssbrowse,ssretent,clretent,ssflags,clflags|grep vrEF|wc -l

223
mminfo -q volume=A92944 -r client,name,savetime,level,ssbrowse,ssretent,clretent,ssflags,clflags|grep -v vrEF

client name date lvl browse retent clretent ssflags clflg
s-cc-rwp01 E:\ 02/23/09 full 03/23/09 03/23/09 03/23/09 vrEiF E

So the "9 browsable savesets" which seem to be stopping this tape recycling do not get listed as being in the media database.

2 Intern

 • 

1.1K Posts

June 22nd, 2009 03:00

So what I am going to look at now is deleting the tape and scanning in then running the same queries again...

2 Intern

 • 

1.1K Posts

June 24th, 2009 02:00

Scanning the tape back in discovered an interesting result; the savesets returned by mminfo matched the recyclable tapes but the nsrim -V volume seemed to be referencing ana additional 9 browsable savesets that were not listed in the mminfo output.

I'm now running a media scavenge on this box and seeing what results this gives.

2 Intern

 • 

1.1K Posts

June 24th, 2009 02:00

Ran the media scavenge successfully; looked at tape A92944 again - ran nsrim -V then checked the savesets:

Cross check completed successfully

A92944: 45 GB used, 67 save sets, read-only, 9 browsable save sets, 58 recyclable save sets


124 root@bkpsrv30:->mminfo -q volume=A92944|wc -l
59
125 root@bkpsrv30:-


We still appear to be referencing the phantom savesets....?

6 Operator

 • 

14.4K Posts

 • 

56.2K Points

June 24th, 2009 02:00

Try to add -av so that lines goes

mminfo -av -q volume=A92944|wc -l
No Events found!

Top