Automedia management has nothing to do with it. Picking up expired tapes is automatic anyway. It is very strange that mminfo doesn't find anything... did you try something like mminfo -avot -q volume=$VOLUME -r name,ssid,ssflags,sumflags,volid,location ? If not try it and see what do you get.
I agree just re-labelling the tape does fix the issue. the problem is if during the night we get the problem we expect automedia management to pick up the volume re-label and carry on with backups.
The reason I mentioned the scanner was just to show you guys how it can be fixed. It is really bizarre as if we run scanner -v "volume name" and then a split second later we abort it all the information for the volume is reconstructed. It is almost like the mediadb forgets about the volume but if we prod it enough all the information comes back to it.
I have searched through the dameon.log but nothing "weird" occurs. i will search again and post some outputs from it.
Do mminfo -av -q volume=000130 -r volid,location,type,volflags,state,labeled
In above check: - is volid the same as returned by nsrjb command - is the location correct - is type the same as the type in NSR device is configured for - any dirty marks for volflags - what is the state - when was last labeled
Of course, checking your daemon.log for when last time this tape was used would be ok too.
Do you do staging, if so that may explain it. When all savesets are staged off a volume that volume it will become recyclable when the nsrim process runs (which is once every 24 hours). When a backup runs it will first use appendable media, then recyclable volumes in the same pool, then recyclable volumes in other pools (if allowed), then blank media (if automedia management enabled). BTW your information is slightly inconsistent - you state:
These tapes then just sit there and do not get automatically recycled
Having previously stated:
n our media database there a number of recyclable volumes...
D:\Program Files\Legato\nsr\logs>mminfo -av -q volume=000130 -r ssid,savetime,sumsize,ssflags,volretent 6095:mminfo: no matches found for the query
volume 000129 is behaving as expected, however volume 000130 is the one that is showing the problems i explained earlier. Volume 000130 will sit there without being recognised by the media DB, the only way for it to become recognised again is for us to manually re-label it.
I assume as you have not confirmed you are doing staging that it means you are not doing any staging? I would just like to ascertain if there are any reasons for the tape being empty of savesets...
It's really strange no data is shown there... even there should be some. I wonder if this is related somehow to what happens to original savesets (I guess that would be the only thing). I know this would take time and might result in nothing, but I would check if possible if tapes (cloning pool) which seem to be fine still have expired savesets in backup pool. I guess it is much harder to check that for those showing nothing as you might not know what it supposed to be there unless you use script which keeps track of that.
Could you also check clretent with mminfo? In 7.4.4 there was an issue with this which did create mess with expiration of tapes and their usability. Not sure if that got fixed in 7.4.4.3 or 7.4.4.4. I miss exactly the details now top of my head, but I remember it did cause some strange things to happen as well. Perhaps EMC will give you jumbo patch and that might address the issue.
The problem only seems to be in tapes from the clone pool. The only problem we have is that the backup pool has a shorter retetntion than the clone pool thus recycling tapes much quicker, however I will keep my eye out for any tapes with the original savesets which could be causing the issue.
The problem is very strange though, I may log a call with EMC to see if they can shed any light on it.
There is no staging going on in the datazone, only cloning. These 2 volumes both belong to the weekly clone pool. The devices they are configured for are IBM LTO3 and the tapes are LTO3. I can't see any errors in volflag. Here are some more outputs:
Please note the last mminfo where savesets are listed for 000129 but not 000130 even though they are both full. As far the volid is concerned they seem to be the same.
Mar 13 18:27:44 xxxxxxxxxxxxx: NetWorker media: (info) LTO Ultrium-3 tape 000130 will be over-written
Mar 16 00:17:25 xxxxxxxxxxxxxl: NetWorker media: (info) verification of volume "000130", volid 45786528 succeeded.
D:\Program Files\Legato\nsr\logs>mminfo -av -q "volume=000130" -r volid,location,type,volflags,state,labeled volid location type flags state labeled 45786528 LTO Ultrium-3 E 13/03/2009
D:\Program Files\Legato\nsr\logs>mminfo -av -q volume=000130,volume=000129 -r volume,pool,volretent,%used volume pool expires (%) 000129 Weekly Clone expired full 000130 Weekly Clone expired full
The mminfo specifies saveset fields; the tape without savesets on it is not going to return any information. I want to see some data we can compare... Or that is what I meant to say
ble1
4 Operator
•
14.4K Posts
0
May 18th, 2009 04:00
2dam
85 Posts
0
May 18th, 2009 04:00
The reason I mentioned the scanner was just to show you guys how it can be fixed. It is really bizarre as if we run scanner -v "volume name" and then a split second later we abort it all the information for the volume is reconstructed. It is almost like the mediadb forgets about the volume but if we prod it enough all the information comes back to it.
I have searched through the dameon.log but nothing "weird" occurs. i will search again and post some outputs from it.
Thanks for your response.
I will keep you posted
2dam
ble1
4 Operator
•
14.4K Posts
0
May 18th, 2009 04:00
If you know one such tape, do greo daemon.log for that tape to see what happened to it and if along the way something outstanding didn't occur.
ble1
4 Operator
•
14.4K Posts
0
May 18th, 2009 05:00
In above check:
- is volid the same as returned by nsrjb command
- is the location correct
- is type the same as the type in NSR device is configured for
- any dirty marks for volflags
- what is the state
- when was last labeled
Of course, checking your daemon.log for when last time this tape was used would be ok too.
DavidHampson
2 Intern
•
1.1K Posts
0
May 18th, 2009 05:00
BTW your information is slightly inconsistent - you state:
These tapes then just sit there and do not get automatically recycled
Having previously stated:
n our media database there a number of recyclable volumes...
- so are these volumes recyclable or not?
2dam
85 Posts
0
May 18th, 2009 05:00
It sounds like I haven't explained myself properly so let me try again giving you some outputs.
I have picked 2 volumes which are expired and in a recyclable state:
D:\Program Files\Legato\nsr\logs>mminfo -av -q volume=000129 -r ssid,savetime,sumsize,ssflags,volretent
ssid date size ssflags expires
4072344306 14/03/2009 39 MB vrEF expired
3954903839 14/03/2009 16 MB vrEF expired
3921349427 14/03/2009 22 MB vrEF expired
3703245671 14/03/2009 16 MB vrEF expired
3820687932 14/03/2009 18 MB vrEF expired
3116045089 14/03/2009 23 MB vrEF expired
3032159027 14/03/2009 130 MB vrEF expired
549133312 14/03/2009 48 MB vrEF expired
3619365345 14/03/2009 11 MB vrEF expired
2998609011 14/03/2009 17 MB vrEF expired
1136342954 14/03/2009 92 MB vrEF expired
515593877 14/03/2009 48 MB vrEF expired
414935867 14/03/2009 15 MB vrEF expired
D:\Program Files\Legato\nsr\logs>mminfo -av -q volume=000130 -r ssid,savetime,sumsize,ssflags,volretent
6095:mminfo: no matches found for the query
volume 000129 is behaving as expected, however volume 000130 is the one that is showing the problems i explained earlier. Volume 000130 will sit there without being recognised by the media DB, the only way for it to become recognised again is for us to manually re-label it.
I hope this makes more sense.
Thanks for the responses so far
2dam
2dam
85 Posts
0
May 18th, 2009 05:00
D:\Program Files\Legato\nsr\logs>nsrim -V 000130
10082:nsrim: Cannot find any save sets in volume 000130
000130: 435 GB used, 0 save sets, recyclable
we are not staging in our datazone, only cloning.
DavidHampson
2 Intern
•
1.1K Posts
0
May 18th, 2009 06:00
ble1
4 Operator
•
14.4K Posts
1
May 18th, 2009 06:00
ble1
4 Operator
•
14.4K Posts
0
May 18th, 2009 06:00
ble1
4 Operator
•
14.4K Posts
0
May 18th, 2009 06:00
2dam
85 Posts
0
May 18th, 2009 06:00
The problem is very strange though, I may log a call with EMC to see if they can shed any light on it.
Thanks
2dam
2dam
85 Posts
0
May 18th, 2009 06:00
There is no staging going on in the datazone, only cloning. These 2 volumes both belong to the weekly clone pool. The devices they are configured for are IBM LTO3 and the tapes are LTO3. I can't see any errors in volflag. Here are some more outputs:
Please note the last mminfo where savesets are listed for 000129 but not 000130 even though they are both full. As far the volid is concerned they seem to be the same.
Mar 13 18:27:44 xxxxxxxxxxxxx: NetWorker media: (info) LTO Ultrium-3 tape 000130 will be over-written
Mar 16 00:17:25 xxxxxxxxxxxxxl: NetWorker media: (info) verification of volume "000130", volid 45786528 succeeded.
D:\Program Files\Legato\nsr\logs>mminfo -av -q "volume=000130" -r volid,location,type,volflags,state,labeled
volid location type flags state labeled
45786528 LTO Ultrium-3 E 13/03/2009
D:\Program Files\Legato\nsr\logs>mminfo -av -q volume=000130,volume=000129 -r volume,pool,volretent,%used
volume pool expires (%)
000129 Weekly Clone expired full
000130 Weekly Clone expired full
D:\Program Files\Legato\nsr\logs>mminfo -av -q volume=000130,volume=000129 -r volume,pool,volretent,ssid
volume pool expires ssid
000129 Weekly Clone expired 3116045089
000129 Weekly Clone expired 3954903839
000129 Weekly Clone expired 3921349427
000129 Weekly Clone expired 549133312
000129 Weekly Clone expired 2998609011
000129 Weekly Clone expired 414935867
000129 Weekly Clone expired 3619365345
000129 Weekly Clone expired 4072344306
000129 Weekly Clone expired 3820687932
000129 Weekly Clone expired 3703245671
000129 Weekly Clone expired 3032159027
000129 Weekly Clone expired 1136342954
000129 Weekly Clone expired 515593877
Any ideas you both have will be muchly appreciated.
Thanks for your responses so far.
2dam
DavidHampson
2 Intern
•
1.1K Posts
0
May 18th, 2009 06:00
Or that is what I meant to say
ble1
4 Operator
•
14.4K Posts
0
May 18th, 2009 06:00
David, you are getting old. That output is from tape which works fine