This post is more than 5 years old
24 Posts
0
6911
Restoring expired save set information into client index with scanner
Follow up from the previous thread (https://community.emc.com/thread/224763 - restoring save set after retention and browse dates passed).
I decided to try scanner to restore information about expired (recoverable, but not browsable) save sets into client index. However some steps are unclear to me.
Hrvoje Crvelin wrote: 'you can always removed ssid from mdb if this is on tape (take mminfo for it before including position on tape) and then run scanner on it. scanner will ask you about position and ssid and will bring back record as if policy has been applied today. But in general, this will take much longer.'
1. What is 'position'? Media file number and media record number?
Scanner is supposed to 'rebuild' the indexes (both media and file index). This term suggest that the whole index is modified. On the other hand for 'the -S ssid option, only entries for the specified save set are copied to the online file index'.
2. Does that mean that the rest of client index is left intact if '-S' option is used (i.e. only part of index concerning given save set is modified)?
3. What about media index in such case?
4. '-i' option modifies both media and client index. Is there no way to modify only client index by using scanner?
5. Man pages advises actually using nsrck to recover client index, and using scanner only for media index. I assume this advice does not apply in my situation?
I'm using STK SL500 Jukebox with two LTO-Ultrium-4 devices. Auto media management and auto clean is on. Two volumes (SD64LT and SD29LT) are mounted in those devices and used for current backups. Save sets I need to recover are on two other volumes (SD56LT and SD34LT). So in order to run scanner to recover for example save set from SD56LT I would need to unmount currently mounted volume (for example SD64LT) from one of devices, and then mount SD56LT there. I wonder how would it affect scheduled backups.
6. SD56LT is full and not expired - that means nothing could be written there. Is that correct?
7. What would happend if some backup was ongoing when I unmounted the volume it was written to (i.e. SD64LT)? Would unmount operation wait for the backup to complete? If not what would happen? Would part of the backup be on unmounted volume and the rest on SD29LT (the volume that is left intact)? Or would backup be corrupted?
8. Should I unmount SD56LT and mount SD64LT back after scanning, or could I just left SD56LT there?
sd@sd1:~ $ /usr/sbin/nsr/mminfo -V -s bdlbkp -m -c sdbkp
state volume written (%) expires read mounts capacity flags
...
SD29LT 177 GB 22% 05/03/16 0 KB 226 800 GB
...
SD56LT 2412 GB full 03/03/16 3783 KB 241 800 GB
...
SD64LT 2218 GB 100% 05/03/16 0 KB 233 800 GB
...
SD64LT mounted on /dev/rmt/1cbn
SD29LT mounted on /dev/rmt/0cbn
I would appreciate any comments.
ble1
14.3K Posts
0
February 9th, 2016 04:00
1) yes, position is media file number and record. scanner will rebuild (merge) what you are scanning unless already there.
2) Correct.
3) It will be populated
4) you can use -m, but then only media records will be populated. Some backups, like RMAN, to be restored require index too.
5) if you have index available correct. As for tapes, mark one drive as read only, make sure autoeject is not set on and load tape with no mount option and run scanner.
6) correct (unless you are referring to full status based on native uncompressed value - in that case tape might still be appendable)
7) you can't unmount volume in use.
8) you can do whatever you want.
bingo.1
2.4K Posts
1
February 9th, 2016 05:00
Please allow me to add some more information here to explain the details ...
1. What is 'position'? Media file number and media record number?
Scanner is supposed to 'rebuild' the indexes (both media and file index). This term suggest that the whole index is modified. On the other hand for 'the -S ssid option, only entries for the specified save set are copied to the online file index'.
>> You can scan the whole media. However, this is more useful, if you do not have any info about the contents.
For a specific save set (and you still have the information from the media db) it is more beneficial to provide the file mark and record numbers so that NW can fast forward the media to this position. Otherwise it can take hours days to read through the whole tape until you reached the desired position.
2. Does that mean that the rest of client index is left intact if '-S' option is used (i.e. only part of index concerning given save set is modified)?
>> Client file index (CFI) info is stored per save set. Consequently you just rebuild the files for the specific save set and the others will not be touched at all.
3. What about media index in such case?
>> It is rebuild, if necessary. However, if the entry for the save set still exists, only its status will be changed appropriately (in this case to 'browsable').
4. '-i' option modifies both media and client index. Is there no way to modify only client index by using scanner?
>> See above. Do not forget that both information are related. It does not make sense to add CFI information if there is no reference in the media index (MI). The MI is the more important db.
5. Man pages advises actually using nsrck to recover client index, and using scanner only for media index. I assume this advice does not apply in my situation?
>> nsrck will recover the index to a certain point-in-time. Good for a disaster recover where a lot of save sets will be involved. However, what you want in this case is just the rebuild of a single save set's metadata.
I'm using STK SL500 Jukebox with two LTO-Ultrium-4 devices. Auto media management and auto clean is on. Two volumes (SD64LT and SD29LT) are mounted in those devices and used for current backups. Save sets I need to recover are on two other volumes (SD56LT and SD34LT). So in order to run scanner to recover for example save set from SD56LT I would need to unmount currently mounted volume (for example SD64LT) from one of devices, and then mount SD56LT there. I wonder how would it affect scheduled backups.
>> As usual, NW will wait for the next tape it needs. So if you backup simultaneously, NW can just use one of the drives for each process.
BTW - I am a bit picky here: technically, you must not necessarily mount the tape to be able to run 'scanner'. This is important as you must have a known media (known by the MI) to be able to mount it. One major benefit of 'scanner' is that the tape must not necessarily be mounted ... so even an unknown tape can be scanned.
6. SD56LT is full and not expired - that means nothing could be written there. Is that correct?
>> That depends whether you have set the tape to full of whether the tape has been really filled (by reaching the End-of-Tape mark). You can set an unmounted & unloaded tape to full at any time ... and vice-versa.
7. What would happend if some backup was ongoing when I unmounted the volume it was written to (i.e. SD64LT)? Would unmount operation wait for the backup to complete? If not what would happen? Would part of the backup be on unmounted volume and the rest on SD29LT (the volume that is left intact)? Or would backup be corrupted?
>> No further comment.
The only way to unload the tape would be if you hard reset the tape drive or if power has been lost.
8. Should I unmount SD56LT and mount SD64LT back after scanning, or could I just left SD56LT there?
>> As mentioned above, the tape must not necessarily be mounted to run 'scanner' successfully.
After a short 'idle timeout', NW will instruct the jukebox to (unmount &) unload it.
tolejniczak
24 Posts
0
February 9th, 2016 05:00
3) What I meant was: would behavior for media index be the same as for client index, i.e. would rest of media index be left intact if '-S' option is used (only part of media index concerning given save set would be modified)?
4) I want to restore client index, but to do that I have to use '-i' option, which restores both media and client indexes. I just wondered if restoring media index was necessary.
5) 'If you have index available' - that mean if I have recoverable backup of index?
tolejniczak
24 Posts
0
February 9th, 2016 05:00
3. What about media index in such case?
>> It is rebuild, if necessary. However, if the entry for the save set still exists, only its status will be changed appropriately (in this case to 'browsable').
4. '-i' option modifies both media and client index. Is there no way to modify only client index by using scanner?
>> See above. Do not forget that both information are related. It does not make sense to add CFI information if there is no reference in the media index (MI). The MI is the more important db.
I assume that there is a reference in the media index, because I see save sets in mminfo in with 'cr' flags. So in such case media index would not be rebuild, only its entries for save sets would be changed to browsable?
BTW - I am a bit picky here: technically, you must not necessarily mount the tape to be able to run 'scanner'. This is important as you must have a known media (known by the MI) to be able to mount it. One major benefit of 'scanner' is that the tape must not necessarily be mounted ... so even an unknown tape can be scanned.
OK, but when I run scanner I give a device as an argument. I assumed, that scanner would scan only the volume currently mounted in the device.
tolejniczak
24 Posts
0
February 12th, 2016 02:00
>> 5) if you have index available correct. As for tapes, mark one drive as read only, make sure autoeject is not set on and load tape with no mount option and run scanner.
By 'auto eject' do you mean 'idle device timeout'? How can it be set? I don't see any such option in nsradmin for 'NSR jukebox':
type: NSR jukebox;
name: STK@4.0.0;
comment: ;
description: ;
model: Standard SCSI Jukebox;
enabled: [Yes] No Service ;
control port: /dev/scsi/changer/c6t500104F0009EA857d0;
devices: /dev/rmt/1cbn, /dev/rmt/0cbn;
number devices: 2;
number drives: 2;
SmartMedia update interval: 12;
bar code reader: [Yes] No ;
match bar code labels: [Yes] No ;
jukebox serial number: WWNN=500104F0009EA856;
max parallelism: 2;
STL barcodes: SDW19L, SD46LT, SDW57L, SDW18L, SDW17L, SDW53L, SDW52L, SDW51L, SDW13L, SD54LT, SD24LT, SDW56L, SDW41L, SDW40L, SDW47L, SDW44L, SDW34L, SDW33L,
SD53LT, SDW59L, SDW55L, SDW31L, SDW49L, SDW46L, SDW43L, SDW60L, SDW02L, SD52LT, SDW58L, SDW50L, SDW48L, SDW62L, SD28LT, SDW42L, SDW45L, SDW54L,
SD51LT, SDW63L, SDW64L, SD74LT, SD73LT, SD72LT, SD71LT, SD70LT, SDW16L, SD50LT, SDW61L, SD39LT, SD38LT, "", "", SD35LT, SD34LT, SD33LT, SD59LT,
SD20LT, SD19LT, SD18LT, "", "", SD15LT, SD14LT, SD13LT, SD61LT, "", SDW15L, SD58LT, SDW12L, SDW05L, SDW03L, SDW10L, SDW09L, SDW08L, SD41LT, SDW06L,
SDW04L, "", SD22LT, SD57LT, SD23LT, SD62LT, SD64LT, SDW27L, SDW26L, SDW25L, SDW24L, SD42LT, "", SD21LT, SD56LT, SD68LT, SD69LT, SD01LT, "", SDW22L,
SDW21L, SDW01L, "", "", SD02LT, SD55LT, SD65LT, SD67LT, SD60LT, SD63LT, SDW32L, "", "", "", "", "", SD32LT, SD16LT, SD12LT, SDW23L, SDW14L, SD17LT,
SD45LT, SD44LT, SD43LT, "", "", SD31LT, SDW20L, SD29LT, "", SD27LT, SD26LT, SD25LT, SD76LT, SD75LT, "", "", SD11LT, SD08LT, SD10LT, SD09LT, SD07LT,
SD06LT, SD05LT, SD04LT, SD03LT, "", "", "", "", "", "", "", "", "", "", SDW65L, SDW66L, SDW67L, SDW68L, SDW69L, SD79LT, SD78LT, SD77LT, SD30LT,
SD36LT, SDW07L, SDW11L, CLNU04, CLNU01, CLNU05, CLNU02, CLNU03, SD66LT;
RSM Library GUID: ;
RSM Media Type: ;
RSM Drive GUIDs: ;
RSM Media GUIDs: ;
STL device names: ;
STL device sharing: ;
STL device reservation: ;
STL interface lib: ;
STL identifier: ;
auto clean: [Yes] No ;
verify label on unload: Yes [No];
cleaning slots: 165-169;
default cleanings: 10;
auto media management: [Yes] No ;
application name: ;
application key: ;
read hostname: ;
NDMP jukebox: Yes [No];
NDMP hostname: ;
NDMP jukebox handle: ;
ready: [Yes] No ;
virtual jukebox: Yes [No];
virtual jukebox frameid: ;
By 'no mount option' do you mean 'nsrjb -l -f /dev/rmt/1cbn SD56LT'?
In scanner manual there is a requirement to use 'no-rewind on close' device. An 'n' at the end of device name (for example /dev/rmt/1cbn) means that it is 'no-rewind' device, am I correct?
Concerning the media index, if it would be rebuild by scanner, what would happen with entries concerning other volumes? Would they be erased? Left unchanged? Or would Networker scan all volumes to rebuild index?
bingo.1
2.4K Posts
1
February 13th, 2016 08:00
"I assume that there is a reference in the media index, because I see save sets in mminfo in with 'cr' flags. So in such case media index would not be rebuild, only its entries for save sets would be changed to browsable?"
Correct.
OK, but when I run scanner I give a device as an argument. I assumed, that scanner would scan only the volume currently mounted in the device.
Correct. You must be ensure to load/insert the right media.
By 'auto eject' do you mean 'idle device timeout'? How can it be set? I don't see any such option in nsradmin for 'NSR jukebox'
It does exist ... but it is hidden by default. You must activate the hidden attributes first:
....
nsradmin> option hidden
Hidden display option turned on
Display options:
Dynamic: Off;
Hidden: On;
Raw I18N: Off;
Resource ID: Off;
Regexp: Off;
nsradmin> . type: nsr jukebox
Current query set
nsradmin> show idle device timeout
nsradmin> p
idle device timeout: 20;
nsradmin>
In scanner manual there is a requirement to use 'no-rewind on close' device. An 'n' at the end of device name (for example /dev/rmt/1cbn) means that it is 'no-rewind' device, am I correct?
That's correct. If you don't do that, the tape will be rewound after each write or read. Such is nonsense for a backup device.
Concerning the media index, if it would be rebuild by scanner, what would happen with entries concerning other volumes? Would they be erased? Left unchanged? Or would Networker scan all volumes to rebuild index?
Of course you could also use scanner to rebuild the whole media index tape-for-tape. But this would mean that you have to scan hundreds or thousand of tapes to retrieve all missing info. This could take days, weeks, months ...
Consequently, the better practice is to recover from a media index backup which is part of a bootstrap save set.
'scanner' is only used to partially add data (append) for a specific volume or a save set.
To rebuild the whole media index from the bootstrap save set you use mmrecov (now nsrdr).
Bare in mind that this overwrites the existing media index. But this usually only lasts some minutes.
tolejniczak
24 Posts
0
February 13th, 2016 09:00
>> Of course you could also use scanner to rebuild the whole media index tape-for-tape. But this would mean that you have to scan hundreds or thousand of tapes to retrieve all missing info. This could take days, weeks, months ...
But if I run 'scanner -i -S ssid' for couple of SSIDs that won't happen? Only parts of media index concerning those ssids would be rebuilt?
bingo.1
2.4K Posts
0
February 14th, 2016 02:00
Sure. As I mentioned.
May I suggest that you setup a tiny test enviroment, run some backups to a local AFTD and verify yourself.
For the eval period, you do not even need a license.
tolejniczak
24 Posts
0
February 16th, 2016 06:00
OK, I decided to make the test environment and try there.
Though if you wouldn't mind one more question:
What I wanted to know is what would happen if I made some mistake and scanner started to rebuild the whole media index? I understand that the entries from scanned volume would be erased and replaced by entries scanned by scanner. What about entries from other volumes?
bingo.1
2.4K Posts
0
February 16th, 2016 10:00
They will not get touched at all.
tolejniczak
24 Posts
0
February 23rd, 2016 02:00
I have multiple media file numbers for each SSID:
/usr/sbin/nsr/mminfo -avot -s bdlbkp -r "ssid,cloneid,mediarec,mediafile,name,totalsize,savetime,volume,state,ssretent,ssbrowse" -q "client=sdbkp"
ssid clone id rec file name total date volume state retent browse
1631254415 1441494519 0 677 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 695 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 678 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 694 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 658 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 679 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 680 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 654 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 681 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 682 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 683 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 684 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 685 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 653 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 652 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 686 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 687 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 688 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 693 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 690 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 691 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 692 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 659 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 657 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 660 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 661 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 662 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 663 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 656 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 664 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 665 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 655 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 666 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 667 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 668 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 669 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 670 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 671 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 672 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 673 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 674 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 675 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 676 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
1631254415 1441494519 0 689 RMAN:/home/scripts/sddb_backup.rcv 362 GB 09/06/15 SD56LT 03/03/16 10/18/15
In such case should I give the lowest as argument of scanner?
scanner -f 652 -r 0 -i -S 1631254415 /dev/rmt/0cbn
bingo.1
2.4K Posts
0
February 23rd, 2016 03:00
That's the logic. However, you better forget about the file and record numbers.
You still have the media index which knows exactly where to start.
You can also say that the media index 'maps the logical entity (the save set) to the physical location on tape'.
File and Media Records really make a difference if you run a DR - while your media index is empty. That's when NW will ask you about the physical location to speed up the positioning towards the beginning of the bootstrap save set.
tolejniczak
24 Posts
0
February 23rd, 2016 08:00
If I still have the media index, do I have to load the volume (without mounting) into device before scanning, or could I just mount it?
bingo.1
2.4K Posts
0
February 23rd, 2016 09:00
In this is up to you. Both will work. Try it.
However, if it does not have a media index entry you do not have the chance to mount it.
tolejniczak
24 Posts
0
February 26th, 2016 04:00
I managed to make expired backups browsable with scanner -f -r -i -S /dev/rmt/1cbn . Thanks for helpful comments.
I noticed that after scanning I had to manually extend browse time by nsrmm -S -s bdlbkp -w "03/02/2016" - it was not changed by scanner and if I didn't change it some save sets would return to recoverable state.