8 Krypton

Restoring expired save set information into client index with scanner

Follow up from the previous thread (https://community.emc.com/thread/224763 - restoring save set after retention and browse dates passed).

I decided to try scanner to restore information about expired (recoverable, but not browsable) save sets into client index. However some steps are unclear to me.

Hrvoje Crvelin wrote: 'you can always removed ssid from mdb if this is on tape (take mminfo for it before including position on tape) and then run scanner on it. scanner will ask you about position and ssid and will bring back record as if policy has been applied today.  But in general, this will take much longer.'

1. What is 'position'? Media file number and media record number?

Scanner is supposed to 'rebuild' the indexes (both media and file index). This term suggest that the whole index is modified. On the other hand for 'the -S ssid option, only entries for the specified save set are copied to the online file index'.

2. Does that mean that the rest of client index is left intact if '-S' option is used (i.e. only part of index concerning given save set is modified)?

3. What about media index in such case?

4. '-i' option modifies both media and client index. Is there no way to modify only client index by using scanner?

5. Man pages advises actually using nsrck to recover client index, and using scanner only for media index. I assume this advice does not apply in my situation?

I'm using STK SL500 Jukebox with two LTO-Ultrium-4 devices. Auto media management and auto clean is on. Two volumes (SD64LT and SD29LT) are mounted in those devices and used for current backups. Save sets I need to recover are on two other volumes (SD56LT and SD34LT). So in order to run scanner to recover for example save set from SD56LT I would need to unmount currently mounted volume (for example SD64LT) from one of devices, and then mount SD56LT there. I wonder how would it affect scheduled backups.

6. SD56LT is full and not expired - that means nothing could be written there. Is that correct?

7. What would happend if some backup was ongoing when I unmounted the volume it was written to (i.e. SD64LT)? Would unmount operation wait for the backup to complete? If not what would happen? Would part of the backup be on unmounted volume and the rest on SD29LT (the volume that is left intact)? Or would backup be corrupted?

8. Should I unmount SD56LT and mount SD64LT back after scanning, or could I just left SD56LT there?

sd@sd1:~ $ /usr/sbin/nsr/mminfo -V -s bdlbkp -m -c sdbkp

state volume                  written  (%)  expires     read mounts capacity flags

...

       SD29LT                   177 GB  22%  05/03/16    0 KB   226    800 GB

...

       SD56LT                  2412 GB full  03/03/16 3783 KB   241    800 GB

...

       SD64LT                  2218 GB 100%  05/03/16    0 KB   233    800 GB

...

SD64LT mounted on /dev/rmt/1cbn

SD29LT mounted on /dev/rmt/0cbn

I would appreciate any comments.

0 Kudos
8 Krypton

Re: Restoring expired save set information into client index with scanner

1) yes, position is media file number and record. scanner will rebuild (merge) what you are scanning unless already there.

2) Correct.

3) It will be populated

4) you can use -m, but then only media records will be populated.  Some backups, like RMAN, to be restored require index too.

5) if you have index available correct.  As for tapes, mark one drive as read only, make sure autoeject is not set on and load tape with no mount option and run scanner.

6) correct (unless you are referring to full status based on native uncompressed value - in that case tape might still be appendable)

7) you can't unmount volume in use.

😎 you can do whatever you want.

0 Kudos
8 Krypton

Re: Restoring expired save set information into client index with scanner

3) What I meant was: would behavior for media index be the same as for client index, i.e. would rest of media index be left intact if '-S' option is used (only part of media index concerning given save set would be modified)?

4) I want to restore client index, but to do that I have to use '-i' option, which restores both media and client indexes. I just wondered if restoring media index was necessary.

5) 'If you have index available' - that mean if I have recoverable backup of index?

0 Kudos
8 Krypton

Re: Restoring expired save set information into client index with scanner

Please allow me to add some more information here to explain the details ...

1. What is 'position'? Media file number and media record number?

Scanner is supposed to 'rebuild' the indexes (both media and file index). This term suggest that the whole index is modified. On the other hand for 'the -S ssid option, only entries for the specified save set are copied to the online file index'.

>> You can scan the whole media. However, this is more useful, if you do not have any info about the contents.

For a specific save set (and you still have the information from the media db) it is more beneficial to provide the file mark and record numbers so that NW can fast forward the media to this position. Otherwise it can take hours days to read through the whole tape until you reached the desired position.


2. Does that mean that the rest of client index is left intact if '-S' option is used (i.e. only part of index concerning given save set is modified)?

>> Client file index (CFI) info is stored per save set. Consequently you just rebuild the files for the specific save set and the others will not be touched at all.


3. What about media index in such case?

>> It is rebuild, if necessary. However, if the entry for the save set still exists, only its status will be changed appropriately (in this case to 'browsable').


4. '-i' option modifies both media and client index. Is there no way to modify only client index by using scanner?

>> See above. Do not forget that both information are related. It does not make sense to add CFI information if there is no reference in the media index (MI). The MI is the more important db.


5. Man pages advises actually using nsrck to recover client index, and using scanner only for media index. I assume this advice does not apply in my situation?

>> nsrck will recover the index to a certain point-in-time. Good for a disaster recover where a lot of save sets will be involved. However, what you want in this case is just the rebuild of a single save set's metadata.

I'm using STK SL500 Jukebox with two LTO-Ultrium-4 devices. Auto media management and auto clean is on. Two volumes (SD64LT and SD29LT) are mounted in those devices and used for current backups. Save sets I need to recover are on two other volumes (SD56LT and SD34LT). So in order to run scanner to recover for example save set from SD56LT I would need to unmount currently mounted volume (for example SD64LT) from one of devices, and then mount SD56LT there. I wonder how would it affect scheduled backups.

>> As usual, NW will wait for the next tape it needs. So if you backup simultaneously, NW can just use one of the drives for each process.

BTW - I am a bit picky here: technically, you must not necessarily mount the tape to be able to run 'scanner'. This is important as you must have a known media (known by the MI) to be able to mount it. One major benefit of 'scanner' is that the tape must not necessarily be mounted ... so even an unknown tape can be scanned.

6. SD56LT is full and not expired - that means nothing could be written there. Is that correct?

>> That depends whether you have set the tape to full of whether the tape has been really filled (by reaching the End-of-Tape mark). You can set an unmounted & unloaded tape to full at any time ... and vice-versa.


7. What would happend if some backup was ongoing when I unmounted the volume it was written to (i.e. SD64LT)? Would unmount operation wait for the backup to complete? If not what would happen? Would part of the backup be on unmounted volume and the rest on SD29LT (the volume that is left intact)? Or would backup be corrupted?

>> No further comment.

The only way to unload the tape would be if you hard reset the tape drive or if power has been lost.


8. Should I unmount SD56LT and mount SD64LT back after scanning, or could I just left SD56LT there?

>> As mentioned above, the tape must not necessarily be mounted to run 'scanner' successfully.

After a short 'idle timeout', NW will instruct the jukebox to (unmount &) unload it.

8 Krypton

Re: Restoring expired save set information into client index with scanner

3. What about media index in such case?

>> It is rebuild, if necessary. However, if the entry for the save set still exists, only its status will be changed appropriately (in this case to 'browsable').

4. '-i' option modifies both media and client index. Is there no way to modify only client index by using scanner?

>> See above. Do not forget that both information are related. It does not make sense to add CFI information if there is no reference in the media index (MI). The MI is the more important db.

I assume that there is a reference in the media index, because I see save sets in mminfo in with 'cr' flags. So in such case media index would not be rebuild, only its entries for save sets would be changed to browsable?

BTW - I am a bit picky here: technically, you must not necessarily mount the tape to be able to run 'scanner'. This is important as you must have a known media (known by the MI) to be able to mount it. One major benefit of 'scanner' is that the tape must not necessarily be mounted ... so even an unknown tape can be scanned.

OK, but when I run scanner I give a device as an argument. I assumed, that scanner would scan only the volume currently mounted in the device.

0 Kudos
8 Krypton

Re: Re: Restoring expired save set information into client index with scanner

>> 5) if you have index available correct.  As for tapes, mark one drive as read only, make sure autoeject is not set on and load tape with no mount option and run scanner.

By 'auto eject' do you mean 'idle device timeout'? How can it be set? I don't see any such option in nsradmin for 'NSR jukebox':

                        type: NSR jukebox;

                        name: STK@4.0.0;

                    comment: ;

                description: <STK    SL500          1235 at SCSI Bus 4 Target 0 LUN 0>;

                      model: Standard SCSI Jukebox;

                    enabled: [Yes]  No    Service ;

                control port: /dev/scsi/changer/c6t500104F0009EA857d0;

                    devices: /dev/rmt/1cbn, /dev/rmt/0cbn;

              number devices: 2;

              number drives: 2;

  SmartMedia update interval: 12;

            bar code reader: [Yes]  No ;

      match bar code labels: [Yes]  No ;

      jukebox serial number: WWNN=500104F0009EA856;

            max parallelism: 2;

                STL barcodes: SDW19L, SD46LT, SDW57L, SDW18L, SDW17L, SDW53L, SDW52L, SDW51L, SDW13L, SD54LT, SD24LT, SDW56L, SDW41L, SDW40L, SDW47L, SDW44L, SDW34L, SDW33L,

                              SD53LT, SDW59L, SDW55L, SDW31L, SDW49L, SDW46L, SDW43L, SDW60L, SDW02L, SD52LT, SDW58L, SDW50L, SDW48L, SDW62L, SD28LT, SDW42L, SDW45L, SDW54L,

                              SD51LT, SDW63L, SDW64L, SD74LT, SD73LT, SD72LT, SD71LT, SD70LT, SDW16L, SD50LT, SDW61L, SD39LT, SD38LT, "", "", SD35LT, SD34LT, SD33LT, SD59LT,

                              SD20LT, SD19LT, SD18LT, "", "", SD15LT, SD14LT, SD13LT, SD61LT, "", SDW15L, SD58LT, SDW12L, SDW05L, SDW03L, SDW10L, SDW09L, SDW08L, SD41LT, SDW06L,

                              SDW04L, "", SD22LT, SD57LT, SD23LT, SD62LT, SD64LT, SDW27L, SDW26L, SDW25L, SDW24L, SD42LT, "", SD21LT, SD56LT, SD68LT, SD69LT, SD01LT, "", SDW22L,

                              SDW21L, SDW01L, "", "", SD02LT, SD55LT, SD65LT, SD67LT, SD60LT, SD63LT, SDW32L, "", "", "", "", "", SD32LT, SD16LT, SD12LT, SDW23L, SDW14L, SD17LT,

                              SD45LT, SD44LT, SD43LT, "", "", SD31LT, SDW20L, SD29LT, "", SD27LT, SD26LT, SD25LT, SD76LT, SD75LT, "", "", SD11LT, SD08LT, SD10LT, SD09LT, SD07LT,

                              SD06LT, SD05LT, SD04LT, SD03LT, "", "", "", "", "", "", "", "", "", "", SDW65L, SDW66L, SDW67L, SDW68L, SDW69L, SD79LT, SD78LT, SD77LT, SD30LT,

                              SD36LT, SDW07L, SDW11L, CLNU04, CLNU01, CLNU05, CLNU02, CLNU03, SD66LT;

            RSM Library GUID: ;

              RSM Media Type: ;

            RSM Drive GUIDs: ;

            RSM Media GUIDs: ;

            STL device names: ;

          STL device sharing: ;

      STL device reservation: ;

          STL interface lib: ;

              STL identifier: ;

                  auto clean: [Yes]  No ;

      verify label on unload:  Yes  [No];

              cleaning slots: 165-169;

          default cleanings: 10;

      auto media management: [Yes]  No ;

            application name: ;

            application key: ;

              read hostname: ;

                NDMP jukebox:  Yes  [No];

              NDMP hostname: ;

        NDMP jukebox handle: ;

                      ready: [Yes]  No ;

            virtual jukebox:  Yes  [No];

    virtual jukebox frameid: ;

By 'no mount option' do you mean 'nsrjb -l -f /dev/rmt/1cbn SD56LT'?

In scanner manual there is a requirement to use 'no-rewind on close' device. An 'n' at the end of device name (for example /dev/rmt/1cbn) means that it is 'no-rewind' device, am I correct?

Concerning the media index, if it would be rebuild by scanner, what would happen with entries concerning other volumes? Would they be erased? Left unchanged? Or would Networker scan all volumes to rebuild index?

0 Kudos
8 Krypton

Re: Restoring expired save set information into client index with scanner

"I assume that there is a reference in the media index, because I see save sets in mminfo in with 'cr' flags. So in such case media index would not be rebuild, only its entries for save sets would be changed to browsable?"

   Correct.

OK, but when I run scanner I give a device as an argument. I assumed, that scanner would scan only the volume currently mounted in the device.

   Correct. You must be ensure to load/insert the right media.

By 'auto eject' do you mean 'idle device timeout'? How can it be set? I don't see any such option in nsradmin for 'NSR jukebox'

   It does exist ... but it is hidden by default. You must activate the hidden attributes first:

....

nsradmin> option hidden

Hidden display option turned on

Display options:

        Dynamic: Off;

        Hidden: On;

        Raw I18N: Off;

        Resource ID: Off;

        Regexp: Off;

nsradmin> . type: nsr jukebox

Current query set

nsradmin> show idle device timeout

nsradmin> p

         idle device timeout: 20;

nsradmin>

In scanner manual there is a requirement to use 'no-rewind on close' device. An 'n' at the end of device name (for example /dev/rmt/1cbn) means that it is 'no-rewind' device, am I correct?

   That's correct. If you don't do that, the tape will be rewound after each write or read. Such is nonsense for a backup device.

Concerning the media index, if it would be rebuild by scanner, what would happen with entries concerning other volumes? Would they be erased? Left unchanged? Or would Networker scan all volumes to rebuild index?

   Of course you could also use scanner to rebuild the whole media index tape-for-tape. But this would mean that you have to scan hundreds or thousand of tapes to retrieve all missing info. This could take days, weeks, months ...

   Consequently, the better practice is to recover from a media index backup which is part of a bootstrap save set.

   'scanner' is only used to partially add data (append) for a specific volume or a save set.

   To rebuild the whole media index from the bootstrap save set you use mmrecov (now nsrdr).

   Bare in mind that this overwrites the existing media index. But this usually only lasts some minutes.

8 Krypton

Re: Restoring expired save set information into client index with scanner

>>   Of course you could also use scanner to rebuild the whole media index tape-for-tape. But this would mean that you have to scan hundreds or thousand of tapes to retrieve all missing info. This could take days, weeks, months ...

But if I run 'scanner -i -S ssid' for couple of SSIDs that won't happen? Only parts of media index concerning those ssids would be rebuilt?

0 Kudos
8 Krypton

Re: Restoring expired save set information into client index with scanner

Sure. As I mentioned.

May I suggest that you setup a tiny test enviroment, run some backups to a local AFTD and verify yourself.

For the eval period, you do not even need a license.

0 Kudos
8 Krypton

Re: Restoring expired save set information into client index with scanner

OK, I decided to make the test environment and try there.

Though if you wouldn't mind one more question:

What I wanted to know is what would happen if I made some mistake and scanner started to rebuild the whole media index? I understand that the entries from scanned volume would be erased and replaced by entries scanned by scanner. What about entries from other volumes?

0 Kudos