NOT in media index

Question

Hi,

During recover i am facing this problem. does anyone know how to resolve this problem?

Networker version : NetWorker 7.6.4.5.Build.1088

recover> recover

Recovering 7351 files into their original locations

Volumes needed (all near-line):

171459 at xxx916_101

171485 at xxx916_101

172140 at xxx916_101

170844 at xxx916_101

172086 at xxx916_101

171965 at xxx916_101

Total estimated disk space needed for recover is 104 MB.

Requesting 7351 file(s), this may take a while...

MESSAGES

Tue 19:09:08 /dev/rmt/c115t0d5BESTnb verified label of 171459 (NOT in media index)

Tue 19:09:09 /dev/rmt/c115t0d5BESTnb Eject operation in progress

Tue 19:09:09 /dev/rmt/c115t0d5BESTnb ejected

Tue 19:11:09 /dev/rmt/c115t0d3BESTnb verified label of 171459 (NOT in media index)

Tue 19:11:10 /dev/rmt/c115t0d3BESTnb Eject operation in progress

Tue 19:11:10 /dev/rmt/c115t0d3BESTnb ejected

Tue 19:11:39 /dev/rmt/c115t0d4BESTnb Verify label operation in progress

11:39 /dev/rmt/c115t0d4BESTnb verified label of 171459 (NOT in media index)

any solution related this problem would be highly apprciated.

Thanks in advance!

Aman

bingo.1 · Accepted Answer

Forget the file index info for the moment - even if you would have the info here, it would not help at all if the info in the media index would be missing. This is the more important part.

This log info is pretty important:

7224 09/30/14 22:10:58 nsrd [Jukebox `svg916_101', operation # 10]. Expected volume ID `1327551246' for volume `171459'. The actual volume ID is `1898572862'.

So it is obvious that NW has a reference to

- volume 171459 with volume ID 1327551246 but now reports to have read

- volume 171459 with volume ID 1898572862

Could it be that you got one with the same bar code but which has been written by another NW server ?

If so, you would need to scan it to retrieve the metadata. However, NW does not support having more than one volume with the same name but 2 different vol IDs (I think this feature was deleted/broken with NW 7.0). The consequence is that you could only add a tape with the same volume after you deleted the existing one.

However, BEFORE you do that, i would run "scanner -n -i ... > file" to verify what is the true content of the tape.

ble1 · Answer

Strange to see that it is not in media index when it was recognized before by recover which did say its location as well.  If you do mminfo for these volume, do you get correct data back?

Lman2 · Answer

Thanks Hrvoje,

i tried to find volume through using below command but couldn't see anything wrong with this volume.

Hp-ux:root /etc # mminfo -mav 171459

state volume written (%) expires read mounts capacity volid next type

171459 195 GB full 10/29/14 0 KB 265 800 GB 1327551246 82 LTO Ultrium-4

Please suggest if any other way to do mminfo.

Regards,

Aman

ble1 · Answer

Assuming your library is called xxx916_101 (as that is what first post suggests as per recover output), you can do nsrjb -v -j xxx916_101 and check if volid will match the one you get with mminfo. As this is NW7, nsrck -m might be good idea too (btw, nsrck -X as suggested by bingo is -L3 and you can run -L6 to get more deep check and isolate it to affected client).

Are you restoring specific data or whole saveset? If whole saveset, if above would fail to get any results, I would also suggest to run ssid restore and to see what is happening. Right now it looks like some mess or mismatch which of those suggested checks should fix, but... there is always but somewhere....

bingo.1 · Answer

There are 2 issues you face here, either individually or in combination:

- the volumes are 'near-line' not 'online'

There were issues on older NW versions where the location was not correctly updated.

- Verify the location by running mminfo -q "volume=171459" -r "volume,location"'

- Then correct the location with mmlocate –u – n 171459 name_of_your_jukebox

- the volumes and the save set tables in the media db are out of sync

You should be able to correct that by running nsrck -X

Lman2 · Answer

Hi Hrvoje,

yes, it is matched.

#######################################################################################

hp-ux:root / # mminfo -q "volume=171459" -r "volume,location"

volume location

171459 svg916_101

hp-ux:root / # nsrjb -vvv -j svg916_101 | grep 171459

setting verbosity level to `3'

413: 171459 full P917SauvPrd01 171459 1327551246 no

########################################################################################

i found this info from logs.

38752 09/30/14 22:10:44 nsrd /dev/rmt/c115t0d1BESTnb Eject operation in progress

67986 09/30/14 22:10:52 nsrmmgd Unloading volume `171459' from device `/dev/rmt/c115t0d1BESTnb' to slot 413.

7224 09/30/14 22:10:58 nsrd [Jukebox `svg916_101', operation # 10]. Expected volume ID `1327551246' for volume `171459'. The actual volume ID is `1898572862'.

Lman2 · Answer

L6 was also performed , but i wanted to see while restore index - i have same situation or not.

nsrck: /opt/networker/index/se1163 contains 0 records occupying 0 KB

nsrck: checking index for 'se1163'

nsrck: /opt/networker/index/se1164 contains 0 records occupying 0 KB

nsrck: checking index for 'se1164'

nsrck: /opt/networker/index/se11645 contains 0 records occupying 0 KB

nsrck: Completed checking 268 client(s)

[1] + Done nsrck -L6 &

Lman2 · Answer

yes, result is same after this recover command. i performed nsrck -m

below is logs from daemon.raw

0 1412113008 2 0 0 1 3663 0 xxx101-979 nsrmmdbd 2 %s 1 0 61 09/30/14 23:36:48 nsrmmdbd: media db manage oper

10103 1412113008 2 0 0 1 3663 0 xxx101-979 nsrmmdbd 28 media db verifying checksums 0

42506 1412113008 2 0 0 1 3656 0 xxx101-979 nsrd 9 %s %s: %s 3 0 14 media database 0 6 notice 49 43 10110 34

10105 1412113019 2 0 0 1 3663 0 xxx101-979 nsrmmdbd 31 media db is compressing records 0

9870 1412113044 2 0 0 1 3663 0 xxx101-979 nsrmmdbd 45 media db is consistency checking the database 0

10111 1412113058 2 0 0 1 3663 0 xxx101-979 nsrmmdbd 27 media db is record checking 0

40027 1412113058 2 0 0 1 3663 0 xxx101-979 nsrmmdbd 65 %s, volid:%s %s name:%s flags:0x%08lx kb:%s files:%lu tm:%s %s %s 10 11 38 9806:MCHK error volume with invalid id 0 1 0 0 1 0 22 6 BM4229 2 1 0 36 1 0 2 1 0 35 5 undef 35 5 undef 35 5 undef

40027 1412113058 2 0 0 1 3663 0 xxx101-979 nsrmmdbd 65 %s, volid:%s %s name:%s flags:0x%08lx kb:%s files:%lu tm:%s %s %s 10 11 38 9806:MCHK error volume with invalid id 0 1 0 0 1 0 22 6 BM3226 2 1 0 36 1 0 2 1 0 35 5 undef 35 5 undef 35 5 undef

40027 1412113058 2 0 0 1 3663 0 xxx101-979 nsrmmdbd 65 %s, volid:%s %s name:%s flags:0x%08lx kb:%s files:%lu tm:%s %s %s 10 11 38 9806:MCHK error volume with invalid id 0 1 0 0 1 0 22 6 BM2773 2 1 0 36 1 0 2 1 0 35 5 undef 35 5 undef 35 5 undef

9777 1412113092 2 0 0 1 3663 0 xxx101-979 nsrmmdbd 40 media db is cross checking the save sets 0

9778 1412113106 2 0 0 1 3663 0 xxx101-979 nsrmmdbd 38 media db is cross checking the volumes 0

0 1412113134 2 0 0 1 3663 0 xxx101-979 nsrmmdbd 2 %s 1 0 63 09/30/14 23:38:54 nsrmmdbd: media db manage operation completed

42506 1412113134 2 0 0 1 3656 0 xxx101-979 nsrd 9 %s %s: %s 3 0 14 media database 0 6 notice 49 45 10112 36 media db manage operation completed

ble1 · Answer

L7 is restore of index &#xa0; I believe suggestion was L6.

ble1 · Answer

lman wrote:

i am performing restore using specific ssid's but also tried from recover prompt and result is same.

So, if you do recover -s -S -d you get the same? If yes, we can exclude any cross index-mdb issue as ssid restore uses only mdb. In that case do nsrck -m and check daemon.raw for anything reported (as it does not report to standard output).

Lman2 · Answer

Hi bingo , thanks for quick view.

i have checked and found that volume location is good so i didn't make any changes.

hp-ux:root / # mminfo -q "volume=171459" -r "volume,location"

volume location

171459 svg916_101

hp-ux:root / # mmlocate -n 171459

Volume Location

171459 svg916_101

moreover i have performed nsrck -L7 which is successful.

hp-ux:root /etc # nsrck -L7 sb123

nsrck: checking index for 'sb123'

9343:nsrck: The file index for client 'sb123' will be recovered.Requesting 1 recover session(s) from server.

Recover completion time: Tue Sep 30 22:12:05 2014

nsrck: /opt/networker/index/sb123 contains 3453670 records occupying 583 MB

nsrck: Completed checking 1 client(s)

i am performing restore using specific ssid's but also tried from recover prompt and result is same.

ble1 · Answer

When I said to check volume id, I was referring to volume id which with mminfo you get with volid report. That one you compare against the one which nsrjb has in its db.  It seems that according to you log these two do not match.

Lman2 · Answer

hp-ux:root / # mminfo -avot -q volume=171459 -r 'savetime(20),olabel,labeled,volid,barcode'

date time orig lbl labeled volid barcode

09/23/14 03:55:15 08/26/11 09/23/14 1327551246 171459

09/23/14 03:56:27 08/26/11 09/23/14 1327551246 171459

09/23/14 03:56:59 08/26/11 09/23/14 1327551246 171459

09/23/14 03:57:22 08/26/11 09/23/14 1327551246 171459

09/23/14 13:01:06 08/26/11 09/23/14 1327551246 171459

09/23/14 13:01:45 08/26/11 09/23/14 1327551246 171459

09/23/14 13:02:01 08/26/11 09/23/14 1327551246 171459

09/23/14 13:50:46 08/26/11 09/23/14 1327551246 171459

09/23/14 19:20:08 08/26/11 09/23/14 1327551246 171459

09/23/14 19:20:22 08/26/11 09/23/14 1327551246 171459

09/23/14 19:20:29 08/26/11 09/23/14 1327551246 171459

09/23/14 19:20:37 08/26/11 09/23/14 1327551246 171459

09/23/14 19:20:40 08/26/11 09/23/14 1327551246 171459

09/23/14 19:20:45 08/26/11 09/23/14 1327551246 171459

09/23/14 21:45:29 08/26/11 09/23/14 1327551246 171459

09/23/14 22:00:50 08/26/11 09/23/14 1327551246 171459

09/23/14 22:01:16 08/26/11 09/23/14 1327551246 171459

09/23/14 22:01:22 08/26/11 09/23/14 1327551246 171459

09/23/14 22:02:07 08/26/11 09/23/14 1327551246 171459

09/23/14 22:02:51 08/26/11 09/23/14 1327551246 171459

09/23/14 22:03:23 08/26/11 09/23/14 1327551246 171459

09/23/14 22:03:53 08/26/11 09/23/14 1327551246 171459

09/23/14 22:05:23 08/26/11 09/23/14 1327551246 171459

09/23/14 22:06:16 08/26/11 09/23/14 1327551246 171459

09/23/14 22:06:36 08/26/11 09/23/14 1327551246 171459

09/23/14 22:06:50 08/26/11 09/23/14 1327551246 171459

09/23/14 22:07:25 08/26/11 09/23/14 1327551246 171459

09/23/14 22:08:01 08/26/11 09/23/14 1327551246 171459

09/23/14 22:08:21 08/26/11 09/23/14 1327551246 171459

09/23/14 22:08:27 08/26/11 09/23/14 1327551246 171459

09/23/14 22:08:35 08/26/11 09/23/14 1327551246 171459

09/23/14 22:08:38 08/26/11 09/23/14 1327551246 171459

09/23/14 22:08:56 08/26/11 09/23/14 1327551246 171459

09/23/14 22:09:01 08/26/11 09/23/14 1327551246 171459

09/23/14 22:09:31 08/26/11 09/23/14 1327551246 171459

09/23/14 22:30:53 08/26/11 09/23/14 1327551246 171459

09/23/14 22:31:26 08/26/11 09/23/14 1327551246 171459

09/23/14 22:31:35 08/26/11 09/23/14 1327551246 171459

09/23/14 22:31:44 08/26/11 09/23/14 1327551246 171459

09/23/14 23:15:31 08/26/11 09/23/14 1327551246 171459

09/23/14 23:29:09 08/26/11 09/23/14 1327551246 171459

09/24/14 00:15:21 08/26/11 09/23/14 1327551246 171459

09/24/14 00:16:28 08/26/11 09/23/14 1327551246 171459

09/24/14 00:16:53 08/26/11 09/23/14 1327551246 171459

09/24/14 00:16:57 08/26/11 09/23/14 1327551246 171459

09/24/14 00:17:02 08/26/11 09/23/14 1327551246 171459

09/24/14 00:17:08 08/26/11 09/23/14 1327551246 171459

09/24/14 00:17:14 08/26/11 09/23/14 1327551246 171459

09/24/14 00:17:19 08/26/11 09/23/14 1327551246 171459

09/24/14 00:18:54 08/26/11 09/23/14 1327551246 171459

09/24/14 00:19:22 08/26/11 09/23/14 1327551246 171459

09/24/14 00:29:46 08/26/11 09/23/14 1327551246 171459

09/24/14 00:36:13 08/26/11 09/23/14 1327551246 171459

09/24/14 00:36:22 08/26/11 09/23/14 1327551246 171459

09/24/14 00:37:07 08/26/11 09/23/14 1327551246 171459

09/24/14 00:37:12 08/26/11 09/23/14 1327551246 171459

09/24/14 01:15:11 08/26/11 09/23/14 1327551246 171459

09/24/14 01:40:49 08/26/11 09/23/14 1327551246 171459

09/24/14 02:57:46 08/26/11 09/23/14 1327551246 171459

09/24/14 03:45:43 08/26/11 09/23/14 1327551246 171459

09/24/14 03:49:16 08/26/11 09/23/14 1327551246 171459

ble1 · Answer

Do following:

mminfo -avot -q volume=171459 -r 'savetime(20),olabel,labeled,volid,barcode'

This will list savetime of savesets, first time tape was labeled, last time labeled, volume id and barcode associated with volume.

ble1 · Answer

ok, now, this tape, according to mdb, has been labeled first time in August and last time 7 days ago. From what NW mdb knows it is volume with barcode (and volume name) 171459 and volid 1327551246 ,

The issue you have is that when tape is loaded and label gets verified, it finds volid which is 1898572862. I have no idea what that is, but we can ask again NW; do:

mminfo -q volid=1898572862 -r -r 'savetime(20),olabel,labeled,volume,barcode'

This is similar to previous command except that we now query this different volid which is found on tape and we wish to see if this has been in mdb before and if yes under which volume name. Do you reuse barcodes by any chance or print them?

NetWorker

NOT in media index

Was this post helpful?