Highlighted
bakr1
1 Copper

why the tape drives not working nor discoverable for netapp filer in cluster mode.

hi all ,

we are facing a strange issue, briefly our backup structure is as follow:

we have two NetApp filers : 1st  7-mode  and the 2nd cluster mode connected to a SAN switch and connected to the backup server which have EMC networker v 8.1 and then connected to IBM tape library which have 10 drives.(we use NDMP backup)

backup was forking fine till suddenly the cluster mode filer was unable to take backups; suddenly the networker gives error read open error the device is in use for the two drives dedicated for this filer so i had to re-configure the library on the networker and assign another two tape drives but apparently the previous two tapes drives are not available to be assigned again for this particular filer while being available to the others.

this error message kept on coming again and again, now am out of any tape drive for this filer they seem to be undiscoverable or they just disappeared.

running inquire -N -on 1st filer am able to see my 10 drives

running inquire -N -2nd filer (cluster mode) am able to see only a library and other tape drives are seen busy as below

scsidev@255.255.255:IBM ULT3580-TD6     C7S4|Tape, nrst3a

S/N: 00078B00A9

ATNN=IBM     ULT3580-TD6     00078B00A9

WWNN=500507604409B204

WWPN=500507604449B204

PORT=00000001

encryption unsupported

0 Tape, 1 Autochanger (Jukebox), Total: 1

Device 'nrst6a' is busy. It cannot be opened at this time.

Device 'nrst7a' is busy. It cannot be opened at this time.

Device 'nrst1a' is busy. It cannot be opened at this time.

Device 'nrst4a' is busy. It cannot be opened at this time.

Device 'nrst0a' is busy. It cannot be opened at this time.

Device 'nrst2a' is busy. It cannot be opened at this time.

Device 'nrst5a' is busy. It cannot be opened at this time.

  Device 'nrst8a' is busy. It cannot be opened at this time.

any ideas why could this be happening here ?

Labels (1)
0 Kudos
7 Replies
ble1
6 Indium

Re: why the tape drives not working nor discoverable for netapp filer in cluster mode.

cluster mode can be bitch.  When you storage tape show-tape-drive and system node hardware tape drive show - how many and which do you see assigned?  Which OnTap version do you use?  How do you access drives - over node or cluster alias?

0 Kudos
bakr1
1 Copper

Re: why the tape drives not working nor discoverable for netapp filer in cluster mode.

all are showed perfectly from the netapp filer as below:

OnTap version 8.2.4.p6

we use node scope level.

BPcDOT::> tape drive show

  (system node hardware tape drive show)

Node   Device Id Drive Description    NDMP Path

------ --------- -------------------- --------------------------------------

rmeegyss002

       Main-SAN-Switch:1-1.126        nrst0l nrst0m nrst0h nrst0a

                 IBM LTO-6 ULT3580    rst0l rst0m rst0h rst0a

                                      urst0l urst0m urst0h urst0a

       Main-SAN-Switch:1-14.126       nrst8l nrst8m nrst8h nrst8a

                 IBM LTO-6 ULT3580    rst8l rst8m rst8h rst8a

                                      urst8l urst8m urst8h urst8a

       Main-SAN-Switch:1-2.126        nrst2l nrst2m nrst2h nrst2a

                 IBM LTO-6 ULT3580    rst2l rst2m rst2h rst2a

                                      urst2l urst2m urst2h urst2a

       Main-SAN-Switch:1-3.126        nrst3l nrst3m nrst3h nrst3a

                 IBM LTO-6 ULT3580    rst3l rst3m rst3h rst3a

                                      urst3l urst3m urst3h urst3a

       Main-SAN-Switch:1-4.126        nrst5l nrst5m nrst5h nrst5a

                 IBM LTO-6 ULT3580    rst5l rst5m rst5h rst5a

                                      urst5l urst5m urst5h urst5a

       Main-SAN-Switch:1-5.126        nrst1l nrst1m nrst1h nrst1a

                 IBM LTO-6 ULT3580    rst1l rst1m rst1h rst1a

                                      urst1l urst1m urst1h urst1a

       Main-SAN-Switch:1-6.126        nrst7l nrst7m nrst7h nrst7a

                 IBM LTO-6 ULT3580    rst7l rst7m rst7h rst7a

                                      urst7l urst7m urst7h urst7a

       Main-SAN-Switch:1-7.126        nrst6l nrst6m nrst6h nrst6a

                 IBM LTO-6 ULT3580    rst6l rst6m rst6h rst6a

                                      urst6l urst6m urst6h urst6a

       Main-SAN-Switch:1-8.126        nrst4l nrst4m nrst4h nrst4a

                 IBM LTO-6 ULT3580    rst4l rst4m rst4h rst4a

                                      urst4l urst4m urst4h urst4a

       Main-SAN-Switch:1-9.126        nrst9l nrst9m nrst9h nrst9a

                 IBM ULT3580-TD2      rst9l rst9m rst9h rst9a

                                      urst9l urst9m urst9h urst9a

10 entries were displayed.

0 Kudos
ble1
6 Indium

Re: why the tape drives not working nor discoverable for netapp filer in cluster mode.

Do you have single or multiple nodes as owner of the drives?

0 Kudos
bakr1
1 Copper

Re: why the tape drives not working nor discoverable for netapp filer in cluster mode.

1 single node

0 Kudos
ble1
6 Indium

Re: why the tape drives not working nor discoverable for netapp filer in cluster mode.

And volumes writing to that tape as on the same volume or do they send data over interconnect?  Since you started this thread I was wondering if tape goes off/busy due some failover mechanism which keeps hanging it in some strange state.I assume you already checked for active NDMP sessions on node to see there is nothing running?  What happens if you restart ndmp service?  Are those tape drives back as not busy?

0 Kudos
bakr1
1 Copper

Re: why the tape drives not working nor discoverable for netapp filer in cluster mode.

they are still listed busy though we restarted the NDMP services, we tried also restarting the backup server and nothing actually happened, i agree with you that there might be a service that make the tape drives go busy but actually the drives are working fine with the other netapp and they are not seen busy.

0 Kudos
ble1
6 Indium

Re: why the tape drives not working nor discoverable for netapp filer in cluster mode.

Yes, but others are mode7, right?  NDMP was not top of the head when cluster mode came out and you can see with each new version OnTap is closing this gap, but there are still certain things left to be desired.  I have cluster mode, but we don't use DSA instead (as we use DD so I just push things over network).

0 Kudos