
UNSOLVED
I/O Device Error: Bad Tape, Bad Drive, or Bad Networker?
Running Legato Networker 7.2.2 J on Windows 2003. Recently upgraded from 7.1.3.
Seeing issues in our daemon.log indicating that I/O errors are aborting backups or are finishing tapes long before they've reached capacity. This is resulting in several of our larger save sets to go un-backed up without multiple retries. This issue is spanning multiple servers of various purposes, but seems to be tied mostly to larger save sets (200g+).
This issue started, more or less, after the upgrade. Could be a coincidence. Doesn't seem to be consistent with tape numbers (some are much older, some are fairly new). Doesn't seem to be consistent with drives (this example spans two different ones).
Below is a daemon.log sample. There are others related to this that are occurring fairly regularly. I figured I'd check here to get an opinion as to source before I escalated to EMC Support.
Thank you for any help our advice you can provide!
------------------------------------------------------------------
03/29/07 09:43:13 nsrd: media warning: \\.\Tape0 writing: The request could not be performed because of an I/O device error., at file 35 record 25505
03/29/07 09:43:13 nsrd: media notice: LTO Ultrium-2 tape XXXX on \\.\Tape0 is full
03/29/07 09:43:13 nsrd: media notice: LTO Ultrium-2 tape XXXX used 70 GB of 190 GB capacity
03/29/07 09:43:13 nsrd: media warning: verification of volume "XXXX", volid 3388920884 failed, read open error: drive status is Drive reports no error - but state is unknown
03/29/07 09:43:13 nsrd: media notice: verification of volume "XXXX", volid 3388920884 failed, volume is being marked as full.
03/29/07 09:43:13 nsrmmd #30: Diagnostic: remember_as_high: no available sop for ssid 3658139812!
03/29/07 09:43:13 nsrd: write completion notice: Writing to volume XXXX complete
03/29/07 09:43:13 nsrd: media notice: Save set (3658139812) client1.domain.com:F:\ volume XXXX on \\.\Tape0 is being terminated because: Media verification failed
03/29/07 09:43:13 nsrd: client1.domain.com:F:\ done saving to pool 'POOL' (XXXX) 276 GB
03/29/07 09:43:40 savegrp: command 'save -s 10.231.4.13 -g POOLGrp -LL -m client1.domain.com -l full -q -W 78 -N F:\ F:\ ' for client client1.domain.com exited with return code 255.
03/29/07 09:43:40 savegrp: client1.domain.com:F:\ will retry 1 more time(s)
03/29/07 09:43:42 nsrd: media info: suggest mounting XXYY on tapeserver.domain.com for writing to pool 'POOL'
03/29/07 09:43:42 nsrd: media waiting event: Waiting for 1 writable volumes to backup pool 'POOL' tape(s) on tapeserver.domain.com
03/29/07 09:43:43 nsrd: \\.\Tape3 Eject operation in progress
03/29/07 09:44:13 nsrd: media info: loading volume XXYY into \\.\Tape3
03/29/07 09:44:25 nsrd: \\.\Tape3 Verify label operation in progress
03/29/07 09:44:40 nsrd: \\.\Tape3 Mount operation in progress
03/29/07 09:44:54 nsrd: media event cleared: Waiting for 1 writable volumes to backup pool 'POOL' tape(s) on tapeserver.domain.com
03/29/07 09:44:54 nsrd: client1.domain.com:F:\ saving to pool 'POOL' (XXYY)
03/29/07 09:53:57 nsrd: media warning: \\.\Tape3 writing: The request could not be performed because of an I/O device error., at file 2 record 24247
03/29/07 09:53:57 nsrd: media notice: LTO Ultrium-2 tape XXYY on \\.\Tape3 is full
03/29/07 09:53:57 nsrd: media notice: LTO Ultrium-2 tape XXYY used 1551 MB of 190 GB capacity
03/29/07 09:53:57 nsrd: media info: verification of volume "XXYY", volid 3372143768 succeeded.
03/29/07 09:53:57 nsrd: write completion notice: Writing to volume XXYY complete
03/29/07 09:53:57 nsrd: media info: suggest mounting XXXY on tapeserver.domain.com for writing to pool 'POOL'
03/29/07 09:53:57 nsrd: media waiting event: Waiting for 1 writable volumes to backup pool 'POOL' tape(s) on tapeserver.domain.com
03/29/07 09:53:58 nsrd: \\.\Tape3 Eject operation in progress
03/29/07 09:54:26 nsrd: media info: loading volume XXXY into \\.\Tape3
Seeing issues in our daemon.log indicating that I/O errors are aborting backups or are finishing tapes long before they've reached capacity. This is resulting in several of our larger save sets to go un-backed up without multiple retries. This issue is spanning multiple servers of various purposes, but seems to be tied mostly to larger save sets (200g+).
This issue started, more or less, after the upgrade. Could be a coincidence. Doesn't seem to be consistent with tape numbers (some are much older, some are fairly new). Doesn't seem to be consistent with drives (this example spans two different ones).
Below is a daemon.log sample. There are others related to this that are occurring fairly regularly. I figured I'd check here to get an opinion as to source before I escalated to EMC Support.
Thank you for any help our advice you can provide!
------------------------------------------------------------------
03/29/07 09:43:13 nsrd: media warning: \\.\Tape0 writing: The request could not be performed because of an I/O device error., at file 35 record 25505
03/29/07 09:43:13 nsrd: media notice: LTO Ultrium-2 tape XXXX on \\.\Tape0 is full
03/29/07 09:43:13 nsrd: media notice: LTO Ultrium-2 tape XXXX used 70 GB of 190 GB capacity
03/29/07 09:43:13 nsrd: media warning: verification of volume "XXXX", volid 3388920884 failed, read open error: drive status is Drive reports no error - but state is unknown
03/29/07 09:43:13 nsrd: media notice: verification of volume "XXXX", volid 3388920884 failed, volume is being marked as full.
03/29/07 09:43:13 nsrmmd #30: Diagnostic: remember_as_high: no available sop for ssid 3658139812!
03/29/07 09:43:13 nsrd: write completion notice: Writing to volume XXXX complete
03/29/07 09:43:13 nsrd: media notice: Save set (3658139812) client1.domain.com:F:\ volume XXXX on \\.\Tape0 is being terminated because: Media verification failed
03/29/07 09:43:13 nsrd: client1.domain.com:F:\ done saving to pool 'POOL' (XXXX) 276 GB
03/29/07 09:43:40 savegrp: command 'save -s 10.231.4.13 -g POOLGrp -LL -m client1.domain.com -l full -q -W 78 -N F:\ F:\ ' for client client1.domain.com exited with return code 255.
03/29/07 09:43:40 savegrp: client1.domain.com:F:\ will retry 1 more time(s)
03/29/07 09:43:42 nsrd: media info: suggest mounting XXYY on tapeserver.domain.com for writing to pool 'POOL'
03/29/07 09:43:42 nsrd: media waiting event: Waiting for 1 writable volumes to backup pool 'POOL' tape(s) on tapeserver.domain.com
03/29/07 09:43:43 nsrd: \\.\Tape3 Eject operation in progress
03/29/07 09:44:13 nsrd: media info: loading volume XXYY into \\.\Tape3
03/29/07 09:44:25 nsrd: \\.\Tape3 Verify label operation in progress
03/29/07 09:44:40 nsrd: \\.\Tape3 Mount operation in progress
03/29/07 09:44:54 nsrd: media event cleared: Waiting for 1 writable volumes to backup pool 'POOL' tape(s) on tapeserver.domain.com
03/29/07 09:44:54 nsrd: client1.domain.com:F:\ saving to pool 'POOL' (XXYY)
03/29/07 09:53:57 nsrd: media warning: \\.\Tape3 writing: The request could not be performed because of an I/O device error., at file 2 record 24247
03/29/07 09:53:57 nsrd: media notice: LTO Ultrium-2 tape XXYY on \\.\Tape3 is full
03/29/07 09:53:57 nsrd: media notice: LTO Ultrium-2 tape XXYY used 1551 MB of 190 GB capacity
03/29/07 09:53:57 nsrd: media info: verification of volume "XXYY", volid 3372143768 succeeded.
03/29/07 09:53:57 nsrd: write completion notice: Writing to volume XXYY complete
03/29/07 09:53:57 nsrd: media info: suggest mounting XXXY on tapeserver.domain.com for writing to pool 'POOL'
03/29/07 09:53:57 nsrd: media waiting event: Waiting for 1 writable volumes to backup pool 'POOL' tape(s) on tapeserver.domain.com
03/29/07 09:53:58 nsrd: \\.\Tape3 Eject operation in progress
03/29/07 09:54:26 nsrd: media info: loading volume XXXY into \\.\Tape3
Responses (0)
Solutions (0)
