NetWorker: NDMP Data Recovery Fails Due to Halted or ABORTED Savesets
Summary: Restore fails with Already at end of tape and Could not initialize media errors, caused by savesets on a Network Data Management Protocol (NDMP) tape with file numbers not 3 or multiples of 3. ...
Symptoms
NetWorker Data Management Protocol (NDMP) restore fails. The nsrndmp_recover output shows the following messages:
nsrndmp_recover:ssid'477171455': NDMP Service Log: RESTORE: Already at the end of tape. nsrndmp_recover:ssid'477171455': NDMP Service Log: RESTORE: Could not initialize media. nsrndmp_recover:ssid'477171455': NDMP Service Log: RESTORE: RESTORE IS ABORTED nsrndmp_recover:ssid'477171455': Data server halted: Error during the restore. nsrndmp_recover:ssid'477171455': NDMP Service Log: Dump aborted. nsrndmp_recover:ssid'477171455': Tape server halted: Error during the restore. nsrndmp_recover:ssid'477171455': NDMP Service Log: MoveletInput: Internal Error. nsrndmp_recover:ssid'477171455': NDMP data server has an internal error. nsrndmp_recover:ssid'477171455': Error during File NDMP Extraction. nsrndmp_recover:ssid'477171455': NDMP recover failed. nsrndmp_recover:ssid'477171455': Error during NDMP recover NDMP retrieval: child failed with status of 1
Cause
These errors can appear for several reasons, but in this case the issue is caused by halted or aborted save sets occurring on an NDMP backup tape.
Check the media database entries for the NDMP tape using:
mminfo -q volume=volume-name -r"name,client,ssid,mediafile,mediarec,mediamark"
If there are any save sets on the tape where the file number is not equal to 3 or a multiple of 3, then the normal NDMP recovery results in a failure to recover the data as the normal behavior causes NDMP tapes to automatically position the NDMP tape to file number 3 or a multiple of 3 to begin recovering the data.
Resolution
If the NDMP data save sets file position on the tape is not 3 or a multiple of 3 then the following steps allow the data to be recovered:
- Create an empty file in
nsr\debugdirectory calledndmp_auto_pos
- Linux:
/nsr - Windows (Default):
C:\Program Files\EMC NetWorker\nsr
- Stop and restart the NetWorker server services or daemons:
- Linux:
systemctl restart networker - Windows (PowerShell):
net stop nsrexecd /y ; net start nsrd- If the Windows host is also a NetWorker Management Console (NMC) server, you must also start the NMC service:
net start gstd
- If the Windows host is also a NetWorker Management Console (NMC) server, you must also start the NMC service:
- Retry the NDMP recovery
Once the NDMP data has been recovered, remove the ndmp_auto_pos file from the nsr\debug directory and stop and restart the NetWorker server services or daemons so that normal NDMP recovery behavior is restored.
Additional Information