NetWorker: NDMP restores of large PowerScale (Isilon) save sets failing with error: Invalid archive format: Inappropriate file type or format
Summary: When recovering large folder structures with many thousands of files. Some of the data is restored but some is omitted. Rerunning the restore of the specific missing files or folders works successfully. All data can be recovered, but it is time-consuming and repetitive. Affects Isilon Multi-Stream backups and restores. Any reference to Isilon is for PowerScale. ...
Symptoms
Some of the data is restored but some is omitted.
Rerunning the restore of the specific missing files or folders works successfully.
All data can be recovered, but it is time-consuming and repetitive.
Error messages seen in the restore logs are:
nsrndmp_recover:ssid'XXXXXXXXXX': NDMP Service Error: Failed to parse data at XXXXXXXXXXXXXXX: Invalid archive format: Inappropriate file type or format
Cause
By default, the Isilon settings for DMA are set to generic.
Run this on the Isilon cluster command to see current settings
isi ndmp settings global view
Example output:
Service: True
Port: 10000
DMA: generic
Bre Max Num Contexts: 64
MSB Context Retention Duration: 300
MSR Context Retention Duration: 600
Stub File Open Timeout: 15
Enable Redirector: False
Enable Throttler: False
Throttler CPU Threshold: 50
On the Isilon, the setting mentioned in the Isilon OneFS 9.4.0.0 Documentation page 54.
Also documented in NetWorker online guide here, showing the command you must use.
Resolution
Before taking any backups with NetWorker backup software of the Isilon cluster using NDMP, the setting must be set for dma=emc
Configure and enable NDMP backup OneFS prevents NDMP backups by default. Before you can perform NDMP backups, you must enable NDMP backups and configure NDMP settings.
1. Enable NDMP backup by running the following command:
isi ndmp settings global modify --service=true
2. Configure NDMP backup by running the isi ndmp settings set command. The following command configures OneFS to interact with NetWorker:
isi ndmp settings global modify --dma=emc
The setting defining dma=emc is critical and must be set for all Isilon installations where the backup is performed by the Dell NetWorker backup software.
Once it is set, you see:
Service: True
Port: 10000
DMA: emc
Bre Max Num Contexts: 64
MSB Context Retention Duration: 300
MSR Context Retention Duration: 600
Stub File Open Timeout: 15
Enable Redirector: False
Enable Throttler: False
Throttler CPU Threshold: 50
3. Perform a NEW FULL BACKUP for the new setting to take effect.
If there are any restores required prior to the dma=emc was set, use the command below to restore the data that was backed up:
Restore it to a new alternate location on the same Isilon:
nsrndmp_recover -s {backup server name} -c {NDMP client name} -S {SSID} -v off -m {NDMP client name}::{target to restore data path "{source of data to be restored" >recover.log 2>&1
Example:
nsrndmp_recover -s networker01.com -c isilon-clu.com -S {SSID} -v off -m isilon-clu.com::/ifs/xyz/folder1/restore_folder "/ifs/xyz/new_folder99/data_to_be_restored" >recover.log 2>&1