NetWorker upgrade to V7.6.x with NDMP causes backups to fail

This Ask the expert event is now open for questions. Drop your question here and Rita will reply to it. We look forward to your participation.

Regards,

Mark

For those of you who have already upgraded to NetWorker V7.6.x I am sure you have already experienced the connection issues that arise when using NetWorker's NDMPdsa feature to perform NDMP backups of your NAS filers.

For those planning the upgrade or configuring NDMPdsa for the first time, perhaps this discussion will help you prepare for some of the configuration issues that may cause the connection failures to occur.

There are many factors that can cause the failure message:

"nsrndmp_save: data connect: failed to establish connection"

The cause of the failure really depends on your environment and configuration. Things to consider, but are not limited to:

• Is the Firewall enabled and running on your W2K8 or Windows7 NetWorker Server?
• If using a Linux NetWorker server, do you have the iptables defined?
• Do you have an entry in the /etc/hosts file of the NetWorker Server for the NetWorker Server itself?
• Do you have multiple NIC cards active in the NetWorker server?
• A firewall exists between the NetWorker Server and the NAS filer and TCP ports are blocked
• NDMP Service is not started on the NAS server
• NDMP user name or password issue, or NDMP password authentication method is incorrect

If you are preparing to configure NDMPdsa for the first time, or have recently upgraded to NetWorker V7.6.x and your NDMPdsa backups are suddenly failing, please post your questions here and I will be happy to address them.

Hello Rita,

I know that TCP port 10000 is required and needs to be open on the NAS for NetWorker NDMP connections. However, during NDMP-DSA backups I see a number of other ports used from the NAS back to NetWorker besides port 10000. Is that expected? Do I need to make sure that more than port 10000 is open between the NAS and NetWorker? If yes, what is the port range?

Thanks!

Hi George:

Port 10000 is the default port used by NDMP to do the initial handshake between the NAS filer and the NetWorker server for both NDMP backups using NDMP devices as well as NDMPdsa backups (using a non-NDMP device on the NetWorker Server). 

In addition to port 10000 NDMP also uses the NetWorker Server port range for passing the metadata, messages, etc.  So in addition to having port 10000 open you must also consider the NetWorker Port range to insure that these ports can be used for NDMP as well.

If you have a firewall between your NetWorker Server and the NAS filer, port 10000 as well as the NetWorker Port Range will need to be open there as well.  The default NetWorker Port Range is 7937-9936.

Regards,

Rita Mines

i am sorry for asking this question..
can RSA Encryp a image? if it can how it solve the biggest number of factorial..?
i am sorry for dusturb you...

Hi Drinia:

Not sure I understand your question?  Are you attempting to use a VPN connection via RSA security and do backups of something via NDMP?

Regards,

Rita Mines

Hello Rita,

i have checked all your points, firewall and so on.

i have always the following error messages.

savegrp: suppressed 2 lines of output.
NDMP Service Debug: The process id for NDMP service is 0x63da60
42794:nsrndmp_save: Performing backup to Non-NDMP type of device
42658:nsrdsa_save: DSA savetime = 1334919167
NDMP Service Debug: The process id for NDMP service is 0x63da60
42909:nsrndmp_save: Performing DAR Backup..
83563:nsrndmp_save: Performing full backup, BASE_DATE = 0
Host = xxx.xxx..com (xxx.xxx.xxx.xxx) port = 9218
42597:nsrndmp_save: data connect: failed to establish connection
42886:nsrndmp_save: Unable to start the NDMP backup process.
42913:nsrndmp_save: Save session closed with NW server successfully

backup failed.
:  Failed with error(s)

On the Celerra i have to error messages

Session 212 (thread ndmp212) NdmpdMover::connect: connect error: Connection refused

Session 212 (thread ndmp212) Invalid state to stop mover (0).

The error messages on the celerra are at the same time, when the backup job starts

Regards Frank

HI Frank:

What platform is your NetWorker server?  Windows 2008, Windows 2003, Solaris, etc.?

Is this a new install?  Or has this never worked?

If it has never worked, the first thing I would suggest is to add an entry to the /etc/hosts file of the NetWorker server for the NetWorker itself.  Yes, that's right.....add the NetWorker Server to it's own /etc/hosts file.  The entry must be very specific based on the "Client Name" that the NetWorker server is known by in NetWorker.

If the NetWorker Servers client name is the FQDN (Fully Qualified Domain Name) then the entry should be:

IP ADDRESS   FQDN  Shortname

If on the other hand, the client name of the NetWorker server is the shortname, it must be:

IP ADDRESS Shortname FQDN

Also, do you have multiple NIC's in the NetWorker server?  If yes, how are they configured failover or load balancing?

If this is Windows2008 is the windows firewall disabled?

Looking forward to your response.

Regards,

Rita Mines

Hi Rita

If you dont have index (failed/deleted/corrupted), can you rebuild the index by scanning the data and browse thereafter?

Backup using ndmp dsa method. I knw it's not supported in native ndmp format

Thank you

Hi Capa:

Although you are using NDMPdsa the same rules apply for the index as would be true when doing the backup to an NDMP device.  Keep in mind that regardless of which method you use (NDMP or NDMPdsa) the NAS filer still generates the backup image and passes the raw metadata to NetWorker to build the index.  The only thing that is really different with NDMPdsa is that the backup image is passed back to NetWorker where the "dsa" (data service agent) processing is done to put the backup image in the NetWorker proprietary tape format so that it can then be written to the non-NDMP NetWorker device.

As with NDMP, when doing NDMPdsa the index cannot be rebuilt using scanner -i from the backup tape.  This is because format of the backup image itself is NOT in the NetWorker proprietary format.

If the index generation failed at the time of backup it also cannot be "rebuilt" since it never really generated at the time of backup.  If the index was built successfully at the time of backup (regardless of whether you ran just an NDMP backup or NDMPdsa) you can restore the index backup from tape back into the index directory and then run nsrck -L6 -c -t to rebuild the index.  (Where is the name of the NDMP client and is the date/time of the backup).

Hope that helps.  Please let me know if I can clarify anything further.

Regards,

Rita Mines

Hi Rita

Had same issue/query.

In the Scanner Command Usage Guide

Scanner usage with NDMP data

Scanner of NDMP data will import data into the media database only. The “-i” option isn’t available as the index is generated as part of the backup itself (also known as “History” file).

While writing to tape, NDMP data will be written with a different behavior than normal data. Each NDMP saveset will take three filemarks instead of one.

NDMP is a different protocol from OTF and is written to tape differently than OTF formatted data. NDMP does not support interleaving (multiplexing) of datastreams and each NDMP saveset is separated by three consecutive filemarks.

Physically, the file numbers are still written to tape and fn2 will be seen while running a scanner command. Fn2 is a header for the NDMP saveset that NetWorker recognizes. The filemarks are enumerated in the same way, and on a NetWorker volume, data still begins at filenumber two (meaning the second NDMP saveset begins at file 5, third at 8, and so on).

This will only be the case for direct NDMP backup. NDMP DSA backup is a hybridization of NDMP and OTF and retains an OTF structure for NDMP data – thus separating with a single filemark instead of 3.

A NDMP scanned-in media will therefore only be usable for saveset recoveries. While a NDMP DSA backup can be treated as a filesystem backup, thus scanner –i will work normally.

NDMP DSA backup also behaves similarly to OTF backups in that it records saverec index stubs, allowing scanner with the –i switch to rebuild indices for savesets scanned in. Regular NDMP volumes cannot use the –i switch to rebuild index data; the only recovery possible will be saveset recovery.

Hi Thierry:

First let me clarify that this information is NOT correct.  An NDMP backup is an NDMP backup image regardless of whether NDMPdsa was used or it was a direct "NDMP" backup.  The only real difference between the two is that the NDMPdsa backup is sent to the NetWorker server or a NetWorker Storage Node where "dsa processing" is nothing more than "putting a wrapper around the NAS filer's native backup stream" so that it then looks like a NetWorker proprietary tape format.  Underneath that "dsa wrapper" it is still a backup image in the NAS filer's native backup format.  Therefore, scanner cannot read it to generate an index from that image.

That being said, can you please tell me where you found this information?  If it is from EMC I will do my best to get it corrected ASAP.

Regards,

Rita Mines

https://powerlink.emc.com/nsepn/webapps/btg548664833igtcuup4826/km/live1//en_US/Offering_Technical/Technical_Documentation/300-012-537.pdf

Hi Rita

Thanks for clarification as it has caused confusions.. it was dowloaded from powerlink as attached..

Hi Rita,

thanks for the fast reply.

I forgot the entry in the /etc/hosts file, now everything is working fine in the backup.

With NDMP Recovery we have a strange issue, the Networker Server opens over 20000 Network Sockets to the NS-480

and after a while he can´t open more sockets and the server hangs.

Regard Frank