Avamar: Fatal Server Connection Problem When Using NDMP to Data Domain

• Backup process aborts with "Fatal server connection problem, aborting initialization."
• Log reports "illegal operation" (DDR result code: 5037) when attempting to open the Data Domain server.
• Message "Problem logging into the DDR server: '', only GSAN communication was enabled."
• Fatal error indicating missing file "/ddr_files.xml.”
• Backup not posted to DDR and not committed.

Relevant log excerpt:

2016-07-12 23:57:30 avtar Info <19156>: - Establishing a connection to the Data Domain system with encryption (Connection mode: A:3 E:2).
2016-07-12 23:58:40 avtar Warning <18125>: Calling DDR_OPEN returned result code:5037 message:illegal operation
2016-07-12 23:58:40 avtar Error <10542>: Data Domain server "berbedd640.corp.quick-restaurants.com" open failed DDR result code: 5037, desc: illegal operation
2016-07-12 23:58:40 avtar Error <10509>: Problem logging into the DDR server:'', only GSAN communication was enabled.
2016-07-12 23:58:40 avtar FATAL <17964>: Backup is incomplete because file "/ddr_files.xml" is missing
2016-07-12 23:58:40 avtar Info <10642>: DDR errors caused the backup to not be posted, errors=0, fatals=0
2016-07-12 23:58:40 avtar Info <12530>: Backup was not committed to the DDR.
2016-07-12 23:58:40 avtar FATAL <8941>: Fatal server connection problem, aborting initialization. Verify correct server address and login credentials and that your server supports un-encrypted network connections.

The NDMP backup failed because the Avamar client could not establish a valid connection to the Data Domain system.

• Hostname resolution failure - the Data Domain server name could not be resolved to an IP address, preventing any network communication.
• Network connectivity problem - without a resolvable address, the client could not reach the Data Domain appliance, leading to connection time‑outs.
• Authentication and protocol mismatch - the client attempted an encrypted connection (mode A:3 E:2) while the Data Domain server either did not support the requested encryption level or required different credentials, resulting in DDR error code 5037 ("illegal operation").
• Missing required DDR file - the backup process could not locate /ddr_files.xml, a symptom of the failed initialization caused by the connection issue.

Relevant log excerpts illustrating the failure:

2016-07-12 23:57:30 avtar Info <19156>: - Establishing a connection to the Data Domain system with encryption (Connection mode: A:3 E:2).
2016-07-12 23:58:40 avtar Warning <18125>: Calling DDR_OPEN returned result code:5037 message:illegal operation
2016-07-12 23:58:40 avtar Error <10542>: Data Domain server "berbedd640.corp.quick-restaurants.com" open failed DDR result code: 5037, desc: illegal operation
2016-07-12 23:58:40 avtar Error <10509>: Problem logging into the DDR server:'', only GSAN communication was enabled.
2016-07-12 23:58:40 avtar FATAL <17964>: Backup is incomplete because file "/ddr_files.xml" is missing
2016-07-12 23:58:40 avtar FATAL <8941>: Fatal server connection problem, aborting initialization. Verify correct server address and login credentials and that your server supports un-encrypted network connections.

Fixing "Fatal server connection problem, aborting initialization" for NDMP backups:

Resolution steps:

1. Open the hosts file on the NDMP node and add the Data Domain’s fully‑qualified domain name (FQDN) with its correct IP address.

2. Save the file and clear any DNS cache.

3. Verify that the NDMP node resolves the FQDN to the expected IP address.

4. Confirm that the NDMP configuration uses the FQDN (not a short hostname) for the Data Domain target.

5. If encryption is enabled, ensure that the Data Domain supports the selected mode or switch to an unencrypted connection if required.

6. Restart the NDMP service to apply the changes.

7. Run a test NDMP backup and review the logs for successful connection.

Commands and examples:

• Edit the hosts file (Linux example)

sudo vi /etc/hosts

• • Add an entry (replace with actual values):

192.168.10.50   data-domain.example.com

• Flush the DNS cache (if applicable):

sudo systemd-resolve --flush-caches

• Verify resolution:

ping -c 4 data-domain.example.com

PING data-domain.example.com (192.168.10.50) 56(84) bytes of data.
64 bytes from 192.168.10.50: icmp_seq=1 ttl=64 time=0.123 ms
...

• Restart the NDMP service (the service name may vary)

sudo systemctl restart ndmp.service

• Initiate a test backup (replace the command with the one applicable to the environment)

avtar -b -s  -t  -c

Verification:

1. Check the NDMP log after the test backup. Successful entries should show a connection established without "illegal operation" or "Fatal server connection problem" messages.

2. Confirm that the file /ddr_files.xml is present in the backup directory.

3. Run a regular backup schedule and ensure it completes without errors.