NetWorker VMware Protection-vProxy: File-level Restore is not completing with message "Failed to log in to Data Domain service Client. Time out exceeded while awaiting headers"

Summary: The NetWorker VMware Protection (NVP) integration is configured with the vProxy appliance. File-level Restore (FLR) operations fail to mount the designated saveset. The error in vProxy vflrd log files located on the vProxy at /opt/emc/vproxy/runtime/logs/vflrd indicates a connectivity issue between the vProxy and Data Domain. ...

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

VMware VMs are protected with the NetWorker VMware Protection (NVP) vProxy appliance.

VM backups are successful.

VM Image restores are performed successfully.

VM Instant Recover and File Level restore fail with the following error logged in the recover logs.  
159373:nsrvproxy_flr: vProxy Log: YYYY-MM-DDTHH:MM:SSZ ERROR: [@(#) Build number: 54] Failed to login to DataDomain service at 'https://mydatadomain.mydomain.com:3009/rest/v1.0/auth': Post "https://mydatadomain.mydomain.com:3009/rest/v1.0/auth": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
159373:nsrvproxy_flr: vProxy Log: YYYY-MM-DDTHH:MM:SSZ INFO: [@(#) Build number: 54] Unmounting after mount failure: Unable to create NFS export at 'mydatadomain.mydomain.com:/data/col1/mynetworkerserver/myvproxy.mydomain.com-d771b76b-9e3f-42ac-aa20-c85823aeb6ag': Unable to login while creating NFS export at 'mydatadomain.mydomain.com:/data/col1/mynetworkerserver/myvproxy.mydomain.com-d771b76b-9e3f-42ac-aa20-b85823aeb6ag': Failed to login to DataDomain service at 'https://mydatadomain.mydomain.com:3009/rest/v1.0/auth': Post "https://mydatadomain.mydomain.com:3009/rest/v1.0/auth": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Saved recover session logs are found on the NetWorker server under:
  • Linux: /nsr/logs/recover
  • Windows: C:\Program Files\EMC NetWorker\nsr\logs\recover
vProxy recover session logs are found under:
  • /opt/emc/vproxy/runtime/logs/vflrd
  • /opt/emc/vproxy/runtime/logs/recycle/vflrd

Cause

Port 3009 is blocked between the vProxy appliance and Data Domain. Port 3009 is required for vProxy File Level Recovery and Instant Recover processes. 

Resolution

Correct the port 3009 communication between the vProxy and Data Domain. A full list of required ports is provided in the Port Requirements section of NetWorker VMware Integration Guide available on Dell Support Product Page under documentation. 

Port requirements and troubleshooting are also documented in knowledge base article 203350, NVP vProxy: Troubleshooting Network Connectivity For Backup and Restore Operations.  

Port 3009 should respond on the Data Domain from the vProxy appliance using the curl command:

curl -v DD_address:3009


Example of curl Command Response
Figure 1: Example of curl Command Response

 

Affected Products

NetWorker
Article Properties
Article Number: 000212329
Article Type: Solution
Last Modified: 21 Apr 2023
Version:  2
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.