VM backups failing with vCenter login errors
Summary: Backups are failing when using rules or tags.
Symptoms
VM backups that are using rules or tags are failing.
Regular backups complete - selected using policy
Logs show the following error:
Action backup vmware-vproxy 'backup' has initialized as 'vproxysave job' with job id 224276 libCURL SSL support is Enabled. Inventory session status returned with a failure. Error: 'Failed to load inventory from 'VCenter_server.FQN.com': ServerFaultCode: Cannot complete login due to an incorrect user name or password.'.
And:
NWSERVERNAME nsrdisp_nwbg NSR critical Inventory session status returned with a failure. Error: 'Failed to load inventory from 'VCENTER_NAME': ServerFaultCode: Cannot complete login due to an incorrect user name or password.'. lNWSERVERNAME nsrdisp_nwbg NSR notice Reading the XML from RAP without running nsrvim again
Cause
The user, or password, or both were changed for VM Backup.
The nsrvisd is using the password.
The change was never reflected in the nsrvisd inventory process.
Review the logs in:/nsr/vproxy/logs/nsrvisd
Or/opt/nsr/vproxy/logs/nsrvisd
The log reflects the old user - password should be encrypted.
Resource Definition: {"Config":{"SessionId":"92-23993-9966cdbbd1","LogTag":"@(#) Build number: 41","LogLevel":"INFO","IdleTimeout":0,"AcceptedLanguage":"","VimServer":{"HostName":"VCENTER_SERVER","TcpPort":443,"UserName":"administrator@vsphere.local","UserPassword":"********","ThumbPrint":"","HypervisorMode":"vSphere"},"RefreshInventory":10},"Actions":{"ChangeSession":"/api/v1/InventorySessions/92fba-7a3a-48eb-84-e706cdbbd1","DeleteSession":"/api/v1/InventorySessions/92fbeef3a-48eb-84a6-70736cdbbd1","CancelActivity":"/api/v1/InventorySessions/92fbeefa-7a3a-48eb-84a6-e70736cdbbd1/activity","GetSession":"/api/v1/InventorySessions/92fbe
Resolution
This is an issue, Dell engineering investigation is under progress.
Workaround is clear the state file - InventorySessions.gob
The state file can be located under./nsr/vproxy/logs/nsrvisd/InventorySessions.gob
OR/opt/nsr/vproxy/logs/nsrvisd/InventorySessions.gob
Rename InventorySessions.gob.mv InventorySessions.gob InventorySessions.gob_old
Stop nsrvisd - using kill.
ps -eaf | grep nsrvisd kill <pid>
NSRVISD is restarted.
Verify it did restart:
ps -eaf | grep nsrvisd