NVP-vProxy: View update fails, error "Unable to fetch data from vCenter: EOF was observed that violates the protocol."
Summary: When refreshing the VMware view, or when running a backup, you might get an error message indicating that NetWorker is unable to fetch data from vCenter.
Symptoms
The NetWorker VMware Protection integration is configured with the vProxy Appliance.
The VMware View update or VMware backups fail with the following error:
Unable to fetch data from vCenter: EOF was observed that violates the protocol. The client probably provided invalid authentication information.
Cause
The NetWorker server is not able to establish an SSL connection to vCenter Server over port 443.
Resolution
In this scenario, OpenSSL connections fail due to a firewall/router blocking the connection from NetWorker server to vCenter server.
Additional Information
OpenSSL can be used to simulate the SSL connection failure to the vCenter server. The OpenSSL command-line utility is available on Linux servers, but a third party OpenSSL must be used on Windows Servers. https://wiki.openssl.org/index.php/Binaries
Example of failed connection:
nsr:~ # openssl s_client -connect xxx.xxx.xxx.xxx:443 socket: Connection timed out connect:errno=110
Example of successful connection:
nsr:~ # openssl s_client -connect xxx.xxx.xxx.xxx:443
CONNECTED(00000003)
depth=0 CN = myvcenter.mydomain.com, C = US
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = myvcenter.mydomain.com, C = US
verify error:num=27:certificate not trusted
verify return:1
depth=0 CN = myvcenter.mydomain.com, C = US
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
0 s:/CN=myvcenter.mydomain.com/C=US
i:/CN=CA/DC=vsphere/DC=local/C=US/ST=California/O=myvcenter.mydomain.com/OU=VMware
---
Server certificate
-----BEGIN CERTIFICATE-----
MIID0zCCArugAwIBAgIJAOZ1ABoYvxvSMA0GCSqGSIb3DQEBCwUAMIGjMQswCQYD
VQQDDAJDQTEXMBUGCgmSJomT8ixkARkWB3ZzcGhlcmUxFTATBgoJkiaJk/IsZAEZ
FgVsb2NhbDELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExJTAjBgNV
BAoMHGFscGluZS5wbHN1cHBvcnQubGFiLmVtYy5jb20xGzAZBgNVBAsMElZNd2Fy
ZSBFbmdpbmVlcmluZzAeFw0xODA1MDQyMTAzMjBaFw0yMDA1MDQwOTAzMjBaMDQx
JTAjBgNVBAMMHGFscGluZS5wbHN1cHBvcnQubGFiLmVtYy5jb20xCzAJBgNVBAYT
AlVTMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAy5a94in23wV11aik
RcgqzjmUZkUA/rJBt+aisK8TwsSQCQSQ0WaPtEJhqrJmiituS0fKshig40skGSqR
rK6eKbc8zGJ0ZSwhevzef6XdceaC2Tzxm7qPRsFi9mRdK+V7nDs0V8tAA2ugRd9e
xy7t8H/xDU5JMQVkl1Ma5g/6pvdU9H86KGHuAQDNmF2VVwdet6wwUgJHjnPFVmuc
RLlj0CovvWF/AAmWRu30gJD7ADDU8Q9S3YWp3RLeUuoe9oDlT8mVnAFAHl2CPMJm
9618UoCobd6la1jw02g76VTX1aSDmBfhSjBrzZkjVpOAZ/6YwLayVpv10OYTKR0x
UU5CewIDAQABo3gwdjALBgNVHQ8EBAMCA6gwJwYDVR0RBCAwHoIcYWxwaW5lLnBs
c3VwcG9ydC5sYWIuZW1jLmNvbTAdBgNVHQ4EFgQUEplX0+MbJDaHPRXPXRawmc5M
bkUwHwYDVR0jBBgwFoAUftmJodEnkPcySgo/SVi2YndesRcwDQYJKoZIhvcNAQEL
BQADggEBAFP7F4ubvlw2LTpgLWp9self2S4Gb22GsZ8WieSYF03GnDcGOxWNZL8m
ATW6PwhmW0p4PkdMdyNLM5ieDmOZH/axSF0wkPXbzXSpzcXtmangkwO64KWFKn6I
qzvaLn0IE88oIw8B0HDS9D2mmwYBEoCLhRfPXHhd5hQ10LyHQbWJoVZTFj3pSu7V
X3TFf84ptAWudECsWpR0cLVlLQ6M/dvd8U7Sagy+eggZNFen+E4OutADlXBAZgM8
k7Q4RA+gnfVe52j/jxzZyH6RYkbTABHM3+5jNnEwQEod/HDjfVasI9AP8RZy9+5H
sAxtGBwVATuOapNLAWUO8/XFWx1ti4Y=
-----END CERTIFICATE-----
subject=/CN=myvcenter.mydomain.com/C=US
issuer=/CN=CA/DC=vsphere/DC=local/C=US/ST=California/O=myvcenter.mydomain.com/OU=VMware
---
No client certificate CA names sent
---
SSL handshake has read 1463 bytes and written 427 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID:
Session-ID-ctx:
Master-Key: E819FEB55B419CAD67F08493CF0730835907289777707C8D40D798258B70E29BA7C214B24B4D4F452C4F830D8348787B
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1561409686
Timeout : 300 (sec)
Verify return code: 21 (unable to verify the first certificate)
---
NVP vProxy: Troubleshooting Network Connectivity For Backup and Restore Operations