RecoverPoint for Virtual Machines: Custom Multichain Certificate Causes Updates to Truststore to Fail
Summary: Using a custom (not default or a self-signed) certificate, a change on one RecoverPoint Appliance (RPA) of MTU value is NOT propagated to other appliances.
Symptoms
Look for the following errors around "addCertificateAuthority" in installationLogs/server.log:
at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:131) ~[jaxws-rt-2.3.3.jar:2.3.3] at com.sun.proxy.$Proxy560.addCertificateAuthority(Unknown Source) ~[?:?] at com.kashya.installation.server.commands.global.AddCertificateAuthorityWorker.callWebServiceInternal(AddCertificateAuthorityWorker.java:20) ~[classes/:?] at com.kashya.installation.server.infocollect.ClientWorker.execute(ClientWorker.java:220) ~[classes/:?]
Cause
Due to MTU mismatches, there are communication errors.
Resolution
Workaround:
There are three possible workarounds:
Workaround #1:
Use a self-signed certificate, instead of a custom certificate. Details can be found in the RecoverPoint for Virtual Machines Security Configuration Guide.
- Add vCenter certificate to RecoverPoint side:
- Manually add the new VC cert in base64 format (CA cert only) under Trusted Store of RPA.
- Log in into the admin menu.
- Go to Options.
[2] Setup > [8] Advanced options > [2] Security options > [2] Certificates management > [2] Truststore management > [2] Add trusted certificate
- Add the CA certificate to the trusted store of RPA.
You must open the certificate in a readable view and copy the whole certificate including:-----BEGIN CERTIFICATE REQUEST-----To-----END CERTIFICATE REQUEST-----
Paste it in PuTTY then add a # in a new line and press Enter. - Run the command from the RPA system CLI.
update_vcenter_server_registration -f
Workaround #2:
Temporarily use the default certificate or a self-signed certificate using the below procedure, to update MTU sizes. Then after confirming that the MTU change propagated to all RPAs, reinstall the custom certificate. Below are the steps to update web certificate and MTU configuration:
A. Update certificate
- Create a customer certificate:
Openssl req -newkey rsa:2048 -x509 -sha256 -days 365 -nodes -out certificate.pem -keyout privatekey.pem
- Replace the RPAs Web certificate with a newly created one.
Use an SSH client to log in to the vRPA Boxmgmt CLI as user = admin. From the Main menu, select:
Setup > Advanced Options > Security Options > Certificates Management > Keystore Management > Change Web Server Certificate.
- Add Web certificate to truststore.
Use an SSH client to log in to the vRPA Boxmgmt CLI as user = admin. From the Main menu, select:
Setup > Advanced Options > Security Options > Certificates Management > Truststore Management > Add trusted certificate
- Restart tomcat service on
RPA - systemctl restart tomcat9
B. Update MTU values
- Access with admin the SC RPA and change the MTU value to something different than the current MTU:
[2] Setup --> [1] Modify Settings --> [3] MTU configuration --> [2] configure MTU values
- Open a new CLI session to the second RPA and confirm that the MTU is modified (if an existing session to the second RPA is already opened this does not reflect the MTU change, a new session is required)
Workaround #3:
Keep the custom certificate and manually update the MTU at each RPA.
The detailed procedure for workaround #3 is as follows.
ssh admin@<RPA IP>
Main Menu **
[1] Installation
[2] Setup
[3] Diagnostics
[4] Cluster operations
[5] Shutdown/Reboot operations
[6] System management CLI
[Q] Quit
Change the MTU configuration:
[2] Setup --> [1] Modify Settings --> [3] MTU configuration --> [2] Configure MTU values
The user can set the MTU separately for each link in their network configuration.
A typical configuration would have a separate link for each of LAN, WAN, Data - but each configuration may have more or less links.
For each link, individually modify the MTU setting to the wanted value (generally 1500 or 9000).
After setting one value, all values will be printed (LAN, WAN, Data), then the user is asked to confirm applying the change.
When all the values are set correctly, enter 'Q' to exit this menu.
No reboot is required for the new settings to take effect.
Repeat for each RPA in the system. Ensure that the same values are used on all RPAs.
Resolution:
Dell Technologies engineering is investigating this issue. A permanent fix is still in progress. Contact the Dell Technologies Customer Support Center or your service representative for assistance and reference this solution ID.