PowerFlex: How to add and approve MDM certificates on Gateway via RESTAPI

Below are the acronym used in the command provided below to add the MDM Certificate. Please replace them at per your environment.

<mdm-password> - MDM password to login to Scaleio
<Gateway-ip> - Gateway IP being used to connect via restapi
<token> - The token you received from step 1
<hostname> - replace with hostname (Ensure you have different hostname for each MDM)
<hostname_mentioned in /etc/hosts> - the MDM hostname mentioned in /etc/hosts

The following steps includes example for your reference where 192.168.2.126 is the gateway and Node1 is one of the MDM nodes for which the MDM certs needs to be added to the Gateway.
You could run this command from any host which has curl installed and has connectivity to the Gateway.
Rebooting the Gateway daemon is not required
You can check the available MDM certs in Gateway keystore via command provided in Step 5 below.

Step 1) Obtain the token

# curl -k --basic -u admin:<mdm-password> https://<Gateway-ip>/api/gatewayLogin 

eg:# curl -k --basic -u admin:hagsjfs https://xx.yy.uu.ii/api/gatewayLogin 
"YWRtaW46MTYxMjM4NTI5NDgyODo2YmRiOGFhNGQxNzk2NWY1OWJmZmE1NDU1MWU2MjlkMw"

Step 2) Ensure you add the MDM IP address and hostname in /etc/hosts on the Gateway(IM) server.

Step 3) From the above step the token you receive needs to be used. Get the certificates of the MDM's which needs to be added :

# curl -k -u foo:<token> https://<Gateway-ip>/api/getHostCertificate/Mdm?host=<hostname_mentioned_in_/etc/hosts> > /tmp/mdmcert_<hostname>

eg:curl -k -u foo:YWRtaW46MTYxMjM4NTI5NDgyODo2YmRiOGFhNGQxNzk2NWY1OWJmZmE1NDU1MWU2MjlkMw https://xx.yy.uu.ii/api/getHostCertificate/Mdm?host=Node1 > /tmp/Node1.crt

Step 4) Install the certificate into gateway keystore

# curl -k -u foo:<token> --form "file=@/tmp/mdmcert_<hostname>" https://<Gateway-ip>/api/trustHostCertificate/Mdm


eg:# curl -k -u foo:YWRtaW46MTYxMjM4NTI5NDgyODo2YmRiOGFhNGQxNzk2NWY1OWJmZmE1NDU1MWU2MjlkMw --form "file=@/tmp/Node1.crt" https://xx.yy.uu.ii/api/trustHostCertificate/Mdm

Step 5) Ensure the certs are seen. In the alias you see, you should refer to the CN name which should be unique for every node. The below command needs to be run from the Gateway.

# /usr/bin/keytool -list -keystore /opt/emc/scaleio/gateway/webapps/ROOT/WEB-INF/classes/certificates/truststore.jks  -storepass changeit| grep mdm -A1 

eg: 
# /usr/bin/keytool -list -keystore /opt/emc/scaleio/gateway/webapps/ROOT/WEB-INF/classes/certificates/truststore.jks  -storepass changeit| grep mdm -A1
ou=asd, o=emc, c=us, st=massachusetts, l=hopkinton, cn=node1, givenname=mdm, Feb 3, 2021, trustedCertEntry,
Certificate fingerprint (SHA1): C6:99:F2:8C:82:4A:25:44:36:AF:90:51:43:B9:27:98:7C:9D:F1:6C

Step 6)Repeat the same procedure to add the MDM certificate to Gateway Keystores in case missing.

Step 7)Restart the Gateway Daemon once MDM certs are added from all the MDM nodes

# service scaleio-gateway restart

We would see the following error on the Gateway WebBrowser when the certificate were changed or missing on gateway keystore -

"The primary MDM's certificate is unapproved. click here to see the certificate details"

Openstack command may also fails when the MDM certificates are not added to the Gateway keystore.

Gateway(via Web-Browser) will always connect with the Master MDM for authentication. If Gateway(web-browser) logs an Certificate error, this would mean that the certificate was changed on the Primary MDM and needs to be added to the gateway keystore.
Gateway will not check for Secondary MDM certificates unless we switch to MDM cluster to it.

Other method of adding the MDM certificates to Gateway Keystore -

1) Switch MDM cluster to the MDM node on which the certificate has been replaced or the certs is missing on Gateway keystore. 
Login to Gateway Web-browser, Click on Maintain Tab  and select "Retrieve System Topology". You would see the below screen where you could select "click here" and approve the MDM certificate after ensuring the CN name is different .
This procedure requires you to switch MDM cluster.

2) Transfer the MDM certificate to the Gateway node and then use the Keytool command to incorporate these certificates into the Gateway keystore. After adding the certificates from all MDM nodes, you need to restart the Gateway daemon.