Cloudlink: KMIP CA Certificate is expired

In the Cloudlink webUI in SYSTEM > Monitoring the following alarm was triggered:

KMIP CA Certificate is expired

This alarm was introduced in Cloudlink 8.1 so it is common to see this alarm after an upgrade.

The KMIP CA certificate is the issuer certificate used to sign the KMS Server certificates and the KMIP Client certificate. Here are a few ways to check if the KMIP CA certificate is expired:

1) Check if the customer has the kmipclient_key_cert.zip which would have been downloaded when the KMIP Client was first created. Extract this .zip file and the ca.pem is the KMIP CA certificate and the cert.pem is the KMIP Client certificate. If you rename ca.pem to ca.cer, you can open the file on a Windows server.

2) Navigate to the https://cloudlink.ip.address:5696 in a browser and select the site information next to the URL it may say Not Secure. Then select Certificate Details and select the Details tab. In the Certificate Hierarchy highlight KMIP CA and you can find details about this certificate. These instructions are for Chrome and may be different in other browsers.

3) In the Cloudlink webUI go to KMIP SERVER > Information > Actions > Download Server Certificate. This downloads a server_certs.zip file which includes the ca.pem which is the KMIP CA certificate and the cert.pem is the KMS Server certificate. If you rename ca.pem to ca.cer, you can open the file on a Windows server.

There does not appear to be any impact from an expired KMIP CA certificate. However it is impactful when the KMIP Client or KMS Server certificates are expired. Typically the KMIP CA, KMIP Client, and KMS Server certificates expire simultaneously. When you renew the KMIP CA certificate, this will also automatically renew the KMS Server certificates and force you to renew the KMIP Client certificate.
 

Confirm if the customer is using KMIP in Cloudlink. If the customer is not using KMIP in Cloudlink, then you can silence the alarm by following the instructions at the bottom of this KB. Typically VxRail environments use KMIP, and PowerFlex environments do not use KMIP but that is not always the case. It would be possible for PowerFlex customers to use KMIP if they have configured a Key Provider in vCenter. In the Cloudlink webUI in SYSTEM > License check what type of licenses have been applied. Typically PowerFlex customer will only have a Capacity license and will not have a KMIP license. You will also notice that the KMIP SERVER tab will only be visible if a KMIP license has been applied.
 

How to renew the KMIP CA certificate:

Take snapshots of all Cloudlink VMs before changing any certificates (do not need to include memory). Also take backups from the Cloudlink webUI in SYSTEM > Backup > Generate New Backup and then Actions > Download Backup. Confirm that customer can locate the Cloudlink backup key (cckey.pem) which should have been saved during deployment of Cloudlink.

Cloudlink 8.1 and above:

Starting in Cloudlink 8.1 you can generate a new self-signed KMIP CA certificate by clicking KMIP SERVER > Information > Actions > Change CA Certificate. Changing the KMIP CA certificate causes the current KMIP Client certificate to become invalid. You must generate a new KMIP Client certificate in KMIP SERVER > Clients > Actions > Generate New Certificate. You have to reestablish trust in vCenter > Configure > Key Providers > Establish Trust > Make KMS Trust vCenter > KMS certificate and private key > and you will upload the cert.pem and key.pem.
Changing the KMIP CA certificate automatically renews the KMS Server certificates.
You have to reestablish trust in vCenter > Configure > Key Providers > Establish Trust > Make vCenter Trust KMS.

Cloudlink 7.x:

Older versions of Cloudlink do not have the option to renew the KMIP CA certificate. I would recommend upgrading to Cloudlink 8.1 or later (if possible) in order to renew this certificate. If upgrading is not an option you can apply your own custom KMIP CA certificate by going to KMIP SERVER > Information > Actions > Upload CA Signed PEM. This certificate can be created using openssl or if the organization has their own Certificate Authority.

Here are instructions for using OpenSSL to generate a self-signed certificate intended to replace the CloudLink 7.x KMIP CA certificate:

1. Use any Linux server (not CloudLink) and confirm OpenSSL is installed by running the command:

   openssl version

2. Create a file called template.cfg by running the command:

   vi template.cfg

3. Paste the information within the box below. For the blue entries, modify and replace with the relevant information.

[req]
default_bits           = 2048
distinguished_name     = req_distinguished_name
req_extensions         = v3_req

[req_distinguished_name]
C =Country(2 letter code)
ST =State
L =Locality(city)
O =Organization
OU =OrgUnit
CN =CommonName

C_default =US
ST_default =utah
L_default =salt lake city
O_default =dell
OU_default =dell
CN_default =KMIP CA

[ v3_req ]
subjectAltName   = @alt_names
keyUsage         = critical, digitalSignature, keyCertSign, cRLSign
extendedKeyUsage = serverAuth, clientAuth
basicConstraints = critical, CA:true, pathlen:1
subjectKeyIdentifier=hash

[alt_names]
DNS.1   = KMIP CA

4. Run the command:

openssl req -newkey 2048 -keyout kmipca.key -config template.cfg -x509 -days 730 -out kmipca.crt -extensions v3_req -nodes

This outputs two files called kmipca.crt and kmipca.key. You will upload these files in Cloudlink in System > Cluster > Actions > Upload CA Signed PEM > Third Party PEM. Save these files and keep them somewhere safe. This command sets -days to 730 which is 2 years but you can adjust as needed.

How to clear the 'KMIP CA Certificate is expired' alarm in an environment where KMIP is not in use:
 

Option 1) The easiest solution would be to silence the alarm in SYSTEM > Alarms Configuration > select KMIP CA certificate is expired alarm > Actions > Change State to Ignored

Option 2) Some organizations scan their environment and do not allow expired certificates. Even though the KMIP CA certificate is not being used, some customers might prefer to renew the certificate rather than silence the alarm. Since the customer does not have a KMIP license we cannot renew the certificate from the webUI because the KMIP SERVER tab is missing. We could use the restapi to renew or apply a new KMIP CA certificate. Take snapshots of all Cloudlink VMs before changing certificates. Also take backups from the Cloudlink webUI in SYSTEM > Backup > Generate New Backup and then Actions > Download Backup. Confirm that you can locate the Cloudlink backup key (cckey.pem) which should have been saved during deployment of Cloudlink.
   
   a) Use the restapi to renew the self-signed KMIP CA certificate (only possible in Cloudlink 8.1 or later):
      In the Cloudlink webUI, go to ABOUT > Rest Documentation > Open Documentation
      Find the button that says POST /kmip/cacertificate  change KMIP CA certificate
      This generates a new KMIP CA certificate that expires in a year.
      Before renewing the cert, we could also increase the certificate lifetime using /cloudlink/rest/kmip/certificate/lifetime.
      We would not want to set the maximum lifetime of 25 years since there is a known issue of openssl certificates that expire in 2050 or later. We would probably want to set this to 15 years or less.

   b) Use the restapi to upload your own custom KMIP CA certificate:
       In the Cloudlink webUI, go to ABOUT > Rest Documentation > Open Documentation
       Find the button that says POST /cloudlink/rest/kmip/certificate/upload
 

Related articles:

• CloudLink: Cluster Node Certificate is Expired
• Dell CloudLink: The Web TLS Certificate is Expired