PowerStore: Certificate Renewal Procedures

Background

Certificate expiration alerts appear when a certificate is due to expire soon, or when a certificate auto-renewal fails.

• For user-imported certificates, the user must upload a new certificate to replace the expiring certificate.
• For auto-renewed certificates, if there is failure, a recovery script must be used to manually renew the certificate.

PowerStore uses certificates for authenticating and validating TLS connections with external systems. Certificates are valid for specific periods, usually one year or longer. When a certificate expires the associated service may stop working, or run in a degraded mode.

To prevent service impact, certificate expiration must be monitored to give the user time to obtain and install a new certificate before the old certificate expires.

PowerStoreOS version 4.1.0.0 (or later) automatically monitors certificates for expiration, checking once every midnight (UTC). When a certificate is found to be expiring soon, PowerStore generates alerts depending on whether the certificate was generated and signed internally (by the PowerStore CA) or signed by a private CA and imported by the user.

Note: In general PowerStore does not support 3rd Party CAs signing the GUI certificate. The browser forum prohibits CAs from signing certs with restricted IPs in the certificate, since private IPs are included in the restricted list and also required in the certificate by PowerStore the CA will decline the request. The only possibility to get a certificate signed by a 3rd Party CA is to use for the management network public IPs provided by an ISP or IANA.

This article refers to a certificate signed by a private enterprise CA (for example, using OpenSSL, Microsoft CA, or something similar) 

Private Certificate Authority signed certificates

Alerts are generated when PowerStore determines a CA signed certificate expires soon.

The alerts generated:

• Minor alert when the certificate expires in 30 days and a new certificate has not been imported
• Major alert when the certificate expires in 15 days and a new certificate has not been imported
• Critical alert when the certificate expires in 7 days and a new certificate has not been imported
• Critical alert when a certificate has expired and a new certificate has not been imported

PowerStore CA-signed Certificates

PowerStore CA-signed certificates are automatically renewed when they are due to expire in 90 days. If the automatic renewal fails, PowerStore attempts to renew the certificate every twenty-four hours until the renewal is successful.

The alerts generated:

• Major alert when the automatic renewal fails 90 days before expiration
• Major alert when the automatic renewal fails at 60 days before expiration
• Critical alert when the automatic renewal fails at 30 days before expiration
• Critical alert when the certificate expires

Steps to resolve:

Private CA-signed certificates

The following describes the steps for each service. When a new certificate is successfully uploaded, one or more alerts is automatically acknowledged.

PowerStore Cluster and GUI Certificate

1. General Steps for replacing PowerStore cluster CA signed Server certificate:

1. To generate a CSR to import a private Certificate Authority (CA) signed server certificate:

   Steps

   1. Select Settings and under Security select Signed Cluster Certificate.
   2. Select Generate CSR.
      The Generate CSR slide out appears.
   3. Type in the information that is used to generate the CSR.
   4. Click Generate.
      The Generate CSR slide out changes to show the next steps that must be taken along with the certificate text.
   5. Click Copy to clipboard to copy the certificate text to your clipboard.
   6. Click Close.
   7. Have the certificate signed by the certificate authority (CA) so that it can be imported as a Mutual Authentication Certificate.

2. To import a signed cluster certificate using the PowerStore Manager:

   Steps

   1. Select Settings and under Security select Signed Cluster Certificate.
      The Signed Cluster Certificate page appears.
   2. Select Import.
   3. Do one of the following:
      • Select "Select Certificate File," then locate and select the file to import.
      • Select "Paste Certificate Text," then copy and paste the certificate text in the text box.
   4. Select Import.

2. Management HTTP service is expiring:

1. Follow GeneralSteps_for_Replacing_PowerStore_cluster_Third_Party_Server_Certificate to replace the PowerStore cluster Third-Party Server Certificate.
2. If the local cluster is participating in a replication relationship, ensure that the replication connection is still healthy.

3. Management HTTP service has expired:

1. If the local cluster is not in a replication relationship, follow GeneralSteps_for_Replacing_PowerStore_cluster_Third_Party_Server_Certificate to manually renew the certificate.

2. If the local cluster is in a replication relationship, at this point, most likely you see that the management connection is broken between the two clusters. (Even it does not show as broken, once new TLS connection is established, the connection status is updated as broken.) Run the following three steps:

   1. Run the pstcli reset the local cluster's server certificate to the default PowerStore CA signed.

      pstcli -user <<username>>  -password <<Password>> -destination localClusterIP /x509_certificate reset_certificates -service Management_HTTP -scope External

   2. Run the pstcli exchange to reestablish the replication management connection. (Do ensure that replication connection is recovered)

      pstcli -user admin -password <<Password>> -destination localClusterIP /x509_certificate exchange -address remoteClusterIP -port 443 -service Replication_HTTP -username <<username>> -password <<Password>>

   3. Follow GeneralSteps_for_Replacing_PowerStore_cluster_Third_Party_Server_Certificate to refresh the certificates.

Replication CA Server

1. In case the CA is expiring on the remote system, follow Management_HTTP_service_is_expiring to refresh the server certificate on the remote system, it automatically renews the Replication CA Server.
2. In case the CA has expired, on the remote system, follow Management_HTTP_service_has_expired to refresh the server certificate on the remote system, it automatically renews the Replication CA Server.

SecurID

1. Select Settings and under Security, select Authentication, then select the RSA SecurID tab.
2. To import a new CA certificate chain to replace the existing CA certificate that is being used by PowerStore, click Import on the left side of the screen above the Certificates table:
3. Select and either import the certificate file or paste the certificate text (in PEM format) into the Import RSA SecurID Certificate slide out panel.
   Note: If there are multiple CA certificates in the chain, they should all be imported in a single certificate file or text block, in ascending order; for example, the root CA certificate should be last. If imported as a text block, add a carriage return between the end of one certificate and the start of the next.

4. Multiple certificates are supported, so the existing CA certificate remain until it expires. When the existing CA certificate expires and a replacement has been imported, it is automatically deleted.

   To support smooth CA certificate rollover, we support co-existing of multiple CA certificates. In order to ensure the process to go through smoothly, before the CA certificate expires:

   1. Upload the new CA certificate on PowerStore, before switching to new the new CA certificate signed server certificate on the SecurID Server.
   2. On the SecurID server, switch to use the new CA signed server cert.

LDAPS
When LDAPS is configured, PowerStore connects to the LDAP server using TLS. PowerStore requires the CA certificate chain file to be uploaded, to properly verify the server certificate received from the LDAP server when the TLS session is established.

1. Steps:

   Note: If there are multiple CA certificates in the certificate chain, they should all be imported in a single certificate file, in ascending order; for example, the root CA certificate should be last.
   1. Select Settings and under Security, select Authentication. Then select the LDAP tab.
   2. Click Import on the left side of the screen above the Certificates table.

      The Upload File dialog box appears.

   3. Click Choose File.
   4. Browse to the wanted certificate file, then select the file and click Open.
   5. After the file upload completes, click Apply to save the configuration changes.

2. Multiple certificates are supported, so the existing CA certificate remain until it expires. When the existing CA certificate expires and a replacement has been imported, it is automatically deleted.

   To support smooth CA certificate rollover, we support co-existing of multiple CA certificates. In order to ensure the process to go through smoothly, before the CA certificate expires:

   1. Upload the new CA certificate on PowerStore, before switching to new the new CA certificate signed server certificate on the LDAP Server.
   2. On the LDAP server, switch to use the new CA signed server cert.

Remote Syslog

1. Customer can import a Server CA Certificate for one-way authentication and/or Mutual Authentication Certificate for two-way authentication. The Server CA Certificate is required for TLS encryption.

2. To import a certificate using the PowerStore Manager, do the following:

   Steps:

   1. Click Settings and under Security select Remote Logging.
      The Remote logging page appears.
   2. Under Certificates, select Import and select the type of certificate to import.
      The slide out for the respective certificate appears.
   3. Do one of the following:
      • Select "Select Certificate File," then locate and select the file to import.
      • Select "Paste Certificate File," then copy and paste the certificate text in the text box.
   4. Select Import.
      The certificate should appear in the list under Certificates.

3. While the mutual cert is expiring, to ensure the no disruptive renewal for the mutual certification, if it is allowed on the remote system before starting the process of replacing the mutual auth certificate, ensure the CA certificate that you are about to use to sign the new mutual certificate is in your Remote sys log's trust store.

4. Generating a CSR is applicable to a Mutual Authentication Certificate, which is used in two-way authentication between PowerStore and the remote syslog server. To generate a CSR using the PowerStore Manager, do the following:

   Steps:

   1. Select Settings and under Security select Remote Logging.
      The Remote logging page appears.
   2. Select Generate CSR.
      The Generate CSR slide out appears.
   3. Type in the information that is used to generate the CSR.
   4. Click Generate.
      The Generate CSR slide out changes to show the next steps that must be taken along with the certificate text.
   5. Click Copy to clipboard to copy the certificate text to your clipboard.
   6. Click Close.
   7. Have the certificate signed by the certificate authority (CA) so that it can be imported as a Mutual Authentication Certificate.
   8. Import the signed Mutual Authentication Certificate (Client CA Certificate) to PowerStore.

5. Multiple certificates chains are supported for CA server cert, but only the latest is used. An existing CA certificate chain that is superseded by a later CA certificate chain remain until it expires. If the existing CA certificate chain is expired and a replacement certificate chain is been imported, the expired certificate chain is automatically deleted.

KMIP
While the mutual cert is expiring, to ensure the no disruptive renewal for the mutual certification, if it is allowed on the remote system before starting the process of replacing the mutual auth certificate, ensure the CA certificate that you are about to use to sign the new mutual certificate is in your KMIP server's trust store.

To renew the Client key and certificate, a Certificate Signing Request (CSR) must be generated on the PowerStore system, sent to the applicable Certificate Authority, then the resulting (signed) certificate imported into the PowerStore.

To generate a CSR using the PowerStore Manager, do the following:

1. Select Settings and under Security select KMIP.
   The KMIP configuration page appears.
2. Select Generate CSR.
   The Generate CSR slide out appears.
3. Type in the information that is used to generate the CSR.
4. Click Generate.
   The Generate CSR slide out changes to show the next steps that must be taken along with the certificate text.
5. Click Copy to clipboard to copy the certificate text to your clipboard.
6. Click Close.
7. Note that a Mutual Authentication Certificate entry has been created. This is used later, when importing the signed certificate.
8. Have the certificate signed by the certificate authority (CA) so that it can be imported as a Mutual Authentication Certificate.
9. To import the signed Mutual Authentication Certificate to PowerStore:
   1. Select the Mutual Authentication Certificate entry created when the CSR was generated.
   2. Click the Import button and select Import Client Certificate
      The Import Client Certificate slide out appears.
   3. Select Certificate from the Import Options, and select the wanted Certificate option - File or Text - to import the certificate.
   4. Specify the filename or paste the certificate into the textbox, then click Import.

To renew the Server CA certificate:

1. Select Settings and under Security select KMIP.
   The KMIP configuration page appears.
2. Click the Import button and select Import Server CA certificate.
   The Import Client Certificate slide out appears.
3. Select the wanted Import option - File or Text - to import the certificate.
4. Specify the filename or paste the certificate into the textbox, then click Import.

Block & File Import from External Storage & Remote Backup to Data Domain

1. Renew the expired certificate on the remote system.
2. For remote backup type remote systems, "Update Certificate" action is available in (Protection → Remote Systems). (Migration → Import External Storage) is applicable to block & file import type remote systems. These actions fetch the renewed certificate from a remote system, display it to the user for confirmation/acceptance and then update it in PowerStore.
3. The expiring or expired certificate from PowerStore is automatically deleted from PowerStore while updating the renewed certificate.

VASA CA signed Server Certificate
In this case, generating CSR is used for generating the VASA provider's server certificate. To generate a CSR using the PowerStore Manager, do the following:

1. Select Settings and under Security select VASA Certificate.
   The VASA Certificate page appears.
2. Select Generate CSR.
   The Generate CSR slide out appears.
3. Type in the information that is used to generate the CSR.
4. Click Generate.
   The Generate CSR slide out changes to show the next steps that must be taken along with the certificate text.
5. Click Copy to clipboard to copy the certificate text to your clipboard.
6. Click Close.
7. The certificate authority (CA) must sign the certificate so that it can be imported.

The vCenter must trust the CA of the certificate being imported, otherwise VASA functionality becomes unavailable. To accomplish this, upload to the VMware vCenter trust store the signing CA chain that was used in step 7 of this sub-section.

To import a certificate using the PowerStore Manager, do the following:

1. Click Settings and, under Security, select VASA Certificate.
   The VASA Certificate page appears.
2. To prevent the VASA server certificate from being overwritten by the vCenter, ensure the Enable/Disable toggle is set to Enabled.
3. Select Import.
   The Import Server Certificate slide out appears.
4. Do one of the following:
   1. Select "Select Certificate File," then locate and select the file to import.
   2. Select "Paste Certificate Text," then copy and paste the certificate text in the text box.
5. Select Import.

The certificate detail information should appear in the VASA Certificate page. Also, the VASA entry appearing on the Certificate page (Settings > Security > Certificate) should be identified by the Service as VASA_HTTP and Scope as vasa.

PowerStore CA-signed Certificates

There are two serviceability scripts for Certificate Renewal: svc_renew_vasa_self_signed_certificate and svc_renew_certificates.

svc_renew_vasa_self_signed_certificate renew the VASA Server PowerStore CA signed certificate.

svc_renew_certificates renew the PowerStore CA signed certificates such as PowerStore Cluster Server, PowerStore Cluster Client, and Replication Mutual Authentication.

Usage:

[SVC:service@835ZX23-B user]$ svc_renew_vasa_self_signed_certificate -h
usage: svc_renew_vasa_self_signed_certificate [-h] [--force] [--verbose] [--debug]
                                  [--quiet]

    This tool is used to renew the` VASA Server PowerStore CA signed certificate.


optional arguments:
  -h, --help  show this help message and exit
  --force
  --verbose
  --debug
  --quiet

--------------------------------------------------------------------------------------

[SVC:service@835ZX23-B user]$ svc_renew_certificates -h

usage: svc_renew_certificates [-h] [-d] [-v] {list,run} ...

  This tool is used to renew the PowerStore CA signed certificates such as;
  PowerStore Cluster Server, PowerStore Cluster Client, and Replication Mutual Authentication Certificate

  To list the the types of certificates available:
     svc_renew_certificates list

  To renew the specific certificate, or all:
     Renew all certificates:
        svc_renew_certificates run all

     Renew the Replication Mutual Authentication Certificate:
        svc_renew_certificates run replication_mutual_authentication

     Renew the PowerStore Cluster Server Internal certificate:
        svc_renew_certificates run powerstore_cluster_server_internal

     Renew the PowerStore Cluster Server External certificate:
            svc_renew_certificates run powerstore_cluster_server_external

     Renew the PowerStore Cluster Client certificate:
            svc_renew_certificates run powerstore_cluster_server_client

optional arguments:
  -h, --help     show this help message and exit
  -d, --debug    Increase logging level to debug and print logs to console
  -v, --verbose

action:
  {list,run}
    list         List the available certificates for renewal
    run          Renews the certificate type specified for renewal, or all