Data Domain: SSL Certificate Import Fails with "Invalid x509 v3 Extension" for Cloud Tier or UI

During SSL configuration for:

• Cloud Tier integration (such as, with ECS using HTTPS), or
• DD UI access using an externally signed certificate,

The following error may be encountered:

Invalid CA certificate x509 v3 extension

Additional log entries may appear in /ddr/var/log/messages.engineering

• sms: NOTICE: smu_sec_extract_certs_from_PKCS12: **** Invalid host certificate x509 v3 extension. https needs either server or no extension
  sms: NOTICE: sms_trust_add_app_certs_job: **** Invalid host certificate x509 v3 extension.

This error can occur due to one or more of the following reasons:

1. Incorrect Certificate Type

   • For Cloud Tier: The certificate must be a CA certificate, not an endpoint certificate.
   • For DD UI: The certificate must be a host/server certificate without inappropriate extensions.

2. Improper Key Generation

   • Sometimes, the private key used to generate the certificate (such as, on ECS for an F5 Load Balancer) was incorrectly created.

3. CSR Mismatch

   • Some Certificate Authorities (CAs) may ignore the requested extensions in the CSR and return a certificate with incompatible x509 v3 extensions.

For Cloud Tier (CT) Integration with ECS using F5 Load Balancer:

Reference: ECS administration guide

1. Generate Private Key on ECS

   • Log in to an ECS node or a connected system.
   • Run:
     • openssl genrsa -des3 -out server.key 2048
   • Enter and confirm a passphrase.
   • Remove the passphrase before uploading the key to ECS.
   • Set permissions:
     • chmod 0400 server.key
2. Generate Certificate on F5 Using ECS Key
   • Follow the steps in the relevant Dell EMC ECS with F5 integration guide.

3. Import Certificate into Data Domain

   • Refer to the KB: How to add certificates to a Data Domain for use with Long Term Reterntion (LTR) to the cloud.

Note: Ensure the certificate being imported is the CA certificate that signed the endpoint certificate, not the endpoint certificate itself.

For DD UI (HTTPS Access):

1. Generate CSR from Data Domain

   • Use DD's built-in tools to generate the CSR.
   • Submit the CSR to your CA for signing.

2. Import the Signed Certificate into DD

   • Ensure that the returned certificate has appropriate extensions (that is, server or none).

3. Troubleshooting

   • If import fails, inspect the certificate using:
     • openssl x509 -in /path/to/certificate.pem -text -noout
   • Review the x509 v3 extensions for compatibility.

Tip: The private key never leaves the DD when using the CSR method, ensuring secure handling. 

Certificate Validation Examples

A common reason for the error "Invalid CA certificate x509 v3 extension" is importing a certificate that is not a root CA or lacks the correct x509 extensions.

Example: Incorrect Endpoint Certificate:

• • Subject: CN=objects.ilandcloud.com
    X509v3 Basic Constraints: critical
        CA:FALSE

  • This certificate is for a web server, not a CA. It cannot be used as a trusted certificate in DD for Cloud Tier.

Correct Intermediate CA Certificate:

• • Subject: CN=Let's Encrypt Authority X3
    X509v3 Basic Constraints: critical
        CA:TRUE, pathlen:0

  • This is a CA certificate, but it is not self-signed. It is signed by the root CA.

Correct Root CA Certificate:

• • Subject: CN=ISRG Root X1
    Issuer: CN=ISRG Root X1
    X509v3 Basic Constraints: critical
        CA:TRU

  • This self-signed root CA certificate is the correct one to import into DD.
    • You may also import intermediate CA certificates if needed, but the root CA is typically required for trust validation.

Example: Incorrect UI Certificate Extensions:

• X509v3 Extended Key Usage:
      TLS Web Client Authentication, E-mail Protection

• This certificate is marked for client authentication and email protection - not suitable for DD UI HTTPS access.

  Recommended CSR Generation for DD UI:

• adminaccess certificate cert-signing-request generate key-strength 4096bit \
    country ES state Madrid city SomeCity org-name DELL org-unit DPS \
    common-name www.example.com basic-constraint CA:FALSE

• Ensure that the CA honors the requested extensions when signing the certificate.