PowerFlex Presentation Server's Web UI Fails To Load

Impacted Versions

• PowerFlex 3.5.x
• PowerFlex 3.6.0.x

The presentation server service starts, but the web page fails to load the initial login screen.

[root@host1 .config]# systemctl status mgmt-server.service

● mgmt-server.service - Scaleio MGMT Server
   Loaded: loaded (/etc/systemd/system/mgmt-server.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2023-1-09 05:30:03 EST; 11s ago
Main PID: 29700 (java)
   CGroup: /system.slice/mgmt-server.service
    
        └─29700 /bin/java -Xmx4g -Dlog4j2.formatMsgNoLookups=true -Djna.tmpdir=/opt/emc/scaleio/mgmt-server/tmp -Djava.io.tmpdir=/opt/emc/scaleio/mg...
Dec 09 05:30:08 host1 java[29700]: at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:320)
...
Dec 09 05:30:08 host1  java[29700]: at java.lang.Thread.run(Thread.java:750)

The presentation server log shows the following errors:

/opt/emc/scaleio/mgmt-server/logs/scaleio.log:

Suppressed: com.google.common.util.concurrent.ServiceManager$FailedService: HttpdService [FAILED]
Caused by: java.lang.IllegalStateException: KeyStores with multiple certificates are not supported on the base class org.eclipse.jetty.util.ssl.SslContextFactory. (Use org.eclipse.jetty.util.ssl.SslContextFactory$Server or org.eclipse.jetty.util.ssl.SslContextFactory$Client instead)
    at org.eclipse.jetty.util.ssl.SslContextFactory.newSniX509ExtendedKeyManager(SslContextFactory.java:1288)
    at org.eclipse.jetty.util.ssl.SslContextFactory.getKeyManagers(SslContextFactory.java:1270)
    at org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:372)
    at org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:243)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
    at org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:97)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
    at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:321)
    at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81)
    at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:234)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.eclipse.jetty.server.Server.doStart(Server.java:401)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at com.emc.vxflexos.webui.backend.httpd.HttpdService.startUp(HttpdService.java:31)
    at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62)
    at com.google.common.util.concurrent.Callables$4.run(Callables.java:119)
    at java.lang.Thread.run(Thread.java:750)

Run the following command to verify whether the presentation server uses multiple SAN entries. This can be run against the certificate the customer is renewing or replacing that has multiple SAN extension entries.

[root@host1 /]# openssl x509 -noout -text -in <location_of_new_signed_cert> | grep -A1 -i 'Subject Alternative Name'
            X509v3 Subject Alternative Name:
                DNS:host1, DNS:host1.cn

The presentation server, also known as the mgmt-server, lacks support for handling multiple Subject Alternative Name (SAN) extension entries. This issue can arise when the certificates of the mgmt-server, featuring several SAN extension entries, are renewed or replaced on the affected version.

Impact
Failure to load the web UI for the mgmt-server results in an inability to manage the PowerFlex cluster through the User Interface (UI). This makes it difficult to manage the PowerFlex system. 

Root Cause
The issue occurs when the Jetty framework seen below, the base class,

org.eclipse.jetty.util.ssl.SslContextFactory 

attempts to process multiple certificates in a KeyStore. This is an operation that it is not designed to handle.

The impacted presentation server version is not equipped to manage certificates that contain more than one Subject Alternative Name (SAN) extension entry.

This leads to a failure when it encounters such certificates.

This issue is addressed in PowerFlex 3.6.1 and later.

Workaround

1. Use a certificate that contains a single Subject Alternative Name (SAN) extension entry. This aligns with the current limitations of the mgmt-server. It should allow normal operation.
2. Upgrade the mgmt-server to version 3.6.1 or later. This version includes improved support for multiple SAN extension entries, and it is not necessary to adjust the certificates.