PowerFlex: SVM OS Conversion Fails When MDM Authentication Is Enabled SDS_AUTHENTICATION_FAILED

• SVM OS conversion completes , but SDS service fails to reconnect to cluster.
• The query SDS command's output shows the SVM listed but its State is Disconnected.
• MDM event logs show SDS_RECONNECTED immediately followed by SDS_AUTHENTICATION_FAILED with error: "Failed loading the authentication key-pair":

  2025-10-10 16:20:36.649 SDS_RECONNECTED INFO SDS: Sds-esxi249.chronex.lab (ID 39c9b4dc00000003) reconnected
  2025-10-10 16:20:36.651 SDS_AUTHENTICATION_FAILED ERROR SDS: Sds-esxi249.chronex.lab (ID 39c9b4dc00000003) failed authentication (Failed loading the authentication key-pair)

• MDM authentication has been explicitly enabled (not the default configuration): 

  scli --query_all | grep -i "MDM connection"
  MDM connection authentication: Enabled

• MDM System clocks may show incorrect time (e.g., 1970‑01‑01) indicating missing NTP configuration.
• Chronyc tracking reports Offline or shows no valid NTP source.

Impact

• OS conversion cannot be completed while MDM authentication is enabled.
• Converted SDS nodes remain offline and cannot rejoin the cluster.
• Storage pools may become DEGRADED due to missing SDS capacity.

Important: MDM authentication is disabled by default in PowerFlex. This issue only affects environments where authentication has been explicitly enabled for enhanced security.

When MDM authentication is enabled, the SDS service requires valid certificates to communicate with the MDM. During OS conversion, the SDS service is reinstalled and loses its certificate credentials. When the SDS attempts to reconnect, the MDM authentication layer blocks registration because the SDS cannot present valid certificates.

Also, if NTP is not properly configured on the MDM cluster nodes, the system clock may be incorrect (commonly showing 1970‑01‑01). Certificates generated with invalid timestamps are rejected by the MDM, resulting in certificate issuance failure events. This prevents successful certificate generation even after authentication is re-enabled.

The OS conversion process does not automatically handle the MDM authentication workflow, requiring manual intervention to disable authentication, allow reconnection, and regenerate certificates.

1. Before starting the OS conversion, validate that NTP is configured on all PowerFlex MDM nodes:

chronyc tracking

Example:

svm-esxi246:~ # chronyc tracking
Reference ID    : 0AEA7154 (CGee-10-234-113-84.Chronex.lab)
Stratum         : 4
Ref time (UTC)  : Wed Oct 29 14:45:52 2025
System time     : 0.000019126 seconds slow of NTP time
Last offset     : -0.000027579 seconds
RMS offset      : 0.000036048 seconds
Frequency       : 10.327 ppm slow
Residual freq   : -0.062 ppm
Skew            : 0.298 ppm
Root delay      : 0.033223286 seconds
Root dispersion : 0.037000805 seconds
Update interval : 129.4 seconds
Leap status     : Normal

If NTP is not configured, configure an NTP server and validate:

chronyc add server 10.10.10.1 prefer
systemctl restart chronyd
chronyc tracking

2. Verify MDM authentication status:

scli --query_all | grep -i "MDM connection"

Example:

scli --query_all | grep -i "MDM connection"
MDM connection authentication: Enabled

3. If MDM authentication is enabled, temporarily disable MDM authentication before proceeding with OS conversion:

scli --set_component_authentication_properties --dont_use_authentication

Example:

scli --query_all | grep -i "MDM connection" 
MDM connection authentication: Disabled

4. Perform the OS conversion using PFMP.

5. Verify the SDSs come online after conversion. Expected: SDSs show a Connected status:

scli --query_all_sds

6. Re‑enable MDM authentication after a successful reconnection of the SDSs, if required:

scli --set_component_authentication_properties --use_authentication

7. Verify the SDSs remains online with authentication enabled. Expected: SDSs show a Connected status: 

scli --query_all_sds

8. To Regenerate the certificates for 1 or all SDSs:

For a Single SDS:
scli --generate_certificate --sds_id 39c9b4dc00000003--i_am_sure

For All SDS's:
for sds_id in $(scli --query_all_sds | grep "SDS ID:" | awk '{print $3}'); do scli --generate_certificate --sds_id $sds_id --i_am_sure; done
Successfully generated a new certificate
Successfully generated a new certificate
Successfully generated a new certificate
Successfully generated a new certificate

Impacted Versions

PFMP 4.6.1