PowerFlex 4.X: MDS systems showing Empty

Summary: This article guides through resolving inventory issues in the MDS Powerflex System caused by missing MDM IP addresses, as indicated by MDS kubectl logs and LIA logs on the MDMs.

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

  • MDS Powerflex System inventory is not displayed.
  • MDS kubectl logs show a 404 error when trying to fetch MdmIps from the LCM manager, stating that mdm.ip.addresses is missing and failing to connect to the cluster.
23:21:28.383 [|https-openssl-nio-443-exec-4] INFO  c.e.s.s.services.GeneralServiceImp - Can't fetch MdmIps from LCM manager at https://thin-deployer:9433/asm/device/mdm_cluster_info?certname=scaleio-block-legacy-gateway-mds. Request returned response code 404, respons
e message: Not Found
23:21:28.383 [|https-openssl-nio-443-exec-4] ERROR c.e.s.s.services.GeneralServiceImp - mdm.ip.addresses is missing
23:21:28.383 [|https-openssl-nio-443-exec-4] INFO  c.e.s.s.services.GeneralServiceImp - Got Cluster IPs
23:21:28.383 [|https-openssl-nio-443-exec-4] ERROR c.e.s.s.s.ConnectionServiceImpl - Failed connecting to cluster
23:21:28.383 [|https-openssl-nio-443-exec-4] ERROR c.e.s.s.r.SystemRepositoryImpl -
com.emc.s3g.scaleio.common.gateway.ScaleIORemoteRuntimeException: Failed connecting to cluster: no MDM IP is set
  • LIA logs on MDMs indicate failure in running scli --reset_management_trust, likely due to the mgmt_ca.pem certificate not being added to the MDM truststore.
2024/01/03 18:06:18.061168 MED:7f905831bdb0:liaCmd_ResetMgmtTrust:05502: Executing command : scli --reset_management_trust --mdm_port
 8611 --i_am_sure
2024/01/03 18:06:18.377107 LOW:7f905831bdb0:liaCmd_ResetMgmtTrust:05508: Run command [scli --reset_management_trust --mdm_port 8611 -
-i_am_sure] returned Value: 1 [0-is success]
2024/01/03 18:06:18.377119 LOW:7f905831bdb0:liaCmd_ResetMgmtTrust:05539: MDM Reset Security command failed, cmdRc = 1

Cause

The root cause of the inventory issue in the MDS Powerflex System is the improper setup of the necessary certificates prior to the import process. The mgmt_ca.pem certificate, in particular, must be correctly established within the MDM truststore to ensure successful MDM IP address retrieval during the import. The failure to recognize this certificate leads to connectivity problems and subsequent inventory issues.

Resolution

1- Check MDM Configuration Files:

Go to.

/opt/emc/scaleio/mdm/cfg

And list the certificates to ensure they are generated.

Verify the mgmt_ca.pem certificate's presence and the last few characters. 2- Update CLI Trust Store:

cat /root/.scli/cli_truststore.pem

If the mgmt_ca.pem certificate is not found in the CLI trust store, add it with.

scli --add_certificate --certificate_file /opt/emc/scaleio/mdm/cfg/mgmt_ca.pem.

3- Spread the Certificate Across MDMs:

Ensure the mgmt_ca.The pem certificate is present on all MDMs, except for the tiebreaker.

4- Remove and Re-add MDS Gateway:

From the resource page, remove the MDS gateway temporarily for troubleshooting. 

5- Back-Up Certificates:

Create backups of mgmt_ca.pem and mdm_mno_certificate.pem using the cp command.

cp mgmt_ca.pem mgmt_ca.pem.bk
cp mdm_mno_certificate.pem mdm_mno_certificate.pem.bk

6- Rediscover MDS:

Once all steps are completed, rediscover the MDS to RE-ESTABLISH proper system functionality.

Affected Products

PowerFlex appliance R640

Products

PowerFlex rack, VxFlex Ready Nodes, PowerFlex custom node, PowerFlex appliance R650, PowerFlex appliance R6525, PowerFlex appliance R660, PowerFlex appliance R6625, Powerflex appliance R750, PowerFlex appliance R760, PowerFlex appliance R7625 , PowerFlex custom node, PowerFlex custom node R650, PowerFlex custom node R6525, PowerFlex custom node R660, PowerFlex custom node R6625, PowerFlex custom node R750, PowerFlex custom node R760, PowerFlex custom node R7625, PowerFlex appliance R740XD, PowerFlex appliance R7525, PowerFlex appliance R840 ...
Article Properties
Article Number: 000220860
Article Type: Solution
Last Modified: 29 Dec 2025
Version:  3
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.