PowerFlex Manager 3.6 Storage-Only deployment fails while adding NVDIMM

• Storage only deployment with NVDIMM fails with below error.

#<ASM::Provider::Elementmanager::Scaleio::WrappedRestClientException: {"message"=>"Error with details", "httpStatusCode"=>500, "errorCode"=>0, "details"=>[{"errorMessage"=>"Could not open SDS device", "error"=>"TGT_ADD_DEV_OPEN_ERROR", "rc"=>71, "info"=>[{"type"=>"EET_ACC_DEV_KEY", "objId"=>nil, "str"=>nil, "num"=>nil, "mdmKey"=>nil, "protectionDomainKey"=>nil, "storagePoolKey"=>nil, "accelerationPoolKey"=>nil, "sdsKey"=>nil, "deviceKey"=>{"id"=>"cd4f0e1f00050000", "sdsKey"=>{"id"=>"d760414200000005", "name"=>"Sds-001.powerflex.com", "hostname"=>nil, "port"=>nil}, "path"=>"/dev/dax3.0", "name"=>"/dev/dax3.0"}, "volumeKey"=>nil}]}]}>

• Attempting to add the dax devices as Acceleration Devices in PowerFlex result in the error message "Could not open SDS device."

[root@flex002 ~]# scli --add_sds_device --sds_name Sds-001.powerflex.com --device_path /dev/dax3.0 --device_name /dev/dax3.0 --acceleration_pool_name AP-1 --force_device_takeover
Error: MDM failed command.
Could not open SDS device
        Acceleration Device cd4f0e1f00050000 Name: /dev/dax3.0, Path: /dev/dax3.0
        SDS d760414200000005 Name: Sds-001.powerflex.com

• After rebooting the storage node, adding dax devices as Acceleration Devices in PowerFlex succeeds.

• After configuring NVDIMM devices, before they can be added as acceleration devices to a PowerFlex Acceleration Pool a reboot is required on storage-only nodes. This is a new issue introduced in PowerFlex 3.6.
• As of today, PowerFlex Manager does not perform this reboot in its automation. Therefore storage-only service deployments with NVDIMM devices fail.

Note: This issue is fixed in PowerFlex 4.X.

PowerFlex Manager configures the NVDIMM dax devices and then adds them to PowerFlex one node at a time.

Due to this issue, storage-only service deployments fail when PowerFlex Manager attempts to add the dax devices for the first node. The remaining nodes will not have their dax devices configured at all.

The workaround is to manually run the PowerFlex Manager script to configure dax devices on all the SO nodes in the service and then reboot each node.

Steps:

# Copy dax config script from PFxM Appliance to SO node 
#SSH to PFxM and run the following command to copy the script from PFxM to the SO node
scp /opt/asm-deployer/scripts/configure_ndctl_namespaces.py root@SO_NODE_IP:/tmp

Ex.
scp /opt/asm-deployer/scripts/configure_ndctl_namespaces.py root@10.20.30.15:/tmp

# Connect via SSH to the SO node and run dax config script that was copied from PFxM to the SO node
cd /tmp
python configure_ndctl_namespaces.py 4 # where 4 is number of nvdimms on node

# Reboot SO node
shutdown -r now

Retry the service deployment in PowerFlex Manager.