PowerFlex Device will not add to Storage Pool

Summary: Device will not be added to the storage pool due to I/O errors.

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

The add device operation failed, due to IO errors.

The Failed Add showed up in the MDM logs as follows:

2017-07-07 09:57:20.178 OPEN_SDS_DEVICE_FAILED    ERROR            Could not open a device on SDS: 473a915e00000009 (Path: /dev/mapper/svm_sdz). Error message: Add SDS device IO error
2017-07-07 09:55:42.916 OPEN_SDS_DEVICE_FAILED    ERROR            Could not open a device on SDS: 473a915e00000009 (Path: /dev/mapper/svm_sdz). Error message: Add SDS device IO error
2017-07-07 09:55:42.921 CLI_COMMAND_FAILED        WARNING          Command add_sds_device was not successful. Error code: Add SDS device IO error [19087506] ID: 0000000000000000
2017-07-07 09:57:15.457 MDM_CLI_CONF_COMMAND_RECEIVED INFO         Command add_sds_device received, User: ''. [19090058] SDS: Name: SDS05, SDS Device:name SDS05-SVM_SDZ path /dev/mapper/svm_sdz, Storage Pool: ENCPOOL1, Test Mode: 'Test and activate' Test Time: 0 force_device_takeover flag: Omitted
2017-07-07 09:57:20.175 SDS_DEV_ERROR_REPORT      ERROR            Device error reported on SDS: SDS05, Device: /dev/mapper/svm_sdz. State: NORMAL upDownState: UP processState: DEV_ERR_INPROGRESS

 

Impact 

Unable to add a device, reduced capacity.

Cause

SDS Trace shows add device operation failing with IO errors due to unresponsive/slow disk:

07/07 09:57:19.011555 0x7fd5438ffeb0:mosAsyncIO_DoIOTimedSyncIntern:01336: Time budget exceeded during IO to /dev/mapper/svm_sdz (Give-up on inflight IO)
07/07 09:57:19.011569 0x7fd5438ffeb0:mosAsyncIO_WriteZeroTimedSync:00811: mosAsyncIO_DoIOTimedSync returned ERROR, offset = 55574528, writeSize = 1048576
07/07 09:57:20.011293 0x7fd5438ffeb0:mosAsyncIO_DoIOTimedSyncIntern:01336: Time budget exceeded during IO to /dev/mapper/svm_sdz (Give-up on inflight IO)
07/07 09:57:20.011300 0x7fd5438ffeb0:mosAsyncIO_WriteZeroTimedSync:00811: mosAsyncIO_DoIOTimedSync returned ERROR, offset = 0, writeSize = 1048576
07/07 09:57:20.011304 0x7fd5438ffeb0:contDev_SendDeviceError:01632: Sending device error to MDM: DevId:97dfea7500090017 deviceName: /dev/mapper/svm_sdz readError: FALSE WriteError: TRUE
07/07 09:57:20.011471 0x7fd5438ffeb0:phyDev_InitWithBuf:02152: Io error /dev/mapper/svm_sdz, size 409600
07/07 09:57:20.011482 0x7fd5438ffeb0:contDev_SendDeviceError:01632: Sending device error to MDM: DevId:97dfea7500090017 deviceName: /dev/mapper/svm_sdz readError: TRUE WriteError: TRUE
07/07 09:57:20.015684 0x7fd5438ffeb0:contCmd_RemDevIntern:01022: DevId 97dfea7500090017 - Start rc = SUCCESS
07/07 09:57:20.015703 0x7fd5438ffeb0:contCmd_AddDev:01401: DevId 97dfea7500090017 - Done rc = TGT_ADD_DEV_IO_ERROR

/var/log/messages showed that the HBA Driver had issues with the disk, aborts, and timeouts:

Jul  7 09:57:15 scaleio05 kernel: mpt3sas_cm0: log_info(0x31120303): originator(PL), code(0x12), sub_code(0x0303)
Jul  7 09:57:15 scaleio05 kernel: mpt3sas_cm0: log_info(0x3112010c): originator(PL), code(0x12), sub_code(0x010c)

Resolution

After the faulty HW is replaced, go over all the drives (Including root drive: satadom):

  1. Zero out the first MB of the drive
  2. Restart the SDS process on SVM
  3. Validate the drive works without errors and encrypts in CloudLink
  4. Add to the Storage Pool 

Affected Products

PowerFlex rack, ScaleIO
Article Properties
Article Number: 000305768
Article Type: Solution
Last Modified: 09 Apr 2025
Version:  1
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.