PowerFlex에서 SDS 디바이스 오류를 지울 수 없음 - 디바이스 상태가 OS 수준에서 오프라인임

시나리오

SDS 디바이스가 실패하거나 오류를 보고하는 경우 SDS "clear device error"를 사용하려고 하면 디바이스가 오류 또는 실패 상태로 유지됩니다.

증상

ScaleIO 시스템 이벤트에서 디스크 디바이스 오류 또는 장애를 보고합니다.

 799   2016-01-22 17:28:39.818 SDS_DEV_ERROR_REPORT      ERROR    Device error reported on SDS: 10.3.1.21, Device: /dev/sdb. State: NORMAL upDownState: UP processState: DEV_ERR_INPROGRESS devErrState: REPORT 

Query SDS가 디스크 디바이스 오류 또는 장애를 보고함:

 ScaleIO-10-1-1-202:~ # scli --query_sds --sds_id 9780122600000003

Device information (total 8 devices):
         1: Name: ScaleIO-6a0a6209  Path: /dev/sdb  Original-path: /dev/sdb  ID: 851f01c100030000
                Storage Pool: SASPOOL, Capacity: 1675 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
         2: Name: ScaleIO-6a0a620a  Path: /dev/sdc  Original-path: /dev/sdc  ID: 851f01c200030001
                Storage Pool: SASPOOL, Capacity: 1675 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
         3: Name: ScaleIO-6a0a620b  Path: /dev/sdd  Original-path: /dev/sdd  ID: 851f01c300030002
                Storage Pool: SASPOOL, Capacity: 1675 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
         4: Name: ScaleIO-6a0a620c  Path: /dev/sde  Original-path: /dev/sde  ID: 851f01c400030003
                Storage Pool: SSDPOOL, Capacity: 1489 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
         5: Name: ScaleIO-6a0a620d  Path: /dev/sdf  Original-path: /dev/sdf  ID: 851f01c500030004
                Storage Pool: SASPOOL, Capacity: 1675 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Error
         6: Name: ScaleIO-6a0a620e  Path: /dev/sdg  Original-path: /dev/sdg  ID: 851f01c600030005
                Storage Pool: SSDPOOL, Capacity: 1489 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
         7: Name: ScaleIO-6a0a620f  Path: /dev/sdh  Original-path: /dev/sdh  ID: 851f01c700030006
                Storage Pool: SASPOOL, Capacity: 1675 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
         8: Name: ScaleIO-6a0a6210  Path: /dev/sdi  Original-path: /dev/sdi  ID: 851f01c800030007
                Storage Pool: SASPOOL, Capacity: 1675 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal

SVM/Linux 메시지 파일은 오프라인 디바이스를 보고합니다.

 Jan 22 17:28:35 ScaleIO-10-1-1-201 kernel: [45678.865605] end_request: I/O error, dev sdg, sector 1138313984
Jan 22 17:28:37 ScaleIO-10-1-1-201 kernel: [45681.452800] sd 2:0:6:0: [sdg] task abort on host 2, ffff8800b83f6e80
Jan 22 17:28:37 ScaleIO-10-1-1-201 kernel: [45681.452877] sd 2:0:1:0: [sdb] task abort on host 2, ffff8801b7476d80
Jan 22 17:28:37 ScaleIO-10-1-1-201 kernel: [45681.453086] sd 2:0:8:0: [sdh] task abort on host 2, ffff8800b83f6280
Jan 22 17:28:37 ScaleIO-10-1-1-201 kernel: [45681.453109] sd 2:0:8:0: [sdh] task abort on host 2, ffff8800a37a6c80
Jan 22 17:28:37 ScaleIO-10-1-1-201 kernel: [45681.453133] sd 2:0:9:0: [sdi] task abort on host 2, ffff8800b83f6b80
Jan 22 17:28:47 ScaleIO-10-1-1-201 kernel: [45691.537180] sd 2:0:5:0: rejecting I/O to offline device

ESXi VMkernel 로그가 디스크 디바이스에 오류를 보고합니다.

 2016-01-22T09:40:21.801Z cpu1:33420)ScsiDeviceIO: 7024: Could not detect setting of QErr for device naa.614187704f3b47001e34b585468abf85. Error Not supported.
2016-01-22T09:40:21.801Z cpu1:33420)ScsiDeviceIO: 7538: Could not detect setting of sitpua for device naa.614187704f3b47001e34b585468abf85. Error Not supported.
2016-01-22T09:40:21.801Z cpu5:33593)NMP: nmp_ThrottleLogForDevice:3178: Cmd 0x28 (0x439e1a830cc0, 0) to dev "naa.614187704f3b47001e34b585468abf85" on path "vmhba1:C2:T1:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x6 0x29 0x0. Act:NONE
2016-01-22T09:40:21.801Z cpu5:33593)ScsiDeviceIO: 2607: Cmd(0x439e1a830cc0) 0x28, CmdSN 0xd62 from world 0 to dev "naa.614187704f3b47001e34b585468abf85" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x6 0x29 0x0.
2016-01-22T09:40:21.801Z cpu5:33593)ScsiCore: 1609: Power-on Reset occurred on naa.614187704f3b47001e34b585468abf85
2016-01-22T09:40:21.844Z cpu5:33593)NMP: nmp_ThrottleLogForDevice:3178: Cmd 0x1a (0x439e1a830cc0, 0) to dev "naa.614187704f3b47001e34b585468abf85" on path "vmhba1:C2:T1:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0. Act:NONE
2016-01-22T09:40:21.844Z cpu5:33593)ScsiDeviceIO: 2645: Cmd(0x439e1a830cc0) 0x1a, CmdSN 0xd66 from world 0 to dev "naa.614187704f3b47001e34b585468abf85" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2016-01-22T09:40:21.844Z cpu1:33420)ScsiDevice: 3835: Successfully registered device "naa.614187704f3b47001e34b585468abf85" from plugin "NMP" of type 0
2016-01-22T09:40:21.844Z cpu1:33420)NMP: nmp_DeviceUpdateProtectionInfo:569: Set protection info for device 'naa.614187704f3b47001e34b585468abf85', Enabled: 0 ProtType: 0x0 Guard: 0x0 ProtMask: 0x0
2016-01-22T22:27:49.085Z cpu19:33115)WARNING: NMP: nmpDeviceTaskMgmt:2284: Attempt to issue lun reset on device naa.614187704f3b47001e34b585468abf85. This will clear any SCSI-2 reservations on the device.

영향

디스크 디바이스는 실패한 상태로 유지됩니다.
SDS 디바이스 오류는 지울 수 없습니다.

해결 방법

SVM - Linux 환경:

1. 디스크 디바이스의 현재 상태를 확인합니다.

[root@ssltest ~]# cat /sys/block/sdx/device/state
offline 

2. 디스크 디바이스가 "오프라인"으로 표시된 경우 다음 명령을 사용하여 디스크 디바이스를 "온라인"으로 변경합니다.

echo "running" > /sys/block/sdx/device/state

3. SCLI 또는 UI를 사용하여 SDS 디바이스 오류를 지웁니다 .

Windows 환경:

1. "Logical Disk Manager" 또는 디스크 파트를 사용하여 디스크 디바이스의 현재 상태를 확인합니다.

C:\>diskpart
Microsoft DiskPart version 6.1.7601
Copyright (C) 1999-2008 Microsoft Corporation.
On computer: ISENABLOVSL1C
DISKPART> list disk
  Disk ###  Status         Size     Free     Dyn  Gpt
  --------  -------------  -------  -------  ---  ---
  Disk 0    Online          238 GB      0 B
DISKPART> 

2. 디스크 디바이스가 "오프라인"으로 표시된 경우 다음 명령을 사용하여 디스크 디바이스를 "온라인"하거나 "Logical Disk Manager"를 사용합니다.

DISKPART> online disk

모든 OS에서 온라인 디스크 장치에 대한 대체 옵션:

1. 가능한 경우 SDS를 유지 보수 모드로 전환합니다.
2. SDS 재부팅 서버
3. 유지 보수 모드 종료 (1단계에서 수행한 경우)
4. SDS 디바이스 오류 지우기 (UI 또는 CLI 사용)
5. ScaleIO에서 디바이스 상태 확인