SC_DISK_ERR1 CHECK DATA PROTECT (Sense key: 07 DATA PROTECT) recorded on an AIX host attached to a Dell Technologies array
Resumen: Write are failing from time to time, leading to an LVM error (LVM_IO_FAIL - EWRPROTECT) for no obvious reason, as if the device was "write protected." The error was reported as: "AIX shows disk operation error on array with PowerPath. No error showed in the array…." ...
Síntomas
Write occurs at a non-yet allocated location of a thin device. The device is not fully allocated as per the bellow example:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION hdiskpower0 active 2047 1176 70..00..287..409..410
The presence of free PPs (Physical Partitions) proves that, most probably, not all the storage for this device has been provisioned yet, although there is not a direct 1 to 1 relationship between allocated storage and free PPs.
Environment:
DELL EMC SW: PowerPath for AIX (any release)
DELL EMC HW: VNX/Unity using thin devices
DELL EMC HW: VMAX/PMAX
Non-DELL EMC SW: MPIO
OS: AIX
Issue: AIX shows disk operation error on Dell Technologies array with PowerPath. No error showed in the array (as initially reported) from time to time, writes are failing with the following characteristic sense bytes: 0102 0000 7000 0700
When using "summ" (IBM free decoder) to decode the error, it translates into "CHECK DATA PROTECT; ASC/ASCQ 2707"
Error SC_DISK_ERR1 as they appear in errpt -a, with characteristic sense bytes:
SENSE DATA 0A00 2A00 6F56 24E0 0000 2004 0000 0000 0000 0000 0000 0000 0102 0000 7000 0700 0000 000A 0000 0000 2707 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 ... SCSI_CMD: 2A WRITE(10) Status validity : 01 SCSI status field is valid SCSI status : 02 CHECK (SCSI sense is valid) SCSI Sense code : Response code: 70 Current errors --- Indicator bits: 00 Sense key: 07 DATA PROTECT
Other symptoms: Attempt to increase the size of the file system on the VG hold by the device is failing:
# chfs -a size=+25G /data chfs:/dev/data : There is an input or output error chfs failed : There is an input or output error
Causa
The device is a thin device, and the pool on the array is 100% full. As a consequence, any attempt to write data at a not yet allocated position is failing .... with a (misleading) write protected status.
Resolución
If from the array perspective the device is indeed a thin device and if it is confirmed that the pool is 100% full, space must be added to the pool to allow expansion of the device and provisioning.