PowerFlex: storcli add vd complains that "controller has data in cache for offline or missing virtual disks"

This can happen if a Virtual Disk/VD is missing because of pulling a hard drive without following proper procedure. Subsequent attempts to add the VD back might result in the above error.

This can happen on VxRack Node and other systems using LSI controller and storcli.

storcli add vd command returns error that "controller has data in cache for offline or missing virtual disks."
 

# /opt/MegaRAID/storcli/storcli64 /c0 add vd type=raid0 drives=252:4 direct wb ra Controller = 0 Status = Failure Description = controller has data in cache for offline or missing virtual disks Unable to add virtual disks. 

When a hard drive is pulled without following the correct procedure, the LSI Controller will have preserved cache for any write operations on the respective VD that have not yet been committed to the disk.

This prevents attempts to add VDs (using other PDs as well). If the server reboots at this point, a POST message is displayed on the console and does not allow proceeding past it, unless the preserved cache is cleared in the controller BIOS.
 

The preserved cache must be cleared before proceeding to add virtual disks. During reboot, this can be done using controller BIOS. It can also be done with storcli:

To verify that there is a preserved cache:

# /opt/MegaRAID/storcli/storcli64 /c0 show preservedCache Controller = 0 Status = Success Description = None ----------- VD State ----------- 1 Missing -----------

In the above example, the VD #1 is in "missing" state and does not show in "storcli /c0 show" output.

To delete the preserved cache:

# /opt/MegaRAID/storcli/storcli64 /c0/v1 delete preservedCache

Or

# /opt/MegaRAID/storcli/storcli64 /c0/vall delete preservedCache

Note: this operation will invalidate data on the disk. This should not cause issues to ScaleIO as there will be two copies for all data.