Dell Unity: Unisphere reports Errors or Faults after CNA or I/O Module Commit
Summary: I/O Module or CNA ports fails to complete the commit. (Dell Correctable)
Symptoms
Unisphere may report the following issues if an I/O Module or CNA commit operation does not complete successfully.
- SP shutting down, degraded (one or both), or in critical failure (one or both)
- One SP may report rebooting
- Storage Pool reports "An operation is in progress. Please wait"
- When creating new objects the status is OK, Need Attention/An operation is in progress.
- I/O Module commit from UI or CLI is failing
To confirm where the issue resides, we can start my confirming if the array is still reporting any uncommitted ports use the following command.
uemcli /net/port/unc show -detail
If the commit is for an I/O Module, we can use the following command to confirm it is present.
uemcli /env/iomodule show -detail
Finally we can use the following log file to determine what the last reported step of the commit was. In the below log, we can see that the I/O module or CNA commits last completed task was the 16th of the total 22 tasks must have completed the commit.
/EMC/C4Core/log/EMCSystemLogBackup.log:
A 03/25/19 14:13:32.867 IOModule 30010 [INFO] User: Starting the hardware configuration commit operation A 03/25/19 14:13:35.281 IOModule 30014 [INFO] User: Completed task <1> of <22> (Waiting for system ready state) A 03/25/19 14:14:05.282 IOModule 30014 [INFO] User: Completed task <2> of <22> (System health checks) A 03/25/19 14:14:08.306 IOModule 30014 [INFO] User: Completed task <3> of <22> (Validating hardware configuration) A 03/25/19 14:14:08.626 IOModule 30014 [INFO] User: Completed task <4> of <22> (Enabling automatic restart) A 03/25/19 14:14:09.744 IOModule 30014 [INFO] User: Completed task <5> of <22> (Clearing system counters) A 03/25/19 14:14:10.959 IOModule 30014 [INFO] User: Completed task <6> of <22> (Clearing system counters) A 03/25/19 14:14:30.914 IOModule 30014 [INFO] User: Completed task <7> of <22> (Stopping services on SP) A 03/25/19 14:14:31.728 IOModule 30014 [INFO] User: Completed task <8> of <22> (Setting hardware persist flag) A 03/25/19 14:14:34.345 IOModule 30014 [INFO] User: Completed task <9> of <22> (Final tasks before reboot) A 03/25/19 14:24:21.219 IOModule 30014 [INFO] User: Completed task <10> of <22> (Rebooting peer SP) A 03/25/19 14:44:14.793 IOModule 30014 [INFO] User: Completed task <11> of <22> (Restarting services) A 03/25/19 14:44:52.059 IOModule 30014 [INFO] User: Completed task <12> of <22> (Waiting for system ready state) A 03/25/19 14:45:02.997 IOModule 30014 [INFO] User: Completed task <13> of <22> (Stopping services on SP) A 03/25/19 14:45:03.516 IOModule 30014 [INFO] User: Completed task <14> of <22> (Setting hardware persist flag) A 03/25/19 14:45:05.732 IOModule 30014 [INFO] User: Completed task <15> of <22> (Final tasks before reboot) A 03/25/19 14:45:18.255 IOModule 30014 [INFO] User: Completed task <16> of <22> (Rebooting SP) A 03/25/19 14:45:25.419 IOModule 30013 [INFO] User: Rebooting the SP to commit the hardware configuration
We can also see in the following log that the I/O Module or SLIC still reports in a running state.
/EMC/CEM/log/cemtracer_health_services.log:
25 Mar 2019 14:13:33 - [Health] INFO ./../../components/providers/osls/Health/src/SuppressionUtils.cpp:645] suppression event: 4 isSoftWareUpgradeRunning = 0 _isSWUpgradeRunning = 0 isSLICCommitRunning = 1 isReboot = 0
25 Mar 2019 14:13:33 - [Health] INFO ../../../components/providers/osls/Health/src/SuppressionUtils.cpp:674] SLIC commit in progress, suppressing alerts
Cause
N/A
Resolution
Contact your Service Provider for Resolution quoting this Knowledge Article.