Dell Unity: XT systems could experience an unexpected SP reboot on UnityOS 5.0.x
Summary: Storage Processors (SPs) may panic due to a memory leak.
Symptoms
Three conditions must be met for the system to be considered impacted:
- The system must be a Dell Unity 380/480/680/880.
- The system must have one or more, or a combination of both, 25G I/O modules ClearskyX or Lightblade.
- The system's memory utilization must be 90% or above.
Unity XT systems may experience an unexpected SP reboot due to a memory leak in the driver for the 25 Gb Ethernet I/O module.
Cause
The uptime has ranged from 90 to 150 days on most of these SPs before they experienced the unexpected reboot.
Resolution
To determine if the system is impacted, check for those above two conditions first, by logging in through SSH and running the below commands:
- # svc_diag -s cru | egrep "sp|slic|iom"
Displaying CRU information: spa: OK iom: OK WARNADO_IOM_BOM_B_REV_C slic0: OK CLEARSKY_X slic1: OK VORTEXQ_X spb: OK iom: OK WARNADO_IOM_BOM_B_REV_C slic0: OK CLEARSKY_X slic1: OK VORTEXQ_X - # uemcli -sslPolicy accept -noHeader /metrics/value/rt -path sp.*.memory.summary.utilization show -interval 5
Timestamp |SP |summary | |System | |Memory | |Utilization | |% --------------------+----------+----------- 2020-06-29 19:21:45 |spa | 94.68 |spb | 61.91 2020-06-29 19:21:50 |spa | 94.63 |spb | 61.91 2020-06-29 19:21:55 |spa | 94.74 |spb | 61.90 2020-06-29 19:22:00 |spa | 94.68 |spb | 61.91
Fix:
This issue has been fixed in version 5.0.3.0.5.014.
It is highly recommended to upgrade the UnityOS to 5.0.3 or later. Instructions on how to upgrade the UnityOS can be found at Dell Unity: How to upgrade the UnityOS (User Correctable)
Workaround:
Since this is a memory leak, reboot both storage processors periodically.
Instructions for how to reboot a Storage Processor can be found in Dell Unity: How to Reboot a Storage Processor (User Correctable)