PowerFlex SDS Panic Upon SDBG Server Startup
Summary: SDS fails to start up.
Symptoms
Scenario
SDS decouple and upon the attempt to start up, it crashes repeatedly, where the suspected issue is SDBG server startup.
Symptoms
Repeated panic upon startup attempt stack trace:
08/03 20:56:58.388914 Panic in file /data/build/workspace/ScaleIO-Common-Job/src/tgt/tgt.c, line 303, function main, PID 50379.Panic Expression rc== 65 PANIC_ID_tgt_1511773103687. /opt/emc/scaleio/sds/bin/sds-3.5.1100.107(mosDbg_PanicPrepare+0x13a) [0x93b62a] /opt/emc/scaleio/sds/bin/sds-3.5.1100.107(main+0x2ce) [0x4cd18e] /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fb153287555] (...) 08/03 21:13:59.562976 Panic in file /data/build/workspace/ScaleIO-Common-Job/src/tgt/tgt.c, line 303, function main, PID 123025.Panic Expression rc== 65 PANIC_ID_tgt_1511773103687. /opt/emc/scaleio/sds/bin/sds-3.5.1100.107(mosDbg_PanicPrepare+0x13a) [0x93b62a] /opt/emc/scaleio/sds/bin/sds-3.5.1100.107(main+0x2ce) [0x4cd18e] /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f72ec9c5555]
No SDS traces due to log rotation at the single encounter of this issue on the customer's site.
Impact
The system will be running in a single point of failure, in case of an additional component failure the customer will experience a DU situation.
Cause
Unknown - a suspected root cause is special SELinux rules that had been implemented, however, this is not certain and was determined as a possible root cause only as this was the only out of the ordinary factor on the customer's system.
An SDBG server startup issue could also be caused by the component’s SDBG listen port being used by another process on the system, insufficient resources, and other OS-related issues.
Resolution
Workaround
Reboot the SDS host - rebooting the host will allow the SDS to startup properly.
Impacted Versions
Unknown - the issue occurred on version 3.5.1.1 yet it's not clear which versions are impacted.
Fixed In Version
This issue is not fixed.