Connectrix MDS Series: FC Module in Powered down status due to System Error code "0x42630007 mauler obt bist failure"
Summary: Hardware failure is detected by the Built-In Self Test (BIST) on one of the F16 ASICs
Symptoms
BIST (Built-In Self Test) Failure encountered on Port Blade Module .`show module`
Mod Ports Module-Type Model Status
--- ----- ----------------------------------- ------------------ ----------
8 48 2/4/8/10/16 Gbps Advanced FC Module DS-X9448-768K9 powered-dn
`show module internal exceptionlog module 8`
Module Slot Number: 8
Device Id : 204
Device Name : F16 Generic Driver
Device Errorcode : 0xccc01400
Device ID : 204 (0xcc)
Device Instance : 01 (0x01)
Dev Type (HW/SW) : 04 (0x04)
ErrNum (devInfo) : 00 (0x00)
System Errorcode : 0x42630007 mauler obt bist failure
Error Type : Minor error
PhyPortLayer : Fibre Channel
Port(s) Affected : fc8/9-16
Error Description : hardware init failure
DSAP : 0 (0x0)
UUID : 1129 (0x469)
Time : Fri Jun 26 01:05:56 2020
(Ticks: 5EF59044 jiffies)
Mod Power-Status Reason
--- ------------ ---------------------------
8 powered-dn Reset (powered-down) because module does not boot
show logging log:
2020 Jun 26 01:04:06 CMISAN-10 %MODULE-2-MOD_SOMEPORTS_FAILED: Module 8 (Serial number: XXXXXXXXXXX) reported failure on ports fc8/9-16 (Fibre Channel) due to mauler obt bist failure in device DEV_F16_CMN (device error 0xccc01400)
2020 Jun 26 01:05:01 CMISAN-10 %PORT-5-MODULE_BRINGUP_NOT_ALLOWED: Not allowing module 8 to come up for reason: Device Name:[0xC4] Instance:[38] Error Type:[diag error] code:[0] (0xcc426300)
Cause
BIST (Built-In Self Test) is a part of POST (Power On Self Test). BIST is basically a diagnostic test meant to detect hardware failures. The different switches that are part of the MDS platform, supports a whole set of diagnostic tests, and that includes BIST.
When a chip gets corrupted, as a result BIST fails. This is an issue with hardware failure on one of the F16 ASICs inside Port blade module.
In the above case - BIST is getting failed for one of the ASIC which is responsible for ports (9-16).
Resolution
The Module needs to be replaced as this is a Hardware ASIC Failure.