RE: MD3000i - RAM Parity error

I have exactly the same issue, MD3000i having both controller in boot loop after power failure. Can see similar logs by connecting to the console. If I am able to arrange the controller from used market,

what would be the procedure of replacement,

Will the data be intact?

Do i need to configure IPs etc after replacing the controller it will somehow get from somewhere?

Highly appreciate your help.

Below are the console logs for your reference from left controller.

04/01/21-17:19:23 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:2 prevNumActivePhys:0 numActivePhys:1
04/01/21-17:19:23 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: IOCPort: chan:1, FAILED to DEGRADED
04/01/21-17:19:23 (GMT) (tSasEvtWkr): WARN: sasIocPhyUp: Initializing Channel 1: Attached SAS Address: 5a4badb44e398510
04/01/21-17:19:25 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:3 prevNumActivePhys:1 numActivePhys:2
04/01/21-17:19:25 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: IOCPort: chan:1, DEGRADED to OPTIMAL
04/01/21-17:19:28 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:19:28 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:19:29 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:19:29 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:19:29 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:19:29 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:19:29 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:19:29 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:19:30 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e0460 4c0c8 bytes , result 0
04/01/21-17:19:41 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, FAILED to DEGRADED
04/01/21-17:19:41 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, DEGRADED to OPTIMAL
04/01/21-17:19:41 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:xa - WidePort: #0 - Alt Controller, chan:1 encl:0, subExp:255, FAILED to DEGRADED
04/01/21-17:19:41 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:xa - WidePort: #0 - Alt Controller, chan:1 encl:0, subExp:255, DEGRADED to OPTIMAL
04/01/21-17:19:41 (GMT) (tSasCfg105): NOTE: Alt Controller path up - chan:0 phy:18 itn:1
04/01/21-17:19:41 (GMT) (tSasCfg106): NOTE: Alt Controller path up - chan:1 phy:16 itn:2
04/01/21-17:19:55 (GMT) (IOSched): NOTE: Connect Initiator: 1 - channel: 1,devHandle: x2c, SAS Address: 5a4badb44e398500
04/01/21-17:19:55 (GMT) (iconMgr2): WARN: sasNetworkManager::healthCheckEvent - Unknown Event (-3)
04/01/21-17:19:55 (GMT) (IOSched): NOTE: Connect Initiator: 2 - channel: 0,devHandle: x2b, SAS Address: 5a4badb44e398501
04/01/21-17:19:55 (GMT) (iconMgr2): NOTE: CheckInMonitor: Check-in failed (NoUnmarshalFuncException Error)
04/01/21-17:19:56 (GMT) (iacTask5): NOTE: fbm:altValidateSubModelID: Sub-Model IDs Validated
04/01/21-17:19:58 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:19:58 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:19:58 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:19:58 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:19:58 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:19:58 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:19:58 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:19:58 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:19:59 (GMT) (tRAID): WARN: QLStartAdapter: ControllerErrorCount exceeds threshold.
04/01/21-17:19:59 (GMT) (tRAID): ERROR: QLInitializeDevice: QLStartAdapter failed
04/01/21-17:19:59 (GMT) (tRAID): ERROR: QLAddDevice: controller/device/chip initialization failed.
04/01/21-17:19:59 (GMT) (tRAID): ERROR: qlgEnableHostInterface: QLInitializeDevice failed.
04/01/21-17:20:00 (GMT) (tRAID): NOTE: ********************************************************************************
04/01/21-17:20:00 (GMT) (tRAID): NOTE: QLogic Target Applicat
-=<###>=-
Attaching interface lo0... done

Adding 9768 symbols for standalone.
Error
04/01/21-17:20:13 (GMT) (tRootTask): NOTE: I2C transaction returned 0x0423fe00

WARNING: Reset by alternate controller

Current date: 04/01/21 time: 06:51:19

Send for Service Interface or baud rate change
04/01/21-17:20:30 (GMT) (tRAID): NOTE: SOD Sequence is Normal, 0
04/01/21-17:20:30 (GMT) (tRAID): NOTE: SOD: removed SAS host from index 0
04/01/21-17:20:30 (GMT) (tRAID): NOTE: In iscsiIOQLIscsiInitDq. iscsiIoFstrBase = 0x0
04/01/21-17:20:30 (GMT) (tRAID): NOTE: Turning on tray summary fault LED
04/01/21-17:20:32 (GMT) (tRAID): NOTE: SYMBOL: SYMbolAPI registered.
esmc0: LinkUp event
04/01/21-17:20:35 (GMT) (tRAID): NOTE: Initiating Drive channel: ioc:0 bringup
04/01/21-17:20:38 (GMT) (tRAID): NOTE: IOC Firmware Version: 00-24-63-00
04/01/21-17:20:39 (GMT) (tNetCfgInit): NOTE: Network Ready
04/01/21-17:20:47 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:0 phy:0 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:20:47 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:0 phy:1 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:20:48 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:2 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:20:48 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:3 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:20:48 (GMT) (tSasCfg016): NOTE: Alt Controller path up - chan:0 phy:18 itn:1
04/01/21-17:20:48 (GMT) (tSasCfg021): NOTE: Alt Controller path up - chan:1 phy:16 itn:2
04/01/21-17:20:57 (GMT) (tRAID): NOTE: IonMgr: Drive Interface Enabled
04/01/21-17:20:58 (GMT) (tRAID): NOTE: SOD: Instantiation Phase Complete
04/01/21-17:20:58 (GMT) (tRAID): NOTE: Inter-Controller Communication Channels Opened
04/01/21-17:20:58 (GMT) (tSasDiscCom): NOTE: SAS Discovery complete task spawned
04/01/21-17:20:58 (GMT) (IOSched): NOTE: New Initiator: 1 - channel: 1,devHandle: x2b, SAS Address: 5a4badb44e398500
04/01/21-17:20:58 (GMT) (tRAID): NOTE: LockMgr Role is Slave
04/01/21-17:20:58 (GMT) (IOSched): NOTE: New Initiator: 2 - channel: 0,devHandle: x1a, SAS Address: 5a4badb44e398501
04/01/21-17:20:58 (GMT) (sasCheckExpanderSet): NOTE: Expander Firmware Version: 0116-e05c
04/01/21-17:20:58 (GMT) (sasCheckExpanderSet): NOTE: Expander SAS address: Hi = x5842b2b4 Low = x51accd10
04/01/21-17:20:59 (GMT) (tRAID): NOTE: spmEarlyData: Using cached data
04/01/21-17:21:03 (GMT) (tSasDiscCom): WARN: SAS: Initial Discovery Complete Time: 30 seconds
04/01/21-17:21:03 (GMT) (tRAID): NOTE: WWN baseName 0004842b-2b51accd (valid==>SigMatch)
04/01/21-17:21:03 (GMT) (tRAID): NOTE: ionEnableHostInterfaces is waiting for a channel to become ready
04/01/21-17:21:03 (GMT) (tRAID): NOTE: ionEnableHostInterfaces waited 800ms for a channel to become ready
04/01/21-17:21:03 (GMT) (tRAID): NOTE: IonMgr: Host Interface Enabled
04/01/21-17:21:03 (GMT) (tRAID): NOTE: SOD: Pre-Initialization Phase Complete
04/01/21-17:21:26 (GMT) (iacTask7): NOTE: ACS: Acs Needed on Alt: No, StateCode: 0, ReasonCode = 2
04/01/21-17:21:26 (GMT) (tRAID): NOTE: ACS: autoCodeSync(): Process start. Comm Mode: 0, Status: 1
04/01/21-17:21:26 (GMT) (tRAID): NOTE: SOD: Code Synchronization Initialization Phase Complete
04/01/21-17:21:29 (GMT) (NvpsPersistentSyncM): NOTE: NVSRAM Persistent Storage updated successfully
04/01/21-17:21:29 (GMT) (tRAID): NOTE: USM Mgr initialization complete with 0 records.
04/01/21-17:21:30 (GMT) (tRAID): NOTE: EDR - recieved 1 small records
04/01/21-17:21:30 (GMT) (tRAID): NOTE: EDR - recieved 0 large records
04/01/21-17:21:32 (GMT) (tRAID): NOTE: Acquire 0.020 secs
04/01/21-17:21:35 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e4340 4c0c8 bytes , result 0
04/01/21-17:22:03 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:22:03 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:22:03 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:22:03 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:22:03 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:22:03 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:22:03 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:22:03 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:22:05 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e4340 4c0c8 bytes , result 0
04/01/21-17:22:07 (GMT) (tHckReset): WARN: Alt Ctl Reboot:
Reboot CompID: 0x408
Reboot reason: 0x6
Reboot reason extra: 0x1
04/01/21-17:22:07 (GMT) (tHckReset): NOTE: holding alt ctl in reset
04/01/21-17:22:07 (GMT) (tHckReset): NOTE: releasing alt ctl from reset
04/01/21-17:22:07 (GMT) (tHckReset): NOTE: LockMgr Role is Master
04/01/21-17:22:07 (GMT) (tHckReset): NOTE: sas: Peering Disabled (Health Check) Event: -4
04/01/21-17:22:07 (GMT) (IOSched): NOTE: Disconnect Initiator 1 - channel: 1, devHandle: x2b, I/O: 0, SAS Address: 5a4badb44e398500
04/01/21-17:22:07 (GMT) (IOSched): WARN: sasRemoveInitiator - No I/O's - send EventAck: event: 00000018, eventContext: 0002002b
04/01/21-17:22:07 (GMT) (sasTgtSendEventAck): WARN: sasTgtSendEventAckTask - SendEventAck for host: 1
04/01/21-17:22:07 (GMT) (IOSched): NOTE: Disconnect Initiator 2 - channel: 0, devHandle: x1a, I/O: 0, SAS Address: 5a4badb44e398501
04/01/21-17:22:07 (GMT) (IOSched): WARN: sasRemoveInitiator - No I/O's - send EventAck: event: 00000018, eventContext: 0002001a
04/01/21-17:22:07 (GMT) (sasTgtSendEventAck): WARN: sasTgtSendEventAckTask - SendEventAck for host: 2
04/01/21-17:22:07 (GMT) (tSasEvtWkr): NOTE: sasIocPhyDown: chan:1 phy:2 prevNumActivePhys:2 numActivePhys:1
04/01/21-17:22:07 (GMT) (tSasEvtWkr): NOTE: sasIocPhyDown: IOCPort: chan:1, OPTIMAL to DEGRADED
04/01/21-17:22:08 (GMT) (tSasEvtWkr): NOTE: sasIocPhyDown: chan:1 phy:3 prevNumActivePhys:1 numActivePhys:0
04/01/21-17:22:08 (GMT) (tSasEvtWkr): NOTE: sasIocPhyDown: IOCPort: chan:1, DEGRADED to FAILED
04/01/21-17:22:08 (GMT) (tSasEvtWkr): NOTE: sasRemoveExpanders: removing expanders on chan:1 level:0 and below
04/01/21-17:22:08 (GMT) (tSasEvtWkr): NOTE: sasLinkDown: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, OPTIMAL to DEGRADED
04/01/21-17:22:08 (GMT) (tSasEvtWkr): NOTE: sasLinkDown: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, DEGRADED to FAILED
04/01/21-17:22:24 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:2 prevNumActivePhys:0 numActivePhys:1
04/01/21-17:22:24 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: IOCPort: chan:1, FAILED to DEGRADED
04/01/21-17:22:24 (GMT) (tSasEvtWkr): WARN: sasIocPhyUp: Initializing Channel 1: Attached SAS Address: 5a4badb44e398510
04/01/21-17:22:26 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:3 prevNumActivePhys:1 numActivePhys:2
04/01/21-17:22:26 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: IOCPort: chan:1, DEGRADED to OPTIMAL
04/01/21-17:22:32 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:22:32 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:22:33 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:22:33 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:22:33 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:22:33 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:22:33 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:22:33 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:22:34 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e4340 4c0c8 bytes , result 0
04/01/21-17:22:41 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, FAILED to DEGRADED
04/01/21-17:22:41 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, DEGRADED to OPTIMAL
04/01/21-17:22:41 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:xa - WidePort: #0 - Alt Controller, chan:1 encl:0, subExp:255, FAILED to DEGRADED
04/01/21-17:22:41 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:xa - WidePort: #0 - Alt Controller, chan:1 encl:0, subExp:255, DEGRADED to OPTIMAL
04/01/21-17:22:41 (GMT) (tSasCfg069): NOTE: Alt Controller path up - chan:0 phy:18 itn:1
04/01/21-17:22:41 (GMT) (tSasCfg070): NOTE: Alt Controller path up - chan:1 phy:16 itn:2
04/01/21-17:22:56 (GMT) (IOSched): NOTE: Connect Initiator: 1 - channel: 1,devHandle: x2c, SAS Address: 5a4badb44e398500
04/01/21-17:22:56 (GMT) (iconMgr4): WARN: sasNetworkManager::healthCheckEvent - Unknown Event (-3)
04/01/21-17:22:56 (GMT) (IOSched): NOTE: Connect Initiator: 2 - channel: 0,devHandle: x2b, SAS Address: 5a4badb44e398501
04/01/21-17:22:56 (GMT) (iconMgr4): NOTE: CheckInMonitor: Check-in failed (NoUnmarshalFuncException Error)
04/01/21-17:22:56 (GMT) (iacTask6): NOTE: fbm:altValidateSubModelID: Sub-Model IDs Validated
04/01/21-17:23:01 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:23:01 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:23:02 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:23:02 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:23:02 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:23:02 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:23:02 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:23:02 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:23:03 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e4340 4c0c8 bytes , result 0
04/01/21-17:23:16 (GMT) (iacTask7): NOTE: ACS: Acs Needed on Alt: No, StateCode: 0, ReasonCode = 2
04/01/21-17:23:19 (GMT) (iacTask7): NOTE: sas: checkInAlt Peering Enabled
04/01/21-17:23:30 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:23:30 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:23:31 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:23:31 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:23:31 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:23:31 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:23:31 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:23:31 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:23:33 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e4340 4c0c8 bytes , result 0
04/01/21-17:24:00 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:24:00 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:24:01 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:24:01 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:24:01 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:24:01 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:24:01 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:24:01 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:24:02 (GMT) (tRAID): WARN: QLStartAdapter: ControllerErrorCount exceeds threshold.
04/01/21-17:24:02 (GMT) (tRAID): ERROR: QLInitializeDevice: QLStartAdapter failed
04/01/21-17:24:02 (GMT) (tRAID): ERROR: QLAddDevice: controller/device/chip initialization failed.
04/01/21-17:24:02 (GMT) (tRAID): ERROR: qlgEnableHostInterface: QLInitializeDevice failed.
04/01/21-17:24:02 (GMT) (tRAID): NOTE: ********************************************************************************
04/01/21-17:24:02 (GMT) (tRAID): NOTE: QLogic Target Applicat
-=<###>=-
Attaching interface lo0... done

Adding 9768 symbols for standalone.
Error
04/01/21-17:24:16 (GMT) (tRootTask): NOTE: I2C transaction returned 0x0423fe00

WARNING: Reset by alternate controller

Current date: 04/01/21 time: 06:55:22

Send for Service Interface or baud rate change
04/01/21-17:24:33 (GMT) (tRAID): NOTE: SOD Sequence is Normal, 0
04/01/21-17:24:33 (GMT) (tRAID): NOTE: SOD: removed SAS host from index 0
04/01/21-17:24:33 (GMT) (tRAID): NOTE: In iscsiIOQLIscsiInitDq. iscsiIoFstrBase = 0x0
04/01/21-17:24:33 (GMT) (tRAID): NOTE: Turning on tray summary fault LED
04/01/21-17:24:35 (GMT) (tRAID): NOTE: SYMBOL: SYMbolAPI registered.
esmc0: LinkUp event
04/01/21-17:24:38 (GMT) (tRAID): NOTE: Initiating Drive channel: ioc:0 bringup
04/01/21-17:24:41 (GMT) (tRAID): NOTE: IOC Firmware Version: 00-24-63-00
04/01/21-17:24:42 (GMT) (tNetCfgInit): NOTE: Network Ready
04/01/21-17:24:50 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:0 phy:0 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:24:50 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:0 phy:1 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:24:51 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:2 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:24:51 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:3 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:24:53 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, FAILED to DEGRADED
04/01/21-17:24:53 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, DEGRADED to OPTIMAL
04/01/21-17:24:53 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:xa - WidePort: #0 - Alt Controller, chan:1 encl:0, subExp:255, FAILED to DEGRADED
04/01/21-17:24:53 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:xa - WidePort: #0 - Alt Controller, chan:1 encl:0, subExp:255, DEGRADED to OPTIMAL
04/01/21-17:24:53 (GMT) (tSasCfg033): NOTE: Alt Controller path up - chan:0 phy:18 itn:31
04/01/21-17:24:53 (GMT) (tSasCfg034): NOTE: Alt Controller path up - chan:1 phy:16 itn:32
04/01/21-17:25:00 (GMT) (tRAID): NOTE: IonMgr: Drive Interface Enabled
04/01/21-17:25:01 (GMT) (tRAID): NOTE: SOD: Instantiation Phase Complete
04/01/21-17:25:01 (GMT) (tRAID): NOTE: Inter-Controller Communication Channels Opened
04/01/21-17:25:01 (GMT) (tRAID): NOTE: LockMgr Role is Master
04/01/21-17:25:01 (GMT) (tRAID): WARN: FBM:validateSubModel: Exception - Alt controller not ready
04/01/21-17:25:01 (GMT) (tSasDiscCom): NOTE: SAS Discovery complete task spawned
04/01/21-17:25:01 (GMT) (tRAID): NOTE: spmEarlyData: Using cached data
04/01/21-17:25:01 (GMT) (sasCheckExpanderSet): NOTE: Expander Firmware Version: 0116-e05c
04/01/21-17:25:01 (GMT) (sasCheckExpanderSet): NOTE: Expander SAS address: Hi = x5842b2b4 Low = x51accd10
04/01/21-17:25:06 (GMT) (tSasDiscCom): WARN: SAS: Initial Discovery Complete Time: 30 seconds
04/01/21-17:25:08 (GMT) (IOSched): NOTE: New Initiator: 1 - channel: 1,devHandle: x2c, SAS Address: 5a4badb44e398500
04/01/21-17:25:08 (GMT) (IOSched): NOTE: New Initiator: 2 - channel: 0,devHandle: x2b, SAS Address: 5a4badb44e398501
04/01/21-17:25:08 (GMT) (iacTask1): NOTE: fbm:altValidateSubModelID: Sub-Model IDs Validated
04/01/21-17:25:13 (GMT) (tRAID): NOTE: WWN baseName 0004842b-2b51accd (valid==>SigMatch)
04/01/21-17:25:13 (GMT) (tRAID): NOTE: IonMgr: Host Interface Enabled
04/01/21-17:25:13 (GMT) (tRAID): NOTE: SOD: Pre-Initialization Phase Complete
04/01/21-17:25:29 (GMT) (tRAID): NOTE: ACS: autoCodeSync(): Process start. Comm Mode: 0, Status: 1
04/01/21-17:25:30 (GMT) (iacTask2): NOTE: ACS: Acs Needed on Alt: No, StateCode: 0, ReasonCode = 2
04/01/21-17:25:30 (GMT) (tRAID): NOTE: SOD: Code Synchronization Initialization Phase Complete
04/01/21-17:25:31 (GMT) (tRAID): NOTE: CheckInMonitor: Check-in failed (NoUnmarshalFuncException Error)
04/01/21-17:25:32 (GMT) (NvpsPersistentSyncM): NOTE: NVSRAM Persistent Storage updated successfully
04/01/21-17:25:33 (GMT) (tRAID): NOTE: USM Mgr initialization complete with 0 records.
04/01/21-17:25:33 (GMT) (tRAID): WARN: Received NoUnmarshalFuncException Error adding small edr records from alt controller
04/01/21-17:25:35 (GMT) (tRAID): WARN: spm: unable to exchange features, assuming none

04/01/21-17:25:36 (GMT) (tRAID): NOTE: DBRead 0.298 secs
04/01/21-17:25:36 (GMT) (tRAID): NOTE: sas: Peering Disabled (Alt in SOD)
04/01/21-17:25:36 (GMT) (iacTask5): NOTE: sas: checkInAlt Peering Enabled
04/01/21-17:25:38 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e7be0 4c0c8 bytes , result 0

^^ 04/01/21-17:26:05 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:26:05 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:26:06 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:26:06 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:26:06 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:26:06 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:26:06 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:26:06 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:26:07 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e7be0 4c0c8 bytes , result 0
04/01/21-17:26:34 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:26:34 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:26:35 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:26:35 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:26:35 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:26:35 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:26:35 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:26:35 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:26:36 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e7be0 4c0c8 bytes , result 0
04/01/21-17:26:38 (GMT) (tHckReset): WARN: Alt Ctl Reboot:
Reboot CompID: 0x408
Reboot reason: 0x6
Reboot reason extra: 0x1
04/01/21-17:26:38 (GMT) (tHckReset): NOTE: holding alt ctl in reset
04/01/21-17:26:39 (GMT) (tHckReset): NOTE: releasing alt ctl from reset
04/01/21-17:26:39 (GMT) (tHckReset): NOTE: sas: Peering Disabled (Health Check) Event: -4
04/01/21-17:26:39 (GMT) (IOSched): NOTE: Disconnect Initiator 1 - channel: 1, devHandle: x2c, I/O: 0, SAS Address: 5a4badb44e398500
04/01/21-17:26:39 (GMT) (IOSched): WARN: sasRemoveInitiator - No I/O's - send EventAck: event: 00000018, eventContext: 0002002c
04/01/21-17:26:39 (GMT) (sasTgtSendEventAck): WARN: sasTgtSendEventAckTask - SendEventAck for host: 1
04/01/21-17:26:39 (GMT) (tSasEvtWkr): NOTE: sasIocPhyDown: chan:1 phy:2 prevNumActivePhys:2 numActivePhys:1
04/01/21-17:26:39 (GMT) (tSasEvtWkr): NOTE: sasIocPhyDown: IOCPort: chan:1, OPTIMAL to DEGRADED
04/01/21-17:26:39 (GMT) (IOSched): NOTE: Disconnect Initiator 2 - channel: 0, devHandle: x2b, I/O: 0, SAS Address: 5a4badb44e398501
04/01/21-17:26:39 (GMT) (IOSched): WARN: sasRemoveInitiator - No I/O's - send EventAck: event: 00000018, eventContext: 0002002b
04/01/21-17:26:39 (GMT) (sasTgtSendEventAck): WARN: sasTgtSendEventAckTask - SendEventAck for host: 2
04/01/21-17:26:39 (GMT) (tSasEvtWkr): NOTE: sasIocPhyDown: chan:1 phy:3 prevNumActivePhys:1 numActivePhys:0
04/01/21-17:26:39 (GMT) (tSasEvtWkr): NOTE: sasIocPhyDown: IOCPort: chan:1, DEGRADED to FAILED
04/01/21-17:26:39 (GMT) (tSasEvtWkr): NOTE: sasRemoveExpanders: removing expanders on chan:1 level:0 and below
04/01/21-17:26:39 (GMT) (tSasEvtWkr): NOTE: sasLinkDown: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, OPTIMAL to DEGRADED
04/01/21-17:26:39 (GMT) (tSasEvtWkr): NOTE: sasLinkDown: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, DEGRADED to FAILED
04/01/21-17:26:55 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:2 prevNumActivePhys:0 numActivePhys:1
04/01/21-17:26:55 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: IOCPort: chan:1, FAILED to DEGRADED
04/01/21-17:26:55 (GMT) (tSasEvtWkr): WARN: sasIocPhyUp: Initializing Channel 1: Attached SAS Address: 5a4badb44e398510
04/01/21-17:26:57 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:3 prevNumActivePhys:1 numActivePhys:2
04/01/21-17:26:57 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: IOCPort: chan:1, DEGRADED to OPTIMAL
04/01/21-17:27:04 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:27:04 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:27:04 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:27:04 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:27:04 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:27:04 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:27:04 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:27:04 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:27:06 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e7be0 4c0c8 bytes , result 0
04/01/21-17:27:13 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, FAILED to DEGRADED
04/01/21-17:27:13 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, DEGRADED to OPTIMAL
04/01/21-17:27:13 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:xa - WidePort: #0 - Alt Controller, chan:1 encl:0, subExp:255, FAILED to DEGRADED
04/01/21-17:27:13 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:xa - WidePort: #0 - Alt Controller, chan:1 encl:0, subExp:255, DEGRADED to OPTIMAL
04/01/21-17:27:13 (GMT) (tSasCfg069): NOTE: Alt Controller path up - chan:0 phy:18 itn:31
04/01/21-17:27:13 (GMT) (tSasCfg070): NOTE: Alt Controller path up - chan:1 phy:16 itn:32
04/01/21-17:27:27 (GMT) (IOSched): NOTE: Connect Initiator: 1 - channel: 1,devHandle: x2c, SAS Address: 5a4badb44e398500
04/01/21-17:27:27 (GMT) (iconMgr2): WARN: sasNetworkManager::healthCheckEvent - Unknown Event (-3)
04/01/21-17:27:27 (GMT) (IOSched): NOTE: Connect Initiator: 2 - channel: 0,devHandle: x2b, SAS Address: 5a4badb44e398501
04/01/21-17:27:27 (GMT) (iconMgr2): NOTE: CheckInMonitor: Check-in failed (NoUnmarshalFuncException Error)
04/01/21-17:27:28 (GMT) (iacTask7): NOTE: fbm:altValidateSubModelID: Sub-Model IDs Validated
04/01/21-17:27:33 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:27:33 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:27:34 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:27:34 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:27:34 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:27:34 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:27:34 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:27:34 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:27:35 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e7be0 4c0c8 bytes , result 0
04/01/21-17:27:47 (GMT) (iacTask3): NOTE: ACS: Acs Needed on Alt: No, StateCode: 0, ReasonCode = 2
04/01/21-17:27:50 (GMT) (iacTask1): NOTE: sas: checkInAlt Peering Enabled
04/01/21-17:27:55 (GMT) (tRAID): NOTE: QLInitializeFW: Time 001/010 FwState 18
04/01/21-17:27:56 (GMT) (tRAID): NOTE: QLInitializeFW: Time 002/010 FwState 18
04/01/21-17:27:57 (GMT) (tRAID): NOTE: QLInitializeFW: Time 003/010 FwState 18
04/01/21-17:27:58 (GMT) (IOSched): NOTE: QLIsrDecodeMailbox: Port 0 Link up.
04/01/21-17:27:58 (GMT) (tRAID): NOTE: QLInitializeFW: Time 004/010 FwState 0
04/01/21-17:27:58 (GMT) (tRAID): NOTE: QLInitializeFW: port = 1, IPv4 Enable = 1, IPv6 Enable = 0
04/01/21-17:27:58 (GMT) (tRAID): NOTE: QLInitializeFW: IP Address 192.168.131.102:3260
04/01/21-17:27:58 (GMT) (tRAID): NOTE: QLInitializeFW: Firmware waiting for DHCP lease. State 18
04/01/21-17:27:58 (GMT) (tRAID): NOTE: QLInitializeFW: Time 000/010 FwState 18
04/01/21-17:27:59 (GMT) (IOSched): NOTE: QLIsrDecodeMailbox: Async Event Code 8002 received
04/01/21-17:27:59 (GMT) (IOSched): ERROR: QLDoInterruptServiceRoutine: PortFatal interrupt. PortFatalErrorStatus 00002000 CSR 0000d508 AS 2 AF 800001
04/01/21-17:27:59 (GMT) (IOSched): ERROR: QLDoInterruptServiceRoutine: Local RAM Parity Fatal Error occured
04/01/21-17:27:59 (GMT) (IOSched): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:27:59 (GMT) (IOSched): NOTE: QLIsrDecodeMailbox: Async Event Code 8002 received
04/01/21-17:27:59 (GMT) (IOSched): ERROR: QLDoInterruptServiceRoutine: PortFatal interrupt. PortFatalErrorStatus 00002000 CSR 0000d708 AS 2 AF 800009
04/01/21-17:27:59 (GMT) (IOSched): ERROR: QLDoInterruptServiceRoutine: Local RAM Parity Fatal Error occured
04/01/21-17:27:59 (GMT) (IOSched): NOTE: QLProcessSystemError: Restart RISC

 

MD3000i having both controller in boot loop after power failure. Can see logs by connecting to the consoles of both controller. If I am able to arrange the controller from used market, my questions are;

Is there a way to recover any controller without replacement?

what would be the procedure of replacement?

Will the data be intact?

Do i need to configure IPs etc after replacing the controller it will somehow get from somewhere?

Highly appreciate your help.

Below are the console logs for your reference from left controller.

-=<###>=-
Attaching interface lo0... done

Adding 9768 symbols for standalone.
Error
04/01/21-17:20:13 (GMT) (tRootTask): NOTE: I2C transaction returned 0x0423fe00

WARNING: Reset by alternate controller

Current date: 04/01/21 time: 06:51:19

Send for Service Interface or baud rate change
04/01/21-17:20:30 (GMT) (tRAID): NOTE: SOD Sequence is Normal, 0
04/01/21-17:20:30 (GMT) (tRAID): NOTE: SOD: removed SAS host from index 0
04/01/21-17:20:30 (GMT) (tRAID): NOTE: In iscsiIOQLIscsiInitDq. iscsiIoFstrBase = 0x0
04/01/21-17:20:30 (GMT) (tRAID): NOTE: Turning on tray summary fault LED
04/01/21-17:20:32 (GMT) (tRAID): NOTE: SYMBOL: SYMbolAPI registered.
esmc0: LinkUp event
04/01/21-17:20:35 (GMT) (tRAID): NOTE: Initiating Drive channel: ioc:0 bringup
04/01/21-17:20:38 (GMT) (tRAID): NOTE: IOC Firmware Version: 00-24-63-00
04/01/21-17:20:39 (GMT) (tNetCfgInit): NOTE: Network Ready
04/01/21-17:20:47 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:0 phy:0 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:20:47 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:0 phy:1 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:20:48 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:2 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:20:48 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:3 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:20:48 (GMT) (tSasCfg016): NOTE: Alt Controller path up - chan:0 phy:18 itn:1
04/01/21-17:20:48 (GMT) (tSasCfg021): NOTE: Alt Controller path up - chan:1 phy:16 itn:2
04/01/21-17:20:57 (GMT) (tRAID): NOTE: IonMgr: Drive Interface Enabled
04/01/21-17:20:58 (GMT) (tRAID): NOTE: SOD: Instantiation Phase Complete
04/01/21-17:20:58 (GMT) (tRAID): NOTE: Inter-Controller Communication Channels Opened
04/01/21-17:20:58 (GMT) (tSasDiscCom): NOTE: SAS Discovery complete task spawned
04/01/21-17:20:58 (GMT) (IOSched): NOTE: New Initiator: 1 - channel: 1,devHandle: x2b, SAS Address: 5a4badb44e398500
04/01/21-17:20:58 (GMT) (tRAID): NOTE: LockMgr Role is Slave
04/01/21-17:20:58 (GMT) (IOSched): NOTE: New Initiator: 2 - channel: 0,devHandle: x1a, SAS Address: 5a4badb44e398501
04/01/21-17:20:58 (GMT) (sasCheckExpanderSet): NOTE: Expander Firmware Version: 0116-e05c
04/01/21-17:20:58 (GMT) (sasCheckExpanderSet): NOTE: Expander SAS address: Hi = x5842b2b4 Low = x51accd10
04/01/21-17:20:59 (GMT) (tRAID): NOTE: spmEarlyData: Using cached data
04/01/21-17:21:03 (GMT) (tSasDiscCom): WARN: SAS: Initial Discovery Complete Time: 30 seconds
04/01/21-17:21:03 (GMT) (tRAID): NOTE: WWN baseName 0004842b-2b51accd (valid==>SigMatch)
04/01/21-17:21:03 (GMT) (tRAID): NOTE: ionEnableHostInterfaces is waiting for a channel to become ready
04/01/21-17:21:03 (GMT) (tRAID): NOTE: ionEnableHostInterfaces waited 800ms for a channel to become ready
04/01/21-17:21:03 (GMT) (tRAID): NOTE: IonMgr: Host Interface Enabled
04/01/21-17:21:03 (GMT) (tRAID): NOTE: SOD: Pre-Initialization Phase Complete
04/01/21-17:21:26 (GMT) (iacTask7): NOTE: ACS: Acs Needed on Alt: No, StateCode: 0, ReasonCode = 2
04/01/21-17:21:26 (GMT) (tRAID): NOTE: ACS: autoCodeSync(): Process start. Comm Mode: 0, Status: 1
04/01/21-17:21:26 (GMT) (tRAID): NOTE: SOD: Code Synchronization Initialization Phase Complete
04/01/21-17:21:29 (GMT) (NvpsPersistentSyncM): NOTE: NVSRAM Persistent Storage updated successfully
04/01/21-17:21:29 (GMT) (tRAID): NOTE: USM Mgr initialization complete with 0 records.
04/01/21-17:21:30 (GMT) (tRAID): NOTE: EDR - recieved 1 small records
04/01/21-17:21:30 (GMT) (tRAID): NOTE: EDR - recieved 0 large records
04/01/21-17:21:32 (GMT) (tRAID): NOTE: Acquire 0.020 secs
04/01/21-17:21:35 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e4340 4c0c8 bytes , result 0
04/01/21-17:22:03 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:22:03 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:22:03 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:22:03 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:22:03 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:22:03 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:22:03 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:22:03 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:22:05 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e4340 4c0c8 bytes , result 0
04/01/21-17:22:07 (GMT) (tHckReset): WARN: Alt Ctl Reboot:
Reboot CompID: 0x408
Reboot reason: 0x6
Reboot reason extra: 0x1
04/01/21-17:22:07 (GMT) (tHckReset): NOTE: holding alt ctl in reset
04/01/21-17:22:07 (GMT) (tHckReset): NOTE: releasing alt ctl from reset
04/01/21-17:22:07 (GMT) (tHckReset): NOTE: LockMgr Role is Master
04/01/21-17:22:07 (GMT) (tHckReset): NOTE: sas: Peering Disabled (Health Check) Event: -4
04/01/21-17:22:07 (GMT) (IOSched): NOTE: Disconnect Initiator 1 - channel: 1, devHandle: x2b, I/O: 0, SAS Address: 5a4badb44e398500
04/01/21-17:22:07 (GMT) (IOSched): WARN: sasRemoveInitiator - No I/O's - send EventAck: event: 00000018, eventContext: 0002002b
04/01/21-17:22:07 (GMT) (sasTgtSendEventAck): WARN: sasTgtSendEventAckTask - SendEventAck for host: 1
04/01/21-17:22:07 (GMT) (IOSched): NOTE: Disconnect Initiator 2 - channel: 0, devHandle: x1a, I/O: 0, SAS Address: 5a4badb44e398501
04/01/21-17:22:07 (GMT) (IOSched): WARN: sasRemoveInitiator - No I/O's - send EventAck: event: 00000018, eventContext: 0002001a
04/01/21-17:22:07 (GMT) (sasTgtSendEventAck): WARN: sasTgtSendEventAckTask - SendEventAck for host: 2
04/01/21-17:22:07 (GMT) (tSasEvtWkr): NOTE: sasIocPhyDown: chan:1 phy:2 prevNumActivePhys:2 numActivePhys:1
04/01/21-17:22:07 (GMT) (tSasEvtWkr): NOTE: sasIocPhyDown: IOCPort: chan:1, OPTIMAL to DEGRADED
04/01/21-17:22:08 (GMT) (tSasEvtWkr): NOTE: sasIocPhyDown: chan:1 phy:3 prevNumActivePhys:1 numActivePhys:0
04/01/21-17:22:08 (GMT) (tSasEvtWkr): NOTE: sasIocPhyDown: IOCPort: chan:1, DEGRADED to FAILED
04/01/21-17:22:08 (GMT) (tSasEvtWkr): NOTE: sasRemoveExpanders: removing expanders on chan:1 level:0 and below
04/01/21-17:22:08 (GMT) (tSasEvtWkr): NOTE: sasLinkDown: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, OPTIMAL to DEGRADED
04/01/21-17:22:08 (GMT) (tSasEvtWkr): NOTE: sasLinkDown: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, DEGRADED to FAILED
04/01/21-17:22:24 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:2 prevNumActivePhys:0 numActivePhys:1
04/01/21-17:22:24 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: IOCPort: chan:1, FAILED to DEGRADED
04/01/21-17:22:24 (GMT) (tSasEvtWkr): WARN: sasIocPhyUp: Initializing Channel 1: Attached SAS Address: 5a4badb44e398510
04/01/21-17:22:26 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:3 prevNumActivePhys:1 numActivePhys:2
04/01/21-17:22:26 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: IOCPort: chan:1, DEGRADED to OPTIMAL
04/01/21-17:22:32 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:22:32 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:22:33 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:22:33 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:22:33 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:22:33 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:22:33 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:22:33 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:22:34 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e4340 4c0c8 bytes , result 0
04/01/21-17:22:41 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, FAILED to DEGRADED
04/01/21-17:22:41 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:x9 - WidePort: #1 - Alt Controller, chan:0 encl:0, subExp:255, DEGRADED to OPTIMAL
04/01/21-17:22:41 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:xa - WidePort: #0 - Alt Controller, chan:1 encl:0, subExp:255, FAILED to DEGRADED
04/01/21-17:22:41 (GMT) (tSasEvtWkr): NOTE: sasLinkUp: expDevHandle:xa - WidePort: #0 - Alt Controller, chan:1 encl:0, subExp:255, DEGRADED to OPTIMAL
04/01/21-17:22:41 (GMT) (tSasCfg069): NOTE: Alt Controller path up - chan:0 phy:18 itn:1
04/01/21-17:22:41 (GMT) (tSasCfg070): NOTE: Alt Controller path up - chan:1 phy:16 itn:2
04/01/21-17:22:56 (GMT) (IOSched): NOTE: Connect Initiator: 1 - channel: 1,devHandle: x2c, SAS Address: 5a4badb44e398500
04/01/21-17:22:56 (GMT) (iconMgr4): WARN: sasNetworkManager::healthCheckEvent - Unknown Event (-3)
04/01/21-17:22:56 (GMT) (IOSched): NOTE: Connect Initiator: 2 - channel: 0,devHandle: x2b, SAS Address: 5a4badb44e398501
04/01/21-17:22:56 (GMT) (iconMgr4): NOTE: CheckInMonitor: Check-in failed (NoUnmarshalFuncException Error)
04/01/21-17:22:56 (GMT) (iacTask6): NOTE: fbm:altValidateSubModelID: Sub-Model IDs Validated
04/01/21-17:23:01 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:23:01 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:23:02 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:23:02 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:23:02 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:23:02 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:23:02 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:23:02 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:23:03 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e4340 4c0c8 bytes , result 0
04/01/21-17:23:16 (GMT) (iacTask7): NOTE: ACS: Acs Needed on Alt: No, StateCode: 0, ReasonCode = 2
04/01/21-17:23:19 (GMT) (iacTask7): NOTE: sas: checkInAlt Peering Enabled
04/01/21-17:23:30 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:23:30 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:23:31 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:23:31 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:23:31 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:23:31 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:23:31 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:23:31 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:23:33 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e4340 4c0c8 bytes , result 0
04/01/21-17:24:00 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:24:00 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:24:01 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:24:01 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:24:01 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:24:01 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:24:01 (GMT) (tRAID): NOTE: QLRebootTimer: Status after Get FW State 4543
04/01/21-17:24:01 (GMT) (tRAID): NOTE: QLRebootTimer: QLGetFwState failed
04/01/21-17:24:02 (GMT) (tRAID): WARN: QLStartAdapter: ControllerErrorCount exceeds threshold.
04/01/21-17:24:02 (GMT) (tRAID): ERROR: QLInitializeDevice: QLStartAdapter failed
04/01/21-17:24:02 (GMT) (tRAID): ERROR: QLAddDevice: controller/device/chip initialization failed.
04/01/21-17:24:02 (GMT) (tRAID): ERROR: qlgEnableHostInterface: QLInitializeDevice failed.
04/01/21-17:24:02 (GMT) (tRAID): NOTE: ********************************************************************************
04/01/21-17:24:02 (GMT) (tRAID): NOTE: QLogic Target Applicat
-=<###>=-
Attaching interface lo0... done

Adding 9768 symbols for standalone.
Error
04/01/21-17:24:16 (GMT) (tRootTask): NOTE: I2C transaction returned 0x0423fe00

WARNING: Reset by alternate controller

Current date: 04/01/21 time: 06:55:22

 

------------- Console LOGs from Right Controller -------------------

-=<###>=-
Attaching interface lo0... done

Adding 9768 symbols for standalone.
Error
04/01/21-17:28:31 (GMT) (tRootTask): NOTE: I2C transaction returned 0x0423fe00

Current date: 04/01/21 time: 07:59:20

Send for Service Interface or baud rate change
04/01/21-17:28:48 (GMT) (tRAID): NOTE: SOD Sequence is Normal, 0
04/01/21-17:28:48 (GMT) (tRAID): NOTE: SOD: removed SAS host from index 0
04/01/21-17:28:48 (GMT) (tRAID): NOTE: In iscsiIOQLIscsiInitDq. iscsiIoFstrBase = 0x0
04/01/21-17:28:48 (GMT) (tRAID): NOTE: Turning on tray summary fault LED
04/01/21-17:28:50 (GMT) (tRAID): NOTE: SYMBOL: SYMbolAPI registered.
esmc0: Link change detected, LinkDown may take a long time to detect
0x36d600 (tNetTask): esmc0: LinkUp event
04/01/21-17:28:53 (GMT) (tRAID): NOTE: Initiating Drive channel: ioc:0 bringup
04/01/21-17:28:55 (GMT) (tNetCfgInit): NOTE: Network Ready
04/01/21-17:28:56 (GMT) (tRAID): NOTE: IOC Firmware Version: 00-24-63-00
04/01/21-17:29:05 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:0 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:29:05 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:1 phy:1 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:29:06 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:0 phy:2 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:29:06 (GMT) (tSasEvtWkr): NOTE: sasIocPhyUp: chan:0 phy:3 prevNumActivePhys:2 numActivePhys:2
04/01/21-17:29:06 (GMT) (tSasCfg016): NOTE: Alt Controller path up - chan:1 phy:18 itn:1
04/01/21-17:29:06 (GMT) (tSasCfg021): NOTE: Alt Controller path up - chan:0 phy:16 itn:2
04/01/21-17:29:15 (GMT) (tRAID): NOTE: IonMgr: Drive Interface Enabled
04/01/21-17:29:16 (GMT) (tRAID): NOTE: SOD: Instantiation Phase Complete
04/01/21-17:29:16 (GMT) (tRAID): NOTE: Inter-Controller Communication Channels Opened
04/01/21-17:29:16 (GMT) (tSasDiscCom): NOTE: SAS Discovery complete task spawned
04/01/21-17:29:16 (GMT) (IOSched): NOTE: New Initiator: 1 - channel: 0,devHandle: x2b, SAS Address: 5842b2b451accd00
04/01/21-17:29:16 (GMT) (IOSched): NOTE: New Initiator: 2 - channel: 1,devHandle: x1a, SAS Address: 5842b2b451accd01
04/01/21-17:29:16 (GMT) (tRAID): NOTE: LockMgr Role is Slave
04/01/21-17:29:16 (GMT) (sasCheckExpanderSet): NOTE: Expander Firmware Version: 0116-e05c
04/01/21-17:29:16 (GMT) (sasCheckExpanderSet): NOTE: Expander SAS address: Hi = x5a4badb4 Low = x4e398510
04/01/21-17:29:16 (GMT) (tRAID): NOTE: spmEarlyData: Using cached data
04/01/21-17:29:20 (GMT) (tSasDiscCom): WARN: SAS: Initial Discovery Complete Time: 30 seconds
04/01/21-17:29:20 (GMT) (tRAID): NOTE: WWN baseName 0004842b-2b51accd (valid==>SoftRst)
04/01/21-17:29:20 (GMT) (tRAID): NOTE: ionEnableHostInterfaces is waiting for a channel to become ready
04/01/21-17:29:21 (GMT) (tRAID): NOTE: ionEnableHostInterfaces waited 1800ms for a channel to become ready
04/01/21-17:29:21 (GMT) (tRAID): NOTE: IonMgr: Host Interface Enabled
04/01/21-17:29:21 (GMT) (tRAID): NOTE: SOD: Pre-Initialization Phase Complete

04/01/21-17:29:36 (GMT) (iacTask1): NOTE: ACS: Acs Needed on Alt: No, StateCode: 0, ReasonCode = 2
04/01/21-17:29:36 (GMT) (tRAID): NOTE: ACS: autoCodeSync(): Process start. Comm Mode: 0, Status: 1
04/01/21-17:29:36 (GMT) (tRAID): NOTE: SOD: Code Synchronization Initialization Phase Complete
04/01/21-17:29:38 (GMT) (NvpsPersistentSyncM): NOTE: NVSRAM Persistent Storage updated successfully
04/01/21-17:29:39 (GMT) (tRAID): NOTE: USM Mgr initialization complete with 0 records.
04/01/21-17:29:39 (GMT) (tRAID): NOTE: EDR - recieved 1 small records
04/01/21-17:29:39 (GMT) (tRAID): NOTE: EDR - recieved 0 large records
04/01/21-17:29:41 (GMT) (tRAID): NOTE: Acquire 0.020 secs
04/01/21-17:29:41 (GMT) (tRAID): WARN: spmEarlyDataMgr: No unused entries for target
04/01/21-17:29:45 (GMT) (tRAID): NOTE: QLStartFw: Downloading Driver's FW image 03.00.01.47 from 032e5640 4c0c8 bytes , result 0
04/01/21-17:29:47 (GMT) (tRAID): NOTE: ********************************************************************************
04/01/21-17:29:47 (GMT) (tRAID): NOTE: QLogic Target Application, Version 2.01.08 6-13-2005 (W2K)
04/01/21-17:29:47 (GMT) (tRAID): NOTE: iSCSI Target Application
04/01/21-17:29:47 (GMT) (tRAID): NOTE: ********************************************************************************
04/01/21-17:29:48 (GMT) (tRAID): NOTE: QLInitializeFW: iSNS Server 0.0.0.0:3205
04/01/21-17:29:48 (GMT) (tRAID): NOTE: QLInitializeFW: ISNSServerIPv6Addr 00:00:00:00:00:00:00:00 :3205
04/01/21-17:29:48 (GMT) (tRAID): NOTE: QLInitializeFW: iSCSI Name iqn.1984-05.com.dell:powervault.md3000i.60026b900057a287000000004b49f5b6
04/01/21-17:29:48 (GMT) (tRAID): NOTE: QLInitializeFW: port = 0, IPv4 Enable = 1, IPv6 Enable = 0
04/01/21-17:29:48 (GMT) (tRAID): NOTE: QLInitializeFW: IP Address 192.168.130.102:3260
04/01/21-17:29:48 (GMT) (tRAID): NOTE: QLInitializeFW: Firmware waiting for DHCP lease. State 18
04/01/21-17:29:48 (GMT) (tRAID): NOTE: QLInitializeFW: Time 000/010 FwState 18
04/01/21-17:29:49 (GMT) (tRAID): NOTE: QLInitializeFW: Time 001/010 FwState 18
04/01/21-17:29:50 (GMT) (tRAID): NOTE: QLInitializeFW: Time 002/010 FwState 18
04/01/21-17:29:51 (GMT) (IOSched): NOTE: QLIsrDecodeMailbox: Port 0 Link up.
04/01/21-17:29:51 (GMT) (IOSched): NOTE: QLIsrDecodeMailbox: Async Event Code 8002 received
04/01/21-17:29:51 (GMT) (IOSched): ERROR: QLDoInterruptServiceRoutine: PortFatal interrupt. PortFatalErrorStatus 00002000 CSR 0000c508 AS 2 AF 800001
04/01/21-17:29:51 (GMT) (IOSched): ERROR: QLDoInterruptServiceRoutine: Local RAM Parity Fatal Error occured
04/01/21-17:29:51 (GMT) (IOSched): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:30:16 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0069, completion timeout
04/01/21-17:30:16 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x69
04/01/21-17:30:17 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:30:17 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:30:17 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:30:17 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE failed. Stat f000
04/01/21-17:30:17 (GMT) (tRAID): WARN: QLInitializeFW: QLGetFwState failed.
04/01/21-17:30:17 (GMT) (tRAID): NOTE: QLInitializeAdapter: QLInitializeFW failed
04/01/21-17:30:17 (GMT) (tRAID): ERROR: QLEnable: Enable lun error
04/01/21-17:30:42 (GMT) (tRAID): WARN: QLMailboxCommand: Cmd = 0026, completion timeout
04/01/21-17:30:42 (GMT) (tRAID): WARN: QLMailboxCommand: command completion timeout, cmd = 0x26
04/01/21-17:30:43 (GMT) (tRAID): NOTE: Qlogic coredump file written to '3S91VL1:/tmp/QLogic_Coredump_port_0_3S91VL1',rc 204E50, expected 204E50
04/01/21-17:30:43 (GMT) (tRAID): WARN: Qlogic coredump file write failed.fclose returned -1

04/01/21-17:30:43 (GMT) (tRAID): NOTE: QLProcessSystemError: Restart RISC
04/01/21-17:30:43 (GMT) (tRAID): NOTE: QLInitializeAdapter: MBOX_CMD_GET_FLASH f000. Unable to check MAC
04/01/21-17:30:43 (GMT) (tRAID): ERROR: QLEnable: Enable lun error

Exception: Reset
cpsr: 60000013 (Unknown Program Counter)

Registers:
r0 = 0 r1 = 35de3a8 r2 = 35de3a8 r3 = 0
r4 = 1bedde2 r5 = 1 r6 = 38b8388 r7 = 0
04/01/21-17:30:43 (GMT) (t5): WARN: QLUtmEventNotify: pDevExt 3294570 port 1 Event code 8002 pUtmTaGetTeb is null.
r8 = 400 r9 = 400 r10 = 32d117c r11/fp = 1c23ba0
r12/ip = 1 r13/sp = 1c23b64 r14/lr = 58b49c pc = 0
cpsr = 60000013

Stack Trace:

======== STACK SHOW ========
Showing for task id = 0x1c24080 (tRAID), Running
FP=0x1c23ba0, SP=0x1c23b64, PC=0x0
Current executing task id = 0x1c24080 (tRAID); not interrupted

Frame Ptr Ret Addr Return Name + Offset Called Name + Offset
========== ========== ================================ ========================
0x1c24050 0x0019f9c0 vxTaskEntry + 0x14 [fuzzy]
0x1c24048 0x0019f9c0 vxTaskEntry + 0x14 sodMain
0x1c23fd4 0x0061e528 sodMain + 0x1c8 _Z17sodInitializationv
0x1c23fc4 0x0061d558 _Z17sodInitializationv + 0x18 _Z32sodInitializeApplicationServicesv
0x1c23fb4 0x0061d2f8 _Z32sodInitializeApplicationServicesv + 0xb8 _Z13sodLogStartupPFvvE
0x1c23e58 0x0061ce30 _Z13sodLogStartupPFvvE + 0xb0 _ZN3ion10initializeEv
0x1c23df4 0x00ae005c _ZN3ion10initializeEv + 0x7c _ZN3ion10IonManager10initializeEv
0x1c23d74 0x00aaeb98 _ZN3ion10IonManager10initializeEv + 0x438 _ZN5b_isn19IscsiNetworkManager10initializeEv
0x1c23bf4 0x0051fd9c _ZN5b_isn19IscsiNetworkManager10initializeEv + 0x4fc QLTA_Main
0x1c23ba8 0x00501fd8 QLTA_Main + 0x238 QLBM_RegisterImmDataBufs
0x1c23b94 0x0058b730 QLBM_RegisterImmDataBufs + 0x30 QLBM_Register4032ImmDataBufs

Note: At least one "[fuzzy]" is indicated. A fuzzy frame entry is not a true
stack frame; rather, an address within VxWorks code space was found in the
stack, but it may not be a legitimate entry in the call list (or it may be).
Error in task 0x1c24080: Bad stack pointer (sp=0x1c24f4c)

********
Task Id: 0x1c24080
Name: "tRAID"
Status: 0x00 (ready)
Options: 0x9001 (suprvsr)
Priority: 125
Stack base: 0x1c24080
Stack end: 0x1c1f080
Stack size: 0x5000 (20480)
Stack margin: 0x3264 (12900)
Stack limit: 0x1c1f080
Pend queue: 0x2e5c70
Last errno: 0x860002

-=<###>=-
Attaching interface lo0... done

Adding 9768 symbols for standalone.
Error
04/01/21-17:30:48 (GMT) (tRootTask): NOTE: I2C transaction returned 0x0423fe00

WARNING: Reset by alternate controller

 

 

Hello,

 

You'll need to replace the controllers. Since both controllers are having problems, there would be nowhere for them to pull configuration data from, you'd need to configure them as though it were a new deployment. The data should be ok, provided the controller didn't write corrupt the array. I would expect that your data is likely fine, but I can't say that as an absolute fact at this point. My recommendation would be to go ahead and proceed with hardware replacement at this time, because it will be needed regardless of the data's current state.

Thank You so much for your reply @Dell-DylanJ I really appreciate your quick response. 

Two quick questions. 

Can I remove both faulty controllers and arrange and install only one the controller, just for the sake of data recovery. or I have to replace both? 

Any safety tips for the installation of replacement-controller?

I hope I will get the replacement from eBay.

Regards,