Good Morning,

I am having a issue communication with the controller on module in slot 1.

Storage array had been reporting two errors. [1] virtual disk not on preferred path. - disk group owned by slot 1 was currently owned by slot 0; and [2]  that the battery was bad on module slot 1.

The Raid module was taken offline and new battery installed. The raid was reinserted and we can ping the nic, however the iSCSI channels are not communicating with the MDSM.

It appears that the controller is cycling repeatedly attempting to initialize, as all the virtual servers came to almost a halt and the vCenter software reported over a 4000 millisecond latency on IOs across the board. It seems to me that the card cannot pull the configuration, but am at a loss how to correct.

The only way to keep the array functional at this time is to keep slot 1 offline. Storage is not my area of expertise by a long shot and would greatly appreciate if someone could review the log below and offer some guidance. Thanks in advance for any help provided. Log as follows:

09/05/14-21:32:30 (GMT) (tNetCfgInit): NOTE:  Network Ready
0x36d600 (tNetTask): esmc0: LinkUp event
09/05/14-21:32:30 (GMT) (tRAID): NOTE:  SYMBOL: SYMbolAPI registered.
09/05/14-21:32:33 (GMT) (tRAID): NOTE:  Initiating Drive channel: ioc:0 bringup
09/05/14-21:32:36 (GMT) (tRAID): NOTE:  IOC Firmware Version: 00-24-63-00
09/05/14-21:32:45 (GMT) (tSasEvtWkr): NOTE:  sasIocPhyUp: chan:1 phy:0 prevNumAc
tivePhys:2 numActivePhys:2
09/05/14-21:32:45 (GMT) (tSasEvtWkr): NOTE:  sasIocPhyUp: chan:1 phy:1 prevNumAc
tivePhys:2 numActivePhys:2
09/05/14-21:32:46 (GMT) (tSasEvtWkr): NOTE:  sasIocPhyUp: chan:0 phy:2 prevNumAc
tivePhys:2 numActivePhys:2
09/05/14-21:32:46 (GMT) (tSasEvtWkr): NOTE:  sasIocPhyUp: chan:0 phy:3 prevNumAc
tivePhys:2 numActivePhys:2
09/05/14-21:32:46 (GMT) (tSasCfg018): NOTE:  Alt Controller path up - chan:1 phy
:18 itn:1
09/05/14-21:32:46 (GMT) (tSasCfg021): NOTE:  Alt Controller path up - chan:0 phy
:16 itn:2
09/05/14-21:32:46 (GMT) (tRAID): NOTE:  IonMgr: Drive Interface Enabled
09/05/14-21:32:46 (GMT) (tRAID): NOTE:  SOD: Instantiation Phase Complete
09/05/14-21:32:47 (GMT) (tRAID): NOTE:  Inter-Controller Communication Channels
Opened
09/05/14-21:32:47 (GMT) (IOSched): NOTE:  New Initiator:  1 - channel: 0,devHand
le: x2a, SAS Address: 50026b9432cd7f00
09/05/14-21:32:47 (GMT) (tSasDiscCom): NOTE:  SAS Discovery complete task spawne
d
09/05/14-21:32:47 (GMT) (tRAID): NOTE:  LockMgr Role is Slave
09/05/14-21:32:47 (GMT) (IOSched): NOTE:  New Initiator:  2 - channel: 1,devHand
le: x19, SAS Address: 50026b9432cd7f01
09/05/14-21:32:47 (GMT) (sasCheckExpanderSet): NOTE:  Expander Firmware Version:
 0116-e05c
09/05/14-21:32:47 (GMT) (sasCheckExpanderSet): NOTE:  Expander SAS address: Hi =
 x50026b94 Low = x32cd8910
09/05/14-21:32:49 (GMT) (tRAID): NOTE:  spmEarlyData: Using cached data
09/05/14-21:33:00 (GMT) (tSasDiscCom): WARN:  SAS: Initial Discovery Complete Ti
me: 30 seconds
09/05/14-21:33:00 (GMT) (tRAID): NOTE:  WWN baseName 00040026-b932cd7f (valid==>
SigMatch)
09/05/14-21:33:00 (GMT) (tRAID): NOTE:  IonMgr: Host Interface Enabled
09/05/14-21:33:00 (GMT) (tRAID): NOTE:  SOD: Pre-Initialization Phase Complete
09/05/14-21:33:14 (GMT) (tRAID): NOTE:  ACS: autoCodeSync(): Process start. Comm
 Mode: 0, Status: 1
09/05/14-21:33:14 (GMT) (tRAID): NOTE:  SOD: Code Synchronization Initialization
 Phase Complete
09/05/14-21:33:15 (GMT) (NvpsPersistentSyncM): NOTE:  NVSRAM Persistent Storage
updated successfully
09/05/14-21:33:15 (GMT) (tRAID): NOTE:  USM Mgr initialization complete with 0 r
ecords.
09/05/14-21:33:15 (GMT) (tRAID): NOTE:  EDR - recieved 1 small records
09/05/14-21:33:15 (GMT) (tRAID): NOTE:  EDR - recieved 0 large records
09/05/14-21:33:16 (GMT) (tRAID): NOTE:  Acquire              0.017 secs
09/05/14-21:33:18 (GMT) (tRAID): NOTE:  QLStartFw: Downloading Driver's FW image
 03.00.01.47 from 032110a0 4c0c8 bytes , result 0
09/05/14-21:33:45 (GMT) (tRAID): WARN:  QLMailboxCommand: Cmd = 0069, completion
 timeout
09/05/14-21:33:45 (GMT) (tRAID): WARN:  QLMailboxCommand: command completion tim
eout, cmd = 0x69
09/05/14-21:33:46 (GMT) (tRAID): NOTE:  Qlogic coredump file written to 'CFSKZK1
:/tmp/QLogic_Coredump_port_0_CFSKZK1',rc 204E50, expected 204E50
09/05/14-21:33:46 (GMT) (tRAID): WARN:  Qlogic coredump file write failed.fclose
 returned -1

09/05/14-21:33:46 (GMT) (tRAID): NOTE:  QLProcessSystemError: Restart RISC
09/05/14-21:33:46 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE fail
ed.  Stat f000
09/05/14-21:33:46 (GMT) (tRAID): NOTE:  QLRebootTimer: Status after Get FW State
 4543
09/05/14-21:33:46 (GMT) (tRAID): NOTE:  QLRebootTimer: QLGetFwState failed
09/05/14-21:33:47 (GMT) (tRAID): NOTE:  QLStartFw: Downloading Driver's FW image
 03.00.01.47 from 032110a0 4c0c8 bytes , result 0
09/05/14-21:34:14 (GMT) (tRAID): WARN:  QLMailboxCommand: Cmd = 0069, completion
 timeout
09/05/14-21:34:14 (GMT) (tRAID): WARN:  QLMailboxCommand: command completion tim
eout, cmd = 0x69
09/05/14-21:34:15 (GMT) (tRAID): NOTE:  Qlogic coredump file written to 'CFSKZK1
:/tmp/QLogic_Coredump_port_0_CFSKZK1',rc 204E50, expected 204E50
09/05/14-21:34:15 (GMT) (tRAID): WARN:  Qlogic coredump file write failed.fclose
 returned -1

09/05/14-21:34:15 (GMT) (tRAID): NOTE:  QLProcessSystemError: Restart RISC
09/05/14-21:34:15 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE fail
ed.  Stat f000
09/05/14-21:34:15 (GMT) (tRAID): NOTE:  QLRebootTimer: Status after Get FW State
 4543
09/05/14-21:34:15 (GMT) (tRAID): NOTE:  QLRebootTimer: QLGetFwState failed
09/05/14-21:34:17 (GMT) (tRAID): NOTE:  QLStartFw: Downloading Driver's FW image
 03.00.01.47 from 032110a0 4c0c8 bytes , result 0
09/05/14-21:34:44 (GMT) (tRAID): WARN:  QLMailboxCommand: Cmd = 0069, completion
 timeout
09/05/14-21:34:44 (GMT) (tRAID): WARN:  QLMailboxCommand: command completion tim
eout, cmd = 0x69
09/05/14-21:34:45 (GMT) (tRAID): NOTE:  Qlogic coredump file written to 'CFSKZK1
:/tmp/QLogic_Coredump_port_0_CFSKZK1',rc 204E50, expected 204E50
09/05/14-21:34:45 (GMT) (tRAID): WARN:  Qlogic coredump file write failed.fclose
 returned -1

09/05/14-21:34:45 (GMT) (tRAID): NOTE:  QLProcessSystemError: Restart RISC
09/05/14-21:34:45 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE fail
ed.  Stat f000
09/05/14-21:34:45 (GMT) (tRAID): NOTE:  QLRebootTimer: Status after Get FW State
 4543
09/05/14-21:34:45 (GMT) (tRAID): NOTE:  QLRebootTimer: QLGetFwState failed
09/05/14-21:34:46 (GMT) (tRAID): NOTE:  QLStartFw: Downloading Driver's FW image
 03.00.01.47 from 032110a0 4c0c8 bytes , result 0
09/05/14-21:35:13 (GMT) (tRAID): WARN:  QLMailboxCommand: Cmd = 0069, completion
 timeout
09/05/14-21:35:13 (GMT) (tRAID): WARN:  QLMailboxCommand: command completion tim
eout, cmd = 0x69
09/05/14-21:35:14 (GMT) (tRAID): NOTE:  Qlogic coredump file written to 'CFSKZK1
:/tmp/QLogic_Coredump_port_0_CFSKZK1',rc 204E50, expected 204E50
09/05/14-21:35:14 (GMT) (tRAID): WARN:  Qlogic coredump file write failed.fclose
 returned -1

09/05/14-21:35:14 (GMT) (tRAID): NOTE:  QLProcessSystemError: Restart RISC
09/05/14-21:35:14 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE fail
ed.  Stat f000
09/05/14-21:35:14 (GMT) (tRAID): NOTE:  QLRebootTimer: Status after Get FW State
 4543
09/05/14-21:35:14 (GMT) (tRAID): NOTE:  QLRebootTimer: QLGetFwState failed
09/05/14-21:35:15 (GMT) (tRAID): NOTE:  QLStartFw: Downloading Driver's FW image
 03.00.01.47 from 032110a0 4c0c8 bytes , result 0
09/05/14-21:35:42 (GMT) (tRAID): WARN:  QLMailboxCommand: Cmd = 0069, completion
 timeout
09/05/14-21:35:42 (GMT) (tRAID): WARN:  QLMailboxCommand: command completion tim
eout, cmd = 0x69
09/05/14-21:35:43 (GMT) (tRAID): NOTE:  Qlogic coredump file written to 'CFSKZK1
:/tmp/QLogic_Coredump_port_0_CFSKZK1',rc 204E50, expected 204E50
09/05/14-21:35:43 (GMT) (tRAID): WARN:  Qlogic coredump file write failed.fclose
 returned -1

09/05/14-21:35:43 (GMT) (tRAID): NOTE:  QLProcessSystemError: Restart RISC
09/05/14-21:35:43 (GMT) (tRAID): ERROR: QLGetFwState: MBOX_CMD_GET_FW_STATE fail
ed.  Stat f000
09/05/14-21:35:43 (GMT) (tRAID): NOTE:  QLRebootTimer: Status after Get FW State
 4543
09/05/14-21:35:43 (GMT) (tRAID): NOTE:  QLRebootTimer: QLGetFwState failed
09/05/14-21:35:44 (GMT) (tRAID): WARN:  QLStartAdapter: ControllerErrorCount exc
eeds threshold.
09ÂÿError
09/05/14-21:35:58 (GMT) (tRootTask): NOTE:  I2C transaction returned 0x0423fe00

WARNING: Reset by alternate controller

Controller resets and above repeats over and over again.