Hello, all!
There was a problem with the disk array. It consists of two CommVault MD3200i and three PowerVault md1220. The disk array is connected to the ESXi server as four datastores. At some point, three datastores stopped working.
Nobody knows the password for access Service Interface. The system was set up long before I got a job here.
Here is the download log:
System startup messages:
========================
ERROR 85.01: Ethernet MAC address is not set
10/05/22-02:08:48 (tNetCfgInit): NOTE: Network Ready
10/05/22-02:08:48 (tSystem): ERROR: FPGA FW is out of date
"Rhone03 rev17" currently in use
"Rhone03 rev20" available for update
Current date: 10/05/22 time: 02:09:12
Send <BREAK> for Service Interface or baud rate change
10/05/22-02:09:12 (tRAID): SOD Sequence is Normal, 0 on controller A
10/05/22-02:09:12 (tRAID): WARN: SOD change to normal power mode - rebooting...
{6)672+#^+Pm)e>2 (:~:)erlv?
Reset, Power-Up Diagnostics - Loop 1 of 1
3600 Processor DRAM
01 Data lines Passed
02 Address lines Passed
3300 NVSRAM
01 Data lines Passed
4410 Ethernet 82574 1
01 Register read Passed
02 Register address lines Passed
6D40 Bobcat
02 Flash Test Passed
3700 PLB SRAM
01 Data lines Passed
02 Address lines Passed
7000 SE iSCSI BE2 1
01 Register Read Test Passed
02 Register Address Lines Test Passed
03 Register Data Lines Test Passed
3900 Real-Time Clock
01 RT Clock Tick Passed
Diagnostic Manager exited normally.
10/05/22-02:09:23 (tSystem): ERROR: FPGA FW is out of date
"Rhone03 rev17" currently in use
"Rhone03 rev20" available for update
10/05/22-02:09:30 (tNetCfgInit): NOTE: eth0: LinkUp event
10/05/22-02:09:30 (tNetCfgInit): NOTE: Network Ready
Current date: 10/05/22 time: 02:09:48
Send <BREAK> for Service Interface or baud rate change
10/05/22-02:09:47 (tRAID): NOTE: Set Powerup State
10/05/22-02:09:48 (tRAID): SOD Sequence is Normal, 0 on controller A
10/05/22-02:09:48 (tRAID): NOTE: loading flash file: ECTDell
10/05/22-02:09:48 (tRAID): NOTE: unloading flash file: ECTDell
10/05/22-02:09:49 (tRAID): NOTE: Turning on tray summary fault LED
10/05/22-02:09:49 (tRAID): NOTE: Installed Protocols: <MTPs: INIC SAS USB > <ITPs: RDMA ISER IPOIB > <STPs: FCP RmtDMA iSCSI SAS >
10/05/22-02:09:49 (tRAID): NOTE: Required Protocols: <MTPs: INIC SAS > <ITPs: UNK > <STPs: iSCSI SAS >
10/05/22-02:09:49 (tRAID): NOTE: loading flash file: iSCSI
10/05/22-02:09:50 (tRAID): NOTE: loading flash file: MTLSEBET1G
*** Disabled FULL TRACE ***
10/05/22-02:09:51 (tRAID): WARN: MTLSE: before mtl_create_pool:IO task size: 0x140
10/05/22-02:09:52 (tRAID): NOTE: DSM: Current revision 7
10/05/22-02:09:52 (tRAID): NOTE: SYMBOL: SYMbolAPI registered.
10/05/22-02:09:52 (tRAID): NOTE: cmgrCtlr: Board Manager HostBoardData Model Name for slot 0: 0501
10/05/22-02:09:52 (tRAID): NOTE: RCBBitmapManager total RPA size = 1761607680
10/05/22-02:09:52 (tRAID): NOTE: init: ioc: 0, PLVersion: 13-075-70-00
10/05/22-02:09:52 (tRAID): ERROR: findChannelFromIocPhy: channel not found, ioc=0, phy=0
10/05/22-02:09:52 (tRAID): ERROR: findChannelFromIocPhy: channel not found, ioc=0, phy=1
10/05/22-02:09:52 (tRAID): ERROR: findChannelFromIocPhy: channel not found, ioc=0, phy=2
10/05/22-02:09:52 (tRAID): ERROR: findChannelFromIocPhy: channel not found, ioc=0, phy=3
10/05/22-02:09:52 (tRAID): ERROR: findChannelFromIocPhy: channel not found, ioc=0, phy=4
10/05/22-02:09:52 (tRAID): ERROR: findChannelFromIocPhy: channel not found, ioc=0, phy=5
10/05/22-02:09:52 (tRAID): ERROR: findChannelFromIocPhy: channel not found, ioc=0, phy=6
10/05/22-02:09:52 (tRAID): ERROR: findChannelFromIocPhy: channel not found, ioc=0, phy=7
10/05/22-02:09:53 (IOSched): NOTE: SAS Expander Added: expDevHandle:x11 enclHandle:x2 numPhys:25 port:2 ioc:0 channel:0 numEntries:22
10/05/22-02:09:53 (IOSched): NOTE: SAS Expander Added: expDevHandle:x12 enclHandle:x3 numPhys:25 port:3 ioc:0 channel:1 numEntries:22
10/05/22-02:09:53 (IOSched): NOTE: SAS Expander Added: expDevHandle:x2d enclHandle:x4 numPhys:38 port:2 ioc:0 channel:0 numEntries:22
10/05/22-02:09:53 (IOSched): NOTE: SAS Expander Added: expDevHandle:x2e enclHandle:x5 numPhys:38 port:3 ioc:0 channel:1 numEntries:22
10/05/22-02:09:53 (IOSched): NOTE: SAS Expander Added: expDevHandle:x31 enclHandle:x6 numPhys:38 port:2 ioc:0 channel:0 numEntries:22
10/05/22-02:09:53 (IOSched): NOTE: SAS Expander Added: expDevHandle:x36 enclHandle:x7 numPhys:38 port:3 ioc:0 channel:1 numEntries:22
10/05/22-02:09:53 (IOSched): NOTE: SAS Expander Added: expDevHandle:x58 enclHandle:x8 numPhys:38 port:2 ioc:0 channel:0 numEntries:22
10/05/22-02:09:53 (IOSched): NOTE: SAS Expander Added: expDevHandle:x5d enclHandle:x9 numPhys:38 port:3 ioc:0 channel:1 numEntries:22
10/05/22-02:09:53 (IOSched): NOTE: SAS Expander Added: expDevHandle:x8c enclHandle:xa numPhys:38 port:2 ioc:0 channel:0 numEntries:22
10/05/22-02:09:53 (IOSched): NOTE: SAS Expander Added: expDevHandle:x91 enclHandle:xb numPhys:38 port:3 ioc:0 channel:1 numEntries:22
10/05/22-02:09:54 (tSasEvtWkr): NOTE: Alt controller path up on channel:0 devH:x2c expDevH:x11 phy:16 itn:13
10/05/22-02:09:54 (tSasExpChk): NOTE: Local Expander Firmware Version: 00.00.80.15
10/05/22-02:09:54 (tSasExpChk): NOTE: Enabling this controller's expander phys, channel=0
10/05/22-02:09:54 (tSasExpChk): NOTE: sasExpanderCheck send icon connection up event channel:0
10/05/22-02:09:54 (sasEnPhys): NOTE: sasEnableDrivePhysTask: 0 phys enabled on channel 0
10/05/22-02:09:55 (tRAID): NOTE: CrushMemoryPoolMgr: platform and memory (CPU MemSize:2048), adjusted allocating size to 65536 for CStripes
10/05/22-02:09:56 (tRAID): SOD: Instantiation Phase Complete
10/05/22-02:09:56 (tRAID): NOTE: Inter-Controller Communication Channels Opened
10/05/22-02:09:56 (tRAID): NOTE: LockMgr Role is Master
10/05/22-02:09:56 (tSasEvtWkr): NOTE: Alt controller path up on channel:1 devH:x2b expDevH:x12 phy:12 itn:28
10/05/22-02:09:56 (tSasExpChk): NOTE: Alternate Expander Firmware Version: 00.00.80.15
10/05/22-02:09:56 (tSasExpChk): NOTE: Enabling alternate controller's expander phys, channel=1
10/05/22-02:09:56 (tSasExpChk): NOTE: sasExpanderCheck send icon connection up event channel:1
10/05/22-02:09:56 (sasEnPhys): NOTE: sasEnableDrivePhysTask: 0 phys enabled on channel 1
10/05/22-02:09:57 (IOSched): NOTE: New Initiator: ioc: 0, port: 3, devHandle: x2b, sasAddress: 5842b2b46b93df08
10/05/22-02:09:57 (tSasInitWkr): NOTE: New Initiator: 1 - channel:1, devHandle:x2b, SAS Address:5842b2b46b93df08
10/05/22-02:09:57 (IOSched): NOTE: discoveredEncl: trayId:4 slotCount:24 eli:500c04f2966d8600
10/05/22-02:09:58 (IOSched): NOTE: discoveredEncl: trayId:1 slotCount:12 eli:500c04f2e541a400
10/05/22-02:09:58 (tHckReset): NOTE: Firmware running line - ON
10/05/22-02:09:58 (IOSched): NOTE: New Initiator: ioc: 0, port: 2, devHandle: x2c, sasAddress: 5842b2b46b93df0c
10/05/22-02:09:58 (tSasInitWkr): NOTE: New Initiator: 2 - channel:0, devHandle:x2c, SAS Address:5842b2b46b93df0c
10/05/22-02:09:58 (IOSched): NOTE: discoveredEncl: trayId:3 slotCount:24 eli:500c04f2b9ac6100
10/05/22-02:09:59 (IOSched): NOTE: discoveredEncl: trayId:2 slotCount:24 eli:500c04f21e64cd00
10/05/22-02:09:59 (IOSched): NOTE: discoveredEncl: trayId:2 slotCount:24 eli:500c04f21e64cd00
10/05/22-02:10:00 (IOSched): NOTE: discoveredEncl: trayId:3 slotCount:24 eli:500c04f2b9ac6100
10/05/22-02:10:00 (iconMgr2): NOTE: HealthCheck: Alt Ctl: 3 Icon_CheckIn, state: 0 Start
10/05/22-02:10:00 (iconMgr2): NOTE: HealthCheckManager: Notify Event 3 Ctl_CheckIn
10/05/22-02:10:00 (IOSched): NOTE: discoveredEncl: trayId:1 slotCount:12 eli:500c04f2e541a400
10/05/22-02:10:00 (IOSched): NOTE: discoveredEncl: trayId:4 slotCount:24 eli:500c04f2966d8600
10/05/22-02:10:00 (tSasDiscCom): NOTE: SAS: Initial Discovery Complete Time: 44 seconds since last power on/reset, 11 seconds since sas instantiated
10/05/22-02:10:00 (tSasDiscCom): NOTE: sasDiscoveryCompleteFirstTime send icon connection up event channel:0
10/05/22-02:10:00 (tSasDiscCom): NOTE: sasDiscoveryCompleteFirstTime send icon connection up event channel:1
10/05/22-02:10:00 (iacTask1): NOTE: fbm:altValidateSubModelID: Sub-Model IDs Validated
10/05/22-02:10:00 (tRAID): NOTE: WWN baseName 0002782b-cb5357f8 (valid==>SigMatch)
10/05/22-02:10:00 (tRAID): NOTE: spmEarlyData: No data available
10/05/22-02:10:01 (tRAID): NOTE: sebeReadEEPROMChecksum: EEPROM checksum match: 0x81986278
10/05/22-02:10:01 (tRAID): NOTE: ServerEngines BladeEngine 2 Firmware Update Utility v2.901.242.0
Copyright 2009 ServerEngines Corpo
10/05/22-02:10:01 (tRAID): NOTE:
10/05/22-02:10:01 (tRAID): NOTE: Found BE card 0x0 at 2.0 bar1-base 0xf0200000 bar2-base 0xf0220000 dev 0x3231954
10/05/22-02:10:01 (tRAID): NOTE: flash_rbver = 2.0.3.768
10/05/22-02:10:01 (tRAID): NOTE: flash_armfwver = 2.901.326.0
10/05/22-02:10:02 (tRAID): NOTE: ufi_rbver = 2.0.3.768
10/05/22-02:10:02 (tRAID): NOTE: ufi_armfwver = 2.901.326.0
10/05/22-02:10:02 (tRAID): NOTE: Flash on BladeEngine at 02 : 00 matches UFI SEBE2T1G
10/05/22-02:10:03 (tRAID): NOTE: Target:irq_ptr:0x0xe3ac38, func:0x0, bcmId:0x0
10/05/22-02:10:03 (tRAID): NOTE: VendorID 0x19a2 DeviceId 0x210 bcmId 0 BDF 2.0.0
10/05/22-02:10:03 (tRAID): NOTE: Target:VKI_INT_CONNECT success:status:0x0
10/05/22-02:10:03 (tRAID): ERROR: Target:ISR registered:irq_ptr:0xe3ac38, phba:0x322ee88, func:0x0
10/05/22-02:10:03 (tRAID): NOTE: BcmId [0] PCI [2.0.0]
10/05/22-02:10:03 (tRAID): NOTE: DevId 0x0210 VendId 0x19a2
10/05/22-02:10:03 (tRAID): NOTE: BAR0 IO base 0x00000000 size 0
10/05/22-02:10:03 (tRAID): NOTE: BAR1 MEM base 0xf0200000 size 1
10/05/22-02:10:03 (tRAID): NOTE: BAR2 MEM base 0xf0220000 size 1
10/05/22-02:10:03 (tRAID): NOTE: BAR4 MEM base 0xf0240000 size 1
10/05/22-02:10:03 (tRAID): NOTE: MPU_EP_SEMAPHORE 0x0000c000
Driver Build Date:Jan 18 2017, Time:16:06:27
ServerEngines iSCSI Target Driver loaded
iSCSI Firmware Version:2.901.326.0
iSCSI Driver Version:2.901.242.0
Found BladeEngine2 in Gen2 x8 PCIe slot
10/05/22-02:10:05 (tRAID): NOTE: RPA memory region : start 0x471977e00 size 16777216 (ConIO-size 16777216) registered for pre-post data buffers on port 0
10/05/22-02:10:05 (tRAID): NOTE: sebeReadEEPROMChecksum: EEPROM checksum match: 0x81986278
10/05/22-02:10:05 (tRAID): NOTE: Target:irq_ptr:0x0xe3aaf0, func:0x1, bcmId:0x1
10/05/22-02:10:05 (tRAID): NOTE: VendorID 0x19a2 DeviceId 0x210 bcmId 1 BDF 2.0.1
10/05/22-02:10:05 (tRAID): NOTE: Target:VKI_INT_CONNECT success:status:0x0
10/05/22-02:10:05 (tRAID): ERROR: Target:ISR registered:irq_ptr:0xe3aaf0, phba:0x322f5a8, func:0x1
10/05/22-02:10:05 (tRAID): NOTE: BcmId [1] PCI [2.0.1]
10/05/22-02:10:05 (tRAID): NOTE: DevId 0x0210 VendId 0x19a2
10/05/22-02:10:05 (tRAID): NOTE: BAR0 IO base 0x00000000 size 0
10/05/22-02:10:05 (tRAID): NOTE: BAR1 MEM base 0xf0260000 size 1
10/05/22-02:10:05 (tRAID): NOTE: BAR2 MEM base 0xf0280000 size 1
10/05/22-02:10:05 (tRAID): NOTE: BAR4 MEM base 0xf02a0000 size 1
10/05/22-02:10:05 (tRAID): NOTE: MPU_EP_SEMAPHORE 0x0000c000
Driver Build Date:Jan 18 2017, Time:16:06:27
ServerEngines iSCSI Target Driver loaded
iSCSI Firmware Version:2.901.326.0
iSCSI Driver Version:2.901.242.0
Found BladeEngine2 in Gen2 x8 PCIe slot
10/05/22-02:10:07 (tRAID): NOTE: RPA memory region : start 0x472977e00 size 16777216 (ConIO-size 16777216) registered for pre-post data buffers on port 1
10/05/22-02:10:07 (tRAID): NOTE: sebeReadEEPROMChecksum: EEPROM checksum match: 0x81986278
10/05/22-02:10:08 (tRAID): NOTE: Target:irq_ptr:0x0xe3a9a8, func:0x2, bcmId:0x2
10/05/22-02:10:08 (tRAID): NOTE: VendorID 0x19a2 DeviceId 0x210 bcmId 2 BDF 2.0.2
10/05/22-02:10:08 (tRAID): NOTE: Target:VKI_INT_CONNECT success:status:0x0
10/05/22-02:10:08 (tRAID): ERROR: Target:ISR registered:irq_ptr:0xe3a9a8, phba:0x322fcc8, func:0x2
10/05/22-02:10:08 (tRAID): NOTE: BcmId [2] PCI [2.0.2]
10/05/22-02:10:08 (tRAID): NOTE: DevId 0x0210 VendId 0x19a2
10/05/22-02:10:08 (tRAID): NOTE: BAR0 IO base 0x00000000 size 0
10/05/22-02:10:08 (tRAID): NOTE: BAR1 MEM base 0xf02c0000 size 1
10/05/22-02:10:08 (tRAID): NOTE: BAR2 MEM base 0xf02e0000 size 1
10/05/22-02:10:08 (tRAID): NOTE: BAR4 MEM base 0xf0300000 size 1
10/05/22-02:10:08 (tRAID): NOTE: MPU_EP_SEMAPHORE 0x0000c000
Driver Build Date:Jan 18 2017, Time:16:06:27
ServerEngines iSCSI Target Driver loaded
iSCSI Firmware Version:2.901.326.0
iSCSI Driver Version:2.901.242.0
Found BladeEngine2 in Gen2 x8 PCIe slot
10/05/22-02:10:10 (tRAID): NOTE: RPA memory region : start 0x473977e00 size 16777216 (ConIO-size 16777216) registered for pre-post data buffers on port 2
10/05/22-02:10:10 (tRAID): NOTE: sebeReadEEPROMChecksum: EEPROM checksum match: 0x81986278
10/05/22-02:10:10 (tRAID): NOTE: Target:irq_ptr:0x0xe3a860, func:0x3, bcmId:0x3
10/05/22-02:10:10 (tRAID): NOTE: VendorID 0x19a2 DeviceId 0x210 bcmId 3 BDF 2.0.3
10/05/22-02:10:10 (tRAID): NOTE: Target:VKI_INT_CONNECT success:status:0x0
10/05/22-02:10:10 (tRAID): ERROR: Target:ISR registered:irq_ptr:0xe3a860, phba:0x32303e8, func:0x3
10/05/22-02:10:10 (tRAID): NOTE: BcmId [3] PCI [2.0.3]
10/05/22-02:10:10 (tRAID): NOTE: DevId 0x0210 VendId 0x19a2
10/05/22-02:10:10 (tRAID): NOTE: BAR0 IO base 0x00000000 size 0
10/05/22-02:10:10 (tRAID): NOTE: BAR1 MEM base 0xf0320000 size 1
10/05/22-02:10:10 (tRAID): NOTE: BAR2 MEM base 0xf0340000 size 1
10/05/22-02:10:10 (tRAID): NOTE: BAR4 MEM base 0xf0360000 size 1
10/05/22-02:10:10 (tRAID): NOTE: MPU_EP_SEMAPHORE 0x0000c000
Driver Build Date:Jan 18 2017, Time:16:06:27
ServerEngines iSCSI Target Driver loaded
iSCSI Firmware Version:2.901.326.0
iSCSI Driver Version:2.901.242.0
Found BladeEngine2 in Gen2 x8 PCIe slot
10/05/22-02:10:12 (tRAID): NOTE: RPA memory region : start 0x474977e00 size 16777216 (ConIO-size 16777216) registered for pre-post data buffers on port 3
10/05/22-02:10:13 (tRAID): SOD: Pre-Initialization Phase Complete
10/05/22-02:10:18 (ssmTimer): NOTE: Power Supply 0 VPD recovered on retry 1
10/05/22-02:10:19 (ProcessHandlers): WARN: DriveMgr::processPredictedFailure - DriveMgr not initialized
10/05/22-02:10:59 (ProcessHandlers): WARN: DriveMgr::processPredictedFailure - DriveMgr not initialized
10/05/22-02:11:10 (iacTask4): NOTE: cmgrCtlr: Board Manager HostBoardData Model Name for slot 0: 0501
10/05/22-02:11:10 (tRAID): NOTE: ACS: autoCodeSync(): Process start. Comm Status: 1
10/05/22-02:11:10 (iacTask7): NOTE: ACS: Acs Needed on Alt: No, StateCode: 0, ReasonCode = 2
10/05/22-02:11:16 (iacTask1): NOTE: cmgrCtlr: Board Manager HostBoardData Model Name for slot 0: 0501
10/05/22-02:11:16 (tRAID): NOTE: iocfi: Peering Disabled (Alt in SOD)
10/05/22-02:11:17 (tRAID): SOD: Code Synchronization Initialization Phase Complete
10/05/22-02:11:20 (iconMgr2): NOTE: cmgrCtlr: Board Manager HostBoardData Model Name for slot 0: 0501
10/05/22-02:11:28 (NvpsPersistentSyncM): NOTE: NVSRAM Persistent Storage updated successfully
10/05/22-02:11:35 (ProcessHandlers): WARN: DriveMgr::processPredictedFailure - DriveMgr not initialized
10/05/22-02:11:35 (tRAID): NOTE: SAFE: Process new features
10/05/22-02:11:35 (tRAID): NOTE: SAFE: Process legacy features
10/05/22-02:11:46 (tRAID): NOTE: sas: Peering Disabled (Alt in SOD)
10/05/22-02:11:46 (tRAID): NOTE: b_isn: Peering Disabled (Alt in SOD)
10/05/22-02:11:46 (tRAID): NOTE: ion: Peering Disabled (Alt in SOD)
10/05/22-02:11:46 (iacTask1): NOTE: sas: checkInAlt Peering Enabled
10/05/22-02:11:50 (tRAID): WARN: mtlsebe::MtlInterface::parameterValidation
10/05/22-02:11:52 (tRAID): WARN: mtlsebe::MtlInterface::parameterValidation
10/05/22-02:11:53 (tRAID): WARN: mtlsebe::MtlInterface::parameterValidation
10/05/22-02:11:54 (tRAID): WARN: mtlsebe::MtlInterface::parameterValidation
10/05/22-02:11:55 (mtlseMgmtTask): WARN: Port[0] MTLSE Link event: link UP
10/05/22-02:11:55 (mtlseMgmtTask): WARN: Port[1] MTLSE Link event: link UP
10/05/22-02:11:55 (mtlseMgmtTask): WARN: Port[2] MTLSE Link event: link UP
10/05/22-02:11:55 (mtlseMgmtTask): WARN: Port[3] MTLSE Link event: link UP
10/05/22-02:11:55 (iacTask3): NOTE: b_isn: checkInAlt Peering Enabled
10/05/22-02:11:55 (iacTask4): NOTE: ion: checkInAlt Peering Enabled
10/05/22-02:11:55 (tRAID): WARN: mtlsebe::MtlInterface::parameterValidation
10/05/22-02:11:55 (tRAID): WARN: mtlsebe::MtlInterface::parameterValidation
10/05/22-02:11:55 (tRAID): WARN: mtlsebe::MtlInterface::parameterValidation
10/05/22-02:11:55 (tRAID): WARN: mtlsebe::MtlInterface::parameterValidation
10/05/22-02:11:55 (tRAID): WARN: MTLSE: group iSCSI Mtl::parameterCommitCompleted without any other parameter events.
10/05/22-02:11:55 (tRAID): WARN: MTLSE: group iSNS Mtl::parameterCommitCompleted without any other parameter events.
10/05/22-02:12:00 (tRAID): NOTE: iSCSI Queue Depth is(QLTA_MAX_TRBS) = 2560
10/05/22-02:12:00 (tRAID): NOTE: The sizeof TRBext =104 TpbExt =54
10/05/22-02:12:01 (tTLoginJ): NOTE: New conn:0xbfe1060 CID:x0 NORM sess:0xbce5128 I:iqn.1998-01.com.vmware:localhost:1809487803:35 ISID:x003d02001c00 TSIH:x2 prt:3
10/05/22-02:12:01 (tTLoginJ): NOTE: New conn:0xbfe1db8 CID:x0 NORM sess:0xbced020 I:iqn.1998-01.com.vmware:esxi-r720-03-5890897e ISID:x00023d000014 TSIH:x3 prt:3
10/05/22-02:12:03 (tTLoginJ): NOTE: New conn:0xbfe2b10 CID:x0 NORM sess:0xbcf5020 I:iqn.1998-01.com.vmware:localhost:1599539411:35 ISID:x003d02001c00 TSIH:x4 prt:3
10/05/22-02:12:10 (iacTask2): WARN: mtlsebe::MtlInterface::parameterValidation
10/05/22-02:12:10 (iacTask2): WARN: mtlsebe::MtlInterface::parameterValidation
10/05/22-02:12:10 (iacTask2): WARN: mtlsebe::MtlInterface::parameterValidation
10/05/22-02:12:10 (iacTask2): WARN: mtlsebe::MtlInterface::parameterValidation
10/05/22-02:12:10 (iacTask8): WARN: MTLSE: group iSCSI Mtl::parameterCommitCompleted without any other parameter events.
10/05/22-02:12:10 (iacTask8): WARN: MTLSE: group iSNS Mtl::parameterCommitCompleted without any other parameter events.
10/05/22-02:12:12 (ProcessHandlers): WARN: DriveMgr::processPredictedFailure - DriveMgr not initialized
10/05/22-02:12:18 (tRAID): NOTE: CacheMgr::cacheOpenMirrorDevice:: mirror device 0xf00011
10/05/22-02:12:19 (tRAID): NOTE: CCM:readCheckpointFromNvsram for this 0xa5162e00 alt 0xfffffffe
10/05/22-02:12:19 (tRAID): WARN: CCM: validateCacheMem() cache memory is invalid
10/05/22-02:12:19 (tRAID): NOTE: CCM: validateCacheMem() Initializing my partition
10/05/22-02:12:20 (tRAID): WARN: CCM: setupRcbBitmapRegion(): Sync RPA RCB bitmap region from pstor/dbm
10/05/22-02:12:20 (tRAID): NOTE: Cache: Restoring RCB region of this controller ...
10/05/22-02:12:20 (tRAID): NOTE: Cache: RCB region restore of this controller completed
10/05/22-02:12:20 (tRAID): NOTE: Cache: Restoring RCB region of alternate controller ...
10/05/22-02:12:20 (tRAID): NOTE: Cache: RCB region restore of alternate controller completed
10/05/22-02:12:20 (tRAID): NOTE: restoreRCBs: m_MetadataIndexTable.size(): 308536, addr 0x5a857d4
10/05/22-02:12:20 (tRAID): NOTE: Cache: Dirty cache blocks to be restored: 2244
10/05/22-02:12:20 (tRAID): NOTE: Cache: Restoring dirty cache blocks ...
10/05/22-02:12:21 (tRAID): NOTE: CCM: sodMirrorCheckIn() waiting for alt cache mirror check in...
10/05/22-02:12:29 (tRAID): NOTE: doRecovery: myMemory:1, IORCB:0, NEW:0, OLD:2244
10/05/22-02:12:29 (tRAID): NOTE: buildBackupCCBList: mirror:0, unmirror:0, wb:0, wbm:2244, wt:0, backupListSize:2244
10/05/22-02:12:29 (tRAID): NOTE: buildBackupMetadata: CME entries completed successfully for all local RCBs
10/05/22-02:12:29 (tRAID): WARN: Cache: Applied 0 bitmaps from pstor bitmap region
10/05/22-02:12:30 (tRAID): NOTE: doRecovery: myMemory:0, IORCB:0, NEW:0, OLD:0
10/05/22-02:12:30 (tRAID): NOTE: CCM: syncRCBBitmapRegionToAlt: Sync RPA RCB bitmap region to the alternate
10/05/22-02:12:30 (tRAID): WARN: PSTOR: pstorRpaSyncRemoteCopy is invoked in syncMultple
10/05/22-02:12:37 (tRAID): NOTE: UWManager::recoverMyNvsramIWLog: adding to NVSRAM IW QUEUE unit 1 blk 0x0000000abf8 blks 1 own A method M
10/05/22-02:12:37 (tRAID): NOTE: UWMgr findIWLogs: Found IW log drive. Devnum 0x10207 tray=0 slot=8 ssd=0 qos=3 controller=0
10/05/22-02:12:37 (tRAID): NOTE: UWMgr findIWLogs: Found IW log drive. Devnum 0x10208 tray=0 slot=9 ssd=0 qos=3 controller=0
10/05/22-02:12:37 (IWTask): NOTE: UWMgr: IW logging started
10/05/22-02:12:46 (tRAID): WARN: *** Volume 0 is now FAILED ***
10/05/22-02:12:47 (tRAID): WARN: *** Volume 2 is now DEGRADED ***
10/05/22-02:12:47 (tRAID): WARN: *** Volume 2 is now FAILED ***
10/05/22-02:12:47 (tRAID): WARN: *** Volume 4 is now DEGRADED ***
10/05/22-02:12:47 (tRAID): WARN: *** Volume 4 is now FAILED ***
10/05/22-02:12:51 (IOSched): NOTE: send running in IOSched task with classID 73.
10/05/22-02:13:10 (ProcessHandlers): WARN: PfaManager observed PFAEvent on devnum 0x00010009
10/05/22-02:13:29 (tRAID): NOTE: MIB creation complete - size 15
10/05/22-02:13:31 (tRAID): SOD: Initialization Phase Complete
==============================================
Title: Disk Array Controller
Copyright 2008-2017 NetApp, Inc. All Rights Reserved.
Name: RC
Version: 08.20.21.61
Date: 01/18/2017
Time: 16:03:41 CST
Models: 2660
Manager: devmgr.v1120api13.Manager
==============================================
10/05/22-02:13:31 (tRAID): sodMain Normal sequence finished, elapsed time = 223 seconds
10/05/22-02:13:31 (tRAID): sodMain complete
10/05/22-02:13:32 (ProcessHandlers): NOTE: Crush Volume 0x2 background op active, do not start media scan agent
10/05/22-02:13:32 (ProcessHandlers): NOTE: Crush Volume 0x4 background op active, do not start media scan agent
10/05/22-02:13:32 (ProcessHandlers): NOTE: Crush Volume 0x0 background op active, do not start media scan agent
10/05/22-02:13:34 (iacTask7): NOTE: cmgrCtlr: Board Manager HostBoardData Model Name for slot 0: 0501
10/05/22-02:13:37 (iconMgr6): NOTE: cmgrCtlr: Board Manager HostBoardData Model Name for slot 0: 0501
10/05/22-02:13:43 (IOSched): NOTE: UWMgr: Alt Mirror Check In Complete
10/05/22-02:13:43 (IWTask): NOTE: UWMgr: IW peering started
10/05/22-02:13:43 (IWTask): NOTE: UWMgr: IW logging stopped
10/05/22-02:13:43 (IWTask): NOTE: UWMgr: IW mirroring started
10/05/22-02:13:43 (IWTask): NOTE: UWMgr: IW peering stopped
10/05/22-02:13:43 (CrushEval): NOTE: Vol SSID 0 can't run any operations!!
10/05/22-02:13:44 (CrushEval): NOTE: Vol SSID 2 can't run any operations!!
10/05/22-02:13:45 (CrushEval): NOTE: Vol SSID 4 can't run any operations!!
10/05/22-02:13:45 (ProcessHandlers): ERROR: symrpc::VolumeGroupStateNotValidException, caught in completeMoveDriveExtents
10/05/22-02:13:45 (ProcessHandlers): ERROR: symrpc::VolumeGroupStateNotValidException, caught in completeMoveDriveExtents
10/05/22-02:13:45 (ProcessHandlers): ERROR: symrpc::VolumeGroupStateNotValidException, caught in completeMoveDriveExtents
10/05/22-02:13:45 (iacTask6): NOTE: DSM: Async upgrade completed on Slave (0 drives 0 requested)
10/05/22-02:13:45 (ProcessHandlers): ERROR: symrpc::VolumeGroupStateNotValidException, caught in completeMoveDriveExtents
10/05/22-02:13:45 (ProcessHandlers): ERROR: symrpc::VolumeGroupStateNotValidException, caught in completeMoveDriveExtents
10/05/22-02:13:45 (ProcessHandlers): ERROR: symrpc::VolumeGroupStateNotValidException, caught in completeMoveDriveExtents
10/05/22-02:13:45 (ProcessHandlers): ERROR: symrpc::VolumeGroupStateNotValidException, caught in completeMoveDriveExtents
10/05/22-02:13:45 (ProcessHandlers): ERROR: symrpc::VolumeGroupStateNotValidException, caught in completeMoveDriveExtents
10/05/22-02:13:45 (ProcessHandlers): ERROR: symrpc::VolumeGroupStateNotValidException, caught in completeMoveDriveExtents
10/05/22-02:13:45 (ProcessHandlers): ERROR: symrpc::VolumeGroupStateNotValidException, caught in completeMoveDriveExtents
10/05/22-02:13:45 (ProcessHandlers): NOTE: SYMbol available
10/05/22-02:13:46 (ProcessHandlers): SOD: sodComplete Notification Complete
10/05/22-02:15:49 (bdbmSync): NOTE: OBB Synchronization: Successful
10/05/22-02:18:33 (utlTimer1): NOTE: sasDEClearRecoveryState:
10/05/22-02:20:21 (ProcessHandlers): WARN: PfaManager observed PFAEvent on devnum 0x00010009
10/05/22-02:30:25 (ProcessHandlers): WARN: PfaManager observed PFAEvent on devnum 0x00010009