Unsolved

1920

February 12th, 2020 22:00

Powervault md3200 controller pci lockdown

Hello.I have an issue with our MD3200. When controller A starts up, it writes "Controller has been locked down due to PCI errors". Connecting with a serial cable, I enter the command "lemclearlockdown", after pci lockdown does not disappear and the controller remains in a locked state. After using the "clearHardwareLockdown" and after "lemclearlockdown" from 5-6 times and, controller the controller boots into "in service mode", when i change a state of controller into online mode controller crashes with error and shuts down.

Help me please. I provided capture events via serial cable and inserted the controller into md3200.

 

I else tried to change controllers by slots else but problem hasn't fix on that controller.

Reset, Power-Up Diagnostics - Loop 1 of 1
3600 Processor DRAM
01 Data lines Passed
02 Address lines Passed
3300 NVSRAM
01 Data lines Passed
4410 Ethernet 82574 1
01 Register read Passed
02 Register address lines Passed
6D40 Bobcat
02 Flash Test Passed
3700 PLB SRAM
01 Data lines Passed
02 Address lines Passed
6D50 LSISAS2008 IOC 1
01 Register Read Test Passed
02 Register Address Lines Test Passed
03 Register Data Lines Test 02/13/20-04:23:17 (tDiag): ERROR: bcmPciCfgWrite error for bus 2, dev 0, func 0, reg 0x21, size 1.
Passed
3900 Real-Time Clock
01 RT Clock Tick Passed
Diagnostic Manager exited normally.

Controller has been locked down due to PCI errors:

================= EXCEPTION LOG =================
Serial number: 44N001C
Entry count: 442
Wrap-arounds: 5
First entry time:
Current date/time: FEB-13-2020 04:23:18 AM

---- Log Entry #353 FEB-06-2020 12:57:43 AM ----
02/06/20-09:40:37 (tShell0): ASSERT: Assertion failed: m_Instance != 0, file /u/symsm/ccm_wa/symbios/RAIDCore-2683.1.171/e10_784_26x0_mercury-07.84.53.60/Application/RAIDLib/lemLockdownErrMgmt.h, line 247

---- Log Entry #354 FEB-06-2020 12:58:11 AM ----

Root Complex TLP header[0] 30008000
Root Complex TLP header[1] 01200033
Root Complex TLP header[2] 00000000
Root Complex TLP header[3] 00000000


PCI SERR Exception
PLX PCI-E Switch (Unit 0)
VID 0x10b5 DID 0x8632 B0:D0:F0
PCI Status = 0x4010
Bridge Secondary PCI Status = 0x4000
PLX PCI-E Bridge to Host Card (Unit 1)
VID 0x10b5 DID 0x8632 B1:D4:F0
PCI Status = 0x4010
PCI-E Device Status = 0x0005
PCI-E AER Uncorrectable Status = 0x00000020
Header Log 0 = 0x00000000
Header Log 1 = 0x00000000
Header Log 2 = 0x00000000
Header Log 3 = 0x00000000
PCI-E AER Correctable Status = 0x00001101
Host-side SAS (Unit 0)
VID 0x1000 DID 0x0072 B2:D0:F0
PCI-E Device Status = 0x0001
PCI-E AER Uncorrectable Status = 0x00004000
Header Log 0 = 0x00000000
Header Log 1 = 0x00000000
Header Log 2 = 0x00000000
Header Log 3 = 0x00000000
PCI-E AER Correctable Status = 0x000011C1
Host-side SAS (Unit 1)
VID 0x1000 DID 0x0072 B2:D0:F0
PCI-E Device Status = 0x0001
PCI-E AER Correctable Status = 0x00000081

---- Log Entry #355 FEB-06-2020 12:58:41 AM ----

Root Complex TLP header[0] 30008000
Root Complex TLP header[1] 01200033
Root Complex TLP header[2] 00000000
Root Complex TLP header[3] 00000000

Outbound Completion Error Stat 0x00204800, Addr 0x00000000F0200000

PCI SERR Exception
Meteor PCI-E (Unit 0)
VID 0x1000 DID 0x0064 B255:D255:F255
PCI Status = 0x4010
PCI-E Device Status = 0x0004
PCI-E AER Uncorrectable Status = 0x00004000
Header Log 0 = 0x00000000
Header Log 1 = 0x00000000
Header Log 2 = 0x00000000
Header Log 3 = 0x00000000
PLX PCI-E Switch (Unit 0)
VID 0x10b5 DID 0x8632 B0:D0:F0
PCI Status = 0x4010
Bridge Secondary PCI Status = 0x4000
PCI-E Device Status = 0x0004
PCI-E AER Uncorrectable Status = 0x00040000
Header Log 0 = 0x30008000
Header Log 1 = 0x00000033
Header Log 2 = 0x00000000
Header Log 3 = 0x00000000
PLX PCI-E Bridge to Host Card (Unit 1)
VID 0x10b5 DID 0x8632 B1:D4:F0
PCI Status = 0x4010
PCI-E Device Status = 0x0005
PCI-E AER Uncorrectable Status = 0x00000020
Header Log 0 = 0x00000000
Header Log 1 = 0x00000000
Header Log 2 = 0x00000000
Header Log 3 = 0x00000000
PCI-E AER Correctable Status = 0x00001100
Host-side SAS (Unit 0)
VID 0x1000 DID 0x0072 B2:D0:F0
PCI-E Device Status = 0x0001
PCI-E AER Correctable Status = 0x000011C1
Host-side SAS (Unit 1)
VID 0x1000 DID 0x0072 B2:D0:F0
PCI-E Device Status = 0x0001
PCI-E AER Correctable Status = 0x00000001

---- Log Entry #356 FEB-06-2020 12:59:10 AM ----

Root Complex TLP header[0] 30008000
Root Complex TLP header[1] 01200033
Root Complex TLP header[2] 00000000
Root Complex TLP header[3] 00000000


PCI SERR Exception
PLX PCI-E Switch (Unit 0)
VID 0x10b5 DID 0x8632 B0:D0:F0
PCI Status = 0x4010
Bridge Secondary PCI Status = 0x4000
PLX PCI-E Bridge to Host Card (Unit 1)
VID 0x10b5 DID 0x8632 B1:D4:F0
PCI Status = 0x4010
PCI-E Device Status = 0x0005
PCI-E AER Uncorrectable Status = 0x00000020
Header Log 0 = 0x00000000
Header Log 1 = 0x00000000
Header Log 2 = 0x00000000
Header Log 3 = 0x00000000
PCI-E AER Correctable Status = 0x00001100

---- Log Entry #357 FEB-06-2020 12:59:14 AM ----

WARNING: Restart by watchdog time out

---- Log Entry #358 FEB-06-2020 12:59:40 AM ----

Root Complex TLP header[0] 30008000
Root Complex TLP header[1] 01200033
Root Complex TLP header[2] 00000000
Root Complex TLP header[3] 00000000


PCI SERR Exception
PLX PCI-E Switch (Unit 0)
VID 0x10b5 DID 0x8632 B0:D0:F0
PCI Status = 0x4010
Bridge Secondary PCI Status = 0x4000
PLX PCI-E Bridge to Host Card (Unit 1)
VID 0x10b5 DID 0x8632 B1:D4:F0
PCI Status = 0x4010
PCI-E Device Status = 0x0005
PCI-E 02/13/20-04:23:22 (tNetCfgInit): NOTE: eth0: LinkUp event
AER Uncorrectable Status = 0x00000020
Hea02/13/20-04:23:22 (tNetCfgInit): NOTE: Acquiring network parameters for interface gei0 using DHCP
der Log 0 = 0x00000000
Header Log 1 = 0x00000000
Header Log 2 = 0x00000000
Header Log 3 = 0x00000000
PCI-E AER Correctable Status = 0x00001100
Host-side SAS (Unit 0)
VID 0x1000 DID 0x0072 B2:D0:F0
PCI-E Device Status = 0x0001
PCI-E AER Uncorrectable Status = 0x00004000
Header Log 0 = 0x00000000
Header Log 1 = 0x00000000
Header Log 2 = 0x00000000
Header Log 3 = 0x00000000
PCI-E AER Correctable Status = 0x000011C1
Host-side SAS (Unit 1)
VID 0x1000 DID 0x0072 B2:D0:F0
PCI-E Device Status = 0x0001
PCI-E AER Correctable Status = 0x00001081

---- Log Entry #359 FEB-06-2020 01:01:37 AM ----
02/06/20-09:44:30 (tShell0): ASSERT: Assertion failed: m_Instance != 0, file /u/symsm/ccm_wa/symbios/RAIDCore-2683.1.171/e10_784_26x0_mercury-07.84.53.60/Application/RAIDLib/lemLockdownErrMgmt.h, line 247

---- Log Entry #360 FEB-06-2020 01:02:05 AM ----

Root Complex TLP header[0] 30008000
Root Complex TLP header[1] 01200033
Root Complex TLP header[2] 00000000
Root Complex TLP header[3] 00000000


PCI SERR Exception
PLX PCI-E Switch (Unit 0)
VID 0x10b5 DID 0x8632 B0:D0:F0
PCI Status = 0x4010
Bridge Secondary PCI Status = 0x4000
PLX PCI-E Bridge to Host Card (Unit 1)
VID 0x10b5 DID 0x8632 B1:D4:F0
PCI Status = 0x4010
PCI-E Device Status = 0x0005
PCI-E AER Uncorrectable Status = 0x00000020
Header Log 0 = 0x00000000
Header Log 1 = 0x00000000
Header Log 2 = 0x00000000
Header Log 3 = 0x00000000
PCI-E AER Correctable Status = 0x00001100
Host-side SAS (Unit 0)
VID 0x1000 DID 0x0072 B2:D0:F0
PCI-E Device Status = 0x0001
PCI-E AER Uncorrectable Status = 0x00004000
Header Log 0 = 0x00000000
Header Log 1 = 0x00000000
Header Log 2 = 0x00000000
Header Log 3 = 0x00000000
PCI-E AER Correctable Status = 0x000011C1
Host-side SAS (Unit 1)
VID 0x1000 DID 0x0072 B2:D0:F0
PCI-E Device Status = 0x0001
PCI-E AER Correctable Status = 0x00001101

---- Log Entry #361 FEB-06-2020 01:02:35 AM ----

February 20th, 2020 09:00

Hello Vladplayer1993,

What is the current version of firmware that is on your MD3200?  Are you seeing the lockdown error on both controllers or on just one controller?

Please let us know if you have any other questions.

2 Posts

February 20th, 2020 17:00

I am running an MD3420 with 2 Esxi hosts hanging off of it. If you have it cabled correctly it will not skip a beat if one of the controllers goes down. DAS is going to be faster than a SAN or NAS.

 

February 20th, 2020 20:00

Hello, the current version of firmware on my md 3200 is 07.84.53.60. I'm seeing the lockdown error on just one controller (controller-1). The second controller (controller-2) is functioning properly. I also tried to change controllers by slots but problem hasn't fix on that controller (controller-1)

7 Posts

February 22nd, 2020 09:00

I am running an MD3420 with 2 Esxi hosts hanging off of it. If you have it cabled correctly it will not skip a beat if one of the controllers goes down. DAS is going to be faster than a SAN or 

February 24th, 2020 11:00

Hello Vladplayer1993,

What you are going to need to do is to clear hardware lockdown.  If after you clear the hardware lockdown and your controller reboots, if you are still getting the error then you will need to replace your controller.

Please let us know if you have any other questions.

22 Posts

July 5th, 2022 16:00

please give steps to clear this hardware lockdown and db corruption fix.

Moderator

 • 

4.1K Posts

July 5th, 2022 23:00

Hi @dharmie,

 

I've responded to you at https://dell.to/3yGnYI9;

 

Please avoid posting on multiple post, as you might obtain multiple replies which may cause unintended various troubleshooting causing more damages to your issue. 

Top