PowerVault MD3800f in Lockdown

Hi @DiegoLopez thank you for the reply. Do you mind telling me how to do lemClearLockdown is after or before pressing S .

Appreciate your support

Hello @mazen.ballat,

Ok, so you can clear the lockdown running the lemClearLockdown via the serial cable connected to the affected controller and that should clear the error.  

Then you need to reboot the controller and after the controller reboots capture all the boot. If after restarting the same error comes back again and it does get stuck in lockdown again you need to find the root of the problem.

Regards.

The easiest way to get you unit out of lockdown when you have changed the hard drives is to do the following

1. remove 1 controller and have the serial cable connected to the other controller

2. when you see ### press ctrl and b

3. you will have a menu appear, select 10 for serial interface, 1 for console only then q to quit and r to reboot

4. when the controller has rebooted and you see the lockdown message press enter for console prompt and type LemClearLockdown (note case sensitive)

5. let the controller reboot

6. repeat with the alternative controller, removing the controller you have just worked on

7. insert both controllers and see if MDSN can detect the unit, it may still be marked as needs attention, this is due to being in recovery mode, follow the recovery guru's instructions to clear the recovery mode

Regards

Thank you @DiegoLopez I will give it a try and let you know

Hi @DiegoLopez 

I apologize for the late reply as I wasn't able to try the solution till today due to the lockdown.

I have removed the controller and pressed Ctrl+B however no menu appeared but the below output:

-=<###>=- reading fw/active/System

8276785 bytes read
Adding 18179 symbols for standalone.
wrproxy:Instantiating /bd0 as rawFs, device = 0x10001

RTC Error: Real-time clock device is not working
usrSecurity: error adding default user '(null)' to the system (errno = 0x16)!
Instantiating /ram as rawFs, device = 0x50001
Formatting /ram for DOSFS
Instantiating /ram as rawFs, device = 0x50001
Formatting...Retrieved old volume params with %38 confidence:
Volume Parameters: FAT type: FAT32, sectors per cluster 0
0 FAT copies, 0 clusters, 0 sectors per FAT
Sectors reserved 0, hidden 0, FAT sectors 0
Root dir entries 0, sysId (null) , serial number 1c60000
Label:" " ...
Disk with 1024 sectors of 512 bytes will be formatted with:
Volume Parameters: FAT type: FAT12, sectors per cluster 1
2 FAT copies, 1010 clusters, 3 sectors per FAT
Sectors reserved 1, hidden 0, FAT sectors 6
Root dir entries 112, sysId VXDOS12 , serial number 1c60000
Label:" " ...
OK.

WARNING: Reset by alternate controller
12/07/20-06:51:49 (tNetCfgInit): NOTE: eth0: LinkDown event
12/07/20-06:51:49 (tNetCfgInit): NOTE: eth1: LinkDown event
12/07/20-06:51:49 (tNetCfgInit): NOTE: Network Ready
Current date: 12/07/20 time: 06:52:08

Send for Service Interface or baud rate change
12/07/20-06:52:08 (tRAID): SOD Sequence is Normal, 0 on controller A
12/07/20-06:52:08 (tRAID): NOTE: loading flash file: ECTDell
12/07/20-06:52:08 (tRAID): NOTE: unloading flash file: ECTDell
12/07/20-06:52:08 (tRAID): NOTE: Turning on tray summary fault LED
12/07/20-06:52:08 (tRAID): WARN: Error threshold exceeded: "Client_Corrupt_DB_Detected_On_Alt"
12/07/20-06:52:08 (tRAID): WARN: Error threshold exceeded: "Client_Corrupt_DB_Detected_On_Alt"
12/07/20-06:52:09 (tRAID): NOTE: Installed Protocols:
12/07/20-06:52:09 (tRAID): NOTE: Required Protocols:
12/07/20-06:52:09 (tRAID): NOTE: loading flash file: Fibre
12/07/20-06:52:09 (tRAID): NOTE: loading flash file: QLFC
12/07/20-06:52:11 (tRAID): NOTE: DSM: Current revision 7
12/07/20-06:52:11 (tRAID): NOTE: SYMBOL: SYMbolAPI registered.
12/07/20-06:52:12 (tRAID): NOTE: RCBBitmapManager total RPA size = 3656384512
12/07/20-06:52:12 (tRAID): NOTE: PCI DevIdl 0x1 is GEN3 MPT Adapter - Host side
12/07/20-06:52:12 (tRAID): NOTE: init: ioc: 0, PLVersion: 08-000-16-00
12/07/20-06:52:12 (tRAID): NOTE: init call sasConfigForTbm

Change TBM to Ext Mode ...
12/07/20-06:52:12 (sntpEventTask): NOTE: sntpEventHandler:SOD not complete; skipping NTP sync.
12/07/20-06:52:13 (tRAID): NOTE: Enable TBM Ext Mode OK
Cable Management is NOT initialized. Code: x00000000
12/07/20-06:52:13 (tRAID): NOTE: Initiating Host ioc:1 bringup
12/07/20-06:52:13 (tRAID): WARN: Check/Do IOC FW download chanIdx 8 isHost=1
12/07/20-06:52:13 (tRAID): NOTE: IOC HW detected, index:8 type:host chan:8 chipID:97 rev:2
12/07/20-06:52:15 (IOSched): NOTE: SAS Expander Added: expDevHandle:x9 enclHandle:x2 numPhys:30 port:0 ioc:0 channel:0 numEntries:22
12/07/20-06:52:15 (tRAID): NOTE: IOC:1 Firmware Version: 08-00-16-00
12/07/20-06:52:18 (tRAID): NOTE: Configuring IOC Diagnostic Trace for ioc:1
12/07/20-06:52:18 (tRAID): NOTE: Diagnostic Trace Post message successful
12/07/20-06:52:18 (tRAID): NOTE: IOC Snapshot Post message successful
12/07/20-06:52:18 (tSasExpChk): NOTE: sasExpanderCheck: alternate:0, start
12/07/20-06:52:18 (tSasExpChk): NOTE: Local Expander Firmware Version: 00.00.90.20
12/07/20-06:52:18 (tSasExpChk): NOTE: Enabling this controller's expander phys, channel=0
12/07/20-06:52:18 (tSasExpChk): NOTE: sasExpanderCheck: alternate:0, complete
12/07/20-06:52:18 (sasEnPhys): NOTE: sasEnableDrivePhysTask: 0 phys enabled on channel 0
12/07/20-06:52:19 (tSasDiscCom): NOTE: SAS: Initial Discovery Complete Time: 38 seconds since last power on/reset, 10 seconds since sas instantiated
12/07/20-06:52:20 (qlfcTask2): NOTE: exchanges 4101, target 2048
12/07/20-06:52:22 (qlfcTask3): NOTE: exchanges 4101, target 2048
12/07/20-06:52:23 (qlfcTask4): NOTE: exchanges 4101, target 2048
12/07/20-06:52:24 (qlfcTask5): NOTE: exchanges 4101, target 2048
12/07/20-06:52:24 (tRAID): NOTE: usbd: discovered device, drive:0
12/07/20-06:52:24 (tRAID): NOTE: CrushMemoryPoolMgr: platform and memory (CPU MemSize:4096), adjusted allocating size to 262144 for CStripes
12/07/20-06:52:26 (tRAID): SOD: Instantiation Phase Complete
12/07/20-06:52:25 (tRAID): NOTE: FTDC: Deferring format string copies due to pending restore
12/07/20-06:52:25 (tRAID): WARN: Error threshold exceeded: "Client_Corrupt_DB_Detected_On_Alt"
12/07/20-06:52:25 (tRAID): WARN: LEM:Client threshold Exceeded
12/07/20-06:52:25 (tRAID): NOTE: LEM: checkError(); Error threshold exceeded;SUSPENDED_LOCKDOWN;
12/07/20-06:52:25 (tRAID): WARN: sodLockdownCheck: sodSequence update: SuspLockdown
12/07/20-06:52:25 (tRAID): WARN: No attempt made to open Inter-Controller Communication Channels 0-0
12/07/20-06:52:25 (tRAID): WARN: Error threshold exceeded: "Client_Corrupt_DB_Detected_On_Alt"
12/07/20-06:52:25 (tRAID): WARN: Client reported lockdown: "Client_Corrupt_DB_Detected_On_Alt"
12/07/20-06:52:25 (tRAID): NOTE: Flashing tray summary fault LED
12/07/20-06:52:25 (tRAID): WARN: Suspending Start-Of-Day Task
12/07/20-06:52:42 (sntpEventTask): NOTE: sntpEventHandler:SOD not complete; skipping NTP sync.
12/07/20-06:53:12 (sntpEventTask): NOTE: sntpEventHandler:SOD not complete; skipping NTP sync.

Can you please check and let me know what I have missed.

Thank you in advance for your great support.

Hello again @mazen.ballat,

From the serial output I can see the error message: “WARN:  Error threshold exceeded: "Client_Corrupt_DB_Detected_On_Alt” 

If your system has a dual controller. Can you swap the MD controller and check it the other one is completing SodMain?

If so, follow this steps:

1. Insert controller which completes SodMain and allow start of day to complete, let’s call this controller-0 .

2. Insert other controller and we should expect that controller to held in a reset state. We will call this controller-1.

3. From controller 0 issued the shell command -> cmgrSetAltToOptimal and/or -> altResetRelease

4. Establish a serial connection to controller-1 ASAP.

5. Issue the command -> lemClearLockdown on controller-1

6. Issue the command -> bdbmClearPrimaryDatabase on controller 1.

7. This should complete SodMain on controller-1.

8. From MDSM, go to “Find Reports” and then “Clear Recovery Mode” to make array optimal.

If the other controller is not accessible neither. You wont be able to perform this operation and will have to call support and ask for an out of warranty ticket support so a level 2 checks the issue. 

Regards.

Hi @DiegoLopez 

Thank you for your reply. I have swapped the MD controllers and SodMain completed at controller-0 as the output below. I have inserted controller-1 and used the commands you recommended as below. However when I connected the serial to controller-1 I was not able to type anything.

reading fw/active/System

8276785 bytes read

Adding 18179 symbols for standalone.
wrproxy:Instantiating /bd0 as rawFs, device = 0x10001

RTC Error: Real-time clock device is not working
usrSecurity: error adding default user '(null)' to the system (errno = 0x16)!
Instantiating /ram as rawFs, device = 0x50001
Formatting /ram for DOSFS
Instantiating /ram as rawFs, device = 0x50001
Formatting...Retrieved old volume params with %38 confidence:
Volume Parameters: FAT type: FAT32, sectors per cluster 0
0 FAT copies, 0 clusters, 0 sectors per FAT
Sectors reserved 0, hidden 0, FAT sectors 0
Root dir entries 0, sysId (null) , serial number 1c60000
Label:" " ...
Disk with 1024 sectors of 512 bytes will be formatted with:
Volume Parameters: FAT type: FAT12, sectors per cluster 1
2 FAT copies, 1010 clusters, 3 sectors per FAT
Sectors reserved 1, hidden 0, FAT sectors 6
Root dir entries 112, sysId VXDOS12 , serial number 1c60000
Label:" " ...
OK.

Reset, Power-Up Diagnostics - Loop 1 of 1
3600 Processor DRAM
01 Data lines Passed
02 Address lines Passed
3300 NVSRAM
01 Data lines Passed
7102 QLOGIC EP8324 FC 1
01 Register Read Test Passed
02 Register Address Lines Test Passed
03 Register Data Lines Test Passed
7103 QLOGIC EP8324 FC 2
01 Register Read Test Passed
02 Register Address Lines Test Passed
03 Register Data Lines Test Passed
7106 QLOGIC EP8324 FC 3
01 Register Read Test Passed
02 Register Address Lines Test Passed
03 Register Data Lines Test Passed
7107 QLOGIC EP8324 FC 4
01 Register Read Test Passed
02 Register Address Lines Test Passed
03 Register Data Lines Test Passed
3900 Real-Time Clock
01 RT Clock Tick Passed
Diagnostic Manager exited normally.
12/08/20-07:11:24 (tNetCfgInit): NOTE: eth0: LinkDown event
12/08/20-07:11:24 (tNetCfgInit): NOTE: eth1: LinkDown event
12/08/20-07:11:24 (tNetCfgInit): NOTE: Network Ready
Current date: 12/08/20 time: 07:11:42

Send for Service Interface or baud rate change
12/08/20-07:11:42 (tRAID): NOTE: Set Powerup State
12/08/20-07:11:42 (tRAID): SOD Sequence is Normal, 0 on controller A
12/08/20-07:11:43 (tRAID): NOTE: loading flash file: ECTDell
12/08/20-07:11:43 (tRAID): NOTE: unloading flash file: ECTDell
12/08/20-07:11:43 (tRAID): NOTE: Turning on tray summary fault LED
12/08/20-07:11:43 (tRAID): NOTE: Installed Protocols:
12/08/20-07:11:43 (tRAID): NOTE: Required Protocols:
12/08/20-07:11:43 (tRAID): NOTE: loading flash file: Fibre
12/08/20-07:11:44 (tRAID): NOTE: loading flash file: QLFC
12/08/20-07:11:46 (tRAID): NOTE: DSM: Current revision 7
12/08/20-07:11:46 (tRAID): NOTE: SYMBOL: SYMbolAPI registered.
12/08/20-07:11:46 (tRAID): NOTE: RCBBitmapManager total RPA size = 3656384512
12/08/20-07:11:47 (tRAID): NOTE: PCI DevIdl 0x1 is GEN3 MPT Adapter - Host side
12/08/20-07:11:47 (tRAID): NOTE: init: ioc: 0, PLVersion: 08-000-16-00
12/08/20-07:11:47 (tRAID): NOTE: init call sasConfigForTbm

Change TBM to Ext Mode ...
12/08/20-07:11:47 (sntpEventTask): NOTE: sntpEventHandler:SOD not complete; skipping NTP sync.
12/08/20-07:11:48 (tRAID): NOTE: Enable TBM Ext Mode OK
Cable Management is NOT initialized. Code: x00000000
12/08/20-07:11:48 (tRAID): NOTE: Initiating Host ioc:1 bringup
12/08/20-07:11:48 (tRAID): WARN: Check/Do IOC FW download chanIdx 8 isHost=1
12/08/20-07:11:48 (tRAID): NOTE: IOC HW detected, index:8 type:host chan:8 chipID:97 rev:2
12/08/20-07:11:49 (IOSched): NOTE: SAS Expander Added: expDevHandle:x9 enclHandle:x2 numPhys:30 port:0 ioc:0 channel:0 numEntries:22
12/08/20-07:11:50 (tRAID): NOTE: IOC:1 Firmware Version: 08-00-16-00
12/08/20-07:11:53 (tRAID): NOTE: Configuring IOC Diagnostic Trace for ioc:1
12/08/20-07:11:53 (tRAID): NOTE: Diagnostic Trace Post message successful
12/08/20-07:11:53 (tRAID): NOTE: IOC Snapshot Post message successful
12/08/20-07:11:53 (tSasExpChk): NOTE: sasExpanderCheck: alternate:0, start
12/08/20-07:11:53 (tSasExpChk): NOTE: Local Expander Firmware Version: 00.00.90.20
12/08/20-07:11:53 (tSasExpChk): NOTE: Enabling this controller's expander phys, channel=0
12/08/20-07:11:53 (tSasExpChk): NOTE: sasExpanderCheck: alternate:0, complete
12/08/20-07:11:53 (sasEnPhys): NOTE: sasEnableDrivePhysTask: 0 phys enabled on channel 0
12/08/20-07:11:54 (tSasDiscCom): NOTE: SAS: Initial Discovery Complete Time: 39 seconds since last power on/reset, 10 seconds since sas instantiated
12/08/20-07:11:55 (qlfcTask2): NOTE: exchanges 4101, target 2048
12/08/20-07:11:56 (qlfcTask3): NOTE: exchanges 4101, target 2048
12/08/20-07:11:57 (qlfcTask4): NOTE: exchanges 4101, target 2048
12/08/20-07:11:59 (qlfcTask5): NOTE: exchanges 4101, target 2048
12/08/20-07:11:59 (tRAID): NOTE: usbd: discovered device, drive:0
12/08/20-07:11:59 (tRAID): NOTE: CrushMemoryPoolMgr: platform and memory (CPU MemSize:4096), adjusted allocating size to 262144 for CStripes
12/08/20-07:12:00 (tRAID): ERROR: FTDC metadata reinit: invalid, moved or RPA size changed
12/08/20-07:12:00 (tRAID): NOTE: DQ FTDC: Reinitialized DQ Trace History metadata: signature was invalid
12/08/20-07:12:00 (tRAID): NOTE: WLC Dedup: Reinitializing metadata: signature was invalid
12/08/20-07:12:00 (tRAID): SOD: Instantiation Phase Complete
12/08/20-07:12:00 (tRAID): NOTE: FTDC: Deferring format string copies due to pending restore
12/08/20-07:12:00 (tRAID): WARN: sodLockdownCheck: sodSequence update: CtlrLockdown
12/08/20-07:12:00 (tRAID): WARN: No attempt made to open Inter-Controller Communication Channels 0-0
12/08/20-07:12:00 (tRAID): NOTE: Failing The Alternate Controller
12/08/20-07:12:00 (tRAID): WARN: Alt Ctl Reboot:
Reboot CompID: 0x401
Reboot reason: 0x6
Reboot reason extra: 0x0
12/08/20-07:12:00 (tRAID): NOTE: holding alt ctl in reset
12/08/20-07:12:00 (tRAID): NOTE: Alt firmware running deasserted: 0 ms
12/08/20-07:12:00 (tRAID): NOTE: HealthCheck: Alt Ctl: 1 Reset_Failure, state: 0 Start
12/08/20-07:12:00 (tRAID): NOTE: LockMgr Role is Master
12/08/20-07:12:01 (tRAID): NOTE: WWN baseName 000400a0-98b5032b (valid==>SigMatch)
12/08/20-07:12:01 (tRAID): NOTE: spmEarlyData: No data available
12/08/20-07:12:01 (qlfcTask2): NOTE: FC/FCoE FW Version 7/5/1
12/08/20-07:12:01 (qlfcTask3): NOTE: FC/FCoE FW Version 7/5/1
12/08/20-07:12:01 (qlfcTask4): NOTE: FC/FCoE FW Version 7/5/1
12/08/20-07:12:01 (qlfcTask5): NOTE: FC/FCoE FW Version 7/5/1
12/08/20-07:12:01 (tRAID): SOD: Pre-Initialization Phase Complete
12/08/20-07:12:02 (utlTimer1): NOTE: fcnChannelReport ==> -2 -3 -4 -5
12/08/20-07:12:07 (utlTimer1): NOTE: fcnChannelReport ==> =2 =3 =4 =5
12/08/20-07:12:08 (ssmSpin): NOTE: FTDC: Freeze for pending restore from backup
12/08/20-07:12:11 (tRAID): WARN: CmgrLockdownException: CORRUPT DBM DATABASE DETECTED - LOCKDOWN Type=6
12/08/20-07:12:11 (tRAID): WARN: ACS: testCommunication: Domi Exception caught: IconSendInfeasibleException Error
12/08/20-07:12:11 (tRAID): NOTE: ACS: autoCodeSync(): Process start. Comm Status: 0
12/08/20-07:12:11 (tRAID): WARN: ACS: autoCodeSync(): Skipped since alt not communicating.
12/08/20-07:12:11 (tRAID): SOD: Code Synchronization Initialization Phase Complete
12/08/20-07:12:11 (tRAID): WARN: Lockdown case-DB CORRUPT. NVPS did not update from Sstor.
12/08/20-07:12:11 (tRAID): NOTE: Controller is in lockdown; skipping sendStagedFirmwareUpdateEvent()
12/08/20-07:12:12 (tRAID): SOD: Initialization Phase Complete
==============================================
Title: Disk Array Controller
Copyright 2013-2016 NetApp, Inc. All Rights Reserved.

Name: RC
Version: 08.25.08.60
Date: 08/18/2016
Time: 10:51:24 CDT
Models: 2701
Manager: devmgr.v1125api15.Manager
==============================================

12/08/20-07:12:12 (tRAID): NOTE: SYMbol available
12/08/20-07:12:12 (tRAID): WARN: *********************************************************
12/08/20-07:12:12 (tRAID): WARN: *********************************************************
12/08/20-07:12:12 (tRAID): WARN: ** **
12/08/20-07:12:12 (tRAID): WARN: ** WARNING!!!!!! **
12/08/20-07:12:12 (tRAID): WARN: ** **
12/08/20-07:12:12 (tRAID): WARN: ** This controller is locked down due to detected **
12/08/20-07:12:12 (tRAID): WARN: ** corruption in the primary database. **
12/08/20-07:12:12 (tRAID): WARN: ** In order to prevent any configuration loss, the **
12/08/20-07:12:12 (tRAID): WARN: ** backed up database must be restored. **
12/08/20-07:12:12 (tRAID): WARN: ** **
12/08/20-07:12:12 (tRAID): WARN: ** **
12/08/20-07:12:12 (tRAID): WARN: *********************************************************
12/08/20-07:12:12 (tRAID): WARN: *********************************************************
12/08/20-07:12:12 (tRAID): SOD: sodComplete Notification Complete
12/08/20-07:12:12 (tRAID): sodMain CtlrLockdown sequence finished, elapsed time = 30 seconds
12/08/20-07:12:12 (tRAID): sodMain complete
12/08/20-07:12:12 (tRAID): WARN: SOD lockdown sequence complete - suspending...
cmgrSetAltToOptimal
altResetRelease

Can you please take a look and let me know what am I doing wrong.

Thank you in advance for your support

Hello mazen.ballat,

When you tried to clear the lockdown did it say it was successful or did it fail?  If it stated successful when the controller rebooted was it showing the lockdown on the controller again?  If that is the case then you may need to replace the controller. 

Hello @DELL-Sam L 

I have tried as below can you please take a look and let me know if I am doing it wrong as I did not get any result. Excuse my lack of experience

reading fw/active/System

8276785 bytes read

Adding 18179 symbols for standalone.
wrproxy:Instantiating /bd0 as rawFs, device = 0x10001

RTC Error: Real-time clock device is not working
usrSecurity: error adding default user '(null)' to the system (errno = 0x16)!
Instantiating /ram as rawFs, device = 0x50001
Formatting /ram for DOSFS
Instantiating /ram as rawFs, device = 0x50001
Formatting...Retrieved old volume params with %38 confidence:
Volume Parameters: FAT type: FAT32, sectors per cluster 0
0 FAT copies, 0 clusters, 0 sectors per FAT
Sectors reserved 0, hidden 0, FAT sectors 0
Root dir entries 0, sysId (null) , serial number 1c60000
Label:" " ...
Disk with 1024 sectors of 512 bytes will be formatted with:
Volume Parameters: FAT type: FAT12, sectors per cluster 1
2 FAT copies, 1010 clusters, 3 sectors per FAT
Sectors reserved 1, hidden 0, FAT sectors 6
Root dir entries 112, sysId VXDOS12 , serial number 1c60000
Label:" " ...
OK.

Reset, Power-Up Diagnostics - Loop 1 of 1
3600 Processor DRAM
01 Data lines Passed
02 Address lines Passed
3300 NVSRAM
01 Data lines Passed
7102 QLOGIC EP8324 FC 1
01 Register Read Test Passed
02 Register Address Lines Test Passed
03 Register Data Lines Test Passed
7103 QLOGIC EP8324 FC 2
01 Register Read Test Passed
02 Register Address Lines Test Passed
03 Register Data Lines Test Passed
7106 QLOGIC EP8324 FC 3
01 Register Read Test Passed
02 Register Address Lines Test Passed
03 Register Data Lines Test Passed
7107 QLOGIC EP8324 FC 4
01 Register Read Test Passed
02 Register Address Lines Test Passed
03 Register Data Lines Test Passed
3900 Real-Time Clock
01 RT Clock Tick Passed
Diagnostic Manager exited normally.
12/15/20-05:32:32 (tNetCfgInit): NOTE: eth0: LinkDown event
12/15/20-05:32:32 (tNetCfgInit): NOTE: eth1: LinkDown event
12/15/20-05:32:32 (tNetCfgInit): NOTE: Network Ready
Current date: 12/15/20 time: 05:32:50

Send for Service Interface or baud rate change
12/15/20-05:32:50 (tRAID): NOTE: Set Powerup State
12/15/20-05:32:50 (tRAID): SOD Sequence is Normal, 0 on controller A
12/15/20-05:32:51 (tRAID): NOTE: loading flash file: ECTDell
12/15/20-05:32:51 (tRAID): NOTE: unloading flash file: ECTDell
12/15/20-05:32:51 (tRAID): NOTE: Turning on tray summary fault LED
12/15/20-05:32:51 (tRAID): NOTE: Installed Protocols:
12/15/20-05:32:51 (tRAID): NOTE: Required Protocols:
12/15/20-05:32:52 (tRAID): NOTE: loading flash file: Fibre
12/15/20-05:32:52 (tRAID): NOTE: loading flash file: QLFC
12/15/20-05:32:54 (tRAID): NOTE: DSM: Current revision 7
12/15/20-05:32:54 (tRAID): NOTE: SYMBOL: SYMbolAPI registered.
12/15/20-05:32:54 (tRAID): NOTE: RCBBitmapManager total RPA size = 3656384512
12/15/20-05:32:55 (tRAID): NOTE: PCI DevIdl 0x1 is GEN3 MPT Adapter - Host side
12/15/20-05:32:55 (tRAID): NOTE: init: ioc: 0, PLVersion: 08-000-16-00
12/15/20-05:32:55 (tRAID): NOTE: init call sasConfigForTbm

Change TBM to Ext Mode ...
12/15/20-05:32:55 (sntpEventTask): NOTE: sntpEventHandler:SOD not complete; skipping NTP sync.
12/15/20-05:32:56 (tRAID): NOTE: Enable TBM Ext Mode OK
Cable Management is NOT initialized. Code: x00000000
12/15/20-05:32:56 (tRAID): NOTE: Initiating Host ioc:1 bringup
12/15/20-05:32:56 (tRAID): WARN: Check/Do IOC FW download chanIdx 8 isHost=1
12/15/20-05:32:56 (tRAID): NOTE: IOC HW detected, index:8 type:host chan:8 chipID:97 rev:2
12/15/20-05:32:57 (IOSched): NOTE: SAS Expander Added: expDevHandle:x9 enclHandle:x2 numPhys:30 port:0 ioc:0 channel:0 numEntries:22
12/15/20-05:32:57 (IOSched): NOTE: SAS Expander Added: expDevHandle:xa enclHandle:x3 numPhys:30 port:1 ioc:0 channel:1 numEntries:22
12/15/20-05:32:58 (tRAID): NOTE: IOC:1 Firmware Version: 08-00-16-00
12/15/20-05:33:01 (tRAID): NOTE: Configuring IOC Diagnostic Trace for ioc:1
12/15/20-05:33:01 (tRAID): NOTE: Diagnostic Trace Post message successful
12/15/20-05:33:01 (tRAID): NOTE: IOC Snapshot Post message successful
12/15/20-05:33:01 (SasEvtWkr0): NOTE: Alt controller path up on channel:0 devH:xe expDevH:x9 phy:16 itn:2
12/15/20-05:33:01 (tSasExpChk): NOTE: sasExpanderCheck: alternate:0, start
12/15/20-05:33:01 (SasEvtWkr0): NOTE: Alt controller path up on channel:1 devH:xd expDevH:xa phy:12 itn:5
12/15/20-05:33:01 (tSasExpChk): NOTE: sasExpanderCheck: alternate:1, start
12/15/20-05:33:01 (tSasExpChk): NOTE: Local Expander Firmware Version: 00.00.90.20
12/15/20-05:33:01 (tSasExpChk): NOTE: Enabling this controller's expander phys, channel=0
12/15/20-05:33:01 (tSasExpChk): NOTE: sasExpanderCheck send icon connection up event channel:0
12/15/20-05:33:01 (tSasExpChk): NOTE: sasExpanderCheck: alternate:0, complete
12/15/20-05:33:01 (tSasDiscCom): NOTE: SAS: Initial Discovery Complete Time: 38 seconds since last power on/reset, 10 seconds since sas instantiated
12/15/20-05:33:01 (tSasDiscCom): NOTE: sasDiscoveryCompleteFirstTime send icon connection up event channel:0
12/15/20-05:33:01 (tSasDiscCom): NOTE: sasDiscoveryCompleteFirstTime send icon connection up event channel:1
12/15/20-05:33:02 (sasEnPhys): NOTE: sasEnableDrivePhysTask: 0 phys enabled on channel 0
12/15/20-05:33:02 (tSasExpChk): NOTE: Alternate Expander Firmware Version: 00.00.90.20
12/15/20-05:33:02 (tSasExpChk): NOTE: Enabling alternate controller's expander phys, channel=1
12/15/20-05:33:02 (tSasExpChk): NOTE: sasExpanderCheck send icon connection up event channel:1
12/15/20-05:33:02 (tSasExpChk): NOTE: sasExpanderCheck: alternate:1, complete
12/15/20-05:33:02 (sasEnPhys): NOTE: sasEnableDrivePhysTask: 0 phys enabled on channel 1
12/15/20-05:33:02 (qlfcTask2): NOTE: exchanges 4101, target 2048
12/15/20-05:33:04 (qlfcTask3): NOTE: exchanges 4101, target 2048
12/15/20-05:33:05 (qlfcTask4): NOTE: exchanges 4101, target 2048
12/15/20-05:33:06 (qlfcTask5): NOTE: exchanges 4101, target 2048
12/15/20-05:33:06 (tRAID): NOTE: usbd: discovered device, drive:0
12/15/20-05:33:06 (tRAID): NOTE: CrushMemoryPoolMgr: platform and memory (CPU MemSize:4096), adjusted allocating size to 262144 for CStripes
12/15/20-05:33:07 (tRAID): ERROR: FTDC metadata reinit: invalid, moved or RPA size changed
12/15/20-05:33:07 (tRAID): NOTE: DQ FTDC: Reinitialized DQ Trace History metadata: signature was invalid
12/15/20-05:33:07 (tRAID): NOTE: WLC Dedup: Reinitializing metadata: signature was invalid
12/15/20-05:33:07 (tRAID): SOD: Instantiation Phase Complete
12/15/20-05:33:07 (tRAID): NOTE: FTDC: Deferring format string copies due to pending restore
12/15/20-05:33:07 (tRAID): WARN: sodLockdownCheck: sodSequence update: CtlrLockdown
12/15/20-05:33:07 (tRAID): NOTE: Inter-Controller Communication Channels Opened
12/15/20-05:33:07 (tRAID): NOTE: Failing The Alternate Controller
12/15/20-05:33:07 (tRAID): WARN: Alt Ctl Reboot:
Reboot CompID: 0x401
Reboot reason: 0x6
Reboot reason extra: 0x0
12/15/20-05:33:07 (tRAID): NOTE: holding alt ctl in reset
12/15/20-05:33:08 (tRAID): NOTE: Alt firmware running deasserted: 0 ms
12/15/20-05:33:08 (tRAID): NOTE: HealthCheck: Alt Ctl: 1 Reset_Failure, state: 0 Start
12/15/20-05:33:08 (tRAID): NOTE: LockMgr Role is Master
12/15/20-05:33:08 (tRAID): NOTE: WWN baseName 000400a0-98b5032b (valid==>SigMatch)
12/15/20-05:33:08 (tRAID): NOTE: spmEarlyData: No data available
12/15/20-05:33:08 (qlfcTask2): NOTE: FC/FCoE FW Version 7/5/1
12/15/20-05:33:08 (qlfcTask3): NOTE: FC/FCoE FW Version 7/5/1
12/15/20-05:33:08 (qlfcTask4): NOTE: FC/FCoE FW Version 7/5/1
12/15/20-05:33:08 (qlfcTask5): NOTE: FC/FCoE FW Version 7/5/1
12/15/20-05:33:08 (tRAID): SOD: Pre-Initialization Phase Complete
12/15/20-05:33:10 (utlTimer1): NOTE: fcnChannelReport ==> -2 -3 -4 -5
12/15/20-05:33:14 (IOSched): NOTE: Alt controller path down on channel:0 devH:xe expDevH:x9 phy:4 itn:2
12/15/20-05:33:14 (IOSched): NOTE: Alt controller path down on channel:1 devH:xd expDevH:xa phy:0 itn:5
12/15/20-05:33:15 (utlTimer1): NOTE: fcnChannelReport ==> =2 =3 =4 =5
12/15/20-05:33:16 (ssmSpin): NOTE: FTDC: Freeze for pending restore from backup
12/15/20-05:33:18 (tRAID): WARN: CmgrLockdownException: CORRUPT DBM DATABASE DETECTED - LOCKDOWN Type=6
12/15/20-05:33:19 (tRAID): WARN: ACS: testCommunication: Domi Exception caught: IconSendInfeasibleException Error
12/15/20-05:33:19 (tRAID): NOTE: ACS: autoCodeSync(): Process start. Comm Status: 0
12/15/20-05:33:19 (tRAID): WARN: ACS: autoCodeSync(): Skipped since alt not communicating.
12/15/20-05:33:19 (tRAID): SOD: Code Synchronization Initialization Phase Complete
12/15/20-05:33:19 (tRAID): WARN: Lockdown case-DB CORRUPT. NVPS did not update from Sstor.
12/15/20-05:33:19 (tRAID): NOTE: Controller is in lockdown; skipping sendStagedFirmwareUpdateEvent()
12/15/20-05:33:19 (tRAID): SOD: Initialization Phase Complete
==============================================
Title: Disk Array Controller
Copyright 2013-2016 NetApp, Inc. All Rights Reserved.

Name: RC
Version: 08.25.08.60
Date: 08/18/2016
Time: 10:51:24 CDT
Models: 2701
Manager: devmgr.v1125api15.Manager
==============================================

12/15/20-05:33:19 (tRAID): NOTE: SYMbol available
12/15/20-05:33:19 (tRAID): WARN: *********************************************************
12/15/20-05:33:19 (tRAID): WARN: *********************************************************
12/15/20-05:33:19 (tRAID): WARN: ** **
12/15/20-05:33:19 (tRAID): WARN: ** WARNING!!!!!! **
12/15/20-05:33:19 (tRAID): WARN: ** **
12/15/20-05:33:19 (tRAID): WARN: ** This controller is locked down due to detected **
12/15/20-05:33:19 (tRAID): WARN: ** corruption in the primary database. **
12/15/20-05:33:19 (tRAID): WARN: ** In order to prevent any configuration loss, the **
12/15/20-05:33:19 (tRAID): WARN: ** backed up database must be restored. **
12/15/20-05:33:19 (tRAID): WARN: ** **
12/15/20-05:33:19 (tRAID): WARN: ** **
12/15/20-05:33:19 (tRAID): WARN: *********************************************************
12/15/20-05:33:19 (tRAID): WARN: *********************************************************
12/15/20-05:33:19 (tRAID): SOD: sodComplete Notification Complete
12/15/20-05:33:19 (tRAID): sodMain CtlrLockdown sequence finished, elapsed time = 29 seconds
12/15/20-05:33:19 (tRAID): sodMain complete
12/15/20-05:33:19 (tRAID): WARN: SOD lockdown sequence complete - suspending...
LemClearLockdown

 Thank you for your support

Hi @DiegoLopez 

I did just that and I got the below output

Press within 5 seconds: for Service Interface, for baud rate
Current date: 12/15/20 time: 10:51:31

Enter Password to access Service Interface (60 sec timeout):

Service Interface Main Menu
==============================

1)Display IP Configuration
2)Display 7-segment LED codes
Q)Quit Menu

Enter Selection: 1

Select Ethernet Port
==============================
1) Ethernet Port: 1
2) Ethernet Port: 2
Q) Quit

Enter Selection: 2

Ethernet Port Configuration
==============================
Device Name : gei1
IP Address : 192.168.40.2
Subnet Mask : 255.255.255.0
Gateway IP Address : 0.0.0.0
IP acquired using : Error in assignment
Speed : Speed Autonegotiation
Duplex Mode : Auto Autonegotiation
Link Status : Up

IPv6 Address Info
=================
Stateless Address Autoconfiguration enabled : 1

Link Local IP Address : ::

Routable Address 0 : ::
Routable Address 1 : ::
Routable Address 2 : ::
Routable Address 3 : ::
Routable Address 4 : ::
Routable Address 5 : ::
Routable Address 6 : ::
Routable Address 7 : ::
Routable Address 8 : ::
Routable Address 9 : ::

Router Address 0 : ::
Router Address 1 : ::
Router Address 2 : ::
Router Address 3 : ::
Router Address 4 : ::
Router Address 5 : ::
Router Address 6 : ::
Router Address 7 : ::
Router Address 8 : ::
Router Address 9 : ::

Service Interface Main Menu
==============================

1)Display IP Configuration
2)Display 7-segment LED codes
Q)Quit Menu

Enter Selection: 2

Code: 0E LF
Condition: Controller lockdown with limited use of SYMbol

Thank you in advance for your support

Honestly I don't think that is running the command properly. I only see the LemClearLockdown command one time and I do not see the result.

Can you please, follow the steps to launch commands on serial and let us know if you are able to follow them:

Here are the steps that you will need to get connected via serial cable.

1. Startup a terminal emulation program like PuTTY, Tera Term, Minicom, or HyperTerminal using terminal settings 115200-8-n-1.
2. Send a from the terminal shell. To do this, use within HyperTerminal or PuTTY, or use from within Minicom on Linux systems.
3. When prompted for input within five seconds, press for the service interface menu.
4. The user will then be prompted for a password. The password is: supportDell
5. Then you should have 4 choices and one of them is the option to change the IP address.
6. Repeat steps 1-5 on second controller if you have a second controller.

Regards.

Thank you @DiegoLopez do you mind giving me the steps for clearing the lockdown. Excuse my lack of experience on this matter.

Thank you in advance for your support 

Ok... so basically what happen is that your system is in a lockdown state, is that all configuration was not removed from your drives prior to them being removed.  When you powered up your MD3800f with the new drives the database is looking for the old drives & since it didn’t find them it is locking the database to protect the configuration.

The solution is to clear the lockdown on both controllers with the password reset cable that came with your MD3800f. If you are not able to clear the lockdown with the instructions provided. It is a signal that there might be another problem. Then you might need to open a service request with you local Dell support for a lvl 2 agent fixing the issue. Or even replace the controller.

Hope this helps.
Regards.

Thank you @DiegoLopez I thought there was another way. I did not work so I will open a support case.