MD3200i with 2both controllers in OFFLINE status

Environment Details:

Comprised of a Storage MD3200i connected to an expansion enclosure MD 1200 in a 2 PE R710 cluster environment with a VmWare ESXi hypervisor 4.1.0,262047.

Problem description:

The RAID slot 1 controller module no longer appears on the PowerVaultMSDM platform, so it was not possible to perform the OFFLINE replacement procedure.

A new controller was acquired and replaced after the system shutdown (servers, MD3200 and MD 1200 in that order).

After turning on the system (in the reverse order of shutdown) all the datastores and VMs worked correctly.

However, in visual inspection of the controllers it was possible to see that the new controller has the access activity LED on while the old controller is turned off, and the network activity link of the management port is not lit either.

When accessing the PowerVault Modular Disk Storage Management 10.70.G6.11 for verification of the array it informed that it needed to be updated.

The latest version of PVMDSM (11.25.0G06.0026) available on the Dell website as DELL_MDSS_Consolidated_RDVD_6_5_0_1 was installed on another device to verify the array.

After the automatic scan the manager located the array with alert that it is necessary to be verified.

When opening the AWM it was verified that the controller RAID 0 and 1 are as OFFLINE on the platform.

The MD 1200 firmware is 1.01.

I would like to know what can be done to re-enable the controllers and also regarding the EMM firmware update and the RAID controller if it can be done and what would be the best way to proceed in this case.

Below are the images of the errors presented and the information of both controllers, where it is possible to verify that the firmwares are divergent.

Raid Controller slot 0Raid Controller slot 1Guru Alerts

Hello Luiz Mello microtel,

Your issue is that you have firmware mismatch with your controllers.  Your controller in slot0 is running firmware from September 2010.  Your controller in slot 1 is running firmware September 2016.  There is a critical bridge update that is required on firmware version 07.75.28.60, that your controller in slot 0 needs so that the controllers can sync.  What you are going to need to do is to update your controller in slot 0 to match your controller in slot1.  If you remove the controller in slot1 your controller in slot 0 should go back online.  Once the controller in slot 0 is updated then you can insert the replacement controller back into your MD3200 & it should allow you to online the controller. 

Your EMM firmware will be updated when you upgrade the controller firmware on controller0.

Here are the steps to do the firmware for controller0.

Steps to update the firmware:

It is highly recommended to gather support bundle from the "Support" tab of the MD Storage Manager before performing any firmware upgrade operations. Also recommend getting another support bundle after the upgrade is complete.

Download the files from the links.

MD Storage Manager:

3.0.1.18 - ftp://ftp.dell.com/FOLDER00676032M/1/DELL_MDSS_Consolidated_RDVD_v3.0.1.18.iso

Hard Disk Drive Firmware Updates MD32xx

A06 - 7/25/2012 http://ftp.us.dell.com/FOLDER00704949M/1/HDD_Firmware_A06.zip

MD32xx SAS RAID Controller Firmware

Version 07.80.62.60 and NVSRAM N26X0-780890-001, A07, Released 7/2/2012

Requires Bridge Firmware 07.75.28.60 (included in zip) and MDSM 3.0.0.18 or newer

http://ftp.us.dell.com/FOLDER00636678M/1/MD3200_MD3220_Firmware_Package_07_80_62_60.zip

If your current firmware is not 7.75.28.60 A06 8/3/11 – you MUST step to 7.75.28.60 before going to 07.80.62.60. This firmware is included with the 07.80.62.60 download.

1. Gather support bundle in MDSM
2. WARNING: If you have a single controller PowerVault MD3200/MD3600 series storage array you must stop all I/O operations before starting the firmware upgrade.
3. Extract the firmware to folders and remember location.
4. Burn or mount the ISO for the resource cd.
5. Uninstall the MDSM from hosts (reboot required)
6. Install MDSM from resource DVD (reboot required)
7. Clear the Major Event Log.
8. Update to 07.75.28.60 if not already there, then update to latest firmwarIf you receive an error while checking the SPM database, ensure that you have an out of band management connection to both RAID controllers.

9. Verify connection and data.
10. Update the hard drive firmware.
11. Reboot stack and verify all is optimal.
12. Power down the server(s)
13. Power down the MD32x0(i)
14. Power down any attached storage (MD12xx)
15. Leave the power off for 2-3 minutes

Please allow for at least 60-85 minutes for the updates.  Time may vary depending on how long each reboot takes.

Please let us know if you have any other questions.

Hi Sam,

Thanks for your help, I really aprecciate it.

I have some questions about your procedural recommendations.

First, just to check my understanding about the initial procedure:

I must to shutdown the entire system (Servers, MD 3200i and MD1200), remove my controller in slot 1 and restart MD 3200i to verify that  my controller in slot 0 will respond, right?

Secondly, I know  I need to update the firmware 07.70.06.63 currently running on my controller in slot 0 to the firmware bridge 07.75.28.60 first.
You mentioned firmware Version 07.80.62.60 in the procedure, but the firmware on my controller in slot 1 already is 08.20.16.60. 
Can I upgrade from 07.75.28.60 to 08.20.16.60 directly or not? And if so, is there a compatibility matrix document for this firmware version (08.20.16.60) or can you tell me what HDD & EMM firmware  I should use it?

And finally  I've downloaded a file from http://ftp.us.dell.com/FOLDER00636678M/1/MD3200_MD3220_Firmware_Package_07_80_62_60.zip.

Would this file be only for a SAS storage or would it also work for an iSCSI Storage MD3200i?

Regards,

Hello Luiz Mello microtel,

In this case you can remove the controller while the system is up.  If your controller in slot0 doesn’t come back online, then you will need to reboot your MD3200i. You can go from 07.75.28.60 to 08.20.16.60 directly.  It is best to go to 07.80.62.60 first.  Sorry about that here is the link to the iSCSI version. http://ftp.us.dell.com/FOLDER00636659M/1/MD3200i_MD3220i_Firmware_Package_07_80_62_60.zip

Please let us know if you have any other questions.

Hi Sam,

Thanks for your answer.

Which EMM & HDD firmware version is compatible with firmware version 08.20.16.60?

Do you have this matrix compatibility document for firmware version 08.20.16.60 ?

Best Regards,

Hello Luiz Mello microtel,

The latest version of firmware for your MD1200 is 1.06.  Your EMM should be upgraded when your MD3200i controllers get their firmware upgrade.  Here is the link to the latest firmware for your HDD/SSD.

https://www.dell.com/support/home/us/en/19/drivers/driversdetails?driverId=R2GNY&osCode=W12R2&productCode=powervault-md3200i

Please let us know if you have any other questions.

Hi Sam,

I need to download MDSM version 10.70.G6.11, can you help me on this?

Tks in advance

Hi Sam,

I need to download MDSM version 10.70.G6.11, can you help me on this?

Tks in advance

Hello Luiz Mello microtel,

That version is no not available as that version has been removed.  The oldest version that is still available is 11.10.0A06.0005.

Please let us know if you have any other questions.