Server failure Dell PowerEdge R720xd

Question

Hey, After restarting the server following some changes in the system (on other servers).I restarted the server and encountered several errors:All the disks started flashing amber (13 disks) after a few minutes some became solid green, and all the rest appear as failed state. (Before the restart there was no disk failure)No hardware changes were made to the server.There are 11 disks in RAID 6 (include hotspare)and 2 disks in RAID 1on which the operating system sits.In iDrac I received a message of - RAID 6 Degraded and RAID 1 disappeared. I wish you could help me or at least direct me, I don't have much experience with hardware problems of this magnitude Details of the server : PowerEdge R720xd BIOS Version 2.2.2 Firmware version 1.57.57 (Build 04) Lifecycle controller firmware 2.21.21.21 I would appreciate any help, advice or guidance on how to proceed from here. Regards, Osher.

DELL-Charles R · Accepted Answer

Hello sci-hpc,

I'm sorry to see the failed drives. I would recommend to consider if you have a backup and if you need this data but do not have a valid backup that you should contact a Data Recovery company.

Along with what Shine mentioned; the other very concerning thing I see is all the drives report 0GB in size (capacity).

Do the drives spin up? Are there any lights lit on the drives?

I can't imagine this happen with just a restart unless some previous issues have been occurring or you have experienced an issue that regular firmware updates could have mitigated. The system is very far behind in firmware and it could be some issue happened that may have been resolved with regular updates. I can't say for sure I'm just trying to give possibilities.

Have you reviewed the iDRAC System Event Log for any issues before this happened?

So after you consider the status of your backup you can try these steps below. I say to consider if you need data recovery as any troubleshooting could make it harder for data recovery if it is possible at all.

The array configuration is stored in two places; the drives and the controller.

You may try shut down, unseat all drives, boot to the controller and clear the configuration and shut down.

Then seat all the drives and boot into the Controller BIOS to see if it imported the arrays or has a foreign configuration that you can import.

Next steps you can update the system firmware and retry to import:

You will need to step up the BIOS and iDRAC together

iDRAC is thirteen revisions behind

2.63.60.62, A00	27 May 2019	Do 7th
2.61.60.60, A00	05 Dec 2018
2.60.60.60, A00	25 Jun 2018	Do 6th
2.52.52.52, A00	06 Mar 2018
2.50.50.50, A00	26 Sep 2017	Do 5th
2.41.40.40, A00	14 Nov 2016
2.40.40.40, A00	12 Oct 2016
2.30.30.30, A00	30 Mar 2016	Do 4th
2.21.21.21, A00	09 Nov 2015
2.20.20.20, A00	10 Sep 2015	Do 3rd
2.15.10.10, A00	25 Jun 2015
2.10.10.10, A00	05 Apr 2015	Do 2nd
1.66.65, A00	01 Dec 2014	Do 1st
1.57.57, A00	06 May 2014	You are here

BIOS is seven revisions behind

2.8.0	01 Oct 2019	Do 7th
2.7.0	27 Jun 2018	Do 6th
2.6.1	07 Mar 2018	Do 5th
2.5.4	17 Feb 2016	Do 4th
2.5.2	01 Apr 2015	Do 3rd
2.4.3	09 Oct 2014	Do 2nd
2.2.3	30 May 2014	Do 1st
2.2.2	06 Mar 2014	You are here

I can't tell about the PERC controller or the hard drive firmware.

You can find the firmware here : https://dell.to/3jIShZB

DELL-Shine K · Answer

Is both picture taken same time. On second picture I could see many drives failed and in first picture RAID 6 shown as degraded. RAID 6 will fail if more than 2 drives are failed.

You can perform AC Power Cycle (Remove the power cable wait for one minute and power on the server) and see whether it recover the physical disks.

sci-hpc · Answer

Hi,Thanks for the answer, I did it and the current situation is that all disks have become failed.Am I missing something? Or it is a more serious hardware failure. Regards, Osher.

sci-hpc · Answer

Thank you very much for all the help, I will try all the options..hope it works.

PowerEdge Hardware General

Was this post helpful?