This post is more than 5 years old
3 Posts
0
85457
Raid Permanently Degraded on SAS 6/IR.
Hello,
How do you update an array from "Permanently Degraded" to "Feeling Better" or "Online" on a Poweredge R310 with a SAS 6/IR controller? Is it automatic?
Here is the background:
I had a Poweredge 310 with SAS 6/IR go offline early this morning. The system would not boot (no bootable devices found, press F1 to try again). The SAS controller reported the array as "Permanently Degraded". Disk0 was offline. Disk1 was online, no failure predicted. I removed Disk0 and moved Disk1 to slot0. Rebooted the system and it booted into Windows. I put a new drive into slot1 and waited. Nothing happened. I went into OpenManage and the disk0 was listed as "Online, no failure predicted" and disk1 is "ready". I changed disk1 to a hot global spare and it began rebuilding.
Here is a summarized rundown from the Windows log. To be clear, no one was present at the store during this time:
10:02:25pm- SAS reports lost link on PHY0, Virtual Disk degraded, Physical disk device removed 0:0:0
10:02:27pm- Physical device re-inserted, rebuild started
11:56:57pm- Link Lost on PHY0
11:57:12pm- Link Restored PHY0
1:46:34am- Unexpected Sense Device 0:0:1
1:46:40am- Unrecoverable disk media error during rebuild. Disk 0:0:1
1:46:46am- Unrecoverable disk media error during rebuild. DIsk 0:0:0
1:49:50am- Bad block table is 80% full. Disk 0:0:0 and 0:0:1
1:50:05am- Bad block table is full. Physical disk 0:0:0 and 0:0:1
1:50:07am- Device 0:0:0 failed.
2:23:03am- \Device\Harddisk0\DR0 has a bad block
3:54:53am- Last entry. An unexpected media sense device 0:0:1.
theflash1932
7 Technologist
7 Technologist
•
16.3K Posts
0
December 10th, 2013 11:00
Disk 0 is bad and needs to be replaced, and disk 1 experienced write failures, causing data loss. Because the RAID controller cannot manage the missing data, it has marked the array as permanently damaged, because no matter what is done, it will never be able to fully rebuild a drive to make the array healthy again, because it knows it is missing some amount of data. Unfortunately, the array needs to be deleted and recreated with healthy disks, then restored from a backup.
dgeronimos
3 Posts
0
December 10th, 2013 11:00
Ah, a late night.
What event clued you in to write errors on disk1?
Thanks,
theflash1932
7 Technologist
7 Technologist
•
16.3K Posts
0
December 10th, 2013 11:00
1:46:40am- Unrecoverable disk media error during rebuild. Disk 0:0:1
However, if the array is permanently degraded, by definition, it means that the remaining disk's data is incomplete - this event is simply what the controller sent to Windows as to its inability to read/write some location on the disk (in a PERC controller log, it would actually give the address), and since that disks is the only remaining disk, if it can't be read, it no longer exists. Before creating a new array, I would run diagnostics on disk 1 just to be safe, as it may be bad too and just hasn't tripped the SMART sensors yet.
dgeronimos
3 Posts
0
December 10th, 2013 12:00
I planned on replacing both of them and I have system images. Replacing the array requires off-hours work, whereas rebuilding the array (as it is doing now) does not.
Thanks!