Failed drive / array won't rebuild

Question

I have a dell poweredge 1600sc server running windows 2000 server. there are 2 seagate hard disks mirrored in RAID 1. A technician in the office restarted the server to eject the tape from its drive and discovered this beeping sound upon restarting. I suspect he powered off the server to quickly restart - no proof of that. The beeping sound led us to find the array had failed. The disk in bay 0 on controller 0 is now reading "failed". The disk in bay 1 is online.

I did a rebuild straight away but the rebuild completes and the disk still reads "failed". I powered down the server and inserted a new disk into it but the rebuild takes place and the disk still reads "failed". I read the previous failed disk on another machine and can see all contents and I feel this disk is not at all faulty.

What could be the problem now as I need to get the virtual disk out of the degraded state. The system is as PERC Subsystem 1 - CERC ATA100/4ch Controller 0

Any similar problems faced by anybody please help as I fear the second good disk that is currently working might go down again.
Thanks.

BGetchel · Answer

Did you try (with both original drives installed) to DELETE the array (non-destructive) and define a NEW array, using the same drives and settings (ie: stripe size, etc)? Upon renewing the array, so NOT initialize -- just SAVE and reboot. Everything SHOULD operate properly afterward.   It isn't uncommon for a controller to 'lose' an array -- even though there is no physical problem. Deleting (do NOT initialize or clear individual drive) the array and specifying a new one with precisely the same stripe size almost always gets things back to normal.       Brendan

Nasak · Answer

Is there a good documentation I can use to carry out this task? The last informative help I received from a Dell documentation on Dell website took me about 1 day to locate. If you could, just point me in the right way - do I use CTRL-M or GUI Array Manager etc. I don't see a delete option in array manager for Array group but there is a delete button available for the virtual disk. NB - the old disk is obviously out of sync with the current working online one. Will there be issues I need to consider prior to the deletion of the array? Thanks.

BGetchel · Answer

I don't think there's actual documentation on this, at least I've never seen it.

Yes, use the BIOS-level "CNTL-M" to do this.

However, not only has this happened to me with just about every RAID controller I've ever used, both SCSI and ATA (including my current PERC3/DC and PERC4/DC boards) that I've become very comfortable performing this function -- even though it seems very counterintuitive on the surface. Basically it's only happening at the controller level, not at the drive level. The controller THINKS you're deleting the array but the metadata written to the drive(s) remains intact, so when you define a new array (do NOT use EASY Config option -- use NEW Config) identical to the old one (select drives, Enter, F10, then be sure to choose same stripe size).. Just save the new array as you normally would only do NOT initialize the new array.

Once you do that it should work correctly. It's a fairly common solution, so it's not just me ;-)

If you're in/near Connecticut I might be able to help you out. You can email me.

pcmeiners · Answer

This  is for the newest adapters but it is fairly detailed http://206.246.253.6/forum/topic.asp?TOPIC_ID=15270

Nasak · Answer

Since Drive 0 on Controller 0 is currently out of sync with disk1 on controller 1, will there be an automatic rebuild after I delete and recreate the array?
After bringing up the server will the status of disk 0 be still failed until I perform a rebuild on this same disk0? I just want to be sure I understand what I will be about to do. Thanks heaps.

BGetchel · Answer

It shouldn't rebuild at all, because you're deleting the array and creating a new one, so no "rebuild" should be involved -- it'll just think it's an entirely new array. Just make sure you don't initialize.

After you create the new array it *should* perform as it did prior to failure -- provided your hardware is functional.

Nasak · Answer

@Nasak wrote:
I have a dell poweredge 1600sc server running windows 2000 server. there are 2 seagate hard disks mirrored in RAID 1. A technician in the office restarted the server to eject the tape from its drive and discovered this beeping sound upon restarting. I suspect he powered off the server to quickly restart - no proof of that. The beeping sound led us to find the array had failed. The disk in bay 0 on controller 0 is now reading "failed". The disk in bay 1 is online.

I did a rebuild straight away but the rebuild completes and the disk still reads "failed". I powered down the server and inserted a new disk into it but the rebuild takes place and the disk still reads "failed". I read the previous failed disk on another machine and can see all contents and I feel this disk is not at all faulty.

What could be the problem now as I need to get the virtual disk out of the degraded state. The system is as PERC Subsystem 1 - CERC ATA100/4ch Controller 0

Any similar problems faced by anybody please help as I fear the second good disk that is currently working might go down again.
Thanks.

I believe I have now created a new array.
Upon rebooting, I get this message:
" Windows 2000 could not strat because the following file is missing or corrupt

\WINNT\SYSTEM32\CONFIG\SYSTEMced
startup options for WINDOWS 2000, press F8

You can attempt to repair this file by starting Windows 2000 Setup using the original setup floppy disk or CD-ROM
Select 'r' at the first screen to start repair.

I am worried as when I insert the setup disk and choose 'r' for repair it says it cannot find a hard disk. Did I miss something on the new array creation steps to cuase this.

pcmeiners · Answer

Searched in google for    SYSTEMced  , many references http://support.microsoft.com/?kbid=269075

BGetchel · Answer

Go to http://www.grc.com/ and pick up 'Spinrite.' Create the boot disk(s) and run the utility on your array before trying to repair/re-install Windows. If there are any lost or damaged files it should be able to recover them for you.   It sounds as though there may have been some data loss prior to the recreation somehow. Spinrite is an EXCELLENT program (but it takes many hours) and works very well on RAID arrays. We had a major hardware catastrophe on one of our RAID5 arrays but it was still able to retrieve some critical data (though it took 48 hours). It was a good sized array however (1TB), so the smaller the array the faster the process will be.     Brendan

Nasak · Answer

I am unable to get a copy of SpinRite since my organistaion still does not operate one of the payment methods required by GRC company. Any other tool you can suggest for me to at least boot to the recovery console of my server? Each time I run the WIN2K setup program from the install CD I keep getting the message :

"Setup did not find any hard disk drives installed in your computer - Make sure any hard disk drives are powered on and properly connected to your computer, and that any disk related hardware configuration is correct. This may involve running a manufacturer - supplied diagnostic or setup program. Setup cannot continue. To quit setup, press F3.

I suspect the registry hive is corrupt as per all the readings I have done - I need a solution to get to the recovery console. I'll point out that my ERD is not current.
Any help will be greatly appreciated. Thanks.

BGetchel · Answer

When booting with the Win2k CD are you pressing 'F6' to install the appropriate RAID controller driver first? If you don't it will not recognize any disks.

Nasak · Answer

I installed the drivers (taken from Dell Website) using F6 and was able to go to the recovery console. I carried out fix as per MS website tip for my previous error. Now I get stop error 0xc0000218 specific to SOFTWARE hive.
I suspect a total registry corruption.
I used Norton ghost on the 6th of August to ghost this working hard disk (disk to disk copy) of the array that was working but now cannot load windows. Will it be possible to restore registry files from the the ghosted disk?
Can I simply take this disk to another server and boot up to load windows and EXH2K or it is not as easy as it is in that with all the raid settings and all?

Suge · Answer

Setup:

Dell PowerEdge 600sc

4GB RAM

Windows Server Ent 2K3

Drives: 2 - 80GB OS mirrored

Drives: 4 - 250GB Data RAID 5

RAID Controller: CERC ATA100/4ch RAID

I recently had one of the OS disk crash. My vender sent me a 160GB replacement. In the process of sync'ing the OS drives the RAID stated beeping

When I checked I was getting the error:

Configuration of NVRAM & drives mismatch (normal mismatch).

Checking the card documentation... it stated the problem as being:

The configuration stored on the RAID card does not match the configuration stored on the drives.

The documentation stated to do the following:

When prompted, press a key to run the BIOS Configuration Utility. Select Configure—> View/Add Configuration from the Management

Menu. Use View/Add Configuration to examine both the configuration in the non-volatile random access memory (NVRAM), and the configuration

stored on the hard drives. Resolve the problem by selecting one of the configurations.

After checking both places (NVRAM and config stored on drives)... I found the NVRAM had 1 data drive up (0) and 3 data drives failed (1, 2, & 3).

The config stored on the drive showed 2 data drives (0 & 1) up and 2 drives down (2 & 3).

I removed the RAID card and allowed the 2 OS drives to sync (reestablish the mirror)... they did so with no problems. However I still have the issue with

the RAID card... Any assistance would be helpful...

Thanks

Shug

Worked like a charm.... Thanks Brendan.... Life saver..

PowerEdge HDD/SCSI/RAID

Failed drive / array won't rebuild

Was this post helpful?