Predicted Drive Failure

Hello JeffACC,

It sounds like you could have a puncture with all the pred fail drives. 

Virtual disk bad blocks are due to presence of unrecoverable bad blocks on one or more member physical disks.

First I recommend to confirm you have a validated backup.

You can search the controller log for "Puncture" see what shows up.

To get controller log: in Open Manage, expand Storage>select the controller.

Click Information/Configuration hyper link at top.

Use the ‘Controller Task’ drop down>

select Export Log, click Execute and Export on next page.

Log will be in Windows folder with file name LSI_####.log

Basic steps for clearing virtual disk bad blocks:

1. Get file level, not block level, backup of the data

2. Update the firmware of the hard drives

3. Run a Consistency Check on the virtual disk (via OMSA)

4. Run a patrol read and Clear the bad block table for the virtual disk (via OMSA)

   (Then monitor for any new bad blocks or other issues)

5. Run diagnostics: Boot to  F11 on Dell Splash screen, selecting  Boot Manager -> System Utilities -> Launch Dell Diagnostics.  Note any messages and continue testing.

You will need to replace any drive that still shows pred fail.

I'll supply some other reference information for you:

How to Fix a RAID Puncture

https://dell.to/3lJUe5S

Double Faults and Punctures in RAID Arrays

https://dell.to/3wMMExR

How to Handle Puncturing (Bad Blocks) on Virtual Disks for PowerEdge servers

https://dell.to/3wS62bh

Hello JeffACC,

That is good to see. Yes you can continue with the action plan.

Please let me know how it goes.

Hi Charles, Thanks for the detailed instructions.

Sorry for the delay, I was out of the office until this morning.

I ran the LSI log and scanned for "puncture" and found none.

Do I still continue on with your original instructions?

Will do, Thanks Charles.

Hi Charles,

Can I run the Drive Consistency checks during the day with people logged in to the server? 

JeffACC, 

Yes you can run it while live, you can also schedule it via OpenManage as well, as seen here. I say that as it is recommended to run about every 30 days, so it is easier to schedule the task. 

Let me know if this helps.

I did run and the three virtual drives are still displaying Predicted Drive Failure.  I will need to update the firmware on the drives and raid card but a bit leery  to install these firmware patches as I know they will need a reboot and am not 100% sure it will come back online.

I would suggest exporting a TSR, so that we can get a better idea as to what is occurring.

You can find the instructions for the TSR here.

After you export it, upload the TSR to upload. dell. com, then private message Charles and I the svc tag used to upload, so that we can locate it. 

Thanks.

SERVICE TAG (removed by moderator)

TSR Uploaded

Please let me know if you need anything further.  Thanks for all your help!

SERVICE TAG [removed by moderator]

TSR Uploaded

Hello JeffACC,

Thank you for the service tag. I will collect the log, review and update you.

Please note I removed the service tag from your post as that is private information.

Have you Run diagnostics: Boot to  F11 on Dell Splash screen, selecting  Boot Manager -> System Utilities -> Launch Dell Diagnostics.  Note any messages and continue testing.

The firmware will not fix a pred fail drive. It can prevent pred fail if it is applied before it reaches pred fail. You will need to replace any drive that still shows pred fail.

Make sure  you have a validated backup.

Thanks Charles,

Will do!

Hello JeffACC,

The log does not indicate a puncture and that is good.

You can continue with the action plan replacing each pred fail drive one at a time.

Run consistency check

Offline pred fail drive and replace

After rebuild completes run consistency check again and repeat for each pred fail drive.