dreading
1 Copper

Dell MD3260 - Reconstruction progress frozen

Hello,

We recently replaced a couple of failed drives in our MD3260 storage array, and have been experiencing issues with the rebuild process.  For context, our array contains thirty 4 TB HDDs arranged as follows:

  • Disk Group 1 (dg1, 50.934 TB RAID 5, degraded)
    • Virtual Disk 1 (dg1-vd1, 25.467 TB, degraded)
    • Virtual Disk 2 (dg1-vd2, 25.467 TB, degraded - reconstruction in progress)
  • Disk Group 2 (dg2, 50.934 TB RAID 5, optimal)
    • Virtual Disk 1 (dg2-vd1, 25.467 TB, optimal)
    • Virtual Disk 2 (dg2-vd2, 25.467 TB, optimal)

We replaced one drive (0-0-7) in dg1 and one drive (0-3-3) in dg2.  The drive we replaced in dg2 rebuilt without issue on both virtual disks, but the rebuild of dg1-vd1 is frozen and I originally believed it to be due to unreadable sectors but now I'm not sure...  To summarize the issues I've been seeing over this process:

  • Physical disk 0-0-7 has been replaced but is stuck rebuilding.  The physical disk is not showing up as "degraded", but has not left "replaced" for several weeks.  I tried re-seating the physical disk, but that seems to have caused the other virtual disks in the disk group to want to reconstruct too.
  • dg1-vd1 is showing as "degraded" on the storage manager, while dg1-vd2 is showing as "degraded - reconstruction in progress".  dg1-vd1 has had 8.5 hours left on its reconstruction for several weeks, and dg1-vd2 has been waiting on this.
  • However...  dg1-vd2 (and not dg1-vd1) is showing up with unreadable sectors in the unreadable sectors log, showing a logical failure on Physical Disk 0-0-8 and a physical failure on Physical Disk 0-1-1.  Both of these disks still show up as "Optimal".
  • More confusingly still, when I collected support data, it claimed the logical failure was on "Enclosure 0, Slot 7", and the physical failure was on "Enclosure 0, Slot 12".
  • Storage -> Disk Group -> Advanced -> Check Consistency... is greyed out on the management GUI.

We've been working on temporarily moving all data off the device for expected maintenance.  My primary question is with all data off of the virtual disks, would I be able to delete and re-create the offending virtual disks even with reconstruction operations pending/in-progress?  I've found conflicting information as to whether or not that will cause other problems.  If that is not possible, or if there's an easier method to fixing this than deletion/re-creation, what do you suggest as a next course of action?

Thanks in advance for any help you can provide

Mike

0 Kudos
1 Reply
Moderator
Moderator

Re: Dell MD3260 - Reconstruction progress frozen

Hello Mike,

First off when you replace a drive in an MD32xx you don’t want to reseat the drive if it stops rebuilding as that will cause the drive to be flagged as bad.  If you can send me the support bundle from your MD3260, then I can review it to see what the best way would be to get the virtual disk back online.  You may need to destroy and recreate the virtual disk and then restore data but will need to look at the logs to be sure.  I will send you a private message so that you can send me the support bundle.

Please let us know if you have any other questions.

DELL-Sam L
Dell | Social Outreach Services - Enterprise
Download the Dell Quick Resource Locator app today to access PowerEdge support content on your mobile device! (iOS, Android, Windows)

0 Kudos