Data Domain: How to understand and monitor disk reconstruction
Summary: This article explains how disk reconstruction works on Data Domain systems and how to monitor reconstruction progress after a disk failure.
Instructions
PURPOSE
Provide an understanding of the Data Domain (DD) Disk reconstruction process.
This article explains how disk reconstruction works on Data Domain systems and how to monitor reconstruction progress following a disk failure. Understanding this process helps ensure continued data protection and allows you to identify when additional actions are required.
Related Information
If you received a disk failure alert (EVT-STORAGE-00002) and are waiting for a replacement disk, see Data Domain: Failed, Absent, Unknown, or Powered-off Disk.
This article provides a disk replacement checklist, including steps to verify reconstruction activity and maintain system stability.
Understanding Data Domain Disk Reconstruction:
Data Domain systems use RAID 6 to protect data, even in the event of multiple disk failures. Monitoring disk reconstruction status is critical to ensure that the system continues to operate within protected limits.
-
RAID 6 Protection: Supports multiple simultaneous disk failures without data loss (RAID 6 has double parity protection per disk group).
-
Reconstruction Rules:
- A maximum of two disk reconstructions can occur simultaneously (across two disk groups).
- Exception: If two disks fail within the same Disk Group (DG), only one reconstruction can proceed at a time. The second will begin only after the first completes.
-
Global Sparing Behavior:
- Data Domain uses global spares across all external enclosures.
- If all spares in one enclosure are consumed and not replaced, the system pulls a spare from another enclosure with the matching disk size.
- This may result in a Disk Group spanning multiple enclosures. While it is ideal to keep RAID and spare disks within the same enclosure, this configuration does not impact performance.
-
Reconfiguration Guidance: If global sparing has occurred, contact support to verify system health and assist with disk enclosure reconfiguration.
-
Controller Disk Behavior: Head Unit (Controller) disks do not reconstruct to external shelf spare disks.
-
Best Practice: Replace failed disks as soon as possible to maintain optimal protection and system stability.
Monitoring Disk Reconstruction Status on Data Domain
To monitor disk reconstruction progress on a Data Domain system, use the following DD-CLI command:
# disk show state
OR
# storage show all
Command Output Includes:
- Current disk state
- Reconstruction activity
- Disk group undergoing reconstruction
- Percent completion of reconstruction
Special Case: If a second disk failure occurs in the same disk group (for example, dg1 or dg2), the output indicates a reconstruction pending status. This means that the second reconstruction does not begin until the first has completed.
Example :
sysadmin@hostname# disk show state
Enclosure Disk
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
--------- ---------------------------------------------
1 s . . .
2 . . R . . . . . . . . . . . F
3 F . . . . . . . . . . . . . R
4 . s . F . . . . . . . . . . .
--------- ---------------------------------------------
Legend State Count
------ ---------------------------- -----
. In Use Disks 44
s Spare Disks 1
R Spare (reconstructing) Disks 2
F Failed Disks 3
------ ---------------------------- -----
Total 49 disks
Disk Disks Progress Remaining
Group Reconstructing (%) (minutes)
----- -------------- -------- ---------
dg1 2.3 0 238
dg2 3.15 99 10
dg3 (Pending)
----- -------------- -------- ---------