Data Domain: How to Perform a General Health Check

Applies to:

• All Data Domain Operating System (DDOS) versions
• All current models 

12-Step Health Check:

1. Connect to the DD system by using SSH (for example PuTTY) as an administrative user.
2. Check that the Filesystem is enabled.
   • # system show serialno
# date
# filesys status
The filesystem is enabled and running.
3. Check that the running DDOS version is current and supported for the DD model.
   • # system show model
# system show version
   • Find the latest DDOS Releases: PowerProtect DD: Software Versions
4.  Address any active alerts that impact the health of the system
   • # alerts show current
   • Reference: Data Domain: How to Check Alerts on a Data Domain System.
5. Ensure that /data is below 90%.
   • To maintain expected performance levels, Data Domain recommends keeping 'use%' below 90%.
   • # df
     • Example output:

       • Active Tier:
         Resource           Size GiB    Used GiB   Avail GiB   Use%   Cleanable GiB*
         ----------------   --------   ---------   ---------   ----   --------------
         /data: pre-comp           -   7259347.5           -      -                -
         /data: post-comp   304690.8    251252.4     53438.5    82%           51616.1 
         /ddvar                 29.5        12.5        15.6    44%                -
         ----------------   --------   ---------   ---------   ----   --------------

   • Reference: Data Domain: How to resolve Capacity Issues 
6. Verify the status of the disks: 
   1. # disk show state
      • There should be no -  Failed (F), Error (E), Absent disks (A) or Reconstructing (R) disks
        • All disks MUST be 'In-use or 'Spare'
      • Example Output:

        • sysadmin## disk show state
          Enclosure   Disk
                      1  2  3  4  5  6  7  8  9  10 11 12 13 14 15 16
          ---------   ------------------------------------------------
          1           .  .  .  .  s  .  .  .  .  .  .  .
          2           .  .  .  .  .  .  .  .  .  A  .  .  .  .  S  R
          3           E  .  .  .  .  .  .  .  .  C  .  .  .  .  .  .
          ---------   ------------------------------------------------
          Legend   State          Count
          ------   ------------   -----
          .        In Use Disks   25
          s        Spare Disks    1
          R        Spare (reconstructing) Disks 1
          C        Copy Recovery Disks 1
          A        Absent Disks   1
          E        Exceeded Error Threshold
          ------   ------------   -----

        • Reference: Data Domain: How to Identify and Resolve Disk States
   2. Check disk reliability output to see if proactive disk replacement is needed.
      • Ensure that there are no disks with "Reallocated Sectors" above 1000 or increasing daily.
      • # disk show reliability-data
        • Example Output:

        • Disk Show Reliability-Data
          --------------------------
          Disk         ATA Bus   Reallocated   Temperature
           (enc/disk)   CRC Err   Sectors
          ----------   -------   -----------   -----------
          1.1          0         0             29 C   84 F
          1.2          0         0             29 C   84 F
          1.3          0         0             29 C   84 F
          1.4          0         0             27 C   81 F
          2.1          0         0             26 C   79 F
          2.2          0         0             25 C   77 F
          2.3          0         0             24 C   75 F
          2.4          0         0             24 C   75 F
          2.5         89         0             25 C   77 F
          2.6          0         0             25 C   77 F
          2.7          0         3156          24 C   75 F
          2.8          0         0             23 C   73 F
          2.9          0         0             24 C   75 F
          2.10         0         0             24 C   75 F
          2.11         0         0             23 C   73 F
          2.12         0         0             23 C   73 F
          2.13         0         0             25 C   77 F
          2.14         0         0             24 C   75 F
          2.15         0         0             22 C   72 F
          2.16         0         0             22 C   72 F

7. Test the SAS/Backend Storage communications on the connected SAS ports for 5 minutes.
   • If there is an error, check and reseat the cables or LCCs accordingly.
     • # disk port show summary
# enclosure show topology
# enclosure test topology <port#> 5 minutes
     • Reference: Data Domain: SAS Cable Configuration, Topology checks, and Testing
8. Address any reported misconfiguration:
   • # enclosure show misconfiguration
     • Example Output:

     • Enclosure Show Misconfiguration
       -------------------------------
       Memory Risers:
           No misconfiguration found.
       Memory DIMMs:
           No misconfiguration found.
       IO Cards:
           No misconfiguration found.
       CPUs:
           No misconfiguration found.
       Disks:
           No misconfiguration found.

9. Check for any errors relating to Replication (if configured)
   • # replication status
   • Reference: Data Domain: How to Check Replication Status
10. Check and confirm the status of VTL (if configured):
    • # vtl status  
    • Reference: Data Domain: Troubleshooting VTL Target Visibility Issues
11. Check and confirm the status of High Availability (HA) -- if configured:
    • # ha status
# ha status detailed
    • Health check for Data Domain HA (DDHA) appliances
12. Run Hardware and system health checks (DDOS >= 8.3.x)
    • # support healthcheck hardware
# system health check
    • Address any issues accordingly.
    • Example Output:

      • HARDWARE Health Check Summary:
        +-------------------+--------+
        | Component         | Status |
        +-------------------+--------+
        | Storage Disk      | PASS   |
        | Power-Supply Unit | PASS   |
        | FAN               | PASS   |
        | SAS Controller    | PASS   |
        | QAT               | PASS   |
        | NvRAM             | PASS   |
        | DIMMs             | PASS   |
        | IO Cards          | PASS   |
        | CPU               | PASS   |
        | NIC H/W Errors    | PASS   |
        +-------------------+--------+

      • PowerEdge-based Data Domain systems (for example: DD6400, 6900, 9910) systems can also connect to iDRAC to check system hardware status:
        • Connect to iDRAC and check system status and health -  gather a TSE Log (if necessary).
        • Reference: Data Domain: How to gather TSR logs

In all health check cases, once the steps above have been completed, reboot the DD system.

• # system reboot

After the system has rebooted, check:

• # alerts show current 
  • There should be no new or outstanding alerts
• # filesys status
  • The filesystem should be enabled and running.

If any further assistance is required, open a Service Request with your contracted Support Provider.