Avamar: Data Domain Integration: Administrator UI Reports an incorrect Data Domain Status due to a time mismatch
Summary: A time mismatch between a Data Domain and the Avamar can result in "The Avamar server's time is ahead of DD's" in the Avamar Administrator UI.
Symptoms
The following appears in the Avamar Administrator UI:
The Avamar server's time is ahead of DD's.
The bottom-right corner of the Administrator UI can also display "Data Domain Unresponsive".
- There is no impact to backups to Avamar or Data Domain
- There is no impact to Avamar maintenance activities.
Cause
The Network Time Protocol (NTP) is not running or disabled.
Resolution
1. Ensure that all Avamar maintenance activities are completed without issue.
a. Log in to the Avamar server and load the keys (Avamar: How to Log in to an Avamar Server and Load Various Keys).
b. Run the status.dpn command and verify that all Avamar maintenance is completing without issue. See Avamar: How to understand the output generated by the status.dpn command for more information about the output.
The following is a sample output when the maintenance was completed without issue:
Last checkpoint: cp.20231030150317 finished Mon Oct 30 08:03:36 2023 after 00m 19s (OK)
Last GC: finished Mon Oct 30 07:00:38 2023 after 00m 02s >> recovered 632.51 KB (OK)
Last hfscheck: finished Mon Oct 30 08:03:04 2023 after 02m 00s >> checked 37 of 37 stripes (OK)
2. Check the date and time on Avamar and on the Data Domain. This output to see if there is a match.
a. If not already logged in, login to the Avamar server and load the keys (Avamar: How to Log in to an Avamar Server and Load Various Keys).
b. Check the data/time on Avamar using the "date" command:
date
Mon Oct 30 10:07:55 PDT 2023
c. Access the Data Domain server in Admin or SE mode (How To Access a Data Domain System from an Avamar Server).
d. Check the date and time on the Data Domain using the "date" command:
date
Thu Oct 30 09:34:41 PDT 2023
3. Compare the output.
The example above shows that the time of the Avamar grid and the Data Domain are different.
4. Verify that the Network Time Protocol (NTP) is configured and running on the Data Domain:
a. Check the configuration:
ntp show config
NTP is currently disabled.
No NTP servers configured; using multicast mode.
-- Or --
# Server
- ------------
1 10.162.204.1
2 10.166.1.1
3 10.166.1.2
- ------------
Showing NTP servers configured manually.
b. Check the status:
ntp status
**** NTP daemon is not running.
-- Or --
Status Disabled
Current Clock Time Unknown
Clock Last Synchronized Unknown
Clock Last Synchronized With Time Server Unknown
5. Perform the actions below depending on the output of the above.
- If NTP is disabled and not configured:
Verify with the customer that this is expected.
- If the NTP name servers are incorrect, correct them:
- Add one or more remote time servers:
ntp add timeserver <time server list>
- Delete one or more remote time servers:
ntp del timeserver <time server list>
- If NTP is not running, disable and re-enable it:
ntp disable
NTP disabled.
ntp enable
NTP enabled.
- If NTP is disabled, re-enable it:
ntp enable
NTP enabled.
6. Check the status and configuration again, and verify that it is configured correctly, and enabled.
ntp show config
ntp status
Status Enabled
Current Clock Time Thu Oct 30 18:25:23.682 2023
Clock Last Synchronized Unknown
Clock Last Synchronized With Time Server Unknown
-- Or --
Status Enabled
Current Clock Time Thu Oct 30 18:25:23.682 2023
Clock Last Synchronized Thu Oct 30 17:52:59.403 2023
Clock Last Synchronized With Time Server 10.10.10.240
7. Correct the time:
- If the time on the Avamar is incorrect, see Avamar: How to configure NTP on an Avamar multi-node grid using the asktime utility to correct the time on Avamar using the
asktimeutility.
- If the time on the Data Domain is incorrect:
a. Disable NTP:
ntp disable
NTP disabled.
b. Correct the time on the Data Domain using the "system set date" command:
system set date <MMDDhhmm[[CC]YY]>
Example:
system set date 10301013
Example output
** System date will be adjusted by 1939 seconds. This change can
potentially cause issues for applications running on this system.
Are you sure? (yes|no|?) [no]: yes
ok, proceeding.
Thu Oct 30 10:13:00 PDT 2023
c. Enable NTP:
ntp enable
NTP enabled.
If the issue is still not resolved, see Avamar: Troubleshooting Data Domain Integration Errors in the Avamar Administrator UI (Resolution Path)