Avamar: Data Domain Integration: Backups to DD fail with an SSL library error due to a time mismatch

Avamar backups to Data Domain (DD) fail with an "SSL library error": 

...
2019-11-20 18:39:04 avtar Info <19155>: - Establishing a connection via token to the Data Domain system with certificate authentication (Connection mode: A:2 E:2).
2019-11-20 18:39:07 avtar Warning <18125>: Calling DDR_OPEN_VIA_TOKEN returned result code:5341 message:SSL library error
2019-11-20 18:39:07 avtar Error <10542>: Data Domain server "data.domain.local" open failed DDR result code: 5341, desc: SSL library error
2019-11-20 18:39:07 avtar Error <10509>: Problem logging into the DDR server:'', only GSAN communication was enabled.
2019-11-20 18:39:07 avtar FATAL <17964>: Backup is incomplete because file "/ddr_files.xml" is missing
2019-11-20 18:39:07 avtar Info <10642>: DDR errors caused the backup to not be posted, errors=0, fatals=0
2019-11-20 18:39:07 avtar Info <12530>: Backup was not committed to the DDR.
2019-11-20 18:39:07 avtar FATAL <8941>: Fatal server connection problem, aborting initialization. Verify correct server address and login credentials.
2019-11-20 18:39:07 avtar Info <6149>: Error summary: 4 errors: 10542, 8941, 10509, 17964
2019-11-20 18:39:07 avtar Info <8468>: Sending wrapup message to parent
2019-11-20 18:39:07 avtar Info <5314>: Command failed (4 errors, 1 warning, exit code 10008: cannot establish connection with server (possible network or DNS failure)

Communication between Avamar and the Data domain is successful.
(The Additional Info section has some examples of how to test this.)

The Management Console Server (MCS) logs (/usr/local/avamar/var/mc/server_log/mcserver.log.0) report an "Unparsable date" error:

11/20-17:36:44.00780 [DdrCache_thread#505209] com.avamar.mc.datadomain.DdrSnmpSystemProperties.getTime
WARNING: Unparseable date: "" with format "EEE MMM dd HH:mm:ss X yyyy"
11/20-17:36:44.00780 [DdrCache_thread#505209] com.avamar.mc.datadomain.DdrSnmpInfo.getSystemPropertiesInfo
WARNING: Warning! com.avamar.mc.datadomain.DdrSnmpInfo:getSystemPropertiesInfo info current time mills number 0

The Avamar UI reports an alert related to Data Domain:

The Avamar server's time is ahead of DD's.

There are time differences between Avamar and the Data Domain.

For example:

The date command run on Avamar:

admin@avamar:~/#: date

Wed Nov 20 18:47:56 PST 2019 

The date command run on Data Domain:

ddboost@data.domain# date

Wed Nov 20 05:08:37 PST 2019

Ensure that the times on the Avamar and Data Domain match.

For more information about time settings, see Avamar: Data Domain Integration: Administrator UI Reports an incorrect Data Domain Status due to a time mismatch.

Ping:

admin@avamar:~/>: ping -c3 data.domain.local

PING data.domain.local (xxx.xx.xx.xx) 56(84) bytes of data.
64 bytes from xxx.xx.xx.xx: icmp_seq=1 ttl=64 time=0.328 ms
64 bytes from xxx.xx.xx.xx: icmp_seq=2 ttl=64 time=1.14 ms
64 bytes from xxx.xx.xx.xx: icmp_seq=3 ttl=64 time=0.671 ms

--- data.domain.local ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 0.328/0.715/1.146/0.335 ms

Telnet to ports 111, 2049, 2051 and 2052:

telnet data.domain.local 111

Trying xxx.xx.xx.xx...
Connected to data.domain.local
Escape character is '^]'.
^CConnection closed by foreign host.

telnet data.domain.local 2049

Trying xxx.xx.xx.xx...
Connected to data.domain.local.
Escape character is '^]'.
^CConnection closed by foreign host.

telnet data.domain.local 2051

Trying xxx.xx.xx.xx...
Connected to data.domain.local
Escape character is '^]'.
^CConnection closed by foreign host.

telnet data.domain.local 2052

Trying xxx.xx.xx.xx...
Connected to data.domain.local
Escape character is '^]'.
^CConnection closed by foreign host.

Secure Shell (ssh):

admin@avamar:~/>: ssh-agent bash
admin@avamar:~/#: ssh-add .ssh/ddr_key

Identity added: .ssh/ddr_key (.ssh/ddr_key)

admin@avamar:~/>: ssh ddboost@data.domain.local

EMC Data Domain Virtual Edition
Last login: Tue Nov 19 03:26:27 PST 2019 from 172.22.37.43 on pts/0


Welcome to Data Domain OS 6.2.0.5-608869
----------------------------------------
ddboost@data.domain#