DLm attached Data Domain alert: Error communicating with ESRS server: esrs.server.name

DLm Consle and DD alerts show current command show:

p0-53 Fri Feb 5 12:00:03 2021 ERROR SystemMaintenance ESRSGateway=esrs.server.name EVT-SMTOOL-00003: Error communicating with ESRS gateway: esrs.server.name.

The configuration is not strictly needed for a DD behind a DLm, however if it's done incorrectly this alert occurs

To resolve this issue, validate the ip address of the ESRS server.  If the DLm is using ESRS and sending alerts via connectemc, you should be able to check the SRS/VE configuraiton via the DLm tools and use that value.   

Test ConnectEMC from the DLm Tools for the DD.  This may work, but capture the results anyway.  Next, connect to the DD 
1) support connectemc config show
ConnectEMC Configuration:
ESRS gateway IP/hostname: esrs.server.name
Registered device IP(s): 10.175.210.140
2) support connectemc test
Sending test message through ConnectEMC...

**** Failed to send test message through ConnectEMC: HTTP request error (401)
3) net hosts show
Hostname Mappings:
<same ip address as in the DLm tools for SRS/VE > -> esrs.server.name 
192.168.100.10 -> VTE1
192.168.100.20 -> VTE2
192.168.200.10 -> VTE1-DATA
192.168.200.20 -> VTE2-DATA
192.168.200.30 -> VTE3-DATA
192.168.200.40 -> VTE4-DATA
192.168.200.50 -> VTE5-DATA
192.168.200.60 -> VTE6-DATA
192.168.200.70 -> VTE7-DATA
192.168.200.80 -> VTE8-DATA
192.168.203.110 -> dlm.host.name

If the DD ESRS Server is different from the DLm, then it likely needs to be corrected.  In the above example, it's good.

4) net show set
port enabled state DHCP IP address netmask type additional setting
/prefix length
------- ------- ------- ---- -------------------------- --------------- ---- -------------------------------------------------------------------
ethMa no down ipv4 n/a n/a n/a
eth1a no down no n/a n/a n/a
eth1b no down no n/a n/a n/a
eth1c yes running n/a n/a n/a n/a bonded to veth1
eth1d yes running n/a n/a n/a n/a bonded to veth1
eth3a yes running n/a n/a n/a n/a bonded to veth2
eth3b yes running n/a n/a n/a n/a bonded to veth2
eth3c yes running n/a n/a n/a n/a bonded to veth2
eth3d yes running n/a n/a n/a n/a bonded to veth2
eth4a yes running n/a n/a n/a n/a bonded to veth2
eth4b yes running n/a n/a n/a n/a bonded to veth2
eth4c yes running n/a n/a n/a n/a bonded to veth2
eth4d yes running n/a n/a n/a n/a bonded to veth2
eth5a yes running n/a n/a n/a n/a bonded to veth3
eth5b yes running n/a n/a n/a n/a bonded to veth3
eth5c no down no n/a n/a n/a
eth5d no down no n/a n/a n/a
eth6a no down no n/a n/a n/a
eth6b no down no n/a n/a n/a
eth6c no down no n/a n/a n/a
eth6d no down no n/a n/a n/a
veth1 yes running no 192.168.100.111 255.255.255.0 n/a failover: eth1c,eth1d
fe80::260:16ff:fe9b:d56e** /64
veth1:1 yes running no 192.168.100.110 255.255.255.0 n/a
veth1:6 yes running no ef80::dd1 /64 n/a
veth2 yes running no 192.168.203.110 255.255.128.0 n/a lacp hash xor-L3L4: eth3a,eth3b,eth3c,eth3d,eth4a,eth4b,eth4c,eth4d
fe80::260:16ff:fe99:d2b4** /64
veth2:1 yes running no 192.168.203.111 255.255.128.0 n/a
veth2:2 yes running no 192.168.203.112 255.255.128.0 n/a
veth2:3 yes running no 192.168.203.113 255.255.128.0 n/a
veth2:4 yes running no 192.168.203.114 255.255.128.0 n/a
veth2:5 yes running no 192.168.203.115 255.255.128.0 n/a
veth2:6 yes running no 192.168.203.116 255.255.128.0 n/a
veth2:7 yes running no 192.168.203.117 255.255.128.0 n/a
veth3 yes running no 10.175.210.140 255.255.255.128 n/a lacp hash xor-L3L4: eth5a,eth5b
fe80::260:16ff:fe99:c8b4** /64

Above, we can see the veth3 ip address 10.175.210.140, which is a 10g, LACP bonded interface.  veth1 is an internal ip address in use by the DLm ( see the DLm VTE /etc/hosts file).  previously, we observed that the registered ip address for the esrs server is 10.175.210.140, or veth3.

5) sysadmin@dlm8500drmf-mgmt# net ping esrs.server.name interface veth3
PING uspgh-esrs-p2 (10.0.115.234) from 10.175.210.140 veth3: 56(84) bytes of data.
From dlm8500drmf-node0.amer.thermo.com (10.175.210.140) icmp_seq=2 Destination Host Unreachable
From dlm8500drmf-node0.amer.thermo.com (10.175.210.140) icmp_seq=3 Destination Host Unreachable
From dlm8500drmf-node0.amer.thermo.com (10.175.210.140) icmp_seq=4 Destination Host Unreachable
^C

the results from #5 shows that there is no route to the esrs server via veth3.

6) net ping esrs.server.name interface veth1
PING uspgh-esrs-p2 (10.0.115.234) from 192.168.100.111 veth1: 56(84) bytes of data.
64 bytes from uspgh-esrs-p2 (10.0.115.234): icmp_seq=1 ttl=54 time=11.4 ms
64 bytes from uspgh-esrs-p2 (10.0.115.234): icmp_seq=2 ttl=54 time=11.4 ms
64 bytes from uspgh-esrs-p2 (10.0.115.234): icmp_seq=3 ttl=54 time=11.3 ms

veth1 is able to reach the esrs server.

7)  net route show gateway
IPv4 Default Gateways
gateway IP source
------------- ------
192.168.100.8 static    <--  this should always be 192.168.100.8, the DLm VTE h/a IP address.

register the Data Domain veth1 IP address to the esrs gateway.

This command requires a Dell / EMC support website login / password.  Customer or Dell Technologies employee.  

8) support connectemc device register 192.168.100.111 esrs-gateway esrs.server.name
Enter Dell EMC Support username: mydellusername
Enter Dell EMC Support password: 

Device registered to ESRS gateway esrs.server.name.
ddhostname# support connectemc test
Sending test message through ConnectEMC...
Test message successfully sent through ConnectEMC.

If the above steps are not successful in resolving the issue, open a service request for the Data Domain unit.