Avamar: Data Domain Integration: Editing the DD fails when the MCS ddrcache is out of sync
Summary: Editing the Data Domain fails with "Failed to connect to Data Domain system" when the Management Console Server (MCS) ddrcache goes out of sync.
Symptoms
The following error is returned when trying to edit the Data Domain properties from Avamar (Both the UI and command-line):
Update of Data Domain system initiated...
Failed to edit Data Domain system.
Event Information:
Event Code: 30941
Category: APPLICATION
Severity: PROCESS
Summary: Failed to connect to Data Domain system.
The Management Console Server (MCS) logs (/usr/local/avamar/var/mc/server_log/mcserver.log*) on the Avamar Utility Node report the following exceptions:
08/03-13:26:59.00840 [RMI TCP Connection(9000)-192.168.99.113#133475] com.avamar.mc.util.MCException.logException
WARNING: java.lang.NullPointerException
at com.avamar.mc.datadomain.DataDomainService._updateDdr(DataDomainService.java:1225)
at com.avamar.mc.datadomain.DataDomainService.updateDdr(DataDomainService.java:1019)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
...
at java.lang.Thread.run(Unknown Source)
Also, the following DDR_POLLING_FAILURE events:
WARNING: DDR_POLLING_FAILURE: <event-source NodeID="test-ave-company.com" ProgramName="com.avamar.mc.datadomain.DdrCache" ddr-id="" ddr-name="" gsan-version="" hardware-id="UNKNOWN" source-hardware-id="UNKNOWN"/> ddrCommandFailureCount = 7
The Data Domain shows up as empty using the following command:
mccli dd show-prop --name=<IP of the Data Domain>
1,30943,Data Domain system does not exist. Cause
The Data Domain was added using the IP Address.
If there are DNS-related issues where the Data Domain was added using the IP address instead of the Fully Qualified Host Name (FQDN), the MCS DNS cache can get out of sync. This in turn throws the error while editing the Data Domain.
The Avamar Data Domain Integration Guide recommends that a Data Domain is NOT added using the IP address.
Resolution
Create a Service Request with the Dell Technologies Avamar Support team to investigate the issue. Reference this Knowledge Article when creating the Service Request.