IDPA: "dpacli -servicemode" command fails on DP4400 appliance with error "Failed to execute DPA CLI command: null"

When the "dpacli" command is issued to put the appliance in service mode, the command fails with the following error:

xxxxxxacm:/usr/local/dpatools/logs # dpacli -servicemode
Failed to execute DPA CLI command: null
usage: Data Protection Appliance Tools
 -servicemode   Shut down VMs (IDPA components) except ACM and ESXi. It's
                a prerequisite for FRU/CRU operations and FW update in
                service-mode

'dpacli.log' can be verified to review the error for this issue: (location: /usr/local/dpatools/logs/dpacli.log)
The following error can be seen in the dpacli.log:

2019-11-04T19:53:18.958Z ERROR [] [DPACLI-System-akka.actor.default-dispatcher-4] [][][][][] com.emc.dpa.dputils.DPAUtils - Exception occured while test connection of DPA app server {}
java.lang.NullPointerException: null
        at com.emc.dpa.dputils.persistence.ACMConfigUtils.getServiceIP(ACMConfigUtils.java:181)
        at com.emc.dpa.dputils.DPAUtils.testDPAAppServerOutOfSyncWhileShutDown(DPAUtils.java:77)
        at com.emc.dpa.dputils.DPApplianceUtils.testDPAOutOfSync(DPApplianceUtils.java:173)
        at com.emc.dpa.dputils.DPApplianceUtils.testOutOfSync(DPApplianceUtils.java:270)
        at com.emc.dpa.dputils.DPApplianceUtils.isOKShutdown(DPApplianceUtils.java:57)
        at com.emc.dpa.dputils.DPApplianceUtils.shutdownServices(DPApplianceUtils.java:87)
        at com.emc.dpa.ptresources.DPAResourceBuilder.shutdownServices(DPAResourceBuilder.java:977)
        at com.emc.dpa.ptresources.PTCLIService.shutdownServices(PTCLIService.java:98)
        at com.emc.dpa.common.akka.DPAHostAdapter.receiveDPAOpMessage(DPAHostAdapter.java:76)
        at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
        at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
        at scala.PartialFunction.applyOrElse(PartialFunction.scala:123)
        at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)
        at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
        at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
        at akka.actor.Actor.aroundReceive(Actor.scala:517)
        at akka.actor.Actor.aroundReceive$(Actor.scala:515)
        at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:180)
        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:588)
        at akka.actor.ActorCell.invoke(ActorCell.scala:557)
        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
        at akka.dispatch.Mailbox.run(Mailbox.scala:225)
        at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
        at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2019-11-04T19:53:18.958Z INFO [] [DPACLI-System-akka.actor.default-dispatcher-4] [][][][][] com.emc.dpa.dputils.DPAUtils - Checking connection with DPA datastore.
2019-11-04T19:55:20.380Z ERROR [] [main] [][][][][] com.emc.dpa.dpacli.DPACliUtils - Exception is thrown when parsing CLI command: {}

As the above logs indicate that a test connection to DPA Application Server and DPA Datastore Server is failing. Verify if the DPA is configured on this IDPA. Log in to the IDPA ACM UI and verify if DPA is configured on the "Data Protection Advisor" dashboard. Also the 'selskuconfig.xml' file can be verified to check if DPA is configured on this appliance: Location: /usr/local/dataprotection/var/configmgr/server_data/skuconfig/selskuconfig.xml head -50 /usr/local/dataprotection/var/configmgr/server_data/skuconfig/selskuconfig.xml Verify the DPA shows as unconfigured as below:

<ConfiguredProducts>             <DPS>true</DPS>             <DPA>false</DPA>             <Avamar>true</Avamar>             <DD>true</DD>             <DPC>true</DPC>             <DDCDR>true</DDCDR>             <IDPA_VAPP>true</IDPA_VAPP>         </ConfiguredProducts>

As per the above output, it is clear that DPA is not configured on this appliance however dpatools still does a test query to DPA which is an issue on DPATOOLS. This issue is fixed on dpatools version 2.0.0-17.

This is a known issue with DPATOOLS version 2.0.0-16 or lower and if fixed in DPATOOLS version 2.0.0-17. Install the new DPATOOLS rpm to resolve this problem.

1. Log in to the ACM ssh session using 'root user

2. Verify the dpatools version on the ACM using the following command:

   rpm -qa | grep -i dpatools

3. If the dpatools version is 2.0.0-16 or lower, Contact IDPA Support to get the newer version of dpatools installed.

4. Once the dpatools has been updated to version 2.0.0-17 or higher, retry the below command again:

   dpacli -servicemode

Also verify if the "dpacli -servicemode" fails due to DataProtectionConfig.xml file not being populated properly.
Resolution: KB Article