IDPA: ACM UI shows error "Backup Server viewuser user password is out of sync. Please update the latest password" on Backup Server Dashboard

ACM UI shows error "Backup Server viewuser user password is out of sync. Please update the latest password" on Backup Server Dashboard.

When user clicks on Password out of sync popup and attempts to update the password, UI shows an error "Avamar viewuser user test connection failed":



Open SSH to ACM as 'root' user and run the following command to confirm if this issue is caused due to MCSDK compatibility where ACM is unable to login to Avamar or Backup Server Postgres Database using MCSDK:
 

grep -i "SQLException.*viewuser.*authentication.*not supported" -A2 /usr/local/dataprotection/var/configmgr/server_data/logs/server.log

Example:

2021-03-30 10:45:25,304 INFO [https-jsse-nio-8543-exec-10]-restadapter.LoginService: Test connection failed for Avamar server viewuser
2021-03-30 10:49:08,451 ERROR [Thread-379779]-avadapter.AvamarUtil: SQLException while checking viewuser connection: org.postgresql.util.PSQLException: The authentication type 10 is not supported. Check that you have configured the pg_hba.conf file to include the client's IP address or subnet, and that it is using an authentication scheme supported by the driver.
2021-03-30 10:57:57,402 ERROR [https-jsse-nio-8543-exec-10]-avadapter.AvamarUtil: SQLException while checking viewuser connection: org.postgresql.util.PSQLException: The authentication type 10 is not supported. Check that you have configured the pg_hba.conf file to include the client's IP address or subnet, and that it is using an authentication scheme supported by the driver.

The above command points out a compatibility issue between ACM and Avamar/Backup Server MCSDK because of which ACM is unable is unable to fetch viewuser password status by logging into Postgres Database on Avamar Server. 

Open SSH to Avamar or Backup Server and verify if the viewuser password works on manual login attempt.

• Verify the viewuser user login works using the following command:

  • psql -p 5555 mcdb -U viewuser -h <IP or hostname of this Avamar server>

Note: If login works with the current viewuser password or common password, then we have the correct password. 

Once we have identified the correct password, click on the password viewuser user password out of sync pop up and attempt to update this password. If we see Test connection failure then proceed to the resolution section.

Compatibility issue between ACM and Avamar or Backup Server. This issue may happen if Avamar has been upgraded to higher version however ACM is yet to be upgraded. For example in case of IDPA DP8X models , Avamar is not part of IDPA Upgrade workflow and is upgraded as part of a separate workflow. 

Recommendation for IDPA DP8X Upgrades:

• IDPA Upgrade workflow from ACM must be started first. 
• Once the IDPA Upgrade workflow reaches the ESXi upgrade phase, Avamar upgrade can then be started to avoid such compatibility issues. 

Note: If the above recommendations are not followed, then that may cause such password out of sync errors.

1. Open SSH to Avamar Server/ Backup Server as 'admin user and verify the following details:

mcserver.sh --version

2. Open SSH to ACM as 'root' user and verify the version of IDPA:

rpm -qa | grep -i dataprotection

3. Verify the ACM version matches respective Avamar version as per the stack table provider below:
 

IDPA Version (ACM Version)	Avamar or Backup Server Version
2.2	7.5.1.101
2.3.x	18.2.0.134
2.4.x	18.2.0.134
2.5	19.1.0.38
2.6.x	19.3.0.149

Note: If the ACM and Avamar versions noted on Step 1 and 2 do not match the above table then this would explain the compatibility issue. 

4. Upgrade the rest of the IDPA or Avamar to match the IDPA Stack versions. If this is a DP8X appliance in the middle of an upgrade, then this error is expected and wait for the upgrade process to complete fully.