Avamar: Symptom Code 22409 - A checkpoint validation (hfscheck) is overdue due to Incorrect mcserver.xml permissions

The event may be generated when hfscheck has been running successfully daily.

Error messages received by High Priority Events, ConnectEMC (DialHome), or Events Management in the Management Console Server (MCS) UI: 

22409 ERROR A checkpoint validation (hfscheck) of Avamar Server checkpoint data is overdue.

summary="A checkpoint validation (hfscheck) of Avamar Server checkpoint data is overdue."
type="ERROR"
code="22409"

Symptom: 22409, Desc: A checkpoint validation (hfscheck) of server checkpoint data is overdue

1. Log in to the Avamar Utiility Node as admin.

2. Load the admin keys. See Avamar: How to Log in to an Avamar Server and Load Various Keys for instructions on loading keys.

3. Verify the permissions of the mcserver.xml files.

It is possible that more than one variation of the mcserver.xml file exists in the directory. Duplicates could have been created during troubleshooting or other Management Console Server (MCS) related work.

Check the permissions of the files by typing the following:

ls -al /usr/local/avamar/var/mc/server_data/prefs/*mcserver.xml*

The correct permissions and ownership are as follows:

-rw-r--r-- 1 admin admin 

Sample output:

-rw-r--r-- 1 admin admin 60916 Sep 28 14:26 /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml
-rw-r----- 1 root  admin 60906 May  9  2023 /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml-2023-05-12
-rw------- 1 admin admin 61125 Jan 20  2023 /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml.2023-01-20

• • • • • Correct ownership and permissions for mcserver.xml
        • Incorrect ownership, but correct permissions for mcserver.xml-2023-05-12
        • Correct ownership, but incorrect permissions for mcserver.xml-2023-01-20

4. If the permissions or ownership is not correct, update accordingly using either the "chmod" or "chown" command as appropriate. Elevate to root if required.

Using the sample output above, the commands required would be:

chown admin:admin /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml-2023-05-12

chmod 644 /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml-2023-01-20

5. If permissions were changed, stop and start MCS per Avamar: How to Restart Management Console Server (MCS)

6. Perform a manual flush: 

mcserver.sh --flush

7. If the flush is successful, monitor the Avamar UI to ensure code 22409 no longer appears.

8. If the issue is still not resolved, return to Avamar: Symptom Code 22409: A checkpoint validation (hfscheck) of server checkpoint data is overdue (Resolution Path)