NetWorker: authc commands fail with "unable to find valid certification path"

• NetWorker server is deployed on a stand-alone (non-clustered) system.
• NetWorker auth commands (authc_config, authc_mgmt) fail with the following error reported:

[root@networker-mc bin]# authc_mgmt -u administrator -e find-all-users
Enter password: 
ERROR [main] (DefaultLogger.java:190) - Error executing command. Failure: I/O error on POST request for https://localhost:9090/auth-server/api/v1/sec/authenticate [localhost]: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target; nested exception is javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

• This issue happens regardless if using local NetWorker authentication, Active Directory (AD), or Lightweight Directory Access Protocol (LDAP) authentication.

There is a mismatch in the signature of the emcauthctomcat certificates. The emcauthctomcat is configured by default during NetWorker deployment. This certificate exists in three places:

Linux:

• /nsr/authc/conf/authc.keystore
• /opt/nsr/authc-server/conf/authc.truststore
• /opt/nre/java/latest/lib/security/cacerts

Windows:

• C:\Program Files\EMC NetWorker\nsr\authc-server\tomcat\conf\authc.keystore
• C:\Program Files\EMC NetWorker\nsr\authc-server\conf\authc.truststore
• C:\Program Files\NRE\java\jre#.#.#_###\lib\security\cacerts

[root@networker-mc bin]# ./keytool -list -keystore /opt/nre/java/latest/lib/security/cacerts -storepass changeit | grep -A1 emcauth 
emcauthctomcat, Oct 7, 2022, trustedCertEntry, 
Certificate fingerprint (SHA-256): 3B:18:1E:DF:39:ED:5B:4B:CF:9F:92:22:E8:D9:96:54:E0:21:A4:EB:06:D6:36:32:03:76:5E:CC:BA:B1:15:6B

[root@networker-mc bin]# ./keytool -list -keystore /opt/nsr/authc-server/conf/authc.truststore  | grep -A1 emcauthctom 
Enter keystore password:  
emcauthctomcat, Oct 7, 2022, trustedCertEntry, 
Certificate fingerprint (SHA-256): 3B:18:1E:DF:39:ED:5B:4B:CF:9F:92:22:E8:D9:96:54:E0:21:A4:EB:06:D6:36:32:03:76:5E:CC:BA:B1:15:6B

[root@networker-mc bin]# ./keytool -list -keystore /nsr/authc/conf/authc.keystore | grep -A1 emcauthctomcat
Enter keystore password: 
emcauthctomcat, Jun 29, 2022, PrivateKeyEntry, 
Certificate fingerprint (SHA-256): 93:97:0D:ED:DF:B1:73:62:D0:E1:95:C9:EB:67:3E:EE:4D:2E:55:9F:D7:9D:5E:FD:CE:81:E3:88:23:8E:0C:C9

Correct the certificate mismatch.

1. Create a copy of the existing keystore files:
   

Linux:

• /nsr/authc/conf/authc.keystore
• /opt/nsr/authc-server/conf/authc.truststore
• /opt/nre/java/latest/lib/security/cacerts

Windows:

• C:\Program Files\EMC NetWorker\nsr\authc-server\tomcat\conf\authc.keystore
• C:\Program Files\EMC NetWorker\nsr\authc-server\conf\authc.truststore
• C:\Program Files\NRE\java\jre#.#.#_###\lib\security\cacerts

2. On the NetWorker server, open an admin or root command prompt.

3. Stop NetWorker server services:
   Linux: nsr_shutdown
   Windows: net stop nsrd

4. Change the directory to the JRE \bin dir.

5. Using the following command syntax, delete the emcauthctomcat certificates from the keystore locations where mismatch is observed.

   Linux:
   ./keytool -delete -alias emcauthctomcat -keystore /path/to/keystore -storepass password

   Windows:
   keytool -delete -alias emcauthctomcat -keystore "C:\path\to\keystore" -storepass password

   NOTE: The Java keystore password, regardless if NRE or Oracle Java Runtime Environment (JRE), is changeit. The authc keystore is the user-defined keystore password set while using the NetWorker installation wizard (Windows) or /opt/nsr/authc-server/scripts/authc_configure.sh script (Linux).

Example:

[root@networker-mc bin]# ./keytool -delete -alias emcauthctomcat -keystore /opt/nre/java/latest/lib/security/cacerts -storepass changeit  

[root@networker-mc bin]# ./keytool -delete -alias emcauthctomcat -keystore /opt/nsr/authc-server/conf/authc.truststore
Enter keystore password:  
[root@networker-mc bin]#

6. The default emcauthctomcat certificate should exist in the following location:
   Linux: /nsr/authc/conf/emcauthctomcat.cer
   Windows: C:\Program Files\EMC NetWorker\nsr\authc-server\tomcat\conf\emcauthctomcat.cer

7. Import the default emcauthctomcat certificate to the keystore locations:
   Linux:
   ./keytool -import -alias emcauthctomcat -keystore /path/to/keystore -storepass password -file /nsr/authc/conf/emcauthctomcat.cer

   Windows:
   keytool -import -alias emcauthctomcat -keystore "C:\path\to\keystore" -storepass password -file "C:\Program Files\EMC NetWorker\nsr\authc-server\tomcat\conf\emcauthctomcat.cer"

Example:

[root@networker-mc bin]# ./keytool -import -alias emcauthctomcat -keystore /opt/nsr/authc-server/conf/authc.truststore  -file /nsr/authc/conf/emcauthctomcat.cer
Enter keystore password:  
Owner: CN=networker-mc.emclab.local, OU=NetWorker, O=DELL, L=Round Rock, ST=TX, C=US
Issuer: CN=networker-mc.emclab.local, OU=NetWorker, O=DELL, L=Round Rock, ST=TX, C=US
Serial number: bd1993a1
Valid from: Wed Jun 29 12:16:53 EDT 2022 until: Sun Jun 23 12:16:53 EDT 2047
Certificate fingerprints:
         SHA1: E8:7B:C8:DF:4D:24:57:C4:63:34:1F:E8:6D:AA:1F:84:79:61:92:26
         SHA256: 93:97:0D:ED:DF:B1:73:62:D0:E1:95:C9:EB:67:3E:EE:4D:2E:55:9F:D7:9D:5E:FD:CE:81:E3:88:23:8E:0C:C9
Signature algorithm name: SHA512withRSA
Subject Public Key Algorithm: 3072-bit RSA key
Version: 3

Extensions: 

#1: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
  DNSName: localhost
  IPAddress: 127.0.0.1
  DNSName: networker-mc.emclab.local
]

Trust this certificate? [no]:  y
Certificate was added to keystore
[root@networker-mc bin]# ./keytool -import -alias emcauthctomcat -keystore /opt/nre/java/latest/lib/security/cacerts -file /nsr/authc/conf/emcauthctomcat.cer   
Enter keystore password:  
Certificate already exists in keystore under alias <emcnwuiserv>
Do you still want to add it? [no]:  y
Certificate was added to keystore

8. Use the keytool -list command to confirm certificate the emcauthctomcat signatures match in each of the keystores:
   Linux: ./keytool -list -keystore /path/to/keystore -storepass password | grep -A1 emcauth
   Windows: keytool -list -keystore "C:\path\to\keystore" -storepass password

9. Start NetWorker services:
   Linux: systemctl start networker
   Windows: net start nsrd

10. Attempt to use an authc_config or authc_mgmt command:
    authc_config -u Administrator -e find-all-users

Example:

[root@networker-mc bin]# authc_mgmt -u administrator -e find-all-users
Enter password: 
The query returns 2 records.
User Id User Name           
1000    administrator       
1001    svc_nmc_networker-mc

NetWorker: authc commands on clustered Red Hat server report "unable to find valid certification path to the requested target."
NetWorker: How To Enable AUTHC DEBUG for Troubleshooting Purposes