NetWorker: Log in to NMC Fails with "An error occurred while validating user credentials"

The following characterizes the behavior observed with this issue:

• Unable to log in to NMC after rebooting the NetWorker server.
• When attempting to log in to the NMC, receive an error while validating credentials.

The following messages appear:

An error occurred while validating user credentials. Verify that Networker Authentication Service is running. [An error occurred, a low level system api call failure Server Message: Make sure that the server is running].

And/Or

An error occurred while validating user credentials. Verify that Networker Authentication Service is running. [Failed to connect to NetWorker_ServerName:9090; No Error. Server Message : Make sure that the server is running.]

This can be temporarily resolved by restarting ALL NetWorker services, but the issue reappears on the next reboot.

Issue may be intermittent.

The server is still completing operations. Visible in logs and through NetWorker server nsrwatch command.

This error can appear if there is a port conflict for one of the ports used by the AUTHC service. This is identifiable in the NetWorker server's \nsr\logs\daemon.raw log:

NetWorker_ServerName nsrctld NSR notice authservice: SEVERE: StandardServer.await: create[localhost:8009]:
NetWorker_ServerName nsrctld NSR notice authservice: java.net.BindException: Address already in use: JVM_Bind

And/Or

NetWorker_ServerName nsrctld NSR notice authservice: SEVERE: StandardServer.await: create[localhost:8005]:
NetWorker_ServerName nsrctld NSR notice authservice: java.net.BindException: Address already in use: JVM_Bind

NetWorker: How to use nsr_render_log

In this case, 8009 (Tomcat AJP Connector Port) and 8005 (Tomcat Security Shutdown) ports cannot be assigned as another application is using them. This can occur if there are one or more other applications installed on this server which use Apache Tomcat. The NetWorker server (nsrd) controls the authservice and cannot be user controlled. The following may be seen in the daemon.raw:

NetWorker_ServerName nsrctld NSR notice Daemon authservice terminated.

Option 1:
Adjust the ports being used by the other component so that there is no port conflict with Apache Tomcat used by NetWorker's Authentication Service.

Option 2:
Change the ports used by NetWorker's Tomcat instance by completing the following steps:

1. Stop the services on the NetWorker server complete the following:
   Linux: nsr_shutdown
   Windows: net stop nsrexecd /y

2. Identify a port not being used by any of the other running services:
   netstat -ab > netstat_ports.txt (e.g: 8089, 8087, and so on…)

3. Open the server.xml file under:
   Linux: /nsr/authc/conf/server.xml
   Windows: [NetWorker Install Path]\nsr\authc-server\tomcat\conf

   Note: NetWorker Install Path is to be replaced with the full path of your NetWorker installation directory.

4. Modify the file by replacing port 8009 to 8089 (chosen in step 2).
   For 8005 conflict, modify:
   <Server port="8005" shutdown="SHUTDOWN">
   TO:
   <Server port="8085" shutdown="SHUTDOWN">

   For 8009 conflict, modify:
   <!-- Define an AJP 1.3 Connector on port 8009 --> <Connector port="8009" protocol="AJP/1.3" redirectPort="8443"/>
   TO:
   <!-- Define an AJP 1.3 Connector on port 8089 --> <Connector port="8089" protocol="AJP/1.3" redirectPort="8443"/>

5. Save the file and reboot the NetWorker server.

   NOTE: Services can be restarted; however, the issue will not be confirmed as resolved until the next OS reboot:

   Linux: systemctl start networker
   Windows:
   net start nsrd
net start gstd (Only required if NMC is installed on NetWorker server)

6. Connect to the NMC.