NetWorker Admin UI Monitoring of Policy’s Show Details Does Not Complete with a Progress of 38 Percent

After leaving the NMC > Monitoring > Policies > Show Details for a particular Policy's Workflow open longer than the Service Token Timeout period, the access to the Show Details for this policy results in 38% progress (38 percent) which goes no further. Even after 10 minutes the 38% progress remains.

The NMC UI must End Task in Windows 2012.

Reviewing Windows Task Manager on the NetWorker for Windows server, which is also the NMC server and the authentication server, shows:

Java (TM) Web Launcher (2) (jp2launcher.exe) is using 10% to 45% CPU.
gstd.exe and java.exe are using 10% to 30% CPU, and fluctuating.
nsrexecd.exe is using between 2% and 15% CPU and fluctuating.

The gstd.raw file is reporting (notice the frequency in the timestamps):

C:\<directory>\Management\GST\logs>nsr_render_log -B -10 gstd.raw
144502 2017-08-13 11:51:35 AM  5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>:  Security token has expired.
144502 2017-08-13 11:51:35 AM  5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>:  Security token has expired.
144502 2017-08-13 11:51:35 AM  5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>:  Security token has expired.
144502 2017-08-13 11:51:35 AM  5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>:  Security token has expired.
144502 2017-08-13 11:51:35 AM  5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>:  Security token has expired.
144502 2017-08-13 11:51:35 AM  5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>:  Security token has expired.
144502 2017-08-13 11:51:35 AM  5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>:  Security token has expired.
144502 2017-08-13 11:51:35 AM  5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>:  Security token has expired.
144502 2017-08-13 11:51:35 AM  5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>:  Security token has expired.
144502 2017-08-13 11:51:36 AM  5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>:  Security token has expired.
144502 2017-08-13 11:51:36 AM  5 24 8 2600 2596 0 <GST_server_name> nsrd AUTHC critical Unable to set user privileges based on user token for SYSTEM on <GST_server_name>:  Security token has expired.

Stopped the gstd.exe service

C:\<directory>\Management\GST\logs>net stop gstd
.................
The EMC GST Service service was stopped successfully.

Checked the Windows tasklist to ensure that all executables had stopped:

C:\<directory>\Management\GST\logs>tasklist | findstr gst
 
C:\<directory>\Management\GST\logs>tasklist | findstr http
httpd.exe                     1836 Services                   0      6,296 K
httpd.exe                      596 Services                   0      6,568 K
 
C:\<directory>\Management\GST\logs>tasklist | findstr post
postgres.exe                  3296 Services                   0     12,176 K
postgres.exe                   988 Services                   0      4,756 K
postgres.exe                  2156 Services                   0      6,464 K
postgres.exe                  2100 Services                   0      5,536 K
postgres.exe                  1532 Services                   0      5,328 K
postgres.exe                  3648 Services                   0      6,684 K
postgres.exe                  3132 Services                   0      4,940 K

Used taskkill to end the errant httpd.exe and postgres.exe:

C:\<directory>\Management\GST\logs>TASKKILL /F /IM httpd.exe
SUCCESS: The process "httpd.exe" with PID 1836 has been terminated.
SUCCESS: The process "httpd.exe" with PID 596 has been terminated.
 
C:\<directory>\Management\GST\logs>TASKKILL /F /IM postgres*
SUCCESS: The process "postgres.exe" with PID 3296 has been terminated.
SUCCESS: The process "postgres.exe" with PID 988 has been terminated.
SUCCESS: The process "postgres.exe" with PID 2156 has been terminated.
SUCCESS: The process "postgres.exe" with PID 2100 has been terminated.
ERROR: The process "postgres.exe" with PID 1532 could not be terminated.
Reason: There is no running instance of the task.
ERROR: The process "postgres.exe" with PID 3648 could not be terminated.
Reason: The operation attempted is not supported.
ERROR: The process "postgres.exe" with PID 3132 could not be terminated.
Reason: The operation attempted is not supported.

C:\<directory>\Management\GST\logs>

Verified that the gstd processes are no longer listed in the tasklist:

C:\<directory>\Management\GST\logs>tasklist | findstr gst
C:\<directory>\Management\GST\logs>tasklist | findstr http
C:\<directory>\Management\GST\logs>tasklist | findstr post

Started the NMC services:

C:\<directory>\Management\GST\logs>net start gstd
The EMC GST Service service is starting.
The EMC GST Service service was started successfully.

THE ISSUE PERSISTS! The errant httpd.exe and postgres.exe are not the issue.

Working through instructions in the Security and Configuration Guide, stopping the gstd.exe, and ensuring all are stopped:

tasklist | findstr gst
tasklist | findstr http
tasklist | findstr post

With the GST service down, run:

gstauthcfg -c -t -h New_authentication_service_hostname -p port_number

Notice that the default port number is 9090.

Start the gstd service.

This message is now returned to the gstd.raw:

C:\<directory>\Management\GST\logs>nsr_render_log -B -10 gstd.raw
0 2017-08-13 1:02:34 PM  2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect]
0 2017-08-13 1:03:35 PM  2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect]
0 2017-08-13 1:04:36 PM  2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect]
0 2017-08-13 1:05:38 PM  2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect]
0 2017-08-13 1:06:39 PM  2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect]
0 2017-08-13 1:07:40 PM  2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect]
0 2017-08-13 1:08:41 PM  2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect]
0 2017-08-13 1:09:43 PM  2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect]
0 2017-08-13 1:10:44 PM  2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect]
0 2017-08-13 1:11:45 PM  2 0 0 3800 4240 0 <GST_server_name> gstd NSR warning Unable to get token for service account of NMC server from authentication service. Unable to authenticate user [Unauthorized access: The username or password is incorrect]

C:\<directory>\Management\GST\logs>

1. Log in to the NetWorker Management Console as administrator
2. Click the Setup Window icon
3. On the toolbar, click Setup
4. From the Setup menu, select Configure Service Account

5. This launches the Configure Service Account

6. Click Ok, which prompts back to the Setup Window. Nothing is returned when the NMC server updates the account with new credentials.
7. Check the gstd.raw log file, and the messages are stopped.
8. The Show Messages for the policy is now available.