NetWorker: NetWorker Server Services Are Not Coming Up "NetWorker Is Unable To Continue Its Startup Sequence"
Summary: The NetWorker server software is deployed on a Red Hat Enterprise Linux Pacemaker (PCS) cluster. After upgrading NetWorker from 19.6.1.x (or earlier) to 19.7.0.0-19.7.0.3 or 19.8.0.0-19.8.0.1, NetWorker server services do not start. ...
Symptoms
NetWorker server is deployed on a Red Hat Enterprise Linux pacemaker (PCS) cluster.
NetWorker server is upgraded from 19.6.1.x (or earlier) to one of the impacted versions:
- 19.7.0.0 to 19.7.0.3
- 19.8.0.0 to 19.8.0.3
- 19.9.0.0 to 19.9.0.1
After upgrade, NetWorker server services do not start.
- The active node's
/nsr_share/nsr/logs/daemon.raw
84419 MM/DD/YYYY HH:mm:SS AM 5 5 0 3128739648 1735039 0 CLUSTER_NODE_HOSTNAME nsrd NSR critical NetWorker is unable to continue its startup sequence due to a server hostname change to "CLUSTER_VIRTUAL_IP". Please verify that the server's hostname and its aliases are properly represented in the local host database (eg. /etc/hosts)and DNS. 78366 MM/DD/YYYY HH:mm:SS AM 1 5 0 3128739648 1735039 0 CLUSTER_NODE_HOSTNAME nsrd NSR notice Beginning check of client attributes 78367 MM/DD/YYYY HH:mm:SS AM 1 5 0 3128739648 1735039 0 CLUSTER_NODE_HOSTNAME nsrd NSR notice Finished check of client attributes 83273 MM/DD/YYYY HH:mm:SS AM 1 5 0 3128739648 1735039 0 CLUSTER_NODE_HOSTNAME nsrd NSR notice Startup in process (step 3 of 5); checking resource types in the RAP database... 198264 MM/DD/YYYY HH:mm:SS AM 1 5 0 3128739648 1735039 0 CLUSTER_NODE_HOSTNAME nsrd NSR notice Populating InactiveClientsCache. 198266 MM/DD/YYYY HH:mm:SS AM 1 5 0 3128739648 1735039 0 CLUSTER_NODE_HOSTNAME nsrd NSR notice Populating InactiveClientsCache completed in 16 microseconds 202656 MM/DD/YYYY HH:mm:SS AM 1 5 0 3128739648 1735039 0 CLUSTER_NODE_HOSTNAME nsrd NSR notice Populating DNS cache... 203404 MM/DD/YYYY HH:mm:SS AM 1 5 0 3128739648 1735039 0 CLUSTER_NODE_HOSTNAME nsrd NSR notice Checking resource types in the RAP database... 95297 MM/DD/YYYY HH:mm:SS AM 5 3 17 3128739648 1735039 0 CLUSTER_NODE_HOSTNAME nsrd RAP critical Error encountered while re-signing lockbox '/nsr/lockbox/CLUSTER_HOSTNAME_FQDN/clb.lb': The Lockbox file was not found. 88783 MM/DD/YYYY HH:mm:SS AM 2 3 17 3128739648 1735039 0 CLUSTER_NODE_HOSTNAME nsrd RAP warningAttempt to allocate server storage node 'CLUSTER_HOSTNAME_FQDN' again. 199164 MM/DD/YYYY HH:mm:SS AM 1 5 0 3039790848 1735039 0 CLUSTER_NODE_HOSTNAME nsrd NSR notice Filling DNS cache with the active clients only. 95297 MM/DD/YYYY HH:mm:SS AM 5 3 17 3128739648 1735039 0 CLUSTER_NODE_HOSTNAME nsrd RAP critical Error encountered while re-signing lockbox '/nsr/lockbox/CLUSTER_VIRTUAL_IP/clb.lb': The Lockbox file was not found.
Cause
Changes applied in the impacted versions prevent NetWorker services from correlating the VIRTUAL_CLUSTER_IP with the hostname used to configure NetWorker. During service startup, services see the IP address instead of the hostname, causing a mismatch with the 'name' value in the server resource. It also tries to open the /nsr/lockbox using the IP instead of the previously configured Fully Qualified Domain Name (FQDN).
root@NWrhelNode1:~# lcmap type: NSR_CLU_TYPE; clu_type: NSR_LC_TYPE; interface version: 1.0; type: NSR_CLU_VIRTHOST; hostname: 192.1xx.xx.30; local: TRUE; owned paths: /nsr_share; root@NWrhelNode1:~# nsradmin NetWorker administration program. Use the "help" command for help, "visual" for full-screen mode. nsradmin> show name nsradmin> p type: nsr name: nwrhelclus.emclab.local; nsradmin> q root@NWrhelNode1:~# nslookup nwrhelclus.emclab.local Server: 192.1xx.xx.2 Address: 192.1xx.xx.2#53 Name: nwrhelclus.emclab.local Address: 192.1xx.xx.30
The lockbox and NetWorker Server Resource (NSR) 'name' fields work off of static assignment and not address resolution.
Resolution
This issue was addressed in NetWorker 19.8.0.4; however, 19.8 is End of Support Life (EOSL) as of November 11 2025. Upgrade to 19.9.0.2 or later for a code fix: https://www.dell.com/support/home/product-support/product/networker/drivers
See: NetWorker: Red Hat Pacemaker Cluster How to Upgrade NetWorker Server and Best Practices