NetWorker Server's or Client's service disruption in HA (High Availability) cluster nodes that are installed with RHEL (Red Hat Enterprise Linux) 8.2, 8.3, 8.4, 8.6 operating system versions.

73248 MM/DD/YYYY HH:MM:SS AM/PM  5 5 0 1700550464 48537 0 NODE_HOSTNAME nsrd NSR critical Can't start nsrd because /nsr/res (/nsr) is local, and NetWorker is configured as a cluster server. Use cluster manager to check NetWorker service status.

NetWorker uses RHEL command "pcs" in the script '/usr/sbin/lcmap'.  This script is used to identify the cluster logical IP address and shared path of the nws cluster resource. Running the command lcmap should return "owned paths" which will be the shared /nsr location; however in this case it does not:

[root@NODE_HOSTNAME ~]# lcmap
type: NSR_CLU_TYPE;
clu_type: NSR_LC_TYPE;
interface version: 1.0;

[root@NODE_HOSTNAME ~]# 

There are changes to current versions of pcs commands used. As a result, the NetWorker script that uses the command was unable to determine path-ownership in a cluster causing the failures.
 
 

A code fix will be available in NetWorker: 

• 19.7.0.4 (May 5th 2023)
• 19.8.0.2 (May 22nd 2023)
• 19.9.0.0 (May 22nd 2023)

Workaround:

Modify the /usr/sbin/lcmap file on each node to use the correct command syntax.

1. Open the lcmap file: sudo vi /usr/sbin/lcmap

a. Modify the syntax of the GRPS line:

GRPS=`pcs resource show 2>&1 | egrep "^\s*Resource Group:" | awk '{print $3}'`

GRPS=`pcs resource group list 2>&1 | awk '{print $1}' | tr -d ":"`

Replace:

RESOURCES=`pcs resource show ${grp} | egrep "^\s*Resource:" | awk '{print $2}'`
      TMP=`pcs resource show ${res} | egrep "^\s*Attributes:.*ip="`
      TMP=`pcs resource show ${res} | egrep "^\s*Attributes:.*device="`
                        
RESOURCES=`pcs resource show ${Linkedres} | egrep "^\s*Resource:" | awk '{print $2}'`
      TMP=`pcs resource show ${res} | egrep "^\s*Attributes:.*ip="`
      TMP=`pcs resource show ${res} | egrep "^\s*Attributes:.*device="`

With:

RESOURCES=`pcs resource config ${grp} | egrep "^\s*Resource:" | awk '{print $2}'`
      TMP=`pcs resource config ${res} | egrep "^\s*Attributes:.*ip="`
      TMP=`pcs resource config ${res} | egrep "^\s*Attributes:.*device="`
                        
RESOURCES=`pcs resource config ${Linkedres} | egrep "^\s*Resource:" | awk '{print $2}'`
      TMP=`pcs resource config ${res} | egrep "^\s*Attributes:.*ip="`
      TMP=`pcs resource config ${res} | egrep "^\s*Attributes:.*device="`

2. Save the file.
3. Confirm that the shared location is returned by running: sudo lcmap

[root@NODE_HOSTNAME ~]# lcmap
type: NSR_CLU_TYPE;
clu_type: NSR_LC_TYPE;
interface version: 1.0;

type: NSR_CLU_VIRTHOST;
hostname: IP_ADDRESS;
local: TRUE;
owned paths: /nsr_share;

4. Clean up the pcs resource: sudo pcs resource cleanup nws
5. Confirm all resources start up: sudo pcs resource status

[root@NODE_HOSTNAME ~]# pcs resource status
  * Resource Group: NW_group:
    * fs        (ocf::heartbeat:Filesystem):     Started NODE_HOSTNAME
    * ip        (ocf::heartbeat:IPaddr):         Started NODE_HOSTNAME
    * nws       (ocf::EMC_NetWorker:Server):     Started NODE_HOSTNAME

• NetWorker: bootstrap backup and authc commands fail on Clustered Linux server after upgrade to 19.7.0.x or later