NVP vProxy: VM Recovery Not Possible After Setting RDZ

The NetWorker VMware Protection (NVP) solution was configured initially without using a NetWorker Restricted Datazone (RDZ). The NVP solution was changed to use an RDZ configuration.

The symptoms of this issue may include:

• NetWorker server is 19.10.x or later. After 19.10 it became possible to modify the NSR Hypervisor resource (vCenter) to be part of an RDZ.

nsradmin
show restricted data zone; name
print type: nsr hypervisor

[root@nsr ~]# nsradmin
NetWorker administration program.
Use the "help" command for help, "visual" for full-screen mode.
nsradmin> show restricted data zone; name
nsradmin> print type: nsr hypervisor
                        name: vcsa.amer.lan;
        restricted data zone: VMware_Restricted;
nsradmin>

• VM save sets which predate the RDZ fail during restore operations:

nsrvproxy_recover: Virtual machine recovery failed: Invalid save set 'SSID'.  It does not for Restricted Data Zone 'RDZ_NAME'.

Using the ssid in the recovery error message we can see that the save set does include the RDZ details:

mminfo -avot -q vmname=VM_NAME -r savetime,ssid,rdz

Example:

[root@nsr ~]# mminfo -avot -q vmname=rhel-client02.amer.lan -r savetime,ssid,rdz
  date   ssid       rdz
11/12/2024 3224635442
11/13/2024 4097137073
11/14/2024 3895896879
11/15/2024 4097309873
11/16/2024 3879292464
11/17/2024 3661275056                  
11/18/2024 3476794389 VMware_Restricted
11/18/2024 4265341233 VMware_Restricted

NetWorker is working as designed. The following statement is from the NetWorker VMware Integration Guides:

• VMware 19.10 VMware Integration Guide (Rev. 04)
• VMware 19.11 VMware Integration Guide (Rev. 02)

Once a hypervisor is configured to be part of an RDZ, it becomes impossible to recover the backup save sets created before
the vCenter was added to the RDZ without removing the RDZ association from the vCenter. Therefore, it is recommended
to perform a full backup after assigning a vCenter to an RDZ to ensure that all data is properly backed up and recoverable.

 The NetWorker VMware Integration Guide is available through https://www.dell.com/support/home/product-support/product/networker/docs

Make sure that the Save set to be recovered match the Restricted Datazone (RDZ) vCenter. The code is designed to expect a match in the ownership of the vCenter and it has moved (from global admin ownership to RDZ), and it is blocking it to prevent a data breach.

The affected versions are NetWorker 19.10.x onwards.

Workaround

To perform the restore the configuration must be reverted to not include the RDZ. 

1. From the NetWorker Management Console (NMC), go to Protection->Groups. 
2. Open the VMware protection groups associated with the RDZ.
3. Make note of the configuration of any NetWorker group configured with the RDZ. 

• VMs selected in group, or any NSR rule applied to workflow.
• Policy-Workflow the group is assigned to.

4. From Protection->Policies, expand the policy containing the workflow. Open the workflow policy and remove the group which contained the RDZ:

5. Return to Protection->Groups and delete the groups containing the RDZ.
6. From a nsradmin prompt on the NetWorker server, update the vCenter to not include the RDZ.

nsradmin
print type: nsr hypervisor
update restricted datazone:
y
quit

[root@nsr ~]# nsradmin
NetWorker administration program.
Use the "help" command for help, "visual" for full-screen mode.
nsradmin> p type: nsr hypervisor
                        type: NSR hypervisor;
                        name: vcsa.amer.lan;
                     version: 8.0.2;
                     comment: ;
                     service: VMware VirtualCenter;
                    endpoint: "https://vcsa.amer.lan/sdk";
                    username: networker_user@vsphere.local;
                    password: *******;
                     command: nsrvim;
                       proxy: nsr.amer.lan;
              console events: Yes;
                       cloud: No;
              console cancel: Yes;
Snapshot FreeSpace Warning Threshold: 0;
Snapshot FreeSpace Failure Threshold: 0;
        restricted data zone: VMware_Restricted;
nsradmin> update restricted data zone:
        restricted data zone: ;
Update? y
updated resource id 198.0.90.20.0.0.0.0.196.80.99.102.192.168.9.150(11613)
nsradmin> print
                        type: NSR hypervisor;
                        name: vcsa.amer.lan;
                     version: 8.0.2;
                     comment: ;
                     service: VMware VirtualCenter;
                    endpoint: "https://vcsa.amer.lan/sdk";
                    username: networker_user@vsphere.local;
                    password: *******;
                     command: nsrvim;
                       proxy: nsr.amer.lan;
              console events: Yes;
                       cloud: No;
              console cancel: Yes;
Snapshot FreeSpace Warning Threshold: 0;
Snapshot FreeSpace Failure Threshold: 0;
        restricted data zone: ;
nsradmin> q