We've recently upgraded from NetWorker 7.5.1 to 7.6.3 (Windows 2003 R2 Standard SP2 32-bit). As our VCB backup proxy was also the backup server itself, we've naturally had to switch from the (fully-working) VCB backup method to the new VADP backup method.
First of all, we were on version 4.0.0 of vSphere which should have been supported, but it turns out that the minimum spec appears to be 4.0.0 Release 2 - annoying! Fortunately, that got upgraded a week later to version 5, albeit on a new hypervisor (Windows 2008 R2 64-bit).
Some of the VADP backups work great, however we have a dozen or so failing with:
* vm_hostname:*FULL* Could not mount required number of volumes. Exiting.
* vm_hostname:*FULL* The VADP proxy and the VM client are possible clones of each other or derived from the same parent. Please try by setting the transport mode as nbd.
I've also added the application information value of VADP_TRANSPORT_MODE=nbd in the client resource for the particular VM, but it seems to ignore this setting as the save command output (please see below) shows:
Using 'san' as the transport mode for Hard disk 1.
The clients that are failing are mainly Windows, both 2003 and 2008 and both 32- and 64-bit, but there's also a couple of Linux as well. I can't see any commonalities with the failing backups. Most of the servers have up-to-date VMware Tools
Another issue we're experiencing is that one client fails with Unable to find the VM with VM name: vm_host.fqdn.com. This is for the weekly backup with a one month retention. However, the monthly backup with a six year retention works perfectly! As far as I can see, both client resources are configured identically, except for the browse/retention policies.
Has anyone experienced these issues before?
Thanks
B
Output from savegrp command (from /nsr/tmp):
Initializing VADP Configuration.
Using nsr.backup.server.com as the VADP proxy server host for client vm_hostname.
Application information attribute VADP_HOST for client vm_hostname is set to "something.our.fqdn".
Application information attribute VADP_DISABLE_FLR for client vm_hostname is set to default value of "No".
Application information attribute VADP_BACKUPROOT for client vm_hostname is set to "C:\\VCB_Backups".
Application information attribute VADP_TRANSPORT_MODE for client vm_hostname is set to "san|hotadd|nbdssl|nbd".
Application information attribute VADP_MAX_RETRIES for client vm_hostname is set to default value of "0".
Application information attribute VADP_MAX_BACKOFF_TIME for client vm_hostname is set to default value of "10".
80423:nsrvadp_save: Unknown application information attribute found for client vm_hostname.
Trying to reset the client resource application information attributes.
Application information attribute VADP_HYPERVISOR for client vm_hostname is set to "hypervisor.our.fqdn".
80423:nsrvadp_save: Unknown application information attribute found for client vm_hostname.
It turned out that the disks types used were a mix of SCSI and IDE (some of them SCSI and the others IDE), this is a known limitation with VADP. Hence the backup failures.
Space for the VADP mountpoint is not an issue, but I'll pass the disk/LUN information to our VMware admins.
Incidentally, the machines that are failing their VADP backup all had successful VCB backups prior to the NW upgrade. Of course, it's difficult to work out what has broken things since we upgraded NetWorker and vSphere and moved to VADP at essentially the same time.
For SAN mode backups, the VADP proxy requires read access to the SAN LUNs hosting the virtual machines.
Also you have to use the Diskpart utility for SAN and hotadd transport modes , Where an RDM NTFS volume is being used for any of the VMs on the VADP proxy host, Windows will automatically attempt to mount the volume and assign drive letters to VM disks during backup. This may lead to data corruption on the VMs.To prevent Windows from automatically assigning drive letters to the RDM NTFS, perform the following steps.
Note: Steps 1 and 2 are only applicable in the case of SAN transport mode where SAN fabric zoning is already in place such that the VADP proxy host is already displaying the SAN LUNs in Windows disk management. If this does not apply, skip to Step 3.
1. Shut down the Windows proxy.
2. Disconnect the Windows proxy from the SAN or mask all the LUNs containing VMFS volumes or RDM for virtual machines.
3. Start the proxy and log into an account with administrator privileges.
4. Open a command prompt and run the diskpart utility by entering the following:
diskpart
The diskpart utility starts and prints its own command prompt.
5. Disable automatic drive letter assignment to newly discovered volumes by entering the following in the diskpart command prompt:
automount disable
6. Clean out entries of previously mounted volumes in the registry by entering the following in the diskpart command prompt:
Make sure that your machines are not configured with independent persistent disks as VADP does not support the backup and recovery of these disks, Also make sure that you are not using Dynamic Disks ( You can check that from the Disk management console inside each machine )
There is space considerations and recommendations needed for the VADP mount point (VADP_BACKUPROOT), which is C:\VCB_Backups in your case, The VADP mount point cache requires space equal to at least 5-10% of the total amount of data being backed up in th e case of Windows VMs. This space is required for temporarily storing the VMDK index during the backup, and is cleaned up once the backup completes. In the case of Linux or FLR-disabled
Windows VMs, minimal space is required as indicated in the note below.
As an example of how much space is required for a Windows VM:
If the proxy client parallelism is set to 5 so that a maximum of 5 Windows VMs are backed up concurrently, then calculate the total used disk space for the 5 largest Windows VMs in the environment. Allocate at least 10% of this total used space for the VADP_BACKUPROOT mount point.
So, if each VM in the above example has around 2 disks and each disk has 40GB used space.
• Total amount of data being backed up=40GB*2*5=400GB
• Total amount needed for mount point=400*10%=40GB
In this case, ensure that the drive specified for VADP_BACKUPROOT has at least 40GB of free space.
Note: This mount point space is only needed when performing FLR-enabled image level backups of Windows VMs. It is otherwise very minimal (in the order of a few MB per VM) when performing image level backups of Linux VMs or FLR-disabled image level backups of Windows VMs.
I've managed to get the client that was failing its weekly backup backup but not its monthly backup working. I don't remember doing anything to fix it, but it's working nonetheless...
I've also managed to get VADP to use the nbd transport method as specified in the original error. However, the backup is still failing with the same issue:
Obtained the virtual disks for the VM.
Change block tracking is disabled for VM: vm_hostname
File level Recovery from Image level backup is enabled.
MORef for the VM is obtained.
Obtained the virtual disks for the VM.
Using 'nbd' as the transport mode for Hard disk 1.
Starting mountvm operation.
Successfully mounted the VM as a file system.
Could not mount required number of volumes. Exiting.
The VADP proxy and the VM client are possible clones of each other or derived from the same parent. Please try by setting the transport mode as nbd.
I am using nsr 8.0.2.6 on each machine. I am trying to configure VADP backup for a VM having Windows 2003, 32-bit OS. The proxy and storage node are the same VM machine running on Windows 2008 R2, The networker server is running on 2008 R2.
The backup is failing with "
Could not mount required number of volumes. Exiting.
The VADP proxy and the VM client are possible clones of each other or derived from the same parent. Please try by setting the transport mode as nbd "
These files are stored in the following location by default:
\nsr\plugins\VDDK\
Note: Double quotes should be specified in the path even though the path is already
present.
I have checked and the above holds true in my case. The VADP proxy is also the proxy for other VMs and the backups for other VMs is running fine, something is wrong with this particular VM (remember this is a new configuration).
tech88kur
142 Posts
3324
0
Posted June 2nd, 2014 12:00
Hi,
Updating the status of this issue..
It turned out that the disks types used were a mix of SCSI and IDE (some of them SCSI and the others IDE), this is a known limitation with VADP. Hence the backup failures.
Regards
tech88kur