PowerProtect Data Manager SQL: VM application aware backups fail with unable to perform MSSQLDM Agent operation 'catalog_snapshot' on VM
Summary: SQL Server application-consistent data protection for virtual machines. VM SQL Application aware backups are failing.
Symptoms
Backup of the application "MSSQLDM" on the virtual machine "XXXX" is unsuccessful. Backup of the application "MSSQLDM" on the virtual machine "XXXX" is unsuccessful. Unable to perform MSSQLDM Agent operation 'catalog_snapshot' on VM xx.xxx.xxx.xx(vm-xxxx). PowerProtect agent for Microsoft Applications: Unable to find the VSS metadata files in directory C:\Program Files\DPSAPPS\MSAPPAGENT\tmp\VSSMetadata.2021-11-30_20-07-17: 2. This problem may occur if the VM configuration setting Disk.EnableUUID is set to FALSE.
Consult the product documentation for more details.
Cause
Two known causes that have been documented:
Case 1: SQL Server application-consistent virtual machine backups might fail with the following error when the disk.EnableUUID variable for the virtual machine is set to False.
Unable to find VSS metadata files in directory C:\Program Files\DPSAPPS\MSVMAPPAGENT\tmp\VSSMetadata.xxxx.
To resolve this issue, ensure that the disk.EnableUUID variable for the virtual machines in an SQL Server applicationconsistent backup is set to True.
>>The parameter disk.EnableUUID for GBRTEF01DR090 is set to 'TRUE' in this case and the VM was also rebooted.
Case 2: "The virtual machine must use SCSI disks only, and the number of available SCSI slots must at least match the number of disks. For example, a virtual machine with seven disks only requires one SCSI controller, but a virtual machine with eight disks requires two SCSI controllers."
If the VM which you are protecting is not using SCSI controller and then this could be one of the possible reasons we are seeing the error related to Disk.EnableUUID during backup.
>> Confirmed that the VM is using SCSI disks (a total of two disks).
Resolution
In this case, the root cause of the issue was due to the use of dynamic disks in the SQL VM, which the VMware does not support for the AppConsistent aware snapshot.
Once the dynamic disks were removed or converted to basic disk in the SQL VM the AppConsistent snapshots were successfully taken, and no issues were seen.
Here is the link for the same:
https://vdc-repo.vmware.com/vmwb-repository/dcr-public/8f96698a-0e7b-4d67-bb6c-d18a1d101540/ef536a47-27cd-481a-90ef-76b38e75353c/doc/GUID-12AFEE64-4427-4127-9B1B-23F2948FD650.html
The Windows VM must not use dynamic disks. If the VM has dynamic disks, the snapshot is file-system consistent.
PowerProtect Data Manager relies on VMware snapshot technology and can support only what is supported by VADP. Now this is one clear case of VMware limitation.