Avamar Hyper-v backups fail with the error: Checkpoint creation for Virtual machine has failed and will be skipped from backup

Avamar Hyper-V backups may fail or complete with exceptions, having the below error:

avhypervvss Error <43426>: Checkpoint creation for Virtual machine '<VM_NAME_1><VM_GUID_1>' has failed and will be skipped from backup. For details, please check 'winclustersvc.log'.
avhypervvss Info <0000>: --------------------Start [EMC Avamar Plug-in Service for Windows] log ----------

The corresponding portion of winclustersvc.log (with DEBUG enabled), which is located in C:\Program Files\avs\var\ folder on the owner of "EMC Avamar backup cluster client for <Avamar_clustered_client_name>" (can be determined from the Failover Cluster Manager). To enable DEBUG in winclustersvc, check "Additional Info".

winclustersvc INFO<0000>:  VMGroup 'AvamarBackup_<####>' created'  
winclustersvc DEBUG<0000>: <=== CreateNewVMGroup. 
winclustersvc DEBUG<0000>: <=== RctCreateVMGroup. 
winclustersvc INFO<0000>:  Adding targets to vmgroup 'AvamarBackup_<####>' and owner '####' 
winclustersvc DEBUG<0000>: ===> RctAddVMGroupMember. 
winclustersvc DEBUG<0000>: ===> AddVMGroupMember. 
winclustersvc DEBUG<0000>: WMI Query : 'SELECT * FROM Msvm_VirtualSystemCollection WHERE ElementName="AvamarBackup_<####>"'  
winclustersvc DEBUG<0000>: WMI Query : 'SELECT * FROM Msvm_CollectionManagementService'  
winclustersvc DEBUG<0000>: ===> GetVmObjectbyId  
winclustersvc DEBUG<0000>: WMI Query : 'SELECT * FROM Msvm_ComputerSystem WHERE Name="<VM_GUID_1>"'  
winclustersvc INFO<0000>:  System.Management.ManagementException: Returned collection empty
   at EMC.BRS.HypervProxy.Common.WmiUtilities.GetVmObjectbyId(String id, String className, ManagementScope scope) 
winclustersvc ERROR<0000>: No Msvm_ComputerSystem could be found with GUID "<VM_GUID_1>", WMI query returned ManagementException ! error message is Returned collection empty 
winclustersvc INFO<0000>:  System.Management.ManagementException: Returned collection empty
   at EMC.BRS.HypervProxy.Common.WmiUtilities.GetVmObjectbyId(String id, String className, ManagementScope scope)
   at EMC.BRS.HypervProxy.HypervAdvanceBackup.RCTAddVMGroupMember(ManagementScope scope, String groupName, String vmName) 
winclustersvc ERROR<0000>: Exception in Adding VM '<VM_GUID_1>' to group 'AvamarBackup_<####>' with error 'Returned collection empty'.

To reproduce the issue, in PowerShell (as administrator), run the below command (without angle brackets):

Get-Wmiobject -Query "SELECT * FROM Msvm_ComputerSystem WHERE Name='<VM_GUID_1>'" -Namespace "root\HypervCluster\v2"

The query for "VM_GUID_1" returns ZERO (empty result).

For comparison, the query for other VM in the cluster - ‘VM_GUID_2’, which does not have checkpoint issue, works just fine:


PS C:\Users\######> Get-Wmiobject -Query "SELECT * FROM Msvm_ComputerSystem WHERE Name='<VM_GUID_1>'" -Namespace "root\HypervCluster\v2"

snippet:
======

__GENUS                                  : 2
__CLASS                                  : Msvm_ComputerSystem
__SUPERCLASS                             : CIM_ComputerSystem
__DYNASTY                                : CIM_ManagedElement
__RELPATH                                : Msvm_ComputerSystem.CreationClassName="Msvm_ComputerSystem",Name="########"
__PROPERTY_COUNT                         : 45
__DERIVATION                             : {CIM_ComputerSystem, CIM_System, CIM_EnabledLogicalElement,
                                           CIM_LogicalElement...}
__SERVER                                 : #######
__NAMESPACE                              : root\HypervCluster\v2
__PATH                                   : \\#######\root\HypervCluster\v2:Msvm_ComputerSystem.CreationClassName="Ms
                                           vm_ComputerSystem",Name="############"
AvailableRequestedStates                 :
Caption                                  : Virtual Machine
CommunicationStatus                      :
CreationClassName                        : Msvm_ComputerSystem
Dedicated                                :
Description                              : Microsoft Virtual Machine
DetailedStatus                           :
ElementName                              : <VM_NAME>

The WMI info for VM in ‘root\HyperVCluster\v2’ is missing.

Fix/workaround:
Advise the customer to remove the VM in question from the cluster then add it back. 
The WMI info for VM in ‘root\HyperVCluster\v2’ will be re-created.

To enable debug in winclustersvc:
Use Notepad to create winclustersvc.cmd in C:\Program Files\avs\var\ folder on the Hyper-V Node - the owner of "EMC Avamar backup cluster client for <Avamar_clustered_client_name>" and add --debug to it. Re-start "EMC Avamar Plug-in Service for Windows" on the Hyper-V Node.  Re-run the backup to reproduce the issue.