Avamar: Hyper-V Backups Failing | Timed Out While Waiting for a VSS Snapshot

Hyper-V cluster backups fail with the following errors:
From the shared cluster var directory logs:

• avhypervvss Error <0000>: The plugin on remote client HYPERVNODE4 (192.168.1.1) terminated with code 10007: miscellaneous error
  avhypervvss Error <0000>: The remote client HYPERNODE4 failed to start its backup sub-workorder in predefined time. Please examine the log 'Hyper-V_5pm-Hyper-_V-123456789-3032-Hyper-V_VSS.log' on client HYPERVNODE4 for more information.
  avhypervvss Error <13810>: Unable to forward a ctl message to the remote client '192.168.1.1', result code: 3.
  avhypervvss Error <41106>: [AV-CLUSTER0] is unable to wakeup sub-workorder on [HYPERVNODE4].
  avhypervvss Error <42468>: Failed to update snapshot details from remote clients
  avhypervvss Error <41659>: Backup did not complete successfully

   

From the backup log on the affected node (C:\Program Files\avs\var): 

• avhypervvss Error <17114>: Timed out while waiting for a VSS snapshot synchronization event from the primary plugin.
  avhypervvss Error <41632>: Failed to setup and notify primary.
  avhypervvss Error <41659>: Backup did not complete successfully

The failure is caused by communication delays in CTL message exchanges between the primary and secondary nodes in the Hyper-V cluster. These delays prevent timely synchronization of VSS snapshots, resulting in backup failure. 
 

To resolve the issue, increase the timeout values in the avhypervvss.cmd file located in the shared cluster var directory (as in, C:\ClusterStorage\Volume1\Avamar\var):

Add the following flags:

--request-ctl-timeout=600000
--subworkorder-timeout=20

• --request-ctl-timeout=600000: Increases the internal CTL timeout between nodes (default is 60 seconds).
• --subworkorder-timeout=20: Sets the wait time (in minutes) for backend clients to start sub-workorders (default is 5 minutes).

⚠️ Ensure the file is updated on all relevant nodes and restart the backup process.