Avamar: VSS Backup Fails with Error 32 for EFIMicrosoftBootBCD Due to File Access Issue

The following symptoms have been observed:

• Avamar Windows VSS backup completes with exceptions:

[avtar] direlem error:init_ntfs_hardlink: Open file for hardlink information failed 
"\\?\GLOBALROOT\Device\HarddiskVolume1\EFI\Microsoft\Boot\BCD", LastError=32 
(code 32: The process cannot access the file because it is being used by another process)

• The avtar process presents this error:

avtar Error <5139>: File access share error "\\?\GLOBALROOT\Device\HarddiskVolume1\EFI\Microsoft\Boot\BCD" 
(code 32: The process cannot access the file because it is being used by another process).

• Adding the below flag to the avtar.cmd file should address the issue:

--x31=268435456

• In some occasions, the issue persists and a new error is presented:

2024-11-04 21:15:17 avtar Info <5008>: Logging to C:\Program Files\avs\var\TXT_Daily_2030_8hrs-Test_Plantsec02-1730773800009#2-3015-VSS.log
2024-11-04 21:15:17 avtar Info <5174>: - Reading C:\Program Files\avs\var\avtar.cmd
2024/11/05-03:15:17.86100 [avtar]  uflags::parsefile  Printing flags from C:\Program Files\avs\var\avtar.cmd:
.cmd flag [1]: --x31=268435456
.cmd flag [2]: --debug=true

                              xxxxxxxxxxx

  <targetlist>
    <path name="\\?\GLOBALROOT\Device\HarddiskVolume1\EFI\Microsoft\Boot\" />
  </targetlist>

               xxxxxxxxxxxxxxxxxxx

2024/11/05-03:15:28.74600 [avtar]  CBackupBCDUtility::exportTmpBCD out (The operation completed successfully.) return (0) bRet (1)
2024/11/05-03:15:28.74600 [avtar]  CBackupBCDUtility::convert2TempBCDPath pathOld(\\?\GLOBALROOT\Device\HarddiskVolume1\EFI\Microsoft\Boot\BCD) pathNew(C:\Temp\av_bcd_tmp-20241104_211528\BCD)
2024/11/05-03:15:28.74600 [avtar]  filestream::fileopen openBCD convert2new path(\\?\GLOBALROOT\Device\HarddiskVolume1\EFI\Microsoft\Boot\BCD) pathNew(C:\Temp\av_bcd_tmp-20241104_211528\BCD)
2024-11-04 21:15:28 avtar Error <16507>: Path "\\?\GLOBALROOT\Device\HarddiskVolume1\EFI\Microsoft\Boot\BCD": Total processed bytes 24576 is inconsistent with original data size 28672
2024/11/05-03:15:28.85700 [avtar]  ucontainer::base::check_set subsequent error 80 - padded a file with zeros because we were unable to read all data from endfile not being set because we already have this error 0 - success
2024/11/05-03:15:28.85700 [avtar]  upath::parentdir() discarding non-entry element 'HarddiskVolume1'
2024/11/05-03:15:28.85700 [avtar]  upath::parentdir() discarding non-entry element 'Device'
2024/11/05-03:15:28.85700 [avtar]  upath::parentdir() discarding non-entry element 'GLOBALROOT'
2024/11/05-03:15:28.85700 [avtar]  CBackupBCDUtility::isBCDBackupIssue path(\\?\globalroot\device\harddiskvolume1\efi\microsoft\boot\bcd.log) : exit code 32 bRet 1
2024/11/05-03:15:28.85700 [avtar]  CBackupBCDUtility::convert2TempBCDPath pathOld(\\?\GLOBALROOT\Device\HarddiskVolume1\EFI\Microsoft\Boot\BCD.LOG) pathNew(C:\Temp\av_bcd_tmp-20241104_211528\BCD.LOG)
2024/11/05-03:15:28.85700 [avtar]  filestream::fileopen openBCD convert2new path(\\?\GLOBALROOT\Device\HarddiskVolume1\EFI\Microsoft\Boot\BCD.LOG) pathNew(C:\Temp\av_bcd_tmp-20241104_211528\BCD.LOG)

• This behavior results in a backup that completes with errors which later leads to failures of any attempts to perform Bare Metal Recovery (BMR).
• Affected environment: All versions of Avamar VSS Plugin, and Windows 2012 and later when EFI partitions are configured.

This issue has been identified as a software issue occurring with Microsoft Windows 2012 Servers and later versions with EFI partitions configured.
 
Caution: The Boot Configuration Data (BCD) directory quoted in the log cannot be excluded from the backup because it requires to boot into Windows after BMR recovery.

For more details check:

https://learn.microsoft.com/en-us/windows/win32/vss/using-vss-automated-system-recovery-for-disaster-recovery?redirectedfrom=MSDN(External Link)

If the code 32 error is encountered when the Avamar Windows VSS plug-in is backing up "EFI\Microsoft\Boot\BCD". 
Ensure to add the flag "--x31=268435456" to the avtar.cmd file and apply the HotFix (HF) according to the client plug-in version.

• v19.9-100-78 => HF 338870 
• v19.10-100-135 => No HF available, upgrade to build 166 and apply the corresponding HF
• v19.10-100-166 (SP1) => HF 338885
• v19.12-100-186 => HF 338884

To download the hotfix from the Dell Support side, see the steps described in Avamar: How to find and download a product hotfix, patch, installation, or upgrade package from the Dell Support website

To apply the hotfix, follow the instructions provided by Dell Support using README file or see the Dell Knowledge Base article for the specific hotfix.

After applying the hotfix, restart the Avamar Windows VSS plug-in service and attempt to back up the "EFI\Microsoft\Boot\BCD" directory again.

Verify that the issue has been resolved by checking the Avamar logs for any errors related to the "EFI\Microsoft\Boot\BCD" directory.