Avamar - Backup fails is virtual machine contains a IDE type virtual disks

Avamar proxy is running Avamar 19.2,19.3, 19.4 
Virtual machine contains at least one virtual disk using IDE type. 
Forcing backup transport to nbd/nbddl does NOT resolve issue. 
Backup of the same vm using older Avamar 19.1 ( or earlier)  is successful. 

The backup log contains the following lines indicating the virtual disk using adapter type: ide

avvcbimage Info <16041>: VDDK:2021-03-24T18:28:51.613Z error -[28517] [Originator@6876 sub=transport] SupportHotAddTransport: Cannot use hotadd transport to mount Virtual Machineswith disks that are not SCSI, SATA or NVME (adapter type: ide)
avvcbimage Info <16041>: VDDK:2021-03-24T18:28:51.613Z error -[28517] [Originator@6876 sub=transport] Mounting 42229301-63f4-3d56-5782-bf7a49b04a54 using transport hotadd failed :
SupportHotAddTransport: Cannot use hotadd transport to mount Virtual Machines with disks that are not SCSI, SATA or NVME (adapter type: ide).
avvcbimage Info <16041>: VDDK:2021-03-24T18:28:51.614Z error -[28517] [Originator@6876 sub=transport] Cannot use mode hotadd to access [Avamar VMFS1] EMC-vTiger2/EMC-Vtiger.vmdk: Cannot mount using this method. (Mounting 42229301-63f4-3d56-5782-bf7a49b04a54 using transport hotadd failed : SupportHotAddTransport: Cannot use hotadd transport to mount Virtual Machines with disks that are not SCSI, SATA or NVME (adapter type: ide).)

The log then contains a Error the cause job to fails with a refernence to VixDiskLib_GetMetadataKeys

 avvcbimage Info <42151>: backing up virtual disk [Avamar VMFS1] EMC-vTiger2/EMC-Vtiger.vmdk, using virtual disk connection [Avamar VMFS1] EMC-vTiger2/EMC-Vtiger.vmdk
 avvcbimage Info <16041>: VDDK:VixDiskLib: VixDiskLib_GetInfo: Retrieve disk info.
 avvcbimage Info <16041>: VDDK:VixDiskLib: VixDiskLib_FreeInfo: Clean up VixDiskLib.
 avvcbimage Info <16041>: VDDK:VixDiskLib: VixDiskLib_ReadMetadata: Missing metadata value. Error 16052 (The specified key is not found in the disk database) at 7564.
 avvcbimage Error <0000>: [IMG0008] VixDiskLib_GetMetadataKeys([Avamar VMFS1] EMC-vTiger2/EMC-Vtiger.vmdk) returned (0) The operation was successful

The Avamar backup fails with  Error Code: 10007

In Avamar 19.2,19.3  and higher Avamar uses newer VDDK 6.7,  19.4 uses VDDK 7.0.0 or 7.0.1  call  VixDiskLib_GetMetadataKeys/VixDiskLib_ReadMetadata  to get attributes from virtual disk description file,   For IDE  adapter type is IDE the VDDK function fails and vmware VIX API error code  is sent.  

  With debug logs we see the following vmware ERROR code  when IDE adapter type: 

[VcbImageBackupAssistThread] VDDKLIBLOG:VixDiskLib_GetMetadataKeys() key=NULL keySize=0
2021/03/26-13:30:35.72478 [VcbImageBackupAssistThread] VDDKLIBLOG:VixDiskLib_GetMetadataKeys() status=24 requiredLen=238
[VcbImageBackupAssistThread] VDDKLIBLOG:VixDiskLib_GetMetadataKeys() output keys=adapterType keySize=238
[VcbImageBackupAssistThread] VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=ide len=4
avvcbimage Info <16041>: VDDK:VixDiskLib: VixDiskLib_ReadMetadata: Missing metadata value. Error 16052 (The specified key is not found in the disk database) at 6602.
[VcbImageBackupAssistThread] VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=ide len=4
avvcbimage Error <0000>: [IMG0008] VixDiskLib_GetMetadataKeys([Avamar VMFS1] EMC-vTiger2/EMC-Vtiger.vmdk) returned (0) The operation was successful

For a SCSI disk  where the call works properly we should see debug output instead:

[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_GetMetadataKeys() status=24 requiredLen=219
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_GetMetadataKeys() output keys=adapterType keySize=219
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=lsilogic len=9
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=true len=5
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=6527 len=5
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_GetMetadataKeys() key=NULL keySize=0
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=255 len=4
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=63 len=3
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=6527 len=5
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=255 len=4
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=63 len=3
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=5414cb1d3f054ebbeabdd3fbff56735e len=33
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=1 len=2
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=1 len=2
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=11265 len=6
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=60 00 C2 99 87 97 8b 63-65 48 30 bf 5e a2 28 f1 len=48
[VcbImageBackupAssistThread]  VDDKLIBLOG:VixDiskLib_ReadMetadata() output value=14 len=3

This issue is under investigation. 

However for both overall  virtual machine performance  as well as backup performance IDE virtual disk are NOT recommended by either VMware or DellEMC.    IDE virtual disk do NOT support hotadd as such even successful backup logs will contain the following line.

avvcbimage Info <12012>:  Check that the VM being backed up has no IDE virtual hard disks, usually an artifact of a P2v conversion - use SCSI virtual disks instead

If possible please convert the virtual machine disk to SCSI to avoid this issue. 

The following  steps was taken from vmware KB 1016192
If virtual machine guest operating system is NOT using IDE type, skip steps 2-6

1. Power off the virtual machine.
2. From the Edit Settings menu for this virtual machine:
   • Go to Add > Hard Disk > Create New Virtual Disk.
   • Change the Capacity to 1 GB and click Next.
   • Ensure that the Virtual Device Node is set to SCSI.
   • Complete the wizard and select the New SCSI Controller.
   • If the current type is not LSI Logic Parallel Select Change Type, change the type to LSI Logic Parallel.
   • Click OK.
   • Choose the same controller as the adapter type, make sure the SCSI ID should read SCSI 0:0.
3. Power on the virtual machine.
4. From guest ensure new virtual disk is visible to operating system to ensure it has a SCSI driver. 
5. Power off the virtual machine, remove disk from step2. 
6. Locate the datastore path where the virtual machine resides.
   For example:

   # cd /vmfs/volumes/datastore_name/vm_name/

7. From the ESX Service Console, open the primary disk (.vmdk) using the vi editor. For more information, see Editing files on an ESX host using vi or nano (1020302).
8. Look for the line:

   ddb.adapterType = "ide"

9. To change the adapter type to LSI Logic change the line to:

   ddb.adapterType = "lsilogic"

   In some instances the entry appears as ddb.adapterType = "legacyESX". In this case change the line to:

   ddb.adapterType = "lsilogic"

10. Save the file.
11. From VMware Infrastructure/vSphere Client:
     
    1. Click Edit Settings for the virtual machine.
    2. Select the IDE virtual disk.
    3. Choose to Remove the Disk from the virtual machine.
    4. Click OK.
       Caution: Do not choose delete from disk.
12. From the Edit Settings menu for this virtual machine:
    1. Go to Add > Hard Disk > Use Existing Virtual Disk.
    2. Navigate to the location of the disk and select to add it into the virtual machine.
    3. Choose the same controller as in Step 9 as the adapter type. The SCSI ID should read SCSI 0:0.
        
13. If a CDROM device exists in the virtual machine it may need to have the IDE channel adjusted from IDE 0:1 to IDE 0:0. If this option is greyed out, remove the CD-ROM from the virtual machine and add it back. This sets it to IDE 0:0.