RecoverPoint for VMs 6.x: Snapshot Consolidation errors or VM restarts on protected VMs
Summary: RecoverPoint for Virtual Machines(VMs) 6.0.3.1 and later are qualified with VMware vStorage APIs for Data Protection(VADP) backup solutions. However, sometimes protected Virtual Machines may see Snapshot Consolidation errors or Virtual Machine restarts, during backup operations. ...
Symptoms
RecoverPoint for Virtual Machines(VMs) 6.0.3.1 and later environments with VMware vStorage APIs for Data Protection(VADP) backup solutions may have Snapshot Consolidation issues.
The Virtual Machine logs(vmware.log) may show:
2026-01-25T18:19:29.437Z In(05) vcpu-0 - PluginLdr_Load: Loaded plugin 'libvmiof-disk-emcsplitter.so' from '/usr/lib64/vmware/plugin/libvmiof-disk-emcsplitter.so'
2026-01-25T18:19:29.437Z In(05) vcpu-0 - FiltLib: VMKPrivate_FiltModInitDiskInfo failed: "Out of memory (ok to retry)" (195887125).
2026-01-25T18:19:29.437Z In(05) vcpu-0 - FiltLib: FiltLibAttachToFiltMod failed with error: "Operation failed" (1).
2026-01-25T18:19:29.437Z Er(02) vcpu-0 - DISKLIB-LIB : DiskLibFiltLibInit: Failed to create filtLib context: Operation failed (334).
2026-01-25T18:19:29.438Z In(05) vcpu-0 - DISKLIB-LIB : DiskLibOpenInt: Failed to create filtLib context: Operation failed (334).
2026-01-25T18:19:29.459Z In(05) vcpu-0 - DISKLIB-VMFS : "vsan://xxxxafdaa9324281-780967382226xxxx/58ac5263-9888-8e86-a59a-5c6fxxxxxxxx" : closed.
2026-01-25T18:19:29.459Z In(05) vcpu-0 - DISKLIB-LIB : Failed to open '/vmfs/volumes/vsan:xxxxafdaa9324281-780967382226xxxx/f6277563-dc3a-d74a-4dbe-5c6fxxxxxxxx/VM1.vmdk' with flags 0x20a Operation failed (334).
2026-01-25T18:19:29.459Z In(05) vcpu-0 - SNAPSHOT:Failed to open disk /vmfs/volumes/vsan:xxxxafdaa9324281-780967382226xxxx/f6277563-dc3a-d74a-4dbe-5c6fxxxxxxxx/VM1.vmdk: Operation failed (334).
2026-01-25T18:19:29.463Z In(05) vcpu-0 - DISKLIB-CBT : Shutting down change tracking for untracked fid 526917510.
2026-01-25T18:19:29.463Z In(05) vcpu-0 - DISKLIB-CBT : Successfully disconnected CBT node.
2026-01-25T18:19:29.466Z In(05) vcpu-0 - [2949195/3904767744]: SplDiskClose: handle: 6A43AD3A20 volInfo: 0
2026-01-25T18:19:29.466Z In(05) vcpu-0 - [2949195/3904767744]: SplDiskClose: PrivateData was NULL for diskHandle 6A43AD3A20 but diskInfo was found in hash, closing sidecar
2026-01-25T18:19:29.468Z In(05) vcpu-0 - [2949195/3904767744]: SplDiskClose: SplDiskClose_cleanUp :: diskInfo->iof_volume_data_i->diskProperties=6A43AF7018 propIndex=0 memory freed
2026-01-25T18:19:29.468Z In(05) vcpu-0 - [2949195/3904767744]: SplDiskClose: SplDiskClose_cleanUp :: diskInfo->iof_volume_data_i->diskProperties=6A43AF7018 propIndex=1 memory freed
2026-01-25T18:19:29.468Z In(05) vcpu-0 - [2949195/3904767744]: SplDiskClose: SplDiskClose_cleanUp :: diskInfo->iof_volume_data_i->diskProperties=6A43AF7018 propIndex=2 memory freed
2026-01-25T18:19:29.468Z In(05) vcpu-0 - [2949195/3904767744]: [6A43AD3A20] InitCounter: Dec from 2 to 1
2026-01-25T18:19:29.472Z In(05) vcpu-0 - [2949195/3904767744]: SplHeapManager_destroy: VMIOF_HeapDestroy done
2026-01-25T18:19:29.472Z In(05) vcpu-0 - [2949195/3904767744]: SplVolDetails_destroy: heapManager=6AEA6BE480
2026-01-25T18:19:29.472Z In(05) vcpu-0 - [2949195/3904767744]: SplDiskClose: SplVolDetails_destroy: done for handle=6A43AD3A20
2026-01-25T18:19:29.494Z In(05) vcpu-0 - DISKLIB-VMFS : "vsan://xxxxafdaa9324281-780967382226xxxx/f6277563-dc3a-d74a-4dbe-5c6fxxxxxxxx" : closed.
2026-01-25T18:19:29.494Z In(05) vcpu-0 - DISK: Failed to open disk for consolidate '/vmfs/volumes/vsan:xxxxafdaa9324281-780967382226xxxx/f6277563-dc3a-d74a-4dbe-5c6fxxxxxxxx/VM1-000001.vmdk': Operation failed (334) 9021
2026-01-25T18:19:29.496Z In(05) vcpu-0 - AIOGNRC: Failed to open '/vmfs/devices/vsan/xxxx7669-1c59-e285-4b87-5c6f6977xxxx' : Could not find the file (60003) (0x2013).
..
2026-01-25T18:19:30.859Z In(05) vcpu-0 - ConsolidateEnd: Snapshot consolidate complete: Operation failed (5).
2026-01-25T18:40:23.042Z In(05) vcpu-0 - PluginLdr_Load: Loaded plugin 'libvmiof-disk-emcsplitter.so' from '/usr/lib64/vmware/plugin/libvmiof-disk-emcsplitter.so'
2026-01-25T18:40:23.043Z In(05) vcpu-0 - FiltLib: VMKPrivate_FiltModInitDiskInfo failed: "Admission check failed for memory resource" (195887233).
2026-01-25T18:40:23.043Z In(05) vcpu-0 - FiltLib: FiltLibAttachToFiltMod failed with error: "Operation failed" (1).
2026-01-25T18:40:23.043Z Er(02) vcpu-0 - DISKLIB-LIB : DiskLibFiltLibInit: Failed to create filtLib context: Operation failed (334).
2026-01-25T18:40:23.043Z In(05) vcpu-0 - DISKLIB-LIB : DiskLibOpenInt: Failed to create filtLib context: Operation failed (334).
2026-01-25T18:40:23.064Z In(05) vcpu-0 - DISKLIB-VMFS : "vsan://xxxxxxxa9324281-7809673822260c03/ca46dc67-1c0c-ab8c-f322-5c6fxxxxxxxx" : closed.
2026-01-25T18:40:23.064Z In(05) vcpu-0 - DISKLIB-LIB : Failed to open '/vmfs/volumes/vsan:xxxxxxxa9324281-7809673822260c03/ca46dc67-1c0c-ab8c-f322-5c6fxxxxxxxx/VM2.vmdk' with flags 0x20a Operation failed (334).
2026-01-25T18:40:23.064Z In(05) vcpu-0 - SNAPSHOT:Failed to open disk /vmfs/volumes/vsan:xxxxxxxa9324281-7809673822260c03/ca46dc67-1c0c-ab8c-f322-5c6fxxxxxxxx/VM2.vmdk: Operation failed (334).
…
2026-01-25T18:40:23.096Z In(05) vcpu-0 - DISK: Failed to open disk for consolidate '/vmfs/volumes/vsan:xxxxxxxa9324281-7809673822260c03/ca46dc67-1c0c-ab8c-f322-5c6fxxxxxxxx/VM2-000001.vmdk': Operation failed (334) 9021
…
2026-01-25T18:40:23.665Z In(05) vcpu-0 - ConsolidateEnd: Snapshot consolidate complete: Operation failed (5).
2026-01-19T18:26:59.026Z In(05) vcpu-0 - PluginLdr_Load: Loaded plugin 'libvmiof-disk-emcsplitter.so' from '/usr/lib64/vmware/plugin/libvmiof-disk-emcsplitter.so'
2026-01-19T18:26:59.026Z In(05) vcpu-0 - FiltLib: VMKPrivate_FiltModInitDiskInfo failed: "Out of memory (ok to retry)" (195887125).
2026-01-19T18:26:59.026Z In(05) vcpu-0 - FiltLib: FiltLibAttachToFiltMod failed with error: "Operation failed" (1).
2026-01-19T18:26:59.026Z Er(02) vcpu-0 - DISKLIB-LIB : DiskLibFiltLibInit: Failed to create filtLib context: Operation failed (334).
2026-01-19T18:26:59.026Z In(05) vcpu-0 - DISKLIB-LIB : DiskLibOpenInt: Failed to create filtLib context: Operation failed (334).
2026-01-19T18:26:59.048Z In(05) vcpu-0 - DISKLIB-VMFS : "vsan://5260afdaa9324281-7809673822260c03/2f776e69-266e-ffeb-ecf7-5c6f6977eb70" : closed.
2026-01-19T18:26:59.069Z In(05) vcpu-0 - DISKLIB-VMFS : "vsan://5260afdaa9324281-7809673822260c03/2bde1867-be05-d2dd-eecd-5c6f69776bf0" : closed.
2026-01-19T18:26:59.069Z In(05) vcpu-0 - DISKLIB-LIB : Failed to open '/vmfs/volumes/vsan:5260afdaa9324281-7809673822260c03/4e242565-d045-7eb4-d626-5c6f69776fc0/TWAPPINF03-000001.vmdk' with flags 0xa Operation failed (334).
2026-01-19T18:26:59.069Z Er(02) vcpu-0 - DISK: Cannot open disk '/vmfs/volumes/vsan:5260afdaa9324281-7809673822260c03/4e242565-d045-7eb4-d626-5c6f69776fc0/TWAPPINF03-000001.vmdk': Operation failed (334).
2026-01-19T18:26:59.069Z In(05) vcpu-0 - CPT: error syncing group SCSI0
2026-01-19T18:26:59.069Z In(05) vcpu-0 - VMXNET3 user: Ethernet0 Driver Info: version = 24153797 gosBits = 2 gosType = 2, gosVer = 40960, gosMisc = 212
2026-01-19T18:26:59.070Z In(05) vcpu-0 - CPT: vm was stunned for 1243597 us
2026-01-19T18:26:59.070Z In(05) vcpu-0 - Msg_Post: Error
2026-01-19T18:26:59.070Z In(05) vcpu-0 - [msg.iofilter.failure] Operation failed
2026-01-19T18:26:59.070Z In(05) vcpu-0 - [msg.disk.noBackEnd] Cannot open the disk '/vmfs/volumes/vsan:5260afdaa9324281-7809673822260c03/4e242565-d045-7eb4-d626-5c6f69776fc0/TWAPPINF03-000001.vmdk' or one of the snapshot disks it depends
on.
2026-01-19T18:26:59.070Z In(05) vcpu-0 - [msg.checkpoint.continuesync.error] An operation required the virtual machine to quiesce and the virtual machine was unable to continue running.
2026-01-19T18:26:59.070Z In(05) vcpu-0 - ----------------------------------------
2026-01-19T18:26:59.071Z In(05) vcpu-0 - SnapshotVMX_ConsolidateCancel: Requesting snapshot consolidate cancel.
2026-01-19T18:26:59.071Z In(05) vcpu-0 - Msg_Post: Error
2026-01-19T18:26:59.071Z In(05) vcpu-0 - [msg.poweroff.commitOn] Performing disk cleanup. Cannot power off.
2026-01-19T18:26:59.071Z In(05) vcpu-0 - ----------------------------------------
2026-01-19T18:26:59.072Z In(05) vcpu-0 - ConsolidateItemCombine: Failed to unstun the virtual machine while consolidating disks.
2026-01-19T18:26:59.072Z In(05) vcpu-0 - ConsolidateEnd: Failed to stun/unstun virtual machine while consolidating disks.
2026-01-19T18:26:59.072Z In(05) vcpu-0 - VigorTransport_ServerSendResponse opID=58b0175d-8a-c67d seq=7937380: Completed Snapshot.Consolidate request with messages in 1446806 US.
2026-01-19T18:26:59.072Z In(05) vcpu-0 - Turning off snapshot info cache.
2026-01-19T18:26:59.074Z In(05) vcpu-0 - Turning off snapshot disk cache.
2026-01-19T18:26:59.074Z In(05) vcpu-0 - ConsolidateEnd: Snapshot consolidate complete: Unable to save snapshot file (13).
2026-01-19T18:26:59.074Z In(05) vcpu-0 - Vix: [mainDispatch.c:4211]: VMAutomation_ReportPowerOpFinished: statevar=3, newAppState=1881, success=1 additionalError=0
2026-01-19T18:26:59.074Z In(05) vcpu-0 - ConsolidateEnd: Snapshot consolidate requesting to power-off.
If any of the below key error messages are present in the log, it could potentially indicate a problem and have an impact:
vcpu-0 - FiltLib: VMKPrivate_FiltModInitDiskInfo failed: "Admission check failed for memory resource" (195887233).
vcpu-0 - FiltLib: VMKPrivate_FiltModInitDiskInfo failed: "Out of memory (ok to retry)" (195887125).
vcpu-0 - FiltLib: FiltLibAttachToFiltMod failed with error: "Operation failed" (1).
vcpu-0 - DISKLIB-LIB : DiskLibFiltLibInit: Failed to create filtLib context: Operation failed (334).
vcpu-0 - DISKLIB-LIB : DiskLibOpenInt: Failed to create filtLib context: Operation failed (334).
vcpu-0 - DISK: Failed to open disk for consolidate '...': Operation failed (334) 9021
vcpu-0 - ConsolidateEnd: Snapshot consolidate complete: Operation failed (5).
vcpu-0 - [msg.checkpoint.continuesync.error] An operation required the virtual machine to quiesce and the virtual machine was unable to continue running.
vcpu-0 - ConsolidateEnd: Failed to stun/unstun virtual machine while consolidating disks.
vcpu-0 - ConsolidateEnd: Snapshot consolidate requesting to power-off.
Cause
RecoverPoint for Virtual Machines (VMs) 6.0.x Splitter mechanism uses VMware vSphere APIs for I/O Filtering(VAIO) technology.
During Snapshot operations, the Virtual Machine may run out of memory and may fail.
Resolution
Dell Technologies and Broadcom engineering teams are investigating this issue. Contact the Dell Technologies Customer Support Center or your service representative for assistance and reference this solution ID.
Workaround:
Creation / Consolidating of vSphere managed snapshot fails or crashes the VM intermittently
- Log in to vCenter as root
Vi /etc/vmware-vpx/vpxd.cfg(make a copy of thevpxd.cfgbefore modifying)- Add this configuration between the
<vpxd>and the</vpxd>tags:
<cluster> <VMOverheadGrowthLimit>6</VMOverheadGrowthLimit> </cluster>
-
- Save the file using command:
wq!
- Restart the vCenter Services to apply the changes at the vCenter level.
-
Restart the Virtual Machine or
suspendResume(vMotion uses this function as well), otherwise the new value ofVMOverheadGrowthLimitdoes not take e
- Save the file using command:
VMOverheadGrowthLimit=6 means 6*5=30MB
Sometimes, Virtual Machines may require a higher setting to allow for more memory, this can be set to 9, 10 or even 12.
To verify the setting is applied:
After the VPXD configuration file is updated and VPXD service is restarted, all ESXi hosts show the updated setting, in the following line print in VMkernel logs:2026-02-04T17:09:35.100Z In(182) vmkernel: cpu0:525807 opID=2050f035)Config: 725: "VMOverheadGrowthLimit" = 6, Old Value: 4294967295, (Status: 0x0)2026-02-04T17:12:15.138Z In(182) vmkernel: cpu2:525452 opID=f83e3379)Config: 725: "VMOverheadGrowthLimit" = 9, Old Value: 6, (Status: 0x0)
To validate that the Virtual Machine has the correct setting, look at the below line in the vmware.log file in the specific Virtual Machine folder:2026-02-04T17:07:07.170Z In(05) vmx - OvhdMem: memsize 4096 MB VMK fixed 3281 pages var(mem) 25 pages var(cpu) 22 cbrcOverhead 0 pages total 3427 pages
The following table shows the expected vmware.log print based on the VC setting:
| vCenter VMOverheadGrowthLimit setting | ESXi VMkernel log - VMOverheadGrowthLimit | Virtual Machine Vmware.log - VMK fixed |
| 0 or default | 0 or 4294967295 (Default) | 3281 |
| 6 | 6 | 4817 |
| 9 | 9 | 5585 |
| 10 | 10 | 5841 |
| 12 | 12 | 6353 |
Additional Information
All the Symptoms above may occur on any Backup Solution that works at the VM level using, including but not limited to:
- Dell PowerProtect Data Manager(PPDM)
- Dell NetWorker
- Dell Avamar
- Dell Technologies APEX Backup Services, powered by Druva
- Veritas NetBackup
- Veeam Backup
- Commvault
- Rubrik