PowerProtect: vCenter Backup Stops Responding with RecoverPoint for Virtual Machines Present
Summary: The vCenter vpxd service encounters a panic and crashes during vCenter tasks to enable or disable Lightweight Delta (LWD) protection. Task required to enable or disable PowerProtect Data Manager TSDM protection. ...
Symptoms
The vCenter vpxd service encounters a panic and crashes, with the following observed prior to the crash:
- PowerProtect Data Manager TSDM backup
- Virtual machine (VM) added to or removed from TSDM Protection Policy.
- vCenter task to enable or disable LWD protection
"Disable LWD-based data protection"or"EnableLWD-based data protection" - RecoverPoint for Virtual Machines version 6.0.x is configured.
With Virtual Machines (VM) protected by PowerProtect Data Manager (TSDM), the VMware LWD I/O Filter must be enabled on both the VM and its associated disks. However, if RecoverPoint for Virtual Machines I/O filters are also present, this configuration may trigger a panic within the vCenter vpxd service.
vCenter Log: vpxd.log
Log Location: /var/log/vmware/vpxd
At the time of the panic, vpxd log can end a task similar to below:
2025-09-05T13:58:22.692Z info vpxd[3928813] [Originator@6876 sub=provisioning opID=21dfd9d8-d9da-7a] Extra overhead memory required for LWD protect is : 30 2025-09-05T13:58:22.692Z info vpxd[3928813] [Originator@6876 sub=provisioning opID=21dfd9d8-d9da-7a] CurrentOverhead = 50, currentOverheadLimit = 139, availableOverheadResv = 89 2025-09-05T13:58:22.692Z info vpxd[3928813] [Originator@6876 sub=provisioning opID=21dfd9d8-d9da-7a] ExtraOverheadForLWD = 0, needMinLimitBump = false, isLwdIoFilterReconfig = true 2025-09-05T13:58:22.697Z info vpxd[3928813] [Originator@6876 sub=VdbOpJournal opID=21dfd9d8-d9da-7a] Added new journal id=4 type=1 2025-09-05T13:58:22.697Z info vpxd[3928813] [Originator@6876 sub=vpxLro opID=21dfd9d8-d9da-7a-01] [VpxLRO] -- BEGIN lro-2565 -- vm-17025 -- vim.VirtualMachine.reconfigure -- 5287b933-8b08-09b0-f3b8-73d40f6450de(5201c450-9da7-cdaa-c646-94c2c2f4e599)
This coincides with PowerProtect Data Manager requesting the task on the vCenter.
Log: vmdm.log
Log Location: /var/log/brs/vmdm
2025-08-31T18:11:37.307Z INFO [] [https-jsse-nio-0.0.0.0-8447-exec-479] [00000000-0000-4000-a000-000000000000][][][TRACE_ID:825e8776gt67y7u8;JOB_ID:8b461hygt67y776][] [c.e.b.v.v.a.VPEProtectOp.prepareDisks(798)] - VM LinuxVM 4yg76y78-3114-5450-af8c-62gghy78u8i9 have disks included. Enabling LWD filter
Due to the vCenter vpxd failure, PowerProtect Data Manager vmdm reports warnings.
While adding a VM to the Protection Policy:
2025-09-05T14:09:13.082Z WARN [] [vpe-thread-6] [00000000-0000-4000-a000-000000000000][][][TRACE_ID:9562721386df306b;JOB_ID:9ca442dea70d8d9e][] [c.e.b.v.v.a.VPEConfigureOp.logErrorStack(64)] - Unable to configure rhel-client02.amer.lan
While removing a VM from the Protection Policy:
2025-09-05T14:08:43.956Z WARN [] [vpe-thread-6] [00000000-0000-4000-a000-000000000000][][][TRACE_ID:b7d78430774e6984;JOB_ID:8c2a181826f2e416][] [c.e.b.v.v.u.VPEProtectionUtils.unConfigureAsset(853)] - Failed to unconfigure asset
com.emc.brs.vmdm.error.InternalServerErrorException: Unable to wait for Vsan task
at com.emc.brs.vmdm.vpe.util.SdmEnableProtectUtil.waitForVsanTask(SdmEnableProtectUtil.java:506)
at com.emc.brs.vmdm.vpe.util.SdmEnableProtectUtil.configureProtect(SdmEnableProtectUtil.java:473)
at com.emc.brs.vmdm.vpe.util.SdmEnableProtectUtil.configureProtection(SdmEnableProtectUtil.java:257)
at com.emc.brs.vmdm.vpe.util.SdmEnableProtectUtil.configureProtection(SdmEnableProtectUtil.java:272)
at com.emc.brs.vmdm.vpe.util.VPEProtectionUtils.unConfigureAsset(VPEProtectionUtils.java:849)
at com.emc.brs.vmdm.vpe.actions.VPEUnconfigureOp.execute(VPEUnconfigureOp.java:42)
at com.emc.brs.vmdm.vpe.service.VPEActionDispatcher.execute(VPEActionDispatcher.java:408)
at com.emc.brs.vmdm.vpe.service.VPEActionDispatcher.lambda$executeOp$1(VPEActionDispatcher.java:362)
at com.emc.brs.vmdm.vpe.util.ResourceUtils$JobExecutorImpl.lambda$createWorkflowTask$0(ResourceUtils.java:186)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: com.vmware.vsan.sdk.RuntimeFaultFaultMsg: Connection was closed by the remote peer
If VMware LWD I/O filter is not enabled at the start of the backup, the corresponding vCenter task must be completed before the backup can proceed. This occurs when the original configuration job failed. Example warning:
2025-09-05T14:08:43.956Z WARN [] [https-jsse-nio-0.0.0.0-8447-exec-479] [00000000-0000-4000-a000-000000000000][][][TRACE_ID:b7d78430774e6984;JOB_ID:8c2a181826f2e416][] [c.e.b.v.v.a.VPEProtectOp.logErrorStack(64)] - Unable to create backup com.emc.brs.vmdm.error.InternalServerErrorException: Unable to wait for Vsan task
Cause
RecoverPoint version 6.0.x is not supported with any type of backup solution, including PowerProtect Data Manager.
Historically, RecoverPoint for Virtual Machines 5.x has functioned and co-existed with Data Protection products. Since VAIO's introduction in version 6.0, various field issues have emerged when used with vStorage Application Programming Interfaces (APIs) for Data Protection (VADP) or Transparent Snapshots.
After upgrading to version 6.0.x, there have been reports of product issues, including failures and instability in both RP4VMs and the various Data Protection products. The following article describes the issues: RecoverPoint for VMs 6.x: Support for backup solutions (Log in as registered Dell Support user is required to view this article.)
Even though RecoverPoint 6.x-6.0.2.x does not support coexistence with any Data Protection products, the article above contains a potential workaround to increase ESXi VM memory overhead. This workaround DOES NOT apply to the vCenter vpxd panic issue or support of PowerProtect Data Manager TSDM, it related to VADP only.
Contact Dell RecoverPoint Support with questions about RecoverPoint for Virtual Machines support.
Resolution
Action: Unprotect VMs from RecoverPoint for Virtual Machines to prevent additional vCenter service crashes and successfully complete backups.
For VADP backups, RecoverPoint for VMs 6.0.3.1 and above has the correct class and does not cause this issue.
RecoverPoint for Virtual Machines does not support use alongside TSDM.
For the latest RecoverPoint for Virtual Machines 6.0.x support status alongside backup solutions, open a service request with the RecoverPoint team. Reference KB article RecoverPoint for VMs 6.x: Support for backup solutions. (Log in as registered Dell Support user is required to view this article.)