PowerProtect - Multiple VM backups are failing with error "Cannot connect to backup server"
Resumen: PowerProtect Data Manager Appliance DM5500: This article discusses a rare scenario with a heavily loaded system. A VMware VADP workload with specific settings enabled on the policy affects internal connections to the Storage System. ...
Este artículo se aplica a
Este artículo no se aplica a
Este artículo no está vinculado a ningún producto específico.
No se identifican todas las versiones del producto en este artículo.
Síntomas
Affected versions: The DM5500 5.13, 5.14 are affected.
The DM5500 and the vCenter are loaded with running hourly VADP backups with Swap File exclusion and metadata Indexing configured.
After some time, a system with the above configuration experiences VM backups failing with the error:
“Cannot connect to backup server : [5040]".
Symptoms found in the logs:
2023-07-03T05:03:00.219Z DEBUG: [a7355c162f90883e] TaskEngine-0: Concurrent open DD Boost connections = 31. 2023-07-03T05:03:00.219Z DEBUG: [a7355c162f90883e] TaskEngine-0: Access to DD Boost connection delayed by 8.756µs. 2023-07-03T05:03:00.219Z DEBUG: [a7355c162f90883e] Sending state of 'Running' (last: state=Queued, progress=0). 2023-07-03T05:03:08.224Z ERROR: [a7355c162f90883e] TaskEngine-0: Cannot connect to backup server : [5040] calling system(), returns nonzero 2023-07-03T05:03:08.225Z DEBUG: [a7355c162f90883e] TaskEngine-0: Task[1].Undo 2023-07-03T05:03:08.225Z DEBUG: [a7355c162f90883e] TaskEngine-0: Task[0].Undo 2023-07-03T05:03:08.225Z DEBUG: [a7355c162f90883e] TaskEngine-0: Releasing ProxyReservationSessions:53e5a903-afa9-467a-98ab-xxxxxxxxx 2023-07-03T05:03:08.225Z DEBUG: [a7355c162f90883e] TaskEngine-0: Sending release reservation to VISD: release/vm/0
Causa
The system is heavily loaded from a VMware VADP workload perspective with "swapfile exclusion" option.
The "metadata indexing" may or may not be enabled on policies.Over time, the system exhausts all the internal connections to the Storage System.
When the VM Protection Engines or the vCenter runs out of resources, connections that are created to the Storage Systems are not cleaned up.
Over time, connection leakage occurs and this leads to backups stalling.
Resolución
Fix:
Contact Dell Support for the latest information.
Workaround:
1. Identify the VM Protection Engine where the error occurs. Restart the VM Protection Engines in the PowerProtect Data Manager UI:
2. Add additional VM Protection Engines from the UI. This reduces load on the existing VM Protection Engines.
3. Disable the Swap File exclusion option from the Protection policies. This further reduces the load on the VM Protection Engines.
Contact Dell Support for the latest information.
Workaround:
1. Identify the VM Protection Engine where the error occurs. Restart the VM Protection Engines in the PowerProtect Data Manager UI:
2. Add additional VM Protection Engines from the UI. This reduces load on the existing VM Protection Engines.
3. Disable the Swap File exclusion option from the Protection policies. This further reduces the load on the VM Protection Engines.
Productos afectados
PowerProtect DM5500Propiedades del artículo
Número del artículo: 000216108
Tipo de artículo: Solution
Última modificación: 21 sept 2023
Versión: 2
Encuentre respuestas a sus preguntas de otros usuarios de Dell
Servicios de soporte
Compruebe si el dispositivo está cubierto por los servicios de soporte.