PowerProtect Data Manager: Kubernetes backup error 'controller pod is not running'

Summary: PowerProtect Data Manager Kubernetes backup failed with 'controller pod is not running' error.

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.

Symptoms

Kubernetes backups start to fail after recovery of PowerProtect Data Manager from its server disaster recovery (This could also apply to other situations):

ABK0002: Unable to back up namespace 'test' in Kubernetes cluster 'my-k8-cluster'.
Review the error details and summary log in the Jobs window or review the log files at /var/log/brs/cndm and /var/log/brs/cndn/k8s for more detailed information to troubleshoot the issue.
Backup failed: controller pod is not running


The following error can be observed in logs:

2021-07-21T03:49:48.340Z ERROR [] [task-5011a057-340f-40fb-8cd8-12414685d058] [][][][TRACE_ID:a66ce529604914ad;JOB_ID:a9b8915af1637407][] [K8sHelperApi.isDone(90)] - Failed to wait on job com.emc.dpsg.ecdm.baseresourceservice.exception.ValidationServiceException: controller pod is not running
2021-07-21T03:50:14.065Z WARN [] [dsSource-plpd-testcluster] [][][][][] [c.e.b.c.s.p.K8sHealthMonitor.checkPodHealth(200)] - Controller Pod is down, cluster: <k8s_cluster_name>, age=PT153H49M43.065S

The 'kubectl describe pod -n PowerProtect' command output shows:

powerprotect     powerprotect-controller-666ffccbbf-p5rwh                   0/1     ImagePullBackOff   0          6d12h
velero-ppdm      backup-driver-587cfcdf59-2mc8p                             1/1     Running            0          49d
velero-ppdm      velero-5df5fcd896-p68rw                                    1/1     Running            0          49d

Cause

The pod is unable to pull required images from the configured repositories.

Resolution

  • Validate the Kubernetes cluster can access the required repositories to pull images:
  • If the repositories are not accessible, pull the images to the Kubernetes cluster local registry. 
  • On the PowerProtect Data Manager appliance, create the cndm application properties file:
vi /usr/local/brs/lib/cndm/config/application.properties
  • Populate the cndm application properties file with the following:
k8s.docker.registry=fqdn:port For example, k8s.docker.registry=artifacts.example.com:8446
k8s.image.pullsecrets=secret resource name Specify this entry only if you require an image pull secret.
  • Restart the cndm daemon: cndm restart
  • Run a Kubernetes cluster discovery from the PowerProtect Data Manager Interface.

Affected Products

PowerProtect Data Manager
Article Properties
Article Number: 000190024
Article Type: Solution
Last Modified: 11 مايو 2026
Version:  9
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.