DDPC: Datastores Missing After Rapid Cluster Create and Destroy Operations

Summary: After multiple rapid create and destroy operations using the Gateway API /rest/v1/endpoints, an Edge Compute Engine (ECE) may lose all datastores. This occurs when the destroy request is sent before cluster creation completes, causing concurrent operations that leave drives in an unusable state. ...

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

  • All datastores disappear from one endpoint after multiple cluster create/destroy cycles

  • Datastores remain missing for over 30 minutes

  • LVMs formatted as ceph_bluestore with no BTRFS mounts

  • No automatic recovery occurs, even after server reboot

  • Cluster creation shows only partial completion (stage 1) before cleanup arrives

Cause

The Dell Automation Platform Orchestrator (DAPO) accepts destroy requests and sends cleanup commands without checking whether cluster creation has reached a terminal state. When cleanup runs concurrently with in-progress creation:

  • Cleanup deletes cluster certificates while creation needs them for gRPC communication

  • Cleanup attempts to zap OSD drives that are still active, causing zap failures

  • When zap fails, BTRFS re-creation is skipped

  • Drives remain in ceph_bluestore format with no active Ceph cluster to serve them

Resolution

The full resolution will be in the next available DDPC release.

The workaround requires API callers to ensure previous cluster operations complete before triggering new operations with the same devices.

Affected Products

Dell Distributed Private Cloud, Dell Distributed Private Cloud
Article Properties
Article Number: 000502490
Article Type: Solution
Last Modified: 03 سبتمبر 2026
Version:  1
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.