PowerStore Alert ID: POWERDOCTOR_VOLUMES_IN_DESTROYING_STATE

Alert

                                Table 1. Alert ID and Associated Information

PowerStore PowerDoctor has detected one or more volumes that are stuck in the Destroying state. These volumes have had delete operations initiated, but the delete commands have already completed (or no delete command exists) while the volumes remain in the Destroying state. This indicates the volumes are stuck and unable to complete their deletion process.

When this issue occurs, you may observe the following symptoms:

• PowerDoctor Alert: A MINOR severity alert is raised with the message: "Volumes are stuck in Destroying state. Affected volumes failed last deletes and are left in Destroying state."
• Alert Details: Volumes are stuck in Destroying state. Affected volumes failed last deletes and are left in Destroying state.
• Volume State: When querying volume details through the UI, CLI, or REST API, volume state shows as Destroying and the delete operation appears to have completed or is not present.
• Storage Impact: Stuck volumes may continue to consume storage capacity, and storage space cannot be reclaimed until volumes are properly removed.

A volume enters the Destroying state when a delete operation is initiated. Normally, the delete operation completes successfully, and the volume is removed from the system. However, in certain failure scenarios, the delete command may fail while the volume remains in the Destroying state.

Common root causes include:

• Delete Command Failure: The delete command encountered an error during execution after command passed rollback point.
• Interrupted Delete Operation: A system event (node failover, service restart, upgrade) interrupted the delete operation mid-flight.
• Snapshot Bulk Delete Issues: For volume snapshots deleted as part of snapshot aging operation, the bulk aging operation may have completed with failure while some snapshots were left behind.

System Impact
Volumes stuck in Destroying state have the following impacts:

• Storage capacity consumption: Stuck volumes continue to consume capacity that cannot be reclaimed until they are fully removed.
• Upgrade impact (NDU): Volumes in an invalid state may cause pre-check failures and prevent successful upgrades.
• Management operations: No direct impact to host I/O, but may complicate capacity planning and volume management visibility.
• Alert noise: The PowerDoctor alert persists until the condition is resolved.

The PowerDoctor alert is not cleared immediately after all Destroying state volumes are removed. PowerDoctor runs on a scheduled interval (24 hours for this alert), and the alert will clear during the next run.

Users can manually run a Health Check from the UI to validate the current state:

• A successful Volumes Stuck in Destroying State check confirms that the condition is resolved.
• The alert may still be visible until the next scheduled PowerDoctor run.

This delayed alert clearance is expected behavior for all PowerDoctor alerts and is not specific to the Volumes Stuck in Destroying State condition.

Automatic Resolution

PowerDoctor does NOT automatically repair this issue.
Automatic repair is intentionally not provided for safety reasons, as stuck volumes may indicate underlying system issues that require investigation.

Self Service Resolution 

Service Shell command - svc_clear_destroying_volumes has been introduced in PowerStoreOS v5.0 which is designed to identify and clear this condition. 

• The command must be run on the lead appliance.
• If you see this error, SSH to the lead appliance and run the command from there.

The svc_clear_destroying_volumes command:

• Queries the database for volumes in the Destroying state.
• Filters out volumes that have active delete operations in progress.
• Categorizes volumes into Block (not in recycle bin), Block (in recycle bin), and File volumes.
• Allows the user to selectively delete Block volumes.
• Handles concurrent delete operations gracefully (treats "object not found" as success).

Usage

# Basic usage - interactive mode with confirmation prompts
svc_clear_destroying_volumes

# Skip confirmation prompts (auto-confirm all deletions)
svc_clear_destroying_volumes --yes
svc_clear_destroying_volumes -y

Command Line Options

                                                                Table 2. CLI Options

Interactive Prompts

When run without the --yes flag, the script prompts for confirmation before deleting each category of volumes:

• yes - Proceed with deletion of volumes in this category
• no - Skip this category and move to the next
• quit - Skip remaining categories and show final summary

Output Sections

The script organizes volumes into three sections:

Section A: Block Volumes NOT in Recycle Bin

Block volumes in the Destroying state that are not in the recycle bin. These volumes will be deleted using DeleteVolumeCommand.

Section B: Block Volumes IN Recycle Bin

Block volumes in the Destroying' state that are in the recycle bin. These volumes will also be deleted using DeleteVolumeCommand.

Section C: File Volumes

File volumes in the Destroying state. Deletion is not yet implemented for File volumes. Contact Dell Support for assistance with these volumes.

Final Summary

After processing all sections, the script displays a final summary showing:

• Number of volumes found in each section
• Number of volumes successfully deleted
• Number of volumes that failed to delete
• Sections that were skipped by the user

If any deletions failed or if File volumes were found, the script displays Dell Support contact information.

Example Output:

======================================================================
       SVC_CLEAR_DESTROYING_VOLUMES
======================================================================

Querying database for volumes in 'Destroying' state...
Found 3 volume(s) stuck in 'Destroying' state.


Section A: Block Volumes in Destroying State (NOT in Recycle Bin):
------------------------------------------------------------

  [1] Volume:
      ID:           abc12345-1234-5678-90ab-cdef12345678
      Name:         test-volume-1
      Type:         Primary
      Storage Type: Block

Do you want to delete these 1 Block volume(s) in Destroying state (NOT in Recycle Bin)? [yes/no/quit]: yes

Deleting Block volumes in Destroying state (NOT in Recycle Bin)...

  Sending DeleteVolumeCommand for: test-volume-1 (abc12345-1234-5678-90ab-cdef12345678)
  [SUCCESS] Volume deleted successfully:
            Name: test-volume-1
            ID:   abc12345-1234-5678-90ab-cdef12345678

...

======================================================================
                    FINAL SUMMARY
======================================================================

  Section A: Block Volumes in Destroying State (NOT in Recycle Bin)
    Found:   1
    Deleted: 1
    Failed:  0

  Section B: Block Volumes in Destroying State (IN Recycle Bin)
    Found:   0

  Section C: File Volumes in Destroying State
    Found:   0

------------------------------------------------------------
  Total Volumes Found:   1
  Total Deleted:         1
  Total Failed:          0
------------------------------------------------------------

  SUCCESS: All requested volume deletions completed successfully.

======================================================================

Script completed.

Prevention

To minimize the risk of volumes becoming stuck in the Destroying state:

• Avoid interruptions – Do not interrupt delete operations or restart services during active volume deletions.
• Monitor delete operations – When deleting large numbers of volumes or snapshots, monitor progress until completion.
• Maintain system health – Ensure the system is in a healthy state before bulk deletions (no active alerts, all nodes online, management database healthy).
• Keep software up to date – Run the latest PowerStoreOS versions and patches, as newer releases include improvements to delete operation reliability.

File Volume Limitation: File volume deletion is not yet implemented. If File volumes are found in Destroying state, contact Dell Support for manual cleanup assistance.

Contact Dell Technical Support for assistance.

Refer to this KB article number for expedited service.

Related PowerDoctor Checks

This alert is generated by the PowerDoctor health check:

• Check Name: volumes_in_destroying_state_check
• Check Frequency: Every 24 hours (1440 minutes)
• Check Scope: Master appliance only
• Check Type: Periodic health check

Technical Details

The health check performs the following:

1. Queries the management database for volumes in the Destroying state.
2. Checks if associated delete commands exist and their completion status.
3. Identifies volumes where delete command state is COMPLETED OR no delete command exists for the volume.
4. Raises a MINOR alert if one or more stuck volumes are found.
5. Automatically clears the alert when no stuck volumes remain.