Symptoms
When Kubernetes objects (Persistent Volumes (PVs) or DellCSIReplicationGroups (RGs) are issued delete commands, the kubectl command may stop responding indefinitely instead of deleting the PV or RG in question. When the RGs are successfully deleted, their associated SyncIQ policies on related PowerScale storage arrays may not be deleted as intended.
This can create issues when further replication is attempted with the same Kubernetes storage class. This issue can cause scenarios where storage space is not freed up after it is no longer necessary.
Issue is reported here:
https://github.com/dell/csm/issues/523
Cause
Finalizer tags in the CSM-managed Kubernetes objects do not properly honor and address Container Storage Modules (CSM) replication when the object is deleted. This results in the object remaining in a "Terminating" state indefinitely because the conditions in its finalizer are not met.
SyncIQ policy deletion is not properly performed when CSM replication controller and replication sidecar handle the deletion of an RG. This results in the SyncIQ policy remaining on the storage array even after its RG is deleted.
Resolution
Resolution:
Engineering has resolved this issue in Dell CSM for PowerScale 1.6.0.
Workaround:
For Kubernetes objects:
- Cancel the deletion that is not responding (for example, Ctrl+C). Manually access the objects that are failing to delete with the following command:
kubectl edit rg <RG NAME>
kubectl edit pv <PV NAME>
- This opens the relevant object in a text editor. Inside the object, there are several categories of tags, as seen below:
- Manually delete the "finalizers:" line and its entries. In the above case, this is:
- Save the file in the text editor, exit the editor, and then run the deletion command again. This process may be repeated for all replicated PVs and RGs that are failing to delete.
For deleting SyncIQ policies left on the array:
- Log in to the source PowerScale array using its web user interface.
- Go to the Data Protection tab and select SyncIQ.
- From the SyncIQ page, go to the Policies subtab.
- On the policy that must be deleted, click Actions, and then Delete Policy.
- Repeat for the target and destination PowerScale array as needed.
Affected Products
Container Storage Modules
Products
Container Storage Modules Family