ECS: How to enable a bucket lifecycle policy to handle the deletion of incomplete MPUs
Summary: ECS needs a life cycle policy enabled to automatically handle expired incomplete Multi-Part Uploads (MPUs).
Instructions
How to enable automatic deletion of incomplete MPUs.
ECS is designed to handle MPU uploads for S3 data. The object is split into multiple parts and uploaded in parallel. When all parts are uploaded, the application sends a manifest that created the final object from all parts on the ECS.
By default, ECS does not enforce an expiry time between the first and final upload.
Parts that get uploaded but never united to an object (MPU aborted or no manifest sent) occupy space and never get cleaned up.
Instructions
Upload the life cycle configuration. Either use the method described in the ECS Data Access Guide or another tool, like S3 Browser. Here is a life cycle policy to cancel old MPUs:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> <Rule> <ID>lifecycle-v2-expire-non-current-and-dmarkers-and-mpu</ID> <Filter/> <Status>Enabled</Status> <AbortIncompleteMultipartUpload> <DaysAfterInitiation>1</DaysAfterInitiation> </AbortIncompleteMultipartUpload> </Rule> </LifecycleConfiguration>
To upload a bucket life cycle using S3 Browser, connect to the bucket and use the following settings:
If incomplete MPUs exist before a life cycle policy with AbortIncompleteMultipartUpload is added, these will not be affected by the life cycle policy. ECS support must enable the Life cycle Delete Scanner (LDS) to apply the life cycle policy to existing objects. Open a Service Request if the LDS must be enabled on existing objects.