ECS: Checking S3 Object existence

Using HEAD to more efficiently check an object existence.

Object listing is an inherently inefficient way of checking for the existence of a specific object. Index searching is not only relatively slow for this specific request, but it can also affect overall load and performance.

Requesting a specific object is always more efficient than requesting one or more objects that match parameters. Object listing is a form of search where distributed indexes must be queried.

Using HEAD on the object is the best method for checking it exists. With ECS object SDK this is done with the GetObjectMetadataRequest.

 
If the object does NOT exist, the SDK throws the following exception due to a 404 error response:

Exception in thread "main" com.emc.object.s3.S3Exception: Not Found

Here is an example of using the GetObjectMetadataRequest which is used in the getObjectMetadata client method:

try { GetObjectMetadataRequest gom = new GetObjectMetadataRequest(<bucket>, <key>); S3ObjectMetadata om = s3.getObjectMetadata(gom); System.out.println("etag: " + om.getETag()); System.out.println("last modified: " + om.getLastModified().toString()); } catch(com.emc.object.s3.S3Exception e) { System.out.println("What happened: " + e.getMessage()); }

  
If using the AWS java SDK and the object does not exist, the SDK throws the following exception. 

com.amazonaws.services.s3.model.AmazonS3Exception: Not Found

 
Here is an example of it:

try { GetObjectMetadataRequest gom = new GetObjectMetadataRequest(AWSS3Factory.S3_BUCKET, key); s3.getObjectMetadata(gom); } catch(com.amazonaws.services.s3.model.AmazonS3Exception e) { System.out.println("What happened: " + e.getMessage()); }

 Check if the object exists to determine if putObject is needed.

If checking for object existence to determine if putObject is needed, then consider using PutObjectRequests conditional methods:
 

withIfMatch() and withIfNoneMatch().

Both of these methods need an object etag value and uses the headers "if-match," "if-none-match" in the PUT request.
There are also the methods withIfUnmodifiedSince() and withIfModifiedSince() which each takes a Date object.

If the etag or md5 are not available, it would needed to retrieve it with a GetObjectMetadataRequest.
An efficient alternative would be to use the withIfUnmodifiedSince() if the Date was old enough.

SR with ECS technical support

If there is a concern about the existence of an object, open a service request with the ECS support team.