Start a Conversation

Unsolved

A

5 Practitioner

 • 

274.2K Posts

3419

October 15th, 2018 05:00

S3 metadata tags on ECS

​Hi,​

​IHAC that is saying S3 metadata tags are not working on ECS, but I'm not sure they are using them correctly.​

​User uploaded the data and use s3 tags which are set for a bucket , however when they try to query it back it does not work.​

​File name MockFile Metadata associated as below:​

​Metadata key x-amz-meta-country metadatavalue UK​

​Metadata key x-amz-meta-date-created metadatavalue 2018-10-05T08:51:03.293Z​

​Metadata key x-amz-meta-field-1 metadatavalue MockFile​

​Metadata key x-amz-meta-field-2 metadatavalue Test value field1​

​Metadata key x-amz-meta-field-3 metadatavalue Test value field2​

​Metadata key x-amz-meta-field-4 metadatavalue Test value field3​

​Metadata key x-amz-meta-owner metadatavalue Dummy Value​

​Metadata key x-amz-meta-status metadatavalue PROCESSED​

​User query : x-amz-meta-status == 'NEW' and x-amz-meta-country == 'UK'​

​Is there anything wrong with how they are doing this, or is it an issue with ECS?​

​Thanks, Jon.​

281 Posts

October 15th, 2018 07:00

This might be better served by an SR, but at a high level it looks okay. Can you confirm the indexing settings on the bucket? Only other issue is that your query doesn't match the MockFile metadata (status=PROCESSED).

Maybe one other thing: when they're uploading the file, how are they passing the metadata? What SDK? Normally, in the SDKs you only pass 'country' not 'x-amz-meta-country'. If you have S3Browser or Cyberduck you could inspect the object's metadata to see what's there.

5 Practitioner

 • 

274.2K Posts

October 15th, 2018 10:00

Thanks for the reply Jason, I have forwarded on to see what they say.

5 Practitioner

 • 

274.2K Posts

October 16th, 2018 11:00

Hi Jason,

The responses from the customer are as follows;

  1. What SDK are you using?
    • The SDK in use is ECS object client 3.1.1
  2. How is the metadata being passed when uploading the file?
    • The metadata is constructed using the SDK and tagged to the S3object
  3. Normally in the SDKs, you only pass 'country' not 'x-amz-meta-country'.
    • The examples we could see were having the prefix x-amz-meta-country. Having the prefix is not important or us, but the ability to query is vitally important. Please suggest the way we could query.

I have asked if they can try using 'country' instead of 'x-amz-meta-country' and review the status used for the query, but in the mean time, is there anything in 1 and 2 that might give a clue as to the issues they are having?

I have also raised an SR.

Thanks, Jon.

110 Posts

October 16th, 2018 13:00

We have an MD search example in our samples project on github that may be useful to them.  The gist is, when you set the metadata on the object using the client, you do not include the x-amz-meta- prefix, but when you create the index fields, and when you search on them, you do use the prefix.  I realize it's a little confusing, but that is how it works.

No Events found!

Top