Highlighted
jonsmith2
1 Nickel

S3 metadata tags on ECS

Hi,

IHAC that is saying S3 metadata tags are not working on ECS, but I'm not sure they are using them correctly.

User uploaded the data and use s3 tags which are set for a bucket , however when they try to query it back it does not work.

File name MockFile Metadata associated as below:

Metadata key x-amz-meta-country metadatavalue UK

Metadata key x-amz-meta-date-created metadatavalue 2018-10-05T08:51:03.293Z

Metadata key x-amz-meta-field-1 metadatavalue MockFile

Metadata key x-amz-meta-field-2 metadatavalue Test value field1

Metadata key x-amz-meta-field-3 metadatavalue Test value field2

Metadata key x-amz-meta-field-4 metadatavalue Test value field3

Metadata key x-amz-meta-owner metadatavalue Dummy Value

Metadata key x-amz-meta-status metadatavalue PROCESSED

User query : x-amz-meta-status == 'NEW' and x-amz-meta-country == 'UK'

Is there anything wrong with how they are doing this, or is it an issue with ECS?

Thanks, Jon.

Tags (2)
0 Kudos
4 Replies
JasonCwik
2 Iron

Re: S3 metadata tags on ECS

This might be better served by an SR, but at a high level it looks okay. Can you confirm the indexing settings on the bucket? Only other issue is that your query doesn't match the MockFile metadata (status=PROCESSED).

Maybe one other thing: when they're uploading the file, how are they passing the metadata? What SDK? Normally, in the SDKs you only pass 'country' not 'x-amz-meta-country'. If you have S3Browser or Cyberduck you could inspect the object's metadata to see what's there.

0 Kudos
jonsmith2
1 Nickel

Re: S3 metadata tags on ECS

Thanks for the reply Jason, I have forwarded on to see what they say.

0 Kudos
jonsmith2
1 Nickel

Re: S3 metadata tags on ECS

Hi Jason,

The responses from the customer are as follows;

  1. What SDK are you using?
    • The SDK in use is ECS object client 3.1.1
  2. How is the metadata being passed when uploading the file?
    • The metadata is constructed using the SDK and tagged to the S3object
  3. Normally in the SDKs, you only pass 'country' not 'x-amz-meta-country'.
    • The examples we could see were having the prefix x-amz-meta-country. Having the prefix is not important or us, but the ability to query is vitally important. Please suggest the way we could query.

I have asked if they can try using 'country' instead of 'x-amz-meta-country' and review the status used for the query, but in the mean time, is there anything in 1 and 2 that might give a clue as to the issues they are having?

I have also raised an SR.

Thanks, Jon.

0 Kudos

Re: Re: S3 metadata tags on ECS

We have an MD search example in our samples project on github that may be useful to them.  The gist is, when you set the metadata on the object using the client, you do not include the x-amz-meta- prefix, but when you create the index fields, and when you search on them, you do use the prefix.  I realize it's a little confusing, but that is how it works.

0 Kudos