PowerScale: Listing S3 Bucket Objects Loops Infinitely when Using API ListObjects or ListObjectsV2

Summary: The S3 App or S3 command uses S3 API listobject or listobjectv2 to query objects from the S3 bucket. In some instances, PowerScale OneFS S3 returns the wrong result if dirCache is there but it does not fulfill the request. The S3 App stops responding due to list objects experiencing an infinite loop. ...

Bu makale şunlar için geçerlidir: Bu makale şunlar için geçerli değildir: Bu makale, belirli bir ürüne bağlı değildir. Bu makalede tüm ürün sürümleri tanımlanmamıştır.
Diğer kaynaklara göz atın

Symptoms

S3 App stops responding due to listing a bucket which does not complete.

S3 command ls bucket repeatedly gets the same objects returned in a paginated list.

Cause

This may experience an S3 cache issue. 
 
If the client requests ListObjects with max entry = 1000, internally we may get 10,000 entries (if directory is wide) but only return 1000 of them to client, then reusing the remaining 9000 for a future request. On the next request, if there is a cache, and there are still files under the directory which are not yet returned to the client, and the current cache does not have enough entries to fulfill the request, we experience the issue and the wrong results are returned.

Resolution

Dell Engineering is working on a permanent resolution for this issue. A roll-up patch (RUP) patch is expected to be released.

The current workaround is to configure S3 ls DIR_CACHE_TTL to 0 temporarily: 
# isi_for_array -X  "/usr/likewise/bin/lwsm inject s3 s3_ls_set_dir_cache_ttl on 0"
If the S3 process restarts or a node reboots, apply the workaround again on the node since it has reverted back to the default 1 hour when S3 restarted.

To manual DIR_CACHE_TTL: 
#isi_for_array -X  "/usr/likewise/bin/lwsm inject s3 s3_ls_set_dir_cache_ttl off"

Etkilenen Ürünler

Isilon, PowerScale
Makale Özellikleri
Article Number: 000223936
Article Type: Solution
Son Değiştirme: 11 Haz 2024
Version:  3
Sorularınıza diğer Dell kullanıcılarından yanıtlar bulun
Destek Hizmetleri
Aygıtınızın Destek Hizmetleri kapsamında olup olmadığını kontrol edin.