PowerScale: Listing S3 Bucket Objects Loops Infinitely when Using API ListObjects or ListObjectsV2

Sammanfattning: The S3 App or S3 command uses S3 API listobject or listobjectv2 to query objects from the S3 bucket. In some instances, PowerScale OneFS S3 returns the wrong result if dirCache is there but it does not fulfill the request. The S3 App stops responding due to list objects experiencing an infinite loop. ...

Den här artikeln gäller för Den här artikeln gäller inte för Den här artikeln är inte kopplad till någon specifik produkt. Alla produktversioner identifieras inte i den här artikeln.
Kolla in andra resurser

Symptom

S3 App stops responding due to listing a bucket which does not complete.

S3 command ls bucket repeatedly gets the same objects returned in a paginated list.

Orsak

This may experience an S3 cache issue. 
 
If the client requests ListObjects with max entry = 1000, internally we may get 10,000 entries (if directory is wide) but only return 1000 of them to client, then reusing the remaining 9000 for a future request. On the next request, if there is a cache, and there are still files under the directory which are not yet returned to the client, and the current cache does not have enough entries to fulfill the request, we experience the issue and the wrong results are returned.

Upplösning

Dell Engineering is working on a permanent resolution for this issue. A roll-up patch (RUP) patch is expected to be released.

The current workaround is to configure S3 ls DIR_CACHE_TTL to 0 temporarily: 
# isi_for_array -X  "/usr/likewise/bin/lwsm inject s3 s3_ls_set_dir_cache_ttl on 0"
If the S3 process restarts or a node reboots, apply the workaround again on the node since it has reverted back to the default 1 hour when S3 restarted.

To manual DIR_CACHE_TTL: 
#isi_for_array -X  "/usr/likewise/bin/lwsm inject s3 s3_ls_set_dir_cache_ttl off"

Berörda produkter

Isilon, PowerScale
Artikelegenskaper
Artikelnummer: 000223936
Artikeltyp: Solution
Senast ändrad: 11 juni 2024
Version:  3
Få svar på dina frågor från andra Dell-användare
Supporttjänster
Kontrollera om din enhet omfattas av supporttjänster.