PowerScale: Listing S3 Bucket Objects Loops Infinitely when Using API ListObjects or ListObjectsV2

Zhrnutie: The S3 App or S3 command uses S3 API listobject or listobjectv2 to query objects from the S3 bucket. In some instances, PowerScale OneFS S3 returns the wrong result if dirCache is there but it does not fulfill the request. The S3 App stops responding due to list objects experiencing an infinite loop. ...

Tento článok sa vzťahuje na Tento článok sa nevzťahuje na Tento článok nie je viazaný na žiadny konkrétny produkt. V tomto článku nie sú uvedené všetky verzie produktov.
Pozrite si ďalšie zdroje

Symptómy

S3 App stops responding due to listing a bucket which does not complete.

S3 command ls bucket repeatedly gets the same objects returned in a paginated list.

Príčina

This may experience an S3 cache issue. 
 
If the client requests ListObjects with max entry = 1000, internally we may get 10,000 entries (if directory is wide) but only return 1000 of them to client, then reusing the remaining 9000 for a future request. On the next request, if there is a cache, and there are still files under the directory which are not yet returned to the client, and the current cache does not have enough entries to fulfill the request, we experience the issue and the wrong results are returned.

Riešenie

Dell Engineering is working on a permanent resolution for this issue. A roll-up patch (RUP) patch is expected to be released.

The current workaround is to configure S3 ls DIR_CACHE_TTL to 0 temporarily: 
# isi_for_array -X  "/usr/likewise/bin/lwsm inject s3 s3_ls_set_dir_cache_ttl on 0"
If the S3 process restarts or a node reboots, apply the workaround again on the node since it has reverted back to the default 1 hour when S3 restarted.

To manual DIR_CACHE_TTL: 
#isi_for_array -X  "/usr/likewise/bin/lwsm inject s3 s3_ls_set_dir_cache_ttl off"

Dotknuté produkty

Isilon, PowerScale
Vlastnosti článku
Číslo článku: 000223936
Typ článku: Solution
Dátum poslednej úpravy: 11 jún 2024
Verzia:  3
Nájdite odpovede na svoje otázky od ostatných používateľov spoločnosti Dell
Služby podpory
Skontrolujte, či sa na vaše zariadenie vzťahujú služby podpory.