ECS: ADO Buckets and IAM Users are Impacted on ECS 3.8.1.0 Upgrades

Summary: This article discusses the potential impact on upgrades to ECS 3.8.1.0 when Access During Outage (ADO) buckets are configured, and Identity and Access Management (IAM) is used.

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

Users may be unable to write objects using S3 PUT command when the VDC is upgraded to ECS 3.8.1.0.

Users may get the following error: 
403 ACC ESS_DENIED

Cause

This potential data unavailability (DU) scenario may occur if:
  • There are multiple ECS VDCs in a Federation
  • You are upgrading one VDC to ECS 3.8.1.0 General Availability (GA) version
  • There are ADO-enabled buckets
  • There are IAM users accessing these buckets
When the above conditions are all met, the user may experience this issue. This continues until all VDCs in the federation are upgraded to ECS 3.8.1.0.

Resolution

  • ECS 3.8.1.1 contains the fix for this issue. It was released on July 17, 2024. 
  • All ECS 3.8.1 upgrades and installs should be to the latest patch (fourth digit) release. 
  • Engineering has released ECS 3.8.1.0 General Patch 1 (GP1) as of May 28, 2024 to address this issue, until ECS 3.8.1.1 was released. 
Note: Check the "Additional Info" section of this article for steps to check if you have ADO buckets and IAM users.

Additional Information

Check for ADO-Enabled Buckets

Run the following command, the output should be all the buckets that have ADO enabled. If there are no buckets listed, this issue does not impact you.

# svc_bucket list -af

Expected Output Similar To:

admin@ecs-n1:~> svc_bucket list -af |more
svc_bucket v1.1.0 (svc_tools v2.16.0)                 Started 2024-05-27 10:43:04

                                                                                                                                       Bucket     Temp
                                                                 Replication         Owner            Owner           API     FS       Versioning Failed
Bucket Name                            Namespace                 Group               User             VDC             Type    Enabled  Enabled    (TSO)

isilon-source                            ns-test                  rg1              storage             vdc01             S3      False    Disabled   False
isilon-target                            ns-test                  rg1              storage             vdc01             S3      False    Disabled   False
isilon-test                              ns-test                  rg1              isilon-test         vdc01             S3      False    Disabled   False

 

Check if IAM is used

Run the following command. If the output is empty, then IAM is not used to access the buckets. 

# svc_log -f "credential: AKIA" -sr dataheadsvc -start 8d -sc

Expected Output Similar To:

admin@ecs-n1:~> svc_log -f "credential: AKIA" -sr dataheadsvc -start 8d -sc
svc_log v1.0.32 (svc_tools v2.16.0)                 Started 2024-05-27 14:11:30
 
Running on nodes:                        <All nodes>
Time range:                              2024-05-19 14:11:30 - 2024-05-27 14:11:30
Filter string(s):                        'credential: AKIA'
Show filename(s):                        True
Show nodename(s):                        True
Log type(s) to search for each service:  <Main Logs>
Search reclaim logs (if any):            False
Search  logs (if any):                   False
 
Total log entry matches:
 
    r1n1                     : 3
    r1n2                     : 0
    r1n3                     : 14
    r1n4                     : 0
    r1n5                     : 0
 
    Total:                   : 17

Affected Products

ECS, ECS Appliance, ECS Appliance Gen 1, ECS Appliance Gen 2, ECS Appliance Gen 3, ECS Appliance Hardware Series, ECS Appliance Software with Encryption, ECS Appliance Software without Encryption, ECS Software, Elastic Cloud Storage
Article Properties
Article Number: 000225439
Article Type: Solution
Last Modified: 21 Aug 2024
Version:  5
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.