ECS: OBS: Copy Between Buckets Fails with HTTP 403-SignatureDoesNotMatch

Summary: Copying objects between two buckets within the same namespace fails with: 03 (SignatureDoesNotMatch): The request signature we calculated does not match the signature you provided. Check your Secret Access Key and signing method. For more information, see REST Authentication and SOAP Authentication for details. ...

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

Copying objects between two buckets in the same namespace fails with: 
ERROR HTTP 403 (SignatureDoesNotMatch): The request signature we calculated does not match the signature you provided. Check your Secret Access Key and signing method. For more information, see REST Authentication and SOAP Authentication for details.)

Other buckets may be able to copy objects between buckets.

Sample error from client (s3cmd in below example)

user@ubuntu:~$ s3cmd cp s3://sourcebucket/0001.txt s3://destinationbucket/0001.txt --add-header=x-emc-copy-mode:deep
ERROR: Copy failed for: 's3://sourcebucket/0001.txt' (403 (SignatureDoesNotMatch): The request signature we calculated does not match the signature you provided. Check your Secret Access Key and signing method. For more information, see REST Authentication and SOAP Authentication for details.)

Logs from svc_request for this session:

x.x.x.159 02-20 11:43:04 0a3c129f:195052e2352:383d:6d0 s3 GET - x.x.x.253 200 454 4 y-test sourcebucket/0001.txt?acl
x.x.x.159 02-20 11:43:04 0a3c129f:195052e2352:37e3:1045 s3 HEAD - x.x.x.253 200 0 2 y-test sourcebucket/0001.txt
x.x.x.159 02-20 11:43:04 0a3c129f:195052e2352:383d:6d3 s3 PUT No x.x.x.253 400 - 7 y-test destinationbucket/0001.txt
x.x.x.159 02-20 11:43:04 0a3c129f:195052e2352:37e3:1048 s3 PUT No x.x.x.253 403 - 1 - destinationbucket/0001.txt

ECS logs show:

varray mismatch: source bucket has urn:storageos:VirtualArray:IDb.urn:storageos:ReplicationGroupInfo:ID:global, dest bucket has urn:storageos:VirtualArray:ID.urn:storageos:ReplicationGroupInfo:ID:global

Tracing the HTTP 400 requests shows "varray mismatch":

xxx.xxx.x.1 2025-02-20 11:43:04,682 0a3c129f:195052e2352:383d:6d3 x.x.x.159:9020 x.x.x.253:60062 y-test-1 - PUT y-test destinationbucket 0001.txt - HTTP/1.1 400 7 - - 6 - copy - - 'X-Forwarded-For: -' 'x-amz-meta-firstName: -' 'x-amz-meta-lastname: -' 'x-amz-meta-age: -' xxx.xxx.x.1 2025-02-20 11:43:04,682 0a3c129f:195052e2352:383d:6d3 x.x.x.159:9020 x.x.x.253:60062 y-test-1 - PUT y-test destinationbucket 0001.txt - HTTP/1.1 400 7 - - 6 - copy - - 'X-Forwarded-For: -' 'x-amz-meta-firstName: -' 'x-amz-meta-lastname: -' 'x-amz-meta-age: -' xxx.xxx.x.1 2025-02-20T11:43:04,675 [qtp2087040347-23314-0a3c129f:195052e2352:383d:6d3-s3-x.x.x.253] INFO V4Signer.java (line 118) credential: y-test-1/20250220/US-EAST-1/s3/aws4_request, amz_expires: null, amz_signed_headers: content-type;host;x-amz-content-sha256;x-amz-copy-source;x-amz-date;x-amz-meta-s3b-last-modified;x-amz-meta-sha256;x-amz-metadata-directive;x-amz-storage-class;x-emc-copy-mode;x-emc-mtime, amz_signature: 65edd5ab6e7b30a19993f88c160df46b826010f199583a41468272321c229ea0, payloadHash: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855, amz_date: 20250220T114323Z xxx.xxx.x.1 2025-02-20T11:43:04,678 [qtp2087040347-23314-0a3c129f:195052e2352:383d:6d3-s3-x.x.x.253] ERROR S3Exception.java (line 1733) got object access exception. RequestId 0a3c129f:195052e2352:383d:6d3 com.emc.storageos.objcontrol.object.exception.ObjectAccessException: varray mismatch: source bucket has urn:storageos:VirtualArray:<ID>.urn:storageos:ReplicationGroupInfo:<ID>:global, dest bucket has urn:storageos:VirtualArray:<ID>.urn:storageos:<ID>:global

Cause

Most AWS-compliant tools use the s3 copy option when copying between buckets on the same source. This option does not download and upload the object using the client and instead asks the target to perform the copy operation.

When the source and target bucket are hosted on different replication groups the copy fails, ECS allows S3 copy only within the same replication group. The concept of replication groups is not used in Amazon AWS, client tools do not detect that there is a different replication group used.

How to verify the buckets are in different Replication Group:

From CLI run: 

svc_bucket list -n <namespace>

admin@ecsnode2:~> svc_bucket list -n y-test 
svc_bucket v1.1.3 (svc_tools v2.24.0)                 Started 2025-07-29 07:18:45

                                                                                                                                       Bucket     Temp      
                                                                 Replication         Owner            Owner           API     FS       Versioning Failed    
Bucket Name                            Namespace                 Group               User             VDC             Type    Enabled  Enabled    (TSO)     

destinationbucket                      y-test                    local_vdc           y-test-1         VDC1            S3      False    Disabled   False       
destinationiam                         y-test                    local_vdc           urn:ec...t:root  VDC1            S3      False    Disabled   False       
destsamerg                             y-test                    RG1                 y-test-1         VDC1            S3      False    Disabled   False       
iam                                    y-test                    RG1                 urn:ec...t:root  VDC1            S3      False    Disabled   False       
iamsamerg                              y-test                    RG1                 urn:ec...t:root  VDC1            S3      False    Disabled   False       
new-bucket-37494b67                    y-test                    RG1                 y-test-1         VDC1            S3      False    Enabled    False       
s3                                     y-test                    RG1                 y-test-1         VDC1            S3      False    Disabled   False       
sourcebucket                           y-test                    RG1                 y-test-1         VDC1            S3      False    Disabled   False       
sourceiam                              y-test                    RG1                 urn:ec...t:root  VDC1            S3      False    Disabled   False       
steve                                  y-test                    RG1                 y-test-1         VDC1            S3      False    Disabled   False       
versioning-test                        y-test                    RG1                 y-test-1         VDC1            S3      False    Enabled    False

In above example, both buckets with destination in their name belong to local_vdc, while the others are RG1
You could copy between all RG1 buckets as well between both local_vdc buckets, but not between local_vdc and rg1 bucket.

Resolution

This is working as designed.

Download the object from ECS to client and reupload to the new source. Alternatively use buckets within the same namespace and Replication Group.

Affected Products

ECS, ObjectScale, ECS Appliance, ECS Appliance Hardware Series, ECS Appliance Software with Encryption, ECS Appliance Software without Encryption, ObjectScale Appliance Software without Encryption, ObjectScale Software Series
Article Properties
Article Number: 000350675
Article Type: Solution
Last Modified: 30 Jul 2025
Version:  1
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.