Skip to main content
Start a Conversation

Unsolved

manne_p

1 Message

7348

September 17th, 2018 03:00

S3 signature does not match

Hello!

I'm a developer that is trying to integrate (send data) to a ECS instance using the S3 API.

In maybe 1-5% of the requests we get an error stating that signature doesnt match. Obviously the auth is correct, since most requests goes through and data is stored.

The Error thrown by the client is (hand typed so might be some typos):

com.amazonaws.request - Received error response: com.amazonaws.services.s3.model.amazonS3Exception: The request signature we calculated does not match the signature you provided. Check your secret Access Key and signing method.

I'm wondering if someone else experienced this? It's the random bahavior and the small amount of errors that is a bit confusing in my opinion.

JasonCwik

281 Posts

September 18th, 2018 08:00

What version of ECS? Also check the name of the object getting uploaded and/or any metadata you're including. If this is the Java SDK, you can turn on request logging to get all the request headers. https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/java-dg-logging.html#sdk-net-logging-verbose

Amarjit01

2 Posts

April 29th, 2022 07:00

We are using ECS Community Edition v3.6.2.0 have same error reported: 

Caused by: software.amazon.awssdk.services.s3.model.S3Exception: The request signature we calculated does not match the signature you provided. Check your Secret Access Key and signing method.

I am 100% we are using the correct Secret Access Key. To be sure, where can I check the setting for the signing method in the Community edition?

We run a number of tests and we have 2 partial update tests that give this error. The same code executed against the ECS TestDrive (https://object.ecstestdrive.com) are successful.

DELL-Sam L

Moderator

 • 

6.9K Posts

April 29th, 2022 11:00

Hello Amarjit01,

Here are the links to a couple of kb’s that maybe of assistance.

https://dell.to/3rZNlRs

https://dell.to/3MFYted

DELL-Sam L

Moderator

 • 

6.9K Posts

May 3rd, 2022 14:00

 Hello Amarjit01,

Here is what the kb's state.

 

ECS: 3.7: S3: (HTTP 403) The request signature we calculated does not match the signature you provided

Summary: After upgrade to ECS Release 3.7 S3 application(s) show error: (HTTP 403) The request signature we calculated does not match the signature you provided This happens for applicationsSee more

Audience Level: Customer

Article Content

Symptoms

After upgrade to ECS Release 3.7 S3 application(s) show error: (HTTP 403)  The request signature we calculated does not match the signature you provided

This only affects applications utilizing signature version 4. 
s3cmd --host 10.246.151.145:9020 ls s3://restic
ERROR: S3 error: 403 (SignatureDoesNotMatch): The request signature we calculated does not match the signature you provided. Check your Secret Access Key and signing method. For more information, see REST Authentication and SOAP#


from ECS logs, "?location" request are successful but other PUT/GET requests fail: 
 Resp     Bucket/
                                                                                                           Size        Time     Object/
Node            Time             Request ID                       Prot  Type   MPU  Client IP       Status (bytes)     (ms)     Options
10.x.x.x  04-20 10:10:47   0af69791:1802f264522:3d4a:8cb    s3    GET    -    10.x.x.x                200    330         4        restic/?location
10.x.x.x  04-20 10:10:47   0af69791:1802f264522:3c18:b99    s3    GET    -    10.x.x.x                403    330         2        restic/?delimiter=%2F
                                                                                                           

search for the error 403: 
svc_log -f 0af69791:1802f264522:3c18:b99 -sr dataheadsvc
svc_log v1.0.26 (svc_tools v2.3.0)                 Started 2022-04-20 10:15:07

Running on nodes:              
     
      
Time range:                    2022-04-19 10:15:07 - 2022-04-20 10:15:07
Filter string(s):              '0af69791:1802f264522:3c18:b99'
Show nodename(s):              True
Search reclaim logs (if any):  False

169.x.x.x 2022-04-20T10:10:47,896 [qtp2066748233-27517-0af69791:1802f264522:3c18:b99-s3-10.x.x.x] ERROR  https://dell.to/39p37yQ (line 335) Signature mismatch CalcSignature: 692c3f2795f0d41f83202e82b6643f24cfe9e74074b0752e92b1a81d20b861db, ClientSignature: 08bfabe59c94a9b3e36d47be9f570c1f1b9dd93928d267462073ca3a84076f46, StringToSign AWS4-HMAC-SHA256
169.x.x.x 2022-04-20T10:10:47,896 [qtp2066748233-27517-0af69791:1802f264522:3c18:b99-s3-10.x.x.x] ERROR  https://dell.to/39p37yQ (line 335) Signature mismatch CalcSignature: 692c3f2795f0d41f83202e82b6643f24cfe9e74074b0752e92b1a81d20b861db, ClientSignature: 08bfabe59c94a9b3e36d47be9f570c1f1b9dd93928d267462073ca3a84076f46, StringToSign AWS4-HMAC-SHA256

Cause

Signature version 4 incorporates the bucket region into the authentification.
In version 3.7 changes were made to bucket-location API. The response from the API is currently " ", causing the signature mismatch.
This is being reworked and will be updated in a later release.

Invalid request: 
Authorization: AWS4-HMAC-SHA256 Credential=mathias/20220419/ /s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**

A valid request is formed including the region: 
Authorization: AWS4-HMAC-SHA256 Credential=mathias/20220419/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**


Resolution

The fix for this issue will be in release 3.7.0.2

There is two options as a workaround.

First option is to not use signature version 4 and use signature version 2 instead, if applicable. 

The second option is to configure a default location.
Please check the documentation for your application how to properly set the region. The default Region is "us-east-1"

Examples:
minio mc:
https://dell.to/3yhbdUR

restic:
set variable AWS_DEFAULT_REGION to the region
or
-o s3.region=" "

Amarjit01

2 Posts

May 3rd, 2022 14:00

Hello DELL-Sam L 

When trying to access the kb's I get: "This article is permission based. Find another article."

How can I access - I do have a DELL account, but still cannot access the kb's!

DELL-Sam L

Moderator

 • 

6.9K Posts

May 3rd, 2022 14:00

ECS - S3 not working with signature v4 but does with v2

Summary: S3 signature v4 is supported since ECS code version 3.0 but connection might fail when LoadBalancer (LB) or proxy server is configured wrong

Audience Level: Customer

Article Content

Symptoms




Applications such as S3 Browser or CloudBerry Explorer cannot connect to ECS using S3 signature v4 and below error message can be seen: 
The request signature we calculated does not match the signature you provided. Check your Secret Access Key and signing method. For more information, see REST Authentication and SOAP Authentication for details.

Errors shown in dataheadsvc.log:

2018-02-08T01:02:22,160 [qtp1281445260-47251-ac16c920:16153c4ec5b:5428:a-s3-192.168.x.x] ERROR  https://dell.to/3kEBnsm (line 270) Signature mismatch CalcSignature: 1beacc1d4410c0d39a18e99b241a8c430ad7f76f030a54595406901feddfddb1, ClientSignature: 874250cee333e6aba9b12abfd279516408a083da60d0b1cc3f8b196a6fe8cdb1
2018-02-08T01:02:22,160 [qtp1281445260-47251-ac16c920:16153c4ec5b:5428:a-s3-192.168.x.x] ERROR  https://dell.to/3kEBnsm (line 270) Signature mismatch CalcSignature: 1beacc1d4410c0d39a18e99b241a8c430ad7f76f030a54595406901feddfddb1, ClientSignature: 874250cee333e6aba9b12abfd279516408a083da60d0b1cc3f8b196a6fe8cdb1
2018-02-08T01:02:22,160 [qtp1281445260-47251-ac16c920:16153c4ec5b:5428:a-s3-192.168.x.x] ERROR  https://dell.to/3LJy66X (line 178) authenticate failed. RequestId ac16c920:16153c4ec5b:5428:a. Error https://dell.to/38SYBso.api.service.impl.resource.s3.S3Exception

Using S3 signature v2 is working.

Cause

Either LoadBalancer or Proxy Server was changing host header to point to different end point than what was configured.

Resolution


Please get answers to below questions:
  • What ECS Version is installed? S3 Signature v4 is supported since ECS 3.0 (https://dell.to/3LGP4mJ)
  • What version of tools used such as Cloudberry Explorer and/or S3 Browser?
  • Is the connection via Load Balancer or Proxy Server?
  • If yes: Please try the v4 connection by using an ECS node as endpoint
  • If this is working, please have the customer further investigate the settings of Load Balancer / Proxy Server

For Apache the below settings should be verified:
Turn this option on to preserve the Host header 
ProxyPreserveHost Directive
Description:
Use incoming Host HTTP request header for proxy request
Syntax:
ProxyPreserveHost On|Off
Default:
ProxyPreserveHost Off
Context:
server config, virtual host, directory
Status:
Extension
Module:
mod_proxy
Compatibility:
Usable in directory context in 2.3.3 and later.
When enabled, this option will pass the Host: line from the incoming request to the proxied host, instead of the hostname specified in the ProxyPass line.
This option should normally be turned Off. It is mostly useful in special configurations like proxied mass name-based virtual hosting, where the original Host header needs to be evaluated by the backend server.

More details can be found here:
https://dell.to/3KHPdVz



 

Additional Information

View More

View All

No Events found!

Top