This post is more than 5 years old
11 Posts
0
13558
getting Caused by: java.net.SocketException: Connection reset by peer: socket write error
I am getting Caused by: java.net.SocketException: Connection reset by peer: socket write error
when using multi-part upload with SSECustomerKey using Amazon s3 client sdk
// Create the request to upload a part.
UploadPartRequest uploadRequest = new UploadPartRequest()//.withSSECustomerKey(SSE_KEY)
.withBucketName(bucketName)
.withKey(keyName)
.withUploadId(initResponse.getUploadId())
.withPartNumber(i)
.withInputStream(is)
.withPartSize(partSize)
.withSSECustomerKey(SSE_KEY);
UploadPartResult uploadResult = S3_CLIENT.uploadPart(uploadRequest);
while upload works fine with single part upload
PutObjectRequest putRequest = new PutObjectRequest(bucketName, keyName, is, metadata).withSSECustomerKey(SSE_KEY);
S3_CLIENT.putObject(putRequest);
why is dell emc ecs throwing java.net.SocketException: Connection reset by peer: socket write error
chris_arnett
110 Posts
1
May 23rd, 2018 06:00
Thanks for posting that code. This is exactly what we needed to see. The problem is that you are specifying SSE (with server-managed key) in the init request, but specifying SSE-C (customer-managed key) in the part uploads. If you want to use SSE-C (where you manage your own keys), you need to change your init call to the following:
If you want to use SSE (with server-managed keys), then remove the customer managed key assignment in the part uploads (it will inherit the setting from the init request).
kastr1
16 Posts
1
May 18th, 2018 09:00
If you are using ECS 3.2.0.0, this is a known bug with Multi-part uploads.
The fix will be in v3.2.0.1, which should be released 22-May.
Normal uploads will work as will multi-part uploads if the URL is not percent-encoded. For 3.2.0.0, the work-around is to avoid using percent-encoding in multi-part uploads.
naveen_bijalwan
11 Posts
0
May 18th, 2018 10:00
ok thanks. I'll wait for v3.2.0.1.
"For 3.2.0.0, the work-around is to avoid using percent-encoding in multi-part uploads."
-- But I am using ip and port directly like to create AmazonS3 client.
EndpointConfiguration endpointConfiguration = new EndpointConfiguration("*.*.*.*:9021", "US_WEST_2");
Which URL should be percent-encoded?
kastr1
16 Posts
0
May 21st, 2018 12:00
I'm talking about the URL that is the MPU request. For instance:
POST /Bucket1/object.name?uploads
It is common to percent-encode the 'dot' in the URL above:
POST /Bucket1/object%2Ename?uploads
If the URL does contain a percent-encoded character (such as the %2E above), the MPU will fail until you upgrade to ECS 3.2.0.1.
naveen_bijalwan
11 Posts
0
May 22nd, 2018 05:00
Yes but I am doing using aws-java-sdk-s3 and not through rest url and my keyname and bucketName doesn't contain any special characters. As said before that It works for me without .withSSECustomerKey(SSE_KEY) but with server side encryption S3_CLIENT.uploadPart fails
kastr1
16 Posts
0
May 22nd, 2018 06:00
Your original note stated that the upload failed as a multi-part, but worked as a single upload. In BOTH cases you were using withSSECustomerKey(SSE_KEY). It never states that you were successful without "withSSECustomerKey(SSE_KEY)"
From reading the bug report, there should be no association of this error with SSE. Please verify this behavior, and be careful that any request that has no percent-encoded will cause the multi-part upload to succeed.
naveen_bijalwan
11 Posts
0
May 22nd, 2018 09:00
my bad if post is misinterpreted . I meant uploadpart is successful without "withSSECustomerKey(SSE_KEY)". I have tried it number of times but upload part is continuously failing with withSSECustomerKey(SSE_KEY). I am not sure whether withSSECustomerKey(SSE_KEY) introduces any percent-encoded due to encryption but same request is passing without withSSECustomerKey(SSE_KEY).
chris_arnett
110 Posts
0
May 22nd, 2018 15:00
Since you are using the AWS SDK, you have access to the TransferManager class. Using this to upload MPU with SSE-C against ECS Test Drive (ECS v3.2.0) works for me. Here is my code excerpt:
naveen_bijalwan
11 Posts
0
May 23rd, 2018 02:00
But how will TransferManager work with UploadPartRequest object for I don't have single stream but each part is coming as seperate stream of length say 5MB each. Is there something we can do similar to com.emc.vipr.services.s3.model.AppendObjectRequest or com.emc.vipr.services.s3.model.UpdateObjectRequest in com.emc.vipr s3-client package for aws-s3-sdk as well for ViPRS3Client doesn't work with withSSECustomerKey
The current code is
naveen_bijalwan
11 Posts
0
May 23rd, 2018 07:00
Thanks for pointing out the mistake that I was doing. I was doing objectMetadata.setSSEAlgorithm(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION); and then passing it to initRequest but going by suggestion to use
worked for me and I am able to upload file successfully using UploadPartRequest with .withSSECustomerKey(SSE_KEY) attached screenshot. thanks for quick suggestions.