ECS : Comment utiliser s3curl effectuer un téléchargement en plusieurs parties

요약: Comment utiliser s3curl pour effectuer un téléchargement en plusieurs parties (MPU).

이 문서는 다음에 적용됩니다. 이 문서는 다음에 적용되지 않습니다. 이 문서는 특정 제품과 관련이 없습니다. 모든 제품 버전이 이 문서에 나와 있는 것은 아닙니다.
기타 리소스 확인

지침

Cet article de la base de connaissances montre comment utiliser S3curl pour effectuer une opération similaire afin de prendre en charge le dépannage des problèmes liés au MPU.


#Create un fichier de test 500M 
  
dd if=/dev/urandom of=example bs=500M count=1

#check this file's md5sum value. 
Exemple
de MD5sum 9C561EAFC918D7BFAE2A1CAC376836C1

#split fichier en deux parties, example.00 et example.01 
split -n 2 example -d example.

ls -l example*
-rw-r--r-- 1 root root 524288000 Sep  4 02:13 example
-rw-r--r-- 1 root root 262144000 Sep  4 02:14 example.00
-rw-r--r-- 1 root root 262144000 Sep  4 02:14 example.01


# Init Uploads, get the uploadID. mpubucket is the bucket name, examplempu is the object name for our example. 

 

ecssh124:/usr/share/s3curl # ./s3curl.pl  --id walker --post -- "http://10.xxx.xxx.xxx:9020/mpubucket/examplempu?uploads"  |xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   266  100   266    0     0    647      0 --:--:-- --:--:-- --:--:--   648
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Bucket>mpubucket</Bucket>
  <Key>examplempu</Key>
  <UploadId>11c945604d9d4299b52987c289103f62</UploadId>
</InitiateMultipartUploadResult>

#Use Upload 1st part of the file, example.00 , and record the ETag value,

 

ecssh124:/usr/share/s3curl # ./s3curl.pl --debug --put=example.00  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=1"
s3curl: Found the url: host=10.xxx.xxx.xxx; port=9020; uri=/mpubucket/examplempu; query=uploadId=11c945604d9d4299b52987c289103f62&partNumber=1;
s3curl: replaced string: partNumber=1
s3curl: replaced string: uploadId=11c945604d9d4299b52987c289103f62
s3curl: ordinary endpoint signing case
s3curl: StringToSign='PUT\n\n\nMon, 04 Sep 2017 02:23:50 +0000\n/mpubucket/examplempu?partNumber=1&uploadId=11c945604d9d4299b52987c289103f62'
s3curl: exec curl -v -H 'Date: Mon, 04 Sep 2017 02:23:50 +0000' -H 'Authorization: AWS "S3 username":"S3 Secretkey"' -L -H 'content-type: ' -T example.00 http://10.xxx.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=1
* Hostname was NOT found in DNS cache
*   Trying 10.32.169.124...
* Connected to 10.xxx.xxx.xxx (10.xxx.xxx.xxx) port 9020 (#0)
> PUT /mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=1 HTTP/1.1
> User-Agent: curl/7.37.0
> Host: 10.xxx.xxx.xxx:9020
> Accept: */*
> Date: Mon, 04 Sep 2017 02:23:50 +0000
> Authorization: AWS "S3 username":"S3 Secretkey"
> Content-Length: 262144000
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Mon, 04 Sep 2017 02:23:50 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a20a97c:15e46a039b0:a79:3c0
< x-amz-id-2: 428b1d2b3b46e11630cc6afb9812c6047e527d1b6b51f3be014e8b6b71482163
< ETag: "d6ccfd257b8eaa2a7208bbdf2277c821"
< Content-Length: 0
<
* Connection #0 to host 10.32.xxx.xxx left intact


##Use Upload 2nd part of the file,example.01, and record the ETag value,
 

ecssh124:/usr/share/s3curl # ./s3curl.pl --debug --put=example.01  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=2"
s3curl: Found the url: host=10.32.xxx.xxx; port=9020; uri=/mpubucket/examplempu; query=uploadId=11c945604d9d4299b52987c289103f62&partNumber=2;
s3curl: replaced string: partNumber=2
s3curl: replaced string: uploadId=11c945604d9d4299b52987c289103f62
s3curl: ordinary endpoint signing case
s3curl: StringToSign='PUT\n\n\nMon, 04 Sep 2017 02:24:57 +0000\n/mpubucket/examplempu?partNumber=2&uploadId=11c945604d9d4299b52987c289103f62'
s3curl: exec curl -v -H 'Date: Mon, 04 Sep 2017 02:24:57 +0000' -H 'Authorization: AWS "S3 username":"S3 Secretkey"' -L -H 'content-type: ' -T example.01 http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=2
* Hostname was NOT found in DNS cache
*   Trying 10.32.xxx.xxx...
* Connected to 10.32.xxx.xxx (10.32.xxx.xxx) port 9020 (#0)
> PUT /mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=2 HTTP/1.1
> User-Agent: curl/7.37.0
> Host: 10.32.xxx.xxx:9020
> Accept: */*
> Date: Mon, 04 Sep 2017 02:24:57 +0000
> Authorization: AWS "S3 username":"S3 Secretkey"
> Content-Length: 262144000
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Mon, 04 Sep 2017 02:24:57 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a20a97c:15e46a039b0:b02:1
< x-amz-id-2: 428b1d2b3b46e11630cc6afb9812c6047e527d1b6b51f3be014e8b6b71482163
< ETag: "c59d28e23ac4f4b488dbf85199a73fd4"
< Content-Length: 0
<
* Connection #0 to host 10.32.xxx.xxx left intact


#create mpu.xml file, which has the ETag value from above two parts file
 

ecssh124:/usr/share/s3curl # cat mpu.xml
<CompleteMultipartUpload>
  <Part>
    <PartNumber>1</PartNumber>
    <ETag>"d6ccfd257b8eaa2a7208bbdf2277c821"</ETag>
  </Part>
  <Part>
    <PartNumber>2</PartNumber>
    <ETag>"c59d28e23ac4f4b488dbf85199a73fd4"</ETag>
  </Part>
</CompleteMultipartUpload>

#Complete a multi-part upload

 

ecssh124:/usr/share/s3curl # ./s3curl.pl --debug --post mpu.xml  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62" |xmllint --format -
s3curl: Found the url: host=10.32.xxx.xxx; port=9020; uri=/mpubucket/examplempu; query=uploadId=11c945604d9d4299b52987c289103f62;
s3curl: replaced string: uploadId=11c945604d9d4299b52987c289103f62
s3curl: ordinary endpoint signing case
s3curl: StringToSign='POST\n\n\nMon, 04 Sep 2017 02:32:32 +0000\n/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62'
s3curl: exec curl -v -H 'Date: Mon, 04 Sep 2017 02:32:32 +0000' -H 'Authorization: AWS "S3 username":"S3 SecretKey"' -L -H 'content-type: ' -X POST -T mpu.xml http://10.32.169.124:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62
* Hostname was NOT found in DNS cache
*   Trying 10.32.xxx.xxx...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 10.32.xxx.xxx (10.32.xxx.xxx) port 9020 (#0)
> POST /mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62 HTTP/1.1
> User-Agent: curl/7.37.0
> Host: 10.32.xxx.xxx:9020
> Accept: */*
> Date: Mon, 04 Sep 2017 02:32:32 +0000
> Authorization: AWS "S3 username":"S3 SecretKey"
> Content-Length: 257
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
} [data not shown]
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Mon, 04 Sep 2017 02:32:32 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a20a97c:15e46a039b0:b9e:3
< x-amz-id-2: 428b1d2b3b46e11630cc6afb9812c6047e527d1b6b51f3be014e8b6b71482163
< ETag: "b7a3f5201be2a6a4b130018f79d232b3-2"
< Content-Type: application/xml
< Content-Length: 329
<
{ [data not shown]
100   586  100   329  100   257   1257    982 --:--:-- --:--:-- --:--:--  1260
* Connection #0 to host 10.32.xxx.xxx left intact
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Location>http://10.32.xxx.xxx:9020/mpubucket/examplempu</Location>
  <Bucket>mpubucket</Bucket>
  <Key>examplempu</Key>
  <ETag>"b7a3f5201be2a6a4b130018f79d232b3-2"</ETag>
</CompleteMultipartUploadResult>


#list this file
 

ecssh124:/usr/share/s3curl # ./s3curl.pl  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/" |xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   583  100   583    0     0   5152      0 --:--:-- --:--:-- --:--:--  5205
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Name>mpubucket</Name>
  <Prefix/>
  <Marker/>
  <MaxKeys>1000</MaxKeys>
  <IsTruncated>false</IsTruncated>
  <ServerSideEncryptionEnabled>false</ServerSideEncryptionEnabled>
  <Contents>
    <Key>examplempu</Key>
    <LastModified>2017-09-04T02:32:32Z</LastModified>
    <ETag>"b7a3f5201be2a6a4b130018f79d232b3-2"</ETag>
    <Size>524288000</Size>
    <StorageClass>STANDARD</StorageClass>
    <Owner>
      <ID>walkerobj</ID>
      <DisplayName>walkerobj</DisplayName>
    </Owner>
  </Contents>
</ListBucketResult>


#Download this file, and we can know it is same as original file before the split.
 

ecssh124:/usr/share/s3curl # ./s3curl.pl  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu" >exampledownload
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  500M  100  500M    0     0   316M      0  0:00:01  0:00:01 --:--:--  317M
ecssh124:/usr/share/s3curl # md5sum example exampledownload
9c561eafc918d7bfae2a1cac376836c1  example
9c561eafc918d7bfae2a1cac376836c1  exampledownload

 

해당 제품

ECS Appliance

제품

ECS Appliance, ECS Appliance Hardware Gen1 U-Series, ECS Appliance Software with Encryption, ECS Appliance Software without Encryption, Elastic Cloud Storage
문서 속성
문서 번호: 000014693
문서 유형: How To
마지막 수정 시간: 09 9월 2025
버전:  5
다른 Dell 사용자에게 질문에 대한 답변 찾기
지원 서비스
디바이스에 지원 서비스가 적용되는지 확인하십시오.