ECS：s3curlを使用してマルチパート アップロードを実行する方法

Resumen: s3curlを使用してマルチパート アップロード(MPU)を実行する方法。

Este artículo se aplica a Este artículo no se aplica a Este artículo no está vinculado a ningún producto específico. No se identifican todas las versiones del producto en este artículo.
Consulte estos recursos

Instrucciones

このKBでは、S3curlを使用して同様の操作を実行する方法を示し、MPUに関するトラブルシューティングの問題をサポートします。


500Mのテストファイル #Create 
  
dd if=/dev/urandom of=example bs=500M count=1

#check this file's md5sum value. 
md5sum の例
9c561eafc918d7bfae2a1cac376836c1 の例

ファイルを example.00 と example.01 の 2 つの部分に #split します。 
split -n 2 example -d example.

ls -l example*
-rw-r--r-- 1 root root 524288000 Sep  4 02:13 example
-rw-r--r-- 1 root root 262144000 Sep  4 02:14 example.00
-rw-r--r-- 1 root root 262144000 Sep  4 02:14 example.01


# Init Uploads, get the uploadID. mpubucket is the bucket name, examplempu is the object name for our example. 

 

ecssh124:/usr/share/s3curl # ./s3curl.pl  --id walker --post -- "http://10.xxx.xxx.xxx:9020/mpubucket/examplempu?uploads"  |xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   266  100   266    0     0    647      0 --:--:-- --:--:-- --:--:--   648
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Bucket>mpubucket</Bucket>
  <Key>examplempu</Key>
  <UploadId>11c945604d9d4299b52987c289103f62</UploadId>
</InitiateMultipartUploadResult>

#Use Upload 1st part of the file, example.00 , and record the ETag value,

 

ecssh124:/usr/share/s3curl # ./s3curl.pl --debug --put=example.00  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=1"
s3curl: Found the url: host=10.xxx.xxx.xxx; port=9020; uri=/mpubucket/examplempu; query=uploadId=11c945604d9d4299b52987c289103f62&partNumber=1;
s3curl: replaced string: partNumber=1
s3curl: replaced string: uploadId=11c945604d9d4299b52987c289103f62
s3curl: ordinary endpoint signing case
s3curl: StringToSign='PUT\n\n\nMon, 04 Sep 2017 02:23:50 +0000\n/mpubucket/examplempu?partNumber=1&uploadId=11c945604d9d4299b52987c289103f62'
s3curl: exec curl -v -H 'Date: Mon, 04 Sep 2017 02:23:50 +0000' -H 'Authorization: AWS "S3 username":"S3 Secretkey"' -L -H 'content-type: ' -T example.00 http://10.xxx.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=1
* Hostname was NOT found in DNS cache
*   Trying 10.32.169.124...
* Connected to 10.xxx.xxx.xxx (10.xxx.xxx.xxx) port 9020 (#0)
> PUT /mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=1 HTTP/1.1
> User-Agent: curl/7.37.0
> Host: 10.xxx.xxx.xxx:9020
> Accept: */*
> Date: Mon, 04 Sep 2017 02:23:50 +0000
> Authorization: AWS "S3 username":"S3 Secretkey"
> Content-Length: 262144000
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Mon, 04 Sep 2017 02:23:50 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a20a97c:15e46a039b0:a79:3c0
< x-amz-id-2: 428b1d2b3b46e11630cc6afb9812c6047e527d1b6b51f3be014e8b6b71482163
< ETag: "d6ccfd257b8eaa2a7208bbdf2277c821"
< Content-Length: 0
<
* Connection #0 to host 10.32.xxx.xxx left intact


##Use Upload 2nd part of the file,example.01, and record the ETag value,
 

ecssh124:/usr/share/s3curl # ./s3curl.pl --debug --put=example.01  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=2"
s3curl: Found the url: host=10.32.xxx.xxx; port=9020; uri=/mpubucket/examplempu; query=uploadId=11c945604d9d4299b52987c289103f62&partNumber=2;
s3curl: replaced string: partNumber=2
s3curl: replaced string: uploadId=11c945604d9d4299b52987c289103f62
s3curl: ordinary endpoint signing case
s3curl: StringToSign='PUT\n\n\nMon, 04 Sep 2017 02:24:57 +0000\n/mpubucket/examplempu?partNumber=2&uploadId=11c945604d9d4299b52987c289103f62'
s3curl: exec curl -v -H 'Date: Mon, 04 Sep 2017 02:24:57 +0000' -H 'Authorization: AWS "S3 username":"S3 Secretkey"' -L -H 'content-type: ' -T example.01 http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=2
* Hostname was NOT found in DNS cache
*   Trying 10.32.xxx.xxx...
* Connected to 10.32.xxx.xxx (10.32.xxx.xxx) port 9020 (#0)
> PUT /mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=2 HTTP/1.1
> User-Agent: curl/7.37.0
> Host: 10.32.xxx.xxx:9020
> Accept: */*
> Date: Mon, 04 Sep 2017 02:24:57 +0000
> Authorization: AWS "S3 username":"S3 Secretkey"
> Content-Length: 262144000
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Mon, 04 Sep 2017 02:24:57 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a20a97c:15e46a039b0:b02:1
< x-amz-id-2: 428b1d2b3b46e11630cc6afb9812c6047e527d1b6b51f3be014e8b6b71482163
< ETag: "c59d28e23ac4f4b488dbf85199a73fd4"
< Content-Length: 0
<
* Connection #0 to host 10.32.xxx.xxx left intact


#create mpu.xml file, which has the ETag value from above two parts file
 

ecssh124:/usr/share/s3curl # cat mpu.xml
<CompleteMultipartUpload>
  <Part>
    <PartNumber>1</PartNumber>
    <ETag>"d6ccfd257b8eaa2a7208bbdf2277c821"</ETag>
  </Part>
  <Part>
    <PartNumber>2</PartNumber>
    <ETag>"c59d28e23ac4f4b488dbf85199a73fd4"</ETag>
  </Part>
</CompleteMultipartUpload>

#Complete a multi-part upload

 

ecssh124:/usr/share/s3curl # ./s3curl.pl --debug --post mpu.xml  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62" |xmllint --format -
s3curl: Found the url: host=10.32.xxx.xxx; port=9020; uri=/mpubucket/examplempu; query=uploadId=11c945604d9d4299b52987c289103f62;
s3curl: replaced string: uploadId=11c945604d9d4299b52987c289103f62
s3curl: ordinary endpoint signing case
s3curl: StringToSign='POST\n\n\nMon, 04 Sep 2017 02:32:32 +0000\n/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62'
s3curl: exec curl -v -H 'Date: Mon, 04 Sep 2017 02:32:32 +0000' -H 'Authorization: AWS "S3 username":"S3 SecretKey"' -L -H 'content-type: ' -X POST -T mpu.xml http://10.32.169.124:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62
* Hostname was NOT found in DNS cache
*   Trying 10.32.xxx.xxx...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 10.32.xxx.xxx (10.32.xxx.xxx) port 9020 (#0)
> POST /mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62 HTTP/1.1
> User-Agent: curl/7.37.0
> Host: 10.32.xxx.xxx:9020
> Accept: */*
> Date: Mon, 04 Sep 2017 02:32:32 +0000
> Authorization: AWS "S3 username":"S3 SecretKey"
> Content-Length: 257
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
} [data not shown]
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Mon, 04 Sep 2017 02:32:32 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a20a97c:15e46a039b0:b9e:3
< x-amz-id-2: 428b1d2b3b46e11630cc6afb9812c6047e527d1b6b51f3be014e8b6b71482163
< ETag: "b7a3f5201be2a6a4b130018f79d232b3-2"
< Content-Type: application/xml
< Content-Length: 329
<
{ [data not shown]
100   586  100   329  100   257   1257    982 --:--:-- --:--:-- --:--:--  1260
* Connection #0 to host 10.32.xxx.xxx left intact
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Location>http://10.32.xxx.xxx:9020/mpubucket/examplempu</Location>
  <Bucket>mpubucket</Bucket>
  <Key>examplempu</Key>
  <ETag>"b7a3f5201be2a6a4b130018f79d232b3-2"</ETag>
</CompleteMultipartUploadResult>


#list this file
 

ecssh124:/usr/share/s3curl # ./s3curl.pl  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/" |xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   583  100   583    0     0   5152      0 --:--:-- --:--:-- --:--:--  5205
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Name>mpubucket</Name>
  <Prefix/>
  <Marker/>
  <MaxKeys>1000</MaxKeys>
  <IsTruncated>false</IsTruncated>
  <ServerSideEncryptionEnabled>false</ServerSideEncryptionEnabled>
  <Contents>
    <Key>examplempu</Key>
    <LastModified>2017-09-04T02:32:32Z</LastModified>
    <ETag>"b7a3f5201be2a6a4b130018f79d232b3-2"</ETag>
    <Size>524288000</Size>
    <StorageClass>STANDARD</StorageClass>
    <Owner>
      <ID>walkerobj</ID>
      <DisplayName>walkerobj</DisplayName>
    </Owner>
  </Contents>
</ListBucketResult>


#Download this file, and we can know it is same as original file before the split.
 

ecssh124:/usr/share/s3curl # ./s3curl.pl  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu" >exampledownload
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  500M  100  500M    0     0   316M      0  0:00:01  0:00:01 --:--:--  317M
ecssh124:/usr/share/s3curl # md5sum example exampledownload
9c561eafc918d7bfae2a1cac376836c1  example
9c561eafc918d7bfae2a1cac376836c1  exampledownload

 

Productos afectados

ECS Appliance

Productos

ECS Appliance, ECS Appliance Hardware Gen1 U-Series, ECS Appliance Software with Encryption, ECS Appliance Software without Encryption, Elastic Cloud Storage
Propiedades del artículo
Número del artículo: 000014693
Tipo de artículo: How To
Última modificación: 09 sept. 2025
Versión:  5
Encuentre respuestas a sus preguntas de otros usuarios de Dell
Servicios de soporte
Compruebe si el dispositivo está cubierto por los servicios de soporte.