ECS: Cómo usar s3curl para realizar una carga de varias partes

Resumen: Cómo utilizar s3curl para realizar la carga de varias partes (MPU).

Este artículo se aplica a Este artículo no se aplica a Este artículo no está vinculado a ningún producto específico. No se identifican todas las versiones del producto en este artículo.
Consulte estos recursos

Instrucciones

En este artículo de la base de conocimientos, se demuestra cómo utilizar S3curl para realizar una operación similar a fin de ayudar a solucionar problemas relacionados con MPU.


#Create un archivo de prueba de 500 millones 
  
dd if=/dev/urandom of=example bs=500M count=1

#check this file's md5sum value. 
Ejemplo
de MD5Sum 9C561EAFC918D7BFAE2A1CAC376836C1 Ejemplo

#split archivo en dos partes, example.00 y example.01 
split -n 2 example -d example.

ls -l example*
-rw-r--r-- 1 root root 524288000 Sep  4 02:13 example
-rw-r--r-- 1 root root 262144000 Sep  4 02:14 example.00
-rw-r--r-- 1 root root 262144000 Sep  4 02:14 example.01


# Init Uploads, get the uploadID. mpubucket is the bucket name, examplempu is the object name for our example. 

 

ecssh124:/usr/share/s3curl # ./s3curl.pl  --id walker --post -- "http://10.xxx.xxx.xxx:9020/mpubucket/examplempu?uploads"  |xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   266  100   266    0     0    647      0 --:--:-- --:--:-- --:--:--   648
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Bucket>mpubucket</Bucket>
  <Key>examplempu</Key>
  <UploadId>11c945604d9d4299b52987c289103f62</UploadId>
</InitiateMultipartUploadResult>

#Use Upload 1st part of the file, example.00 , and record the ETag value,

 

ecssh124:/usr/share/s3curl # ./s3curl.pl --debug --put=example.00  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=1"
s3curl: Found the url: host=10.xxx.xxx.xxx; port=9020; uri=/mpubucket/examplempu; query=uploadId=11c945604d9d4299b52987c289103f62&partNumber=1;
s3curl: replaced string: partNumber=1
s3curl: replaced string: uploadId=11c945604d9d4299b52987c289103f62
s3curl: ordinary endpoint signing case
s3curl: StringToSign='PUT\n\n\nMon, 04 Sep 2017 02:23:50 +0000\n/mpubucket/examplempu?partNumber=1&uploadId=11c945604d9d4299b52987c289103f62'
s3curl: exec curl -v -H 'Date: Mon, 04 Sep 2017 02:23:50 +0000' -H 'Authorization: AWS "S3 username":"S3 Secretkey"' -L -H 'content-type: ' -T example.00 http://10.xxx.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=1
* Hostname was NOT found in DNS cache
*   Trying 10.32.169.124...
* Connected to 10.xxx.xxx.xxx (10.xxx.xxx.xxx) port 9020 (#0)
> PUT /mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=1 HTTP/1.1
> User-Agent: curl/7.37.0
> Host: 10.xxx.xxx.xxx:9020
> Accept: */*
> Date: Mon, 04 Sep 2017 02:23:50 +0000
> Authorization: AWS "S3 username":"S3 Secretkey"
> Content-Length: 262144000
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Mon, 04 Sep 2017 02:23:50 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a20a97c:15e46a039b0:a79:3c0
< x-amz-id-2: 428b1d2b3b46e11630cc6afb9812c6047e527d1b6b51f3be014e8b6b71482163
< ETag: "d6ccfd257b8eaa2a7208bbdf2277c821"
< Content-Length: 0
<
* Connection #0 to host 10.32.xxx.xxx left intact


##Use Upload 2nd part of the file,example.01, and record the ETag value,
 

ecssh124:/usr/share/s3curl # ./s3curl.pl --debug --put=example.01  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=2"
s3curl: Found the url: host=10.32.xxx.xxx; port=9020; uri=/mpubucket/examplempu; query=uploadId=11c945604d9d4299b52987c289103f62&partNumber=2;
s3curl: replaced string: partNumber=2
s3curl: replaced string: uploadId=11c945604d9d4299b52987c289103f62
s3curl: ordinary endpoint signing case
s3curl: StringToSign='PUT\n\n\nMon, 04 Sep 2017 02:24:57 +0000\n/mpubucket/examplempu?partNumber=2&uploadId=11c945604d9d4299b52987c289103f62'
s3curl: exec curl -v -H 'Date: Mon, 04 Sep 2017 02:24:57 +0000' -H 'Authorization: AWS "S3 username":"S3 Secretkey"' -L -H 'content-type: ' -T example.01 http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=2
* Hostname was NOT found in DNS cache
*   Trying 10.32.xxx.xxx...
* Connected to 10.32.xxx.xxx (10.32.xxx.xxx) port 9020 (#0)
> PUT /mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=2 HTTP/1.1
> User-Agent: curl/7.37.0
> Host: 10.32.xxx.xxx:9020
> Accept: */*
> Date: Mon, 04 Sep 2017 02:24:57 +0000
> Authorization: AWS "S3 username":"S3 Secretkey"
> Content-Length: 262144000
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Mon, 04 Sep 2017 02:24:57 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a20a97c:15e46a039b0:b02:1
< x-amz-id-2: 428b1d2b3b46e11630cc6afb9812c6047e527d1b6b51f3be014e8b6b71482163
< ETag: "c59d28e23ac4f4b488dbf85199a73fd4"
< Content-Length: 0
<
* Connection #0 to host 10.32.xxx.xxx left intact


#create mpu.xml file, which has the ETag value from above two parts file
 

ecssh124:/usr/share/s3curl # cat mpu.xml
<CompleteMultipartUpload>
  <Part>
    <PartNumber>1</PartNumber>
    <ETag>"d6ccfd257b8eaa2a7208bbdf2277c821"</ETag>
  </Part>
  <Part>
    <PartNumber>2</PartNumber>
    <ETag>"c59d28e23ac4f4b488dbf85199a73fd4"</ETag>
  </Part>
</CompleteMultipartUpload>

#Complete a multi-part upload

 

ecssh124:/usr/share/s3curl # ./s3curl.pl --debug --post mpu.xml  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62" |xmllint --format -
s3curl: Found the url: host=10.32.xxx.xxx; port=9020; uri=/mpubucket/examplempu; query=uploadId=11c945604d9d4299b52987c289103f62;
s3curl: replaced string: uploadId=11c945604d9d4299b52987c289103f62
s3curl: ordinary endpoint signing case
s3curl: StringToSign='POST\n\n\nMon, 04 Sep 2017 02:32:32 +0000\n/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62'
s3curl: exec curl -v -H 'Date: Mon, 04 Sep 2017 02:32:32 +0000' -H 'Authorization: AWS "S3 username":"S3 SecretKey"' -L -H 'content-type: ' -X POST -T mpu.xml http://10.32.169.124:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62
* Hostname was NOT found in DNS cache
*   Trying 10.32.xxx.xxx...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 10.32.xxx.xxx (10.32.xxx.xxx) port 9020 (#0)
> POST /mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62 HTTP/1.1
> User-Agent: curl/7.37.0
> Host: 10.32.xxx.xxx:9020
> Accept: */*
> Date: Mon, 04 Sep 2017 02:32:32 +0000
> Authorization: AWS "S3 username":"S3 SecretKey"
> Content-Length: 257
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
} [data not shown]
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Mon, 04 Sep 2017 02:32:32 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a20a97c:15e46a039b0:b9e:3
< x-amz-id-2: 428b1d2b3b46e11630cc6afb9812c6047e527d1b6b51f3be014e8b6b71482163
< ETag: "b7a3f5201be2a6a4b130018f79d232b3-2"
< Content-Type: application/xml
< Content-Length: 329
<
{ [data not shown]
100   586  100   329  100   257   1257    982 --:--:-- --:--:-- --:--:--  1260
* Connection #0 to host 10.32.xxx.xxx left intact
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Location>http://10.32.xxx.xxx:9020/mpubucket/examplempu</Location>
  <Bucket>mpubucket</Bucket>
  <Key>examplempu</Key>
  <ETag>"b7a3f5201be2a6a4b130018f79d232b3-2"</ETag>
</CompleteMultipartUploadResult>


#list this file
 

ecssh124:/usr/share/s3curl # ./s3curl.pl  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/" |xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   583  100   583    0     0   5152      0 --:--:-- --:--:-- --:--:--  5205
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Name>mpubucket</Name>
  <Prefix/>
  <Marker/>
  <MaxKeys>1000</MaxKeys>
  <IsTruncated>false</IsTruncated>
  <ServerSideEncryptionEnabled>false</ServerSideEncryptionEnabled>
  <Contents>
    <Key>examplempu</Key>
    <LastModified>2017-09-04T02:32:32Z</LastModified>
    <ETag>"b7a3f5201be2a6a4b130018f79d232b3-2"</ETag>
    <Size>524288000</Size>
    <StorageClass>STANDARD</StorageClass>
    <Owner>
      <ID>walkerobj</ID>
      <DisplayName>walkerobj</DisplayName>
    </Owner>
  </Contents>
</ListBucketResult>


#Download this file, and we can know it is same as original file before the split.
 

ecssh124:/usr/share/s3curl # ./s3curl.pl  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu" >exampledownload
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  500M  100  500M    0     0   316M      0  0:00:01  0:00:01 --:--:--  317M
ecssh124:/usr/share/s3curl # md5sum example exampledownload
9c561eafc918d7bfae2a1cac376836c1  example
9c561eafc918d7bfae2a1cac376836c1  exampledownload

 

Productos afectados

ECS Appliance

Productos

ECS Appliance, ECS Appliance Hardware Gen1 U-Series, ECS Appliance Software with Encryption, ECS Appliance Software without Encryption, Elastic Cloud Storage
Propiedades del artículo
Número del artículo: 000014693
Tipo de artículo: How To
Última modificación: 09 sept. 2025
Versión:  5
Encuentre respuestas a sus preguntas de otros usuarios de Dell
Servicios de soporte
Compruebe si el dispositivo está cubierto por los servicios de soporte.