ECS: Hoe s3curl te gebruiken? Uploaden in meerdere delen uitvoeren

Samenvatting: Hoe s3curl te gebruiken? Multi-part upload (MPU) uitvoeren.

Dit artikel is van toepassing op Dit artikel is niet van toepassing op Dit artikel is niet gebonden aan een specifiek product. Niet alle productversies worden in dit artikel vermeld.
Bekijk andere hulpbronnen

Instructies

Dit KB-toetsenbord laat zien hoe u S3curl kunt gebruiken en een vergelijkbare bewerking kunt uitvoeren om problemen met het oplossen van problemen rond MPU te ondersteunen.


#Create een testbestand van 500M 
  
dd if=/dev/urandom of=example bs=500M count=1

#check this file's md5sum value. 
MD5SUM voorbeeld
9c561eafc918d7bfae2a1cac376836c1 voorbeeld

#split bestand in twee delen, example.00 en example.01 
split -n 2 example -d example.

ls -l example*
-rw-r--r-- 1 root root 524288000 Sep  4 02:13 example
-rw-r--r-- 1 root root 262144000 Sep  4 02:14 example.00
-rw-r--r-- 1 root root 262144000 Sep  4 02:14 example.01


# Init Uploads, get the uploadID. mpubucket is the bucket name, examplempu is the object name for our example. 

 

ecssh124:/usr/share/s3curl # ./s3curl.pl  --id walker --post -- "http://10.xxx.xxx.xxx:9020/mpubucket/examplempu?uploads"  |xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   266  100   266    0     0    647      0 --:--:-- --:--:-- --:--:--   648
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Bucket>mpubucket</Bucket>
  <Key>examplempu</Key>
  <UploadId>11c945604d9d4299b52987c289103f62</UploadId>
</InitiateMultipartUploadResult>

#Use Upload 1st part of the file, example.00 , and record the ETag value,

 

ecssh124:/usr/share/s3curl # ./s3curl.pl --debug --put=example.00  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=1"
s3curl: Found the url: host=10.xxx.xxx.xxx; port=9020; uri=/mpubucket/examplempu; query=uploadId=11c945604d9d4299b52987c289103f62&partNumber=1;
s3curl: replaced string: partNumber=1
s3curl: replaced string: uploadId=11c945604d9d4299b52987c289103f62
s3curl: ordinary endpoint signing case
s3curl: StringToSign='PUT\n\n\nMon, 04 Sep 2017 02:23:50 +0000\n/mpubucket/examplempu?partNumber=1&uploadId=11c945604d9d4299b52987c289103f62'
s3curl: exec curl -v -H 'Date: Mon, 04 Sep 2017 02:23:50 +0000' -H 'Authorization: AWS "S3 username":"S3 Secretkey"' -L -H 'content-type: ' -T example.00 http://10.xxx.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=1
* Hostname was NOT found in DNS cache
*   Trying 10.32.169.124...
* Connected to 10.xxx.xxx.xxx (10.xxx.xxx.xxx) port 9020 (#0)
> PUT /mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=1 HTTP/1.1
> User-Agent: curl/7.37.0
> Host: 10.xxx.xxx.xxx:9020
> Accept: */*
> Date: Mon, 04 Sep 2017 02:23:50 +0000
> Authorization: AWS "S3 username":"S3 Secretkey"
> Content-Length: 262144000
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Mon, 04 Sep 2017 02:23:50 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a20a97c:15e46a039b0:a79:3c0
< x-amz-id-2: 428b1d2b3b46e11630cc6afb9812c6047e527d1b6b51f3be014e8b6b71482163
< ETag: "d6ccfd257b8eaa2a7208bbdf2277c821"
< Content-Length: 0
<
* Connection #0 to host 10.32.xxx.xxx left intact


##Use Upload 2nd part of the file,example.01, and record the ETag value,
 

ecssh124:/usr/share/s3curl # ./s3curl.pl --debug --put=example.01  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=2"
s3curl: Found the url: host=10.32.xxx.xxx; port=9020; uri=/mpubucket/examplempu; query=uploadId=11c945604d9d4299b52987c289103f62&partNumber=2;
s3curl: replaced string: partNumber=2
s3curl: replaced string: uploadId=11c945604d9d4299b52987c289103f62
s3curl: ordinary endpoint signing case
s3curl: StringToSign='PUT\n\n\nMon, 04 Sep 2017 02:24:57 +0000\n/mpubucket/examplempu?partNumber=2&uploadId=11c945604d9d4299b52987c289103f62'
s3curl: exec curl -v -H 'Date: Mon, 04 Sep 2017 02:24:57 +0000' -H 'Authorization: AWS "S3 username":"S3 Secretkey"' -L -H 'content-type: ' -T example.01 http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=2
* Hostname was NOT found in DNS cache
*   Trying 10.32.xxx.xxx...
* Connected to 10.32.xxx.xxx (10.32.xxx.xxx) port 9020 (#0)
> PUT /mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62&partNumber=2 HTTP/1.1
> User-Agent: curl/7.37.0
> Host: 10.32.xxx.xxx:9020
> Accept: */*
> Date: Mon, 04 Sep 2017 02:24:57 +0000
> Authorization: AWS "S3 username":"S3 Secretkey"
> Content-Length: 262144000
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Mon, 04 Sep 2017 02:24:57 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a20a97c:15e46a039b0:b02:1
< x-amz-id-2: 428b1d2b3b46e11630cc6afb9812c6047e527d1b6b51f3be014e8b6b71482163
< ETag: "c59d28e23ac4f4b488dbf85199a73fd4"
< Content-Length: 0
<
* Connection #0 to host 10.32.xxx.xxx left intact


#create mpu.xml file, which has the ETag value from above two parts file
 

ecssh124:/usr/share/s3curl # cat mpu.xml
<CompleteMultipartUpload>
  <Part>
    <PartNumber>1</PartNumber>
    <ETag>"d6ccfd257b8eaa2a7208bbdf2277c821"</ETag>
  </Part>
  <Part>
    <PartNumber>2</PartNumber>
    <ETag>"c59d28e23ac4f4b488dbf85199a73fd4"</ETag>
  </Part>
</CompleteMultipartUpload>

#Complete a multi-part upload

 

ecssh124:/usr/share/s3curl # ./s3curl.pl --debug --post mpu.xml  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62" |xmllint --format -
s3curl: Found the url: host=10.32.xxx.xxx; port=9020; uri=/mpubucket/examplempu; query=uploadId=11c945604d9d4299b52987c289103f62;
s3curl: replaced string: uploadId=11c945604d9d4299b52987c289103f62
s3curl: ordinary endpoint signing case
s3curl: StringToSign='POST\n\n\nMon, 04 Sep 2017 02:32:32 +0000\n/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62'
s3curl: exec curl -v -H 'Date: Mon, 04 Sep 2017 02:32:32 +0000' -H 'Authorization: AWS "S3 username":"S3 SecretKey"' -L -H 'content-type: ' -X POST -T mpu.xml http://10.32.169.124:9020/mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62
* Hostname was NOT found in DNS cache
*   Trying 10.32.xxx.xxx...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 10.32.xxx.xxx (10.32.xxx.xxx) port 9020 (#0)
> POST /mpubucket/examplempu?uploadId=11c945604d9d4299b52987c289103f62 HTTP/1.1
> User-Agent: curl/7.37.0
> Host: 10.32.xxx.xxx:9020
> Accept: */*
> Date: Mon, 04 Sep 2017 02:32:32 +0000
> Authorization: AWS "S3 username":"S3 SecretKey"
> Content-Length: 257
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
} [data not shown]
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Mon, 04 Sep 2017 02:32:32 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a20a97c:15e46a039b0:b9e:3
< x-amz-id-2: 428b1d2b3b46e11630cc6afb9812c6047e527d1b6b51f3be014e8b6b71482163
< ETag: "b7a3f5201be2a6a4b130018f79d232b3-2"
< Content-Type: application/xml
< Content-Length: 329
<
{ [data not shown]
100   586  100   329  100   257   1257    982 --:--:-- --:--:-- --:--:--  1260
* Connection #0 to host 10.32.xxx.xxx left intact
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Location>http://10.32.xxx.xxx:9020/mpubucket/examplempu</Location>
  <Bucket>mpubucket</Bucket>
  <Key>examplempu</Key>
  <ETag>"b7a3f5201be2a6a4b130018f79d232b3-2"</ETag>
</CompleteMultipartUploadResult>


#list this file
 

ecssh124:/usr/share/s3curl # ./s3curl.pl  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/" |xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   583  100   583    0     0   5152      0 --:--:-- --:--:-- --:--:--  5205
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Name>mpubucket</Name>
  <Prefix/>
  <Marker/>
  <MaxKeys>1000</MaxKeys>
  <IsTruncated>false</IsTruncated>
  <ServerSideEncryptionEnabled>false</ServerSideEncryptionEnabled>
  <Contents>
    <Key>examplempu</Key>
    <LastModified>2017-09-04T02:32:32Z</LastModified>
    <ETag>"b7a3f5201be2a6a4b130018f79d232b3-2"</ETag>
    <Size>524288000</Size>
    <StorageClass>STANDARD</StorageClass>
    <Owner>
      <ID>walkerobj</ID>
      <DisplayName>walkerobj</DisplayName>
    </Owner>
  </Contents>
</ListBucketResult>


#Download this file, and we can know it is same as original file before the split.
 

ecssh124:/usr/share/s3curl # ./s3curl.pl  --id walker  -- "http://10.32.xxx.xxx:9020/mpubucket/examplempu" >exampledownload
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  500M  100  500M    0     0   316M      0  0:00:01  0:00:01 --:--:--  317M
ecssh124:/usr/share/s3curl # md5sum example exampledownload
9c561eafc918d7bfae2a1cac376836c1  example
9c561eafc918d7bfae2a1cac376836c1  exampledownload

 

Getroffen producten

ECS Appliance

Producten

ECS Appliance, ECS Appliance Hardware Gen1 U-Series, ECS Appliance Software with Encryption, ECS Appliance Software without Encryption, Elastic Cloud Storage
Artikeleigenschappen
Artikelnummer: 000014693
Artikeltype: How To
Laatst aangepast: 09 sep. 2025
Versie:  5
Vind antwoorden op uw vragen via andere Dell gebruikers
Support Services
Controleer of uw apparaat wordt gedekt door Support Services.