Highlighted
Not applicable

How to handle interrupted uploads

When attempting to createObject using an InputStream as content the stream abruptly breaks. I can see the library attempting to restart and try again, but it fails because the original stream is no longer usable.

Is there a way to instruct the library to instruct the library to stop uploading and keep whatever has been uploaded in the storage?

The alternative for me would be to successively updateObject with blocks read from the stream, but that would be quite slow, as round-trip to the storage is measured in seconds.

Labels (1)
0 Kudos
Reply
1 Reply
Highlighted
3 Argentum

Re: How to handle interrupted uploads

Lachezar,

CreateObject is a single HTTP transaction.  Should it fail, you must re-send the entire request.  If you have large objects and require a "resume" behavior, then you should buffer the data and send CreateObject followed by multiple UpdateObject (append) calls.  If an append operation fails, you can resume from the last successful append.

If you're using checksums the process is more complex. In that case, you'll have to HEAD the object to find out from what offset to resume.  Then you'll have to issue a new append request from that offset and send the remainder of the buffer.

You should also upload multiple objects in parallel to increase throughput, especially with high latency.

0 Kudos
Reply