Writing objects directly to the ECS store

[Alex_Pyrgiotis]

Hi all,

In the ScaleIO subforum, I've been pointed to see ECS for object

storage. From what I read, it is API-compatible with S3, Swift, Atmos

etc. Is there any tool, however, that can be used to read/write objects

directly to the ECS store, without the HTTP overhead, like Ceph does

with librados?

Thanks,

Alex

[JasonCwik · Accepted solution]

Hi Alex,

ECS does not currently export any access to its internal protocols.  Can you expand upon your use case for direct access?  What are you trying to achieve?

[Alex_Pyrgiotis]

Hi Jason,

You're right, I should have provided more context for our use case. You

can take a look at the thread in ScaleIO that was the basis for my

question.

You'll see that in both of my questions, I'm searching for a librados

equivalent tool in ScaleIO or ECS. In case you don't know what librados

does, it's a library that implements all the low-level operations on a

RADOS cluster. It also provides an API that is the building block for

most of Ceph's interfaces: Ceph Object Gateway and RBD.

Now, why would a third party need such a tool? The reason is that such a

client tool enables one to write a custom interface (business logic) over the

underlying object store. That's what we need and currently do with Ceph.

So, I'll reiterate my question. Given that ECS has two interfaces and

one on the making (Object, HDFS and NFS), is there any client tool that

provides the basis for the above and, most importantly, performs object

requests directly to the ECS store?

[JasonCwik]

Yes, there is an underlying protocol inside ECS, but it is currently not documented and supported for customer use.  I'll forward your request on to product management; this request has come up once before.

Are you simply looking to reduce your latency vs. HTTP-based protocols?  What kinds of latency requirements are you looking at?  What I/O patterns does your application perform?

[Alex_Pyrgiotis]

Jason Cwik wrote:

Yes, there is an underlying protocol inside ECS, but it is currently not documented and supported for customer use.  I'll forward your request on to product management; this request has come up once before.

Yes, this is probably what we're looking for. We'd be very interested to know

more.

Are you simply looking to reduce your latency vs. HTTP-based protocols?  What kinds of latency requirements are you looking at?  What I/O patterns does your application perform?

The benefits of using a librados-equivalent tool are two-fold:

a) First, we reduce the I/O latency with the removal of the HTTP

protocol but, most importantly, we can do I/O asynchronously with callbacks.

b) Second, we can use features of the object store that are not exposed

by the Object interface, such as locking, snapshotting, etc.

As for the latency requirements, we're hoping for at most 500us added latency

to the native speeds of our storage devices. This should roughly cover the cost

of the network latency and internal ECS replication. Also, our hard I/O pattern

consists of an equal amount of reads/writes to multiple new and old objects at

the same time.