Ask the Expert: Making the Right Choice for a Scale-Out Object Storage Cloud Platform

Our Experts would like to know what you think. Please participate in this  poll to provide your valued input. Let us know what is the right choice for you.

Join the discussion here too

How does ECS protect data within the system and how does it compare to other vendors or an OpenStack distribution?

ECS uses an erasure coding scheme to provide storage efficiency without compromising data protection or access. ECS first writes data triple mirrored for high performance, then later splits the data into 12 data fragments and 4 coding fragments, with the resulting 16 fragments dispersed across nodes at the local site. The storage engine can reconstruct a chunk from a minimum of 12 fragments. In this way, ECS can tolerate node failures and still deliver service while also marrying the high ingest performance of mirrored writes with the low storage overhead of erasure coding.

ECS provides strong consistency for data, even when accessed from multiple geographies. In a strongly consistent system, the application is always guaranteed to read the most recent version of the data, and ECS has been optimized to quickly determine where the data resides and return it from the location closest to the application. Most other cloud systems implement eventual consistency schemes, where updates to data propagate lazily and applications may read prior versions of the data for an unspecified period of time, requiring additional and complex application logic to ensure correct behavior.

OK. What about open source options on commodity hardware?  How do these solutions compare to a vendor supported, turnkey solution? Experts? Anyone?

This all depends on the solutions that you are comparing. The great thing is that there are many choices for building your own cloud storage solution which can be run on low-cost commodity hardware. These solutions may have a lower initial cost of acquisition but you should also factor in how long it will take to implement, and how much of your own resources are you going to have to tie up with training and development. Turnkey vendor supported solutions like EMC ECS Appliance are completely vetted from an architectural standpoint and are designed to be quick to deploy. The goal is to get your web infrastructure up and running as quickly as possible to start delivering value to the business.

Regarding a turnkey vendor approach, how are security and authentication handled through a solution like EMC ECS Appliance? How are they handled with Open Source on commodity?

EMC ECS Appliance is designed to be multi-tenant, and can authenticate users via a number of secure methods including LDAP and Microsoft Active Directory.

For Open Source, OpenStack includes a keystone project, which provides an overall authentication and authorization service, and which also can support a number of different backends.

Many IT organizations like the efficiency and REST access of object storage but need to ingest and store lots of small files. Can an object platform do this? Usually object is only good for larger files.

Check out the ECS Appliance Whiteboardto see how applications deployed to infrastructure efficiently utilize all of the storage that’s allocated to them.

ECS Software defined layer is adept at handling both a high volume of small and large files. For example, files that are 1KB to 512KB are handled with a procedure called “box-carting”. When an application writes many small files with high I/O, ECS Software aggregates multiple requests in memory and writes them as one operation, which improves performance by eliminating multiple round trips to the underlying storage and instead writing the aggregated data as a single “box-cart”. For large files, ECS will split the large write into multiple, independent pieces which can be processed in parallel across multiple ECS nodes, thereby taking advantage of all spindles and NICs in a cluster, which enables applications to use the full bandwidth of every node in the system to achieve maximum throughput.

Let's start the week with a basic question about a vendor-supported turnkey solution. Suspect the nature of the APIs might be on peoples' minds (e.g. proprietary, open).

Experts: Does EMC ECS Appliance support the popular APIs like S3?

Thank you, George.

Multi-site access can be difficult to implement in a home grown object storage solution. Experts: How do vendors like EMC overcome this hurdle and implement multi-site access?

A vendor-delivered object-storage platform like EMC ECS Appliance provides strong, consistent views of data regardless of where the data resides. With geo-protection enabled, applications can access data immediately through any ECS Appliance site, regardless of where the last data was written. For example, an application that writes object 1 to site A can access data immediately from site B. Additionally, any update to object 1 in site B can be read instantly from site A. ECS Appliance ensures that the data returned for object 1 is always the latest version, whether you access it from site A or B.

OK. Let's talk about Analytics and Big Data. How does a vendor-supported object storage platform make it easier to implement analytics?