Ask the Expert: Making the Right Choice for a Scale-Out Object Storage Cloud Platform

Welcome to our newest ATE session on ECN. We'd like to open this discussion for replies. Let's make of this an opportunity to respectfully engage our experts with your questions, comments or ideas. Have a great experience!

Hello and welcome to our Ask the Expert event about the critical considerations that go into that very important decision about whether to build or purchase an object storage cloud platform. During the course of the next two weeks, our object storage experts will engage in a lively discussion about the merits and pitfalls of open source and commodity, and commercial vendor offerings.

Let’s get started.

Experts: Starting with some basics. What is object storage and why is it the popular choice for web-scale storage?

Object storage is a method in which to logically store data; primarily unstructured data. Objects are discrete units of storage, characterized by their extended metadata. Each object is assigned a unique identifier which allows a server or end user to retrieve the object without needing to know the physical location of the data. Like files, objects contain data, but unlike files, objects are not organized in a hierarchy. Every object exists at the same level in a flat address space called a storage pool and one object cannot be placed inside another object.

Object storage systems feature a single, flat global namespace, regardless of the number of nodes or locations. This results in location-independence without requiring a file system to govern data placement. This also means it can scale linearly and efficiently to 10s and 100s of petabytes (even exabytes!). Multiple nodes and sites appear as one, logical storage system. Replication and geo-distribution of data is driven by policies rather than dedicated replication and backup infrastructure. Object storage provides geo-distribution with multiple sites in an active-active architecture. This means any site can deliver content in the event of a site failure.

Object is used widely for Web and mobile appications because of its simple archiecture and http/https based access. Access to data is via a REST based API and http/https. Examples of object storage APIs include Amazon S3, OpenStack Swift and EMC Atmos. Developers can write applications to one of the industry lead object storage APIs without having to worry about the infrastructure. Changes to the infrastructure don't require application code changes.

For more info, I recommend viewing the webcast, “How REST & Object Storage Make Next Generation Application Development Simple” to get an in-depth look at object architecture and writing apps to REST based APIs.

Commodity storage is generally considered low cost, high capacity SATA disk drives, off the shelf controllers, and standard X86 based servers.  The intelligence and management is software-defined. This allows these commodity “off the shelf” components to perform reliably. With regard to whether a commodity platform is as reliable as traditional SAN, that depends on a number of factors.

Even a traditional array is comprised of primarily commodity components. But the benefit is there is nothing to build and the software and hardware are optimized to work together. The potential drawback is that it is purpose-built for a specific set of tasks and data. Software-defined storage, as its name implies, is hardware-independent and provides theability to use standard, COTS components. This can dramaticaly lower storge costs. But as far as reliability, commodity components are built to fail and do so frequently. Getting entrprise-grade reliability requires the right scale, the right software and operational skills. 

Continuing along the same line. What is considered commodity storage and components?  Are these commodity components as reliable as a traditional SAN?

OpenStack is often a consideration these days. What are some things an IT department should be doing to make sure they’re thoroughly “kicking the tires” and getting a relevant view of what OpenStack would be like in production?

It is important to evaluate the entire lifecycle of the environment to avoid surprises. Beyond testing application access to the system during normal operating conditions, things to test would include:

• Artificially failing nodes or drives and evaluating the recovery capabilities of the system and also the ease of
  installing replacement parts
• Failing network connections between nodes, or partitioning the system
• How the system monitoring facilities identify the failing node/disk/network and the impact of the failure on system
  performance
• Disaster recovery capabilities in the case of a site outage
• How quickly and seamlessly the system capacity can be expanded
• The ease of upgrading to a new software release, especially when under load.

What are the benefits of on-premise cloud storage compared to hosted clouds?

When a company evaluates the decision whether to build an on-prem vs. using a poublic cloud service, there are a couple general questions they ask.

1. Am I even allowed to use a public cloud? The discusion may end here depending on governance, compliance or other security concerns.
2. Do I have the scale to justify building on-prem? Public cloud services are pretty cheap. However, at scale and over time, it's almost always cheaper to own than to rent in perpetuity.
3. What level of control do I want over the infrastructure and, more importantly, my data? This is usually the biggest consideration. With a significant amount of data, organizations want to maintain control of it. It's a critical business asset; their intellectual property.
4. What is the cost comparison? It's more than the $/GB. Bandwidt charges are often the highest cost in the monthly bill. What are you paying for all the puts and gets? It'ds important to look at the total cost of owndership over time than to go with lowest $/GB.

For smaller organizatons that don't hve the skills or resources to build and operate their own infrastructure, public cloud is a great way to get the technology capabilities they need and get to market faster. For larger businesses, they have more of a decision to make. The good news is that software-defined storage solutions and commodity components have made public cloud economics more accessible to more companies. The next decision is whether to build an evironment from the ground up or use an intregrated vendor platform. 

Why would someone want to consider a turnkey object storage solution as opposed to a home grown solution?

There are a number of reasons why you may want to opt for a turnkey solution instead of piecing together your own, the main reason being that a turnkey solution is vendor supported, which means that the vendor has certified the commodity hardware as a reference architecture. This means that you will still get the cost benefit of the commodity components, but you will also receive the high level of support like you would if you were buying a SAN. Other benefits include a much quicker deployment because you don’t have to go through the trial and error of architectural design as it has already been done for you. As a result, more IT time can be allocated for building applications instead of designing the infrastructure.

Get an inside look at the commodity components within an ECS Appliance rack.

Many IT organizations like to build what they can themselves. What are some of the biggest challenges to designing a home-grown object-storage platform?

Check out the demo to uncover the simple process of creating object buckets within an ECS Appliance.