Ask The Expert: All about ViPR 2.0 Geo-distribution and Protection

This discussion is now open for questions. e look forward to a lively and interactive discussion on this topic. Don't be shy, jump in with you thoughts and questions.

@Kamal,

Are you the expert around ViPR and unstructured file data?

To put it simply, my customer is a very happy with their VPLEX functionality for all block workload. They are looking for us to provide a similar solution for their file data (mainly file shares on CIFS).

@josejperez  We don't have File head access to the unstructured data in ViPR 2.0.  It is in our roadmap, but I don't have timelines to share at this point.  If you can connect offline, I would be happy to learn more about your customer requirements on this. 

Looking forward to this Ask the Experts event! I'll be following the discussion this week from the Gigaom Structure 2014 conference in San Francisco. This will be my first time at Gigaom Structure and I'm thrilled to be leading a workshop on geo-scale cloud storage and global analytics capabilities. Of course, I'll be talking a lot about the new EMC ECS appliance, powered by ViPR, that brings hyperscale capabilities and economics to everyone. If you're at the event, stop by the booth or attend my workshop tomorrow, in Conference Room 4 titled, "Any Workload, Any Device, Any Cloud".

And, if you want to know a little more about the ECS Appliance and ViPR 2.0 check out Chris Ratcliffe's fireside chat with IDC's Vernon Turner:

Just returned from GigaOm Structure conference. The theme was "The Cloud is the Computer", a shout-out to Sun Microsystem's "The Network is the Computer" from the late 90s.   I ran a workshop/presentation titled, "Any device, any app, any cloud" and talked about the new geo-capabilities of ViPR and the ECS appliance, powered by ViPR. We had a great crowd and they asked some good questions. One person asked about storage overhead and what the numbers really are. It was a nice segue to my next slide:

The ECS appliance, powered by ViPR accomplishes this with a hybrid encoding model. Traditional approaches to data protection and replication make extra copies of the data which increases overhead as you add more sites. With ViPR the storage efficiency scales inversely: the more data centers, the more efficient the protection!

There are two primary types of data protection in the industry: geo-mirroring or geo-parity using distributed erasure coding. Customers have been forced to make a choice. Do i want to optimize for data access or storage efficiency. Geo-mirroring means I replicate full copies of a file within and among sites. For example, a 1MB file replicated across four data center with a data protection policy of "2 Local, 3 remote GeoMirror replicas" would take up 5MB of disk space - a storage overhead of 5x. The benefit is that you maintain rapid access to data but it's a very inefficient way to do it. With erasure coding, I distribute fragments of a file with parity across multiple data centers. My policy would simply be, "one distributed geo-parity replica". Using 10 of 16 distributed erasure coding (10 data bits, 6 parity bits) across those same four data centers, I consume 1.66MB of disk space for that same 1MB file. Much more efficient but I have to reconstruct the data over the WAN if I have a disk, node or data center failure. The ViPR approach combines these methods to maintain local access to content while reducing overhead. Over those same four data centers, ViPR would have <1.8x storage overhead but would not require WAN traffic to reconstruct a file in the event of a disk, node, rack or data center failure.

One of the speakers at GigaOm said, "Private cloud efforts will fail if they cannot offer public cloud economics." I'd argue that that is not entirely true - it's more then just pure cost. However, EMC is bringing public cloud economics to both enterprise private clouds or to service providers that want to compete with public IaaS providers. The remarkable storage efficiency of the ECS appliance is one of the many features that enables EMC to change the economics of cloud storage.

What do you think? What else would you like to know about ViPR Services, the ECS appliance, commodity storage and the unique geo-scale capabilities of software-defined storage?

I had another question come up recently. Someone asked what benefit ViPR Services and the ECS appliance bring to developers. Secondly, from the developer's perspective, how is the ECS appliance different/better than EMC Atmos? Great questions. As the product marketing manager for Atmos, I've been part of a team that has been communicating the benefits of object storage to developers for writing Web, mobile and cloud applications for a couple years. We have code examples that show upwards of a 10x reduction in code! To see actual examples, you can view the webcast series that EMC recently produced titled, How REST & Object Storage Make Next Generation Application Development Simple, featuring EMC's own Jason Cwik.

Atmos is a strategic platform for EMC and has a host of features and capabilities that make an app developer's life easier. Atmos supports the Atmos REST API as well as Amazon S3. The object semantics of Atmos make developing Web, mobile and cloud apps easier and a lot less risky. One of the key differences between Atmos and ECS is that, in addition to Object, ECS supports HDFS and Block protocols (and will add File in the future). In addition, Atmos supports either Geo-mirror or geo-parity (distributed erasure coding)  ECS uses a hybrid approach described in my previous post.

So, which one is better? The answer as always is, "It depends". It depends on application architecture and a host of other considerations. But some of the new developer-centric features that ViPR Services 2.0 and the ECS appliance bring include:

• Buckets of data span sites and feature strong consistency. The ECS appliance provides the ability to read from and write to any site. In contrast, Atmos is eventually consistent (as are most object storage platforms). In the  eventually consistent model, the developer is responsible for contending with the possibility of a "stale read". There is always a trade-off between performance and consistency - ECS optimizes for both - it features strong consistency without imposing a significant performance penalty.  
• ECS supports Block and HDFS access, and ECS supports OpenStack Swift in addition to Amazon S3 and Atmos object APIs.
• ViPR Services stores objects in append-only containers and features the ability to atomically append an object. This enables simultaneous updates to an object while maintaining consistency and returning the latest version.

So, again, it depends. The good news for Atmos customers is that they are already on the on-ramp to the ECS appliance. EMC has a host of new capabilities, features, and updates planned for Atmos. Moving forward, ViPR will add support for Atmos arrays as ViPR managed arrays. Looking ahead, customers will be able to expand their object pool to additional resources and get access to the new features in ViPR (such as HDFS and file access). It's an exciting time to be working with Atmos, ViPR and the ECS appliance. Our focus on helping developers create 3rd Platform applications for Web, mobile and Big Data use cases has never been greater. Atmos has been a tremendous success in that regard and now EMC is making a leap forward with ViPR 2.0 and the ECS Appliance.

does Atmos CDP support ViPR? or when will a similar cloud delivery platform be provided?

or is similar "as a service" abilities already built into ViPR?  I only saw ACL's which is not the same

Thank you for the question. ViPR Controller includes its own self-service portal with cloud storage-as-a-service capabilities such as self-service access, multi-tenancy, metering, chargeback, etc. Customers can use the ViPR user portal or can integrate ViPR with an existing cloud stack from VMware, OpenStack or Microsoft Hyper-V. Atmos Cloud Delivery Platform is an add-on to Atmos to offer cloud storage-as-a-service for Atmos only environments.