While today’s business organizations have gained tremendous IT agility and efficiency with technology that automates the management of their computer and network resources, handling growing data storage demands across multiple environments has remained a time-consuming challenge—until now. Software defined storage opens the door for enterprises to achieve automated and on-demand management of their data storage resources to provide the final piece of IT optimization in the cloud.
I am part of a team at EMC IT that is currently incorporating a groundbreaking software defined storage platform into our IT-as-a-Service (ITaaS) model.
Amid the ongoing explosion of data creation and the demand to access and analyze such information quickly, enterprises have been struggling to manage their multi-vendor storage environments. Software defined storage turns physical storage arrays into pools of virtual shared storage resources so that users don’t have to care which platform their data is on. After all, business organizations are interested in data modeling and getting the data storage services they need, and not whether that information is in box A, box B or box C.
The problem has been that provisioning new storage for user applications had remained a manual and time-consuming process, managed one storage platform or array at a time. Say an IT customer wants storage for his or her application and it will involve data in various formats—block, file and object files. Traditionally, each type of data would have to be managed with separate administrative services, using different tools.
The organization’s IT operation would need to determine the customer’s needs, map his requirements to a storage array, validate the attributes the application warrants, and the provision the storage. They would also need to make sure somebody else is not provisioning the storage at the same time, that the require capacity was available, and that the provisioning would take place as a certain time.
As EMC IT has transformed from a traditional IT operation to ITaaS, we had become very aware of the gap in technology to automate storage resources and provide a seamless path to the cloud.
So when EMC software engineers working on a software defined framework, now called ViPR – allowing for the ability to transform multi-vendor storage environments into a single, open platform – asked us to test their technology, we were eager to take on the job. Like everyone else in the IT industry, we were managing discrete elements of storage and looking for a way to manage data as a whole as well as standardize self-provisioning for users. We wanted to create a shopping cart experience for storage as we had done for compute provisioning or getting an email account. Automating storage remained a missing link in our service catalogue.
ViPR enables the end user to tap into such automation while maintaining strict administrative controls and monitoring through storage resource management (SRM).
In August 2012, we started Alpha testing for the ViPR technology in the lab, provisioning some basic block storage on a VMAX and offered feedback to the development team that contributed to ViPR’s portal design, provisioning features and several other enhancements.
In January of 2013, EMC IT installed the new ViPR framework in its data center as one of the early adopters of the technology.
EMC IT has played an extensive role in the overall architecture of the ViPR platform, identifying the standard storage services, defining IT use cases, and performing Alpha and Beta testing. We participated in production deployment and provided significant feedback on identifying the solutions gaps and automation opportunities. It is part of our longstanding EMC approach of using our own technology and leveraging our real-world experience to make our technology better.
We are currently using this new framework to do object provisioning, through which we can provide EMC Atmos, Amazon S3 (Amazon Simple Storage Service) and open stack Swift API compatible storage to developers without having to use the pubic cloud. That means a developer can create products that are compatible with these application program interfaces (APIs) without having to put their data outside the data center. A developer can request cloud storage from our Self Service ViPR portal and use it to develop new apps that can talk to both public and private cloud.
What’s more, such provisioning can be completed in the same day it is requested. This gives EMC developers a lot of flexibility and efficiency.
In a second use case, we are using ViPR for Network File System (NFS)-based file provisioning, allowing users to store and share files from UNIX or Linox machines. This capability, used mostly by engineers developing code or analyzing logs to solve customer problems, is not yet offered in our service catalog but will be soon.
This software defined framework is just the first step in what we see as a multi-year journey to automate and optimize storage management as we have other facets of IT and use the flexibility and efficiencies that brings to accelerate our IT transformation.