Over the past couple of months, I’ve been on a data management road tour of sorts, starting with Cassandra Summit in mid-September, then on to the Gartner BI & Analytics Summit, Dell World, and Oracle OpenWorld, before concluding things at the Hadoop World in Singapore. At each stop, there was lively discussion about the proliferation of traditional and non-traditional data sources as well as debate about whether it should all live in the cloud, on-premises or both.
At Oracle OpenWorld, experts touted the many merits of moving everything to the cloud. A customer at Dell World shared the same sentiment, saying, “If it’s not in the cloud, we won’t be having any of that.” This groundswell of support even has some analysts predicting that in the near future companies will no longer feel the need or urge to stand up in-house custom applications. Instead, everything that’s needed—from apps to experts—will be rented.
So, does this mean that a company running a hosted human resources app could do away with their HR department altogether? Sure, but then they also would need to buy specific HR expertise on an as-needed basis, which could end up being expensive. It all comes down to each specific organization and its ultimate business goals and objectives.
That’s why I worry when everyone gets caught up in the hype that makes everything sound like an “either/or” proposition. Either move everything to the cloud or keep it all on-prem. Embrace non-traditional data sources, like NoSQL, or keep it old school and only use traditional sources like SQL. Isn’t there a middle ground here somewhere?
Practically speaking, most organizations will benefit most from a blend of cloud and on-premises solutions. Likewise, I think the whole SQL vs. NoSQL debate will become irrelevant as companies combine different data sources to address both high-volume data from social media and IoT technologies with highly valuable, transactional, run-your-business kind of data. Instead of arguing endlessly over which one platform or data source is better, there clearly is a place for them all.
I vote for a data management middle ground where cloud-based and on-premises data ecosystems coexist amid an abundance of traditional and non-traditional data sources. For the next five to seven years, I believe the market will continue to be shaped by the emergence of different data sources that give companies multiple ways to cast the net wider for greater business insights.
Cloud-based, on-prem, SQL, NoSQL—they all present both hurdles and opportunities. In my matter-of-fact approach, I would start by defining a company’s assortment of data use cases to shed light on which data sources and platforms might be most beneficial. For instance, if your goal is to collect all the Facebook data in the world, you’ll likely start with a Hadoop cluster. If you need to gather highly sensitive HIPAA-compliant data, then an on-prem data warehouse is a better bet.
It’s equally important to understand the success metrics of any new system—before you build it. So, instead of dividing data sources into traditional and non-traditional camps, you’d be better served by determining whether a high-capacity, high-performance solution fits the bill or do you need a super-accurate and highly available solution? Being ever practical, organizations also need to determine the trade-offs to deploying in the cloud or on-prem. It’s not so much about which source or solution will reign supreme, but rather which combination can best solve a particular business problem.
With my feet firmly on the ground—and not in any specific camp—I’m wary of any wave of hype that positions companies as old-school just because they have a data warehouse. Similarly, moving to the cloud does not make you an automatic innovator. Every organization will have different requirements, which is why it’s so critical to find the solution that meets your practical needs. It’s really that simple, even if the process of getting there is increasingly complicated.
For most companies, an ultimate balance of cloud, on-prem, traditional and non-traditional solutions will come together in a multiplatform or polyglot-styled data infrastructure. In architecting these solutions, many will rely on ever-practical data management tools that deliver Swiss Army knife-like functionality to cut through the complexities. In doing so, they address an ever-growing list of business requirements without ever leaving the data management middle ground.
What do you think about this idea to meet in the middle? Connect with me on Twitter at @joschloss to share your thoughts.