Second wave of big data From services to solutions
For many big data buyers, 2012 was the year of services-led engagements. Suppliers with well-tuned professional and educational services units, such as Dell, IBM and Opera Solutions, helped many early adopters achieve results. Apache Hadoop™ distribution providers, including Cloudera, Hortonworks and MapR, also took part. But as organizational profiles shift from early adopters to early majority, discretionary budgets for big data are expected to be slimmer on average, and the number of highly customized, risk-taking projects may diminish.
Most true believers in data science have already made some kind of big data investment; in fact, they needed little convincing. However, the second wave of big data buyers wants an approach that is not primarily customized and is easy to implement. They look for all-encompassing solutions — infrastructures, platforms, productivity tools and even big data applications — with lower apparent costs than are typically incurred from mainly custom efforts.
Counter to much of the hype surrounding big data, Enterprise Strategy Group (ESG) has fresh evidence to support the notion that although big data investments during 2013 will be healthy, they will not be over the top: Business intelligence (BI)/analytics spending, while above average for IT initiatives, sits in the third tier of initiatives in ESG’s 2013 IT spending intentions research survey.1 When asked which business initiatives they believed will have the greatest impact on their organization’s technology spending decisions over the next 12 months, 44 percent of respondents cited cost-reduction initiatives. Security and business process improvement each came in at 31 percent; multiple selections were allowed. BI/analytics initiatives fell into a group in the 21–25 percent range, which included compliance, mobile computing and collaboration.
In fact, ESG saw the coming of the second wave in the latter half of 2012; the figure on the left is taken from a survey ESG conducted regarding big data.2 The primary concerns include security, integrating data, cleansing data and ensuring analytics performance does not disappoint users. To ESG, these primary concerns clearly reflected an enterprise-class set of needs. The notion of an experimental, throw-money-at-the-problem big data approach was already losing its luster.
Favoring appliances and preconfigured hardware
Given healthy, albeit not extravagant, budgets for big data and buyers who are not quite at the cutting edge, where will big data buyers invest? ESG believes many of these buyers will increasingly show interest in appliances and preconfigured hardware for big data solutions. Though interest continues to grow for using the cloud for parts of big data, many organizations still prefer to opt for infrastructure where they exercise control — but not to the point where they want to spin up their own clusters. And unless the organization is a Web 2.0–style company, ESG does not generally recommend a do-it-yourself (DIY) approach to spinning up and managing big data hardware and infrastructure for two main reasons:
First, few IT shops and almost no lines of business possess the big data expertise on staff to architect, design, spin up and manage their own infrastructure for a full-scale and rapidly growing production environment. Those that take that risk without the requisite skills may quickly head over the edge of their own economic and delivery cliff. Those willing to hire DIY big data skills should expect to pay well more than average for relevant IT and data-science expertise.
Second, organizations are overly optimistic if they believe chief marketing officers and lines of business will chip in massive amounts of budget for capital or operational expenditures that go toward big data projects with an unpredictable return on investment (ROI). The business may provide somewhat of an investment, but the serious projects — not the experimental ones — still answer to the chief financial officer (CFO), just like any other IT-oriented project, regardless of budgetary source. ESG believes, therefore, that serious projects should, if they plan to be successful, allocate a healthy, predictable and dependable portion of budget for infrastructure. Appliances and preconfigured infrastructure fit those criteria.
The trick in these cases is to offer an appliance or preconfigured hardware tuned for a particular big data software solution. To date, ESG has not seen a successful generalized big data infrastructure with universal appeal, and does not expect one to appear in 2013 or probably ever. Instead, from a success perspective, when applicable it’s important to have big data software solution and infrastructure providers walking hand in hand to help their customers with big data projects. That second wave of big data buyers wants the help and assurance from all the primary vendors involved, whether hardware, software, network or hybrid — and they will still likely need plenty of big data services to help get going.
Dell has recognized the appeal of appliances, preconfigured infrastructure and partnerships for big data. For example, the company currently offers a preconfigured Hadoop™ infrastructure that includes the Cloudera’s Distribution including Apache Hadoop (CDH) software. Another example is the SAP® HANA™ appliance from Dell that is configured and highly tuned for HANA, an analytics database being used by an increasing number of SAP organizations to revolutionize operational analytics. Dell also brings a set of well-versed SAP solution experts to the table and has even created a set of services specific to harnessing and managing the in-memory– powered speed of HANA.
Meeting enterprises where they are
The year 2012 also saw an immense amount of investment from venture capitalists in early-stage big data solution providers. Unfortunately, 2013 begins the days of reckoning for some of those investments. ESG, therefore, really likes either the established, larger and well-known solution providers that work well with others in the complex big data solution space, or those relatively new solution providers that specialize in a clear niche. And the smart solution providers from the latter category understand they need to work with reliable and trusted providers like Dell to reach interested but careful big data customers.
Dell formed partnerships with several big data solution specialists during 2012, such as Datameer, ParAccel and Pentaho. These vendors offer holistic solutions — from core BI/analytics with a good dose of big data for medium-to-large companies (as offered by Pentaho) to the blazing-fast massively parallel processing (MPP) analytics database and platform for complex, predictive analytics (from ParAccel). ESG believes Dell’s span of big data solutions can help customers regardless of their current affinity for or expertise in big data.
Defining big data going forward
To capitalize on big data, an organization’s infrastructure requirements depend on its goals and current level of adoption. Has the company invested long-term in advanced analytics to better understand customers or to help make fresh discoveries that drive research and development? Or is the organization still struggling with providing basic BI to its executives? Many companies are in between. But in nearly all cases, to make progress they must adapt their infrastructure to the needs of big data solutions and augment their expertise to some degree.
Much has been written about the three Vs of big data — referring to the fact that big data often involves more data, or volume; more types of data, or variety; and a faster influx of data, or velocity, than in the past. ESG believes that organizations should focus on six rather than three Vs, adding veracity, to help ensure analysts are working with clean data; visualization, to provide end users with rich, flexible, graphical tools to tap into the power of big data; and value, to help the CFO ensure that the organization realizes more benefit from big data than the investment being made in big data.
In 2013, the idea of let’s put up a few clusters, download Hadoop, throw some data at the clusters and MapReduce and see what we can find is not going to cut it. This approach no longer defines big data this year or going forward. Rather, organizations are demanding well-thought-out and proven infrastructures; software solutions that enable quick results; and services that help define realistic projects and results and deliver the combined big data solution — at a price that does not break the bank.
Evan Quinn is Senior Principal Analyst at Enterprise Strategy Group.
Enterprise Strategy Group:
Dell | Cloudera solution:
Dell | SAP HANA solution:
1 “Research report: 2013 IT spending intentions survey,” by Bill Lundell and John McKnight with Jennifer Gahm, Enterprise Strategy Group, January 2013, qrs.ly/ah2z6ao.
2 “Research report: The convergence of big data processing and integrated infrastructure,” by Evan Quinn and Bill Lundell with Brian Babineau, Enterprise Strategy Group, July 2012, qrs.ly/tp2z6as.