Analyze this – big data for Oil & Gas 2016 economics

The concept of a Digital Oil Field is nothing new. Yet the Oil & Gas industry still struggles to leverage broad-scale big data & analytics in a way that makes it as mainstream and clearly understood as interpretation and modelling processes. Granted there has been some success in areas such as predictive maintenance and drilling optimization, but very little in sustained step-change improvements that have redefined the way production is planned and executed – especially when we look back at 2015 Oil & Gas economics and the continued challenge in 2016 and beyond.

One of the key obstacles that make leveraging the power of analytics difficult is the sheer volume and variety of data available to potentially fuel workflows:

  • How do I consolidate data from multiple systems into a single secure environment (a Data Lake) that scales easily and cost-effectively so it can be more easily managed?
  • How can I create models across heterogeneous data sets so that benefits can be seen not only in one section of the Hydrocarbon Value Chain, but across multiple domains to produce exponential benefits?
  • How can I execute analytical workloads at a velocity that is sympathetic to data latency issues so that business planning fundamentals have a larger reliable predictive component?

The good news is that advancements in data storage and computing power have offered solutions to individual technical infrastructure challenges, but putting them together to form a smooth running analytics platform still requires specialist skills.

However, perhaps even more important than the technology challenge is the figuring out of what we should perform meaningful analytics on.

  • Which areas of the business should I look to optimize that will make significant operational impact where savings or increases in value are measured in the tens or hundreds of millions of dollars?

The pessimism around big data and analytics in general comes through in the August 2015 Gartner report ‘The Demise of Big Data, Its Lessons and the State of Things to Come’, in which the following Strategic Planning Assumption is made:

“Through 2018, 90% of deployed data lakes will be useless as they are overwhelmed with information assets captured for uncertain use cases”.

Source: Gartner report – ‘The Demise of Big Data, Its Lessons and the State of Things to Come’ Published: 19 August 2015

So what is the answer?

We certainly cannot give up on big data & analytics any more than we should give up on making exploration and production more efficient. We need to claw back margins lost to low oil price economics, particularly in the Upstream segment of Oil & Gas.

I believe we have found at EMC what appears to be a solid approach that enables the rapid organized processing of data to extract timely actionable insight to drive step-changing efficiencies. It is used to appraise, design and implement digital oilfield and big data & analytics projects to improve productivity and reduce well and facility downtime. We have identified that there are 5 key ingredients for success:

ADDED - analytics

But the bit that takes you over the wall is being able to blend the 5 ingredients effectively, so that one does not get in the way of another – for example, you could have a great analytics engine, but if the data management platform that will feed it scales poorly or cannot deliver quality-checked data rapidly, the entire analytics process is weakened. If you have access to external intelligence but cannot integrate the data in a way that enriches your existing data sets, then the true value of that industry knowledge is never realized.

We believe we’ve figured out a way to do that blending in a repeatable methodology driven through our Oil & Gas Big Data Vision Workshop. In a recent case spanning 10,000 wells, analytics using EMC solutions showed a potential increase in oil production of 8-14%, and decreases in completion costs of more than 5%. In another case, a large Independent was able to uncover that if they spent an extra $40M on unconventional drilling operations, they would see a $200M return over 3 years.

Want to learn more about the Big Data Vision Workshop? Just send an email to

About the Author: Julian Alfred