The Quantity Challenge: Managing the Data Deluge

“Having data doesn’t give you productivity.
Having better decisions gives you productivity.”
   Michael Trick, Professor, Tepper School of Business
   Carnegie-Mellon University

I thought about this quote during the first session that I attended at the SPE’s Annual Technical Conference and Exhibition in Anaheim, California, earlier this week. In the past few years, new technologies have become available that have led to an exponential increase in the data available to operations and management. Upstream oil and gas operations are a case in point, where there now are constant data feeds from sensors embedded in the fields and the drilling equipment, sent to the surface and transmitted via satellite to operations centers.

The question everybody is asking is what we should do with all this data. Roger Hite from Business Fundamentals Group, a consultancy firm to the oil and gas sector, put this clearly in his presentation entitled “Barriers to Implementation of Real-Time Optimization Technology” on Tuesday: “When you collect data, you’ve simply added cost. You need to add decisions to add value”.

However, adding decisions—and, if I may add, good decisions—about operational situations remains challenging. Knowing, at any time point, the location of every truck or the pressure value at every valve, does not mean that we know how to act. Last August I attended the 25th DARPA Systems and Technology Symposium (here at the very same conference center) where a representative from the U.S. Armed Forces told me that they now have evidence showing that the quality of critical decision making sometimes has worsened with more data-rich support systems.

Data is not the problem. Good decisions are the problem.

Decision support systems are usually focused on finding ways of collecting, transmitting, mining, and visualizing data. The last critical piece, decision-making, is left to the user. Frequently, organizations try to overcome this barrier to effective decision making by using various operations analytics tools, including data mining and statistical pattern recognition. However, these approaches provide only a part of the solution.

Making decisions, for humans or computers, is far from straightforward. Usually, we have many constraints, competing objectives, and ad hoc knowledge, and often the problems themselves get exponentially more difficult to solve when the number of components increases. Leaving this challenge to the user alone is often not appropriate. It is therefore interesting to see how developments in search and optimization algorithms (such as those resulting from our work at Actenum) now make it possible to address operational decision making.

At Actenum, we strongly believe in empowering users, not replacing them. We therefore develop our applications as decision proposal tools. This means that the decision process is collaborative—what is sometimes called mixed initiative problem solving—and good decision making is a process where the user and the application play a bit of ping-pong, each of them proposing alternatives, until the user declares bingo!