Digitizing oil and gas production

| Article

The rapid progress of technology such as big data and analytics, sensors, and control systems offers oil and gas companies the chance to automate high-cost, dangerous, or error-prone tasks. Most oil and gas operators are starting to capture these opportunities and would do well to accelerate their efforts. Companies that successfully employ automation can significantly improve their bottom line.

While automation offers many potential benefits in the upstream value chain of exploration, development, and production, some of the biggest opportunities are in production operations, such as reducing unplanned downtime (Exhibit 1). Given the oil and gas industry’s substantial increases in upstream capital investment, optimizing production efficiency1 is essential. Automation creates several opportunities to that end: maximizing asset and well integrity (by which we mean optimizing production without compromising health, safety, and the environment), increasing field recovery,2 and improving oil throughput.3

1
The highest-impact automation opportunities in upstream lie in production operations.

With the substantial production volumes of offshore production platforms, even small improvements in production efficiency will have meaningful financial impact, as additional throughput translates directly into more revenue. In the low-volume regimes of current unconventional mature assets—oil sands, for example—carefully targeted automation steps can cut costs and, more important, can also improve the reliability of production equipment, leading to higher revenues that can extend an asset’s economic life.

Industry challenges

Our benchmarking analysis of North Sea offshore platforms illustrates the efficiency challenge that many oil and gas companies face. Research shows that average production efficiency dropped in the past decade, while the performance gap between industry leaders and other companies widened, from 22 percentage points in 2000 to around 40 percentage points in 2012.

Benchmarking data also illustrate the broad opportunity for improvement. Best-in-class players do not incur higher costs to improve production efficiency, and high performance is not linked to a specific asset type or the maturity of assets. Instead, companies with high production efficiency are often similar in their quality of operations, approach to eliminating equipment defects, equipment choices, and planning and execution of shutdowns. While our benchmarking focuses on North Sea offshore platforms, we expect to see similar patterns in other regions.

Regardless of location, most oil and gas companies also face issues that complicate efforts to achieve sustained production-efficiency improvements. We believe further automation can play a major role in addressing the following industry-wide challenges.

More complex operations. Increasing volume and complexity in hostile, remote locations (for example, arctic, offshore, and deepwater) require reliable remote and automated or semiautomated operations, and logistics optimized for efficiency. Mature assets with declining production need very efficient maintenance schedules to keep production profitable.

Zero tolerance for health, safety, and environmental incidents. This is a nonnegotiable imperative. Recent industry experience has shown that in the current highly regulated environment, such incidents can threaten not only profitability but also the very existence of an operator. Automated production control, monitoring the condition of the equipment, and predictive shutdown systems are now basic requirements to prevent or mitigate catastrophic events in geographically dispersed remote operations.

The talent and experience gap. The industry is in the most dramatic demographic shift in its history, commonly referred to as “the big crew change.” Thousands of petrotechnical professionals will be retiring soon, resulting in a knowledge and experience crisis for the industry. Retention and recruitment are unlikely to fill the gap completely. This development drives efforts to codify many routine analysis and decision-support processes and, where possible, to automate them.

The automation imperative

Automation is not without its own challenges. Today’s intelligent oil field is flush with digitally enabled wired systems, equipment, and components. A typical offshore production platform can have more than 40,000 data tags, not all connected or used. Converting this complex flood of data into better business and operating decisions requires new, carefully designed capabilities for data manipulation, analysis, and presentation, as well as tools to support decision making.

The impact of addressing these automation challenges can be material. Judging by our benchmarking research, improving production efficiency by ten percentage points can yield up to $220 million to $260 million bottom-line impact on a single brownfield asset. For declining assets, automation could extend field life in an economically viable way. The potential could be even more significant for greenfield assets, where required instrumentation can be included from the start as part of the design. However, operators and drilling contractors are not the only players looking for a share of the production-efficiency gains in oil and gas. (See sidebar, “Automation is changing the competitive landscape.”)

We expect industry leaders to increasingly adopt automation in upstream production operations, leading to improved efficiency. As a result, the performance gap with industry laggards could widen further. To illustrate how oil and gas companies can unlock the value of automation, we analyzed production maintenance, where the opportunity is particularly attractive.

Applying automation to maintenance

There are many ways in which automating maintenance can improve production efficiency. For example, radio-frequency-identification tagging of equipment, along with the use of other sensors, can help track activity. Tracking, in turn, enables applications that can monitor the condition of equipment and support predictive maintenance and automated operations shutdowns. These applications minimize risk of catastrophic failures and process disruptions, while maximizing equipment reliability and production efficiency.

But unlocking the value of automation in maintenance isn’t only about having lots of data. Some companies struggle to maintain data quality across their IT networks. Others are not good enough at aggregating data and conducting meaningful analyses. Yet others experience challenges in turning analysis into action. That’s why many oil and gas operators need to identify the information shortfalls or leakages that occur when capturing data from processes, systems, and data stores and move them to where operational and business decisions are made (Exhibit 2). Having identified the leakages, they must then address them by improving the automation of their data flows.

2
Limiting data leakages can improve operational and business decisions.

Many forms of leakage impair automation in maintenance. One example is having only isolated data availability from individual equipment components, as opposed to more network-based availability. Another is having only equipment-level profiles of components at risk, as opposed to comprehensive coverage at the asset level. A third example is to only catalog critical equipment failures rather than conduct extensive root-cause analysis of them.

Automated support of real-time decisions and actions to reduce planned and unplanned downtime require the following elements.

Data capture. This involves automated hardware sensors and manual data capture by engineers. Both should be deployed based on a detailed analysis of use cases, which are a method for gathering the functional requirements of applications.4 Hardware sensors should help ensure sufficient coverage of data, as well as provide redundancy (that is, data backups) for high-value measurements of equipment-performance data. High-precision hardware sensors are usually more costly than low-precision sensors and should be used only in critical cases. Manual data capture is useful when parts of assets are not yet equipped to monitor and measure performance or for inspections and failure analysis. Using regulatory-certified handheld electronic devices for manual data capture improves data accuracy, consistency, and availability.

Data infrastructure and data management. Data infrastructure—ranging from transactional data in relational (such as enterprise-resource-planning) databases to data in Apache Hadoop or similar big data analytics platforms—should allow companies to couple data from disparate sources of equipment and manual capture to aid decision-support analytics. Streaming of real-time data in situations requiring immediate data availability also needs infrastructure support. Technology choices should be made on an individual basis by evaluating the data flow and asking several questions: Is the volume of data high? Is it real time or batch? Are the data unstructured or structured? For example, for unstructured data processing, a solution like Apache Hadoop is more relevant than traditional relational databases.

Data analytics. Industry leaders are using analytical models to predict failures of critical equipment components. The next level of sophistication includes connecting all the parts of the end-to-end production value chain to optimize the balance between production and downstream stages, for example, by adapting upstream production levels to account for expected future demand shifts in downstream retail. It also includes using simulations to test failure scenarios in platform operations and employing text mining for analysis of unstructured input from engineers and operators.

Data visualization and staff training. Software applications are needed that present data and insights in a way that is closely tied to priority operational and business decisions. One example is knowledge systems that suggest actions to maintenance engineers who take into account previous repair approaches as well as the specific history of the asset. Managers and staff must have adequate training to collaborate and work with these applications.

Success factors for automating oil and gas production

In our experience, companies that have successfully pursued automation programs for production efficiency have employed several effective approaches.

Building multidisciplinary teams

Successful automation programs have staff with backgrounds ranging from process automation, process-domain expertise (for example, in maintenance), data management, and cybersecurity to interface design. These multidisciplinary teams include representatives from every aspect of the organization’s IT function. Sometimes the teams also include equipment vendors.

Differentiating greenfield and brownfield automation

In greenfield automation programs, the digital processes are built in during project development to ready the technology for future advances, taking into account the five-to seven-year life cycle of these projects. For brownfield programs, companies develop overlays (for example, upgrades of wireless and mobile) that pull the required data flow out of the platform to support analytics units. This approach helps to avoid being locked in by technology choices from the past. Some companies use a library of reusable software components to achieve economies of scale across offshore platforms. In our experience, this library approach is preferred to mandating a single, centrally developed overlay across platforms.

Thinking big, piloting small, scaling fast

Companies with successful programs think in terms of total life-cycle costs and economics. They build a digitization team and make automation part of a corporate digitization program. Their automation programs are integrated with all aspects of their complex organizations, work processes, and human behaviors. Industry experience and prudent risk management dictate that this level of complexity be thoroughly tested and proved in small-scale pilot implementations. Once the concept is proved, rapid scaling is needed to secure the payoff. Such a scale-up requires tools and capabilities in technology-enabled transformation, change, and risk management.


There is a clear competitive imperative for increasing automation in oil and gas production. Companies that successfully implement big data and analytics, sensors, and other new technologies will be well positioned to meet their industry’s challenges.

Explore a career with us