Chapter 2. Technology and Unconventional Sources in the Global Oil Market
- Rabah Arezki, and Akito Matsumoto
- Published Date:
- September 2017
Technological factors played an important role in the collapse of oil prices that started in June 2014. Macroeconomists often assume that technological innovation results from independent, external forces (is exogenous), but in oil markets innovation is driven by prices. Indeed, high oil prices prompted breakthroughs in technology in extractive industries and led to the emergence of new sources known as “unconventional oil.” Shale oil in particular has important consequences for the oil market outlook in that it not only significantly increases supply but also contributes to more limited and shorter production and price cycles.
Technology has transformed the oil market in powerful ways. Technological innovation and the subsequent adoption of new recovery techniques—including for drilling and processing—have given rise to new sources known as “unconventional oil.” One recent example is shale oil (also known as tight oil), which has become a major contributor to global oil supply. Provided they are effective and widely adopted, improvements in recovery techniques mechanically increase the size of technically recoverable oil reserves. This increase, in turn, changes the outlook for oil supply—with potentially large and immediate implications for oil prices—by changing expectations about the future path of oil production. Increased supply lowers oil prices, but even if this has the effect of reducing investment and hence production, the industry is nonetheless forced to become more efficient, unleashing automatic stabilization forces.
Innovation in recovery techniques typically follows periods of prolonged high prices or changes in regulations that render new techniques more economical. New oil sources often come onstream in times of need—because of, say, the depletion of existing conventional sources—and in places that have economic and institutional systems more favorable to both innovation and the adoption of new recovery techniques. Innovation has led to significant improvements in drilling techniques in particular. The advent of hydraulic fracturing and directional (non-vertical) drilling gave rise to the production of shale oil in the 2000s by allowing for the capture of oil trapped within layers of rock. In the wake of the two oil crises of the 1970s, which dramatically increased oil prices, successive improvements in techniques for deepwater drilling spurred production in the North Sea and the Gulf of Mexico. In both these examples innovation opened new oil sources from relatively high-cost producers and gave rise to tensions with the lower-cost producers from the Organization of Petroleum Exporting Countries (OPEC), who in the 1980s and again more recently responded by strategically moderating their production levels.
This chapter addresses four questions about the role of technology and unconventional oil sources in the global oil market:1
What constitute unconventional oil sources?
Where are the production and reserve centers?
How have investment and production evolved?
What lies ahead?
What Constitute Unconventional Oil Sources?
Today’s unconventional oil sources are extra-heavy oil extracted from oil sands, shale (or tight) oil, and ultra-deepwater oil.2 Unconventional oil is typically more difficult and more expensive to extract and process than conventional oil. The categorization is, of course, time specific because the sources of oil evolve along with improvements in recovery techniques. “Conventional oil” used to refer only to light crude that was easily captured by tapping into a reservoir. But the term now often applies also to heavy oil and deepwater oil, which were once considered unconventional. To give a historical perspective on how “new” oil sources have contributed to the evolution and transformation of the oil market, this chapter adopts a broad, all-encompassing definition of unconventional sources, including those no longer considered unconventional (such as heavy and deepwater oil).
Oil sands are either loose sands or partially consolidated sandstone containing a naturally occurring mixture of sand, clay, and water saturated with a dense and extremely viscous form of petroleum referred to technically as bitumen and colloquially as tar because of its superficially similar appearance. Heavy and extra-heavy oil are characterized by high viscosity, high density, and high concentrations of nitrogen, oxygen, sulfur, and heavy metals. These characteristics raise the costs of extraction, transportation, and refining. Despite the cost and technical difficulties, major oil corporations regard these resources as providing reliable long-term flows of liquid hydrocarbons with substantial payoffs for any incremental improvements in recovery. However, there are environmental concerns about potential damage from extracting and refining these new oil sources, which have often been met with specific safety regulations and standards meant to help limit the risks.
Shale oil (also known as tight oil) is petroleum that consists of light crude oil contained in petroleum-bearing rock formations of low permeability, often shale or tight sandstone. The widescale exploitation of shale oil began with the development of shale gas extraction using a combination of hydraulic fracturing, also called fracking, (a well-stimulation technique in which rock is fractured by a hydraulically pressurized liquid) and directional drilling (the practice of drilling nonvertical wells). These gas-recovery techniques were later widely adopted by the oil industry, primarily in the United States. Shale oil sources are developed by relatively smaller corporations. Shale oil also has a different cost structure: there are lower sunk costs than for conventional oil, and the lag between initial investment and production is much shorter.
Deepwater and ultra-deepwater oil involve offshore production activities that take place at depths of more than 125 meters and 1,500 meters, respectively. Successive improvements in drilling techniques have allowed drilling much farther from coastlines and to much greater depths. The offshore rigs used for ultra-deepwater oil drilling differ very significantly from the rigs used for deepwater drilling: ultra-deepwater rigs are partially submerged in water and can involve dynamic positioning systems, or they can be drill ships—self-propelled offshore drilling rigs that can work beyond a depth of 3,000 meters. Although it has high fixed costs, ultra-deepwater drilling can deliver a steady stream of oil for a very long period, which makes these assets attractive to major international oil corporations.
Where are the Production and Reserve Centers?
Production and reserve centers for unconventional sources are concentrated in a few countries. North America has the highest concentration of economically recoverable proven reserves and production in unconventional sources (Figure 2.1; Table 2.1). These consist of shale oil in the United States and oil sands in Canada. Central and South America also host significant reserves and production centers, comprising heavy and extra-heavy oil and deepwater and ultra-deepwater oil resources in Brazil, Colombia, Ecuador, and Venezuela. The remainder of world reserves and production of unconventional sources are scattered and consist primarily of heavy oil in Europe and deepwater and ultra-deepwater oil in the North Sea and waters off west Africa. It is noteworthy that the Middle East has the highest concentration of conventional oil reserves and production but has a relatively low level of proven reserves and production in unconventional oil.
Figure 2.1.Unconventional Oil, Proven Reserves and Production, 2016
Sources: Rystad Energy research and analysis; and IMF staff calculations.
Note: Production and reserves include: oil sands, heavy, extra heavy, tight and shale, deepwater, and ultra-deepwater oil. A proven reserve is one with a greater than 90% probability that the resource is recoverable and economically profitable. Deepwater is defined at 125-1500 meters. Ultra deepwater is defined at 1500 meters and above. When deepwater (or ultra-deepwater) production was also categorized as heavy (or extra heavy) oil, the production was counted once, as deepwater (or ultra-deepwater). Oil refers to crude oil, condensate, and natural gas liquids.
|Country||Heavy oil||Oil sands and extra heavy oil||Deepwater||Ultra-deepwater||Shale and tight oil||Total|
|Congo, Republic of||-||0.01||0.16||-||-||0.17|
In addition to physical geology, the high concentration of unconventional proven reserves and production reflects the geographical distribution of innovation and the subsequent adoption of new recovery techniques, which in turn reflects the levels of investment in exploration and extraction. Resource economists have long argued that, conceptually, the resource base is uncorrelated to the level of effort applied to explore resources.3 Knowledge about the actual geology is gained through exploration efforts and constantly evolves with technological improvements. In other words, proven reserves and production are governed as much by economic and institutional factors (above-ground factors) as by actual geology (below-ground factors).
Economic factors affecting the geography of exploration and production include proximity to markets and complementarities with available infrastructure. These factors often lead to agglomeration in both production and proven reserves.4 Institutional factors affecting exploration and production include openness to foreign investment and the strength of property rights, including in subsoil assets. Arezki, van der Ploeg, and Toscani (2016) provide empirical evidence of a causal—and economically significant—relationship running from changes in market orientation to discoveries of major hydrocarbon and mineral deposits, over and above increases in resource prices and depletion rates.
The observed differences between known reserves and production across countries reflect differences in production efficiency. These differences can be explained by institutional factors emanating from the ownership structure of the industry. For instance, Wolf (2009) provides evidence that the structure of ownership in the oil sector—that is, the existence of state-owned operators—plays a key role in determining relative efficiency. He finds that, all else equal, non-state-owned oil corporations significantly outperform state-owned ones. Difficulties with production systems can lead to a low propensity to produce from existing reserves. To exploit unconventional sources, oil companies must be able to innovate or to implement new techniques.
Regulatory changes also play a central role in determining the occurrence of innovation and the subsequent adoption of recovery techniques. Consider shale oil in the United States. The existence of large reserves of oil—and gas—in shale formations in the United States was well known long ago, and shale oil production was attempted several times, first in the mid-nineteenth century. Until the mid-2000s, however, extracting oil from shale rock formations was not cost-competitive with other sources. In part a response to price rises driven by the rapid increase in demand from emerging market economies such as China and India, the advent of shale oil production was also the consequence of a regulatory shock in the United States. The expansion of shale oil extraction was aided by a landmark study conducted by the U.S. Environmental Protection Agency in 2004, which found that hydraulic fracturing posed no threat to underground drinking water supplies. Shortly thereafter, with passage of the Energy Policy Act of 2005, chemicals used in hydraulic fracturing were exempted from Safe Drinking Water Act regulations (Gilje, Loutskina, and Strahan 2016).
Shale oil deposits have been identified in several other countries, including Argentina, Australia, Canada, China, Mexico, and Russia. However, except for Argentina and Canada, where shale oil production is gearing up, regulatory obstacles and technological challenges, as well as the fall in oil prices, have delayed or discouraged extraction. Most regulatory obstacles relate to environmental concerns, including water supply quality, and to the need to tailor fracking techniques to more complex rock formations.5 Some countries have gone so far as to ban all exploration and production of shale oil. Overall, the extent to which shale oil production will diffuse globally remains uncertain, contributing to broader uncertainty about the global oil supply outlook.
How Have Investment and Production Evolved?
The adage “necessity is the mother of invention” illustrates the cyclical nature of technological change (Hanlon 2015). The direction of technical change is biased toward specific needs, depending on prevailing forces (Acemoglu 2002). In the oil sector, the need to address the rapid depletion of conventional oil reserves in certain locations and the resulting periods of high oil prices have fostered improvements in recovery techniques. Episodes of high prices have been accompanied by significant increases in research and development expenditures, mostly on the part of major corporations, though at times by smaller corporations (Figure 2.2). The current low-price environment provides few incentives for research into oil-recovery techniques. Lindholt (2015) finds that technological improvements resulting from research and development activity have offset the effects of ongoing depletion on the cost of finding and developing additional reserves of oil around the world. However, he finds that over a longer period, depletion generally outweighs technological progress, most likely because technical improvements are cyclical whereas depletion is constant.6
Figure 2.2.Evolution of Research and Development Expenditure in Selected Integrated Oil and Service Companies
Source: IMF, Primary Commodity Price System; Bloomberg L.P; and IMF staff calculations.
Note: APSP = average petroleum spot price--average of U.K. Brent, Dubai, and West Texas Intermediate, equally weighted. The list of companies included are Baker Hughes, BP P.L.C., Chevron, ExxonMobil Corporation, The Halliburton Company, Royal Dutch Shell plc, Total S.A., and Schlumberger Limited.
The so-called peak oil theory predicted that oil production would top out in the mid-2000s, but this is precisely when the shale revolution began. In many respects, that revolution can be viewed as an endogenous supply response to high prices in the 2000s and hence a challenge to the overly pessimistic view that geological factors limit supply (Arezki and others 2017).7
Historically, global investment and operational expenditures in unconventional oil have closely followed oil price developments (Figure 2.3).8 During episodes of dramatic price movements, as in the late 1970s, investment in the oil sector responded promptly. In late 2008, during the global financial crisis, oil investment plummeted but then rebounded in 2009 following the sharp albeit temporary drop in oil prices. This episode marks an unprecedented increase in global capital expenditure and reflects a prolonged era of high oil prices. The rapid increase in oil demand, especially from large emerging market economies such as China and India, drove oil prices up and encouraged further investment in tight oil formations, ultra-deepwater oil, and extra-heavy oil, all of which were uneconomical at lower oil prices. Comovement between oil prices and capital expenditure is similar for both unconventional sources and conventional sources, but expenditure in unconventional sources embodies technological changes that contribute to changing the response of global oil production. Shale oil requires a lower level of sunk costs than conventional oil, and the lag between initial investment and production is much shorter. Shale oil thus contributes to shorter and more limited oil price cycles (Arezki and Matsumoto 2016a).
Figure 2.3.Historical Evolution of Global Capital and Operational Expenditures
The unprecedented increase in capital expenditure in unconventional sources in the 2000s made these sources central to the global oil market. Shale oil in particular has been a major contributor to global supply growth (Figure 2.4).9 The rapid increase in unconventional sources also helped spur a change in OPEC’s strategic behavior, leading to the dramatic collapse in oil prices (Arezki and Blanchard 2014). Although that abrupt decline in prices led in turn to a reduction in investment and expenditure, the large operational efficiency gains already realized acted as automatic stabilizers.
Figure 2.4.Growth in Unconventional World Oil Production and Real Oil Prices
Source: IMF, Primary Commodity Price System; Bureau of Economic Analysis; Rystad Energy research and analysis; and IMF staff calculations.
Note: APSP = average petroleum spot price--average of U.K. Brent, Dubai, and West Texas Intermediate, equally weighted. Total world production in 2016 was estimated at 96.5 mbd (million barrels a day). Deepwater is defined at 125-1500 meters. Ultra deepwater is defined at 1500 meters and above. When deepwater (or ultra-deepwater) production was also categorized as heavy (or extra heavy) oil, the production was counted once, as deepwater (or ultra-deepwater). Oil refers to crude oil, condensate, and natural gas liquids.
The downward shift in the cost structure induced by lower oil prices is partly temporary. This goes against a commonly held belief that the cost structure—which is often proxied by the breakeven price, or the price at which it is economical to produce a barrel of oil—is constant and driven by immutable factors, such as the nature of the oil extracted and the associated geology (Figure 2.5). In practice, the cost structure depends on a host of factors, including technological improvements and the extent of “learning by doing,” which both permanently reduce costs. In some instances, breakeven prices have fallen in sync with oil prices. That type of shift is explained by operational efficiency gains that help the service industries that support oil production (infrastructure, drilling supplies, transportation, storage, and the like) significantly reduce their costs. For shale oil specifically, the extraordinary resilience to the decline in oil prices can be explained by such important efficiency gains and also by the fact that shale production came online at the onset of an investment cycle in which learning by doing was important (Figure 2.6).10 The shale cost structure is likely to shift back up somewhat because some of the efficiency gains cannot be sustained with an expansion of oil production and with the cost of capital increasing, as it is expected to do as U.S. interest rates rise.
Figure 2.5.Global Oil Supply Cost Curve and Breakeven Prices
Source: Rystad Energy research and analysis; and IMF staff calculations.
Note: NAM = North America. The breakeven price is the Brent oil price at which NPV equals zero, considering all future cash flows using a real discount rate of 7.5%. Oil refers to crude oil, condensate, and natural gas liquids.
Figure 2.6.North American Shale Oil Wells at Different WTI Oil Prices and Cost Deflation Scenarios
Source: Rystad Energy research and analysis.
Note: mbd = million barrels a day. Refers to spudded wells, defined as wells that are drilled but not extracted. At 60 U.S. dollars per barrel approximately 8000 shale wells have to be drilled, with 10% cost deflation, to keep production flat.
The shift in cost structure has not been uniform across unconventional sources. Oil sands production costs have continued to grow at high rates, in part because of the high costs of decommissioning a processing plant. At the same time, there has been less investment in exploring new fields, and this is expected to lower oil sands production in the future. Deepwater and ultra-deepwater oil production have been subject to active upgrading which has made them somewhat resilient to price changes. Here again, lower investment in new fields will likely affect future deepwater and ultra-deepwater oil production, albeit with different patterns across regions owing to below- and above-ground factors.
What Lies Ahead?
The development of unconventional oil sources is inherently uncertain, which becomes apparent when comparing the ability to forecast unconventional relative to conventional production (Figure 2.7).11 Technological improvements and their subsequent adoption—including the extent of learning by doing and the geographic diffusion of new techniques—are hard to predict, owing to the interaction between below- and above-ground factors. All in all, the rising importance of unconventional sources in global supply is not only changing the dynamic response of production to prices, it is also creating more uncertainty about the medium-term forecasts.
Figure 2.7.Unconventional and Conventional Oil Production Outlook Vintages
Source: International Energy Agency.
Note: Replicated from Wachtmeister, Henke, & Höök, 2017. Dates correspond to vintages from forecast.
Added to the uncertainty about technological improvements is uncertainty about the likely output by suppliers of conventional oil. In September 2016 OPEC negotiated an agreement to reduce oil production by 1.8 million barrels a day (mbd) originally for six months that was latter extended for another six months. In principle, this would help rebalance the market by the end of 2017, eliminating an excess supply estimated to be a little less than 1 mbd. In practice, rebalancing oil supply with demand accompanied by stable prices will hinge on the prospects for unconventional sources (Figure 2.8). Annual oil demand growth is commonly projected to be about 1.2 mbd, and this will be met over the next few years by unconventional sources, mainly resources under development for deepwater and ultra-deepwater oil, oil sands, and heavy and extra-heavy oil.
Figure 2.8.Unconventional Oil Outlook
Source: Rystad Energy research and analysis; and IMF staff-calculations.
Note: Deepwater is defined at 125-1500 meters. Ultra deepwater is defined at 1500 meters and above. When deepwater (or ultra-deepwater) production was also categorized as heavy (or extra heavy) oil, the production was counted once, as deepwater (or ultra-deepwater). Oil refers to crude oil, condensate and natural gas liquids.
Without the increase in shale oil supplies, depletion forces and the legacy of low investment would start to kick in and push prices up significantly in a few years. Instead, in the new normal for the oil market, shale oil production will likely be further stimulated by a moderate price increase (Arezki and Matsumoto 2016a). As a result, supply from shale will help moderate what would otherwise be a sharp upward swing in oil prices. Over the medium term, as prices increase further, technical improvements in unconventional oil recovery will be reactivated, which will eventually set off another price cycle.
2002. “Directed Technical Change.” Review of Economic Studies69 (4): 781–809..
2016. “Carbon Taxes, Path Dependency, and Directed Technical Change: Evidence from the Auto Industry.” Journal of Political Economy124 (1): 1–51..
2014. “Seven Questions about the Recent Oil Price Slump.” IMFdirect blog, December22..
2017. “Proximity to Markets, State Capacity and the Production of Fossil Fuels.” Unpublished..
2017. “Oil Prices and the Global Economy.” IMF Working Paper 17/15, International Monetary Fund, Washington, January..
2016. “A ’New Normal’ for the Oil Market.” IMFdirect blog, October27..
2015. “The Price of Oil and the Price of Carbon.” IMFdirect blog, December 2, and VoxEU.org, December 3..
2016. “Shifting Frontiers in Global Resource Wealth: The Role of Policies and Institutions.” CEPR Discussion Paper No. 11553, Centre for Economic Policy Research, London..
1982. “Optimal Pricing, Use and Exploration of Uncertain Natural Resource Stocks.” Journal of Environmental Economics and Management9 (1): 1–10..
2001. “Technological Change, Depletion, and the U.S. Petroleum Industry.” American Economic Review91 (4): 1135–48..
1982. “Exploration and Scarcity.” Journal of Political Economy90 (6): 1279–90..
2016. “Exporting Liquidity: Branch Banking and Financial Integration.” The Journal of Finance71 (3): 1159–84..
2015. “Necessity Is the Mother of Invention: Input Supplies and Directed Technical Change.” Econometrica83 (1): 67–100.
Unconventional Fossil Fuels.” In Introduction to Energy Technology: Depletable and Renewable, edited by M.J.Aziz and A.C.Johnson. Wiley-VCH.“
2015. “The Tug-of-War between Resource Depletion and Technological Change in the Global Oil Industry, 1981–2009.” Energy93 (P2): 1607–16..
2004. “Technological Change and Depletion in Offshore Oil and Gas.” Journal of Environmental Economics and Management47 (2): 388–409..
2005. “Technological Change and Petroleum Exploration in the Gulf of Mexico.” Energy Policy36 (5): 619–32..
2006. “Stochastic Frontier Analysis of Total Factor Productivity in the Offshore Oil and Gas Industry.” Ecological Economics60 (1): 204–15..
2016. “An Energy-Centric Theory of Agglomeration.” Working Paper 22964, National Bureau of Economic Research, Cambridge, Massachusetts..
Nature Climate Change. 2013. “Fracking Fracas.” Editorial. Nature Climate Change3: 429.
2016. “Venezuela’s Growing Risk to the Oil Market.” Columbia University, Center on Global Energy Policy, August..
1978. “The Optimal Exploration and Production of Nonrenewable Resources.” Journal of Political Economy86 (5): 841–61..
2017. “Oil Projections in IEA World Energy Outlooks: Revisions, Accuracy and Uncertainty.” Uppsala University Working Paper. Uppsala, Sweden..
Prepared by Rabah Arezki (team lead), Claudia Berg, Christian Bogmans, Rachel Yuting Fan and Akito Matsumoto (team coleader), with research assistance from Clara Galeazzi, and Lama Kiyasseh. The authors thank Rystad Energy, and Per Magnus Nysveen in particular for very useful discussions and for kindly providing proprietary data on capital expenditures and cost structures.
The focus of this feature is on oil, which refers here to liquids including crude oil, condensate, and natural gas liquids.
See Kleinberg (forthcoming) for a discussion of unconventional sources.
In the exploration model developed by Pindyck (1978), a social planner maximizes the present value of the social net benefits from consumption of oil, and the reserve base can be replenished through exploration and discovery of new fields. Resource exploration and discovery has been investigated either as a deterministic or a stochastic process (see, for example, Pindyck 1978, Arrow and Chang 1982, and Devarajan and Fisher 1982).
Moreno-Cruz and Taylor (2016) propose a spatial model of energy exploitation that determines how the location and productivity of energy resources affect the distribution of economic activity across geographic space. They find that a novel scaling law links the productivity of energy resources to population size, whereas rivers and roads effectively magnify productivity. Arezki and Bogmans (2017) provide evidence for the role of proximity to major markets and state capacity in the production of fossil fuels.
See Nature Climate Change (2013) for a discussion of the pros and cons of fracking.
For the Gulf of Mexico, Managi and others (2004, 2005, 2006) use microlevel data from 1947–98 and find empirical support for the hypothesis that technological change has offset depletion for offshore oil and gas production. For the United States, Cuddington and Moss (2001) present evidence that technological improvements respond to instances of scarcity by analyzing the determinants of the average finding cost for additional petroleum reserves over the period 1967–90.
High oil prices also stimulate technological change in the energy-using sector. Aghion and others (2016) provide evidence that firms in the auto industry tend to innovate more in “clean” (and less in “dirty”) technologies when they face higher fuel prices. The current lower-for-longer oil price environment could, however, delay the energy transition by slowing technological change—and subsequent adoption—directed toward moving away from fossil fuel use (Arezki and Obstfeld 2015).
Investment and oil price series are deflated using a price index of private fixed investment in mining and oilfield machinery in the United States obtained from the Bureau of Economic Analysis website (www.bea.gov).
In 2016, shale oil added 7.9 million barrels a day (mbd) in a market of 96 mbd—4.4 mbd in crude oil, 2.7 mbd in natural gas liquids, and 0.8 mbd in condensate.
Figure 2.6 indicates that under a scenario of no cost deflation, the oil price level required to keep shale production constant is higher than $80 a barrel. With cost deflation of about 40 percent, about what it has been in the recent past, the required price level is only $40 a barrel. The recent rally in oil prices has been followed by signs of recovery in investment and production.
The International Energy Agency (IEA) does not provide specific forecasts for oil production by OPEC. Wachtmeister, Henke, and Höök (2017) present a detailed assessment of the production forecast prepared by the IEA using a narrower definition of unconventional oil sources. Leduc, Moran, and Vigfusson (2013) present evidence of the rather gradual learning in futures markets.