The Forecasting Accuracy of Crude Oil Futures Prices

The efficiency of the crude oil futures market and the forecasting accuracy of futures prices are investigated. The accuracy of forecasts using futures prices is compared with that of forecasts using alternative techniques, including time series and econometric models and judgmental forecasts. The predictive power of futures prices is further explored by comparing the forecasting accuracy of end–of–month prices with weekly and monthly averages, using different weighting schemes. Finally, the paper investigates whether forecasts using futures prices can be improved by incorporating information from other forecasting techniques. [JEL A10, C22, C52, E37]

Abstract

The efficiency of the crude oil futures market and the forecasting accuracy of futures prices are investigated. The accuracy of forecasts using futures prices is compared with that of forecasts using alternative techniques, including time series and econometric models and judgmental forecasts. The predictive power of futures prices is further explored by comparing the forecasting accuracy of end–of–month prices with weekly and monthly averages, using different weighting schemes. Finally, the paper investigates whether forecasts using futures prices can be improved by incorporating information from other forecasting techniques. [JEL A10, C22, C52, E37]

The efficiency of the crude oil futures market and the forecasting accuracy of futures prices are investigated. The accuracy of forecasts using futures prices is compared with that of forecasts using alternative techniques, including time series and econometric models and judgmental forecasts. The predictive power of futures prices is further explored by comparing the forecasting accuracy of end–of–month prices with weekly and monthly averages, using different weighting schemes. Finally, the paper investigates whether forecasts using futures prices can be improved by incorporating information from other forecasting techniques. [JEL A10, C22, C52, E37]

This paper investigates the efficiency and forecasting accuracy of crude oil futures prices. The efficiency of the market is analyzed in terms of the expected returns from trading in the futures contracts. The accuracy of futures price forecasts is analyzed by comparing it with the accuracy of forecasts obtained using a variety of other techniques including random walk and time-series models, as well as forecasts using judgmental and econometric techniques. The paper also explores the predictive power of futures prices by comparing the forecasting accuracy of end-of-month prices with weekly and monthly averages, using a variety of weighting schemes. Finally, the paper examines the improvement in forecasting accuracy when futures prices are combined with forecasts from alternative sources.

The empirical analysis in the paper is based on the New York Mercantile Exchange’s (NYMEX) futures contracts, which constitute by far the most active crude oil futures trading in the world. Trading in crude oil contracts started in April 1983; however, in the initial period the market was relatively limited. Since 1985 activity has increased at a very fast pace indeed. Although there is a growing futures market for Brent crude oil on the International Petroleum Exchange in London, until recently trading was limited, and contracts were available for only a few months ahead. On the NYMEX, in addition to crude oil, there are active futures markets in other energy products, including heating oil, gasoline, and natural gas. But the combined activity in these markets still falls short of the activity in crude oil futures.

The empirical exercise analyzes the behavior of futures prices from June 1985, by which time the crude oil futures market had become highly developed, to October 1990. Although during 1990–91 there was a sharp increase in the maturities available with contracts for up to nearly three years ahead, for most of the period under discussion trading was confined to a period of around nine months to one year, which is the range of maturities that the analysis will consider.

The discussion is organized as follows. Section I examines the emergence and growth of futures markets in crude oil trading during the last decade. Section II provides the empirical evidence on efficiency, and Sections III and IV provide evidence on forecasting accuracy, comparing the accuracy of futures forecasts with that of forecasts from a variety of alternative models. Section IV also analyzes the improvement in forecasting accuracy that may result from combining futures forecasts with those from other techniques. The final section summarizes the main results and notes their implications for forecasting oil prices for different forecast horizons.

I. Futures Market in Crude Oil

Futures markets serve two interrelated purposes. First, they provide an organized forum allowing agents to undertake hedging or speculation. Second, the price of futures contracts provides a summary or consensus view, based on market trading, of the participants’ expectations with regard to the future course of prices. For the market to perform either of these functions efficiently, there has to be sufficient activity in the volume of contracts traded on a regular basis.

Table 1.

Crude Oil Trading on the New York Mercantile Exchange

article image
Source: New York Mercantile Exchange.Note: Contracts are for 1,000 barrels; both the open interest and the volume of contracts traded can vary enormously from month to month.

End-July.

End-May.

Development of Futures Markets

Although there are a number of international forward and futures markets in crude oil, by far the largest is on the New York Mercantile Exchange (NYMEX). Since the introduction of oil futures contracts on NYMEX in April 1983, the volume of trade in oil contracts has grown extremely fast. In 1983 the average daily volume was around 1,200 contracts (with each contract representing 1,000 barrels of oil).1 (See Table 1 for detailed information on the size of the market.) By the end of 1990 the daily average exceeded 100,000 contracts, or 100 million barrels a day, making crude oil futures one of the most heavily traded of any futures contract. At the same time, the total number of contracts outstanding increased from about 5,000 contracts to nearly 280,000 contracts, equivalent to 280 million barrels. In comparison, total world oil production is about 65 million barrels a day. Furthermore, because the futures price is set in open trading, it is accepted as the benchmark from which almost all other prices for crude oil are calculated.

It is worth noting that there are futures contracts in oil products that predate the trading of crude oil contracts. For instance, gasoline futures were introduced on NYMEX in 1978, and heating oil futures were introduced in 1980 (see Hirschfeld (1983) for details). Crude oil futures trading was introduced on the NYMEX during 1978, but the contract was withdrawn because of insufficient volume of trade. One of the main reasons for the insufficient trading was the lack of adequate volatility in crude oil spot prices. As the discussion below shows, this low volatility in turn was due to the market structure, within which spot trading was very limited; most of the oil traded was on the basis of contractual arrangements between oil producers and international oil companies.

The significance of contractual arrangements may be judged from the fact that until the late 1970s almost 90 percent of the world’s oil was sold under long-term contracts based on prices set by the major oil producers, and the other 10 percent was bought and sold informally between the international oil companies. Beginning with the late 1970s, however, there was a fundamental change, as this historic system of long-term, fixed-price contracts negotiated between oil producers and the international oil companies came to be largely replaced with an open market system. By the early 1980s, 90 percent of the world’s oil was available on the spot market. The growing prominence of spot market transactions seems to have been accompanied by much greater short-term volatility in oil prices, especially in recent years (see, for instance, Anderson and others (1990)).

Thus, on a short-term basis (day-to-day, or week-to-week), crude oil prices during the 1980s were considerably more volatile than during the 1970s, when prices were stable—often for months at a time. The incidence of sharp and abrupt price swings, however, does not seem to have been intensified by the growing prominence of the market. If anything, it could be argued that the market, by adjusting prices continuously to the supply-demand imbalances, may have lessened the extent of the abrupt changes (see, for example, Hampton (1991)).

The antecedents of the change from contractual to spot price transactions lay in two major developments during the 1970s: (1) the world’s major oil producers exercised greater control over their oil fields, allowing them to sell their oil to whoever offered the best deal; and (2) major oil companies that previously had owned the oil fields were cut loose to bid for crude oil wherever it was available.

At the same time, the very high real interest rates of the early 1980s, which made the storage of oil expensive, encouraged the formation of the oil futures market. The emergence of this market meant that it was much cheaper for a user to acquire claims on oil by buying a contract that assured delivery when needed than it was to purchase and store actual oil. As with almost all futures trading, actual delivery is made in only a small proportion of contracts; instead, the contracts are usually bought and sold over and over again, often many times in the same day, as financial instruments that provide a form of price and supply insurance to producers and users.

Futures Contracts

Futures trading began in 1983 with delivery for a period of up to 6 months ahead. Initially, even this period seemed likely to easily satisfy the needs of traders, and most of the transactions were concentrated in the first 2 or 3 months. Fairly soon, however, trading in distant months became equally active, and demand grew for longer maturities. Trading was then extended to 9 months and, by 1989, to 12 months. Beyond 9 months, however, trading volume remained limited. Since July 1990 contracts have been extended almost continuously; first, to 15 months, then 18 months, and by April 1991, contracts were available for up to 3 years ahead. However, these most recent lengthenings of maturities cannot yet be used to compare forecasting accuracy. Also, since most active trading in the earlier period was limited to a period up to 9 months ahead, this paper focuses on trading up to these maturities.

A critical factor affecting the accuracy of futures price forecasts is the way the prices are actually determined. To ensure that the determination of prices is completely transparent, some specific rules are used to reach the “settlement price” for the crude oil futures contracts. As discussed in the Appendix, the settlement price is a weighted average of the transactions prices toward the end of the trading session. This is the price that is used in the following analysis of the forecasting accuracy of futures prices (see the Appendix for details).

The contracts are specified for a specific crude—namely, West Texas Intermediate (WTI) to be delivered at a specific point—but the futures market rules allow for delivery of six other grades against the WTI contract.2 NYMEX adds an adjustment factor to the WTI price. Buyers taking delivery of alternative grades may appear to he concerned about getting unexpected or lower-grade oil, but for most substitute crudes, they are actually indifferent. That is because the marginal benefit to the refiner of using these grades generally exceeds their additional cost and the NYMEX adjustment factor.3 There is, in any case, a strong and systematic relationship between the West Texas price and prices of other crudes, so one can infer forecasts of prices for other crudes from the West Texas price. (For details, see Kumar (1991, Appendix II).)

As noted earlier, futures prices frequently set the benchmark for the pricing of other crudes in the spot market. It is also worth noting that while futures prices are not entirely unconstrained in the sense that there are exchange-determined upper and lower limits to price changes in any given day that trading takes place, the constraints are seldom if ever binding. Even when they are binding, the decisions taken usually reflect almost entirely the market conditions (see Appendix).

II. Efficiency of Oil Futures Markets

The first empirical issue to be analyzed is whether there is any bias in crude oil futures prices; that is, do these futures prices tend to exhibit any systematic behavior, which could mean that, ex ante, one could obtain positive returns from speculating in the futures market. If futures prices are to be useful in forecasting future spot prices, the expectation would be that there is no such bias (see, for example, Kaminsky and Kumar (1990a, 1990b)). At any given time, there may be a number of factors that might lead to an expectation of an increase in the future spot price; equally, at other times, there could be factors promoting expectations of a decline. There are a number of techniques for examining this efficiency, but the most rigorous and transparent is to examine the returns from holding futures contracts up to a given maturity.

Table 2 shows the mean excess returns from holding a futures contract and the corresponding t-statistic for the test of the null hypothesis of unbiasedness. The excess return is computed as the change in the contract’s price between two different dates ranging from one to nine months, and covers the eight-year period 1983–90. Although mean excess return is positive for the first five forecast horizons, it is not significantly different from zero over any of the forecast horizons. (The tests of significance use standard errors corrected for autocorrelation using the “method of moments.”) These results suggest, at least superficially, that the null hypothesis of a zero bias in the oil futures prices cannot be rejected. However, the evidence can be consistent with the presence of a time-varying bias. Since there is evidence from other commodity markets that a time-varying bias exists, it is useful to check whether there is such a bias in the oil futures market (see Kaminsky and Kumar (1990a, 1990b).

Table 2.

Test of Unconditional Unbiasedness: Full Sample

article image
Note: The t-statistics use standard errors of means corrected for autocorrelation (using “method of moments”). The i subscript refers to the forecast horizon (i = 1 to 9). The sample is from September 1983 to October 1990.

The procedure used for isolating such bias is to divide the sample into subperiods according to whether the crude oil spot price was increasing or falling. The rationale for such a procedure follows from the results of a number of studies into the behavior of other commodity, and asset, futures prices. These results indicate that, in general, investors consistently underpredict the price of an asset when the asset is appreciating (for example, the U.S. dollar in the early 1980s) and systematically overpredict it when it is depreciating (as was the case after 1985, when the dollar started to depreciate). As Figure 1 shows, the spot oil price was increasing from January 1986 to January 1987 as well as from April 1988 to December 1989. The observations from these two subperiods were pooled as were the observations from a period of declining spot prices. The results of computing the excess returns for the two periods of increasing and declining prices are given in Table 3. The realized excess returns during the period of increasing prices were significantly positive for the first four forecast horizons; for the other forecast horizons the results were not significant. For the period of decreasing prices there were again no significant excess returns for any of the maturities. As has been noted in earlier studies, this evidence, weak and in general statistically insignificant, does not necessarily imply market failure or that the futures are necessarily biased predictors of future spot prices.

Figure 1.
Figure 1.

Crude Oil: Spot Prices and Excess Returns

(September 1984-September 1990)

Citation: IMF Staff Papers 1992, 002; 10.5089/9781451947106.024.A008

Table 3.

Test of Unconditional Unbiasedness: Increasing and Decreasing Spot Prices

article image
Note: The t-statistics use standard errors of means corrected for autocorrelation (using “method of moments”).

III. Futures Price Forecasts

The accuracy of forecasts obtained from using futures prices is measured in relation to the spot price prevailing on the day of the maturity of any given contract. Forecasts using futures prices are generated using prices of contracts for delivery up to ten months ahead. Since oil contracts are terminated in the month preceding the contract month, forecasts up to nine months ahead are thereby obtained. More precisely, trading for any contract is terminated on the third business day, prior to the twenty- fifth calendar day of the month, preceding the delivery month. The reason for this apparently peculiar timing is that for delivery, the pipeline space must be reserved by the twenty-fifth of the month.4

In an initial comparison, futures prices on the last trading day of the month are used, under the premise that the latest price would incorporate the latest information and thereby provide the most accurate and up- to-date forecast. In a subsequent exercise, however, prices prevailing at different times during the month, weighted according to specified criteria, are also examined. Accuracy of forecasts using futures prices is investigated in relation to forecasts obtained from a number of different techniques. These include forecasts using the random walk model, time-series models, judgmental methods, and econometric models.

End-of-Month Prices and Random Walk

Consider, first, a comparison of the forecasts from end-of-month futures prices with the forecasts from a random walk model. The latter can be regarded as postulating that spot prices at period t are the best unbiased predictor of spot prices at any future period, T. 5 The rationale for this hypothesis is that a commodity such as crude oil is subject, on a day-to-day basis, to a large number of influences on its demand and supply, and on its price. Hence, at any given time, the spot price itself reflects the consensus view of the current market situation and provides the best guidance on the future course of prices.

Table 4 gives the results of a comparison of the accuracy of futures prices with that of the random walk model. Forecasts are examined for a period ranging from one to nine months. Forecast error is defined as follows:

Table 4.

Comparison of Futures Prices and Random Walk

article image
Note: Forecasts are for the period December 1985 to October 1990.
Pt,TjST,(1)

where Pt,Tj is the forecast made at time I for a future spot price at time T by using the weighting scheme, j (discussed below); and ST is the realized spot price prevailing at time T. For the random walk model

Pt,Tj=St,

St is the actual spot price at time t; and T – t denotes the forecast horizon, one to nine months.

Three different criteria for comparing forecast accuracy are utilized. The first is the mean absolute error (MAE), which is the absolute value of the deviation of the predicted value from the realized value. The second and main criterion is the root mean square error (RMSE), which attaches a higher weight to larger absolute errors. The third is Theil’s U-statistic, which adjusts for trend changes.

The results in Table 4 tabulate the forecast errors from using these three forecast accuracy criteria for one to four months, and six and nine months, respectively. These results suggest two main conclusions: first, for virtually all forecast horizons, and for the different accuracy criteria, futures prices provide more accurate forecasts than the random walk model; and second, as the length of the forecast horizon increases, the accuracy of the forecasts, whether using futures prices or the random walk model, diminishes markedly. In general, however, as Figures 2, 3, and 4 also emphasize, futures prices provide a fairly accurate forecast of future spot prices for a forecast horizon of up to six months.

Figure 2.
Figure 2.

Crude Oil: Spot Prices and One-Month-Ahead Forecasts

(September 1984–November 1990)

Citation: IMF Staff Papers 1992, 002; 10.5089/9781451947106.024.A008

Note: For explanation of monthly average and 1/ei, see text below.
Figure 3.
Figure 3.

Crude Oil: Spot Prices and Three-Month-Ahead Forecasts

(September 1984–November 1990)

Citation: IMF Staff Papers 1992, 002; 10.5089/9781451947106.024.A008

Figure 4.
Figure 4.

Crude Oil: Spot Prices and Six-Month-Ahead Forecasts

(September 1984—November 1990)

Citation: IMF Staff Papers 1992, 002; 10.5089/9781451947106.024.A008

To the extent that the random walk model may be regarded as “naive,” a next step would be to examine whether one can improve on the accuracy of futures prices by using other more sophisticated but readily available econometric or time-series models. Before considering these alternatives, it is worth enquiring whether on an a priori basis it is appropriate to use anything but the latest price. One hypothesis would be that if the market is efficient, and the results discussed in Section II above suggest that it is, then it must be the case that the latest price embodies all the relevant information and provides the best possible prediction of the future spot price. Hence, combining the latest price with other (preceding) prices would not lead to any increase, and may actually lead to a diminution, in the forecast accuracy. An alternative hypothesis would be that if there is a time-varying risk premia with a transitory and a permanent component, then an averaging procedure may dampen, or even cancel out, the temporary component. Such a procedure may, therefore, improve on the end-of-period prices. It might also be argued that to the extent that there are speculative bubbles in the market, some sort of averaging procedure might improve the forecasts. In this regard, the results of a number of recent studies into the efficiency of different asset markets are quite illuminating. Most of these studies suggest that even in markets that are regarded as approximately efficient, there may be evidence of speculative bubbles. For instance, Lo and Mackinlay (1988) found a high degree of autocorrelation in weekly stock returns, while Fama and French (1988) and Poterba and Summers (1988) found similar results for returns spanning one year or more.

Predetermined Weighting Schemes

Given the above considerations, it is appropriate to see whether using prices prevailing during the month, rather than only at the end of the month, provides any incremental improvement in forecast accuracy of futures prices. The problem then is essentially an empirical one of choosing the optimal weighting scheme for combining or averaging different intramonthly prices. The case of the end-of-month price would be simply a special case where the weight on intramonth prices is equal to zero. The end-of-month price is compared with three other schemes in which weights are also predetermined: (1) last 5 days price, which is a simple average of the futures (settlement) price on the last 5 trading days of any given month; (2) monthly average, which is a simple average of the futures price for each of the trading days in any given month; since the number of trading days in the month can vary from 20 to 23 days, the average is based on closing prices on the last 20 trading days in the month; and (3) exponentially declining weights, whereby the weights for the 20 trading days are computed according to the inverse of the exponential function; that is, the weights, Wt, are defined as follows: Wi=1/ei,i=1,2,...,n. Here, 1 refers to the latest observation that is given the greatest weight, 2 refers to the last but one observation, and so on.

In addition to the above three, essentially ad hoc schemes and the end-of-month price, a scheme whereby weights were determined endogenously to minimize the forecast error (using a maximum likelihood technique) was also applied. (This methodology and its results will be discussed presently.)

Consider first the results for the above four schemes, given in Table 5. For the one-month-ahead forecast, using the end-of-month price yields the smallest forecasting error. According to the RMSE criterion, the monthly average gives forecast errors that are about 15 percent higher than the end-of-month price and the exponential scheme. It is notable, however, that monthly average forecast errors are still smaller than the errors generated by the random walk model (last column). As the forecast horizon increases, the difference in the forecast accuracy of the end-of-month and exponential scheme vis-à-vis the monthly average declines monotonically, so that for the six-month-ahead forecasts there is virtually no difference in the errors generated by the different schemes. As results in Kumar (1991, Appendix Table 2) indicate, this remains the case up to the nine-month forecast when the criterion is the MAE; however, when RMSE is the criterion, the monthly average has a negligible edge over the other schemes. It is also worth noting that regardless of the weighting scheme, the forecast errors using the random walk model are always larger than those using futures prices. Furthermore, all forecast errors increase monotonically with the increase in the forecast horizon. (See Figure 5 for a comparison of the actual spot prices and forecasts for nine-month horizons, respectively.)

Table 5.

Comparison of Alternative Weighting Schemes for Futures Prices

article image
Note: Forecasts are for the period December 1985 to October 1990. Criterion for accuracy is the root mean square forecast error.
Figure 5.
Figure 5.

Crude Oil: Spot Prices and Nine-Month-Ahead Forecasts

(September 1984—November 1990)

Citation: IMF Staff Papers 1992, 002; 10.5089/9781451947106.024.A008

Endogenous Weighting Scheme

The above four schemes apply predetermined weights to futures prices. An interesting extension is to see if the weighting scheme could be determined endogenously by the data, and whether that would reduce the out-of-sample prediction errors. For this purpose, the following equation was fitted to the data on spot and futures prices:

logST=α+Σi=1kλilogPt, T+ϵt.(2)

Here, as before, ST denotes the realized spot price at time T, and Pt, T denotes the futures price at time t for the contract maturing at T; a and λi are the estimated parameters. The equation was estimated separately for each of the nine forecast horizons. Given the nonlinearity, the estimation procedure used a standard maximum likelihood function (MINDIS). Initially, the equation was estimated using daily observations; that is, K = 20, and ST was regressed on 20 variables constituting the prices at the end of each of the trading days in any given month.6 However, because of considerable collinearity between the daily observations (for almost all the forecast horizons), it was not possible to achieve convergence. Therefore, estimation was undertaken using prices at the end of every fifth day (using data from 1986 to 1988). Using the estimates of λi, normalized weights (summing to 1) were computed for each of the forecast horizons. These weights were then used to compute the out-of-sample forecasts and forecast errors for the period November 1988 to October 1990. In order to compare these forecasts with those obtained from using extraneous weights, forecast errors were recomputed for this period for the end-of-month, monthly average, and the exponentially declining weights.

Results of the above exercise are provided in Table 6 and Figure 6. Consider first Figure 6, which compares the weighting scheme for the different forecast horizons as generated by the estimation and the extraneous weights for the other schemes. (These other weights are, of course, invariant across forecast horizons.) As the figure shows, the pattern of weights is related to the length of the forecast horizon. The weights decline gradually with lag length when the horizon is distant, but for a nearby horizon the decline is very steep. For instance, according to the estimation, with a one-month-ahead forecast, the greatest weight should be placed on the latest futures price. But with a nine-months-ahead forecast, the average should be relatively more equally weighted. This result is fairly plausible and can be explained in terms of the deficiency of lagged prices with respect to the information they contain. This deficiency is more serious relative to a short forecast than to a long forecast; that is, for the short forecast, the information (or the consensus) embodied in the latest price is likely to be much more important than the information in the lagged prices and, hence, should be given the greatest weight. For the distant forecast, given the relatively greater uncertainty, somewhat greater weight can be given to the lagged prices. Some support for the above result, and the explanation, can be found in a study recently undertaken by Feinstein (1989). In that study, a volatility indicator for the stock market was constructed, by using a weighted average of current and lagged implied volatilities from call options. Feinstein reached similar conclusions about the relationship between weights and the forecast horizon.

Table 6.

Comparison of MINDIS-Weighted Forecasts

article image
Note: MINDIS k a maximum likelihood algorithm for nonlinear estimation, Estimation period was December 1985 to October 1988, and the forecasts were generated for November 1988 to October 1990.
Figure 6.
Figure 6.

Weighting Schemes

Citation: IMF Staff Papers 1992, 002; 10.5089/9781451947106.024.A008

a Using MINDIS for one, three, six, and nine months.

Next to be considered is how the forecast errors using these endogenously determined weights compare with the errors generated by the other weighting schemes. The new weights were used to construct forecasts for the period November 1988 to October 1990. Given that these forecasts are for the out-of-sample period. it would not necessarily be the case that the forecast errors would be smaller than the other weighting schemes. This is borne out to some extent by the results in Table 6. Compared to the simple monthly average (scheme 2 above), the forecast errors using the new weights are generally smaller. However, comparing the last-day forecast with the newly weighted forecast shows that, apart from the first and the third month, the former has marginally smaller errors.

IV. Time Series and Econometric and Judgmental Forecasts

This section discusses the relative accuracy of futures prices compared to forecasts obtained from time-series and structural econometric models, as well as forecasts obtained using judgmental methods. A number of time-series models were developed specifically for this study, and forecasts using these models are discussed below; the other two types of forecasts were obtained from existing sources. Despite a plethora of forecasts for crude oil price for the medium and long term (5 to 20 years), the absence of systematic short-term forecasts is quite remarkable. The forecasts examined appear to be the best available. In the case of both econometric models (given the inevitable “add-on” factors or adjustments) and judgmental forecasts, it is very difficult to ensure that, during the process of forecasting. information provided in the futures prices was not used. An effort was made to select forecasts based on the “structural” factors, which were less likely to be influenced by the information embodied in the futures prices. However, in the case of both judgmental and econometric forecasts, the frequency at which forecasts have been made has often been limited. This and a number of other issues are examined in detail below.

Time-Series Models

Consider, first, a comparison of the forecast accuracy of futures prices with the forecasts generated from time-series models. A major advantage of this technique is that the data requirements are very limited—only the data for the actual variable being modeled are needed, which in this case is simply the spot price. The main problem is to ensure that the forecasts obtained from this method correspond precisely to the forecasts implicit in the futures prices. This necessitates estimating models in such a way so that forecasts can be obtained for the day of the maturity of the contract. This requirement, in turn, necessitates estimation of the models using daily spot price data. Apart from the considerations relating to the time period, the key issue is the specification of the model. In order to provide a rigorous test of the accuracy of futures prices, the time- series model has to be very carefully specified. For this purpose, the estimation was initially undertaken using the most general form of the time-series generating process—that is, the autoregressive integrated moving-average-process (ARIMA). The process is defined by the equation

Φ(β)(1β)dqt=θ0+θ(β)at,(3)

where Φ(β) and ɵ(β) are operators in β of degree p and q, respectively, and the roots of Φ(β) = 0 and Φ(β) = 0 lie outside the unit circle; atis a white-noise process. [E(at)=0andvar(at)=σ2]. The process is thus of order (p, d, q). This model is very general, subsuming autoregressive, moving average, and mixed autoregressive moving-average models, and the integrated forms of all three. (See, for instance. Box and Jenkins (1976, pp. 85–91) and Granger and Newbold (1986. pp. 25–28).

The preliminary analysis strongly suggested that over the sample period, a more specific autoregressive moving-average-process (ARMA)—that is. d = 0—was appropriate, implying stationarity in the spot price series. In such a case, the roots of the polynomials lie within the unit circle. A number of models were estimated using this more restrictive specification. (The ARMA model was preferred because it achieved as good a fit as an AR model, using fewer parameters.) The best fit was given by a first-order autoregressive process and a moving average process of order two (see, for instance. Box and Jenkins (1976, pp. 85–91) and Granger and Newbold (1986, pp. 25–28)); that is, the equation is of the form

P¯t=Φ1P¯t1+atθ1at1θ2at2,(4)
Table 7.

Comparison of Accuracy of ARMA Model and Futures Prices

article image
Note: The autoregressive moving average (ARMA) model was estimated using daily spot price data, starting with January 2, 1986 to end-December 1986, to get the first set of forecasts for one to nine months ahead. The model was then re-estimated for February 1, 1986 to end-January 1987 for the next set of forecasts, and so on; MAE is mean absolute error; RMSE is root mean square error.

or more generally

Φ(β)Pt¯=ɵ(β)at,(5)

where Pt¯ is now the deviation of Pt from its mean.

The results of forecasts obtained using this model, and the comparable forecasts using futures prices, are shown in Table 7, using the absolute mean error and the root mean square error for forecast horizons from one to nine months. As noted earlier, the increase in forecast error as the length of forecast horizon increases from one to six months is quite marked; between six and nine months, however, there is either no increase or even a marginal improvement. Compared to the futures prices, however, regardless of the forecast horizon, the ARMA model does less well, although the difference is marked only for the shortest forecasts horizon.

Judgmental Forecasts and Econometric Models

Next, consider the forecasts obtained using judgmental methods or econometric models.7 The objective is again to compare the accuracy of forecasts using these alternative techniques with that of forecasts obtained from futures prices. Clearly, the comparison can be important, given that futures prices are readily available, whereas the alternative techniques can require considerable investments, necessitating following developments in the oil industry, model building, and so on. In theory, such a comparison should not pose any problem: given the ubiquitous nature of oil price forecasting models and forecasts, one could simply take these forecasts and compare them with those generated by futures prices. More formally, denote by Pt,Tj the price forecast obtained at time i for T. using the forecasting technique, j;Pt,Tf, the price of the futures contract at time t for maturity at T; and ST, the spot price at T. Then, the comparison would simply be between

(Pt,TjST)and(Pt,TfST),(6)

using any one of the forecasting accuracy criteria.

In practice, however, unlike the time-series model estimated above, a number of important constraints arise when this comparison is attempted.

The first constraint is that in the alternative forecasts, T refers not to any specific day, as in the futures forecast, but to an average of a lengthy period, such as a year, or at best, a quarter. (Forecasts on a quarterly basis are available only for a very short length of time.) For the above comparison to be undertaken, therefore, it is necessary to transform Pt,Tf, so that it is comparable to Pt,Tj. One, and perhaps the only, procedure that could be used is to obtain an average of prices for contracts maturing in the year for which the forecast is made. Thus, one would have a sequence of maturities. T – 1, T – 2,…, T – 11—where T now refers to the end of the calendar year for which the alternative forecast is made, and 1,2 refers to the specific months. Even where this can be done, given that futures contracts were only available from 1983 onwards, it would mean that a maximum of only eight observations could be available for the analysis.

However, a second constraint then arises because the forecasts are often made several months prior to the beginning of the calendar year to which they apply. This means that the forecasting horizon, T – t, is usually in excess of 15 months. Given that until September 1990 the most distant contracts were only for a period up to 14 months, it would not be possible to have the contracts that mature throughout the year. Indeed, for the period before 1987, when contracts were available for even shorter periods, it may be possible to use only the first few months of the year for which the forecast is made.8

It is also often quite difficult to ascertain precisely when the alternative forcasts were made; that is, whereas for futures forecasts time t is transparent, it can be highly uncertain for the alternative forecasts. Indeed for the latter, the time at which the forecast is made can often only be guessed at, with a considerable margin of error.

Finally, it is important that the alternative forecasts, especially the judgmental ones, are not influenced by the path of futures prices themselves. In practice, of course, it is impossible to ensure this; all that can be done is to ascertain the extent to which a formal model, or formal analysis, is used in making the forecasts.

Despite the above factors, alternative forecasts were obtained from three main sources: annual forecasts published by the Petroleum Economist, the United States Energy Administration (USEA), and the World Bank.9 Analysis was undertaken separately, using each of the three sets of forecasts, for the period 1984 to 1991. In general the forecasting errors using futures prices were smaller, although not markedly so, compared to the forecasts from the alternative techniques. An examination of the individual forecasts also showed that on an annual basis, for the majority of years the futures prices were more accurate. In view of the considerations stated earlier, these results should be treated with caution, but they do suggest that where futures prices are available, they can provide forecasts that are no worse, and are often better than those obtained from judgmental or econometric methods.

Combined Forecasts

A final exercise was undertaken to examine whether the forecasts obtained from using futures prices can be improved on, in terms of the variance of the forecast errors, by combining them with forecasts obtained from alternative sources. A considerable theoretical literature, and empirical evidence, shows that in many practical situations combined forecasts can outperform individual forecasts in terms of error variance.10 Following Granger and Newbold (1986), this reduction in error variance can be seen readily. Let fn, and tn be the futures and time-series forecasts of the spot price Sn at time n, and effandent and be the two forecast errors, respectively; that is

enf=Snfn(7a)

and

ent=Sntn.(7b)

The combined forecast is Cn=Kfn+(1K)tn, where K and (1 – K) denote the weights; the combined forecast error, enc, is

enc=SnCn=Kenf+(1K)ent,(8)

and the error variance is

σc2=K2σf2+(1K)2σt2+2K(1K)ρσfσt,(9)

where ρ is the correlation between two forecast errors.

It can be seen readily that σc2<min(σf2,σt2), unless p is either exactly equal to σf/σtortoσt/σf. If either equality holds, then the variance of the combined forecast is equal to the smaller of the two error variances.

The main issue here is the choice of weights to be applied to the different forecasts. Using the forecasts from the time-series model, the following weighting scheme was applied:11

Kn=t=nνn1et(2)2t=nνn1(et(1)2+et(2)2),(10)

where enf=Xnfn(j),j=1,2;fn(j) are the two forecasts of X, from futures and time-series models, respectively; and enj are the two forecast errors; n denotes the number of forecasts, and Knis the estimate of the weight to he applied at time n; ν was varied between 1 and 3 to obtain optimal weights that were constrained to lie between zero and unity. The results of the exercise showed only a marginal improvement in terms of reduction in error variance for forecast horizons ranging from six to nine months.12 These results can he interpreted in the context of the close correspondence between the two sets of forecasts (futures and time series) shown earlier, since, as was emphasized, the combination is most likely to result in significant improvement only when individual forecasts are dissimilar in nature.

V. Summary and Conclusions

This paper has analyzed the forecasting accuracy of crude oil futures prices by comparing these forecasts with those obtained from a variety of other techniques. It emphasized the increasing depth and breadth of the oil futures markets and the role that these factors were likely to play in price discovery. The empirical analysis focused on the behavior of futures prices from the inception of the market to end-1990—the most extensive sample data ever used to examine these issues. The following key results were obtained.

  • There did not appear to be any systematic bias in crude oil futures prices. This was shown by a detailed analysis of the average excess returns, which could be obtained from holding futures contracts for different lengths of time.

  • A comparison of end-of-month forecasts obtained from using futures prices with forecasts from a random walk model showed that the former provided more accurate forecasts for all forecast horizons. However, as the length of the forecasting horizon increased, the accuracy of forecasts, whether using futures or the random walk model, diminished markedly.

  • An analysis of intramonth futures prices showed some marginal improvement in forecasting accuracy for forecasts beyond six months, compared to end-of-month prices.

  • When weights are determined endogenously, the weighting scheme appears to be related to the length of the forecast horizon; the weights decline gradually with lag length when the horizon is distant, but for the nearby forecasts the decline is very steep.

  • A number of time-series models were estimated and the forecasts from them were compared with the futures prices. In general, the modelbased forecasts had larger errors compared to the forecasts using futures prices. A similar result was obtained when comparing judgmental and econometric forecasts.

  • Combining forecasts from the time-series models and the futures’ prices yielded only a marginal improvement in term of variance of forecast errors.

The above results clearly suggest that crude oil prices provide forecasts that are, on average, superior to those obtained from alternative techniques for short-term horizons. For more distant horizons, their accuracy does diminish markedly; however, even for these distant horizons the futures forecasts are no worse, and are often better, than those obtained from alternative techniques.

APPENDIX Rules for Pricing of Crude Oil Futures Contracts

This appendix outlines the procedures for determining the settlement price (SP) for crude oil futures contracts on the New York Mercantile Exchange (NYMEX). The SP is a daily price at which the clearing house clears all trades and settles all accounts between clearing members for each contract month. Since the SP is used to determine both the margin calls and invoice prices for deliveries, there are some very precise rules for its determination.13 By the same token, it is the best guide to the market’s views on the future course of prices.

There are two sets of rules, contingent on the volume of trade, for determining the SP. One set of rules applies if at the opening of business on any trading day, a given delivery month has more than 10 percent of the total open interest for all delivery months of the futures contracts.14 The second set of rules applies if the volume criterion is not met. These two sets of rules are considered below.

When the Volume Criterion Is Met

The SP is the weighted average of the transactions prices during the closing range; this range is defined as the last five minutes of trading before the end of the trading session. The weights are given by the number of contracts traded. For instance, suppose for January 1992 delivery, during the trading range, n1 contracts are traded at price pt, and n2 contracts, for p2. The settlement price would then be equal to (p1n1+p2n2)/N, where N=n1+n2.

The reason for having this procedure is that in the so-called open cry system of trading in the futures markets, at any given time there would be a range of prices at which transactions would be occurring. To call any one of those prices the “settlement” price would thus be quite arbitrary. The procedure adopted ensures that a set of representative prices is taken; by taking only the last few minutes of trading, the objective is to have the prices reflect the latest information available to the market.

If there are no transactions in the closing range, the SP is the last trade price, unless a bid higher or offer lower than the last trade price is made in the closing range. Such a higher bid or lower offer is then called the SP.

When the Volume Criterion Is Not Met

For these delivery months, the SP is the price relationship between any given delivery month and the current delivery month. The price relationship itself is based on the last “spread transaction” executed in the closing range between such months. Spread transaction is a trade involving the simultaneous purchase of one futures contract against the sale of another futures contract.15 For instance, on February 1, 1991. a trader may sell March 1991 contract, the current delivery month, and buy March 1992 contract. The difference in the prices of these two contracts would thus determine the “price relationship” and the SP for March 1992.

If there is no such spread transaction in the closing range, the relationship would be established by the last such spread transaction executed that day, unless a bid higher or offer lower than the last transaction is made in the closing range, in which case the last bid or offer for such spread is the SP.

If there are no spread transactions and no bids or offers made during any particular trading day, the spread differential for that day is taken to be the spread differential of the settlement prices for the preceding business day.

In addition to the above two sets of rules, there is a provision in the rules that allows the Settlement Price Committee to establish the SP under specific circumstances.16 There are essentially two such circumstances: (1) if the SP, determined according to either set of rules, is inconsistent with transactions that occurred during the closing range in other delivery months; or (2) if the SP is inconsistent with market information known to the Committee. In either of these two circumstances, the Committee may establish the SP at a level consistent with other transactions or market information. In such an event, the Committee is required to prepare a written record of the basis for any SP so established.

It appears that during the lead up to and after the outbreak of hostilities in the Middle East in late 1990 and early 1991, the Committee had to intervene a number of times to set the SP, especially for some of the distant months for which trade volume was very limited. It should be noted that the Committee may determine the SP for one month, with the above two rules determining the SP for the following month, and the Committee again determining the SP for the month following that. Given that the decisions of the Committee have serious financial implications for traders and other users of the futures market, the settlement prices so determined invariably reflect a consensus view, not only of the Committee members but also of the major traders.

REFERENCES

  • Anderson, G. H., M. F. Bryan, and C. Pike, “Oil and the Economy,” Consensus, Vol. XX, No. 49 (1990).

  • Bopp, Anthony E., and Scott Sitzer, “Are Petroleum Futures Prices Good Predictors of Cash Value,” The Journal of Futures Markets, Vol. 7, No. 6 (1987).

    • Search Google Scholar
    • Export Citation
  • Box, George E. P., and Gwilym M. Jenkins, Time-Series Analysis: Forecasting and Control, rev. ed. (San Francisco: Holden-Day, 1976).

  • Cho, Dong and Gerald S. McDougall, “The Supply of Storage in Energy Futures Markets,” The Journal of Futures Markets, Vol. 10 (December 1990), pp. 61121.

    • Search Google Scholar
    • Export Citation
  • Choe, B., “Rational Expectations and Commodity Price Forecasts,” PRE, WPS 435 (Washington: World Bank, June, 1990).

  • Clemen, Robert T., “Combining Forecasts: A Review and Annotated Bibliography,” International Journal of Forecasting, Vol. 5, No. 4 (1989), pp. 55983.

    • Search Google Scholar
    • Export Citation
  • Dominguez, K. M., “The Volatility and Efficiency of Crude-Oil Futures Contracts,” in Oil and Money: Coping with Price Risk Through Financial Markets, Harvard International Energy Studies (Cambridge, Massachusetts: Harvard University, 1987).

    • Search Google Scholar
    • Export Citation
  • Fama, Eugene F., and Kenneth R. French, “Permanent and Temporary Components of Stock Prices,” Journal of Political Economy, Vol. 98 (April 1988), pp. 24674.

    • Search Google Scholar
    • Export Citation
  • Feinstein, S. P., “Forecasting Stock Market Volatility Using Options on Index Futures,” Economic Review, Federal Reserve Bank of Atlanta (May 1989), pp. 1230.

    • Search Google Scholar
    • Export Citation
  • Granger, C. W. J., and Paul Newbold, Forecasting Economic Time Series, 2nd ed. (Orlando, Florida: Academic Press, 1986).

  • Hampton, Michael, “Cycling Towards Low Prices,” Petroleum Economist, Vol. 58 (April 1991), pp. 911.

  • Hirschfeld, D. J., “A Fundamental Overview of the Energy Futures Market,” The Journal of Futures Markets, Vol. 3, No. 1 (1983).

  • Kaminsky, Graciela, and Manmohan S. Kumar (1990a), “Efficiency in Commodity Futures Markets,” Staff Papers, International Monetary Fund, Vol. 37 (September), pp. 67099.

    • Search Google Scholar
    • Export Citation
  • Kaminsky, Graciela, and Manmohan S. Kumar (1990b), “Risk Premia in Commodity Futures Markets,” IMF Working Paper 90/116 (Washington: International Monetary Fund, December).

    • Search Google Scholar
    • Export Citation
  • Kumar, Manmohan S., “Forecasting Accuracy of Crude Oil Futures Prices,” IMF Working Paper 91/93 (Washington: International Monetary Fund, October 1991).

    • Search Google Scholar
    • Export Citation
  • Lo, Andrew W., and A. Craig Mackinlay, “Stock Market Prices Do Not Follow Random Walks: Evidence From A Simple Specification Test,” Review of Financial Studies, Vol. 11 (1988), pp. 4166.

    • Search Google Scholar
    • Export Citation
  • McNees, Stephen K., “The Role of Judgement in Macroeconomic Forecasting Accuracy,” International Journal of Forecasting, Vol. 6 (October 1990), pp. 28799.

    • Search Google Scholar
    • Export Citation
  • Ma, Cindy W., “Forecasting Efficiency of Energy Futures Prices,” The Journal of Futures Markets, Vol. 9 (October 1989), pp. 393419.

    • Search Google Scholar
    • Export Citation
  • Petroleum Intelligence Weekly, Petroleum Database Services (June 1988).

  • Porteba, James M., and Lawrence H. Summers, “Mean Reversion in Stock Prices: Evidence and Implications,” Journal of Financial Economics, Vol. 22 (October 1988), pp. 2759.

    • Search Google Scholar
    • Export Citation
  • Serletis, Apostolos, “Rational Expectations, Risk and Efficiency in Energy Futures Markets,” Energy Economics, Vol. 13 (April 1991), pp. 11115.

    • Search Google Scholar
    • Export Citation
*

Manmohan S, Kumar, an Economist in the Research Department, is a graduate of the London School of Economics and Political Science. He received his Ph.D. from Cambridge University, where he also taught before joining the IMF.

The author is grateful to Bijan Aghevli, Ernesto Hernandez-Cats, Peter Wickham, Charles Adams, William Perraudin, and Blair Rourke for helpful comments, to Tony Curtis of the CFTC for discussions on the oil futures contracts, and to Raja Hettiarachchi for computing assistance.

1

Each barrel includes roughly 42 gallons of oil.

2

The delivery point is the town of Cushing, in Oklahoma. The six other deliverable grades include two Algerian grades, two Nigerian grades, a Norwegian grade, and UK Brent Blend.

3

See, for instance. Petroleum Intelligence Weekly (1988). According to data provided by Petroleum Database Services, which has individual computer models of all U.S. refineries, the extra profit from five of the six deliverable crude, relative to WTI, exceeded the NYMEX value adjustment by 15 to 7(1 cents a barrel during the second quarter of 1988. North Sea Brent Blend, the most readily available substitute crude, was the only grade that appeared unattractive to buyers at Cushing, since it showed little or no extra profit compared to WTI, but the NYMEX adjustment method still penalized Brent with a slight premium.

4

It is worth noting that the delivery date was changed in 1985 when it was based on the fifth day prior to the twenty-fifth calendar day. An earlier study by Ma (1989) for the period 1984–86 apparently used the same delivery date. Given the extreme sensitivity of prices near the maturity date, the difference of even a couple of days can be important. For a somewhat different methodology. see, for instance, Dominguez (1987); see also Bopp and Sitzer (1987).

5

A related model is a random walk with drift; there was, however, no empirical support for the drift factor.

6

As noted above, given the variable number of trading days, the last 20 days in the month were utilized.

7

In the tests below, given the “adjustments” applied to forecasts obtained from econometric models, no attempt was made to separate the econometric and the judgmental forecasts. See, for instance, McNees (1990).

8

These two factors are particularly relevant when considering comparisons made, for instance, by Choe (1990).

9

The forecasts by the USEA are often in terms of constant dollars—they were converted into nominal dollars using the expected inflation rate.

10

For a succinct summary of this literature, see Granger and Newbold (1986, pp. 266–276) and Clemen (1989).

11

For a rationale of this type of weighting scheme, see Granger and Newbold (1986, p. 269).

12

Thus, for instance, for the six-month-ahead forecast, the error variance using futures and time-series models was 0.03577 and 0.04267, respectively (using ν = 1). The error variance of the combined forecast was 0.03447.

13

These rules are set out in the NYMEX Rule Guide. The rules for energy contracts (for crude oil, gasoline, as well as fuel oil) are given by Rule 6.52. The rules are set by the Exchange and approved by the Commodity Futures Trading Commission (CFTC). I am particularly grateful to Tony Curtis of the CFTC for his advice in interpreting these rules.

14

Open interest is defined as the total number of futures contracts, long or short, that have been entered into and not yet liquidated by an offsetting transaction or fulfilled by delivery. The term is interchangeable with “open contracts” and “open commitments.”

15

There are a number of different types of spread transactions: the intramarket spread—consisting of buying one month and selling another month in the same commodity: the intercommodity spread—consisting of a long position in one commodity and a short position in a related commodity: and the intramarket spread—consisting of buying a commodity at one exchange and selling the same commodity at another exchange. For the determination of the crude oil SP, the first of these spread transactions is relevant.

16

This committee consists of three members, including a floor trader, a floor broker, and an oil market expert.

IMF Staff papers: Volume 39 No. 2
Author: International Monetary Fund. Research Dept.
  • View in gallery

    Crude Oil: Spot Prices and Excess Returns

    (September 1984-September 1990)

  • View in gallery

    Crude Oil: Spot Prices and One-Month-Ahead Forecasts

    (September 1984–November 1990)

  • View in gallery

    Crude Oil: Spot Prices and Three-Month-Ahead Forecasts

    (September 1984–November 1990)

  • View in gallery

    Crude Oil: Spot Prices and Six-Month-Ahead Forecasts

    (September 1984—November 1990)

  • View in gallery

    Crude Oil: Spot Prices and Nine-Month-Ahead Forecasts

    (September 1984—November 1990)

  • View in gallery

    Weighting Schemes