Forecasting Accuracy of Crude Oil Futures Prices

This paper undertakes an investigation into the efficiency of the crude oil futures market and the forecasting accuracy of futures prices. Efficiency of the market is analysed in terms of the expected excess returns to speculation in the futures market. Accuracy of futures prices is compared with that of forecasts using alternative techniques, including time series and econometric models, as well as judgemental forecasts. The paper also explores the predictive power of futures prices by comparing the forecasting accuracy of end-of-month prices with weekly and monthly averages, using a variety of different weighting schemes. Finally, the paper investigates whether the forecasts from using futures prices can be improved by incorporating information from other forecasting techniques.

Abstract

This paper undertakes an investigation into the efficiency of the crude oil futures market and the forecasting accuracy of futures prices. Efficiency of the market is analysed in terms of the expected excess returns to speculation in the futures market. Accuracy of futures prices is compared with that of forecasts using alternative techniques, including time series and econometric models, as well as judgemental forecasts. The paper also explores the predictive power of futures prices by comparing the forecasting accuracy of end-of-month prices with weekly and monthly averages, using a variety of different weighting schemes. Finally, the paper investigates whether the forecasts from using futures prices can be improved by incorporating information from other forecasting techniques.

I. Introduction

This paper undertakes an investigation into the efficiency and forecasting accuracy of crude oil futures prices. Efficiency of the market is analyzed in terms of the expected returns from trading in the futures contracts. Accuracy of futures price forecasts is analyzed by comparing it with the accuracy of forecasts obtained using a variety of other techniques including random walk and time series models, as well as forecasts using judgmental and econometric techniques. The paper also explores the predictive power of futures prices by comparing the forecasting accuracy of end-of-month prices with weekly and monthly averages, using a variety of weighting schemes. Finally, the paper examines the improvement in forecasting accuracy when futures prices are combined with forecasts from alternative sources.

The empirical analysis in the paper is based on the New York Mercantile Exchange’s (NYMEX) futures contracts, which constitute by far the most active crude oil futures trading in the world. Trading in crude oil contracts started in April 1983; however, in the initial period, the market was relatively limited. Since 1985, activity has increased at a very fast pace indeed. While there is a growing futures market for Brent crude oil at the International Petroleum Exchange in London, until recently trading on this was limited, and contracts were available for only a few months ahead. On the NYMEX, in addition to crude oil, there are active futures markets in other energy products, including heating oil, gasoline, and natural gas. But the combined activity in these markets still falls short of the activity in crude oil futures.

The empirical exercise analyzes the behavior of futures prices from June 1985, by which time the crude oil futures market had become highly developed, to October 1990. The analysis considers maturities ranging from one month to nine months. Although during the last year there has been a sharp increase in the maturities available with contracts for up to nearly three years ahead, for most of the period under discussion the trading was confined to a period of around nine months to one year.

The discussion in the paper is organized as follows: Section II examines the emergence and growth of futures markets in crude oil trading during the last decade. Section III provides the empirical evidence on efficiency, while Sections IV and V provide evidence on forecasting accuracy, which compares the accuracy of futures forecasts with that of forecasts from a variety of alternative models; Section V also analyzes the improvement in forecasting accuracy which may result from combining futures forecasts with those from other techniques. A last section summarizes the main results and notes their implications for forecasting oil prices for different forecast horizons.

II. Futures Market in Crude Oil

1. Development of futures market

Futures markets serve two interrelated purposes: the first is a provision of an organized forum allowing agents to undertake hedging, or speculation. Their second purpose is that related to price discovery; the price of futures contracts provides a summary or consensus view, based on market trading, of the participants’ expectations with regard to the future course of prices. For the market to perform either of these functions efficiently, there has to be sufficient activity in terms of the volume of contracts traded on a regular basis.

While there are a number of international forward and futures markets in crude oil, by far the largest futures market is on the New York Mercantile Exchange (NYMEX). Since the introduction of the oil futures contracts on NYMEX in April 1983, volume of trade in oil contracts has grown almost exponentially. In 1983, the average daily volume was around 1,200 contracts, (with each contract representing 1,000 barrels of oil). 1/ By the end of 1990, the daily average exceeded 100,000 contracts, or 100 million barrels a day, making crude oil futures one of the most heavily traded of any futures contract. At the same time, the total number of contracts outstanding has increased from around 5,000 contracts to nearly 280,000 contracts, equivalent to 280 million barrels. In comparison, total world oil production is about 65 million barrels a day. Furthermore, because the futures price is set in open trading, it is accepted as the benchmark from which almost all other prices for crude oil are calculated.

It is worth noting that there are futures contracts in oil products which predate the trading of crude oil contracts. For instance, gasoline futures were introduced on NYMEX in 1978 while heating oil futures were introduced in 1980. 2/ During 1978 crude oil futures trading was begun on the NYMEX, but because of insufficient volume of trade the contract was withdrawn. One of the main reasons given for insufficient trading was the lack of adequate volatility in crude oil spot prices. As the discussion below shows, the low volatility in turn was due to the market structure whereby spot trading was very limited; most of the oil traded was on the basis of contractual arrangements between oil producers and international oil companies.

The significance of contractual arrangements can be judged from the fact that until the late 1970s, almost 90 percent of the world’s oil was sold under long-term contracts based on prices set by the major oil producers, and the other 10 percent was bought and sold informally between the international oil companies. Beginning with the late 1970s, however, there was a fundamental change, with this historic system of long-term, fixed price contracts negotiated between oil producers and the international oil companies being largely replaced with an open market system. By the early 1980s, 90 percent of the world’s oil was available on the spot market. The move to the preponderance of spot market transactions seems to have been accompanied by considerably greater short-term volatility in oil prices in the recent years. 3/

Thus, on a short-term basis (day-to-day, or week-to-week), crude oil prices during the 1980s were considerably more volatile than during the 1970s, when the prices were stable often for months at a time. The incidence of sharp abrupt swings in prices, however, does not seem to have been increased any marked way by the move to greater market role. If anything, it could be argued that by adjusting prices continuously to the supply-demand imbalances, the extent of the abrupt changes may be lessened. 4/

The antecedents of the change from contractual to spot price transactions lay in two major developments during the 1970s: (i) the world’s major oil producers exercised greater control over their oil fields, allowing them to sell their oil to whomever offered the best deal; (ii) major oil companies that previously had owned the oil fields were cut loose to bid for crude oil wherever it was available.

At the same time, the very high real interest rates of the early 1980s, which made the storage of oil an expensive activity, encouraged the formation of the oil futures market. The emergence of this market meant that it was much cheaper for a user to acquire claims on oil by buying a contract that assured delivery when needed than it was to purchase and store actual oil. As with almost all futures trading, actual delivery is made in only a small proportion of contracts: instead the contracts usually are bought and sold over and over again, often many times in the same day, as financial instruments that provide a form of price and supply insurance to users and producers.

2. Futures contracts

The futures trading began in 1983 with delivery for a period of up to six months ahead. Initially, even this period seemed likely to easily satisfy the needs of traders, and most of the transactions were concentrated in the first two or three months. Fairly soon, however, trading in distant months became equally active and there was demand for longer maturities. Trading was then extended to nine months and by 1989 to twelve months. Beyond nine months, however, trading volume remained limited. Since July 1990, the contracts have been extended almost continuously; first, they were extended to 15 months, then to 18 months, and by April 1991, contracts were available for up to three years ahead. For the purposes of comparing forecasting accuracy, the most recent lengthening of maturities could not be utilized. For the earlier periods also, most active trading was limited to a period up to nine months ahead, as the paper focuses on trading at these maturities.

A critical factor affecting the accuracy of the futures prices’ forecast is the way the prices are actually determined. To ensure that the determination of prices is completely transparent, there are some very specific rules for arriving at the “settlement price” for the crude oil futures contracts. As discussed in Appendix I, the “settlement price” is a weighted average of the transactions prices, towards the end of the trading session. This is the price used in the following analysis of the futures prices’ forecasting accuracy (see Appendix I for details).

The contracts are specified for a specific crude--namely, the “West Texas Intermediate” (WTI) to be delivered at a specific point--but the futures market rules allow for delivery of six other grades against the WTI contract. 5/ An adjustment factor is added by NYMEX to the WTI price. Buyers taking delivery of alternative grades may appear to be concerned about getting unexpected or less-valued oil. But for most substitute crudes, the buyers are actually indifferent. That is because the marginal benefit to the refiner of utilizing these grades generally exceeds their additional cost and the NYMEX adjustment factor. 6/ There is, in any case, a strong and systematic relationship between the West Texas price and prices of other crudes. So that from the West Texas price, one can infer forecasts of prices for other crudes (see Appendix II for a detailed analysis of this issue).

In any case, as noted earlier, the futures prices frequently set the benchmark for the pricing of other crudes in the spot market. It is also worth noting that while the futures prices are not entirely unconstrained in the sense that there are exchange-determined upper and lower limits to price changes in any given day, the constraints are seldom if ever binding. Even when they are binding, the decisions taken usually reflect almost entirely the market conditions (see Appendix I).

III. Efficiency of Oil Futures Markets

The first empirical issue to be analyzed is whether there is any bias in crude oil futures prices, that is, do these futures prices tend to exhibit any systematic behavior, which could mean that ex ante one could obtain positive returns from speculating in the futures market. If futures prices are to be useful in forecasting future spot prices, the expectation would be that there is no such bias. At any given time, there may be a number of factors which might lead to an expectation of an increase in the future spot price; equally, at other times, there could be factors leading to an expectation of a decline. So that on average, ex ante, there should be no systematic bias (see, for example, Kaminsky and Kumar (1990 and 1991)). There are a number of techniques for examining this efficiency, but the most rigorous and transparent is to examine the returns from holding futures contracts up to a given maturity.

Table 1 shows the mean excess returns from holding a futures contract and the corresponding t-statistic for the test of the null hypothesis of unbiasedness. The excess return is computed as the change in the contract’s price between two different dates ranging from one to nine months, and covers the eight-year period 1983-90. Although mean excess return is positive for the first five forecast horizons, it is not significantly different from zero over any of the forecast horizons. (The tests of significance use standard errors corrected for autocorrelation using the “method of moments”). These results suggest, at least superficially, that the null hypothesis of a zero bias in the oil futures prices cannot be rejected. However, the evidence can be consistent with the presence of a time varying bias. Since there is evidence from other commodity markets that a time varying bias exists, it is useful to check whether there is such a bias in the oil futures market (see Kaminsky and Kumar (op.cit.)).

Table 1.

Test of Unconditional Unbiasedness: Full Sample

article image
Note: The t-statistics used standard errors of means corrected for autocorrelation (using “method of moments”). The i subscript refers to the forecast horizon (i=1 to 9). The sample is from September 1983 to October 1990.

The procedure used for isolating such bias is to divide the sample into sub-periods according to whether the crude oil spot price was increasing or falling. The rationale for such a procedure follows from the results of a number of studies into the behavior of other commodity, and asset, futures prices. These results indicate that, in general, investors consistently underpredict the price of an asset when the asset is appreciating (for example, the U.S. dollar in the early 1980s) and systematically overpredict it when it is depreciating (as was the case after 1985, when the dollar started to depreciate). As Chart 1 shows, the spot oil price was increasing from January 1986 to January 1987 as well as from April 1988 to December 1989. The observations from these two sub-periods were pooled as were the observations from a period of declining spot prices. The results of computing the excess returns for the two periods of increasing and declining prices are given in Table 2. The realized excess returns during the period of increasing prices were significantly positive for the first four forecast horizons; for the other forecast horizons, the results were not significant. For the period of decreasing prices, there were again no significant excess returns for any of the maturities. As has been noted in earlier studies, this evidence, weak and in general statistically insignificant, does not necessarily imply market failure or that the futures are necessarily biased predictors of future spot prices.

Chart 1.
Chart 1.

Crude Oil: Spot Prices and Excess Returns

(September 1984 - September 1990)

Citation: IMF Working Papers 1991, 093; 10.5089/9781451951110.001.A001

Table 2.

Test of Unconditional Unbiasedness: Increasing and Decreasing Spot Prices

article image
Note: The t-statisctics use standard errors of means corrected for autocorrelation (using “method of moments”).

IV. Futures Price Forecasts

The accuracy of forecasts obtained from using futures prices is measured in relation to the spot price prevailing on the day of the maturity of any given contract. Forecasts using futures prices are generated using prices of contracts for delivery up to ten months ahead. Since oil contracts are terminated in the month preceding the contract month, forecasts up to nine months ahead are thereby obtained. More precisely, trading for any contract is terminated on the third business day, prior to the twenty-fifth calendar day of the month, preceding the delivery month. The reason for this apparently peculiar timing is that for delivery, the pipeline space must be reserved by the 25th of the month. 7/

1. End-of-month prices and random walk

In the initial exercise, futures prices on the last trading day of the month are utilized, under the premise that the latest price would incorporate the latest information and thereby provide the most accurate and up-to-date forecast. In a subsequent exercise, however, prices prevailing at different times during the month, weighted according to specified criteria, are also examined. Accuracy of forecasts using futures prices is investigated in relation to forecasts obtained from a number of different techniques. These include forecasts using the random-walk model, time-series models, judgmental methods, and econometric models.

Consider first a comparison of the forecasts from end-of-month futures prices with the forecasts from a ‘random walk’ model. The latter can be regarded as postulating that spot prices at period t are the best unbiased predictor of spot prices at any future period T. 8/ The rationale for this hypothesis is that a commodity such as crude oil is subject, on a day-to-day basis, to a large number of influences on its demand and supply, and on its price. Hence, at any given time, the spot price itself reflects the consensus view as to the current market situation and provides the best guidance as to the future course of prices.

Table 3 provides the results of a comparison of the accuracy of the futures prices, with that of the random walk model. Forecasts are examined for a period ranging from one to nine months. Forecast error is defined as follows:

Pt,TjST(1)
Table 3.

Comparison of Futures Prices and Random Walk

article image
Note: Forecasts are for the period December 1985 to October 1990.

where Pt,Tj is the forecast made at time t for a future spot price at time T by using the weighting scheme j (discussed below). Thus in the case of the futures forecast, the price at maturity is taken; for the random walk model

Pt,Tj=St

ST is the realized spot price at time T; T-t denotes the forecast horizon, 1 to 9 months.

Three different criteria for comparing forecast accuracy are utilized. The first is the mean absolute errors (MAE). This is the absolute value of the deviation of the predicted value from the realized value. Secondly, and the main one, is the root mean square error, (RMSE), which attaches a higher weight to larger absolute errors. Thirdly, Theil’s “U” statistic which, in addition, adjusts for trend changes, is utilized.

The results in Table 3 tabulate the forecast errors from using these three forecast accuracy criteria for one to four months, and six and nine months, respectively. These results suggest two main conclusions: the first is that for virtually all forecast horizons, and for the different accuracy criteria, futures prices provide more accurate forecasts than the random walk model. The second conclusion is that as the length of the forecast horizon increases, the accuracy of the forecasts, whether using futures prices or the random walk model, diminishes markedly. In general, however, as Charts 2 to 4 also emphasize, futures prices provide a fairly accurate forecast of future spot prices for up to a six-month forecast horizon.

Chart 2.
Chart 2.

Crude Oil: Spot prices and one month ahead Forecasts 1/

September 1984 - November 1990

Citation: IMF Working Papers 1991, 093; 10.5089/9781451951110.001.A001

1/ Price (logorithms, dollars/barrel).
Chart 3.
Chart 3.

Crude Oil: Spot prices and three months ahead Forecasts 1/

September 1984 - November 1990

Citation: IMF Working Papers 1991, 093; 10.5089/9781451951110.001.A001

1/ Price (logarithms, dollars/barrel).
Chart 4.
Chart 4.

Crude Oil: Spot prices and six months ahead Forecasts 1/

September 1984 - November 1990

Citation: IMF Working Papers 1991, 093; 10.5089/9781451951110.001.A001

1/ Price (logarithms, dollars/barrel).

To the extent that the random walk model may be regarded as “naive”, an obvious next step would be to examine whether one can improve upon the accuracy of the futures prices by using other more sophisticated but readily available econometric or time-series models. Before considering these alternatives, it is worth enquiring whether on an a priori basis, it is appropriate to use anything but the latest price. One hypothesis would be that if the market is efficient, and the results discussed in Section III above suggest that it is, then it must be the case that the latest price embodies all the relevant information and provides the best possible prediction of the future spot price. Hence combining the latest price with other (preceding) prices would not lead to any increase, and may actually lead to a decline, in the forecast accuracy. An alternative hypothesis would be that if there is a time varying risk premia with a transitory and a permanent component, then an averaging procedure may dampen, or even cancel out, the temporary component. Such a procedure may, therefore, improve on the end-of-period prices. It might also be argued that to the extent that there are speculative bubbles in the market some sort of averaging procedure might improve the forecasts. In this regard the results of a number of recent studies into the efficiency of different asset markets are quite illuminating. Most of these studies suggest that even in markets that are regarded as approximately efficient, there may be evidence of speculative bubbles. For instance, Lo and Mackinley (1988) find a high degree of autocorrelation in weekly stock returns, while Fama and French (1988) and Poterba and Summers (1989) found similar results for returns spanning one year or more.

2. Predetermined weighting schemes

Given the above considerations, it may appear worthwhile to examine whether using prices prevailing during the month provide any incremental improvement in forecast accuracy of futures prices. The problem then is essentially an empirical one of choosing the optimal weighting scheme for combining different prices. The case of the end-of-month price would be simply a special case where the weight on intra-month prices is equal to zero. The end-of-month price is compared with three other schemes in which weights are also predetermined:

(i) Last five days’ price: this is a simple average of the futures’ (settlement) price on the last five trading days of any given month.

(ii) Monthly average: this is a simple average of the futures’ price for each of the trading days in any given month. Since the number of trading days in the month can vary from 20 to 23 days, the average is based on closing prices on the last 20 trading days in the month.

(iii) Exponentially declining weights: The weights for the 20 trading days are computed according to the inverse of the exponential function; that is, the weights, Wi, are defined as follows: Wi = 1/ei, i=1,2…n. Here 1 refers to the latest observation which is given the greatest weight, 2 refers to the last but one observation and so on.

In addition to the above four schemes, a scheme whereby weights were determined endogenously to minimize the forecast error (by using a maximum likelihood function) was also applied. (This methodology and its results are discussed presently.)

Consider first the results for the above four schemes, given in Table 4. For the one month ahead forecast, using end-of-the-month price yields the smallest forecasting error. According to the RMSE criterion, the monthly average gives forecast errors which are around 15 percent higher than the end-of-the-month price and the exponential scheme. It is worth noting, however, that monthly average forecast errors are still smaller than the errors generated by the random walk model (last column). As the forecast horizon increases, the difference in the forecast accuracy of the end-of-the month and exponential scheme vis-à-vis the monthly average declines monotonically, so that for the six-month ahead forecasts there is virtually no difference in the errors generated by the different schemes. As results in Appendix Table 2 indicate, this remains the case up to nine months forecast when the criterion is the MAE; however, when RMSE is the criterion, the monthly average has a negligible edge over the other schemes. It is also worth noting that regardless of the weighting scheme, the forecast errors using the random walk model are always larger than those using futures prices. Furthermore all forecast errors increase monotonically with the increase in the forecast horizon. (See Chart 5 for a comparison of the actual spot prices and forecasts for 9 month horizons respectively.)

Table 4.

Comparison of Alternative Weighting Schemes for Futures Prices

article image
Note: Forecasts are for the period December 1985 to October 1990. Criteria for accuracy is the “Root Mean Squared Forecast Error.”
Chart 5.
Chart 5.

Crude Oil: Spot prices and nine months ahead Forecasts 1/

September 1984 - November 1990

Citation: IMF Working Papers 1991, 093; 10.5089/9781451951110.001.A001

1/ Price (logarithms, dollars/barrel).

3. Endogenous weighting scheme

The above four schemes apply predetermined weights to futures prices. An interesting extension would be to see if the weighting scheme could be determined endogenously by the data, and whether that reduces the out-of-sample prediction errors. This was examined by fitting the following equation to the data on spot and futures’ prices:

logST=α+Σi=1kλilogPt,T+εt(2)

Here ST denotes the realized spot price at time T and Pt,T denotes the futures price at time t for the contract maturing at T; α and λi are the estimated parameters. The equation was estimated separately for each of the nine forecast horizons. Given the non-linearity, the estimation procedure used a standard maximum likelihood function (MINDIS). Initially the equation was estimated using daily observations; that is, K = 20 and ST was regressed on 20 variables constituting the prices at the end of each of the trading days in any given month. 9/ However, because of considerable collinearity between the daily observations, (for almost all the forecast horizons) it was not possible to achieve convergence. Therefore, estimation was undertaken using prices at the end of every fifth day (using data from 1986 to 1988). Using the estimates of λi, normalized weights (summing to 1) were computed for each of the forecast horizons. These weights were then used to compute the out-of-sample forecasts and forecast errors for the period November 1988 to October 1990. In order to compare these forecasts with those obtained from using extraneous weights, forecast errors were recomputed for this period for the end-of-the-month, monthly average and the exponentially declining weights.

Results of the above exercise are provided in Table 5 and Chart 6. Consider first Chart 6, which compares the weighting scheme for the different forecast horizons as generated by the estimation, and the extraneous weights for the other schemes. (These other weights are, of course, invariant across forecast horizons.) As the chart shows, the pattern of weights is related to the length of the forecast horizon. The weights decline gradually with lag length when the horizon is distant, but for a nearby horizon the decline is very steep. For instance, when undertaking one month ahead forecast, according to the estimation, greatest weight should be placed on the latest futures price. But in undertaking a nine-month ahead forecast the average should be relatively more equally weighted. This result is fairly plausible and can be explained in a number of ways. One explanation may be in terms of the deficiency of lagged prices with respect to the information they contain. This deficiency is more serious relative to a short forecast than to a long forecast. That is, for the short forecast, the information (or the consensus) embodied in the latest price is likely to be much more important than the information in the lagged prices, and hence should be given the greatest weight. For the distant forecast, given the relatively greater uncertainty, somewhat greater weight can be given to the lagged prices. Some support for the above result, and the explanation, is found in a rather different study recently undertaken by Feinstein (1989). In that study, a volatility indicator for the stock market was constructed, by using weighted average of current and lagged implied volatilities from call options. That study reached very similar conclusions as to the relationship between weights and the forecast horizon.

Table 5.

Comparison of “MINDIS” Weighted Forecasts

article image
Note: “MINDIS” is a maximum likelyhood Algorithim for non-linear estimation. Estimation period was December 1985 to October 1988 and the forecasts were generated for November 1988 to October 1990.
Chart 6:
Chart 6:

WEIGHTING SCHEMES

Citation: IMF Working Papers 1991, 093; 10.5089/9781451951110.001.A001

Next consider how the forecast errors using these endogenously determined weights compare with the errors generated by the other weighting schemes. The new weights were used to construct forecasts for the period November 1988 to October 1990. Given that these forecasts are for the out-of- sample period, it would not necessarily be the case that the forecast errors would be smaller than the other weighting schemes. This is borne out to some extent by the results in Table 5. Compared to the simple monthly average (Scheme II above), the forecast errors using the new weights are generally smaller. However, comparing the last-day forecast with the new weights forecast, shows that apart from the first and the third month, the former has marginally smaller errors.

V. Time Series, and Econometric and Judgmental Forecasts

This section discusses the relative accuracy of futures prices as compared to forecasts obtained from time-series and structural econometric models, as well as forecasts obtained using judgmental methods. A number of time series models were developed specifically for this study and forecasts using these models are discussed below; the other two types of forecasts were obtained from existing sources. Despite a plethora of forecasts for the crude oil price for the medium and long term (five to twenty years), the absence of systematic short-term forecasts is quite remarkable. The forecasts examined appear to be the best available. Both in the case of econometric models (given the inevitable “add-on” factors or adjustments), and judgmental forecasts, it is very difficult to ensure that, during the process of forecasting, information provided in the futures prices was not used. An effort was, however, made to select forecasts based on the “structural” factors and which were less likely to be influenced by the information embodied in the futures prices. However, in the case of both judgmental and econometric forecasts, the frequency at which forecasts have been made has often been limited. This, and a number of other issues, are examined in detail below.

1. Time-series models

Consider first a comparison of the accuracy of futures’ price forecasts with the forecasts generated from time-series models. A major advantage of this technique is, of course, that the data requirements are very limited--only the data for the actual variable being modelled is needed, which in this case is simply the spot price. The main problem which arises is, however, to ensure that the forecasts obtained from this method correspond precisely to the forecasts implicit in the futures prices. This necessitates estimating models in such a way so that forecasts can be obtained for the day of the maturity of the contract. This in turn necessitates estimation of the models using daily spot price data. Apart from the considerations relating to the time period, the key issue is the specification of the model. In order to provide a rigorous test of futures’ price accuracy, it is obviously critical that the time-series model should have the best specification possible. For this purpose, the estimation was initially undertaken using the most general form of the time series generating process--that is, the autoregressive integrated moving average process (ARIMA). The process is defined by the equation:

φ(β)(1β)dqt=θ0+θ(β)at(3)

where φ(β) and θ(β) are operators in β of degree p and q, respectively, and the roots of φ(β)=0 and φ(β)=0 lie outside the unit circle. at is white noise process [E(at)=0 and var (at)=σ2]. The process is thus of order (p, d, q). This model is very general, subsuming autoregressive, moving average, and mixed auto-regressive-moving average models, and the integrated forms of all three. 10/

The preliminary analysis strongly suggested, however, that over the sample period, a more specific autoregressive moving average process (ARMA), that is, d = 0, was appropriate implying stationarity in the spot price series.) In such a case the roots of the polynomials lie within the unit circle. A number of models were estimated using this more restrictive specification. (The reasons for preferring the ARMA model was that it achieved as good a fit as an AR model but using fewer parameters.) The best fit was given by a first order autoregressive process and a moving average process of order two. 11/ That is, the equation is of the form

P¯t=φ1P¯t1+atθ1at1θ2at2(4)

or more generally

φ(B)P¯t=θ(B)at(5)

where P¯t is now the deviation of Pt from its mean.

The results of forecasts obtained using this model, and the comparable forecasts using futures prices, are shown in Table 6 using the absolute mean error the root mean square errors for forecast horizons from one to nine months. As noted earlier, the increase in forecast error of the model as the length of forecast horizon increases from one to six months is quite marked; between six and nine months however, there is either no increase or even a marginal improvement Compared to the futures prices, however regardless of the forecast horizon, the ARMA model does less well, although it is only for the shortest forecast horizon that the difference is marked.

Table 6.

Comparison of ARMA Model and Futures Prices Accuracy

article image
Notes: The ARMA model was estimated using daily spot price data, starting with 2nd January 1986 to end-December 1986, to get the first set of forecasts for 1 to 9 months ahead. The model was then reestimated for 1st February 1986 to end-January 1987 for the next set of forecasts, and so on.

2. Judgmental forecasts and econometric models

Next consider the forecasts obtained using judgmental methods or econometric models. 12/ The objective is again to compare the accuracy of forecasts using these alternative techniques with that of forecasts obtained from futures prices. Clearly the comparison can be important given that the futures prices are readily available while the alternative techniques can require considerable investments in model building, following developments in the oil industry, etc.. In theory, such a comparison should not pose any problems: given the ubiquitous nature of oil price forecasting models and forecasts, one would simply take these forecasts and compare them with the futures prices. More formally, denote by

Pt,Tj the price forecast obtained at time t for T, using the forecasting technique j; and

Pt,Tf the price of the futures contract at time t, for maturity at T; and ST the spot price at T.

Then the comparison would simply be between

(Pt,TjST)and(pt,TfST)(6)

using any one of the forecasting accuracy criteria.

In practice, however, unlike the time-series model estimated above, there are a number of important constraints which arise in making this comparison.

The first of these constraints arises because in the alternative forecasts T refers not to any specific day, as in the futures forecast, but to a specific period such as a year, or at best a quarter. (Unfortunately forecasts on a quarterly basis are available only for a very short period.) For the above comparison to be undertaken, therefore, it is necessary to transform Pt,Tf so that it is comparable to Pt,Tj. One, and perhaps the only procedure which could be used for this is to obtain an average of prices for contacts maturing in the year for which the forecast is made. Thus one would have a sequence of maturities, T-1, T-2, … T-11--where T now refers to the end of the calendar year for which the alternative forecast is made. Even where this can be done, given that futures contracts were only available from 1983 onwards, it would mean that a maximum of only eight observations could be available for the analysis.

However, a second constraint then arises from the fact that the forecasts are often made several months prior to the beginning of the calendar year to which they apply. This means that the forecasting horizon T-t is usually in excess of fifteen months or so. Given that until September 1990, the most distant contracts were only for a period up to fourteen months, it would not be possible to have the contracts which mature throughout the year. Indeed for the period before 1987, when contracts were available for even shorter periods, it may be possible to have only the first few months of the year for which the forecast is made. 13/

Often, it is also quite difficult to ascertain precisely when the alternative forecasts were made; that is, while for futures, the time t is transparent, it can be highly uncertain for the alternative forecasts. Indeed for the latter, the time at which the forecast is made can often only be guessed at, with a considerable margin of error.

Finally, it is important that the alternative forecasts, especially the judgmental ones are not influenced by the path of futures prices themselves. In practice, of course, it is impossible to ensure this; all that can be done is to ascertain the extent to which a formal model, or formal analysis, is utilized in making the forecasts.

In view of the above considerations, forecasts were obtained from three main sources: annual forecasts published by the Petroleum Economist, the United States Energy Administration, and the World Bank. 14/ Analysis was undertaken, separately, using each of the three sets of forecasts, for the period 1984 to 1991. In general, the forecasting errors using futures prices were smaller, although not markedly so, compared to the forecasts from the alternative techniques. An examination of the individual forecasts also showed that on an annual basis, for the majority of years, the futures prices were more accurate. However, in view of the earlier considerations, these results should be treated with caution. But they do suggest that where futures prices are available, they can provide forecasts which are no worse, and are often better, compared to those obtained from judgmental or econometric methods.

3. Combined forecasts

A final exercise was undertaken to examine whether the forecasts obtained from using futures prices can be improved on, in terms of the variance of the forecast errors, by combining them with forecasts obtained from alternative sources. There is a considerable theoretical literature, and empirical evidence, to show that in many practical situations combined forecasts can outperform individual forecasts in terms of error variance. 15/ Following Granger and Newbold (1986) this reduction in error variance can be seen readily: Let fn and tn be the futures and time series forecasts of the spot price Sn at time n, and enf and ent be the two forecast errors, respectively. That is,

enf=Snfn

and

ent=Sntn.(7)

The combined forecast is Cn = Kfn + (1-K)tn, where K and (1-K) denote the weights, and the combined forecast error enc is

enc=SnCn=Kenf+(1K)ent(8)

and the error variance is

σc2=K2σf2+(1K)2σt2+2K(1K)ρσfσt(9)

where ρ is the correlation between two forecast errors.

It can be seen readily that σc2<min(σf2,σt2) unless either ρ is exactly equal to σft or to σtf. If either equality holds, then the variance of the combined forecast is equal to the smaller of the two error variances.

The main issue here is the choice of weights to be applied to the different forecasts. Using the forecasts from the time series model, the following weighting scheme was applied: 16/

Kn=Σt=nvn1et(2)2Σt=nvn1(et(1)2+et(2)2)(10)

where enj=xnfn(j),j=1,2

fn(j) are the two forecasts of Xn from futures and time series models, respectively, and enj are the two forecast errors; n denotes the number of forecasts and Kn is the estimate of the weight to be applied at time n; V was varied between 1 and 3 to obtain optimal weights which were constrained to lie between zero and unity. The results of the exercise showed only a marginal improvement in terms of reduction in error variance for forecasts horizons ranging from six to nine months. 17/ These results can be interpreted in the context of the earlier findings of the close correspondence between the two sets of forecasts (futures and time series), since, as has been emphasized, the combination is most likely to result in significant improvement only when individual forecasts are very dissimilar in nature.

VI. Summary and Conclusions

This paper has analyzed the forecasting accuracy of crude oil futures prices by comparing these forecasts with those obtained from a variety of other techniques. It emphasized the increasing depth and breadth of the oil futures markets and the role that these factors were likely to play in price discovery. The empirical analysis focused on the behavior of the futures prices from the inception of the market to end-1990, thus using the most extensive sample data ever utilized to examine these issues. The following key results were obtained:

  • (1) There did not appear to be any systematic bias in crude oil futures prices. This was shown by an analysis of the mean excess returns which could be obtained from holding of futures contracts.

  • (2) A comparison of end-of month forecasts obtained from using futures prices with forecasts from a random walk model showed that the former provided more accurate forecasts for all forecast horizons. However, as the length of the forecasting horizon increased, the accuracy of forecasts, whether using futures or the random walk model, diminished markedly.

  • (3) An analysis of intra-month futures prices showed some marginal improvement in forecasting accuracy compared to end-of-the month prices.

  • (4) When weights are determined endogenously, it appears that the weighting scheme is related to the length of the forecast horizon; the weights decline gradually with lag length when the horizon is distant, but for the nearby forecasts the decline is very steep.

  • (5) A number of time series models were estimated and the forecasts from them compared with the futures prices. In general the model-based forecasts have larger errors compared to the forecasts using futures prices. A similar result was obtained when comparing judgmental and econometric forecasts.

  • (6) Combining forecasts from the time series models and the futures’ prices yielded only a marginal improvement in term of variance of forecast errors.

The above results clearly suggest that crude oil prices provide forecasts which are, on average, superior to those obtained from alternative techniques for short-term horizons. For more distant horizons, their accuracy does diminish markedly; however, even for these distant horizons the futures forecasts are no worse, and are often better, than those obtained from alternative techniques.

Forecasting Accuracy of Crude Oil Futures Prices
Author: Mr. Manmohan S. Kumar
  • View in gallery

    Crude Oil: Spot Prices and Excess Returns

    (September 1984 - September 1990)

  • View in gallery

    Crude Oil: Spot prices and one month ahead Forecasts 1/

    September 1984 - November 1990

  • View in gallery

    Crude Oil: Spot prices and three months ahead Forecasts 1/

    September 1984 - November 1990

  • View in gallery

    Crude Oil: Spot prices and six months ahead Forecasts 1/

    September 1984 - November 1990

  • View in gallery

    Crude Oil: Spot prices and nine months ahead Forecasts 1/

    September 1984 - November 1990

  • View in gallery

    WEIGHTING SCHEMES