1. Development of futures market

Futures markets serve two interrelated purposes: the first is a provision of an organized forum allowing agents to undertake hedging, or speculation. Their second purpose is that related to price discovery; the price of futures contracts provides a summary or consensus view, based on market trading, of the participants’ expectations with regard to the future course of prices. For the market to perform either of these functions efficiently, there has to be sufficient activity in terms of the volume of contracts traded on a regular basis.

While there are a number of international forward and futures markets in crude oil, by far the largest futures market is on the New York Mercantile Exchange (NYMEX). Since the introduction of the oil futures contracts on NYMEX in April 1983, volume of trade in oil contracts has grown almost exponentially. In 1983, the average daily volume was around 1,200 contracts, (with each contract representing 1,000 barrels of oil). ^1/ By the end of 1990, the daily average exceeded 100,000 contracts, or 100 million barrels a day, making crude oil futures one of the most heavily traded of any futures contract. At the same time, the total number of contracts outstanding has increased from around 5,000 contracts to nearly 280,000 contracts, equivalent to 280 million barrels. In comparison, total world oil production is about 65 million barrels a day. Furthermore, because the futures price is set in open trading, it is accepted as the benchmark from which almost all other prices for crude oil are calculated.

It is worth noting that there are futures contracts in oil products which predate the trading of crude oil contracts. For instance, gasoline futures were introduced on NYMEX in 1978 while heating oil futures were introduced in 1980. ^2/ During 1978 crude oil futures trading was begun on the NYMEX, but because of insufficient volume of trade the contract was withdrawn. One of the main reasons given for insufficient trading was the lack of adequate volatility in crude oil spot prices. As the discussion below shows, the low volatility in turn was due to the market structure whereby spot trading was very limited; most of the oil traded was on the basis of contractual arrangements between oil producers and international oil companies.

The significance of contractual arrangements can be judged from the fact that until the late 1970s, almost 90 percent of the world’s oil was sold under long-term contracts based on prices set by the major oil producers, and the other 10 percent was bought and sold informally between the international oil companies. Beginning with the late 1970s, however, there was a fundamental change, with this historic system of long-term, fixed price contracts negotiated between oil producers and the international oil companies being largely replaced with an open market system. By the early 1980s, 90 percent of the world’s oil was available on the spot market. The move to the preponderance of spot market transactions seems to have been accompanied by considerably greater short-term volatility in oil prices in the recent years. ^3/

Thus, on a short-term basis (day-to-day, or week-to-week), crude oil prices during the 1980s were considerably more volatile than during the 1970s, when the prices were stable often for months at a time. The incidence of sharp abrupt swings in prices, however, does not seem to have been increased any marked way by the move to greater market role. If anything, it could be argued that by adjusting prices continuously to the supply-demand imbalances, the extent of the abrupt changes may be lessened. ^4/

The antecedents of the change from contractual to spot price transactions lay in two major developments during the 1970s: (i) the world’s major oil producers exercised greater control over their oil fields, allowing them to sell their oil to whomever offered the best deal; (ii) major oil companies that previously had owned the oil fields were cut loose to bid for crude oil wherever it was available.

At the same time, the very high real interest rates of the early 1980s, which made the storage of oil an expensive activity, encouraged the formation of the oil futures market. The emergence of this market meant that it was much cheaper for a user to acquire claims on oil by buying a contract that assured delivery when needed than it was to purchase and store actual oil. As with almost all futures trading, actual delivery is made in only a small proportion of contracts: instead the contracts usually are bought and sold over and over again, often many times in the same day, as financial instruments that provide a form of price and supply insurance to users and producers.

2. Futures contracts

The futures trading began in 1983 with delivery for a period of up to six months ahead. Initially, even this period seemed likely to easily satisfy the needs of traders, and most of the transactions were concentrated in the first two or three months. Fairly soon, however, trading in distant months became equally active and there was demand for longer maturities. Trading was then extended to nine months and by 1989 to twelve months. Beyond nine months, however, trading volume remained limited. Since July 1990, the contracts have been extended almost continuously; first, they were extended to 15 months, then to 18 months, and by April 1991, contracts were available for up to three years ahead. For the purposes of comparing forecasting accuracy, the most recent lengthening of maturities could not be utilized. For the earlier periods also, most active trading was limited to a period up to nine months ahead, as the paper focuses on trading at these maturities.

A critical factor affecting the accuracy of the futures prices’ forecast is the way the prices are actually determined. To ensure that the determination of prices is completely transparent, there are some very specific rules for arriving at the “settlement price” for the crude oil futures contracts. As discussed in Appendix I, the “settlement price” is a weighted average of the transactions prices, towards the end of the trading session. This is the price used in the following analysis of the futures prices’ forecasting accuracy (see Appendix I for details).

The contracts are specified for a specific crude--namely, the “West Texas Intermediate” (WTI) to be delivered at a specific point--but the futures market rules allow for delivery of six other grades against the WTI contract. ^5/ An adjustment factor is added by NYMEX to the WTI price. Buyers taking delivery of alternative grades may appear to be concerned about getting unexpected or less-valued oil. But for most substitute crudes, the buyers are actually indifferent. That is because the marginal benefit to the refiner of utilizing these grades generally exceeds their additional cost and the NYMEX adjustment factor. ^6/ There is, in any case, a strong and systematic relationship between the West Texas price and prices of other crudes. So that from the West Texas price, one can infer forecasts of prices for other crudes (see Appendix II for a detailed analysis of this issue).

In any case, as noted earlier, the futures prices frequently set the benchmark for the pricing of other crudes in the spot market. It is also worth noting that while the futures prices are not entirely unconstrained in the sense that there are exchange-determined upper and lower limits to price changes in any given day, the constraints are seldom if ever binding. Even when they are binding, the decisions taken usually reflect almost entirely the market conditions (see Appendix I).

1. End-of-month prices and random walk

In the initial exercise, futures prices on the last trading day of the month are utilized, under the premise that the latest price would incorporate the latest information and thereby provide the most accurate and up-to-date forecast. In a subsequent exercise, however, prices prevailing at different times during the month, weighted according to specified criteria, are also examined. Accuracy of forecasts using futures prices is investigated in relation to forecasts obtained from a number of different techniques. These include forecasts using the random-walk model, time-series models, judgmental methods, and econometric models.

Consider first a comparison of the forecasts from end-of-month futures prices with the forecasts from a ‘random walk’ model. The latter can be regarded as postulating that spot prices at period t are the best unbiased predictor of spot prices at any future period T. ^8/ The rationale for this hypothesis is that a commodity such as crude oil is subject, on a day-to-day basis, to a large number of influences on its demand and supply, and on its price. Hence, at any given time, the spot price itself reflects the consensus view as to the current market situation and provides the best guidance as to the future course of prices.

Table 3 provides the results of a comparison of the accuracy of the futures prices, with that of the random walk model. Forecasts are examined for a period ranging from one to nine months. Forecast error is defined as follows:

Table 3.

Comparison of Futures Prices and Random Walk

Note: Forecasts are for the period December 1985 to October 1990.

Table 3.

Comparison of Futures Prices and Random Walk

Forecast Accuracy Criterion	Horizon	Futures Prices	Random Walk
1. MAE	1	0.0870	0.0921
	2	0.1436	0.1438
	3	0.1722	0.1798
	4	0.1875	0.1925
	6	0.2260	0.2355
	9	0.2676	0.2836
2. RMSE	1	0.1104	0.1211
	2	0.1717	0.1799
	3	0.2080	0.2188
	4	0.2240	0.2378
	6	0.2573	0.2741
	9	0.2929	0.3172
3. Thiel’s ‘U’	1	0.0369	0.0404
	2	0.0573	0.0600
	3	0.0694	0.0730
	4	0.0747	0.0793
	6	0.0859	0.0914
	9	0.0977	0.1058

Note: Forecasts are for the period December 1985 to October 1990.

View Table

Table 3.

Comparison of Futures Prices and Random Walk

Forecast Accuracy Criterion	Horizon	Futures Prices	Random Walk
1. MAE	1	0.0870	0.0921
	2	0.1436	0.1438
	3	0.1722	0.1798
	4	0.1875	0.1925
	6	0.2260	0.2355
	9	0.2676	0.2836
2. RMSE	1	0.1104	0.1211
	2	0.1717	0.1799
	3	0.2080	0.2188
	4	0.2240	0.2378
	6	0.2573	0.2741
	9	0.2929	0.3172
3. Thiel’s ‘U’	1	0.0369	0.0404
	2	0.0573	0.0600
	3	0.0694	0.0730
	4	0.0747	0.0793
	6	0.0859	0.0914
	9	0.0977	0.1058

Note: Forecasts are for the period December 1985 to October 1990.

where ${\mathrm{P}}_{\mathrm{t},\mathrm{T}}^{\mathrm{j}}$ is the forecast made at time t for a future spot price at time T by using the weighting scheme j (discussed below). Thus in the case of the futures forecast, the price at maturity is taken; for the random walk model

S_T is the realized spot price at time T; T-t denotes the forecast horizon, 1 to 9 months.

Three different criteria for comparing forecast accuracy are utilized. The first is the mean absolute errors (MAE). This is the absolute value of the deviation of the predicted value from the realized value. Secondly, and the main one, is the root mean square error, (RMSE), which attaches a higher weight to larger absolute errors. Thirdly, Theil’s “U” statistic which, in addition, adjusts for trend changes, is utilized.

The results in Table 3 tabulate the forecast errors from using these three forecast accuracy criteria for one to four months, and six and nine months, respectively. These results suggest two main conclusions: the first is that for virtually all forecast horizons, and for the different accuracy criteria, futures prices provide more accurate forecasts than the random walk model. The second conclusion is that as the length of the forecast horizon increases, the accuracy of the forecasts, whether using futures prices or the random walk model, diminishes markedly. In general, however, as Charts 2 to 4 also emphasize, futures prices provide a fairly accurate forecast of future spot prices for up to a six-month forecast horizon.

View Full Size

Crude Oil: Spot prices and one month ahead Forecasts ^1/

September 1984 - November 1990

Citation: IMF Working Papers 1991, 093; 10.5089/9781451951110.001.A001

^1/ Price (logorithms, dollars/barrel).

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Crude Oil: Spot prices and three months ahead Forecasts ^1/

September 1984 - November 1990

Citation: IMF Working Papers 1991, 093; 10.5089/9781451951110.001.A001

^1/ Price (logarithms, dollars/barrel).

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Crude Oil: Spot prices and six months ahead Forecasts ^1/

September 1984 - November 1990

Citation: IMF Working Papers 1991, 093; 10.5089/9781451951110.001.A001

^1/ Price (logarithms, dollars/barrel).

• Download Figure
• Download figure as PowerPoint slide

To the extent that the random walk model may be regarded as “naive”, an obvious next step would be to examine whether one can improve upon the accuracy of the futures prices by using other more sophisticated but readily available econometric or time-series models. Before considering these alternatives, it is worth enquiring whether on an a priori basis, it is appropriate to use anything but the latest price. One hypothesis would be that if the market is efficient, and the results discussed in Section III above suggest that it is, then it must be the case that the latest price embodies all the relevant information and provides the best possible prediction of the future spot price. Hence combining the latest price with other (preceding) prices would not lead to any increase, and may actually lead to a decline, in the forecast accuracy. An alternative hypothesis would be that if there is a time varying risk premia with a transitory and a permanent component, then an averaging procedure may dampen, or even cancel out, the temporary component. Such a procedure may, therefore, improve on the end-of-period prices. It might also be argued that to the extent that there are speculative bubbles in the market some sort of averaging procedure might improve the forecasts. In this regard the results of a number of recent studies into the efficiency of different asset markets are quite illuminating. Most of these studies suggest that even in markets that are regarded as approximately efficient, there may be evidence of speculative bubbles. For instance, Lo and Mackinley (1988) find a high degree of autocorrelation in weekly stock returns, while Fama and French (1988) and Poterba and Summers (1989) found similar results for returns spanning one year or more.

2. Predetermined weighting schemes

Given the above considerations, it may appear worthwhile to examine whether using prices prevailing during the month provide any incremental improvement in forecast accuracy of futures prices. The problem then is essentially an empirical one of choosing the optimal weighting scheme for combining different prices. The case of the end-of-month price would be simply a special case where the weight on intra-month prices is equal to zero. The end-of-month price is compared with three other schemes in which weights are also predetermined:

(i) Last five days’ price: this is a simple average of the futures’ (settlement) price on the last five trading days of any given month.

(ii) Monthly average: this is a simple average of the futures’ price for each of the trading days in any given month. Since the number of trading days in the month can vary from 20 to 23 days, the average is based on closing prices on the last 20 trading days in the month.

(iii) Exponentially declining weights: The weights for the 20 trading days are computed according to the inverse of the exponential function; that is, the weights, W_i, are defined as follows: W_i = 1/eⁱ, i=1,2…n. Here 1 refers to the latest observation which is given the greatest weight, 2 refers to the last but one observation and so on.

In addition to the above four schemes, a scheme whereby weights were determined endogenously to minimize the forecast error (by using a maximum likelihood function) was also applied. (This methodology and its results are discussed presently.)

Consider first the results for the above four schemes, given in Table 4. For the one month ahead forecast, using end-of-the-month price yields the smallest forecasting error. According to the RMSE criterion, the monthly average gives forecast errors which are around 15 percent higher than the end-of-the-month price and the exponential scheme. It is worth noting, however, that monthly average forecast errors are still smaller than the errors generated by the random walk model (last column). As the forecast horizon increases, the difference in the forecast accuracy of the end-of-the month and exponential scheme vis-à-vis the monthly average declines monotonically, so that for the six-month ahead forecasts there is virtually no difference in the errors generated by the different schemes. As results in Appendix Table 2 indicate, this remains the case up to nine months forecast when the criterion is the MAE; however, when RMSE is the criterion, the monthly average has a negligible edge over the other schemes. It is also worth noting that regardless of the weighting scheme, the forecast errors using the random walk model are always larger than those using futures prices. Furthermore all forecast errors increase monotonically with the increase in the forecast horizon. (See Chart 5 for a comparison of the actual spot prices and forecasts for 9 month horizons respectively.)

Table 4.

Comparison of Alternative Weighting Schemes for Futures Prices

Note: Forecasts are for the period December 1985 to October 1990. Criteria for accuracy is the “Root Mean Squared Forecast Error.”

Table 4.

Comparison of Alternative Weighting Schemes for Futures Prices

Forecast Horizon (Months)	Forecasting Method
	Futures prices				Random Walk
	Last day	Last five days	Monthly average	Exponentially declining weights
1	0.1244	0.1268	0.1432	0.1248	0.1383
2	0.1984	0.1975	0.2067	0.1975	0.2057
3	0.2355	0.2356	0.2456	0.2351	0.2503
4	0.2527	0.2551	0.2558	0.2531	0.2716
5	0.2747	0.2773	0.2773	0.2751	0.2955
6	0.2929	0.2946	0.2945	0.2926	0.3143
7	0.3089	0.3086	0.3076	0.3081	0.3313
8	0.3207	0.3225	0.3204	0.3209	0.3447
9	0.3312	0.3301	0.3309	0.3311	0.3617

Note: Forecasts are for the period December 1985 to October 1990. Criteria for accuracy is the “Root Mean Squared Forecast Error.”

View Table

Table 4.

Comparison of Alternative Weighting Schemes for Futures Prices

Forecast Horizon (Months)	Forecasting Method
	Futures prices				Random Walk
	Last day	Last five days	Monthly average	Exponentially declining weights
1	0.1244	0.1268	0.1432	0.1248	0.1383
2	0.1984	0.1975	0.2067	0.1975	0.2057
3	0.2355	0.2356	0.2456	0.2351	0.2503
4	0.2527	0.2551	0.2558	0.2531	0.2716
5	0.2747	0.2773	0.2773	0.2751	0.2955
6	0.2929	0.2946	0.2945	0.2926	0.3143
7	0.3089	0.3086	0.3076	0.3081	0.3313
8	0.3207	0.3225	0.3204	0.3209	0.3447
9	0.3312	0.3301	0.3309	0.3311	0.3617

Note: Forecasts are for the period December 1985 to October 1990. Criteria for accuracy is the “Root Mean Squared Forecast Error.”

View Full Size

Crude Oil: Spot prices and nine months ahead Forecasts ^1/

September 1984 - November 1990

Citation: IMF Working Papers 1991, 093; 10.5089/9781451951110.001.A001

^1/ Price (logarithms, dollars/barrel).

• Download Figure
• Download figure as PowerPoint slide

3. Endogenous weighting scheme

The above four schemes apply predetermined weights to futures prices. An interesting extension would be to see if the weighting scheme could be determined endogenously by the data, and whether that reduces the out-of-sample prediction errors. This was examined by fitting the following equation to the data on spot and futures’ prices:

Here S_T denotes the realized spot price at time T and P_t,T denotes the futures price at time t for the contract maturing at T; α and λⁱ are the estimated parameters. The equation was estimated separately for each of the nine forecast horizons. Given the non-linearity, the estimation procedure used a standard maximum likelihood function (MINDIS). Initially the equation was estimated using daily observations; that is, K = 20 and S_T was regressed on 20 variables constituting the prices at the end of each of the trading days in any given month. ^9/ However, because of considerable collinearity between the daily observations, (for almost all the forecast horizons) it was not possible to achieve convergence. Therefore, estimation was undertaken using prices at the end of every fifth day (using data from 1986 to 1988). Using the estimates of λⁱ, normalized weights (summing to 1) were computed for each of the forecast horizons. These weights were then used to compute the out-of-sample forecasts and forecast errors for the period November 1988 to October 1990. In order to compare these forecasts with those obtained from using extraneous weights, forecast errors were recomputed for this period for the end-of-the-month, monthly average and the exponentially declining weights.

Results of the above exercise are provided in Table 5 and Chart 6. Consider first Chart 6, which compares the weighting scheme for the different forecast horizons as generated by the estimation, and the extraneous weights for the other schemes. (These other weights are, of course, invariant across forecast horizons.) As the chart shows, the pattern of weights is related to the length of the forecast horizon. The weights decline gradually with lag length when the horizon is distant, but for a nearby horizon the decline is very steep. For instance, when undertaking one month ahead forecast, according to the estimation, greatest weight should be placed on the latest futures price. But in undertaking a nine-month ahead forecast the average should be relatively more equally weighted. This result is fairly plausible and can be explained in a number of ways. One explanation may be in terms of the deficiency of lagged prices with respect to the information they contain. This deficiency is more serious relative to a short forecast than to a long forecast. That is, for the short forecast, the information (or the consensus) embodied in the latest price is likely to be much more important than the information in the lagged prices, and hence should be given the greatest weight. For the distant forecast, given the relatively greater uncertainty, somewhat greater weight can be given to the lagged prices. Some support for the above result, and the explanation, is found in a rather different study recently undertaken by Feinstein (1989). In that study, a volatility indicator for the stock market was constructed, by using weighted average of current and lagged implied volatilities from call options. That study reached very similar conclusions as to the relationship between weights and the forecast horizon.

Table 5.

Comparison of “MINDIS” Weighted Forecasts

Note: “MINDIS” is a maximum likelyhood Algorithim for non-linear estimation. Estimation period was December 1985 to October 1988 and the forecasts were generated for November 1988 to October 1990.

Table 5.

Comparison of “MINDIS” Weighted Forecasts

	Root Mean Square Errors
Forecast Horizon	“MINDIS” weighted	Last day	Monthly average	Exponentially declining weights
1	0.1385	0.1390	0.1622	0.1416
2	0.2189	0.2182	0.2288	0.2185
3	0.2454	0.2481	0.2436	0.2471
4	0.2509	0.2483	0.2534	0.2486
5	0.2644	0.2561	0.2683	0.2574
6	0.2751	0.2730	0.2758	0.2713
7	0.2830	0.2815	0.2816	0.2786
8	0.2893	0.2842	0.2884	0.2828
9	0.2900	0.2865	0.2916	0.2863

Note: “MINDIS” is a maximum likelyhood Algorithim for non-linear estimation. Estimation period was December 1985 to October 1988 and the forecasts were generated for November 1988 to October 1990.

View Table

Table 5.

Comparison of “MINDIS” Weighted Forecasts

	Root Mean Square Errors
Forecast Horizon	“MINDIS” weighted	Last day	Monthly average	Exponentially declining weights
1	0.1385	0.1390	0.1622	0.1416
2	0.2189	0.2182	0.2288	0.2185
3	0.2454	0.2481	0.2436	0.2471
4	0.2509	0.2483	0.2534	0.2486
5	0.2644	0.2561	0.2683	0.2574
6	0.2751	0.2730	0.2758	0.2713
7	0.2830	0.2815	0.2816	0.2786
8	0.2893	0.2842	0.2884	0.2828
9	0.2900	0.2865	0.2916	0.2863

Note: “MINDIS” is a maximum likelyhood Algorithim for non-linear estimation. Estimation period was December 1985 to October 1988 and the forecasts were generated for November 1988 to October 1990.

View Full Size

WEIGHTING SCHEMES

Citation: IMF Working Papers 1991, 093; 10.5089/9781451951110.001.A001

• Download Figure
• Download figure as PowerPoint slide

Next consider how the forecast errors using these endogenously determined weights compare with the errors generated by the other weighting schemes. The new weights were used to construct forecasts for the period November 1988 to October 1990. Given that these forecasts are for the out-of- sample period, it would not necessarily be the case that the forecast errors would be smaller than the other weighting schemes. This is borne out to some extent by the results in Table 5. Compared to the simple monthly average (Scheme II above), the forecast errors using the new weights are generally smaller. However, comparing the last-day forecast with the new weights forecast, shows that apart from the first and the third month, the former has marginally smaller errors.

1. Time-series models

Consider first a comparison of the accuracy of futures’ price forecasts with the forecasts generated from time-series models. A major advantage of this technique is, of course, that the data requirements are very limited--only the data for the actual variable being modelled is needed, which in this case is simply the spot price. The main problem which arises is, however, to ensure that the forecasts obtained from this method correspond precisely to the forecasts implicit in the futures prices. This necessitates estimating models in such a way so that forecasts can be obtained for the day of the maturity of the contract. This in turn necessitates estimation of the models using daily spot price data. Apart from the considerations relating to the time period, the key issue is the specification of the model. In order to provide a rigorous test of futures’ price accuracy, it is obviously critical that the time-series model should have the best specification possible. For this purpose, the estimation was initially undertaken using the most general form of the time series generating process--that is, the autoregressive integrated moving average process (ARIMA). The process is defined by the equation:

where φ(β) and θ(β) are operators in β of degree p and q, respectively, and the roots of φ(β)=0 and φ(β)=0 lie outside the unit circle. a_t is white noise process [E(a_t)=0 and var (a_t)=σ²]. The process is thus of order (p, d, q). This model is very general, subsuming autoregressive, moving average, and mixed auto-regressive-moving average models, and the integrated forms of all three. ^10/

The preliminary analysis strongly suggested, however, that over the sample period, a more specific autoregressive moving average process (ARMA), that is, d = 0, was appropriate implying stationarity in the spot price series.) In such a case the roots of the polynomials lie within the unit circle. A number of models were estimated using this more restrictive specification. (The reasons for preferring the ARMA model was that it achieved as good a fit as an AR model but using fewer parameters.) The best fit was given by a first order autoregressive process and a moving average process of order two. ^11/ That is, the equation is of the form

or more generally

where ${\overline{\mathrm{P}}}_{\mathrm{t}}$ is now the deviation of P_t from its mean.

The results of forecasts obtained using this model, and the comparable forecasts using futures prices, are shown in Table 6 using the absolute mean error the root mean square errors for forecast horizons from one to nine months. As noted earlier, the increase in forecast error of the model as the length of forecast horizon increases from one to six months is quite marked; between six and nine months however, there is either no increase or even a marginal improvement Compared to the futures prices, however regardless of the forecast horizon, the ARMA model does less well, although it is only for the shortest forecast horizon that the difference is marked.

Table 6.

Comparison of ARMA Model and Futures Prices Accuracy

Notes: The ARMA model was estimated using daily spot price data, starting with 2nd January 1986 to end-December 1986, to get the first set of forecasts for 1 to 9 months ahead. The model was then reestimated for 1st February 1986 to end-January 1987 for the next set of forecasts, and so on.

Table 6.

Comparison of ARMA Model and Futures Prices Accuracy

Forecast Horizon (Months)	ARMA Model		Futures Prices
	MAE	RMSE	MAE	RMSE
1	0.065	0.102	0.053	0.068
2	0.101	0.126	0.082	0.110
3	0.120	0.146	0.108	0.141
4	0.142	0.180	0.131	0.167
6	0.180	0.207	0.159	0.197
9	0.174	0.204	0.161	0.192

Notes: The ARMA model was estimated using daily spot price data, starting with 2nd January 1986 to end-December 1986, to get the first set of forecasts for 1 to 9 months ahead. The model was then reestimated for 1st February 1986 to end-January 1987 for the next set of forecasts, and so on.

View Table

Table 6.

Comparison of ARMA Model and Futures Prices Accuracy

Forecast Horizon (Months)	ARMA Model		Futures Prices
	MAE	RMSE	MAE	RMSE
1	0.065	0.102	0.053	0.068
2	0.101	0.126	0.082	0.110
3	0.120	0.146	0.108	0.141
4	0.142	0.180	0.131	0.167
6	0.180	0.207	0.159	0.197
9	0.174	0.204	0.161	0.192

Notes: The ARMA model was estimated using daily spot price data, starting with 2nd January 1986 to end-December 1986, to get the first set of forecasts for 1 to 9 months ahead. The model was then reestimated for 1st February 1986 to end-January 1987 for the next set of forecasts, and so on.

2. Judgmental forecasts and econometric models

Next consider the forecasts obtained using judgmental methods or econometric models. ^12/ The objective is again to compare the accuracy of forecasts using these alternative techniques with that of forecasts obtained from futures prices. Clearly the comparison can be important given that the futures prices are readily available while the alternative techniques can require considerable investments in model building, following developments in the oil industry, etc.. In theory, such a comparison should not pose any problems: given the ubiquitous nature of oil price forecasting models and forecasts, one would simply take these forecasts and compare them with the futures prices. More formally, denote by

${\mathrm{P}}_{\mathrm{t},\mathrm{T}}^{\mathrm{j}}$ the price forecast obtained at time t for T, using the forecasting technique j; and

${\mathrm{P}}_{\mathrm{t},\mathrm{T}}^{\mathrm{f}}$ the price of the futures contract at time t, for maturity at T; and S_T the spot price at T.

Then the comparison would simply be between

using any one of the forecasting accuracy criteria.

In practice, however, unlike the time-series model estimated above, there are a number of important constraints which arise in making this comparison.

The first of these constraints arises because in the alternative forecasts T refers not to any specific day, as in the futures forecast, but to a specific period such as a year, or at best a quarter. (Unfortunately forecasts on a quarterly basis are available only for a very short period.) For the above comparison to be undertaken, therefore, it is necessary to transform ${\mathrm{P}}_{\mathrm{t},\mathrm{T}}^{\mathrm{f}}$ so that it is comparable to ${\mathrm{P}}_{\mathrm{t},\mathrm{T}}^{\mathrm{j}}$. One, and perhaps the only procedure which could be used for this is to obtain an average of prices for contacts maturing in the year for which the forecast is made. Thus one would have a sequence of maturities, T-1, T-2, … T-11--where T now refers to the end of the calendar year for which the alternative forecast is made. Even where this can be done, given that futures contracts were only available from 1983 onwards, it would mean that a maximum of only eight observations could be available for the analysis.

However, a second constraint then arises from the fact that the forecasts are often made several months prior to the beginning of the calendar year to which they apply. This means that the forecasting horizon T-t is usually in excess of fifteen months or so. Given that until September 1990, the most distant contracts were only for a period up to fourteen months, it would not be possible to have the contracts which mature throughout the year. Indeed for the period before 1987, when contracts were available for even shorter periods, it may be possible to have only the first few months of the year for which the forecast is made. ^13/

Often, it is also quite difficult to ascertain precisely when the alternative forecasts were made; that is, while for futures, the time t is transparent, it can be highly uncertain for the alternative forecasts. Indeed for the latter, the time at which the forecast is made can often only be guessed at, with a considerable margin of error.

Finally, it is important that the alternative forecasts, especially the judgmental ones are not influenced by the path of futures prices themselves. In practice, of course, it is impossible to ensure this; all that can be done is to ascertain the extent to which a formal model, or formal analysis, is utilized in making the forecasts.

In view of the above considerations, forecasts were obtained from three main sources: annual forecasts published by the Petroleum Economist, the United States Energy Administration, and the World Bank. ^14/ Analysis was undertaken, separately, using each of the three sets of forecasts, for the period 1984 to 1991. In general, the forecasting errors using futures prices were smaller, although not markedly so, compared to the forecasts from the alternative techniques. An examination of the individual forecasts also showed that on an annual basis, for the majority of years, the futures prices were more accurate. However, in view of the earlier considerations, these results should be treated with caution. But they do suggest that where futures prices are available, they can provide forecasts which are no worse, and are often better, compared to those obtained from judgmental or econometric methods.

3. Combined forecasts

A final exercise was undertaken to examine whether the forecasts obtained from using futures prices can be improved on, in terms of the variance of the forecast errors, by combining them with forecasts obtained from alternative sources. There is a considerable theoretical literature, and empirical evidence, to show that in many practical situations combined forecasts can outperform individual forecasts in terms of error variance. ^15/ Following Granger and Newbold (1986) this reduction in error variance can be seen readily: Let f_n and t_n be the futures and time series forecasts of the spot price S_n at time n, and ${\mathrm{e}}_{\mathrm{n}}^{\mathrm{f}}$ and ${\mathrm{e}}_{\mathrm{n}}^{\mathrm{t}}$ be the two forecast errors, respectively. That is,

and

The combined forecast is C_n = Kf_n + (1-K)t_n, where K and (1-K) denote the weights, and the combined forecast error ${\mathrm{e}}_{\mathrm{n}}^{\mathrm{c}}$ is

and the error variance is

where ρ is the correlation between two forecast errors.

It can be seen readily that ${\sigma }_{\mathrm{c}}^2<\min \left({\sigma }_{\mathrm{f}}^2,{\sigma }_{\mathrm{t}}^2\right)$ unless either ρ is exactly equal to σ_f/σ_t or to σ_t/σ_f. If either equality holds, then the variance of the combined forecast is equal to the smaller of the two error variances.

The main issue here is the choice of weights to be applied to the different forecasts. Using the forecasts from the time series model, the following weighting scheme was applied: ^16/

where ${\mathrm{e}}_{\mathrm{n}}^{\mathrm{j}}={\mathrm{x}}_{\mathrm{n}}-{\mathrm{f}}_{\mathrm{n}}{}^{\left(\mathrm{j}\right)},\mathrm{j}=1,2$

f_n^(j) are the two forecasts of X_n from futures and time series models, respectively, and ${\mathrm{e}}_{\mathrm{n}}^{\mathrm{j}}$ are the two forecast errors; n denotes the number of forecasts and Kn is the estimate of the weight to be applied at time n; V was varied between 1 and 3 to obtain optimal weights which were constrained to lie between zero and unity. The results of the exercise showed only a marginal improvement in terms of reduction in error variance for forecasts horizons ranging from six to nine months. ^17/ These results can be interpreted in the context of the earlier findings of the close correspondence between the two sets of forecasts (futures and time series), since, as has been emphasized, the combination is most likely to result in significant improvement only when individual forecasts are very dissimilar in nature.