Chapter

II How Accurate Are the World Economic Outlook Projections?

Author(s):
International Monetary Fund
Published Date:
January 1994
Share
  • ShareShare
Show Summary Details
Author(s)
José M. Barrionuevo1

Economic activity is affected by decisions that are based on projections of future growth and inflation. Consumption and investment decisions rely heavily on projections of future economic developments. Governments plan budgets and set macroeconomic policies based on forecasts of future economic activity. Because large deviations from anticipated conditions may prove to be costly in terms of lost output and employment, it is important to assess whether forecasts are accurate, given the information that is available when they are made.

A useful discussion about forecasting accuracy needs to provide a qualitative assessment of the way in which various forms of inefficiency in a projection are related. These inefficiencies may be due to the way in which past errors are used to make current projections or because the economic model is not the minimum variance model. Forecasting accuracy has been the subject of numerous examinations over the last three decades.2 Most of the previous studies on forecasting accuracy evaluate the accuracy of a projection based on the characterization of the regression error. A major difficulty with this criterion is that it provides only a joint test of unbiasedness and efficiency. Yet, a distinction between the different properties of a projection would yield information that is useful in improving its accuracy.

This paper evaluates the accuracy of a forecast based on the properties of the forecast error, defined as the difference between the realization and the projection. The optimality conditions of a forecasting optimization problem are used to show that under rational expectations the usual test for efficiency is necessary, yet not sufficient, to ensure efficiency. The necessary and sufficient conditions for efficiency are presented and two statistics are derived to measure these conditions. It is shown that the optimality conditions under rational expectations yield insight into the relation between different kinds of inefficiency. Moreover, the criterion provides simple adjustment factors that reduce the inefficiency of a forecast. This provides a consistent framework to evaluate the accuracy of projections.

The optimality conditions are used as criteria to examine the accuracy of the World Economic Outlook projections of output growth and inflation for industrial and developing countries. The accuracy of growth and inflation projections in each of the business cycles in this period is also examined for the seven major industrial countries. The analysis extends the sample beyond the 1971–86 period covered by Artis (1988), thereby permitting a comparison of the World Economic Outlook projections for 1990–91 with the projections in previous recessions.

The World Economic Outlook projections are conditional on a number of assumptions about economic policies, exchange rates, and commodity prices. The relationship between these factors and deviations from projection outcomes are, however, beyond the assessment presented below. For developing countries that are engaged in Fund-supported stabilization and structural adjustment programs, the projections assume that policies aimed at achieving the growth and inflation objectives are adopted and implemented. Thus, the deviations between the conditional projections and outcomes may be interpreted as a measure of the extent to which the policies specified in the programs were not fully implemented, or as a reflection of the fact that the assumptions about the international economic environment faced by these countries have not always been realized. To evaluate the effect of this sort of conditionality on the World Economic Outlook projections for developing countries, forecasting accuracy tests are performed for a sample of countries that were not engaged in Fund-supported programs. The tests performed for the nonprogram countries correspond more closely to the evaluation of unconditional growth and inflation forecasts and, thus, are similar to those performed for industrial countries.

Time-series models are estimated for output growth and inflation for the seven major industrial countries and the accuracy of the forecasts generated by these models is examined relative to the projections of the World Economic Outlook. To examine the relevance of past errors in forming projections, the fraction of the previous years’ errors in projections of growth and inflation that are incorporated into the forecast of growth and inflation for the current year is calculated. This illustrates how a time-series model incorporates previous errors to generate current forecasts. Moreover, the proposed criterion implies that judgmental projections can be adjusted in a similar fashion to improve the accuracy of a forecast. The forecasts derived from these time-series models are then used as a benchmark to evaluate the relative efficiency of the World Economic Outlook projections.

A Simple Forecasting Accuracy Criterion

The basic idea of Muth’s (1961) rational expectations hypothesis is that expectations of future events are essentially the same as the projections of the economic model that incorporates in a systematic fashion all relevant information, including current and past realizations of economic variables. The motivation behind rational expectations is that agents use the information available to them in an efficient manner. Rational expectations thus implies that agents will not be systematically wrong in making projections, given this information,

where R+1 is the next period realization of a random variable and ξ+ 1 is a stochastic error with E(ξ+1)=0,E(ξ+12)<,E(ξξ+1)=0, E+1 · ɸ) = 0, and ɸ represents a set of current and past values of R.3 Thus, ξ+ 1 is not predictable from information known in the current period. Equation (1) implies that at any particular time the rational expectation can be inaccurate because a random shock occurs. Yet, the projection is correct on average.

Frenkel (1977), Frenkel and Razin (1980), and Artis (1988), among others,4 test for the efficiency of a projection by estimating equation (2) by ordinary least squares,

where F is the forecast of R+1 given information available in the current period, and v+ 1 is the random error of the equation. The conventional criterion implies that a forecast is efficient if the estimated coefficients are α = 0 and γ = 1. Moreover, this criterion is also used to evaluate unbiasedness.5 It is demonstrated below, however, that under rational expectations, this is a necessary but not sufficient condition for efficiency in the sense that a set of projections could satisfy this test and still not be the minimum variance forecast.

A property of the disturbance term that has been extensively examined is the problem of serial correlation in the regression error (v). The standard test is not sufficient, however, for efficiency because the problem of autocorrelation in the forecast error (ξ) has been largely ignored.6 The relation between the forecast error and the forecast itself is derived by subtracting F from both sides of equation (2) and replacing R+1F by ξ+1,

Equation (3) implies that the forecast error (ξ) is linearly related to the forecast by γ—1, a and the random error ν. Note that ν is equal to ξ only if α and γ − 1 are zero, otherwise ξ would also be determined by the forecast.

Rational expectations imposes a series of restrictions on the properties of the forecast error (ξ) that are derived from the following optimization problem. The problem is stated in terms of minimizing the sum of squared forecast errors by choice of parameters α and β in equation (4),

where ξ+ 1 = α + β · F + ν. The model is therefore consistent with the standard statistical criterion of minimizing the sum of squared errors. A comparison of equations (3) and (4) implies that β = γ − 1. The optimality conditions of this problem are given by equations (5) and (6).

Equation (5) implies that the forecast is unbiased if its average error is zero; Holden and Peel (1990) show that this is necessary and sufficient. They also show that the standard criterion for unbiasedness is necessary but not sufficient. In the same fashion as in the theory of finance,7 a forecast is efficient if it reflects all the information that is available at the time the forecast is made. Thus, condition stated in equation (6) implies that a forecast is efficient if the error term is uncorrelated with the forecast itself. Applying E(A·B) = E(AE(B) + cov(A, B) to equation (6), yields

Thus, the optimality condition in equation (6) can be characterized in terms of the average forecast error, the average projection, and the covariance between the forecast error and the forecast itself. Hence, even if the covariance between the forecast error and the forecast itself is zero, the condition in equation (6) would be satisfied only if the average forecast error were zero. By contrast, the conventional criterion implies that E+1 · F) = cov(ν+1, F), since by construction the average residual is zero. This characterizes the importance that unbiasedness has for the efficiency of a projection (equation (6)), unlike the standard criterion that fails to distinguish between these two concepts. It is shown below that the distinction between these two conditions yields the appropriate adjustment factor to improve the accuracy of a forecast. It also illustrates the relevance of the distinction between the regression error (ν) and the forecast error (ξ). Substituting equations (5) and (6) into equation (7), and dividing by the variance of the forecast (σF2) yields

Equation (8) states that the comovement between the forecast error and the forecast itself, relative to the variance of the forecast, should be zero. Note that equation (8) allows a least-squares representation of the relation between the forecast error and the projection. Hence, the optimization problem yields a version of the standard efficiency conditions in which both the average forecast error and β = y—1 are zero.

To demonstrate that β = 0 is not sufficient for efficiency, define F = g(ɸ), where g: ɸ – > F and ɸ = ɸ–1 + ξ. Replacing F = g(ɸ) by g (ɸ–1) + ξ in equation (7) yields

Rational expectations imposes the restriction that cov [ξ+ 1, ξ) = cov[ξ+ 1, g(ɸ-1)] = 0.8 Dividing this restriction by the variance of the forecast error σξ2 yields

Equation (10) implies that the current period forecast error should not be significantly related to last period’s forecast error. It is important to note that if cov(ξ+ 1, ξ) = -cov[ξ+ 1, g(ɸ–1)] ≠ 0 in equation (9), β would be zero even though the forecast errors are serially correlated and, hence, the projection is inefficient. It follows that β = 0 is a necessary, yet not sufficient condition for efficiency. Thus, from equations (8), (9), and (10) the necessary and sufficient conditions for efficiency are that the average forecast error and both β and ρ be zero.

Unlike the standard criterion, this method provides a useful distinction between different forms of inefficiency, and yields insight into the relation between them. For instance, from equations (7) and (8), if ρ is different from zero and β is zero, the forecast is inefficient because the errors of the past are repeated in the present, and hence forecasts could be improved by adjusting them by ρ. If β and ρ are both not equal to zero, the inefficiency is again partly due to the way in which new information is incorporated into projected values. Finally, if ρ is zero and β is different from zero, the inefficiency arises because the model used to derive the forecast is not the minimum variance model, and, thus, the projections could be improved by adjusting them by β.9 If the nature of the relation between different kinds of inefficiency could be established, an adjustment method that would reduce the effect of specific disturbances and, hence, improve the accuracy of a forecast could be produced.

This criterion suggests defining an accurate forecast as one that is both unbiased and efficient. A forecast is unbiased if its average error is zero (equation (5)) and is efficient if the forecast error is not related to information available at the time the projections were made (equations (7) and (9)). Of the two characteristics, unbiasedness is generally regarded as more important because it means that, on average, forecasts are identical to outturns. In addition, unbiasedness is also important because it is a necessary condition for efficiency.

The approach used in this paper fully implements the optimality conditions to assess the accuracy of a forecast and the nature of an inefficiency. To implement this approach, a least-squares regression of the forecast error on a constant is used to test whether the average forecast error is statistically different from zero. The efficiency of a projection can be tested by measuring the statistical significance of the comovement between the forecast error and the forecast itself (the β test), and the comovement between the current period’s forecast error and the previous period’s forecast error (the ρ test). If both of these comovements are not statistically different from zero, a forecast is said to be efficient.10

Forecasting Accuracy of the World Economic Outlook

This section applies the optimality criterion introduced in the previous section to examine the accuracy of the World Economic Outlook projections of output growth and inflation for each of the 7 major industrial countries; the group of 7 largest industrial countries; the group of 14 smaller industrial countries;11 the average of these industrial country groups; each of the groups of developing countries in Africa, Asia, the Middle East, and the Western Hemisphere;12 the average of these developing country groups; and 36 nonprogram developing countries.13

The data are from the published versions of the World Economic Outlook and from earlier unpublished IMF documents. Two sets of projections are examined: the current year forecasts prepared in the spring of the same year, and year ahead forecasts made in the fall for the following year. For the current year forecasts, the outcome is taken to be the figure reported in the World Economic Outlook published in the following spring; and for the year ahead forecasts, the outcome is the estimate published two years later. The sample period is 1971–91 for industrial countries, 1977–91 for developing countries, and 1988–91 for nonprogram developing countries. The smaller industrial countries and the developing countries include several countries for which projections are prepared only once a year. Therefore, for these countries the current year and year ahead forecasts may be similar.

The Seven Major Industrial Countries

Tables 1 and 2 present the results for each of the seven major industrial countries, and for the pooled projections for the seven countries in 1971–91.14 The top panels of Tables 1 and 2 indicate that forecast errors of output growth (real GDP/GNP) and inflation (GDP/GNP deflator) in the current year are all small and not significantly different from zero.15 In addition, the current year growth and inflation forecasts are efficient, except for growth projections for Canada and the pooled, and inflation projections for the United Kingdom. Furthermore, one third of the variation of the forecast error is captured by the predicted variation of the projections for these economies.16 For the remaining countries the fraction of the variation explained by the model ranges from 1 percent for France to 9½ percent for the United States. For the pool, this figure is 5¾ percent.

Table 1.Forecast Error Statistics for Output Growth in the Seven Major Industrial Countries1(In percent)
United StatesJapanGermany2FranceItalyUnited KingdomCanadaSeven Major Industrial Countries
AveragePooled
(Current year forecast)
Average growth2.54.82.42.62.21.62.92.92.7
AFE3−0.1−0.3−0.1−0.1−0.10.1−0.1−0.1
(0.2)(0.4)(0.3)(0.3)(0.3)(0.3)(0.5)(0.2)(0.1)
RMSE40.91.41.41.11.41.12.10.71.4
β50.1−0.20.2−0.1−0.4*−0.1*
(0.1)(0.2)(0.2)(0.2)(0.1)(0.2)(0.1)(0.1)
ρ6−0.1−0.10.30.2−0.20.1
(0.2)(0.2)(0.2)(0.2)(0.2)(0.2)(0.2)(0.2)(0.1)
(Year ahead forecast)
Average growth2.64.82.42.72.31.72.92.92.8
AFE3−0.4−0.5−0.5−0.4−0.4−0.6−0.5−0.4−0.5*
(0.3)(0.6)(0.4)(0.3)(0.5)(0.4)(0.4)(0.3)(0.2)
RMSE41.83.01.91.42.31.91.51.52.1
β5−0.70.1−0.1−0.50.1−0.2*
(0.2)(0.2)(0.4)(0.2)(0.3)(0.3)(0.3)(0.2)(0.1)
ρ6−0.20.10.10.4−0.20.1
(0.2)(0.2)(0.2)(0.2)(0.2)(0.2)(0.2)(0.2)(0.1)

The sample period is 1971–91. The number of observations for the individual country tests is 21, except for the estimates of ρ, which are 20. In the pooled tests there are 147 observations, but 140 for estimation of ρ. The figures in parentheses are standard errors and an * indicates that the error is significantly different from zero at the 5 percent level of significance.

Prior to unification.

Average forecast error is defined as the realization less the forecast.

Root mean squared error.

β is the estimated coefficent from a least-squares regression of the forecast error on the forecast.

ρ is the estimated coefficient from a least-squares estimate of the current period forecast error in the forecast error of the previous period.

The sample period is 1971–91. The number of observations for the individual country tests is 21, except for the estimates of ρ, which are 20. In the pooled tests there are 147 observations, but 140 for estimation of ρ. The figures in parentheses are standard errors and an * indicates that the error is significantly different from zero at the 5 percent level of significance.

Prior to unification.

Average forecast error is defined as the realization less the forecast.

Root mean squared error.

β is the estimated coefficent from a least-squares regression of the forecast error on the forecast.

ρ is the estimated coefficient from a least-squares estimate of the current period forecast error in the forecast error of the previous period.

Table 2.Forecast Error Statistics for Inflation in the Seven Major Industrial Countries1(In percent)
United StatesJapanGermany2FranceItalyUnited KingdomCanadaSeven Major Industrial Countries
AveragePooled
(Current year forecast)
Average inflation5.54.24.27.412.19.96.62.97.1
AFE3−0.5−0.30.60.50.30.2
(0.1)(0.5)(0.2)(0.3)(0.4)(0.4)(0.3)(0.1)(0.2)
RMSE40.62.10.81.21.72.01.40.61.5
β5−0.1−0.10.10.1
(0.1)(0.1)(0.1)(0.1)(0.1)(0.1)(0.1)(0.1)(–)
ρ60.10.1−0.20.2−0.4*0.20.3
(0.2)(0.1)(0.2)(0.2)(0.2)(0.2)(0.2)(0.2)(0.1)
(Year ahead forecast)
Average inflation5.64.14.27.712.010.06.66.07.2
AFE30.2−0.30.20.7*1.11.4*0.6−0.20.5*
(0.3)(0.7)(0.2)(0.3)(0.7)(0.6)(0.5)(0.3)(0.2)
RMSE41.43.30.91.62.92.92.31.42.2
β5−0.10.1−0.10.20.20.1
(0.2)(0.2)(0.1)(0.1)(0.1)(0.2)(0.2)(0.2)(–)
ρ60.30.10.4*0.4*0.30.30.30.3*
(0.2)(0.2)(0.2)(0.2)(0.2)(0.2)(0.2)(0.2)(0.1)

The sample period is 1971–91. The number of observations for the individual country tests is 21, except for the estimates of ρ, which are 20. In the pooled tests there are 147 observations, but 140 for estimation of ρ. The figures in parentheses are standard errors and an * indicates that the error is significantly different from zero at the 5 percent level of significance.

Prior to unification.

Average forecast error is defined as the realization less the forecast.

Root mean squared error.

β is the estimated coefficent from a least-squares regression of the forecast error on the forecast.

ρ is the estimated coefficient from a least-squares estimate of the current period forecast error in the forecast error of the previous period.

The sample period is 1971–91. The number of observations for the individual country tests is 21, except for the estimates of ρ, which are 20. In the pooled tests there are 147 observations, but 140 for estimation of ρ. The figures in parentheses are standard errors and an * indicates that the error is significantly different from zero at the 5 percent level of significance.

Prior to unification.

Average forecast error is defined as the realization less the forecast.

Root mean squared error.

β is the estimated coefficent from a least-squares regression of the forecast error on the forecast.

ρ is the estimated coefficient from a least-squares estimate of the current period forecast error in the forecast error of the previous period.

As one would expect, the current year projections are more accurate than the year ahead projections, which are nonetheless unbiased when evaluated on an individual country basis, with the exception of the inflation projections for the United Kingdom. It is noteworthy that for the pooled sample, year ahead forecasts of growth in 1971–91 overestimated actual growth by ½ of 1 percentage point, while the year ahead forecasts of inflation underestimated actual inflation by the same magnitude. By comparison, Artis found that the average output and inflation forecast errors for 1973–85 were about ¾ of 1 percentage point. The updated results therefore suggest that the bias in the World Economic Outlook’s forecasts was reduced after 1985. For the pooled projections, the results show that the World Economic Outlook’s year ahead forecast of inflation and both current year and year ahead forecasts of output growth are inefficient.17

Comparison of Business Cycles and Recession Years

Charts 1 and 2 show the pooled growth and inflation forecast errors for the seven major industrial countries during the 1971–91 period. A positive (negative) error implies that the actual value was higher (lower) than projected. These charts indicate that prior to 1982 the forecast errors are consistently negative across the group of seven major industrial countries, and are positive thereafter. Tables 3 and 4 suggest that the World Economic Outlook generally overestimated growth and underestimated inflation in 1971–82.18 By comparison, in 1983–91, only output growth in the current year was slightly underestimated, while the year ahead projection of growth and both current year and year ahead projections of inflation showed large reductions in the average error (both absolutely and relative to average growth and inflation) and were unbiased. Furthermore, the average error of the inflation and growth forecasts was only 16 percent of the mean error derived over the entire 1971–91 period. Thus, the accuracy of the World Economic Outlook forecasts improved after 1982. This improvement may partly reflect a more stable economic environment in the 1980s compared with the repeated supply and demand shocks and high inflation in the 1970s.

Chart 1.Forecast Errors in World Economic Outlook Projections for Output Growth1

(In percent)

1Forecast error—defined as realized minus projected—of pooled projections for the seven major industrial countries. Each year consists of seven forecast errors for each of the country. The shaded areas indicate years in which the United States was in recession for two or more quarters, as defined by the National Bureau of Economic Research (NBER).

Chart 2.Forecast Errors in World Economic Outlook Projections for Inflation1

(In percent)

1Forecast error—defined as realized minus projected—of pooled projections for the seven major industrial countries. Each year consists of seven forecast errors for each of the country. The shaded areas indicate years in which the United States was in recession for two or more quarters, as defined by the National Bureau of Economic Research (NBER).

Table 3.Pooled Forecast Error Statistics for Output Growth Over the Business Cycles in the Seven Major Industrial Countries1(In percent)
197419801982199019911971-821983-911986-91
(Current year forecast)
Average growth1.11.3−0.42.50.92.62.92.8
AFE2−1.1*0.2−1.5*−0.2−0.2−0.4*0.3*0.3*
(0.5)(0.3)(0.6)(0.4)(0.3)(0.2)(0.1)(0.1)
RMSE31.80.62.10.80.71.71.01.0
Thei1 statistic40.30.30.60.60.40.40.50.7
β5−0.2−0.10.30.5*−0.1−0.1*0.10.1
(0.2)(0.1)(0.4)(0.2)(0.1)(–)(0.1)(0.1)
ρ60.9*−0.1−0.50.80.7−0.10.10.3
(0.5)(0.4)(1.0)(0.4)(0.2)(0.1)(0.1)(0.1)
(Year ahead forecast)
Average growth0.91.4−0.32.50.92.63.02.8
AFE2−4.0*−1.1*−2.3*−0.5−1.5*−0.9*0.10.1
(1.0)(0.5)(0.7)(0.5)(0.6)(0.2)(0.2)(0.2)
RMSE34.91.42.81.32.01.41.41.4
Theil statistic40.70.60.90.91.20.70.71.1
β5−1.0*0.2−0.11.0*1.2*−0.2*0.20.4
(0.5)(0.3)(0.5)(0.5)(0.3)(0.1)(0.2)(0.3)
ρ61.51.0*−0.51.21.1−0.10.2*0.6*
(1.6)(0.3)(0.3)(0.3)(0.2)(0.1)(0.1)(0.2)

The number of observations for individual years is seven; the number of observations for the longer time periods is seven times the number of years. The figures in parentheses are standard errors and an * indicates that the error is significantly different from zero at the 5 percent level of significance.

Average forecast error is defined as the realization less the forecast.

Root mean squared error.

Theil inequality statistic, defined as the ratio of the RMSE of the World Economic Outlook forecast to the RMSE of the random walk (last period realization) forecast. A ratio value of less than one indicates the World Economic Outlook forecast is better; a value greater than one implies the random walk forecast is better.

β is the estimated coefficent from a least-squares regression of the forecast error on the forecast.

ρ is the estimated coefficient from a least-squares estimate of the current period forecast error in the forecast error of the previous period.

The number of observations for individual years is seven; the number of observations for the longer time periods is seven times the number of years. The figures in parentheses are standard errors and an * indicates that the error is significantly different from zero at the 5 percent level of significance.

Average forecast error is defined as the realization less the forecast.

Root mean squared error.

Theil inequality statistic, defined as the ratio of the RMSE of the World Economic Outlook forecast to the RMSE of the random walk (last period realization) forecast. A ratio value of less than one indicates the World Economic Outlook forecast is better; a value greater than one implies the random walk forecast is better.

β is the estimated coefficent from a least-squares regression of the forecast error on the forecast.

ρ is the estimated coefficient from a least-squares estimate of the current period forecast error in the forecast error of the previous period.

Table 4.Pooled Forecast Error Statistics for Inflation Over the Business Cycles in the Seven Major Industrial Countries1(In percent)
197419801982199019911971-821983-911986-91
(Current year forecast)
Average growth12.811.28.74.24.29.04.03.9
AFE21.0−0.2−0.50.2−0.10.4*−0.10.1
(1.0)(0.8)(0.4)(0.3)(0.3)(–)(0.1)(0.1)
RMSE33.21.91.10.70.71.70.70.6
Theil statistic40.60.70.50.71.20.40.50.6
β5−0.20.20.30.30.1
ρ6(0.3)(0.2)(0.1)(0.2)(0.2)(–)(–)(0.1)
(Year ahead forecast)
Average growth13.211.28.74.24.29.14.03.9
AFE25.3*1.6−0.1*11.41.2*−0.30.2
(1.1)(1.3)(—)(0.5)(0.5)(0.3)(0.2)(0.2)
RMSE35.93.61.11.01.02.81.31.0
Theil statistic41.11.30.51.12.20.70.91.0
β50.70.7*−0.10.10.20.3−0.2*0.1
(0.5)(0.3)(0.1)(0.3)(0.4)(0.7)(0.1)(0.1)
ρ61.1*1.6*0.30.90.10.60.4*
(0.5)(0.6)(0.2)(0.7)(0.2)(0.1)(0.1)(0.1)

The number of observations for individual years is seven; the number of observations for the longer time periods is seven times the number of years. The figures in parentheses are standard errors and an * indicates that the error is significantly different from zero at the 5 percent level of significance.

Average forecast error is defined as the realization less the forecast.

Root mean squared error.

Theil inequality statistic, defined as the ratio of the RMSE of the World Economic Outlok forecast to the RMSE of the random walk (last period realization) forecast. A ratio value of less than one indicates the World Economic Outlook forecast is better; a value greater than one implies the random walk forecast is better.

β is the estimated coefficent from a least-squares regression of the forecast error on the forecast.

ρ is the estimated coefficient from a least-squares estimate of the current period forecast error in the forecast error of the previous period.

The number of observations for individual years is seven; the number of observations for the longer time periods is seven times the number of years. The figures in parentheses are standard errors and an * indicates that the error is significantly different from zero at the 5 percent level of significance.

Average forecast error is defined as the realization less the forecast.

Root mean squared error.

Theil inequality statistic, defined as the ratio of the RMSE of the World Economic Outlok forecast to the RMSE of the random walk (last period realization) forecast. A ratio value of less than one indicates the World Economic Outlook forecast is better; a value greater than one implies the random walk forecast is better.

β is the estimated coefficent from a least-squares regression of the forecast error on the forecast.

ρ is the estimated coefficient from a least-squares estimate of the current period forecast error in the forecast error of the previous period.

In 1986–91, the year ahead pooled projections of growth and both current year and year ahead projections of inflation were unbiased; however, the pooled projections underestimated growth in the current year by about 73 of 1 percentage point. Based on either the β or ρ statistics, the pooled projections for 1986–91 were inefficient, although the Theil statistics indicate that current year forecasts were generally superior to random walk forecasts.19 Moreover, the root mean square errors (RMSE) of the World Economic Outlook and the random walk projections for 1983–91 and 1986–91 were about half of those in 1971–82, except for the year ahead projections of growth.20

In the 1990–91 recession, the World Economic Outlook projections have been reasonably accurate, and in general they compare favorably with the forecasting record in past recessions. All of the growth and inflation projections in 1990 and 1991 were unbiased, with average projection errors not significantly different from zero, except for the year ahead projection for growth in 1991, which was overstated by a large margin. The error reflects the difficulty of predicting major turning points in economic activity; however, even this projection was better than the comparable estimates in the 1974 and 1982 recessions. Moreover, for current year growth and year ahead inflation, the unbiasedness of the 1991 projection was also a significant improvement over the projections for the 1974 and 1982 recessions. In general, the forecast errors are of broadly similar orders of magnitude for the current and the 1980 recessions, on the one hand, and the 1974 and 1982 recessions, on the other. This suggests that forecast errors may be related to the depth of the recession—with larger errors associated with the more severe recessions.

Industrial Country Groups

Table 5 presents the results for each group of industrial countries.21 It shows that deviations between the outcomes and both the current year and year ahead forecasts of output growth and inflation were small and not significantly different from zero. Although both were unbiased, forecasts for the current year were, not surprisingly, more accurate than for the year ahead.

Table 5.Industrial Countries: Deviations of Outcomes from Projections1(In percent)
Output GrowthInflation
Seven major countriesFourteen smaller countriesAll countriesSeven major countriesFourteen smaller countriesAll countries
(Current year forecast)
Average outcome2.92.22.72.97.16.1
AFP2−0.1−0.2−0.10.50.1
RMSE30.70.91.20.61.00.5
Theil statistic40.30.50.30.40.80.3
β50.1−0.2
ρ60.10.30.20.3−0.20.4*
(Year ahead forecast)
Average outcome2.92.42.96.07.16.1
AFE2−0.4−0.4−0.4−0.20.80.3
RMSE31.51.51.41.41.51.3
Theil statistic40.50.80.60.81.11.5
β5−0.40.1−0.3*1.1
ρ60.10.20.10.30.30.4*

The sample period is 1971–91 giving 21 observations, where, except for the estimates of p, the sample size is 20. An * indicates that the projection error is significantly different from zero at the 5 percent level of significance.

Average forecast error. The error is defined as the realization less the forecast.

Root mean squared error.

The Theil inequality statistic is defined as the ratio of the World Economic Outlook RMSE to the RMSE of the random walk forecast, which is its last period’s realization. A Theil inequality statistic less than unity implies that the World Economic Outlook projections are better than the random walk forecasts.

The symbol 0 is the estimated coefficient from a least-squares regression of the forecast error on the forecast.

The symbol ρ is the estimated coefficient from a least-squares estimate of the current period forecast error on the forecast error of the previous period.

The sample period is 1971–91 giving 21 observations, where, except for the estimates of p, the sample size is 20. An * indicates that the projection error is significantly different from zero at the 5 percent level of significance.

Average forecast error. The error is defined as the realization less the forecast.

Root mean squared error.

The Theil inequality statistic is defined as the ratio of the World Economic Outlook RMSE to the RMSE of the random walk forecast, which is its last period’s realization. A Theil inequality statistic less than unity implies that the World Economic Outlook projections are better than the random walk forecasts.

The symbol 0 is the estimated coefficient from a least-squares regression of the forecast error on the forecast.

The symbol ρ is the estimated coefficient from a least-squares estimate of the current period forecast error on the forecast error of the previous period.

For all industrial countries, year ahead forecasts of growth in 1971–91 overestimated actual growth by 0.4 percentage point on average, whereas the year ahead forecasts of inflation underestimated actual inflation by 0.3 percentage point. Artis found that the average forecast errors for output and inflation for 1973–85 were 0.5 percentage point. This suggests that the bias in the World Economic Outlook’s forecasts for industrial countries was reduced after 1985, probably because of the relative stability of output growth and inflation since 1985. In particular, there were enormous projection errors associated with the first oil price shock in 1973–74. The absolute average projection error for the group of 14 smaller industrial countries was higher than that for the major industrial countries, except for the average year ahead growth projection, which was the same for both groups.

The current year and year ahead projections of growth were efficient for the industrial countries. Current year inflation forecasts were efficient for the seven large countries and for the small countries.22 Year ahead inflation forecasts were efficient for the large countries but not for the small countries.23 The Theil statistics indicate that World Economic Outlook projections of growth and inflation for industrial countries were superior to random walk forecasts, except for inflation projections for the smaller countries.

Developing Countries

The conditionality of output and inflation projections is far more important for developing countries than for industrial countries, because many developing countries have adopted Fund-supported stabilization and structural adjustment programs. The World Economic Outlook projections of growth and inflation assumed the full implementation of the policies stipulated in the programs. The deviations between the conditional projections and outcomes are therefore partly a measure of the extent to which the policies specified in the programs were not fully implemented, or partly a reflection of assumptions about the international economic environment that have not always been explicitly realized. Moreover, the economic situation of program countries has tended to be, on balance, worse than that of nonprogram countries, making forecasting more difficult for the former group.

Table 6 presents the results of the statistical tests for the developing country groups as well as for 36 developing countries that were not engaged in Fund-supported programs. The tests performed for the nonprogram countries correspond more closely to the evaluation of unconditional growth and inflation forecasts and, hence, are more comparable with those for industrial countries. The results show that in many cases there have been significant deviations between projections and outcomes for growth and inflation, both on average across all developing countries and for individual groups. In particular, actual growth fell short of the projections, while inflation tended to exceed projected price increases. For the sample of nonprogram developing countries, both inflation and real output growth projections were unbiased in the 1988–91 period.

Table 6.Developing Countries: Deviations of Outcomes from Projections1(In percent)
Output GrowthIn Ration
AverageAfricaAsiaMiddle EastWestern HemisphereNonprogram pooledAverageAfricaAsiaMiddle EastWestern HemisphereNonprogram Pooled
(Current year forecast)
Average outcome3.52.15.72.92.24.945.519.18.922.4179.311.1
AFE2−0.5*−1.0*−0.3−0.7−0.211.1*2.5*1.7*0.277.7*
RMSE31.21.31.32.42.33.618.05.42.48.9156.05.7
Theil statists41.01.00.70.90.90.80.91.40.91.00.80.4
β5−0.3−0.5−0.3−0.3−0.6*0.9*−0.9*0.1−0.4*1.7*−0.2*
ρ60.5−0.1−0.20.30.10.6*−0.2020.10.5*
(Year ahead forecast)
Average outcome3.72.36.13.12.34.746.918.89.222.6200.213.7
AFE2−1.0*−0.8*0.2−1.3−1.7*−0.621.7*2.8*2.6*−1.1141.8*2.4
RMSE31.61.41.72.52.94.230.35.83.716.3237.07.1
Theil statistic41.21.10.91.31.10.91.11.11.41.71.10.7
β5−0.4−0.7*1.2−0.30.1−0.7*1.90.9*−0.8*0.8*5.9*−0.4*
ρ60.3−0.1030.40.40.5*−0.1*−0.10.30.5*

The sample period is 1977–91, except for the year ahead projections of inflation (1979–91) and for the nonprogram countries, for which pooled observations for 36 countries in 1988–91 (144 = 36 × 4 observations) are used. (Nonprogram countries are developing countries without arrangements with the IMF in 1988–91.)

Average forecast error. The error is defined as the realization less the forecast; an * indicates that the projection error is significantly different from zero at the 5 percent level of significance.

Root mean squared error.

The Theil inequality statistic is defined as the ratio of the World Economic Outlook RMSE to the RMSE of the random walk forecast, which is its last period’s realization. A Theil inequality statistic less than unity implies that the World Economic Outlook projections are better than the random walk forecasts.

The symbol β is the estimated coefficient from a least-squares regression of the forecast error on the forecast; an * indicates that the estimated coefficient is significantly different from zero at the 5 percent level of significance; that is, that the error is correlated with the forecast.

The sample ρ period is 1977–91, except for the year-ahead projections of inflation (1979–91) and for the nonprogram countries, for which pooled observations for 36 countries in 1988–91 (144 = 36× 4 observations) are used. (Nonprogram countries are developing countries without arrangements with the IMF in 1988-91).

The sample period is 1977–91, except for the year ahead projections of inflation (1979–91) and for the nonprogram countries, for which pooled observations for 36 countries in 1988–91 (144 = 36 × 4 observations) are used. (Nonprogram countries are developing countries without arrangements with the IMF in 1988–91.)

Average forecast error. The error is defined as the realization less the forecast; an * indicates that the projection error is significantly different from zero at the 5 percent level of significance.

Root mean squared error.

The Theil inequality statistic is defined as the ratio of the World Economic Outlook RMSE to the RMSE of the random walk forecast, which is its last period’s realization. A Theil inequality statistic less than unity implies that the World Economic Outlook projections are better than the random walk forecasts.

The symbol β is the estimated coefficient from a least-squares regression of the forecast error on the forecast; an * indicates that the estimated coefficient is significantly different from zero at the 5 percent level of significance; that is, that the error is correlated with the forecast.

The sample ρ period is 1977–91, except for the year-ahead projections of inflation (1979–91) and for the nonprogram countries, for which pooled observations for 36 countries in 1988–91 (144 = 36× 4 observations) are used. (Nonprogram countries are developing countries without arrangements with the IMF in 1988-91).

Table 7 reports the same tests for the subperiods 1977–85 and 1986–89. In the earlier period, real output growth fell short of the current year projections by 1.1 percentage points for the average of all developing countries, whereas growth exceeded the current year projections by 0.3 percentage point in the later subperiod for this group. For the year ahead projections, output growth fell short of the projections in both subperiods, but the average shortfall declined from 1.4 percentage points to 0.5 percentage point. The RMSE also fell substantially between the subperiods: for the current year forecasts, it fell from 1.5 percentage points to 0.5 percentage point; for the year ahead forecasts, from 2.0 percentage points to 0.9 percentage point. In view of the important assumption of policy implementation, this reduction may suggest improvement in the record of meeting program objectives after 1985, perhaps because of the recent progress toward strengthened policies in many developing countries.24 In addition, the economic environment has been more stable in recent years than in the late 1970s and early 1980s. In contrast, the average deviation between the outcome and the projection for inflation for the developing countries as a whole rose significantly between the periods. Although this result suggests a substantial departure from policy objectives, it must be interpreted with care, because the picture for inflation in 1986–91 was dominated by only a few countries.

Table 7.Developing Countries: Deviations of Outcomes from Projections by Subperiod1(In percent)
Output GrowthInflation
AverageAfricaAsiaMiddle EastWestern HemisphereAverageAfricaAsiaMiddle EastWestern Hemisphere
1977-85(Current year forecast)
Average outcome3.52.35.44.22.533.619.78.227.581.5
AFE2−1.1*−1.1*−0.3−0.7−1.2*4.1*2.11.3*−0.816.7*
RMSF31.51.41.32.32.64.76.31.611.319.6
Theil statistic41.01.00.81.11.01.51.60.81.01.1
β50.20.3−0.90.1−0.5−0.2−1.4*−0.2−0.5−0.3*
ρ60.2−0.2−0.2−0.2−0.2−0.5−0.50.5−0.8*
(Year ahead forecast)
Average outcome3.82.56.04.12.634.119.68.429.689.7
AFE2−1.4*−0.8*−0.2−1.7*−1.8*8.6*2.52.0*−3.640.0*
RMSE32.01.51.72.93.38.43.32.419.541.4
Theil statistic41.21.00.91.81.14.50.61.31.82.6
β50.1−0.4−1.4−0.20.4−1.0−0.7*−0.2−1.0*2.5*
ρ60.20.40.60.5−2.0*−0.40.40.31.1
1986-91(Current year forecast)
Average outcome3.52.85.05.53.263.318.09.714.9326.1
AFE20.3−0.8*0.50.5−0.121.5*3.22.2*1.6169.2*
RMSE30.51.01.22.51.727.73.83.33.6245.6
Theil statistic40.70.80.91.21.20.91.80.70.61.3
β5−0.4−0.1−0.6−1.61.80.9*−2.0*−0.20.1
ρ6−0.6−0.4−0.50.80.60.2—1.4*−0.21.1*
(Year ahead forecast)
Average outcome3.62.95.45.53.361.917.910.114.5329.1
AFE2−0.5−0.80.7−0.9−1.837.0*3.2*3.4*1.9260.5*
RMSE30.91.31.31.92.446.87.64.73.3345.3
Theil statistic40.91.00.91.71.21.10.71.31.02.3
β5−0.7−1.00.7−1.5*2.02.3−0.4−1.2−1.5*
ρ60.41.31.00.1−0.80.4−0.50.8*

The sample period is 1971–91 giving 21 observations, where, except for ρ, the sample size is 20.

Average forecast error. The error is defined as the realization less the forecast. An * indicates that the projection error is significantly different from zero at the 5 percent level of significance.

Root mean squared error.

The Theil inequality statistic is defined as the ratio of the World Economic Outlook RMSE to the RMSE of the random walk forecast, which is its last period’s realization. A Theil inequality statistic less than unity implies that the World Economic Outlook projections are better than the random walk forecasts.

The symbol β the estimated coefficient from a least-squares regression of the forecast error on the forecast.

The symbol ρ is the estimated coefficient from a least-squares estimate of the current period forecast error on the forecast error of the previous period.

The sample period is 1971–91 giving 21 observations, where, except for ρ, the sample size is 20.

Average forecast error. The error is defined as the realization less the forecast. An * indicates that the projection error is significantly different from zero at the 5 percent level of significance.

Root mean squared error.

The Theil inequality statistic is defined as the ratio of the World Economic Outlook RMSE to the RMSE of the random walk forecast, which is its last period’s realization. A Theil inequality statistic less than unity implies that the World Economic Outlook projections are better than the random walk forecasts.

The symbol β the estimated coefficient from a least-squares regression of the forecast error on the forecast.

The symbol ρ is the estimated coefficient from a least-squares estimate of the current period forecast error on the forecast error of the previous period.

The growth projections for the full sample and for both subsamples were generally efficient for the developing countries. In contrast, neither the current year nor year ahead inflation projections satisfied the efficiency tests. According to the Theil statistics, random walk forecasts were superior to the World Economic Outlook projections of both growth and inflation, although the statistics are somewhat lower for the 1986–91 period. The Theil statistics for the pooled sample of nonprogram countries, however, are well below unity, implying that the projections for these countries were superior to random walk forecasts. The difference in the Theil statistics between program and nonprogram countries may reflect unrealized policy objectives in the former.

Chart 3 shows the deviations between outcomes and projections for growth and inflation for the average of industrial countries and the average for developing countries. For output growth, the deviations tended to move in the same direction for both groups, although the magnitude was larger for the developing country group. Chart 3 also shows that the magnitude of the underprediction of inflation for developing countries increased significantly in the last four years of the 1980s due to sharply increasing inflation in a few countries, notably Brazil and Argentina.

Chart 3.Difference Between Outcomes and Projections in the World Economic Outlook Projections1

(In percent)

1Outcome minus year ahead projection.

Time-Series Forecasts

The World Economic Outlook makes judgmental projections that are based on an implicit view of how economic variables are mutually related. Alternatively, projections of the future values of a particular variable can be obtained from its past values. This method does not use any economic knowledge that may be available about a variable. Rather, a model is constructed for the stochastic process that generated the data. Time-series models that replicate output growth and inflation data for the seven major industrial economies are constructed below. The accuracy of the predictions of these models is evaluated using the rational expectations criterion presented earlier and then compared with the World Economic Outlook projections.

The processes generating output growth and inflation can often be represented in a form involving autoregressive (AR) and moving average (MA) components represented as follows,

where y is an economic variable and v is white noise.25 The AR and MA representations of the process characterize the systematic effect of past values of y and past errors, respectively, on the current value of y. Unit root tests suggest that output growth and inflation are stationary across the seven major industrial countries at a 5 percent significance level.26 Thus, among AR and MA representations that satisfied the stationarity and invertibility conditions,27 the process that minimized the Akaike (1974) and (1976) and Schwarz (1978) criteria was chosen as the data-generating process.28

Tables 8 and 9 present nonlinear least-squares estimates of the time-series representations of output growth and inflation for each of the seven major industrial economies in 1950–70.29Table 5 suggests that both past values of growth and past errors were significant determinants of growth in the current period. Indeed, the second lag of output growth was an important determinant of the processes generating growth across the seven major industrial economies, except for Germany (prior to unification). Last period’s growth was the only relevant part of past growth determining current growth in Germany and in the pooled sample. For most of these seven countries, the errors made one and two periods earlier also affect significantly present growth. Campbell and Mankiw (1987) characterize the univariate time-series model of the United States in terms of an AR and MA representation (2, 1) for 1969–84. Table 8, however, indicates that the forecast error made two periods ago is also a significant determinant of current growth in the United States and of the average of the seven major industrial economies.

Table 9 suggests that last period’s inflation accurately characterizes the current inflation rate across most of the major seven industrial countries.30 Inflation in France and Canada was also described by the previous period’s unanticipated inflation. The univariate time-series models of inflation presented in Table 9 are similar to the first-order models that Nelson and Schwert (1977), Pearce (1978), and Fama and Gibbons (1982) estimate on monthly data for the United States in the 1953–71, 1959–76, and 1953–77 periods, respectively.

Table 8.Time-Series Models of Output Growth1
United SlatesJapanGermany2FranceItalyUnited KingdomCanadaSeven Major Industrial Countries
AveragePooled
ARMA (p, q)(2.2)(2.1)(1.0)(2.2)(2.2)(2.0)(2.2)(2.2)(1.1)
p lags21.2121.21.21.221
q lags1.211.21.21.21.21
Constant31.96.41.89.34.64.32.79.2
(0.6)(1.5)(1.3)(0.6)(1.7)(0.8)(1.6)(0.8)
AR lag:4
10.80.70.70.10.7−0.3
(0.3)(0.2)(0.2)(0.2)(0.3)(0.1)
20.4−0.4−0.6−0.5−0.6−0.3−0.8
(0.1)(0.2)(0.1)(0.2)(0.2)(0.3)(0.1)
MA lag:5
11.2−0.9−0.1−0.3−1.0−0.50.6
(0.1)(0.3)(0.1)(0.1)(0.6)(0.2)(0.1)
2−0.80.90.8−0.70.6
(0.4)(0.2)(0.1)(0.3)(0.2)
Prob Q-statistic618.011.561.05.05.038.024.97.010.3

The sample period is 1950–70. In the pooled tests there are 147 observations.

Prior to unification.

Constant estimates and standard errors are in percent; standard errors are in parentheses.

AR is the autoregressive process of the ARMA model.

MA is the moving average process of the ARMA model.

Prob Q-statistic is the probability that a χ2 random variate will exceed the computed value. The hypothesis that the residuals are random noise is rejected if the marginal significance level is smaller than 5 percent. Figures are in percent. The lag length is selected by minimizing the Akaike and Schwarz criteria. The Akaike and Schwarz criterion statistics are 0.0 for each process.

The sample period is 1950–70. In the pooled tests there are 147 observations.

Prior to unification.

Constant estimates and standard errors are in percent; standard errors are in parentheses.

AR is the autoregressive process of the ARMA model.

MA is the moving average process of the ARMA model.

Prob Q-statistic is the probability that a χ2 random variate will exceed the computed value. The hypothesis that the residuals are random noise is rejected if the marginal significance level is smaller than 5 percent. Figures are in percent. The lag length is selected by minimizing the Akaike and Schwarz criteria. The Akaike and Schwarz criterion statistics are 0.0 for each process.

Table 9.Time-Series Models of Inflation1
United SlatesJapanGermany2FranceItalyUnited KingdomCanadaSeven Major Industrial Countries
AveragePooled
ARMA (p, q)(1.0)(1.0)(1.0)(1.1)(1.0)(1.0)(1.1)(1.0)(1.1)
p lags111111111
q lags111
Constant31.12.81.62.11.91.83.41.61.2
(0.6)(1.0)(0.9)(1.2)(1.0)(0.9)(0.8)(0.6)(0.4)
AR lag4
10.60.40.40.60.50.5−0.40.50.6
(0.2)(0.2)(0.2)(0.3)(0.2)(0.2)(0.2)(0.2)(0.1)
MA lag5
1−0.8

(0.2)
0.8

(0.2)
−0.3

(0.1)
Prob Q-statistic650.957.328.031.067.265.014.262.213.7

The sample period is 1950–70. In the pooled tests there are 147 observations.

Prior to unification.

Constant estimates and standard errors are in percent; standard errors are in parentheses.

AR is the autoregressive process of the ARMA model.

MA is the moving average process of the ARMA model.

Prob Q-statistic is the probability that a χ2 random variate will exceed the computed value. The hypothesis that the residuals are random noise is rejected if the marginal significance level is smaller than 5 percent. Figures are in percent. The lag length is selected by minimizing the Akaike and Schwarz criteria. The Akaike and Schwarz criterion statistics are 0.0 for each process.

The sample period is 1950–70. In the pooled tests there are 147 observations.

Prior to unification.

Constant estimates and standard errors are in percent; standard errors are in parentheses.

AR is the autoregressive process of the ARMA model.

MA is the moving average process of the ARMA model.

Prob Q-statistic is the probability that a χ2 random variate will exceed the computed value. The hypothesis that the residuals are random noise is rejected if the marginal significance level is smaller than 5 percent. Figures are in percent. The lag length is selected by minimizing the Akaike and Schwarz criteria. The Akaike and Schwarz criterion statistics are 0.0 for each process.

Forecasting Accuracy of the Time-Series Models

Table 10 presents statistics that measure the accuracy of within-sample forecasts generated by time-series models for each of the seven major industrial countries, for the GDP-weighted average of the seven countries, and for the pooled projections for the seven countries. Time-series forecasts were generated by using the models identified and estimated prior to the forecast sample 1971–91. The models were re-estimated recursively until the end of the sample.31Table 10 indicates that the forecast errors of output growth (real GDP/GNP) and inflation (GDP/GNP deflator) are all small and not significantly different from zero. Moreover, the ratio of the average forecast error to average growth across the seven major industrial economies is, on average, 15 percent, while for inflation, it is only 5 percent.

Table 10.Forecast Error Statistics for Time-Series Models1(In percent)
United SlatesJapanGermany2FranceItalyUnited KingdomCanadaSeven Major Industrial Countries
AveragePooled
(Output growth)
Average growth2.64.82.42.72.31.72.92.92.8
AFE3−0.6−0.1−0.50.3−0.7−0.5−0.3−0.1−0.2
(0.7)(0.6)(0.5)(0.5)(0.7)(0.4)(0.5)(0.4)(0.3)
RMSE41.81.52.32.21.82.02.51.31.9
β5−0.9−0.8−0.7−0.7*−0.9−0.6−0.9*−0.4−0.6
(0.6)(0.5)(0.7)(0.4)(0.6)(0.5)(0.5)(0.6)(0.4)
ρ60.3−0.20.3−0.30.5*0.20.30.2
(0.2)(0.2)(0.2)(0.2)(0.2)(0.3)(0.3)(0.2)(0.2)
(Inflation)
Average growth5.64.04.27.712.010.06.66.07.2
AFE2−0.1−0.6−0.6−0.2−0.9−0.4−0.1−0.3
(0.4)(0.9)(0.3)(0.6)(0.7)(1.1)(0.6)(0.4)(0.3)
RMSE41.72.81.31.82.72.12.11.82.0
β50.30.30.1−0.2−0.1−0.5−0.3−0.3−0.1
(0.2)(0.4)(0.3)(0.2)(0.2)(0.3)(0.2)(0.2)(0.1)
ρ60.20.10.20.5*0.5*−0.30.80.30.2
(0.2)(0.2)(0.2)(0.2)(0.2)(0.2)(0.3)(0.2)(0.1)

The sample period is 1971–91. The number of observations for the individual country tests is 21, except for the estimates of ρ, which are 20. In the pooled tests there are 147 observations, but 140 for estimation of ρ. The figures in parentheses are standard errors. An * indicates the error is significantly different from zero at the 5 percent level of significance.

Prior to unification.

Average forecast error is defined as the realization less the forecast.

Root mean squared error.

β is the estimated coefficent from a least-squares regression of the forecast error on the forecast.

ρ is the estimated coefficient from a least-squares estimate of the current period forecast error in the forecast error of the previous period.

The sample period is 1971–91. The number of observations for the individual country tests is 21, except for the estimates of ρ, which are 20. In the pooled tests there are 147 observations, but 140 for estimation of ρ. The figures in parentheses are standard errors. An * indicates the error is significantly different from zero at the 5 percent level of significance.

Prior to unification.

Average forecast error is defined as the realization less the forecast.

Root mean squared error.

β is the estimated coefficent from a least-squares regression of the forecast error on the forecast.

ρ is the estimated coefficient from a least-squares estimate of the current period forecast error in the forecast error of the previous period.

Year ahead growth and inflation forecasts are efficient, except for growth projections for France, the United Kingdom, and Canada, and inflation projections for France and Italy. The ρ and β tests suggest that the forecast error of inflation in these countries is correlated with the error lagged one period and with the forecast itself. It was suggested earlier that the significance of β is related to the failure to incorporate last period’s unexpected inflation into the inflation projected for the current period. These inefficiencies arise because in this exercise the Q-statistic test evaluates the serial correlation of the residuals lagged one through five periods, thereby reducing the significance of the correlation of lags closer to the current period. The tests for inflation in France and Italy provide an example of the care that should be exercised in drawing conclusions from the β statistic alone: the β statistic suggests that the pooled projections are efficient, but the ρ statistic shows that they are not.

Comparison of Forecasts

A comparison of the year ahead forecast statistics in Tables 1 and 2 with those in Table 10 indicates that the absolute average forecast error of growth and inflation generated by the time-series models is two thirds of the absolute average forecast error of the World Economic Outlook for most of the seven major industrial countries. The pooled growth projections of the time-series model are efficient, while those of the World Economic Outlook are not. Nonetheless, World Economic Outlook and time-series projections of growth for most individual countries and for the average of the seven major industrial economies are efficient. The efficiency restrictions are satisfied less often for pooled projections of the World Economic Outlook than those of the time-series models.

For comparison with time-series models and Artis (1988), the TS Theil and RW Theil statistics are used to compare the projections of the World Economic Outlook with those of time-series models and a random walk, respectively. The RW Theil statistics in Table 11 indicate that World Economic Outlook projections are superior to random walk forecasts. Not surprisingly, 17 out of 18 statistics are below unity. Theil statistics for the pooled projections show that the root mean squared error of the current year projections for growth and inflation are about half those of the random walk projections, while the year ahead projections are about 30 percent better than those of a random walk.

Table 11.Comparison of Forecast Errors1(In percent)
United SlatesJapanGermany2FranceItalyUnited KingdomCanadaSeven Major Industrial Countries
AveragePooled
(Output growth)
Theil statistic TS31.02.00.80.61.31.00.61.21.1
Theil statistic RW40.50.40.70.70.70.80.60.50.7
(Inflation)
Theil statistic TS30.81.20.70.91.11.41.10.81.1
Theil statistic RW40.70.80.60.81.00.50.90.80.7

The sample period is 1971–91. The number of observations for the individual country tests is 21. In the pooled tests there are 147 observations.

Prior to unification.

The Theil inequality statistic TS is defined as the ratio of the RMSE of the World Economic Outlook forecast to the RMSE of the time-series forecast. A ratio value of less than one indicates that the World Economic Outlook forecast is better; a value of greater than one implies that the time series forecast is better.

The Theil inequality statistic RW is defined as the ratio of the RMSE of the World Economic Outlook forecast to the RMSE of the random walk forecast. A ratio value of less than one indicates that the World Economic Outlook forecast is better; a value of greater than one implies that the random walk forecast is better.

The sample period is 1971–91. The number of observations for the individual country tests is 21. In the pooled tests there are 147 observations.

Prior to unification.

The Theil inequality statistic TS is defined as the ratio of the RMSE of the World Economic Outlook forecast to the RMSE of the time-series forecast. A ratio value of less than one indicates that the World Economic Outlook forecast is better; a value of greater than one implies that the time series forecast is better.

The Theil inequality statistic RW is defined as the ratio of the RMSE of the World Economic Outlook forecast to the RMSE of the random walk forecast. A ratio value of less than one indicates that the World Economic Outlook forecast is better; a value of greater than one implies that the random walk forecast is better.

The TS Theil statistics suggest that time-series forecasts of growth and inflation between 1971 and 1991 were superior to World Economic Outlook projections, except for growth projections for Germany, France, and Canada and inflation forecasts for Germany, France, the United States and the average of the seven major industrial countries. The efficiency of growth projections for the United States and the United Kingdom was the same for both forecasts. These results suggest that time-series forecasts provide a more accurate description of growth and inflation than the projections of the World Economic Outlook. Judgmental projections can be improved, however, if they are adjusted by β and ρ. It is shown next that this adjustment roughly corresponds to the error-correction mechanism present in time-series models.

An Adjustment Method

To examine the relevance of shocks in generating forecasts in time-series models, it is useful to calculate the fraction of unexpected growth and inflation for the previous year(s) that is incorporated into the growth and inflation projected for the current year. This exercise illustrates how time-series models incorporate previous errors to generate current forecasts. In the same fashion, a judgmental projection could be adjusted by ρ to improve the accuracy of the forecast. The time-series model that characterizes the process generating growth for the United States and the average of the seven major industrial countries is given by

This process can be described by its expected and unexpected components,

where E(y) is expected growth for the current year assessed four periods earlier, and the unexpected growth (ν) is serially uncorrelated white noise. Equations (12) and (13) imply

Thus, rewriting y-2 in terms of its expected and unexpected components, E–2 (y–2) and ν–2, yields

The estimates of the parameters α and θ in Table 8 for equation (15) indicate that the unexpected growth one and two years earlier are fully incorporated into the expected growth for the current year in the United States. Similar calculations for the time-series process of the average of the seven major industrial countries suggest that one half of the unexpected growth one period earlier is incorporated into the projection of growth in the current year. The time-series model for Germany incorporates two thirds of unexpected growth for the previous year, while France and Italy fully include last period’s unanticipated growth into current expected growth.

The time-series model that characterizes the process generating the inflation rate for most of the seven major industrial economies is given by

where the unexpected component of inflation is represented by η. Rewriting π-1 in terms of its expected and unexpected components, E-1(π-1) and η–1, yields

The estimates of the parameters α and θ in Table 9 indicate that the variance of expected inflation is small relative to the variance of unexpected inflation across most of the seven major industrial economies. Equation (17) then suggests that only three fifths of the unexpected inflation rate for the previous year is incorporated into the expected inflation for the current year in the United States. For Japan and Germany only two fifths of last period’s unexpected inflation is included in the projection of inflation in the current period. This figure is about one half for Italy, the United Kingdom, and the average of the seven major industrial economies. Thus, a failure to make adjustments for large errors reduces significantly the accuracy of a projection.

Concluding Remarks

This paper presents a simple criterion to evaluate the accuracy of a forecast. The criterion is derived from a simple optimization problem under rational expectations. The optimality conditions of this problem imply that the standard efficiency conditions are necessary, but not sufficient. A key feature of the method is the characterization of how different kinds of inefficiency are mutually related. The forecasting accuracy of the World Economic Outlook and time-series models are examined according to this criterion. The results of a number of empirical tests for industrial countries support the following conclusions.

  • The World Economic Outlook current year forecasts of growth and inflation for the seven major industrial countries are unbiased for 1971–91. The current year forecasts of growth reflect an important structural change between 1971–82 and 1983–91. In particular, the 1971–82 forecasts of growth are biased upward, whereas those for 1983–91 are biased downward.

  • The World Economic Outlook year ahead projections overstate growth and understate inflation by ½ of 1 percentage point. This bias occurs because year ahead forecasts overstated growth and understated inflation in 1971–82. After 1982, however, year ahead projections of both growth and inflation are unbiased across the seven major industrial economies.

  • Only current year forecasts of inflation are efficient. Current and year ahead forecasts of growth and year ahead projections of inflation are inefficient in the sense that the projections could be improved by adjusting them by a forecast error correlation factor (ρ) or a projection factor (β), or both.

  • The accuracy of the World Economic Outlook projections for growth and inflation improved after 1985, the last year fully analyzed in the earlier study by Artis. This improvement may partly reflect a more stable environment in the 1980s than in the more volatile 1970s.

  • In the 1990–91 recession, the World Economic Outlook projection errors are lower than in the two previous cyclical downturns, and the projections were generally unbiased, which is a distinct improvement over the forecasts for 1974 and 1982. Possible reasons for this difference are that supply shocks did not play a central role in the current recession and that the current recession has been relatively shallow compared with the other two. Nevertheless, the World Economic Outlook projections failed to anticipate the full extent of the current downturn.

  • Time-series forecasts for 1971–91 are unbiased and efficient, with the exception of inflation forecasts for three countries. The absolute value of the average growth and inflation errors generated by time-series models were half of those derived for the World Economic Outlook.

  • The analysis suggests that a fraction of the unexpected inflation rate and growth are included in the time-series projections for the current year across the seven major industrial countries.

Time-series forecasts outperform the projections of the World Economic Outlook. This suggests that the accuracy of the World Economic Outlook could be improved by the use of such model-based methods. By contrast, the criterion presented in this paper allows a constructive analysis of the projection error that may improve forecast performance over time. Indeed, judgmental projections could be improved significantly if the β and ρ adjustments are included in the projections. Moreover, the model could readily be extended to include assumption-based adjustments. These kinds of adjustments resemble the error-correction mechanism present in time-series models and, thus, provide a more accurate description of future economic activity.

The tests performed for developing countries suggest the following conclusions.

  • There were significant deviations between outcomes and projections of growth and inflation before 1985, but they were small for growth projections in the 1986–91 period. Although the economic environment has been more stable, the improvement in forecast accuracy suggests that policy assumptions have been more frequently met in recent years.

  • The average deviation between the outcome and the projection of inflation for the developing countries as a whole rose significantly between 1977–85 and 1986–91. Only a few countries, however, dominated this result.

  • For the sample of nonprogram developing countries, both inflation and real output growth projections were unbiased in the 1988–91 period.

  • The growth projections were generally efficient for the developing countries. In contrast, neither the current year nor year ahead inflation projections were efficient.

  • Theil statistics suggest that random walk forecasts were superior to the World Economic Outlook projections of both growth and inflation for the groups of developing countries that include program countries. The Theil statistics for the pooled sample of nonprogram countries, however, suggest that the projections for these countries were superior to random walk forecasts. This suggests that there were unrealized policy objectives for some program countries.

Bibliography

    AkaikeH.“A New Look at Statistical Model Identification,”IEEE Transactions on Automatic Control AC-19 (1974). pp. 71623.

    AkaikeH.“Canonical Correlation Analysis of Time Series and the Use of an Information Criterion,” in Systems Identification: Advances and Case Studiesed. by R.K.Mehra and D.G.Lainiotis (New York: New York Academic Press1976) pp. 52107.

    ArtisM.J.“How Accurate is the World Economic Outlook? A Post Mortem on Short-Term Forecasting at the International Monetary Fund,”Staff Studies for the World Economic Outlook World Economic and Financial Surveys (Washington: International Monetary Fund1988) pp. 149.

    BlakeD.M.Beenstock and V.Brasse“The Performance of U.K. Exchange Rate Forecasters,”Economic JournalVol. 96 (1986) pp. 98699.

    CampbellJohn and GregoryMankiw“Are Output Fluctuations Transitory?”Quarterly Journal of EconomicsVol. 102No. 4 (1987) pp. 85780.

    CampbellJohn and PierrePerron“Pitfalls and Opportunities: What Macroeconomists Should Know About Unit Roots” Research Memorandum No. 360 (Princeton: Princeton University, Economic Research ProgramApril1991).

    FamaEugene F.Foundations of Finance (New York: Basic Books, Inc. 1976).

    FamaEugene F. and Michael R.Gibbons“Inflation, Real Returns, and Capital Investment,”Journal of Monetary EconomicsVol. 9 (1982) pp. 297323.

    FrenkelJ.A.“The Forward Exchange Rate, Expectations, and the Demand for Money: The German Hyperinflation,”American Economic ReviewVol. 67No. 4 (1977) pp. 65370.

    FrenkelJ.A. and AssafRazin“Stochastic Prices and Tests of Efficiency of Foreign Exchange Markets,”Economic LettersVol. 6No. 2 (1980) pp. 16570.

    HansenLars P. and Robert J.Hodrick“Forward Exchange Rates as Optimal Predictors of Future Spot Rates: An Econometric Analysis,”Journal of Political EconomyVol. 88No. 5 (1980) pp. 82953.

    HoldenK. and D.A.PeelExpectations: Theory and Evidence (London: Macmillan Press1985).

    HoldenK. and D.A.Peel“On Testing for Unbiasedness and Efficiency of Forecasts,”Manchester School of Economic and Social StudiesVol. 58 (June1990) pp. 12027.

    LevichR.M.“Tests of Forecasting Models and Market Efficiency in the International Money Market” in The Economics of Exchange Rates: Selected Studiesed. by J.A.Frenkel and H.G.Johnson (Reading, Massachusetts: Addison-Wesley1978).

    McNessS.K.“The Rationality of Economic Forecasts,”American Economic ReviewVol. 68No. 2 (1978) pp. 301305.

    MurfinA. and P.Ormerod“The Forward Rate for the U.S. Dollar and the Efficient Markets Hypothesis, 1978–1983,”Manchester School of Economic and Social StudiesVol. 54 (September1984) pp. 29299.

    MuthJ.F.“Rational Expectations and the Theory of Price Movements,”EconometricaVol. 29 (1961) pp. 31535.

    NelsonCharles R. and WilliamSchwert“Short Term Interest Rates as Predictors of Inflation: On Testing the Hypothesis That the Real Rate of Interest Is Constant,”American Economic ReviewVol. 67 (1977) pp. 47886.

    PearceDouglas“Comparing Survey and Rational Measures of Expected Inflation: Forecast Performance and Interest Rate Effects,”Journal of Money Credit and BankingVol. 11 (1978). pp. 44756.

    RomerChristina“The Prewar Business Cycle Reconsidered: New Estimates of Gross National Product, 1869–1918,”NBER Working Paper No. 1969 (July1986).

    SchwarzG.“Estimating the Dimension of a Model,”Annals of Mathematical Statistics 6 (1978) pp. 46164.

    TheilHenriPrinciples of Econometrics (New York: John Wiley & Sons1971).

    UrichT. and P.Wachtel“The Structure of Expectations of the Weekly Money Supply Announcement,”Journal of Monetary EconomicsVol. 13 (1984) pp. 18394.

The author is grateful to Charles Adams, Peter B. Clark, David T. Coe, Robert Ford, Alexander Hoffmaister, Fleming Larsen, Bennett T. McCallum, Enrique Mendoza, Steven A. Symansky, and Michael A. Wattleworth for helpful comments and discussions. Any remaining errors are the author’s responsibility.

Fama (1976) introduces the concept of weak information sets which consist of current and past values of a random process. All variables are denoted in the current period, unless stated otherwise.

See, for example, Artis (1988).

See Murfin and Ormerod (1984) for a discussion of this issue.

The model is consistent with linear rational expectations. It could, however, also accommodate nonlinear specifications by choosing a different g function.

Note that if the average forecast error is not zero, the projection is inefficient even if both β and p are zero.

Least-squares regressions are used to estimate β and ρ.

These countries are Australia, Austria, Belgium, Denmark, Finland, Greece, Ireland, Netherlands, New Zealand, Norway, Portugal, Spain, Sweden, and Switzerland.

Eastern European countries and the states of the former U.S.S.R. are not included in the developing country group.

Nonprogram countries are those that were not engaged in Fund-supported stabilization and structural adjustment programs in 1988–91. The requisite data are available for 36 of these countries.

This period was chosen because growth and inflation projections start in 1971. The pooled time-series/cross-section sample for the current year forecasts have 147 (= 7× 21) observations, while the year ahead forecasts have 140.

For the pooled projections, the average forecast error is only 3½ percent of actual average output growth (0.1 of 1 percentage point compared with average growth of 2.7 percent) and about 2¾ percent of the actual average inflation (0.2 of 1 percentage point compared with average inflation of 7.1 percent).

The predicted variation is β2σF2, where σF2 is the variance of the forecast.

This implies, for example, that if ρ is significant the projection could be improved by adjusting for last period’s error.

Although 1971–82 is somewhat longer than the 1974–82 business cycles, the earlier years were included because the forecast errors in 1971–73 were similar to those in 1974–82 (cf. Charts 1 and 2).

The Theil inequality statistic is defined as the ratio of the root mean squared error of the World Economic Outlook forecast to the root mean squared error of the random walk forecast. If this ratio is less than one, it indicates that the World Economic Outlook forecast is better; that is, it has a lower average error than the random walk forecast. The random walk forecast for next period is the current period’s realization. These statistics are provided for comparison with Artis (1988).

Because growth and inflation are likely to be stationary; that is, they tend to revert to a fixed mean. It should not be difficult for a judgmental projection to outperform a random walk that does not have this property.

The outcomes for the seven major countries are taken from the World Economic Outlook, May 1992 (Washington: International Monetary Fund, May 1992), Annex VIII, pp. 88–93.

These forecasts barely fail the efficiency tests for the industrial countries as a group, the statistic is 1.82 and the critical value is 1.80.

For example, since p is significant in the case of the inflation forecasts of the group of industrial countries, the projection could be improved by adjusting for last period’s error.

See Chapter IV of the World Economic Outlook, October 1992 (Washington: International Monetary Fund, 1992).

θ(L) = (1 – θ1 · Lθ2 · L2 – … –θp · LP) and α(L) = (1–α1 · L–α2 · L2– … –αq · Lq), where p and q are the orders of the AR and MA processes.

Unit roots tests were performed using the augmented Dickey-Fuller and the augmented Phillips-Perron for the 195091 period. The number of lags included in each of these tests was chosen following Campbell and Perron (1991).

Stationarity requires that θ1, + θ2 + … +θp < 1, while invertibility requires that the roots of the characteristic equation α (L) = 1 -α1, · L2 · L − α2 · L2 – … – αq = 0 must all lie outside the unit circle.

Under this, the lag length is selected by minimizing the functions (RSS + 2· K-SEESQ)/T (Akaike) and [RSS + K(logT) · SEESQ]/T, where K and Tare the number of regressors and observations, respectively, and RSS is the residual sum of squares and SEESQ is the standard error of the estimate squared.

Nonlinear least-squares estimations were performed using the Gauss-Newton algorithm with numerical partial derivatives and annual data. The data prior to 1971 were obtained from the International Finance Statistics tape of the International Monetary Fund. For Germany and Italy, data for 1950–55 were obtained from the OECD National Accounts. The data for developing countries were not available prior to 1965.

Interestingly enough, the model for inflation not only describes essentially the same process across the seven countries, but the parameters estimates are also fairly close to each other.

Models estimated over the entire 1950–91 period were also consistent with these specifications.

    Other Resources Citing This Publication