Chapter

I How Accurate Is the World Economic Outlook? A Post Mortem on Short-Term Forecasting at the International Monetary Fund

Author(s):
International Monetary Fund
Published Date:
January 1988
Share
  • ShareShare
Show Summary Details
Author(s)
*

Recent developments in the sphere of international economic policy coordination produced an agreement at the May 1986 Tokyo Summit that the major countries should focus on a set of economic indicators as a means of strengthening the degree of cooperation in macroeconomic policymaking already in existence. The Fund was given the formal responsibility for carrying this suggestion forward. In the subsequent development of this idea (see. in particular, Crockett and Goldstein (1987)), emphasis has been given to a taxonomy of indicators of current economic developments, distinguishing those which are signals of policy posture from those which measure intermediate variables, and which in turn are distinguished from those measuring economic performance. Indicators may be used in a number of ways. On a rising scale of increasing international interdependence, they may provide individual countries with a checklist of variables against which to monitor the short-run progress of their economies; they may provide information on the medium-run sustainability of policies; and they may signal in a formal way the need for multilateral discussion of policies.

Because of the lags in the economic process, it is clear that indicators of current developments cannot be a substitute for forecasting; on the contrary, for any of the purposes listed above forecasts are needed for the evolution of the relevant indicators. Here, the relevance of the present study should become apparent. Students of the analytics of economic policy coordination (see, for example. Cooper (1984)) have long stressed the significance of agreement about propositions in positive economics to the success of international policy coordination: that agreement must embrace both the evaluation of responses of performance indicators to policy indicators and the baseline forecast evolution of the indicators. The Fund’s World Economic Outlook (WEO) has long been in the business of projecting the latter, making forecasts of the development of the performance indicators subject, essentially, to starting assumptions about policies. For the successful functioning of an indicator system, the degree of forecasting accuracy must be tolerably good, given the alternatives. This setting provides more than adequate motivation for an examination of the Fund’s forecasting track record as distilled from the projections published in the World Economic Outlook and publications of the same kind that were circulated internally within the Fund for nearly a decade before regular publication began in 1980.1

Conclusions

The scope of this study was restricted to the examination of the World Economic Outlook forecasting record for the principal performance indicators for the major industrial countries and corresponding aggregates and for groups of non-oil developing countries. Moreover, the analysis only covered the short-term forecasts—that is, those covering the current year and one year ahead. Necessarily set on one side has been the mass of detail presented in the forecasts pertaining to the components of national expenditure and the medium-term “scenarios” formulated in recent years.

Several criteria were used in evaluating the forecasts: the computation and evaluation of various summary statistics of forecast accuracy, bias, and efficiency; comparisons with alternative forecasts—naive forecasts and forecasts produced by the Organization for Economic Cooperation and Development (OECD) and by national forecasting agencies; the examination of turning-point errors and forecast performance in defined episodes; and, finally, some attempt to explain forecast error in terms of unanticipated developments in policy variables and oil prices.

In judging the forecast performance of the World Economic Outlook, a number of points must be kept in mind. Most important, it has to be recognized that the period since the inception of the World Economic Outlook as a regular forecasting exercise has been extraordinarily rich in economic upheavals, which have made the odds against accurate forecasting formidable. It should also be recalled that the objective of the World Economic Outlook is not to forecast the most likely outcome but rather to provide conditional estimates of economic developments under the assumption of unchanged policies and exchange rates. Indeed, the quintessential purpose of the World Economic Outlook exercise is to assist the Fund in carrying out bilateral as well as multilateral surveillance by helping to identify tensions in the projections that may call for policy adjustments or may result in exchange rate changes. Finally, it must remain true that the standard of accuracy required in a forecast is relative to the task for which the forecasts are required. Adapting the results recorded here to this criterion must remain an exercise for the interested reader.

Against this background, the forecast performance appears to have been reasonably accurate, particularly for output and inflation, with the industrial country forecasts generally more accurate than those for the developing countries. Although the average absolute error for year-ahead forecasts of growth in the seven major industrial countries as a group over the 1973–85 period has been slightly above 1 percentage point, this result is strongly influenced by a few outliers attributable to some of the large disturbances experienced over the period, notably the first major round of oil price increases in 1973–74 and the tight monetary conditions in the early 1980s. Excluding 1974 and 1982, the mean absolute error for year-ahead output growth in the seven major industrial countries has been only 0.7 of 1 percentage point. While the biggest errors can be related to the large fluctuations in oil prices, the significance of unforeseen fiscal policy developments in explaining forecast error seems strictly limited. The results also show that forecast accuracy is quite sensitive to forecast lead time, so that the errors typically diminish as more information becomes available, particularly for the industrial countries.

The forecasts for output growth, both for industrial and for developing countries, appear to have suffered from a degree of “optimism bias” in the sense that output forecasts have been mostly on the high side in relation to realized values. This bias in an optimistic direction is concentrated in the 1971–80 period. It undoubtedly reflects the fact that the slowdown in growth in this period was only gradually perceived to be a break in trend growth rather than primarily a cyclical phenomenon. Since 1980, output forecast errors have been more evenly distributed. Moreover, there is little evidence of inefficiency in the World Economic Outlook forecasts, in the sense that the forecast errors cannot systematically be explained by the level of the forecast itself and are not obviously statistically biased. The World Economic Outlook forecasts also appear to be efficient in that they are generally incapable of being improved by adding information from the available forecasts produced by the OECD or by national forecasters.

As between the performance indicators considered, forecasts for the current account of the balance of payments are inferior to those for output and inflation, at least for the industrial countries. This result, which might appear surprising for forecasts that are prepared on an internationally consistent basis, must be qualified in two important respects. First, the current account is the balance of very large gross flows of exports and imports (of goods, services, and transfers); even small errors in the growth estimates of exports or imports may thus show up as large errors in the current account. The second qualification concerns the questionable quality of balance of payments statistics as evidenced by the large and highly volatile discrepancy in the world current account, particularly since the late 1970s. As argued in a recent report on this problem (International Monetary Fund (1987)), large fluctuations in the current account discrepancy have undoubtedly been an important source of error, not only for balance of payments forecasts but also for world growth projections.

On the track record of the World Economic Outlook relative to national forecasting agencies, comparisons are hampered by differences in source dates of available forecasts, making generalization hazardous. However, it would appear that the World Economic Outlook forecasts do not generally provide any distinct improvement over those of national agencies in forecasting national output growth and inflation. Indeed, one outcome of the comparisons with other forecasters is the finding that there is a high degree of common sharing in the principal forecasting errors. The largest of these are traceable to the two large oil price rises, especially to the first (1973–74), though there are turning-point errors outside of these episodes which also appear to be widely shared by national and international forecasters. Although this may seem disappointing, it would be a mistake to conclude that international forecasts such as those of the World Economic Outlook are therefore redundant. Indeed, in most cases, the national forecasts are prepared on the basis of assumptions about each country’s international environment that typically originate from forecasting exercises like those of the Fund or the OECD. Projections prepared on an internationally consistent basis are also necessary as an input into Fund multilateral surveillance activities and into any attempt to coordinate economic policies among countries. Such projections are also required as a basis for monitoring the situation of the indebted developing countries.

The question remains whether the World Economic Outlook’s forecasting accuracy can be significantly improved. It would probably not be helpful to be overly ambitious in this respect. Nevertheless, the results of this study do suggest that there may be scope for improvement in several areas. In particular, it seems clear that a reduction in the magnitude and, more especially, the volatility of the world current account discrepancy could only enhance the quality of an internationally consistent exercise such as the World Economic Outlook. An early implementation of the recommendations contained in the recent report on the discrepancy may well be a necessary condition for significant improvements in the accuracy of the projections. Another conclusion that has emerged from this report is the sensitivity of forecast accuracy to lead time, which underlines the importance of being able promptly to take into account any new information that becomes available. This raises the question of whether the accuracy of the World Economic Outlook could be improved by a more widespread use of formal, model-based methods which would reduce processing time and would allow more frequent ad hoc updates of the forecasts. The ready availability of such methods would permit a given baseline projection based on the judgment of individual desk officers to be adjusted incrementally at short notice for changes in the main exogenous assumptions underlying the projections and would ease the task of providing scenario analyses. Whilst a move in this direction should not be expected to yield early dividends, it is probably also true that a more formal methodology, simply by being more explicit, more easily allows constructive post mortem analysis of forecast error which should help to improve forecast performance over time.

Forecasting Methods and Criteria for Evaluation

Methods

This is not the place to explain the construction of the World Economic Outlook forecasts in detail. (Goldstein (1986) may be consulted for such a description.) But it is essential to spell out some of the principal characteristics of the forecasting process, for these affect the post mortem techniques that can be used.

First, it is important to stress that the forecasts are conditional. They are prepared on certain assumptions about “exogenous” variables: fiscal and monetary policy, exchange rates, and oil prices are the leading variables in question. The basic assumption about policies is that “present policies” will be held unchanged during the forecast period, though “present policies” are interpreted to include any currently known announcements about future policy adaptations and may also “encompass certain policy adaptations or changes that seem likely to occur even though they have not been announced by the authorities.”2 Exchange rates are currently projected at the real (formerly, nominal) levels prevailing at a recent base date, whilst the oil price is also usually projected (in the absence of more specific indicators otherwise) as constant in real terms.

The reasons these variables are treated in this particular way are mixed. The treatment of policies follows the customary practice of national official forecasting and its many derivatives where a prime originating purpose of the forecasting exercise is to provide a consistency check on policy itself. A similar justification applies here too and the World Economic Outlook draws conclusions for desirable policy adjustment from its analysis of the future outlook. For market-based policy instruments such as interest rates, World Economic Outlook projections must also be inhibited by the knowledge that a Fund forecast might move the market in a way which could force the hand of a member government, which would be an embarrassing prospect.3 Somewhat similar considerations may affect the treatment of oil prices, but the practice of the World Economic Outlook here is like that of other forecasters and to this extent reflects a belief that predicting the timing and magnitude of changes in oil prices is a particularly hazardous undertaking.

The fact that World Economic Outlook forecasts are conditional on assumptions about policy and oil prices suggests that, subject to measurement problems, it is important to allow for the falsification of the conditional assumptions in reviewing the track record (see the last section of this paper). The position on exchange rates is rather different. The typical conditional projection in the World Economic Outlook of an unchanged pattern of exchange rates cannot be defended on the argument that exchange rates are a policy instrument, at least not one independent of fiscal and monetary policy, but rather because the undoubted power of the Fund to “move markets” would make the publication of exchange rate forecasts inappropriate.

Strictly speaking, since policy adjustment is not allowed to take the strain of supporting the pattern of exchange rates assumed and the exchange rate is not allowed to take the strain of supporting the set of policies assumed, the collection of conditional assumptions the World Economic Outlook is forced to make about these variables can only be squared with theoretical considerations by invoking “portfolio shifts” of just the right type and magnitude to sustain them. In principle, nothing is more likely than that this assumption of accommodating portfolio shifts will fail. But this cannot mean that it would be right to treat deviations of exchange rates from their “forecast” paths as a reason for forecast error elsewhere. First of all, the failure of the exchange rate assumption to materialize may reflect a failure of other parts of the forecast just as much as the other way around; second, pragmatically but most importantly, despite theoretical considerations, the power of structural models to predict the exchange rate is extremely low. In practice, it is not clear that the conditional exchange rate baseline projection—which is, after all, a form of random walk prediction—can be significantly bettered. Finally, also pragmatically, exchange rate effects take a considerable time to work through onto output (although less time onto prices); they could have little impact within the typical short-term forecast period. On the other hand, it seems fair to say that failures of the exchange rate assumption may have more rapid and noticeable effects on balance of payments forecast errors, and since these turn out to be the most problematic part of the track record, some attempt to relate them to exchange rate forecast errors seems worthwhile.

A second important characteristic of the World Economic Outlook forecasting exercise is that it is comparatively informal, much more so, say, than the leading forecasting models in the United Kingdom. (For a recent review of these the interested reader may consult Wallis et al. (1986).) While model-based exercises are conducted at various stages of the production of the forecast, there is no computer-based “world model” behind the forecast as a whole. This is not necessarily a drawback in itself, but the implication for post mortem analysis is that it is not possible to decompose an ex post forecast error into exogenous variable, judgmental, and model-based error in the way that would be appropriate, and feasible, for a model-based exercise (see Osborn and Teal (1979) for an original exercise of this type). Nevertheless, it should be possible, measurement problems permitting, to relate the forecast errors to exogenous variable errors, as discussed below.

A third important characteristic of the World Economic Outlook forecast procedure is that it has, at its heart, a consistency check not shared by national forecasters. As described in Goldstein (1986), original country-desk-based forecasts, prepared against environmental assumptions specified by the Fund’s Research Department, are aggregated to check for the consistency of their trade and balance of payments implications. Identified discrepancies are then removed by an iterative process in which the country desk forecasts are successively revised, until the check is satisfied. The opportunity, and indeed the need, to conduct this check obviously arises from the closed economy nature of world forecasting which contrasts with the open economy basis of national forecasting. It would be useful to identify a way of confirming the value of consistency checks. One approach might be to compare the ex post accuracy of the initial and final forecasts made in each round, but the records available do not allow this comparison to be made. An additional problem relates to the fact that there is a significant discrepancy in the world current account, which may reduce the value of the consistency check.

Criteria

Given the selection of variables to be examined, the principal tools used for assessing the World Economic Outlook forecasts in the sections below comprise the following: inspection of forecast error summary statistics; investigation of systematic bias in the forecasts taken over a long period; comparison with alternative forecasts; and the investigation of the rationality of forecast error. These checks are supplemented by an identification of outstanding episodes in the track record and an attempt to explain these in more detail by recourse to narrative material. Some explanation of these tools is in order.

Summary statistics. The principal summary statistics deployed in examining the World Economic Outlook track record are the average absolute error of forecast, the root mean square error, and the Theil inequality statistic. Because a forecast error series may display both positive and negative errors, the simple mean may be a highly misleading indicator of accuracy, and for this reason the average absolute error is preferred. As a basis for comparing this statistic among series, the mean absolute value of the realized series itself is also presented. It is commonplace in economic analysis to prefer a measure which penalizes a large deviation more highly than a series of smaller ones of equivalent total size; for analytical tractability a quadratic measure is often used and for this reason the root mean square error (RMSE) is a preferred statistic in studies like the present one. This statistic too needs to be normalized in some fashion to facilitate comparison between series and the study makes use of such a normalization in the Theil inequality statistic, which can be generally defined as the ratio of the RMSE of the forecast under consideration to the RMSE of an alternative forecast. In the main text tables displayed below, this alternative is provided by the naive “no change” forecast, where the forecast for year t of variable x is the t –1 value for x (where x may be the growth rate of real GDP or the rate of inflation).4 (Appendix IV also presents the Theil statistics produced by the alternative naive standard that the forecast of x for year t corresponds to the ten-year moving average of x.)

Realization—forecast regressions. The efficiency of forecasting may be tested by performing the regression of the realizations on the forecasts themselves, as R(t) = a + b F(t) + u(t). A perfect forecast would identify the intercept in such a regression as zero, the slope as unity and yield a correlation coefficient of 1.00. Where knowledge of the realization-forecast relationship itself can reduce the forecast error variance, these conditions will not hold. It seems a natural interpretation, within the terms of this regression framework, to identify a failure of the two expectations about the intercept and slope terms with the presence of bias; but this inference is not necessarily correct.5 The essence of the matter is that the realization-prediction regression detects whether the pattern of forecast errors can be related to the level of the forecast, not whether the average error is significantly different from zero, which can be tested for directly by measuring the average error and asking whether it is significantly different from zero.6 In answering some of these questions it is useful to supplement the results that can be obtained for specific countries (areas or aggregates) by pooling the data. While this procedure permits the benefit of offsetting country error, it enhances the power of significance tests.

Comparisons. “Absolute” measures of forecast accuracy are useless in themselves; they need to be related, on the one hand, to the standards of accuracy required by the purpose for which they are sought and, on the other, to comparable measures generated by alternative forecasting techniques. In the latter category, the normalizations already noted compare the forecasts with those generated by two alternative prediction schemes. It would be possible also to generate univariate time series and multivariate (Bayesian vector autoregression) models as a further source of alternative forecasts; ex post facto it might well prove possible to generate a model in this class which would be superior to the World Economic Outlook forecasts, but the achievement would not be very interesting because the alternative model does not represent a feasible alternative forecasting technique. Even if models of this class, possessing superior forecasting qualities, could be built on a purely ex ante bias, their usefulness and plausibility would be in doubt if they did not enforce consistency and could not accommodate variation for policy or environmental change. Given these drawbacks, this type of alternative was not explored.

The alternative actual forecasts with which the World Economic Outlook forecasts are compared here are those produced by the OECD and, following Llewellyn and Arai (1984), by a set of national forecasters. With the OECD the comparison is with another international agency producing forecasts of a nearly comparable scope in country coverage, assumptions, and detail. However, a difficulty with both types of comparison is that it is not possible to align the forecast dates exactly and so differences in the information sets conditioning the forecasts inescapably contaminate the comparisons.

Explanations for Forecast Errors

Given the conditional nature of the forecasts, explained above, testing the rationality of the forecasts involves assessing the contributions of “innovations” (unexpected changes) in the exogenous variables. The principal difficulties in implementing this approach are measurement problems. While it is possible to derive a reasonably satisfactory series for the innovations in oil prices, it is less easy to do this for fiscal policy and appears not to be feasible for monetary policy. In the case of fiscal policy the problem is less conceptual than practical: series of fiscal policy anticipations and outturns exist, but are for one reason or another less than satisfactory. For monetary policy there is the substantial conceptual problem that indicators like the growth rate of the money supply reflect not only policy but the economy more generally. While measures of fiscal policy like the fiscal impulse or the structural budget balance attempt to normalize for the influence of the economy, no comparable measure exists for monetary policy. Hence, even though fiscal policy and oil prices are not the only driving variables in world economic forecasting, for practical reasons it is only the contribution of innovations in these variables to explanations of the forecast error that is assessed. In a fully “rational” forecast these variables should only appear in the form of current innovations; neither tagged innovations nor actual values should in principle explain current errors if the forecasters have fully taken on board the implications of previous changes and have a correct model of the significance of their own current anticipations of these variables for those they are forecasting.7

The role of systematic analysis of the complete time series of forecasts is not to avoid the challenge of historical analysis of forecast error so much as to provide a context for it and to avoid the trap of choosing specific explanations which fit the facts in any one episode but have no overall power to improve the forecasts in general. Moreover, there is a dimension of forecasting quality which lends itself best to graphical and narrative analysis and this is the question of turning-point error. An allegedly common failing of forecasts is their failure to spot the significant cyclical turning points.

Data Base

The data base used in this study comprises the forecasts in the published versions of the World Economic Outlook and similar data from the earlier comparable unpublished documents. The nature of this data base, in terms of the forecast horizons used and regularity of the forecast exercises conducted, is indicated in Table 1. This shows that while there has been some irregularity in forecast production dates, particularly up to 1982, there has nearly always been a forecast for the year in question produced in the second quarter. An earlier forecast for the year has generally been available in the fourth, and often as early as the third, quarter of the previous year. In the last two years, the first forward look has been taken even earlier, with forecasts for the following year appearing as early as April.8 Besides producing a main forecast, there have been many occasions when uncertainties about principal conditioning variables (such as oil prices and exchange rates) have been felt to be sufficiently acute as to warrant the production of variant “scenarios.”

Table 1.Forecast Horizon Content of World Economic Outlook
World Economic Outlook Date(s)Forecast Horizon
January 12, 19711971
May 27, 19711971
April 13, 19721972
January 31; February 22; March 1, 19731973
June 14; August 9, 19731974
December 21, 1973; January 4 and 31, 19741974
March 14, 19741974
May 22, 23, and 24; June 21, 19741974
December 24 and 31, 19741975
March 31, 19751975
May 21 and 23, 19751975
December 12 and 15–16, 19751976
July 7 and 9; August 11, 19761976
February 22 and 24; March 2–3, 197719771
June 29; July 5 and 11, 19771977
December 27, 19771978
April 3–4, and 10, 19781978
September 6, 197819781
December 1, 19781979
February 9, 13, and 15, 19791979
June 11, 13, and 15, 19791979
August 30, 19791980
May 198021980
August 22, 19801981
June 198121981
August 24, 19811982
April 198221982
August 2, 19821983
May 198321983
August 19; September 16, 19831984
April 198421984
September 198421985
April 198521986
October 198521986
April 198621987
October 198621987
April 198721988

The content of the World Economic Outlook is extraordinarily rich; forecasts are produced not simply for the principal variables of interest in the main countries, but in considerable detail both for these economies and for regional and analytical groupings embracing the entire world economy (with the exception of the U.S.S.R. and other countries of Eastern Europe that are not members of the Fund). In order to make progress in assessing the accuracy of the forecasts, it is necessary to make a number of decisions about which variables and forecasts to exclude.

The identification of forecast error plainly requires a definition of the outturn or realization with which the forecast can be compared. Because of the incidence of revisions of economic data, there is more than one possible series of realizations that might be chosen. Investigators generally take the view that the purpose of forecasts is to be right about the true evolution of the economy and that, at any point in time, that is most nearly revealed by the latest, revised-to-date series of data. This view, though clearly quite a persuasive one, is perhaps too superficial. The latest available set of data is not homogeneous in vintage: early data are many times revised, while latest data are perhaps still preliminary or partial estimates. Re-basing economic series may make it quite inappropriate to use the latest available data as a check on the forecast: the latter will have been formulated on data with a different base, and different properties, and it may not be feasible to reconstruct the data on a consistent base.9 Then again, policy (and short-term forecasting post mortems) will inevitably be based on early, not subsequently revised, data. For all these reasons there is room for choice about the realization series to be used and expedient criteria may legitimately affect the decision. In the present study (as detailed in the next section), three types of realization are deployed, which one is in play at any time being made clear in context. None of the more general conclusions arrived at appears to depend on the particular choice of realization series, though some conclusions are drawn from experiments involving the use of a specific series which were not, or could not be, replicated on the latest available set as was done for all the processing described in the next section.

Forecasting Accuracy

In this section we consider the accuracy of World Economic Outlook forecasts of principal variables over the whole available period, using a selection of the standard criteria discussed in the previous section. This discussion covers three topics: first, the variables selected for study; second, the forecasts for industrial countries; and third, the forecasts for developing countries.

Variables

As indicated in the previous section, World Economic Outlook forecasts embrace a large number of variables for several individually specified countries and aggregate groupings of various kinds. To be useful, a study must suppress secondary detail and select a primary set of variables. Recent discussion of the use of indicators in multilateral surveillance draws attention to the relationship between indicators and the transmission mechanism of economic policy. Thus, a conventional view of the latter directs attention to indicators of policy input (as, for example, the structural budget balance) at one end of the transmission mechanism and indicators of performance at the other (such as output growth, inflation, or the balance of payments); in between stand intermediate variables such as the exchange rate and perhaps interest rates. This study concentrates on the indicators of economic performance, measured by real GNP/GDP growth, GNP/GDP deflator or consumer price inflation, and the current account of the balance of payments. In addition, because the global context of World Economic Outlook forecasts lends trade a particular interest—World Economic Outlook trade forecasts often being cited by national forecasters—export and import volume growth and the development of the terms of trade are also investigated.

The country coverage of the projections examined also needs to be determined. Here, the institutional importance of the Group of Seven major industrial countries, their weight in world output and trade (in 1984–85, 56.9 and 53.5 percent, respectively), and the fact that the World Economic Outlook has consistently provided forecasts for the Group of Seven members individually, dictate that the forecast record for each of these countries and for the group as a whole should be examined. At the same time, aggregates for the industrial countries as a whole and for “Europe” as a group can also be easily and usefully examined. In addition to the industrial countries, the developing countries need also to be examined; none of these is as large in combined trade and output weight as the smallest of the Group of Seven countries, and World Economic Outlook forecasts have traditionally distinguished various groupings of the developing country bloc. The longest standing of such groupings and thus the most amenable to analysis over a reasonably long period of time are the regional groupings, where among the non-oil bloc, Africa, Asia, Europe, the Middle East, and the Western Hemisphere are separately distinguished.

Finally, a choice has to be made of horizon of forecast and vintage of realization or outturn data to be employed. Table 1 gives a brief summary of the projection content of successive World Economic Outlook iterations; the variable dates of these imply that whatever selection is made, no set of forecasts is homogeneous in its timing relative to the forecast horizon. However, a distinction was drawn between two groups of broadly homogeneous forecasts—current-year forecasts, where the forecast for year t is made during the year t itself—and year-ahead forecasts where the forecast for year t is made in year t –1.10

In practice even this distinction proved an ideal rather than a rigorously enforceable practice, as the actual sourcing for the two categories of forecast shown in Table 2 illustrates. The current-year forecasts are considerably more homogeneous in timing, varying by only three months from April to July at the maximum, compared to a maximum variation of seven months from August to March (of the following year!) in the case of the year-ahead forecasts.11 The additional variability nevertheless seemed a price worth paying to obtain a reasonably long series. In the choice of outturn data, the main analysis deploys two categories. For the current-year forecasts, the outturn is identified with the “first available” estimate, the figure reported in the following year’s World Economic Outlook; in the case of the year-ahead forecasts, however, the outturn is identified with the “first settled” estimate, that available in the World Economic Outlook of the following-year-but-one (that is, the year-ahead forecast for 1980 is compared with the outturn data published in the forecast source in 1981). These choices of outturn data had certain specific advantages over the use of latest available estimates: first, the definition of some of the aggregates were changed over the course of time, and the use of these outturn data enabled the resultant inconsistencies to be minimized or even eliminated in a way which would not have been so straightforward with latest available data. Second, the combination of “first settled estimates” as outturn data with the year-ahead forecasts allowed these to be compared with OECD and national forecasts prepared on a similar basis for the paper by Llewellyn and Arai (1984) and extended in the present study. Latest available data were nevertheless used in replication of all the principal computations of the main analysis; a summary of these results appears in Appendix II.

Table 2.Classification of World Economic Outlook Forecasts
Current-Year ForecastsYear-Ahead Forecasts
May 27, 1971January 12, 1971
April 13, 1972Not available for 1972
June 14, 1973January 31, 1973
May 23, 1974December 24, 1973
May 23, 1975December 24 and 31, 1974
July 7–9, 1976December 12, 15, and 16, 1975
June 29; July 5, 1977March 3, 1977
April 3–4, 1978December 27, 1977
June 11, 13, and 15, 1979February 15, 1979
May 1980publishedAugust 30, 1979
June 1981August 22, 1980
April 1982August 24, 1981
May 1983August 2, 1982
April 1984August 19, 1983
April 1985September 1984published
October 1985April 1986
October 1986
Note: Dates refer to those of World Economic Outlook documents, published where stated, otherwise unpublished. The publication lag is generally one to two months.

Summary Statistics: Industrial Countries

Tables 3 to 6 provide evidence of the track record of World Economic Outlook forecasting based on averaging over the whole period: 1971–86 for the current-year forecasts, 1973–85 for the year-ahead forecasts.

Table 3.World Economic Outlook Forecast Accuracy: Industrial Countries’ Output Growth(In percent)
CanadaUnited

States
JapanFranceFederal

Republic of Germany
ItalyUnited

Kingdom
Group

of

Seven
Total

Industrial

Countries
Europe
Current Year (1971–86)
Mean absolute actual
value3.7443.4444.9382.9502.6812.8752.0563.1633.0252.469
Average absolute error0.9060.8881.2380.7131.0501.1060.9310.6190.6250.606
RMSE1.3381.0631.7161.2081.4121.4701.1900.7610.7570.996
Theil’s inequality
statistic0.3880.2780.3930.4390.4480.3620.4440.2440.2570.382
Regression: intercept–0.4160.1060.241–0.321–0.629–0.603–0.034–0.027–0.398–0.686
(0.66)(0.28)(0.26)(0.55)(1.09)(0.95)(0.10)(0.83)(1.18)(1.47)
slope1.0210.9100.8791.0160.9831.2110.8191.0811.0361.106
(0.14)(0.98)(0.74)(0.09)(0.10)(0.95)(1.47)(0.02)(0.39)(0.67)
R¯20.7510.8660.6510.6980.6790.6550.7420.8990.8930.759
Year Ahead (1973–85)
Mean absolute actual
value3.3623.4314.6922.4772.6622.9622.3853.0312.8852.338
Average absolute error1.8001.4541.7921.0851.6311.9381.3921.1301.1081.238
RMSE2.2382.0473.2171.0852.2072.4821.7791.7131.6571.744
Theil’s inequality
statistic0.5960.5040.8240.6480.6560.6270.6730.5440.5550.658
Regression: intercept–1.2440.0202.723–0.671–2.8980.263–0.686–0.811–0.943–1.455
(0.79)(0.02)(1.25)(0.61)(1.69)(0.20)(0.78)(0.76)(0.89)(1.08)
slope1.1030.8860.3090.9701.6010.7471.0851.0661.0921.222
(0.26)(0.49)(1.87)(0.09)(1.12)(0.59)(0.24)(0.23)(0.31)(0.49)
R¯20.3480.525–0.0260.3890.3970.1430.4100.5120.5110.340
Note: The definitions of current-year and year-ahead forecasts are discussed in the text and in Appendix I. Mean absolute actual value is defined as Σ|Ri|/n where Ri is the realization (“actual”) in year i and n the number of years in the sample; mean absolute error is Σ|FiRi|/n where Fi is the forecast for year i. RMSE is Σ(FiRI)2/n and Theil’s inequality statistic is RMSE(F)/RMSE(F,a), where F,a is a naive “no change” forecast. The regression data are for the regression of R, on F and figures in parentheses are t-statistics: those for the intercept test against difference from zero; those against the slope for differences from unity.
Table 4.World Economic Outlook Forecast Accuracy: Industrial Countries’ Inflation(In percent)
CanadaUnited

States
JapanFranceFederal

Republic of

Germany
ItalyUnited

Kingdom
Group

of

Seven
Total

Industrial

Countries
Europe
Current Year (1971–86)
Mean absolute actual
value7.3696.0635.1128.7814.59413.91311.1635.6506.8318.238
Average absolute error1.1360.5441.5750.9500.6881.3061.7810.5560.4250.744
RMSE1.5670.7242.4181.2910.8621.9712.2360.6980.6100.971
Theil’s inequality
statistic0.6260.3510.5290.7220.4980.6110.3430.3600.3260.430
Regression: intercept–0.148–0.173–0.0381.852–0.1912.832–0.2800.1850.3051.183
(0.13)(0.31)(0.04)(1.81)(0.26)(2.20)(0.22)(0.34)(0.61)(1.41)
slope1.0761.0250.9300.8271.0360.8371.0760.9730.9700.904
(0.50)(0.30)(0.53)(1.49)(0.24)(1.79)(0.71)(0.35)(0.43)(0.93)
R¯20.7650.9050.7620.7660.7580.8470.8670.9130.9270.834
Year Ahead (1973–85)
Mean absolute actual
value8.2856.7695.3929.6084.38515.23111.9697.3467.4388.823
Average absolute error1.9381.2922.9381.6380.5153.0312.3231.2151.1001.162
RMSE2.7311.7254.2001.2000.6863.5743.2251.7571.6391.479
Theil’s inequality
statistic0.9270.7410.8310.8360.4291.1050.4940.7940.7730.663
Regression: intercept1.4781.2850.3963.388–1.3328.290–2.4330.9171.0640.201
(0.50)(0.73)(0.18)(1.75)(2.07)(2.31)(1.14)(0.47)(0.55)(0.10)
slope0.9440.8650.8580.7411.2450.4911.3500.9220.8991.052
(0.14)(0.51)(0.46)(1.26)(1.81)(2.09)(1.89)(0.29)(0.38)(0.22)
R¯20.2810.4430.3640.5010.8750.2030.8130.4760.4680.608
Note: For definitions etc., see Note to Table 3.
Table 5.World Economic Outlook Forecast Accuracy: Industrial Countries’ Export Growth(In percent)
CanadaUnited

States
JapanFranceFederal

Republic of

Germany
ItalyUnited

Kingdom
Group

of

Seven
Total

Industrial

Countries
Current Year (1972–86)
Average absolute error3.3072.5604.1702.5732.8473.7532.3531.8871.893
RMSE5.0713.4335.3783.2853.4534.6482.9722.4932.469
Theil’s inequality
statistic0.5210.3660.4750.4390.3780.6210.4250.3660.376
Regression: intercept0.115–0.4561.443–0.424–2.1891.317–0.423–0.668–0.345
(0.06)(0.47)(0.68)(0.25)(1.44)(0.58)(0.30)(0.60)(0.29)
slope1.1581.2470.9371.0481.4160.6151.0551.1821.079
(0.55)(1.82)(0.24)(0.19)(1.88)(1.22)(0.19)(0.99)(0.39)
R¯20.5190.8570.4630.5320.7410.1660.4700.7440.663
Year Ahead (1972–85)
Average absolute error3.4362.4434.4642.5862.8863.9292.4642.0113.015
RMSE5.2333.3725.5673.3393.5214.7993.0682.5883.917
Theil’s inequality
statistic0.5190.3580.4800.4280.3400.6270.4290.3670.569
Regression: intercept0.288–0.7391.741–0.126–1.9761.512–0.367–0.688–2.520
(0.14)(0.75)(0.71)(0.07)(1.23)(0.62)(0.25)(0.57)(0.77)
slope1.1561.2490.9081.0221.4010.5991.0511.1811.315
(0.52)(1.88)(0.32)(0.09)(1.75)(1.20)(0.17)(0.94)(0.59)
R¯20.5190.8710.4040.5180.7380.1460.4640.7400.296
Note: For definitions etc., see Note to Table 3.
Table 6.World Economic Outlook Forecast Accuracy: Industrial Countries’ Import Growth(In percent)
CanadaUnited

States
JapanFranceFederal

Republic of

Germany
ItalyUnited

Kingdom
Group

of

Seven
Total

Industrial

Countries
Current Year (1972–86)
Average absolute error5.2334.1873.9202.3532.4334.2502.9333.1412.740
RMSE6.3445.1574.6442.7593.1575.2383.4223.3653.016
Theil’s inequality
statistic0.5080.2020.2790.2300.5300.4220.4300.2630.276
Regression: intercept–0.112–0.150–2.652–0.3660.205–3.675–0.826–1.321–1.446
(0.05)(0.04)(1.72)(0.36)(0.14)(1.89)(0.63)(0.96)(1.22)
slope1.3011.2741.2591.1620.8291.4231.2291.2651.314
(0.90)(1.46)(1.54)(1.16)(0.86)(1.54)(0.99)(1.34)(1.69)
R¯20.5050.7640.7980.8300.5400.650.660.740.78
Year Ahead (1972–85)
Average absolute error5.2503.7203.7502.4142.5364.3713.0073.0493.954
RMSE6.4294.5074.5032.8283.2575.3803.5053.2814.929
Theil’s inequality
statistic0.5190.3020.3000.2490.5410.4390.4280.3520.566
Regression: intercept–0.579–0.919–3.233–0.4090.209–3.635–1.008–1.684–4.306
(0.25)(0.65)(2.25)(0.39)(0.14)(1.81)(0.73)(1.25)(1.26)
slope1.3321.2881.2661.1580.8291.4501.2381.2731.573
(0.97)(1.79)(1.74)(1.09)(0.83)(1.56)(1.00)(1.43)(1.01)
R¯20.5210.8280.8390.8280.5370.6500.6670.7690.358
Note: For definitions etc., see Note to Table 3.

Subject to a finding (see below) of some bias when the data are pooled, the track record for output growth forecasts, in the first table, is by a small margin the best of the three. The current-year forecasts show comparatively low average absolute errors compared to the mean absolute value of the series, while the Theil statistics indicate that the root mean square error of the forecasts is only some 20–40 percent, in typical cases, of the error that would be incurred by a “naive” forecaster. The realization-forecast regressions provide no indication of inefficiency.12 As might be expected, the current-year forecasts are superior by these criteria to the year-ahead forecasts, where the RMSEs and Theil statistics are higher, the fit of the realization-forecast regression poorer, and average absolute errors in relation to actual mean absolute values higher than they are in the current-year forecasts. Even so, these results appear fairly satisfactory: the Theil statistics are all well below unity, and the average absolute errors are well below the mean absolute value of the output growth series itself.

The track record for inflation (Table 4) is marginally less satisfactory than that for output, though still overall highly acceptable. The superiority of the current-year forecasts again stands out. These forecasts display, with the single exception of Germany, smaller average absolute errors, lower RMSEs, and lower Theil statistics than the year-ahead forecasts. The current-year forecasts provide no evidence of inefficiency, yielding a good fit in the realization-forecast regressions. The year-ahead forecasts provide a poorer fit in these regressions, and for Italy indicate inefficiency; for this same country, moreover, the Theil statistic exceeds unity. Elsewhere, however, the general run of evidence is favorable, even if the performance is not so good as in the nearer-term horizon of the current-year forecast or the comparable output forecasts.

Turning to the evidence on export and import volume forecasts, Tables 5 and 6, the track record now suggests little difference between the current-year and year-ahead forecasts (though the current-year statistics for imports are better than those for exports). In terms of overall quality, both appear equally good, with low Theil statistics suggesting generally that these forecasts provide a distinct improvement on the naive standard. There is no evidence of inefficiency and the overall fits of the realization-forecast regressions are on the whole not unreasonable except for the export growth statistics for Italy.

The record for balance of payments forecasts in Table 7 is considerably less reassuring than for output and inflation. The Theil statistics, especially for the year-ahead forecasts, are notably high, showing that the forecasts are little better than a naive projection, while the average absolute errors are high in relation to the absolute mean values, and in two of the year-ahead forecasts (for France and the Group of Seven) are actually somewhat higher. The realization-forecast regression suggests inefficiency in two cases, both for the year-ahead (Group of Seven and Total Industrial) and for the current-year (Canada and Japan) forecasts, whilst the overall explanatory power of the forecast is rated very low, at least in the year-ahead forecasts. There is, it is important to note, a considerable improvement in forecast accuracy when the forecast horizon is reduced—though the current-year forecasts are still markedly inferior to forecasts of a corresponding term for output or inflation.

Table 7.World Economic Outlook Forecast Accuracy: Industrial Countries’ Balance of Payments on Current Account(In billions of dollars)
CanadaUnited

States
JapanFranceFederal

Republic of

Germany
ItalyUnited

Kingdom
Group

of

Seven
Total

Industrial

Countries
Current Year1 (1973–86)
Mean absolute actual value2.86925.62313.3464.1236.8004.5154.68519.977
Average absolute error2.30010.3796.1802.1214.2002.5792.76412.479
RMSE3.08013.3297.3642.9895.3592.9963.88614.652
Theil’s inequality
statistic0.9910.5880.4980.5650.5770.4710.6710.579
Regression: intercept–1.6640.1970.764–0.6640.8450.5260.377–3.924
(1.77)(0.50)(0.39)(0.68)(0.52)(0.53)(0.34)(0.81)
slope0.3481.1421.1620.8441.0591.0140.9290.732
(2.60)(1.80)(2.03)(0.81)(0.37)(0.06)(0.31)(1.58)
R¯20.0660.9410.9420.5810.7670.6120.5490.573
Year Ahead2 (1973–85)
Mean absolute actual value2.84623.46213.6623.7009.0774.5396.16914.51515.562
Average absolute error1.32311.8387.8853.8005.7464.2854.19217.76221.220
RMSE1.93216.6409.5074.5697.1555.5585.15620.32325.260
Theil’s inequality
statistic0.7640.8170.8400.9041.0580.9440.9230.9120.967
Regression: intercept–0.151–4.9021.028–1.3654.424–1.2951.721–1.151–5.628
(0.20)(0.97)(0.31)(1.03)(1.46)(0.86)(1.08)(0.22)(0.92)
slope0.7681.0391.0730.3330.4780.4091.0930.4140.155
(1.06)(0.27)(0.34)(1.77)(1.47)(1.39)(0.28)(2.09)(2.63)
R¯20.4860.8070.669–0.0190.063–0.0060.450.090.068
Note: For definitions etc., see Note to Table 3.
Table 8.World Economic Outlook Forecast Accuracy: Terms of Trade and Trade Volumes(In percentage changes)
All Industrial CountriesTotal

World

Trade
Industrial

Countries’

Terms of Trade
ImportsExports
Current Year (1972–86)
Average absolute error2.7401.8931.6671.177
RMSE3.0162.4691.9931.522
Theil’s inequality statistic0.2760.3760.3110.319
Regression: intercept–1.446–0.3491.3530.427
(1.22)(0.29)(1.55)(1.64)
slope1.3141.0791.2081.345
(1.69)(0.39)(1.42)(4.69)
R¯20.7800.6630.8400.968
Year Ahead (1972–85)
Average absolute error3.9543.0153.5692.185
RMSE4.9293.9174.3353.150
Theil’s inequality statistic0.5660.5690.6080.547
Regression: intercept–4.306–2.520–4.021–0.620
(1.26)(0.77)(1.06)(1.04)
slope1.5731.3151.4893.075
(1.01)(0.59)(0.77)(3.86)
R¯20.3580.2960.2720.725
Note: For definitions etc., see Note to Table 3.

The relative weakness of the balance of payments forecasts revealed by these summary statistics is not unexpected and is in line with experience at a national level and with the OECD forecasting track record (see the next section). The problem is evidently related to the sizable fluctuations that have been observed in the world current account discrepancy, particularly since the late 1970s. As the reasons for this discrepancy are imperfectly understood, World Economic Outlook projections have to be based on an implicit assumption of relative stability in the projected path of the discrepancy.13 It should also be noted that the current account is the difference between two large flows, each of which has a volume as well as a price component. Relatively small forecast errors in any of the underlying volume or price changes can thus induce relatively large errors in the absolute difference between the nominal flows. Finally, as discussed in Appendix VI, exchange rate innovations may at times have contributed to the errors in current account projections.

Table 8 shows the forecasts for world trade and industrial countries’ terms of trade. (The statistics on industrial countries’ export and import volume forecasts are repeated in the table for convenience.) The overall record for trade conveyed in these figures is one in which the shortening of the forecast horizon contributes greatly to accuracy, with a marked decline in the error statistics (the RMSE and average absolute error more than halve between the year-ahead and current-year forecasts) and a sharp rise in the explanatory power of the forecasts. Turning to the terms of trade, there is again a marked improvement in quality as the forecast horizon is reduced, yet in both cases there is strong evidence of inefficiency, with the forecasts underestimating whenever the terms of trade improve significantly. Inspection of the time series of the errors shows that, while there are large positive forecast errors associated with the two rounds of large oil price increases, these are more than offset by persistent negative forecast errors in the subsequent periods.

Table 9.World Economic Outlook Forecast Accuracy: Non-Oil Developing Countries’ Output Growth(In percent)
AfricaAsiaEuropeMiddle

East
Western

Hemisphere
Total

Non-Oil

Developing

Countries
Current Year (1977–86)1
Mean absolute actual value2.2205.3202.4674.0803.4403.370
Average absolute error1.311.100.661.631.920.96
RMSE1.3551.4260.8142.1972.6051.218
Theil’s inequality statistic0.9420.9460.7961.0891.0081.055
Regression: intercept–1.9034.9481.136–0.630–0.0560.155
(1.48)(1.15)(1.36)(0.34)(0.03)(0.09)
slope1.2460.0650.5180.9870.7730.827
(0.65)(1.24)(1.56)(0.04)(0.51)(0.45)
R¯20.520–0.1240.1840.3870.1860.289
Year Ahead (1979–85)2
Mean absolute actual value2.3435.3862.1504.1863.4713.629
Average absolute error1.501.831.232.662.641.66
RMSE1.6812.1241.3003.1823.7361.947
Theil’s inequality statistic1.0751.3700.7381.9331.1671.206
Regression: intercept0.47525.4812.911–1.793–3.5663.404
(0.17)(5.10)(1.27)(0.35)(0.67)(0.84)
slope0.597–3.390–0.2550.9601.3180.047
(0.56)(5.22)(1.66)(0.05)(0.27)(1.14)
R¯2–0.0980.718–0.2150.0470.039–0.200
Note: For definitions etc., see Note to Table 3.

The direct tests for bias in the forecast errors, in the sense of significant differences from zero, are reported in full in Appendix III where, in addition to the processing of individual country data, tests are also conducted on the pooled set of errors. The results suggest a degree of output optimism in World Economic Outlook forecasts. Although individual country output forecast errors are not significant, they are predominantly of the same sign, so that upon pooling, a significant amount of bias is suggested—on the order of 0.3 percent in the current-year forecasts and somewhat higher at about 0.5 percent in the year-ahead forecasts (if the 1974 error is excluded from these). The output optimism appears to have been most pronounced in the second half of the 1970s, undoubtedly reflecting the fact that the deceleration in growth in many countries was only gradually perceived as a break in trend, rather than as a cyclical downturn. On the other hand there appears to be no bias in the inflation forecasts, at least not if the 1974 year-ahead error is excluded. Interestingly, the inflation and output errors are significantly negatively correlated in the pooled data set, lending support to the contention (see Kenen and Schwartz (1986)) that the implicit World Economic Outlook forecasts for nominal income are more robust than those for either real output growth or inflation.

Summary Statistics: Developing Countries

Statistics of the forecasting record for developing countries are presented in Tables 913. These plainly show a much poorer track record than that for the industrial countries.

Table 10.World Economic Outlook Forecast Accuracy: Non-Oil Developing Countries’ Inflation(In percent)
AfricaAsiaEuropeMiddle

East
Western

Hemisphere
Total

Non-Oil

Developing

Countries
Current Year (1977–86)
Mean absolute actual value19.2408.40026.77025.83082.01036.860
Average absolute error4.311.815.096.5116.136.31
RMSE5.9882.1576.41110.73018.9157.059
Theil’s inequality statistic1.1090.6540.9401.0040.7520.898
Regression: intercept22.212–0.1902.84310.503–5.441–2.851
(3.29)(0.13)(0.32)(1.18)(0.98)(0.58)
slope–0.1731.2691.1040.5761.3271.300
(3.04)(1.25)(0.25)(1.41)(4.17)(1.89)
R¯2–0.0970.7900.4090.2150.9690.880
Year Ahead (1979–85)1
Mean absolute actual value19.6148.42924.01729.62989.70038.114
Average absolute error2.8002.3148.66716.05739.99012.629
RMSE3.7492.70311.06522.07249.96514.767
Theil’s inequality statistic0.5841.0341.3481.7852.5413.042
Regression: intercept5.4252.60853.38148.196–62.253–24.322
(0.89)(0.64)(2.09)(4.07)(1.89)(0.55)
slope0.8300.898–1.272–0.5573.0572.450
(0.49)(0.17)(1.76)(4.69)(3.17)(0.83)
R¯20.4370.159–0.0070.2320.7790.140
Note: For definitions etc., see Note to Table 3. Inflation is measured by consumer price indices.
Table 11.World Economic Outlook Forecast Accuracy: Non-Oil Developing Countries’ Export Growth(In percent)
Current Year (1981–86)1Year Ahead

(1977–85):

Total

Non-Oil

Developing

Countries
AfricaAsiaEuropeMiddle

East
Western

Hemisphere
Total

Non-Oil

Developing

Countries
Mean absolute actual value3.6338.6506.6005.6504.8005.8456.189
Average absolute error3.155.904.128.535.002.143.08
RMSE4.2706.6194.8949.4345.7262.6733.804
Theil’s inequality statistic0.9860.7400.5911.2040.8800.5280.845
Regression: intercept–0.94817.0573.0500.802–6.8650.7952.621
(0.15)(1.42)(0.29)(0.13)(2.39)(0.40)(0.25)
slope0.674–1.1930.689–0.2651.3440.8530.146
(0.27)(0.12)(0.16)(1.00)(0.86)(0.47)(0.44)
R¯2–0.160–0.107–0.213–0.2370.6750.320–0.142
Note: For definitions etc., see Note to Table 3. Regional details for year-ahead data are not available.
Table 12.World Economic Outlook Forecast Accuracy: Non-Oil Developing Countries’ Import Growth(In percent)
Current Year (1979–86)1Year Ahead

(1977–85):

Total

Non-Oil

Developing

Countries
AfricaAsiaEuropeMiddle

East
Western

Hemisphere
Total

Non-Oil

Developing

Countries
Mean absolute actual value4.6256.3503.6007.5388.7385.5205.050
Average absolute error4.9131.8383.4715.62510.1753.2003.744
RMSE6.2493.0225.7037.07410.7114.1345.548
Theil’s inequality statistic1.2460.6461.0290.9620.9130.6271.037
Regression: intercept–3.5200.6873.500–3.873–6.2280.5402.621
(2.00)(0.22)(1.40)(1.49)(1.54)(0.31)(0.25)
slope–0.8860.801–0.6621.0661.1720.8320.146
(1.25)(0.49)(2.46)(0.17)(0.26)(0.52)(0.44)
R¯2–0.1040.266–0.0060.4950.2360.282–0.142
Note: For definitions etc., see Note to Table 3. Regional detail for year-ahead data not available.
Table 13.World Economic Outlook Forecast Accuracy: Non-Oil Developing Countries’ Balance of Payments on Current Account(In billions of dollars)
AfricaAsiaEuropeMiddle

East
Western

Hemisphere
Total

Non-Oil

Developing

Countries2
Current Year (1977–86)1
Mean absolute actual value9.12012.1305.74311.86019.29042.467
Average absolute error2.574.801.472.654.866.68
RMSE3.6956.4112.1133.5476.6398.028
Theil’s inequality statistic0.7400.8851.3160.5480.6720.509
Regression: intercept–4.422–2.554–2.762–2.194–1.644–2.020
(1.16)(0.68)(3.16)(1.38)(0.36)(0.49)
slope0.4940.6680.6080.7270.9520.943
(1.33)(1.46)(2.80)(2.68)(0.23)(0.71)
R¯20.0710.4580.7490.8480.6820.907
Year Ahead (1978–85)2
Mean absolute actual value8.35014.5005.1008.80020.97559.062
Average absolute error2.5386.5881.6603.43810.76321.825
RMSE3.8967.4911.7564.93312.09824.041
Theil’s inequality statistic0.8931.2220.9110.8501.1461.181
Regression: intercept–3.294–8.187–0.743–4.692–5.515–21.479
(0.40)(1.75)(0.60)(0.72)(0.46)(0.86)
slope0.6220.3990.7030.3660.6970.597
(0.47)(2.30)(1.66)(1.10)(0.62)(1.08)
R¯2–0.0620.1600.783–0.0930.1240.184
Note: For definitions etc., see Note to Table 3.

In the output growth forecasts, for example, a majority of the Theil statistics for the year-ahead forecasts exceed unity, while half of those for the current-year forecasts do so. Apparently, a naive prediction of no change in output growth would have been a better forecast in these instances than the actual World Economic Outlook forecasts.14 The fit of the realization-forecast regressions is also problematic, violation of efficiency being particularly strongly indicated for the Asia group in the year-ahead forecasts. Nevertheless, average absolute errors appear reasonably low in relation to the mean absolute value of the outturn series and there appears to be an improvement with the reduction in forecast horizon length upon moving from the year-ahead to the current-year forecast sets. Much the same statements can be made of the record in relation to forecasts of inflation. These are considerably poorer than the corresponding forecasts for the industrial countries, with some notably high Theil statistics in the year-ahead forecasts (where all but one exceed unity), generally low overall explanatory power in the realization-forecast regression, and evidence of inefficiency in several cases.

For export and import volume growth, regional detail is available only for the current-year forecasts; here the evidence is somewhat more reassuring. In the statistics on the export volume forecasts, only one of the Theil statistics exceeds unity (for the Middle East grouping, for which the average absolute forecast error itself exceeds the mean absolute value of the outturn series), although for import growth, Theil statistics exceed unity in three out of the seven cases. The overall explanatory power of the forecasts in the realization-forecast regression is generally low although indications of inefficiency are confined to European import growth forecasts.

The balance of payments forecasts, finally, provide some indication of weakness; the year-ahead forecasts produce three instances of Theil statistics above unity, with evidence of bias in the Asia grouping and generally low explanatory power for the forecasts in the realization-forecast regression. The current-year forecasts are somewhat better. The average absolute errors are generally lower in relation to the absolute mean of the outturns, the Theil statistics (that for Europe excepted) are lower, and the overall explanatory power of the forecasts higher, although with more evidence of departure from the efficiency requirements on the parameters of the regression.

The results of directly testing for bias, both on individual area results and the pooled data as a whole, are again reported in Appendix III. For the developing countries too, these tests suggest a tendency toward output optimism, at least in the year-ahead sample. Some individual area bias in inflation estimates also appears, though this is not significant when the data are pooled.

One reason for weakness in developing country forecasting is the extent to which such forecasts must rely upon projections of commodity prices, themselves known to be associated with large margins of uncertainty. Unfortunately, changes of definition and non-continuities in reporting such forecasts in the World Economic Outlook documents make it impossible to examine more than a small run of years of commodity price projections. Table 14 reports some summary statistics on forecasts made for four individual groups of commodities and the aggregate of interest here, non-oil developing countries’ exports. The variability of these prices is notably high and it is not too surprising that the average absolute errors—ranging from 5.7 to 8.5 percent—are also rather big. Even so, the forecasts do at least compare well with the naive standard and only a proportion appear to infringe the efficiency criteria in the realization-forecast regression.

Table 14.Primary Product Price Forecast Accuracy(In percent)
Agricultural

Raw

Materials
BeveragesFoodMetalsNon-Oil

Developing

Countries’

Exports
Current Year (1981–86)
Mean absolute average value7.4612.2911.678.078.06
Average absolute error5.6528.0957.5628.5335.887
RMSE8.95410.0809.2598.9547.497
Theil’s inequality statistic0.8000.5340.5561.2090.668
Regression: intercept–5.505–1.103–5.112–8.248–5.487
(1.967)(0.287)(1.279)(6.778)(2.313)
slope1.0130.6490.9420.7940.833
(0.025)(1.669)(0.029)(1.098)(0.567)
R¯20.3760.6310.3820.7710.585
Note: For definitions etc., see Note to Table 3.

The summary statistics reviewed, based on the overall record, provide a number of general conclusions. First, industrial country forecasting appears to be much better than that for developing countries. This is not surprising: developed countries are better understood, data streams are not so thin and are more reliable. It should also be borne in mind that the quality of the data analyzed in these tables is less good for the developing countries owing to frequent changes in definitions and coverage.

Second, among the industrial country forecasts, the balance of payments forecasts appear considerably worse than those for output, inflation, and export or import volumes. This should not be cause for great surprise (though it may be cause for concern): the balance of payments is the difference between two series; small changes in these can induce large changes in the difference, which as a data series may be volatile both in behavior and in its revisions. Moreover, the emergence of a relatively large and volatile discrepancy in the world’s aggregate current account—which, in principle, should be in balance—casts some doubt on the quality of balance of payments data even for the industrial countries. Difficulty in forecasting the balance of payments is a common complaint among national forecasters.15

A third conclusion that can be drawn for the industrial countries is that the current-year forecasts are superior to the year-ahead forecasts. While it may not seem surprising (it may even appear obvious) that near-term forecasting is more accurate than longer-term forecasting, such results are not invariably recorded (see, for example, Burns (1986) for a contrary instance).

Fourth, the record appears comparatively free from inefficiency in the sense that country-by-country and area-by-area the parameters of the realization-forecast regression conform by and large to the requirements of efficiency. However, when the data are pooled, direct tests for the significance of forecast bias produce some evidence of an output optimism error (more pronounced for the year-ahead than for the current-year forecasts).

Finally, it should be noted that the relative inferiority of the balance of payments forecasts and of the year-ahead forecasts for the industrial countries does not carry over to the developing countries. For this group, output forecasts are the weakest, with forecasts for export growth and current year balance of payments somewhat better. By and large, these results are similar to those arrived at by Kenen and Schwartz (1986) in their study of the Fund’s forecasting and they were confirmed by additional replicating calculations using the latest available estimates of outturns (Appendix II).

Comparisons

It is natural to enquire how well forecasting in the Fund compares with other forecasts. Here we consider two alternatives, the OECD (see Table 15) and national forecasting agencies. In these comparisons we are able to follow the lead set by Llewellyn and Arai (1984), who have already compared OECD and national forecasting records.

Table 15.Published Basis for OECD-World Economic Outlook Comparison
Forecast

Date
OECD

Economic

Outlook
Fund’s

World Economic

Outlook
Realization

Date
1973Dec. 1972Jan. 1973
1974Dec. 1973Dec. 1973
1975Dec. 1974Dec. 19741973
1976Dec. 1975Dec. 19751974
1977Dec. 1976Mar. 19771975
1978Dec. 1977Dec. 19771976
1979Dec. 1978Feb. 19791977
1980Dec. 1979Aug. 19791978
1981Dec. 1980Aug. 19801979
1982July 1981Aug. 19811980
1983July 1982Aug. 19821981
1984July 1983Aug. 19831982
1985July 1984Sept. 19841983
1986Dec. 1985Oct. 19851984
. .Dec. 1986Oct. 19861985

Comparisons with OECD

There have been a number of analyses of OECD’s track record besides that of Llewellyn and Arai (such as Smyth (1983), Smyth and Ash (1975), and Holden, Peel, and Sandhu (1987)). It would be useful if, in choosing the OECD as a comparison, one is also choosing a forecast which is-—at least in more recent years—less “judgmental” and more model-based than that underlying the World Economic Outlook forecasts, for this would give added point to the comparison. However, it is not clear how far such a contrast is realistic.16

The forecasts compared here are those for output growth, inflation, and the balance of payments on the current account of the Group of Seven countries individually and in aggregate. In Llewellyn and Arai (1984) attention was focused on the OECD’s forecasts a year ahead, using issues of the OECD Economic Outlook for December of year t –1 for forecasts for year t, the realizations coming from the OECD Economic Outlook for December of the following year (t+ 1). An immediate problem is that World Economic Outlook forecasts are not always based on the same information as that conditioning the OECD forecasts, the historically less regular World Economic Outlook round drawing on some forecasts made as early as August of the previous year and as late as March in the year in question. In order to achieve as close a match as possible, since 1981 year-ahead forecasts in July issues of the OECD Economic Outlook have been compared with the World Economic Outlook’s August forecasts. But this is only a partial solution to the problem and the remaining differences in timing, whilst apparently not severe, are unfortunate.17 The fact that calendar time discrepancies between forecast dates are small is uncertain assurance that discrepancies between the corresponding information sets are in the relevant sense also small. The scope of the comparison is from 1973 to 1985.

Table 16.OECD Forecast Accuracy—Summary Statistics: Group of Seven
CanadaUnited

States
JapanFranceFederal

Republic of

Germany
ItalyUnited

Kingdom
Group

of

Seven
Year Ahead (1973–85): Output Growth
(In percent)
Mean absolute actual value3.3773.4004.6542.5312.6852.9542.3152.992
Average absolute error1.7901.1501.6401.0101.5202.1001.1901.070
RMSE2.521.682.811.542.202.481.641.50
Theil’s inequality statistic0.6600.4140.7280.6000.6480.6270.6460.476
Regression: intercept–0.385–0.1062.124–0.062–1.2790.7710.217–0.493
(0.21)(0.15)(1.10)(0.07)(0.85)(0.79)(0.37)(0.57)
slope0.8831.0380.4580.8521.1600.6510.7961.178
(0.24)(1.18)(1.51)(0.52)(0.33)(1.07)(0.84)(0.64)
R¯20.1660.6660.0490.3940.2780.1980.4470.583
Year Ahead (1973–85): Inflation
(In percent)
Mean absolute actual value8.3466.6725.3779.9614.36915.36911.8317.346
Average absolute error2.111.523.351.3620.622.222.741.26
RMSE2.8774.6170.7391.7351.6773.0263.4591.775
Theil’s inequality statistic0.9590.7330.8950.7540.4510.8590.5460.804
Regression: intercept3.2221.585–0.2103.337–0.6436.149–0.9341.178
(1.05)(1.00)(0.72)(1.96)(0.95)(2.69)(0.36)(0.60)
slope0.6410.7560.9680.6851.1960.6731.1650.847
(0.97)(1.10)(0.07)(0.09)(1.23)(0.04)(0.74)(0.59)
R¯20.1420.4690.2350.4850.8310.5810.6900.445
Year Ahead (1973–85): Balance of Payments
(In billions of dollars)
Mean absolute actual value2.82524.79513.2884.0926.9164.4984.89919.385
Average absolute error3.1815.287.513.344.354.723.7916.04
RMSE5.03922.3829.9783.9335.3726.0234.87820.864
Theil’s inequality statistic1.7821.0440.8830.7670.7241.0430.8400.913
Regression: intercept–2.042–7.6201.108–1.4880.562–2.2390.945–9.495
(1.91)(1.19)(0.32)(0.95)(0.33)(1.55)(0.66)(1.44)
slope–0.0201.1551.0590.6391.1630.1111.2210.698
(4.98)(0.78)(0.25)(0.96)(0.34)(2.30)(0.51)(0.81)
R¯2–0.0900.7310.6210.1340.473–0.0830.3610.180
Note: For definitions etc., see Note to Table 3.

Table 16 lists summary statistics for the OECD forecasts which can be compared with corresponding statistics for the industrial countries in the World Economic Outlook (shown as the year-ahead forecasts in Tables 3, 4, and 7). Within Table 16 itself the summary statistics quoted show that the balance of payments is the worst forecast for the OECD too: the Theil statistics are generally higher than for the output and inflation forecasts, while the overall explanatory power in the realization-forecast regression is often very low. Similarly, average absolute errors are larger in relation to the absolute value of the balance of payments than they are for output growth or inflation (indeed they are sometimes bigger than the mean absolute value of the balance of payments itself). These features are identical to those found for the World Economic Outlook forecasts of the same variables. The regression evidence suggests a departure from efficiency in the forecasts of inflation in Italy (just as in the World Economic Outlook forecasts—see Table 2) and, in addition, for the balance of payments forecasts for that country and Canada.

Realizations do not differ greatly between the OECD and World Economic Outlook forecasts and an initial comparison between the average absolute errors and RMSEs of the two seems reasonable. On this basis, the OECD output growth forecasts emerge as slightly superior to those in the World Economic Outlook in that, in a majority of cases, the OECD error is less than that of the World Economic Outlook. For inflation and the balance of payments, however, the evidence suggests the opposite. Thus, the World Economic Outlook balance of payments record does not appear to be capable of improvement by using the unexploited information in OECD forecasts. We turn, in a moment, to consideration of a more formal test of this proposition.

Error triangles. An alternative way of comparing these forecasts is to take the growth, inflation, and balance of payments forecasts together for a particular country. A typical social welfare or decision function would consider a weighted sum of squared prediction errors for these variables as the minimand; the quadratic form implies symmetrical penalties, as seems appropriate in this case, increasing sharply with the value of the error.18 Without access to the weights with which the squared prediction errors are to be combined, the approach cannot be used to quantify exhaustively the relative success of the two organizations. However, where each of the prediction errors for the three variables of interest for a particular country and period is less for one set of forecasts than for the other, the former set is unequivocally superior to (“dominates”) the latter, whatever the appropriate weights may be. In diagrammatic terms (see Chart 1), a dominant forecast provides a triangle in output/inflation/balance of payments error space which lies wholly within the other (the chart displays triangles for Japan and France for illustrative purposes). The forecasts made for each of 13 years, for seven individual countries and the Group of Seven aggregate, provide the basis for 104 such triangles. In addition, there are at least eight period-averages that can be considered. For the single-year comparisons, the absolute error clearly provides the same results as squared errors and Table 17 shows that on this basis there were 22 cases out of the total of 104 in which one set of forecasts was unequivocally better than the other. In just under two thirds of these, the World Economic Outlook forecast turned out to be dominant.

Chart 1.Error Triangles: Illustrative Comparisons of OECD and World Economic Outlook Forecasts

Table 17.Forecast Comparisons: World Economic Outlook and OECD
YearForecasterCountryDominant

Forecast by:1
1973NoneNone0
1974OECDUnited States1
1975OECDFrance1
FundCanada1
1976FundJapan, United States2
1977FundUnited States1
OECDGroup of Seven1
1978NoneNone0
1979FundJapan, Italy2
1980OECDFrance, Group of Seven2
1981OECDFrance1
Canada, United States,
1982FundUnited Kingdom3
1983FundCanada1
OECDUnited States1
France, Japan,
1984FundGroup of Seven3
OECDUnited Kingdom1
1985FundCanada1
Total22
Fund14
OECD8
Table 18.Comparison of Output Forecast Errors: World Economic Outlook and OECD(In percentage points)
CanadaUnited

States
JapanFranceFederal

Republic of

Germany
ItalyUnited

Kingdom
Group

of

Seven
Correlation

Coefficient1
1973OECD–0.80.40.60.0–0.1–0.1–0.3–0.3
Fund–0.80.40.50.0–0.3–0.3–1.00.00.74
1974OECD2.74.49.31.62.94.13.44.0
Fund2.24.910.81.42.62.63.64.80.98
1975OECD2.9–0.2–0.14.25.73.53.61.2
Fund1.90.31.24.75.44.64.02.00.93
1976OECD–0.7–0.3–2.1–2.2–2.5–4.1–2.1–1.6
Fund0.80.2–0.7–1.8–2.2–2.2–0.7–0.80.94
1977OECD0.8–0.40.80.00.9–2.2–0.4–0.4
Fund1.30.31.20.42.2–2.20.60.60.96
1978OECD0.4–0.2–0.6–0.1–0.3–1.6–0.3–0.6
Fund1.10.1–0.80.10.00.4–0.50.00.25
1979OECD1.2–0.3–1.20.2–0.5–1.50.8–0.7
Fund1.31.1–1.10.0–0.4–0.11.00.30.87
1980OECD1.4–1.10.60.70.5–2.0–0.2–0.5
Fund3.00.20.51.01.3–1.21.40.60.87
1981OECD–1.8–1.30.90.70.1–0.80.2–0.5
Fund–2.3–3.41.11.62.30.61.8–0.60.83
1982OECD7.32.71.00.13.12.1–1.01.9
Fund5.82.62.70.63.12.1–0.52.40.96
1983OECD–2.0–1.21.01.82.03.7–1.4–0.2
Fund–1.2–1.6–0.51.80.74.5–1.6–0.40.94
1984OECD–0.3–2.3–2.3–1.1–1.0–0.6–0.4–1.7
Fund–0.8–2.5–1.8–0.3–0.6–0.2–0.5–1.70.86
1985OECD–1.0–0.2–0.80.40.3–0.1–1.3–0.32
Fund–0.91.3–0.40.4–0.10.2–0.90.50.68
Correlation
coefficient0.930.900.980.970.910.950.91

Looking at the comparative performance over the period as a whole, it becomes essential to consult a quadratic error statistic: in fact, the Theil statistics already collected in Tables 3, 4, 7, and 16 fit the bill well enough, and also have the merit that they standardize for the (small) differences in realizations as between the two sets of forecasts. A comparison of these statistics yields the result that, over the whole period, the OECD forecasts dominate those of the World Economic Outlook for France and for the Federal Republic of Germany whilst the World Economic Outlook forecasts for Canada dominate those of OECD.

Error patterns. The evidence so far suggests that there is generally little to choose between the two sets of forecasts; with some slight risk of misrepresentation this might be characterized as saying that the two organizations tend to make the same errors about the same variables for the same countries at the same time. Tables 1820 both confirm this and provide qualifications. These tables show the pattern of forecast errors (forecast minus realization) for the three variables of interest and the simple correlation coefficients between them, both across countries for a particular year and across years for a particular country. The latter are notably high and on the whole higher than the former. Some years of “disagreement” (1978 on output growth, 1982–85 on inflation) thus stand out, but in doing so serve to highlight the fact that the pattern of errors is usually very similar between the two organizations. In Llewellyn and Arai (1984) a similar conclusion was drawn about a comparison of the OECD and national forecasters.

Table 19.Comparison of Inflation Forecast Errors: World Economic Outlook and OECD(In percentage points)
CanadaUnited

States
JapanFranceFederal

Republic of

Germany
ItalyUnited

Kingdom
Group

of

Seven
Correlation

Coefficient1
1973OECD–3.1–2.1–6.3–1.8–0.6–4.0–0.1–2.5
Fund–2.8–2.3–6.3–2.3–0.4–2.3–0.5–2.50.94
1974OECD–7.1–3.6–11.9–3.40.2–6.9–6.3–4.9
Fund–7.4–4.7–8.6–4.40.2–7.6–4.8–4.90.92
1975OECD1.21.27.4–1.3–1.61.0–7.81.0
Fund0.50.59.0–2.1–1.1–1.0–7.80.80.98
1976OECD–0.31.5–0.90.90.7–5.31.10.4
Fund–0.50.6–0.11.11.2–2.80.00.30.92
1977OECD0.6–0.71.30.60.41.5–1.30.0
Fund0.3–0.51.3–0.50.12.7–0.60.00.79
1978OECD0.1–1.30.5–1.40.1–0.61.5–0.5
Fund0.0–1.3–0.8–1.20.1–0.80.4–0.80.84
1979OECD–3.5–1.62.5–0.8–0.3–2.5–4.6–0.8
Fund–3.4–1.10.2–1.5–0.2–1.8–4.2–1.10.93
1980OECD–1.50.62.6–0.5–0.6–3.7–2.30.1
Fund–3.20.14.4–1.80.1–6.2–4.6–0.10.98
1981OECD–0.51.32.2–0.4–0.3–0.93.11.0
Fund–2.0–1.31.6–3.10.2–3.81.2–0.80.81
1982OECD0.41.92.8–1.1–1.30.02.41.4
Fund–0.31.21.50.5–0.32.01.01.00.52
1983OECD4.21.33.14.1–0.5–0.32.21.7
Fund1.61.82.32.11.32.02.21.90.39
1984OECD3.21.21.71.20.61.61.61.6
Fund1.90.41.60.31.14.11.81.1–0.33
1985OECD1.71.40.30.20.8–0.6–0.80.5
Fund1.31.0–0.5–0.40.42.3–1.10.50.29
Correlation
coefficient0.960.890.950.830.670.820.95
Table 20.Comparison of Balance of Payments Forecast Errors: World Economic Outlook and OECD(In billions of dollars)
CanadaUnited

States
JapanFranceFederal

Republic of

Germany
ItalyUnited

Kingdom
Group

of

Seven
Correlation

Coefficient1
1973OECD–0.82–5.845.341.99–5.415.872.383.50
Fund–0.4–6.66.21.6–6.14.72.21.60.99
1974OECD1.75.64.25.2–8.66.455.8020.35
Fund1.46.63.64.6–8.74.64.316.40.99
1975OECD1.15–19.20.7–6.152.1–5.15–2.8–29.35
Fund0.9–18.10.7–6.44.8–5.4–3.8–27.30.99
1976OECD0.456.65–8.23.6–1.43.05–0.753.4
Fund0.44.4–5.43.40.23.7–2.14.60.96
1977OECD0.412.3–10.9–0.451.3–1.8–1.75–0.9
Fund0.19.5–8.9–1.20.1–3.4–3.4–7.20.99
1978OECD0.85–5.85–6.5–5.9–5.8–4.651.5–25.85
Fund0.7–2.8–5.8–7.3–6.1–6.41.9–25.80.92
1979OECD–0.1–7.220.80.87.75–0.355.427.1
Fund1.0–6.719.60.49.9–0.25.629.60.99
1980OECD–5.9–0.21.955.911.415.1–7.021.25
Fund–4.1–5.49.07.314.014.4–4.630.60.91
1981OECD2.513.25–11.451.25–2.95.75–11.95–3.55
Fund–0.212.5–17.22.1–4.56.5–13.2–14.00.98
1982OECD–12.9511.110.15.25–1.550.5–4.657.8
Fund–4.73.81.46.4–10.9–1.6–1.9–7.50.63
1983OECD–10.335.3–0.05–3.00.85–7.252.6518.2
Fund–1.836.0–5.2–6.21.7–2.5–5.516.50.93
1984OECD1.564.0–13.25–3.2–4.31.00.0545.8
Fund1.339.0–10.4–1.1–0.2–0.20.829.20.96
1985OECD2.6512.70–13.200.70–3.204.45–2.601.5
Fund0.2–2.5–9.11.4–7.52.1–5.2–20.60.67
Correlation
coefficient0.870.940.910.960.890.950.85

Non-nested tests. A formal test of the proposition that the OECD forecast could improve the World Economic Outlook forecast and vice-versa was nevertheless implemented. Borrowing from the methodology of non-nested tests as applied to estimated equations, it was proposed to ask whether the OECD (World Economic Outlook) forecast had any power in explaining the error of the World Economic Outlook (OECD) forecast. For this purpose it seemed appropriate to use, not the raw forecast error, but the residuals from the forecast-realization regression, the “structural forecast error.”19 Thus the test conducted was, for each of the three variables (output growth, inflation, and the balance of payments) and for each of the Group of Seven countries and the aggregate to run the regressions:

where E(WEO) and E(OECD) are the respective structural errors and F(WEO), F(OECD) the corresponding raw forecast. These tests were carried out on a subset of the comparison data set (terminating in 1984, and starting in 1974 for the balance of payments). Only in three cases did the results of these regressions suggest unexploited information in the alternative forecast. Both forecasts for the balance of payments of Canada could be improved by knowledge of the other, and a reduction in the World Economic Outlook error on the U.S. balance of payments could be effected by exploiting the OECD forecast. In the circumstances it seems excessively burdensome to report the findings in full.

In conclusion, it is apparent that the World Economic Outlook forecasts compare well with OECD’s and that there is no substantial evidence that there is unexploited information in these forecasts which the World Economic Outlook could benefit from using. As the two groups of forecasters “breathe the same air,” exchange information, and maintain contacts with the same national forecast agencies, the conclusion is perhaps not surprising.

Comparisons with National Forecasts for Industrial Countries

How do World Economic Outlook forecasts compare with national ones? This discussion follows the lead set by Llewellyn and Arai (1984), using a data set similar to theirs but extended by three years to cover the period 1973–85.20

Llewellyn-Arai. First, recall the basis for the Llewellyn-Arai study and its principal conclusions. Llewellyn and Arai approached forecasters in a large number of OECD countries for data on output and inflation forecasts and outturns. They described the terms in which these were to be defined in the following way:… the forecasts sought were those made in October, November, or December for the following calendar year. The forecasters were asked to supply the values of the actuals against which the forecasts should be compared. This seemed the most sensible approach because only those who made the forecasts really know which series, to which base year, and so on they were trying to predict. Forecasters were asked to provide estimates of the actuals made about one year after the outcome…21 The OECD forecasts against which the national data were compared were compiled on a year-ahead basis matching forecasts for year t, taken from the OECD’s Economic Outlook in December of year t –1, against outturns taken from the Economic Outlook in December of year (t+ 1. The coverage of the Llewellyn-Arai paper was ambitious, in terms both of the number of countries covered and the length of time for which series were obtained, the paper deploying data for 1966–82 for all OECD countries with the principal exceptions of Australia, Canada, and Ireland.

The main results of the comparison were based on an analysis of the total forecast error distribution of OECD and the national forecasters, and analysis of that distribution in particular years. From the frequency distribution of forecast errors, it was concluded that there was a slight tendency towards skewness in the direction of “growth optimism” for output but that overall neither one of the two sets of forecasts exhibited a marked superiority over the other. Most important, perhaps, Llewellyn and Arai detected that there were certain years which were marked by positive or negative skewness, suggesting that in these periods a shock external to the OECD (such as the oil price increases) or a system-wide behavioral innovation (the decline in expenditures with rising inflation, for example) was the predominant source of error; in other years individual country errors were largely offsetting (a similar phenomenon in comparing the World Economic Outlook with OECD is noted above). In the case of inflation, the error distribution was found to display a marked negative skewness, though this appeared to reflect most heavily experience in the early years of the series.

The present study sought to update the national forecast series in Llewellyn and Arai for comparison with the World Economic Outlook year-ahead forecast series. Those who supplied the original national forecast data were approached again and details of other forecasts were obtained from the corresponding country desk officers in the Fund. The resultant extensions, obtained for most of the series used in the original study, are shown in Appendix V. They make up series for each of the Group of Seven countries from 1973 to 1985, with the exception of Canada, and in addition cover Finland, Netherlands, Sweden, Switzerland, and Austria.22 As the World Economic Outlook does not publish forecasts for these last countries their contribution is assessed in the context of forecasts for the aggregate “Europe.”

The information conditioning the data set is less consistent than in the original Llewellyn-Arai study. Whilst the “centre of gravity” of the World Economic Outlook year-ahead forecast is only slightly earlier (October as opposed to December) than that of the national forecasts with which they are compared, it is obvious that on average a difference of two months may seriously disadvantage the World Economic Outlook particularly in situations when major disturbances occur. Given the highly significant sensitivity of forecast accuracy to the information set available to the forecaster, this mismatch in source dates implies that comparisons of forecast accuracy on this data set must be treated with care.23

A second source of heterogeneity in the data lies in the sector provenance of the national forecasts; some of these are “official” forecasts, whilst others emanate from the private sector. For the United States and the Federal Republic of Germany, more than one series is available: in the case of the United States as in the original study, Stephen McNees kindly provided details of consensus forecasts. In addition, for the United States, a comparison was also made with the projections published by the Office of Management and Budget in the Mid-Session Budget Review, the dating of which corresponds quite closely to that of the World Economic Outlook year-ahead forecasts since 1980. Prior to 1980, the dating of the World Economic Outlook corresponds more closely to the projections presented in the annual report of the Council of Economic Advisers.

Various hypotheses can be formulated about the possible biases in national forecasts. The political exposure to which official forecasts are subject might be thought to predispose them towards “errors of sentiment”—causing output growth to appear too high, or inflation too low; but it may also be argued that the normative element in official forecasts helps accuracy, for governments may be able to implement mid-term policy corrections to check deviations of variables from their target trajectories. Both private and official national forecasts might suffer from solipsistic bias because they are based on less-than-full information about international developments and the feedback between these and their own projections than about purely domestic developments. By contrast, a characteristic of forecasting by international agencies like the Fund or the OECD is that forecasting at this level requires endogenization to the forecasting process of global variables which are (mostly) treated as exogenous at the national level.

Despite the various possible sources of difference, however, the fact that so much information is passed from one forecaster to another, both domestically and internationally, is bound to narrow the scope for substantial disagreement to persist and thus to limit the extent to which forecast accuracy differs, at least over any reasonably lengthy period of time.

Tables 21 and 22 collect the forecast errors (forecast minus realization) for the different agencies covered, respectively, for output and for inflation. In Table 23, composite series for the Group of Seven and Europe, constructed as a weighted average of the national figures, are compared with the corresponding World Economic Outlook forecasts. In constructing these series the missing 1984 and 1985 figures for Italy were proxied by the World Economic Outlook figures for those dates, and where multiple national forecasts were available, as in the case of the United States and Germany, the one with the lowest average absolute error was selected.

Table 21.Comparison of Year-Ahead Output Growth Forecast Errors1(In percentage points)
United StatesJapanFranceFederal Republic of GermanyItalyUnited KingdomB2
OMROEAConsensusWEOOfficialWEOOfficialWEOConsensusFive Wise

Men
OfficialWHOISCOWEON1ESRWEO
19730.80.20.44.30.50.00.0–0.40.1–0.9–0.3–0.9–1.4–0.8–1.0–0.20
19743.22.94.92.710.82.51.42.62.10.62.63.12.60.43.6+1.00
1975–1.01.00.30.91.24.24.76.15.65.65.43.74.63.54.0+1.00
19760.20.0–0.10.2–0.1–0.7–0.2–1.8–1.6–1.1–1.1–2.2–3.6–4.2–1.3–0.7–0.83
19770.80.00.10.30.91.21.50.43.12.12.62.21.3–2.2–1.70.6+0.67
19781.30.4–0.10.11.3–0.81.20.1–0.40.10.10.0–0.60.4–0.5–0.5+0.09
19792.01.4–0.81.10.2–1.10.30.0–0.4–0.6–0.4–0.4–1.0–1.01.11.0–0.09
19801.2–0.7–1.10.2–0.20.51.11.00.71.00.71.3–2.5–1.21.91.4+0.50
1981–1.71.0–0.7–3.42.01.11.21.60.30.8–0.22.3–0.20.61.11.8+0.33
19825.14.22.42.61.92.71.20.62.21.72.53.11.32.10.1–0.5+0.83
19831.0–3.0–1.3–1.6–0.30.51.11.8–0.3–0.3–0.80.72.74.5–0.8–1.60.0
1984–1.6–1.1–1.3–2.5–1.0–1.8–0.5–0.3–0.6–0.1–0.1–0.6–0.2–0.3–0.5–1.00
19851.61.50.71.30.3–0.40.40.4–0.50.50.0–0.10.2–0.6–0.9+0.27
Average absolute
error 1973–791.07550.9710.7431.0431.4862.3291.4141.2002.0861.6711.6141.8712.0292.3431.3291.629
Omitting 19741.07550.6000.3830.4001.2830.9171.2331.1672.0001.6001.7831.7501.8502.3001.4831.300
1980–852.0331.9171.2501.9330.9501.1670.9170.9500.7670.7330.7171.3501.67532.10030.8001.117
Whole period1.65051.4080.9771.4541.2391.7921.1851.0851.4771.2391.2001.6311.9002.25541.0851.392
Correlation
coefficient.
WEO0.890.680.830.500.880.910.920.860.860.75
Source: Llewellyn and Arai (1984). updated and extended (Appendix V).Note: For sources of national forecasts, see Appendix V.
Table 22.Comparison of Year-Ahead Inflation Forecast Errors1(In percentage points)
United StatesJapanFranceFederal Republic of GermanyItalyUnited KingdomB2
OMROEAConsensusWEOOfficialWEOOfficialWEOConsensusFive Wise MenOfficialWEOISCOWEONIESRWEO
1973–2.3–2.7–2.3–9.6–6.3–2.1–2.3–0.6–0.1–0.6–0.4–3.0–2.3–2.9–0.5–1.00
1974–3.2–3.5–4.7–8.1–8.6–4.1–4.40.31.00.50.2–10.2–7.6–2.8–4.8–0.67
19752.3–0.20.55.19.0–3.0–2.1–1.3–2.3–1.8–1.11.6–1.0–5.6–7.8–0.33
19762.00.90.70.60.0–0.1–1.91.11.40.90.91.2–7.5–2.8–1.10.00.0
19770.1–0.4–0.3–0.51.61.3–0.4–0.50.40.4–0.10.1–1.02.7–1.0–0.6–0.33
1978–1.1–2.3–1.4–1.30.9–0.8–2.0–1.20.1–0.4–0.40.1–1.2–0.8–0.30.4–0.50
1979–2.2–1.6–1.1–1.11.70.2–1.5–1.5–0.3–0.8–0.3–0.2–3.4–1.8–3.2–4.2–0.67
1980–0.1–1.0–0.20.11.8–4.4–2.4–1.8–0.5–0.5–1.00.1–5.8–6.2–0.2–4.6–0.33
19810.81.60.1–1.31.51.6–0.9–3.10.4–0.10.40.2–3.5–3.8–0.41.20.0
19822.12.61.91.21.41.51.10.5–0.3–0.8–0.8–0.30.32.00.01.00.64
19832.31.51.51.81.52.3–0.92.10.30.30.31.3–0.12.01.72.20.67
19841.01.40.70.40.21.6–0.50.30.61.11.11.14.10.51.80.83
19851.31.11.01.0–0.1–0.5–0.3–0.40.3–0.2–0.20.42.30.2–1.10.0
Average absolute
error (1973–79)1.35051.8571.4141.5713.8573.7572.1431.8710.6570.8430.6290.4713.9862.7142.4142.614
Omitting 19741.35051.6331.0671.0503.1502.9501.8171.4500.6830.8170.6830.5172.9501.9002.3502.250
1980–851.2671.5330.9000.9671.0831.9831.0171.3670.4000.5000.6330.5672.42533.50030.5001.983
Whole period1.30051.7081.1771.2922.5772.9381.6231.6380.5380.6850.6310.5153.41843.0001.5312.323
Correlation
coefficient.
WEO0.8090.800.940.920.640.860.800.860.830.80
Source: Llewellyn and Arai (1984), updated and extended (Appendix D).Note: See definitions in Note to Table 21.
Table 23.Output Growth and Inflation Forecast Errors1(In percentage points)
OutputInflation
National forecastsWEONational forecastsWEO
Group

of

Seven
EuropeGroup

of

Seven
EuropeGroup

of

Seven
EuropeGroup

of

Seven
Europe
19730.58–0.320.09–0.50–3.38–2.04–2.49–1.30
19742.391.234.982.30–4.18–3.22–4.86–3.10
19752.204.432.014.80–0.06–1.900.77–2.00
1976–0.09–0.22–0.80–1.70–0.31–1.430.360.40
19770.641.150.581.00–0.07–0.480.020.00
19780.310.39–0.100.30–0.84–0.55–0.89–0.30
1979–0.220.080.230.00–0.73–1.52–0.99–1.70
1980–0.011.030.491.00–0.51–2.00–0.03–2.20
19810.270.47–0.571.500.01–0.88–0.69–1.00
19821.951.272.191.601.200.071.060.10
1983–0.620.14–0.371.001.100.341.901.70
1984–0.93–0.33–1.74–0.400.510.29–0.981.10
19850.44–0.080.610.000.50–0.270.510.20
Average absolute error0.820.861.141.241.031.151.201.16
Correlation
coefficient, WEO0.890.920.950.90
Note: The Group of Seven excludes Canada (WEO errors are assumed for national forecasts for Italy in 1984 and 1985). The aggregate for Europe is based on data for France, Italy, the Federal Republic of Germany, the United Kingdom, the Netherlands, Austria, Sweden, Switzerland, and Finland.

Considering first the individual country forecasts for output growth, it can be seen that there is a fair measure of sympathy between the World Economic Outlook and national forecast errors; but there are some exceptions, and correlation coefficients in nearly every case are lower than those recorded between the OECD and World Economic Outlook country forecasts, Japan affording an extreme example (though with a special explanation—see below). In the case of the Federal Republic of Germany, the World Economic Outlook forecast is strongly correlated with the national forecasts; the larger positive errors of 1975, the smaller ones for the preceding year, and the negative errors for 1977 are common to all four forecasts for Germany. In 1982, sizable positive errors are the common experience. In France, Italy, and the United Kingdom the patterns of error for the national forecast and the World Economic Outlook are very similar. The aggregate Group of Seven and Europe output growth forecast errors are well correlated with the corresponding World Economic Outlook projections.

In the light of the mismatch of source dates discussed earlier, it may be helpful in making comparisons of forecast accuracy to divide the sample into two sub-periods and also to take into account the special features distinguishing the 1974 forecast exercise. The World Economic Outlook forecasts for the years in the first half of the sample (1973–79) were prepared later in the year than those in the second sub-sample (1980–85), as indicated in Table 2. Accordingly, the World Economic Outlook forecast might be expected to reflect a relative information advantage in the first sub-sample and a corresponding disadvantage in the second sub-sample.

The 1974 forecast was prepared in December 1973 in conditions of extreme uncertainty about the supply situation in the oil market. The forecast was in fact made on a “business as usual” assumption, this phrase meaning that no physical supply shortages were assumed. As it happened, the prospect of prolonged and severe constraints of this kind was dispelled when, early in January, decisions were taken by the Organization of Arab Petroleum Exporting Countries (OAPEC) to relax the supply curbs that had been threatened but to substitute further price rises. This news prompted World Economic Outlook forecasters in a January supplement to the December forecast to hazard that “All in all, it seems reasonable to conclude that current prospects for growth … are … somewhat intermediate between (a) the staffs ‘business as usual’ projections … and (b)the much bleaker outlook that followed the imposition of OAPEC cutbacks in oil production, until their recent easing.” This clearly shows that the Fund’s 1974 projections had a consciously artificial element, which knowingly would produce an “optimistic” bias in output forecasts, even though the “business as usual” assumption proved to be less wide of the mark than might have seemed likely at some earlier stages. Of course, national forecasters faced similar problems in coping with the impact of the oil price increase; and some would have had an information advantage (most likely, Japan), though others would not. There is a case, therefore, for omitting 1974 from the comparisons of forecast accuracy.24 When this is done, as can be seen from Table 21, the World Economic Outlook output forecasts generally do appear more accurate (with lower absolute errors) than those of the national agencies in the first sub-sample and (with the single exception of the forecast by the Office of Management and Budget for the United States) less accurate in the second.

For inflation, the World Economic Outlook and national forecasts are generally somewhat more highly correlated, those for France being the prominent exception; here, the weakness in the relationship between the two is especially pronounced for the 1980s (the correlation coefficient between the two forecasts for 1980–85 is only 0.40). In the United States the sign pattern of the errors in the three forecasts is highly consistent, 1982 proving—as for real output—a prominent weakness in the U.S. Mid-Session Budget Review Forecast. In Japan, too, the patterns of the forecasts are very similar overall, 1978 being a prominent exception involving a difference of sign, though there are other years where the difference is bigger, 1975 carrying a particularly large World Economic Outlook error. In Italy and the United Kingdom, a generally high degree of correspondence also includes some large error differences (1974 and 1976 in Italy, 1975 in the United Kingdom being the notable examples). By the average absolute error criterion, the World Economic Outlook inflation forecasts are superior for Italy and the Federal Republic of Germany, but the margin of difference is again generally small (the principal exception being a difference of 0.8 of 1 percent in the national forecaster’s favor in the case of the United Kingdom). In the aggregate series, the inflation forecast errors for the composite Group of Seven and Europe turn out to be very similar indeed. Indeed, having regard to the various qualifications concerning source-date mismatch and the status of the 1974 forecasts and to the fact that for multiple-forecast countries the forecast yielding the lowest average (output) error was used in the national forecast compilation, it could not be said that there is any significant difference between the track records of the World Economic Outlook and the aggregated national forecasts of output and inflation in the Group of Seven and Europe as a whole.

Since there are two forecast variables under consideration here (a comparison with national forecasters regarding the balance of payments forecasts was not possible due to lack of comparable data) it is possible to compute on how many occasions one forecast dominates the other by counting the number of times that one forecast is better, for the pair of variables together, than the other. In such a case, whatever (positive) weighting is given to the different variable forecast misses, the forecast with the lowest pair of errors will dominate the other. In the present case there are 73 instances to consider (6 countries × 13 years minus the two years for which independent Italian forecasts are not available, minus one case in which the errors are identical). Of these 73 possible instances, 49 are decisive, 19 showing World Economic Outlook as the dominant forecaster, and 30 the national forecaster.25 The World Economic Outlook forecast was especially strong in 1978 when for five out of six major countries the World Economic Outlook forecast was superior for both output and inflation. On the other hand, in both 1983 and 1985 the World Economic Outlook was inferior for both output and inflation to five of the national forecasts.26

Turning finally to the question of the relationship across countries between the two forecast errors, the “B” statistic shown in Tables 21 and 22 shows that there are, as Llewellyn and Arai found, a number of years when all or a majority of forecast errors are of a similar sign.27 In the case of output growth, 1974 and 1975 stand out as years of universally or widely shared positive error (over-optimism), as also 1982; whilst 1976 and 1984 are instances of widely shared or universal negative error. In the case of inflation, the bunching of errors occurs in 1973 and 1974 (negative errors), and again in 1982, 1983, and 1984 (positive errors).

As a broad conclusion, it seems that forecasts for the World Economic Outlook are similar to those by national forecasters in the industrial countries for output, although the Fund’s forecasts are more accurate for the 1970s (when 1974 is omitted), and national forecasts are better for the 1980s. The forecasts from both sources for inflation are well correlated.

Some explanation of these episodes is explored in the next section. The national forecast data perform the useful function of identifying periods when an error is (or not) widely spread across countries and forecasting agencies, confirming that the explanation for a particular World Economic Outlook error is (or not) specific to the World Economic Outlook. Since there is, generally, a high correlation between the episodes identified by the B statistics and major errors in World Economic Outlook Group of Seven forecasts (and a correlation across countries between national and World Economic Outlook forecast errors) there is confirmation that, at any rate for the most part. World Economic Outlook errors tend to be shared in some fairly large degree by other agencies; they appear thus to be general products of the imprecise art of economic forecasting rather than errors that are purely specific to the World Economic Outlook.

Explanations of Forecast Errors

This explanation of forecasting error takes two forms. First, an attempt is made to identify the most significant episodes from inspection of the error patterns. These are then examined in more detail, using the World Economic Outlook source documents. This approach captures the uniqueness of each error-episode. The complementary approach pursued in the second part of this section is to attempt to explain systematically the forecast error, using an “innovation-accounting” approach where the error is attributed to deviations from the World Economic Outlook projection of conditional assumptions about fiscal policy and oil prices.

Main Issues

A convenient graphical summary of error patterns appears in Charts 2 and 3, which show, in the top panel the current year forecasts, and in the lower panels, the year-ahead forecast, together with the corresponding realization and error. Chart 2 shows output and Chart 3 inflation for the Group of Seven countries. Table 24 reports these same data, together with B statistics for the World Economic Outlook country errors and the latest available estimates of realization values. A comparison of these data with comparable data for the individual countries indicates, as might be expected, that the Group of Seven series are smoother and average the sometimes discrepant individual country movements. The B statistic reported is an aide-memoire in this respect (as noted earlier, by construction the statistic is bounded by –1 and + 1, giving a zero reading where the country errors are offsetting in sign and a value near ± unity when they are of similar sign).28

Chart 2.World Economic Outlook Forecast Errors: Group of Seven Output

Chart 3.World Economic Outlook Forecast Errors: Group of Seven Inflation

Table 24.Group of Seven: Output and Inflation Errors in World Economic Outlook
OutputInflation
ForecastsOutturnsErrorsB

statistic1
ForecastsOutturnsErrorsB

statistic1
Y/AC/YF/AF/SE1E2Y/AC/YF/AF/SE1E2
19714.54.03.23.21.30.84.24.95.55.5–1.3–0.6–0.86
19725.35.8–0.54.34.20.1
19736.67.56.76.60.00.8–0.144.65.47.17.1–2.5–1.7–1.00
19744.31.3–0.5–0.54.81.8+1.007.310.911.912.2–4.9–1.0–0.86
19750.7–1.5–1.6–1.32.00.1+1.0011.710.810.510.90.80.3–0.14
19764.86.15.65.6–0.80.5–0.717.67.57.17.30.30.40.0
19774.74.53.94.10.60.6+0.717.07.07.17.00.0–0.1+0.14
19784.24.24.04.20.00.2+0.336.36.17.27.1–0.8–1.1–0.14
19793.83.53.53.50.30.00.006.78.07.67.8–1.10.4–0.86
19801.80.81.21.20.6–0.4+0.719.09.69.09.1–0.10.6–0.14
19810.61.61.41.2–0.60.2+0.438.18.98.38.9–0.8–0.6–0.14
19822.00.7–0.4–0.42.41.1+0.717.77.56.76.71.00.8–0.43
19832.41.82.52.8–0.4–0.7+0.146.35.14.74.41.90.4+1.00
19843.53.95.15.2–1.7–1.2–1.004.74.23.63.61.10.6+1.00
19853.53.22.73.00.50.5–0.144.13.73.63.60.50.1+0.14
19863.23.02.40.63.53.13.10.1
Note: Y/A = Year Ahead; C/Y = Current Year; F/A = First Available Estimate; F/S = First Settled Estimate; El, E2 = Y/A–F/S, C/Y–F/A.

Turning-point error. Forecasters are often accused of being able to get things right only when the outlook is unproblematically the same as the past; when things change they fail to spot the difference. Clearly there is some truth in such a charge in the present case. Chart 2 shows how the sharp downturn in output growth between 1973 and 1975 gave rise to positive errors of forecast in 1974, in particular, even on a current year basis. Similarly, the pickup in growth in 1983 and 1984 gave rise to negative errors, particularly in the latter year. On the other hand, the turning-point error occasioned by the very sharp turnaround from the 1976 recession is quite small, even on a year-ahead basis while the prominent positive output error of 1982 comes at a time when output growth simply went on declining.

Turning to the inflation record, the negative error of 1974 coincided with a take-off of inflation, but its subsequent decline through 1976 and 1977 is well tracked. The renewed inflationary surge starting in 1978 before the second round of oil price increases again produces a negative error, however, whilst the subsequent disinflation seems to have been seriously unanticipated in the year-ahead forecasts—though the correction in the current-year forecasts is rather satisfactory.

The period as a whole can be broken up into perhaps four homogeneous sub-periods: The first round of oil price increases and its aftermath (1973 to 1975. say); the subsequent recovery (to 1978–79); the second round of oil price increases and its aftermath (1978–80 to 1982, say) followed by the “dollar shock” and the period up to the present.

Before exploring these episodes, the balance of payments forecast errors should also be recalled. By nature the B statistics for these errors, unlike those for output and inflation, will tend towards zero, as a positive error for one member of the group will need to be accommodated by a negative error for another, except to the extent to which surpluses or deficits against the rest of the world occur, or when there are variations in the global current account discrepancy. Table 25 below thus records both the net balance of payments error for the year ahead for the Group of Seven aggregate, and the absolute error obtained by summing across the individual countries without regard to sign. The latter gives an impression of the extent to which the balance of payments forecasts across countries were out of kilter. In fact the record suggests that the worst balance of payments forecast errors occurred in 1975, in 1979–80, and in 1983 and 1984 when the net error was also relatively large.

Table 25.Group of Seven: Balance of Payments Forecast Error, Year-Ahead Forecasts(In billions of dollars)
Net Error1Absolute Aggregate Error2
19731.627.8
197416.433.8
1975–27.340.1
19764.619.6
1977–7.226.2
1978–25.831.0
197929.643.4
198030.658.8
1981–14.056.2
1982–7.530.7
198316.558.9
198429.253.0
1985–20.628.0

The first round of oil price increases and its aftermath. The first wave of oil price increases in 1973–74 contributed to raising inflation steeply to double-digit levels; in the subsequent adjustment the Group of Seven countries’ output growth declined sharply, culminating in actual declines in 1974 and 1975. The Fund forecasters were not able to take on board the full extent of the oil price rise even in their late 1973 year-ahead projections. The anticipated rise in export unit values of oil exporters for 1974 as a result very dramatically underestimated the increase to come; by May 1974 the estimated rise had been increased by a factor of four, yet still significantly underestimated the actual increase (see Table 39 in Appendix VI).

These substantial discrepancies between forecasts and outcomes contributed powerfully, if uncontroversially, to the negative inflation forecast errors in 1973 and 1974. In no subsequent year was there an inflation error even half as large as the year-ahead error for 1974. It is not surprising, either, that the same error was associated with an unforeseen recession in 1974. As discussed earlier, these forecast errors for 1974 reflected in part the uncertainties regarding the supply situation. The failure to predict the full extent of the worldwide decline in output in 1975, however, is more problematic. Oil prices rose only moderately in this year and in any case the rise was largely anticipated, whilst fiscal policies were switched to an expansionary mode in every one of the major countries except the United Kingdom (see Table 38 in Appendix VI), on a scale which was essentially unforeseen. The forecast error seems thus to have derived largely from a failure to appreciate the full extent to which the oil price rise and the reactive adjustment policies would reduce output; this mistake was widely shared by national forecasters (as indicated in the Fund’s 1974 Annual Report) and the fiscal expansion was itself a response to the belated realization that the oil price increase had left real activity in the major economies in a weak condition. This was well understood by the time that the March 1975 World Economic Outlook came to be issued.29

The 1976–78 recovery. The recovery from the 1973–75 down-turn, which began in 1976, was foreseen as early as the May 1975 World Economic Outlook though it turned out to be understated, perhaps partly because the expansionary stance of policy was underestimated. Output anticipations in the remainder of the recovery period continued to be closely fulfilled and there were few aggregate disturbances until the oil price rise in 1979. Nevertheless there was quite a sizable inflation forecast error in 1978; even in the forecast made as late as April of that year, inflation was being predicted at too low a rate. In good part this seems to have been the outcome of a large error for a specific country, the United States; while the World Economic Outlook forecasters correctly anticipated that inflation in Europe would accelerate in 1978, they underestimated the extent of the inflationary surge in the United States.

The second round of oil price increases and its aftermath. The second round of oil price increases, like the first, contributed rapidly to an acceleration in inflation and to lower output growth in the industrial world. Learning from the experience of 1974–75, and in the light of already-high inflation, countries engaged uniformly in policies of monetary restraint and, in most cases, began to apply the fiscal brake as well. Although the oil price rise itself was not predicted, its scale was of course much smaller in percentage terms than that of 1973–74 and the greater uniformity of response and the experience of the first wave of oil price increases all helped reduce the extent of consequent forecast error. In fact, aside from the initial effect of the oil price rise on the inflation forecasts for 1979, the subsequent output and inflation forecast errors appear quite small. The difficulties appear in the subsequent period.

The 1982 recession. This phase is characterized by quite a large error of optimism about output in 1982, when aggregate output in the Group of Seven GNP declined slightly while the forecasters, even as late as April 1982, looked for a continued rise. One of the reasons was a mini-recovery early in 1982 which was known to forecasters and was thought to presage a continued upswing. In the following two years, errors of the opposite sign were made as the world economy recovered, only to give way to renewed negative errors as the world recovery moderated. The rate of inflation declined steadily over the period persistently, somewhat faster than expected.

The forecast error for output in 1982 which became subsequently apparent in the projections prepared the year before had no obvious prompting in either oil prices or unexpected fiscal developments as the aggregate Group of Seven fiscal impulse was actually somewhat greater than had been anticipated. A continued determination to squeeze out inflation characterized the collective Group of Seven policies at this time, however, and it is pertinent to note, as extensively documented in the April 1982 World Economic Outlook that this cooperation (or rather, mutual emulation) in restraint was concentrated particularly in monetary policy. While this fact was evident at the time, the World Economic Outlook forecasters do not appear to have grasped the true measure of it—or a true measure of its effects. (Nor was the error confined to the World Economic Outlook—it was shared, as shown above, by forecasters at OECD and by national forecasters around the world.)

The suggestion that the underestimating of the strength of contractionary monetary policies and their effects underlay the forecast error can be given some support from analysis of the sectors apparently most seriously affected and the World Economic Outlook’s ex post comments on the “unprecedentedly” high real interest rates which emerged. Table 26 shows two indicators of monetary stringency, real interest rates (short and long) generated as the difference between nominal interest rates and either current inflation or the contemporaneous World Economic Outlook inflation forecast and the growth rate of the real money supply (nominal money growth less current inflation). These data suggest that, if anything, the leap in monetary stringency was initiated in 1981 (or 1980 in the case of the real money stock) and sustained in 1982, so the weakness of the World Economic Outlook forecasts may reflect both the lagged effect of the already accomplished increase in stringency and its unexpected continuation through 1982. It should also be noted that forecasters differed significantly in their assessment of the impact of monetary stringency at the time. For example, although the World Economic Outlook output error for the United States was large it was much smaller than the error in the corresponding Mid-Session Budget Review and Council for Economic Advisers’ forecasts for 1982 (see previous section).

Table 26.Group of Seven: Indicators of Monetary Stringency1(In percent)
Real Interest Rates
(Using actual inflation)(Using WEO inflation forecast)Real Money
Short termLong termShort termLong termSupply Growth
1971–3.02–1.293.57
1972–1.83–2.565.29
19734.35–0.264.372.162.65
1974–1.57–2.73–2.53–1.97–5.03
1975–16.63–1.95–12.67–1.82–2.41
1976–3.451.00–4.151.230.84
1977–5.780.70–4.891.251.22
1978–2.001.12–0.342.293.71
19791.761.291.921.451.68
19803.391.993.081.67–3.66
19815.544.525.304.28–2.23
19824.865.544.214.90–0.17
19834.626.234.055.665.45
19845.777.155.456.843.07
19854.906.134.755.995.04
19863.804.413.734.348.25

The post-1982 recovery. Coming to terms with this mistake, the World Economic Outlook forecasts for 1983 turned out to be remarkably close to the mark on output, predicting well the recovery that would ensue. Inflation, nevertheless, turned out much less strong than anticipated in the early forecasts. It is not difficult to suggest that this was related to an underestimate of the extent to which the world recession would slow home-bred inflation in the industrial countries and, most important, cause primary product prices to decline. Both in 1982 and again in 1983, against a background of declining inflation in the industrial countries, the terms of trade improved to a degree which had not been foreseen, with particularly large falls in primary product prices in 1982, following a decline in the previous year.

The upswing correctly predicted for 1983 was foreseen by World Economic Outlook forecasters to continue through 1984, but not on the scale which in fact ensued. The forecasters did not catch up with this, even in their April 1984 projections. Although the sign of the output error was repeated across the set of Group of Seven countries, much the largest error and by far the greater part of the aggregate error was due to the underestimate of output growth in the United States, a mistake shared with other forecasters, including national agencies. This can be associated partly with an underestimate of the degree of fiscal expansion that policies would provide; for the rest it seems as if the rebound of private spending following the sustained reduction in inflation was larger than anticipated.

In 1985 and again in 1986 forecasting error was small. Against a background in which the sharp expansion of 1984 gave way to lower growth rates, World Economic Outlook forecasters tended to overestimate (although only slightly) the extent of output growth in their year-ahead forecasts; at this time, and especially in outlining the prospects for 1986, the World Economic Outlook forecasters came to lay particular stress on the effect of the decline in oil prices as a factor significantly improving the short-run growth prospects of the industrial countries. In retrospect (see the April 1987 World Economic Outlook which examines the point in some detail), the extent to which the decline in oil prices would have this effect was overestimated; albeit the output error in the predictions for 1986 was not large.

This pursuit of episodic detail clearly suggests that the major forecasting error can be associated with the first round of oil price increases. The second wave of oil price increases also created some obvious problems for forecasters. But some large errors have occurred which cannot be explained in this way. The effect of collective policies of restraint, especially on the monetary side as in 1982, appears to have been underestimated, for example, and private sector responses, first to the pressure of high inflation, then to its decline, have been similarly incompletely understood. Forecaster’s “caution,” a tendency to miss some turning points, particularly those provoked by novel types of disturbance, or to understate the strength of the turn can also be discerned. These weaknesses should be put in perspective: the errors described appear to have been widely shared by other forecasting agencies, national and international, as analyzed in the previous section, and as this evidence suggests, given the state of knowledge, were probably inescapable. In any case, most of the errors seem small and generally were quickly corrected. Just how small is “small” in this context depends of course on the purpose to which the forecasts are put.

Effects of Unanticipated Changes in Fiscal Policy and Oil Prices

This sub-section reports a formal attempt to ascertain the extent to which the World Economic Outlook forecast error can be attributed to a falsification of the environmental assumptions on which the forecasts are conditioned. As noted above, in making a forecast. World Economic Outlook forecasters make assumptions about external environmental factors, of which the most important appear to be oil prices and fiscal policy. Whilst the forecasts, as already noted, also make conditional projections of other variables like exchange rates and interest rates, it is arguable that these are to a considerable extent driven by fiscal policy and oil prices, together with variables which are endogenous to the forecast in any case. Of course, monetary policy may also have an exogenous component which may affect economic developments independently of variations in fiscal policy and oil prices. However, even though this factor clearly played an important role in the early 1980s, the identification and quantification of exogenous monetary policy changes is extremely hazardous.

With this caveat, the task here is to relate the forecasting error to unexpected changes in fiscal policy and oil prices. Two alternative approaches to the one adopted here should be mentioned. One, adopted by Kenen and Schwartz (1986) uses the forecast-realization framework to relate forecast misses to presumptive policy change; their approach, though, is really devised to test the stronger proposition that where the forecast itself projects an “unacceptable” scenario (inflation too high, output growth too low, for example), policies will be revised and will change the outcomes to the extent that it looks as though the forecast was wrong. A second approach, appropriate in the case of forecasts produced from a formal model, would be to estimate equations of the type R(t) = a + bP(t) + cF(t) + v(t), where P is the unexpected change in conditioning factors already expressed through the model in terms of the realized forecast variable. The investigator would then look for a = 0, b = 1, c = 1.30 This approach is not open to us here, since no estimates of oil price and fiscal policy multipliers that are consistent with World Economic Outlook estimates are available. Consequently, the formal approach adopted is to identify the unexpected changes and then to estimate their contribution to the forecast error through simple regressions of the type:

where I is a vector of the unexpected changes in conditioning variables. Some related hypotheses can be explored at the same time. For example Llewellyn and Arai (1984) concluded from their study of OECD and national forecasting errors that OECD forecasters were prone to overlook the lagged effects of large policy change; this hypothesis can be explored by relating forecast error to realized policy (and oil price) variables as well as to the innovations in them.

Measuring the unexpected changes. Because of the presumption that both policy and oil price changes take time to work through the economy to affect variables of interest, exploring the current-year forecasts for the effects of changes in exogenous variables seemed less likely to yield interesting results than an investigation of the year-ahead forecasts; the latter were consequently employed.

The first requirement then is to obtain a measure of the unexpected changes themselves that are relevant to the year-ahead horizons. Fortunately, the World Economic Outlook forecasters have always given an explicit projection for oil prices in the form of a forecast of major oil exporters’ export unit values, so that it is a straightforward matter to compute the unexpected oil price change. This series is shown in Table 27. (There is a discussion above of the context in which some of these errors arose, whilst the forecast and realization series themselves are shown in Table 37 in Appendix VI.)

The measurement of unexpected fiscal policy changes is somewhat less straightforward and is prone to significant observation error. However, World Economic Outlook forecasts have in later years generally provided a quantified forecast of the “fiscal impulse”; the series can only be extended backwards by “translating” the qualitative comment then available to comparable terms. At the same time, estimates of the realized fiscal stimulus have varied and the latest available consistent set of data does not appear to extend back beyond 1977, though at the risk of generating a spurious break these can again be extended backwards by using some earlier World Economic Outlook estimates of the fiscal impulse in these years. More of the forecast figures are available for the narrowly defined central government fiscal impulse than for the broader definition of general government fiscal impulse, though the qualitative comment might sometimes apply equally to either and it is not obvious what assumption to make about the forecasters’ implicit treatment of the “non-central” part of the fiscal impulse when comment is only about the central government part. Data on what the central government fiscal impulse actually was are not available before 1977. However, since the fiscal impulse is (with sign reversed) closely related to the first difference of the structural budget balance, an alternative set of realizations can be obtained from OECD estimates of this concept and in this way, two alternative series on unexpected fiscal policy changes were generated, as shown in Table 27, (A fuller account of the data base generation for these series is given in Appendix VI.)

The results. Given the measurement of the unexpected changes, the next step was to formulate the regressions. For this purpose the year-ahead forecast errors for output growth, inflation, and the balance of payments were related, for each country, to the oil price change, and, separately, to the “own” fiscal policy change and to the “foreign” fiscal policy change. In order to generate suitable estimates of the latter, unexpected changes in individual countries’ fiscal policy were aggregated, using GDP or import weights. In addition, since the series of unexpected oil price changes is dominated by the two rounds of oil price increases, an alternative specification using separate dummy variables for the two events was also used. There was considerable agreement between the various specifications: overwhelmingly, oil price changes (especially those of 1973–74) contributed to inflation forecast errors (those for the Federal Republic of Germany being the only exception); while unexpected oil price changes also accounted for a number of the output and balance of payments errors. By contrast, fiscal policy changes proved relatively unimportant in explaining forecast error and, where they were significant, did not always carry the theoretically expected sign.

Table 27.Unexpected Changes in Driving Variables, 1973–85
Fiscal Policy
CanadaUnited

States
JapanFranceFederal

Republic of

Germany
ItalyUnited

Kingdom
Oil Prices
(In percent of GNP)(In percent)
1973FP110.40.5–1.20.60.3–1.5–2.5
FP220.4–0.2–0.30.31.10.4–3.1–34.8
1974FP110.60.80.30.1–0.41.10.2
FP220.90.90.20.3–1.60.5–0.1–192.8
1975FP11–2.3–3.2–2.2–2.6–1.2–1.5–2.5
FP22–3.3–1.6–2.4–1.1–2.4–2.00.81.2
1976FP110.40.60.31.6–0.20.31.6
FP220.40.8–1.00.50.70.7–1.2–1.0
1977FP11–2.00.70.1–1.20.41.50.3
FP22–0.40.9–0.8–1.91.21.80.4–0.9
1978FP11–0.90.20.3–0.90.4–3.5–1.5
FP22–0.20.6–2.3–1.50.1–1.6–1.6–0.4
1979FP11–0.10.40.0–0.90.20.90.1
FP220.8–0.11.70.1–0.3–1.4–0.1–35.8
1980FP11–0.2–0.7–0.30.50.3–0.41.4
FP22–0.8–0.8–0.11.1–0.50.80.9–43.6
1981FP110.8–0.20.2–1.30.2–0.91.2
FP220.6–0.70.4–1.3–0.2–3.72.70.7
1982FP11–1.9–0.4–0.21.00.6–1.10.2
FP22–0.9–0.9–0.20.40.70.11.08.8
1983FP11–1.2–0.60.2–0.1–0.40.3–0.7
FP22–2.20.30.3–0.10.92.5–1.313.9
1984FP11–1.5–1.0–0.1–0.3–0.81.0–0.6
FP22–1.2–1.10.50.00.0–1.7–1.11.0
1985FP11–1.2–0.30.10.10.2–1.0–0.1
FP22–1.3–0.80.00.40.3–1.40.14.3

Table 28, by way of example, tabulates the results arrived at when treating the oil price changes as a continuous variable. Appendix VI tabulates additionally the results of treating the innovations as a discontinuous series (dummy variables). While the significance of the oil price effect is expected, the relative unimportance of unexpected fiscal policy changes (and the prevalence of “wrong signs”) is somewhat disappointing. In addition to problems with the quality of the fiscal policy data series per se, some other explanations also suggest themselves. To begin with, impulse measures are best regarded as measures of “policy stance”; they are imprecise about timing and weight expenditures and taxation revenues equally, so for these reasons the measure may not be a good one. It is also true that some fiscal actions have long lags and these may be longer than the forecast horizon. Finally, as we know that fiscal policy responds to both the realized and (conditionally) forecast path of economic variables, it may also respond to the World Economic Outlook error if that error is in some sense a measure of the disappointment of desirable targets experienced as the forecast period unfolds—thus for example an ex-post, over-optimistic forecast for output growth may become widely anticipated during the year and result in expansionary revisions of fiscal policy.31This would lead to an observed association of positive forecast error with a negative impact from (own) fiscal impulse changes, the “wrong” sign. Clearly this two-way causation issue is not important for oil prices and neither can it be invoked to explain the “wrong” sign on the external fiscal policy variable. There is always, of course, the possibility that fiscal crowding out is more common than usually supposed.32

Table 28.Explaining Forecast Errors: Unexpected Changes in Driving Variables(Year-ahead forecasts, 1973–85)
CanadaUnited

States
JapanFranceFederal

Republic of

Germany
ItalyUnited

Kingdom
Output growth
Constant0.441–0.354–0.1190.1501.118–0.263–0.080
(0.52)(0.61)(0.20)(0.34)(1.46)(0.32)(0.26)
Own fiscal policy–0.123–0.395–0.486–0.702–1.1650.0430.600*
(0.15)(0.62)(0.97)(1.63)(1.47)(0.10)(3.18)
External fiscal policy–0.752–0.1370.425–1.576*–0.2432.453–1.325*
(0.44)(0.09)(0.39)(2.36)(0.16)(1.70)(2.74)
Oil price–0.012–0.028*–0.050*–0.010.004–0.012–0.023*
(0.83)(2.81)(4.90)(1.29)(0.24)(0.84)(4.38)
R¯2–0.2020.2930.6610.2770.1360.0130.723
Inflation
Constant–0.7180.1570.836–0.1470.363–0.060–0.066
(1.79)(0.49)(0.77)(0.34)(1.34)(0.07)(0.09)
Own fiscal policy–0.812**–0.577–0.2920.7490.1410.414–0.326
(2.11)(1.65)(0.33)(1.78)(0.50)(0.84)(0.70)
External fiscal policy0.3760.958–3.1450.6930.557–1.0662.950*
(0.47)(1.15)(1.63)(1.06)(1.04)(0.68)(2.47)
Oil price0.035*0.024*0.042*0.028*0.0030.045*0.037*
(5.20)(4.55)(2.31)(3.87)(0.48)(2.91)(2.87)
R¯20.8210.6920.4260.5070.0730.3950.434
Balance of payments on current account
Constant–0.4079.313–3.2411.2572.1261.047–3.602*
(0.54)(1.65)(1.02)(1.00)(0.69)(0.56)(1.20)
Own fiscal policy0.6303.7502.1973.249*–5.034–0.292–2.141*
(0.86)(0.60)(0.84)(2.64)(1.58)(0.29)(2.93)
External fiscal policy–1.50014.473–5.0542.4456.9333.342–1.373
(0.98)(1.05)(0.90)(1.28)(1.13)(1.03)(0.73)
Oil price–0.0050.074–0.075–0.0110.088–0.029–0.047*
(0.43)(0.77)(1.39)(0.51)(1.40)(0.92)(2.32)
R¯2–0.134–0.0920.0210.400–0.030–0.0310.475
Note: Figures in parentheses are t-ratios; * and ** denote significance at the 5 percent and 10 percent level, respectively.

Actual policy effects. Even when unexpected policy and oil price changes have been taken fully on board, are their effects fully comprehended? Llewellyn and Arai indicated that they may not have been, and it would be consistent with the suggestion above that coping with a change of a novel and unexpected kind or magnitude should cause forecast error both before the change is fully known and immediately afterwards, before its implications have been taken on board. In the first place, errors arise from unanticipated changes in the multiplicand; then as a result of error in the multiplier. The sign of the latter error is in principle ambiguous.

To test this proposition formally we extended regression equations of the type reported in Table 28 by adding actual oil price changes and actual fiscal policy variables, again using the alternative measures of fiscal policy and alternative weighting schemes. With the exception that lagged actual oil price change often proved significant among the additional variables, none of the variants afforded results offering substantially reconditioned conclusions. Thus the results for oil price changes reported in Table 28 remained substantially unchanged; actual fiscal policy (lagged) turned out to be significant only in the balance of payments results for the U.S., with a (positive) sign, suggesting that World Economic Outlook forecasters may have overestimated the fiscal impact on the current account. Lagged oil price realizations appeared significant in a number of cases, apparently contributing to positive output forecast errors in the Federal Republic of Germany and France, a positive inflation error in Japan (but a negative one in the United Kingdom), and a positive balance of payments forecast error in the Federal Republic of Germany. On this basis it would appear possible to overdo the extent to which large unexpected change yields a residue of “model” error in the immediately succeeding period.

Appendix I Data Definitions

This appendix explains various problems of definition, and the differences between the basic definitions and data used here and those used by Kenen and Schwartz in their 1986 study.33

Changing Definitions

The basis for the principal choices of variables to investigate is indicated in the text. This note discusses some problems arising out of the changing definitions of various aggregates investigated and the solutions adopted.

Industrial Countries

To obtain lengthy series for the Group of Seven and the “Europe” aggregates, not provided in early World Economic Outlook documents, figures were reconstructed from the country detail and “total industrial” category. Group of Seven figures for output growth and inflation were reconstructed from individual country data using preceding year GDP/GNP (current price, current exchange rates) weights. For consistency, later Group of Seven aggregates from the World Economic Outlook, which after 1981 were based on the average of three preceding years’ current price output weights, were also recomputed onto the same basis.34 (In the later OECD comparison, OECD Group of Seven aggregates were regenerated in the same way.) For the early years, figures for the “Europe” sub-aggregate were not presented in World Economic Outlook documents, but were reconstructed as the residual from available data on the “total industrial” and the “non-European” countries shown separately (at the time the latter category comprised only Canada, the U.S., and Japan).

In 1980, the definition of the total industrial group was expanded (and that of Europe correspondingly) to include an additional six countries—Australia, Finland, Iceland, Ireland, New Zealand, and Spain. Thus, in the May 1980 World Economic Outlook, both the forecast and the outturn for the previous year are for this group; and this necessitated the reconstruction of a current-year realization for 1979 using the World Economic Outlook figures for the major countries supplemented by contemporary evidence of the outturn for the other industrial countries. For this purpose figures from OECD Country Surveys and OECD Economic Outlook (July 1980) were used. A similar problem arose for the realization of the year-ahead forecasts, both for 1979 and for 1980 where outcome figures for the major individual countries available in the August 1980 and August 1981 World Economic Outlook reports were supplemented by data from the OECD’s July 1980 and 1981 Economic Outlook.

Developing Countries

A number of classification changes affect the aggregates and regional data for the developing countries. First, the position of China was not recognized in the data until April 1980 and even then could not be included in the May World Economic Outlook of that year. Some series were, therefore, shown both including and excluding China to reduce noncomparabilities. Then, there was a major change of classification in 1980 (described in detail below); finally. South Africa has been treated differently from time to time, with resultant breaks in the series and noncomparable forecast and realization data in some years. Some series were, therefore, shown, both including and excluding China to reduce noncomparabilities.

Reclassification. With effect from the May 1980 World Economic Outlook the previous classification was abandoned in favor of a new one. Before the change, three mutually exclusive groupings were identified:

(1) industrial (14 countries);

(2) primary producers in more developed areas (13, including South Africa). This category was also called “other developed” or “more developed primary producing countries”; and

(3) primary producing countries in the developed areas (all other countries not in (I) or (2)). Also called “less developed countries” or “developing countries.”

This category was in turn split into:

(a) major oil exporters; and

(b) other developing countries.

In the reclassification, category (2) was dissolved, with seven countries joining the industrial country bloc (including Australia and New Zealand) and six joining the developing country bloc (including South Africa). As before, the latter category was split into “oil” and “non-oil” sub-categories. As a result, forecast and realization data for the total and for the regional aggregates for Africa and Europe are not fully comparable for 1979.

China. Data for China were introduced into the Fund’s statistics from April 1980, but the May 1980 World Economic Outlook was already prepared by then and did not reflect the change. The 1981 World Economic Outlook data on output growth and inflation give figures excluding as well as including China and no noncomparability of forecast and realization data arises. In export and import growth the data give rise to a noncomparability in 1981 where actuals (totals) include, but forecasts exclude, China. Subsequent World Economic Outlook give data for the totals (forecast and realization) which include China and for the Asia regional category provide data which exclude China. From the 1985 World Economic Outlook, however, both the total and the regional data include China. This gives rise to another noncomparability in 1984.

Differences from Kenen-Schwartz

While the principal distinction between current-year and year-ahead forecasts made here follows Kenen and Schwartz (1986), the actual classification of sources is a little different, and different outturn data are used in the two studies.

The most important differences in the sources are that, for the current-year forecasts for 1977 this study uses the June-July documents rather than the March 1977 document used by Kenen and Schwartz; and for the 1979 forecasts it uses the June documents rather than the February ones. For the year-ahead forecasts, unlike Kenen and Schwartz, this study uses a January 1973 source for 1973 forecasts and the March 1977 documents to give forecasts for 1977 where they have none; for the sake of a more complete record the study also uses the February 1979 documents for 1979 which give more information than the December 1978 document used by Kenen and Schwartz. Finally, for 1985 and 1986, we use the published World Economic Outlook reports dated September 1984 and October 1985 rather than the internal August-dated documents apparently used by Kenen and Schwartz.

The net effect is a somewhat more complete year-ahead record at the cost of a greater dispersion of source dates. For the current-year forecasts the alterations make little difference in either respect and essentially accommodate the change in year-ahead sources, while ensuring that no single source is used for both kinds of forecast.35

The outturn data used here are, as explained in the main text, “first available” estimates for current-year forecasts and “first settled” estimates for the year-ahead forecasts. Kenen and Schwartz mainly use latest available data (with a supplementation of earlier series in an attempt to cope with definition changes).

Appendix II Replications with Latest Available Outturns

All the principal statistical procedures carried out for the current-year and year-ahead forecasts against their respective first available and first settled outturns were replicated using the latest available data set derived from the data bank underlying the April 1987 World Economic Outlook. Because of the change in definitions, data for the total industrial and Europe aggregates were not reprocessed; nor were the developing country forecast errors reprocessed. Thus the replications pertain to the seven industrial countries for output growth, inflation, exports, imports, and the balance of payments for the two forecast horizons. It would be excessively tedious to report the results in full, and Table 29 summarizes the number of countries (out of seven) for which the error statistics deteriorated upon use of the latest available data set, that is, for which the average absolute errors, root mean square errors, and Theil statistics increased in value and for which the multiple correlation coefficient in the realization-forecast regression declined in value. By subtraction, the number of cases of improvement can be derived. The tabulation conveys the general impression of some deterioration in the forecast when judged against the latest available estimates of outturn. However, the deterioration is generally marginal and the increase in the error statistics rather small. It is interesting to find that the current-year balance of payments forecasts are the chief exception to the general rule, though it still remains true that the balance of payments is on the whole the poorest forecast among the five variables considered.

Table 29.Deterioration in Summary Error Statistics When Latest Available Outturn Data Are Used
AAE

Increases
RMSE

Increases
R¯2

Falls
Theil

Increases
Year ahead
Output4334
Inflation4447
Exports5635
Imports5656
Balance of
payments3532
Current year
Output3335
Inflation5557
Exports5524
Imports4747
Balance of
payments2337
Note: This table covers the seven countries for which the statistics show a deterioration when the latest available data are used.
Appendix III Bias and Efficiency in Forecasts

In the text tables and discussion, attention is drawn to the fit of the realization-forecast regression

A perfect forecast would have a = 0, b = 1, and deviations from these values indicate an inefficiency in the forecast in the sense that the forecasts could be improved by knowledge of these parameters. Subtracting the forecast from each side of (1) gives

This makes it clear that estimates of (1) or (2) are tests of whether the error can be explained by the forecast itself, and also suggests an economical way of examining whether the forecasts are on average biased or not. This is simply to estimate

that is, to regress the error on a constant term. The value of the constant term is then the average error itself and the t-statistic generated on this parameter tests whether the error is significantly different from zero.

Put this way, it is clear that a set of forecasts might fail to pass the efficiency restrictions on the parameters of (1) and (2), yet produce no evidence of average bias. (For example, a highly inefficient set of forecasts might produce offsetting errors.) Holden and Peel (1987) produce examples of this. We would not expect the opposite to hold, however. If the efficiency restrictions hold in (1) and (2), then bias should not be evident in (3). That this is so can be readily seen by noting that (3) can be considered as a restriction of (2) (hence of (1)) in which b is set to unity. But if the joint restrictions are satisfied, the restriction of b to unity should not significantly disturb the estimate of the constant term from zero. It follows that cases where bias is detected yet the t-statistics on the individual coefficients in the realization-forecast regression do not suggest inefficiency are cases in which evaluating the individual coefficients is an insecure means of inferring the results of a test of the validity of the joint restriction. There are a number of examples of this in the developing country estimates in this study.

In conducting the tests of (1) and (3), there seems to be no a priori reason why the often rather small country data sets should not be pooled in the interests of obtaining more precise error bands. (Thus the pooled current year data sample for output and inflation disposes of 7 x 16 = 112 observations as compared with 16 in the individual-country cases whilst the pooled year-ahead sample comprises 91 observations against 15 for individual countries.) In any case the validity of the pooling can be tested for by including country shift and slope dummies.

Industrial Countries

Table 30 summarizes the results of estimating equation (3) for each country, the aggregates, and for the pooled sample. In order to test for the validity of the pooling the equation was first estimated with country dummies and the constant term suppressed, as

Table 30.Industrial Countries: Forecast Average Errors and Significance Levels
CanadaUnited

States
JapanFranceFederal

Republic of Germany
ItalyUnited

Kingdom
Group

of

Seven
Total

Industrial

Countries
EuropePooled

Data
Current year
Output–0.363–0.150–0.363–0.263–0.681–0.119–0.331–0.267–0.425–0.431–0.324
(–1.090)(–0.552)(–0.839)(–0.865)(–2.123)(–0.314)(–1.127)(–1.449)(–2.847)(–1.858)(2.593)
Inflation0.381–0.019–0.4310.444–0.0250.6750.5380.0280.1000.4380.221
(0.393)(–0.100)(–0.703)(1.391)(–0.112)(1.408)(0.959)(0.160)(0.643)(1.945)(1.381)
Balance of
payments–0.071–3.5792.743–0.2931.0570.4930.3360.6930.010
(–0.084)(–1.005)(1.444)(–0.355)(0.726)(0.601)(0.313)(0.171)(0.148)
Year ahead
Output–0.877–0.300–1.054–0.800–1.0000.262–0.515–0.598–0.646–0.838–0.708
(–1.475)(–0.513)(–1.201)(–1.909)(–1.921)(0.202)(–1.049)(–1.290)(–1.467)(–1.900)(3.096)
Inflation1.0770.431–0.4311.023–0.1931.0151.3080.3660.3460.3920.602
(1.487)(0.893)(–0.357)(2.063)(–1.101)(1.027)(1.537)(0.737)(0.748)(1.194)(2.073)
Balance of
payments0.400–5.3611.654–0.5380.950–1.2541.915–2.9236.100–0.287
(0.733)(–1.179)(0.612)(–0.388)(0.502)(–0.802)(1.386)(–0.398)(1.254)(–0.322)
Note: Data show the average error (actual minus forecast) and, in parentheses, the t-statistic obtained from a regression of the error series on a constant term.

From the results obtained a further specialization of the dummies was determined and the equations were re-run with a constant and selected dummy variables. In the event, only in the current-year balance of payments equation was a country dummy (for the United States) significant. In the light of this result, the fully pooled data estimates are displayed alongside the country and aggregate estimates in the table. The results in this table reveal bias, among the individual country estimates, only in a handful of cases: the current-year forecasts of output for the Federal Republic of Germany tend toward a negative bias (over-forecast) as do the year-ahead forecasts for this country and for France. No bias is revealed among the individual country estimates for inflation except for France tn the year-ahead forecasts, and no bias at all is suggested in the balance of payments forecasts, either on a current-year or year-ahead basis.

Nevertheless, as can be seen, for each of the individual countries with the sole exception of Italy, the output bias—though not individually significant—is consistently of the same (negative) sign. It is not too surprising, then, that the pooling reveals this bias to be significant. In a similar way, there is a predominance of positive signs in the year-ahead country inflation errors, significant only for France, but upon pooling, significant generally. The extent of the bias in the case of current-year output forecasts is quite small (0.3 of 1 percentage point), but it is more than twice as large in the year-ahead forecasts and 0.6 of a percentage point in the inflation forecasts on this basis. However, truncating the sample of year-ahead forecasts so as to omit 1974, for which a case can be made (see the text discussion of the comparison of World Economic Outlook and national forecasts), has the effect of removing completely the significance of the average inflation error, and reduces the size of the output bias (to 0.5 of 1 percentage point) without removing its significance.

The data pooling thus suggests that there is a tendency to output optimism in the Fund’s forecasting, with the output forecasts being pitched too high. Less strongly, there is a suggestion that the forecasts for inflation are on average pitched too low. Interestingly, the signs of these errors are offsetting and a regression of inflation forecast error on output error using the pooled data set confirmed a significant negative relationship, a result which might have one of a number of interpretations. A plausible candidate explanation is that it reflects the prevalence of supply side shocks over the period. An implication is that World Economic Outlook forecasts of nominal income are superior to those for its components (as Kenen and Schwartz (1986) also argue).

The pooling procedure can be applied equally to the realization-forecast regression (1); in this case the validity of the pooling can be tested for by proceeding from an initial specification in which shift and slope dummies are included for every country (the constant again suppressed). Again, the results suggested little need for any “country effects,” and the results of the fully pooled data regression are accordingly shown in Table 31.

Table 31.Industrial Countries: Realization-Forecast Regressions on Pooled Data
Constant

Term
SlopeR¯2
Current year
Output–0.1780.9510.764
(–0.906)(0.968)
Inflation0.1171.0130.887
(0.370)(–0.380)
Balance of payments0.4451.1410.945
(0.741)(–5.043)
Year ahead
Output–0.1400.8230.371
(–0.329)(1.584)
Inflation0.2591.0410.721
(0.408)(–0.602)
Balance of payments–0.3021.0710.787
(–0.339)(–1.203)
Note: Terms in parentheses are t-statistics: those for the slope coefficient test for difference from unity.

Developing Countries

Table 32 tabulates average forecast errors (again, actual minus forecast) for the developing country areas, derived from regressing the errors on a constant. The figures in parentheses are t-ratios. Because of the comparative lack of degrees of freedom, the critical values of the f-ratio are somewhat higher than customary: significant estimates are shown by footnote 2. With a smaller data set for individual areas than for the individual Group of Seven countries, pooling is an attractive option and the acceptability of this procedure was assessed by initially regressing the errors on a set of area dummies. Inspection for significant differences between the coefficients attracted by these dummies led to the retention of a separate dummy only for the Western Hemisphere in respect of the inflation forecast errors.36

Table 32.Developing Countries: Average Forecast Errors and Significance Levels
Total

Non-Oil

Developing

Countries
AfricaAsiaEuropeMiddle

East
Western

Hemisphere
Pooled

Data
Current year
Output growth–0.600–1.090–0.400–0.100–0.690–0.860–0.639
(–1.698)(–4.66)1(–0.877)(–0.350)(–0.993)(–1.049)(–2.611)
Inflation6.3102.0501.6305.090–0.77016.1302.0002
(5.982)1(1.093)(3.461)1(3.918)1(–0.216)(4.899)1(1.661)
Balance of0.453
payments0.3902.200–0.7381.430–0.7600.815
(0.212)(0.318)(1.096)(–1.064)(1.322)(–0.346)(0.976)
Year ahead
Output growth–1.171–1.071–0.543–0.500–2.029–2.186–1.288
(–1.845)(–2.025)(–0.648)(–1.096)(–2.027)(–1.767)(–3.297)1
Inflation12.6292.5141.9438.667–3.71439.9862.1192
(4.042)1(2.215)(2.532)1(2.817)1(–0.418)(3.976)1(0.679)
Balance of15.225
payments0.5381.313–0.9002.2131.2130.968
(1.342)(0.369)(0.471)(–0.433)(1.328)(0.267)(0.805)
Note: See note to Table 30 for definitions.

As can be seen, the pooled data would support the contention that, as in the case of the developed countries. World Economic Outlook output forecasts tend toward optimism, though here the bias is only significant for the year-ahead forecasts. Whilst not many of the individual area average output growth forecast errors are statistically significant, they are all of the same sign. The pooling does not suggest a bias in other forecasts, even those for inflation, where most, but not all, forecast errors are positive in sign and most, but not all, are individually significant. Finally, the balance of payments forecasts, neither individually nor pooled, appear to be biased.

Table 33.Developing Countries: Realization-Forecast Regressions on Pooled Data
ConstantSlopeDummy

Shift
Variables

Slope
R¯2
(Western Hemisphere)
Current year
Output growth–0.2880.9110.454
(–0.470)(0.624)
Inflation5.5660.803–11.0070.5250.954
(2.508)(1.572)(–1.761)(3.840)
Balance of–1.2910.8490.677
payments(–0.842)(1.624)
Year ahead
Output growth–0.5520.8380.218
(–0.438)(0.616)
Inflation13.9650.381–76.2182.6760.877
(3.259)(3.252)(–3.262)(5.442)
Balance of–2.9510.7040.410
payments(–1.381)(0.365)
Note: Figures in parentheses are t-statistics; those on the constant and on both dummy variable terms test for difference from zero, whilst those on the slope coefficient test for a difference from unity.

Table 33 tabulates the results of restricted pooled data estimates for the realization-forecast regression, already reported for the individual areas and the aggregate in the text tables. Again, the validity of the pooling was examined by first regressing the outturns on a full set of area shift and slope dummies, leading to a more specialized set of dummies on the basis of the consideration of the significance of differences between the dummy variable coefficients. Further runs then showed that no dummy variables were significant except for those pertaining to inflation in respect of the Western Hemisphere. These regressions suggest that the output and balance of payments forecasts are efficient in the sense that the constant terms are not significantly different from zero, the slope terms not different from unity, The inflation forecasts are another story, however, even when the Western Hemisphere observations are treated separately. The current-year errors suggest that when inflation is above about 7 percent, it is likely to be underforecast, whilst the year-ahead errors suggest that inflation is overforecast at levels below about 37 percent.

Appendix IV: An Alternative Naive Forecast

The Theil statistics reported in the main tables compare the RMSE of the World Economic Outlook forecasts with the RMSE of an alternative naive “no change” forecast, where the forecast for year t is set equal to the realization for t–1.

A plausible alternative naive forecast is one that projects the established trend. Table 34 shows the Theil statistics computed for this alternative naive standard,37 where the trend values were identified with a moving average of the past ten years’ output growth or inflation. (It was not felt appropriate to treat the balance of payments in this way.) Because the computed “trend” is backward looking, this alternative standard might be expected to represent a harder standard to beat for a variable with a comparatively short cycle, like output, than the conventional “no change” naive standard underlying the Theil statistics reported in the text tables. For a variable with an evolving trend or longer cycle, like inflation, however, it is likely to prove an easier standard.

Industrial Countries

This expectation is borne out for the industrial countries where the Theil statistics quoted for output growth in the table are uniformly higher than those reported in the text tables (though not, in general, much higher) whilst those given for inflation are uniformly lower. The relatively poorer standard of accuracy of the year-ahead forecasts is repeated, but there is no case where the trend alternative would have outperformed the World Economic Outlook forecasts.

Developing Countries

Table 34.Alternative Theil Statistics
Current YearYear Ahead
Output growthInflationOutput growthInflation
Industrial countries
Canada0.4310.4120.6590.631
United States0.3440.2580.6180.536
Japan0.4360.4350.8680.659
France0.5040.4020.6830.493
Germany, Fed. Rep. of0.5460.4500.7910.386
Italy0.4560.2970.7830.520
United Kingdom0.4980.2700.7400.401
Group of Seven0.2840.2240.6230.488
Total industrial0.4060.1990.6450.471
Europe0.4390.2700.7300.431
Non-oil developing countries
Africa0.7700.6811.2390.570
Asia0.9110.5721.1700.756
Europe0.2320.5290.5201.038
Middle East1.0550.6252.0041.128
Western Hemisphere0.7140.4330.8610.976
Total0.6240.4610.9380.942
Note: Using ten-year moving averages as the “naive” forecast.

The table also tabulates alternative Theil statistics, generated from a “forecast-trend” assumption for the developing countries; in the same way as for the developed countries, a ten-year moving average of past output growth (inflation) was used as the naive forecast. As can be seen, there are a handful of instances where such a naive forecast would have produced better forecasts (by a minimum RMSE criterion) than the procedures actually used; but a similar statement was already available with respect to “no change” forecasts and in fact in nearly every case the “no change” forecast is a harder standard to beat than the “trend” alternative—that is, the Theil statistics tabulated in Table 34 are nearly always smaller than those reported for the conventional “no change” forecast in the text tables. There are only three exceptions to this, all of them arising in the year-ahead output growth forecasts. Here, for Africa and the Middle East the “trend” naive forecast is better than both the “no change” naive forecast and the forecasts actually made; with respect to Europe, whilst the “trend” naive forecast is a harder standard to beat than the “no change” forecast, the forecasts actually made outperform both.

Appendix V Basic Data Extensions, National Forecasts

As indicated in the main text the comparison with national forecasts was carried out by extending the base already assembled by Llewellyn and Arai in their OECD study (Llewellyn and Arai (1984)). Data were sought from the same institutions on the same basis as in that study, or from corresponding country desk officers in the Fund. The paradigm return would be of forecasts for year t prepared in the last quarter of t-1, with outturns taken from data available in t+1; however, there were numerous variations on the paradigm. Practice regarding the vintage of data used to describe the outturn clearly varied; more important, perhaps, the date of preparation of the forecasts is not constant across forecasters relative to the forecast horizon. Table 35 tabulates the dates recorded for the Summit Seven forecasts discussed in the text. Assuming that forecasts of fourth quarter origin were in fact made available in December and treating the two-month lead time of the Japanese financial year forecasts as equivalent to that of a November forecast for the calendar year, the unweighted “centre for gravity” of these forecasts is December. The World Economic Outlook year-ahead forecasts with which the national forecasts are compared in the main text have a considerable dispersion over the period in their lead time on the forecast; the average of these lead times is two months, giving a centre of gravity for these forecasts of October.38

Table 36 provides the data obtained as updates and additions to the series published by Llewellyn and Arai. Additional detail is included for the U.S. consensus forecasts because of a change in definition of the “actuals” supplied for these forecasts. For the United States, data are also included showing the track record of the projections (or “economic assumptions”) contained in the Mid-Session Budget Reviews, published by the Office of Management and Budget in July-August. Since 1980, the dating of these forecasts thus corresponds quite closely to that of the World Economic Outlook, whereas the dating of the projections of the Council of Economic Advisers contained in its Annual Report (published in February) corresponds more closely to that of the available World Economic Outlook projections prior to 1980.

Table 35.Six Major Industrial Countries: Dates of National Forecasts
ForecastDate1
United StatesOMB (Mid-Session Budget Review)July–August, t –1
CEAFebruary, t
(Annual Report)
ConsensusFourth quarter, t –1
JapanOfficialJanuary, t–1 for FYt
FranceOfficialSeptember, t –1
Germany,
Fed. Rep. ofOfficialJanuary, t
Five Wise MenNovember, t –1
OfficialOctober, t –1
ItalyISCOSeptember–October, t –1
United KingdomNIESRFebruary, t
Note: OMB is the U.S. Office of Management and Budget; CEA is the U.S. Council of Economic Advisers; ISCO is Instituto Nazionale per to Studio della Congiontura; NIESR is National Institute for Economic and Social Research.
Table 36.Updates to Llewellyn-Arai Data Base(In percent or percentage points)
GNPGNP Deflator
ForecastActualErrorForecastActualError
Country: United States
Source: Consensus1
19736.15.90.23.36.0–2.7
19741.1–1.82.95.99.4–3.5
1975–0.8–1.81.09.19.3–0.2
19765.96.0–0.16.05.30.7
19775.04.90.15.55.8–0.3
19784.34.4–0.15.97.3–1.4
19792.43.2–0.87.48.5–1.1
1980–1.3–0.2–1.18.89.0–0.2
19811.21.9–0.79.59.40.1
19820.5–1.92.47.96.01.9
19832.43.7–1.35.33.81.5
19845.26.5–1.34.84.10.7
19853.42.70.74.33.31.0
Source: Council for Economic Advisers
19833.16.1–3.05.64.11.5
19844.55.6–1.15.03.51.5
19854.02.51.54.33.21.1
Source: Office of Management and Budget
(Mid-Session Budget Review)
19766.36.10.27.15.12.0
19775.74.90.86.05.90.1
19785.34.01.36.37.4–1.1
19794.32.32.06.68.8–2.2
19801.0–0.21.28.99.00.1
19810.32.0–1.710.09.20.8
19823.4–1.75.18.05.92.1
19834.43.71.06.54.22.3
19845.26.8–1.64.83.81.0
19854.32.71.64.73.41.3
Country: Japan
Source: Official2
19833.43.7–0.32.10.61.5
19844.15.1–1.01.71.50.2
19854.64.30.31.41.5–0.1
Country: Germany, Federal Republic of
Source: Consensus3
19830.11.3–0.33.53.20.3
19842.02.6–0.62.51.90.6
19852.02.5–0.52.52.20.3
Source: Official
19830.51.3–0.83.53.20.3
19842.51.6–0.13.01.91.1
19852.52.50.02.02.2–0.2
Source: Five Wise Men
19831.01.3–0.33.53.20.3
19842.52.6–0.13.01.91.1
19853.02.50.52.02.2–0.2
Country: France
Source: Official4,5
19841.01.5–0.56.77.2–0.5
19851.81.40.45.55.8–0.3
Country: United Kingdom
Source: National Institute of Economic and Social Research5,6
19831.42.2–0.87.15.41.7
19842.22.5–0.35.65.10.5
19852.83.4–0.65.65.40.2
Country: Austria
Source: Institut fur Wirtschaftsforschung5,6
19830.51.9–1.44.33.31.0
19841.52.2–0.75.35.6–0.3
19853.02.90.14.03.20.8
Country: Finland
Source: Ministry of Finance4,5,6
19843.02.80.28.07.10.9
19853.52.80.76.05.90.1
Country: Netherlands
Source: Central Planbureau6
19830.10.8–0.74.42.91.5
19840.91.8–0.93.02.50.5
19851.82.1–0.31.52.6–1.1
Country: Sweden
Source: National Institute of Economic and Social Research4,6
19842.14.0–1.97.48.0–0.6
19852.12.2–0.14.87.4–2.6
Country: Switzerland
Source: Arbeitsgruppe fur Wirtschaftsprognose
1983–1.41.0–2.44.02.71.3
19841.52.6–1.13.03.3–0.3
19851.74.0–2.32.53.6–1.1
Appendix VI Fiscal Policy, Oil Prices, and the Exchange Rate

This appendix describes further the basis for the fiscal policy assumptions from which the information on unexpected changes described in the text are obtained, the expected signs on variables in the associated regressions, and a summary of some of the additional results obtained for fiscal policy and oil price innovations together with a note on the effect of exchange rate projection error.

Fiscal Policy Assumptions

The World Economic Outlook forecasters reckon to take into account existing policies, including announced and probable changes during the course of the forecast period. The quantification of this policy assumption—in the form of an estimate of the “fiscal impulse”—is more recent, however.39 To fill the gap, the sequence of source documents for the forecasts classified as “year ahead” was examined for qualitative comment that might be translated into quantitative terms. Relevant extracts are noted below and in Table 37 showing the resultant quantified forecasts and realizations. The latter, drawn from the World Economic Outlook 1987 data bank, extend back to 1977. Figures for earlier years are derived from an earlier World Economic Outlook source but do not appear to be fully homogeneous with the later series. In the text, reference is made to an alternative set of realizations, identified as the (negative) of the first difference of estimates of the structural budget balance. For these data the paper by Price and Mueller (1984), supplemented by later figures from issues of the OECD’s Economic Outlook, was used. These data are shown in Table 38. The two alternative realization series (Tables 37 and 38) lead to two corresponding alternative policy innovation measures, FP1 and FP2, respectively.

World Economic Outlook: Extracts on Fiscal Policy

January 1973 World Economic Outlook (for 1973). No basis for any assumption is given for Canada, Japan, or the United States. But for France the “stance of fiscal policy … is expected to be more expansionary than in 1972,” for the Federal Republic of Germany “… some reduction in the stimulative effect of fiscal policy …” is noted, whilst for Italy “the expansionary impact of fiscal policy is expected to increase in 1973… .” For the United Kingdom, comment suggests possibly some contraction.

December 1973 World Economic Outlook (for 1974). The outlook was dominated by consideration of the effect of the oil crisis. For Canada, the United States, France, and the United Kingdom, there is no comment. In Japan, fiscal policy is “likely to be more conservative” while in Germany a slight relaxation is hinted at. The World Economic Outlook is sceptical of efforts in July to restrain fiscal policy.

December 1974 World Economic Outlook (for 1975). For Canada some stimulus is indicated; for the United States the “… broad maintenance of the present fiscal stance.” In Japan “… relaxation of fiscal policy” is noted, with “stimulative action” in the Federal Republic of Germany and “no change” in Italy. In the United Kingdom, the “November budget would ease the cash position of industry.”

December 1975 World Economic Outlook (for 1976). A full set of forecast impulse figures is provided (except for Canada). They are derived from central government fiscal balance data.

March 1977 World Economic Outlook (for 1977). A full set of forecast fiscal impact figures is provided, now including Canada, again for the central government.

December 1977 World Economic Outlook (for 1978). No estimates are given, but there is a comment that “For 1978, a perceptible shift in the orientation of fiscal policies appears to be in the making.” There is also a comment on the stimulatory measures applied in the fourth quarter of 1977 by the United Kingdom, the Federal Republic of Germany, Japan, Canada, and the likely stimulus to come in the United States and Italy. Only France has indicated no relaxation though the World Economic Outlook thinks this “may be modified” in the course of the year.

February 1979 World Economic Outlook (for 1979). A fully quantified set of measures is available, this time with figures on a general government basis also available for three countries.

August 1979 World Economic Outlook (for 1980). No quantification, just a general reference to “caution” in fiscal policy.

Table 37.Forecasts and Realizations of Fiscal Impulse(In percent of GNP)
CanadaStates

United
JapanFranceFederal

Republic of

Germany
ItalyUnited

Kingdom
Forecasts1
19730.00.00.00.25–0.250.25–0.25
19740.00.0–0.250.00.250.250.0
19750.250.00.250.00.500.00.25
19760.0–0.50.0–0.1–0.5–1.0–1.0
1977–0.20.70.3–1.50.10.7–1.3
19780.50.250.50.00.50.250.5
1979–0.5–0.41.1–0.80.3–0.8–0.7
1980–0.25–0.25–0.25–0.25–0.25–0.25–0.25
1981–0.25–0.25–0.25–0.25–0.25–0.25–0.25
1982–0.50.0–0.51.0–0.5–0.5–0.5
1983–0.61.0–0.250.0–0.250.250.0
19840.0–0.6–0.5–0.4–0.20.0–0.3
1985–0.70.0–0.5–0.2–0.3–0.2–0.4
Realizations2
1973–0.4–0.51.2–0.3–0.51.72.2
1974–0.6–0.8–0.5–0.10.6–0.8–0.2
19752.53.22.42.61.71.52.7
1976–0.4–1.1–0.3–1.7–0.3–1.3–2.6
19771.80.00.2–0.3–0.3–0.8–1.6
19781.40.10.20.90.13.82.0
1979–0.4–0.81.10.10.1–1.7–0.8
1980–0.10.40.1–0.8–0.60.1–1.7
1981–1.10.0–0.51.1–0.50.7–1.4
19821.40.4–0.30.0–1.10.6–0.7
19830.61.6–0.50.10.20.00.7
19841.50.4–0.4–0.10.6–1.00.3
19850.50.3–0.6–0.3–0.50.8–0.1
Table 38.Fiscal Policy: Changes in Structural Budget Balance1(In percent of potential GNP)
1973197419751976197719781979198019811982198319841985
Canada+0.4+0.9–3.5+0.4–0.2–0.7+1.3–0.5+0.9–0.4–1.6–1.2–0.6
United States–0.2+0.9–1.6+1.3+0.2+0.3+0.3–0.5+0.9–0.9–0.7–0.5–0.8
Japan–0.3+0.4–2.6–1.0–1.1–2.8+0.6–0.2+0.6+0.3+0.5+1.0+0.5
France0.0+0.3–1.1+0.6–0.4–1.5+0.9+1.3–1.0–0.6–0.1+0.4+0.6
Germany, Fed. Rep. of+1.3–1.8–2.9+1.2+1.1–0.4–0.6–0.2+0.1+1.2+1.1+0.2+0.6
Italy+0.1+0.2–2.0+1.7+1.1–1.8–0.6+1.0–3.4+0.6+2.2–1.7–1.2
United Kingdom–2.8–0.1+0.5–0.2+1.7–2.1+0.6+1.1+2.9+1.5–1.3–0.8+0.5
Sources: For 1973–79, data are from Price and Müller (1984), Table 2; data for later years are from OECD Economic Outlook (December 1984; June 1985; May 1986; December 1986). There may be some minor inconsistencies in the data.

August 1980 World Economic Outlook (for 1981). No overall quantification, but a comment that no tax cut has been assumed for the United States and a general reference to the “restrictive” stance of fiscal policy (with no comment on any likely relaxation).

August 1981 World Economic Outlook (for 1982). Not many figures are quoted, but for 1982, “… estimates indicate a withdrawal of stimulus for all countries except the United States and France. For the United States, no impulse either positive or negative is projected; for France, an expansionary impulse equivalent to 1 percent of GDP.”

August 1982 World Economic Outlook (for 1983). There is a general reference to the policy analysis of the March 1982 World Economic Outlook as “still valid.” This noted, inter alia, that “the general trend towards less expansionary fiscal policies observed since 1979 … is estimated to have continued in 1981” and went on … “It is expected to intensify in 1982 but to moderate in 1983.” For the United States, there is a sceptical discussion of administration policy claims, which allow one to suggest a forecast of expansion, while for the Federal Republic of Germany, the discussion indicates “restraint” for Japan—perhaps a minor withdrawal of stimulus. There is a specific forecast for Canada that the cyclically adjusted budget deficit would decline by less than 1 percent of GNP in 1983, but very general indications for the remaining countries—perhaps no change in France, an unwanted stimulus in Italy.

August 1983 World Economic Outlook (for 1984). A full set of quantified estimates for central government fiscal impulse.

September 1984 World Economic Outlook (for 1985). A full set of quantified estimates for both the central and general government. For 1985, the differences are very small.

Oil Price Forecasts

The oil price series employed is the export unit value of major oil exporters for which World Economic Outlook forecasts are available from the source documents. The full series of forecasts, realizations, and innovations is shown in Table 39. Because the innovations are dominated by the two rounds of oil price increases, dummies for these two shocks were used in the regressions as an alternative to the continuous series.

Regression Evidence

In the regressions the forecast error is regressed on the oil price and fiscal policy innovations. The latter are split into two components—“own” fiscal policies and “external” (other countries’) fiscal policies. The weighting for the latter is supplied either by GNP or by import weights.

Table 39.Oil Prices: Forecasts and Realization(In percentage change)
1973197419751976197719781979
Forecast5.233.06.35.38.50.010.1
Realization40.0225.85.16.39.40.445.9
Error–34.8–192.81.2–1.0–0.9–0.4–35.8
198019811982198319841985
Forecast20.010.54.52.5–1.50.0
Realization63.69.8–4.3–11.4–2.5–4.3
Error–43.60.78.813.91.04.3
Table 40.“Expected” Signs in Regression of Forecast Error on Unexpected Changes
Own Fiscal

Policy
Other Fiscal

Policy
Oil

Prices
Oil

Dummies
Output+++
Inflation+++
Balance of payments++

As the fiscal impulse measure is signed positively for output-expansionary fiscal policy (assuming a broadly “Keynesian” mechanism), positive innovations (forecast minus realization) indicate that policy was less expansionary than predicted, and might be expected to be associated with a positive (forecast minus realization) output error. Thus the expected sign on the fiscal policy innovations in the output error equations is positive, unless there is complete crowding out.40 The signs to be “expected” in the inflation error equations are less obvious to the extent that the inflation response to fiscal policy depends on the response of the exchange rate, for which a priori indications are themselves ambiguous. Finally, in the balance of payments equations, “own country” and “external” policy would on the same broadly Keynesian grounds, be expected to have opposing effects. The pattern of “expected” signs indicated by these considerations is summarized in Table 40.

The regression results are summarized in Table 41 for the measure of fiscal policy FP2. Use of the alternative measure, FP1, does not greatly recondition the results. There is a discussion in the main text of what happens when the regressors are extended to include actual oil price and fiscal policy variables.

Table 41.Effects of Unexpected Changes in Fiscal Policy and Oil Prices on Forecast Error: Significant Results
Fiscal Policy and Oil PricesFiscal Policy and Oil Price Dummies
GNP weightsImport weightsGNP weightsImport weights
Output:At 5%, U.S.: oilAt 5%, U.S.: oilOutput:At 5%, U.S.: D1At 5%, U.S.: D1
Japan: oilJapan: oilJapan: D1Japan: D1
France: FP (other)1France: FP (other)1France: FP(other)1U.K.: D1
U.K.: FP (own)U.K.: FP (own)U.K.: FP (other)1FP (own)
FP (other)1FP (other)1FP (own)FP (other)1
OilOilD1
At 10% additionally,At 10% additionally,At 10% additionally,At 10% additionally,
NoneItaly: FP (other)1NoneGermany: FP (own)1
France: FP (other)1
Inflation:At 5%, Canada: oilAt 5%, Canada: oil
U.S.: oilU.S.: oilInflation:At 5%, Canada: D1. D2At 5%, Canada: D1. D2 FP(own)1
Japan: oilJapan: oilU.S.: D1U.S.: D1
France: oilFrance: oilJapan: D1Japan: D1
U.K.: oilU.K.: oilFrance: D1France: D1
FP (other)FP (other)U.K.: D1, D2U.K.: D1, D2
Italy: oilItaly: oilFP (other)FP (other)
Italy: D1Italy: D1
At 10% additionally,At 10% additionally,At 10% additionally,At 10% additionally,
Canada: FP (own)1NoneFrance: D2France: D2
FP (own)
Italy: D2Italy: D2
Balance of payments:At 5%, U.K.: oilAt 5%, U.K.: oilBalance of payments:At 5%, Japan: D2At 5%, Japan: D2
FP (own)FP (own)U.K.: FP (own)U.K.: FP (own)
France: FP (own)1France: FP (own)1Germany: D2Germany: D2
At 10% additionally,At 10% additionally,At 10% additionally,At 10% additionally,
NoneCanada: FP (other)’U.K.: D1, D2U.K.: D1, D2
France: FP (own)1France: FP (own)1

Exchange Rate Effects

As noted in the text, World Economic Outlook forecasts dispose of a “working assumption” about exchange rates, latterly that real exchange rates remain constant over the forecast horizon at the levels reached in a recent period, formerly that nominal exchange rates will remain at recent levels. The precise nature of these assumptions, for the set of year-ahead forecasts defined in this study is set down in Table 42. The associated ex post error (defined as the forecast minus actual in percent of actual) is shown in Table 43.

The exchange rate assumption is not on quite the same level as the assumptions made about fiscal policy and oil prices and, as argued in the main text, it does not seem appropriate to treat the ex post exchange rate errors in quite the same fashion as the fiscal policy and oil price innovations. In any case, it would not be difficult to argue that, the lags in the process being as long as they are, exchange rate errors would not be likely to impact strongly on output growth or inflation over the typical forecast horizon. It might be supposed that the balance of payments forecasts, on the other hand, could be materially affected by these errors. It should be noted that, as the balance of payments data are defined in dollar terms, they will not, except for the United States, reflect the effects of changes in the value of national currency numeraires. A particular reason for examining the contribution of the exchange rate assumption error to the balance of payments forecast errors is of course simply that the balance of payments appears to be least well forecast among the principal magnitudes (though, as noted in the text this is by no means a World-Economic-Outlook-specific phenomenon).

In light of these considerations the contribution of exchange rate assumption error to balance of payments errors was examined by regressing the latter on the former. In view of the likely lags it was felt appropriate to perform this regression only for the year-ahead forecasts.41

Table 42.Exchange Rate Forecast Assumptions: Year-Ahead Forecasts
WEO SourceYear ForecastConstancy Assumption in WEO
January 19711971(Presumably at recent “fixed” levels)
January 19731973(Presumably at recent “fixed” levels)1
December 19731974“Present” levels2
December 19741975Values of October 21–25, 19743
December 19751976Average October 1975
March 19771977Average December 1976
December 19771978Average November 1977
February 19791979Average December 1978
August 19791980Average July 1979
August 19801981Average July 1980
August 19811982Average May–July 1981
August 19821983Exchange rates “in effect just after the realignment of rates among the European Monetary System countries in mid-June 1982”4
August 19831984Average May 1983
September 19841985Average June 1984
October 19851986Rates on July 22, 19855
October 19861987Real levels of 2 weeks prior to September 5, 1986
Table 43.Exchange Rate Assumption Error(In percent)
CanadaUnited

States
JapanFranceFederal

Republic of

Germany
ItalyUnited

Kingdom
19733.189.46–6.01–4.90–10.2210.663.95
1974–3.01–2.614.047.20–2.298.010.15
19754.693.70–2.89–5.19–1.37–2.668.54
1976–5.11–1.77–3.694.97–6.4320.6611.49
19775.191.58–8.94–0.41–1.643.63–2.42
19786.035.46–11.50–0.55–4.144.521.88
1979–0.98–0.9212.38–1.22–2.69–0.92–6.83
19800.06–1.284.23–1.68–1.393.87–3.75
1981–2.39–13.57–11.0013.699.4015.430.65
1982–1.35–8.065.438.60–6.046.995.61
1983–5.22–3.43–10.706.25–2.054.8111.03
19841.42–9.03–5.556.624.238.317.97
19851.74–6.37–2.110.341.597.091.49
198611.7421.81–22.93–5.36–9.78–4.5014.37

The results of the regressions were to produce insignificant coefficients on the exchange rate error term except in two cases, those of Japan and the United States. In the former case a negative coefficient was found, in the latter a positive one. The negative sign is what would be predicted on a “partial equilibrium” basis where the unexpected exchange rate movement can be thought of as an autonomous factor and volume effects dominate valuation effects. Accordingly, the opposing result found for the United States suggests the significance of J-curve effects. More particularly, the results probably reflect the comparatively large fluctuation in the exchange rate error assumption (Table 43) for these two countries (especially in 1986).

References

    Ash, J.C.K. and D.J.Smyth,“The Accuracy of OECD Forecasts of the International Economy,” paper presented at the International Symposium on Forecasting (London, 1984).

    Burns, Sir Terence,“The Interpretation and Use of Economic Predictions,” Proceedings of the Royal Society, Series A (London, 1986).

    Cooper, Richard N.,“Economic Interdependence and Coordination of Economic Policies,”in Handbook in International Economics, Vol. 2, ed. by RonaldW. Jones and PeterB. Kenen (Amsterdam: North-Holland, 1984).

    Crockett, Andrew and MorrisGoldstein,“Strengthening the International Monetary System: Exchange Rates, Surveillance, and Objective Indicators,” Occasional Paper No. 50 (Washington: International Monetary Fund, 1987).

    Goldstein, Morris,“The Global Effects of Fund-Supported Adjustment Programs,” Occasional Paper No. 42 (Washington: International Monetary Fund, 1986).

    Heller, PeterS., RichardD.Haas, and AhsanMansur,“A Review of the Fiscal Impulse Measure,” Occasional Paper No. 44 (Washington: International Monetary Fund, 1986).

    Holden, K. and D.A.Peel,“On Testing for Unbiassedness and Efficiency of Forecasts,” University College of Aberystwyth Discussion Papers (Aberystwyth, 1987).

    Holden, K., D.A. Peel, and B.Sandhu,“The Accuracy of OECD Forecasts” (mimeographed; University of Liverpool, 1987).

    International Monetary Fund,Report on the World Current Account Discrepancy (Washington: International Monetary Fund, 1987).

    Kenen, P.B. and S.B.Schwartz,“An Assessment of Macroeconomic Forecasts in the International Monetary Fund’s World Economic Outlook,”Working Papers in International Economics (Princeton University: International Finance Section, 1986).

    Llewellyn, John and HaruhitoArai,“International Aspects of Forecasting Accuracy,” OECD Economic Studies (Autumn1984), pp. 73117.

    Llewellyn, John, StephenPotter, and LeeSamuelson,Economic Forecasting and Policy (London: Routledge and Kegan Paul,1985).

    Osborn, Denise R. and FrancisTeal,“An Assessment and Comparison of Two NIESR Econometric Forecasts,” National Institute Economic Review,No. 88 (May1979), pp. 5062.

    Price, Robert W.R. and PatriceMüller,“Structural Budget Indicators and the Interpretation of Fiscal Policy Stance in OECD Economies,” OECD Economic Studies (Paris), No. 3 (Autumn1984), pp. 2871.

    Savage, David,“Assessment of the National Institute’s Forecasts of GDP, 1959–82,” National Institute Economic Review (London), No. 105 (August1983).

    Smyth, David J.,“Short-Run Macroeconomic Forecasting: The OECD Performance,”Journal of Forecasting (New York), Vol. 2 (January–March1983), pp. 3749.

    Smyth, David J., and J.C.K.Ash, “Forecasting Gross National Product, the Rate of Inflation and the Balance of Trade: The OECD Performance,”Economic Journal (London), Vol. 85, (June1975), pp. 36164.

    Theil, Henri,Applied Economic Forecasting (Amsterdam: North-Holland, 1966).

    Wallis, K.F.,et al.(eds.)Models of the UK Economy 3 (London: Oxford University Press, 1986).

Mr. Artis is professor at the University of Manchester. He prepared this report as a consultant to the Fund’s Research Department. The author is grateful to officers of the IMF for inspiring and helping in the execution of this study, most particularly to Flemming Larsen; to John Llewellyn at the OECD for substantial assistance; to Peter Kenen for encouragement and example: to Eric Kiernan and Sue Duckworth for research assistance; and to Hilary Thornton for secretarial services. None of the above bears any responsibility for opinions expressed.

References to the World Economic Outlook are generally to the forecasting process; World Economic Outlook reports have been published annually by the Internalional Monetary Fund since 1980. and twice a year since 1984.

This quotation, which could have been applied in identical or very similar terms to other World Economic Outlook reports, comes from the published World Economic Outlook of May 1980.

Thus, despite the flexibility with which “current policy” is interpreted, World Economic Outlook procedures fall far short of the contemporary identification in the literature of “policy” with a rule for the adjustment of policy instruments (and so “policy change” with a change in the rule) and are closer to the traditional identification of “policy” with the instrument settings themselves (and “policy change” with a change in these settings).

In the original case considered by Theil (1966) the “no change” naive forecast referred to the levels of the series and the RMSE of such a forecast is of course simply the RMSE of the series itself.

See Appendix III for a fuller discussion of these points.

Given the conditional nature of the forecasts, there need be no necessary presumption that forecast errors should be serially uncorrelated or preserve the desirable efficiency and freedom-from-bias properties described above in the regression of realizations on the “raw” forecasts, uncorrected for the effect of “innovations.” For example, because of the nature of the policy and oil price assumptions, these innovations may very well be, themselves, serially correlated. However, as will be shown, there is not a great deal of evidence of bias or inefficiency in the uncorrected forecasts in any case.

For internal purposes, this practice of taking a long forward look was instituted even earlier.

Extreme examples arise when rebasing involves a substantially new representation for a particular activity: for example, the behavior of volume estimates of GDP may be substantially different for an economy which undergoes a resources boom depending upon whether the relevant activity weights are pre- or post-boom.

This distinction follows that in the study of forecasting in the Fund by Kenen and Schwartz, though (as described in Appendix I), the actual classification of forecasts and the outturn data employed here are different from theirs.

The dates referred to are the dates of the documents used in the study, some of which are published, some unpublished. Publication lags have varied between one and two months.

Generally, in commenting on the realization-prediction regression results in what follows, we may take guidance from the t statistics estimated on the coefficients. It is worth noting, though, that the individual t-statistics are not always a secure guide to the results to be obtained from the appropriate joint test of the restrictions. See Appendix III.

The world current account discrepancy has recently been the subject of an extensive analysis that concluded that the main source of the discrepancy arises in the services account; see International Monetary Fund (1987).

Because of delays in the production of data for these countries, however, it must be pointed out that the greater part of “last year’s growth” is estimated in any case.

The following section shows that OECD’s balance of payments forecasting is similarly weaker than its output and inflation records.

An account of OECD forecasting methods appears in Llewellyn et al. (1985).

Full details of the basis for the comparisons are given in Table 15.

Since only deviations from forecasts are being considered, rather than deviations from socially desirable target values, some of the objections about the quadratic form do not apply when that is being used to derive a social welfare function to direct the optimal application of policy.

This eliminates the possibility of merging two logically distinct methods of improving a forecast—on the one hand, using the knowledge to be gained from the realization-forecast regression of the forecast itself, on the other, exploiting the information content of an alternative forecast.

The author is most grateful to John Llewellyn for assisting him in the updating and to those individuals who supplied him with national forecasts and outturns.

Extensions of the series were also not obtained for Italy for 1983–85.

The problem was smaller in the case of the OECD comparison (see pp. 7–8) because this used the July issues of OECD’s Economic Outlook, where appropriate, to align the source dates more closely than is possible here.

Japan’s forecasts are on a fiscal year (April–March basis); see Appendix V.

For purposes of the count, the multiple forecasts for the United States and for the Federal Republic of Germany were reduced to one in each case by choosing that forecast with the overall lowest average absolute error in forecasting output.

This apparent deterioration is consistent with the shift in sourcing dates for the World Economic Outlook forecasts.

In calculating the statistic, a single national forecast was used for each country, following the same selection principle for the two cases of multiple national forecasts as in the computation of the synthetic Group of Seven and “Europe” aggregates.

Because of the odd number of countries making up the Group of Seven, the statistic cannot be zero, except where there is a zero error for an odd number of countries.

The appreciation of events contained in this World Economic Outlook included also the point that the exchange rate assumptions used in the previous December’s projections had already been shown to be substantially out of line with developments.

See, for example, Savage (1983).

Since there is not a great deal of difference between World Economic Outlook and national country forecasts, it may make a good deal of sense to treat the latter (at least where they emanate from a government source) as a picture (given the constraints on doing better) of a desired evolution of the economy—otherwise actual and promised policy would be changed to make it so.

The estimating equations do not provide for any conditioning on monetary policy to help cope with this point.

The basic data used in the study can be obtained from the author or from the Current Studies Division of the Fund’s Research Department.

Current price GDP/GNP data (in billions of dollars at current exchange rates) were conveniently obtained from OECD’s 1986 edition of its National Accounts: 1960–1984, and updated from OECD’s Main Economic Indicators.

An exception is the detailed forecasts of export and import growth, for which the current-year forecast series can only be made complete by using the February 1979 document.

Accordingly the figures reported for these cases are the values of the constant term in a regression which also included a dummy variable for Western Hemisphere observations.

As the ratio of the RMSE of the World Economic Outlook forecast to the RMSE of the “trend as forecast” alternative.

No account in any of this is taken of publication lags which may vary between the forecasts.

The fiscal impulse measure is described in Heller et al. (1986) where it is compared with other measures of fiscal stance, including the structural (cyclically corrected) budget balance. The first difference of the latter (with sign reversed) is a close correlate in principle of the fiscal impulse.

And assuming the absence of any significant reverse feedback from output error to fiscal innovation.

In principle, because of the importance of lags in the export and import responses to exchange rate changes, the within-year profile of the exchange rate error might be examined as well. However, the regressions described were fitted to annual average errors in both variables.

    Other Resources Citing This Publication