Artis, Michael, “How Accurate are the IMF’s Short-term Forecasts? An Examination of the World Economic Outlook”, Staff Studies for the WEO, 1997.
Batchelor, Roy, “How Useful are the Forecasts of Intergovernmental Agencies?” City University Business School, 1997. (Provided by Consensus Forecasts, Inc.)
Beach, William W., Aaron B. Schavey, and Isabel M. Isidro, “How Reliable are IMF Economic Forecasts?,” Heritage Foundation Report 99-0 5, August 27, 1999.
Brash, Donald T., 1998, “Reserve Bank Forecasting: Should We Feel Guilty?,” An address to The New Zealand Society of Actuaries (http://www.rbnz.govt.nz/speeches/sp981021.htm).
Fintzen, David and H.O. Stekler, “Why Did Forecasters Fail to Predict the 1990 Recession,” International Journal of Forecasting 15 (1999), 309-323.
Gallo, Giampiero M., Clive Granger and Yongil Jeon, “The Impact of the Use of Forecasts in Information Sets”, Deutsche Bank Research, Research Notes in Economics and Statistics, September 1999.
Harvey, David I., Stephen J. Leybourne and Paul Newbold, “Analysis of a Panel of UK Macroeconomic Forecasts,” Loughborough University, Department of Economics Research Paper No. 99/8, July 1999.
Holden, Ken and D.A. Peel, “On Testing for Unbiasedness and Efficiency of Forecasts,” The Manchester School Vol. LVIII No. 2, June 1990, 120-27.
Kenen, Peter and Stephen B. Schwartz, “An Assessment of Macroeconomic Forecasts in the International Monetary Fund’s World Economic Outlook,” Princeton University, Working Papers in International Economics G-86-04, December 1986.
McNees, S.K. (1991), Forecasting Cyclical Turning Points: The Record in the Past Three Recessions. In K. Lahiri and G.H. Moore, Leading Economic Indicators: New Approaches and Forecasting. New York: Cambridge University Press.
Nordhaus, William D. “Forecasting Efficiency: Concepts and Applications,” The Review of Economics and Statistics, 69, 1997, 667-74.
Zarnowitz, Victor and Louis A. Lambros, “Consensus and Uncertainty in Economic Prediction”, Journal of Political Economy, 1985, vol. 95, No. 3, 591-621.
I thank Tam Bayoumi, Eduardo Borensztein, Peter Clark, Kajal Lahiri, and Ratna Sahay for valuable suggestions and support. I am also grateful to seminar participants at the IMF and the Federal Reserve Board and to numerous other individuals for comments and data. Grace Juhn provided outstanding research assistance.
Publications such as the IMF’s World Economic Outlook (WEO), the World Bank’s Global Economic Prospects (GEP) and the OECD’s Economic Outlook (EO) contain references to the Consensus forecasts. See, for instance, WEO: Interim Assessment (December 1997, pp. 34-6), Staff Studies for the WEO (December 1997, pp. 23-25) and GEP (1999, p. 9).
Despite the increasing visibility of Consensus Forecasts, there has been very little independent analysis of their accuracy. To my knowledge, the only studies are by Artis (1997), Batchelor (1997), Harvey, Leybourne and Newbold (1999), and Gallo, Granger and Jeon (1999). The first two restrict attention to the G7 countries, the third to the United Kingdom, and the last to the United States, the United Kingdom and Japan.
July forecasts will be used in the comparison with OECD forecasts, whereas the comparison with World Bank forecasts will require some other months, as discussed later.
This choice was implemented by using the real GDP data as reported in the May WEO of the following year. For example, the 1990 forecast was compared to the realization as reported in the May 1991 WEO. In cases where this was not possible because the data were not reported, I attempted to use the first available realization reported in the WEO.
There are two classes of theories for why recessions are not forecast. The first is that the information needed is lacking: forecasters either do not have access to reliable real-time information or lack reliable models for translating available information into predictions of a recession. The second is that the incentives for producing an “outlier” forecast (a recession or a strong boom) are lacking. For instance, some researchers and private forecasters argue that the incentives are tilted towards not predicting a recession. For instance, Zarnowitz argues that “predicting a general downturn is always unpopular, and predicting it prematurely ahead of others may prove quite costly to the forecaster and his customers” (1986, p. 9). Gary Shilling, a private forecaster, states: “Most economists are paid to be cheerleaders. Whistle blowers are unemployable” [Smalhout (2000)].
Forecast smoothing has been found in many other studies of forecasting performance. Nordhaus (1987) advances a couple of conjectures to explain this finding. The first is that forecasters are fearful that “jumpy” or “jagged” forecasts will be treated as inconsistency by their bosses or customers. Second, “studies from behavioral psychology suggest that people tend to hold on to prior views too long.”
The second test of efficiency used here is sometimes also used as a test of unbiasedness. However, as Holden and Peel (1990) demonstrate, this test provides a sufficient, but not a necessary condition for unbiasedness. They suggest that “correct inferences concerning unbiasedness can be obtained by testing whether the forecast error has a mean of zero” (p. 124).
Year-ahead forecasts are reported for industrialized countries; for developing countries the reporting of year-ahead forecasts is less systematic.
Preface to October 1998 WEO.
However, if the near-perfect collinearity results holds up, it can be useful in evaluating the validity of theories on the sources of bias in economic forecasts. For instance, Beach, Schavey and Isidro (1999) have alleged that IMF growth forecasts are too optimistic in cases where the countries have IMF programs because the odds of program failure are not adequately reflected in the forecasts. Since the private sector is not subject to the same pressures, it is difficult to understand why its forecasts end up so close to IMF forecasts. Conversely, there is a large literature that attempts to explain bias in private sector forecasts in terms of the incentives that these forecasters face and strategic behavior among forecasters. Since IMF (and OECD and World Bank) forecasters do not face the same incentives and are not engaged in strategic behavior vis-a-vis the set of private forecasters, it is once again difficult to explain the collinearity in forecasts. My conjecture, therefore, is that the near-perfect collinearity comes about because both private and multilateral forecasters are essentially reliant on official (government) forecasts, and lack either the information or the incentive to deviate too much from the government forecast. This conjecture can be tested by extending the analysis to examine the correlations among private sector, multilateral and government forecasts.
In principle, one could carry out a similar analysis for the year-ahead standard deviation as well.