Further Cross-Country Evidence on the Accuracy of the Private Sector’s Output Forecasts
Author: Miss Grace Juhn and Mr. Prakash Loungani
Publication Date:
18 Apr 2002
eISBN:
9781589060975
Language:
English
Keywords:
SP; standard deviation; IMF projection; private sector forecast; private-sector assessment; forecast error; term recession; output forecast; Fiscal stance; Global financial crisis of 2008-2009; Private capital flows; Trade balance; GDP forecasting; Eastern Europe; Africa; Asia and Pacific
Download PDF (1.9 MB)

This paper evaluates the performance of Consensus Forecasts of real GDP growth for a large number of industrialized and developing countries for the time period October 1989 to December 1999. The questions addressed are the following: How accurate are private sector forecasts? How does their accuracy compare with that of the IMF’s World Economic Outlook? How well do forecasters predict rare events such as recessions or crises? Is discord among forecasters associated with lower forecast accuracy?

Abstract

This paper evaluates the performance of Consensus Forecasts of real GDP growth for a large number of industrialized and developing countries for the time period October 1989 to December 1999. The questions addressed are the following: How accurate are private sector forecasts? How does their accuracy compare with that of the IMF’s World Economic Outlook? How well do forecasters predict rare events such as recessions or crises? Is discord among forecasters associated with lower forecast accuracy?

“… the ability to produce accurate predictions of the course of the economy in the near-term is probably the main criterion by which the public judges the usefulness of our entire profession.”

Victor Zarnowitz (1986, p. 1)

This paper evaluates the performance of Consensus Forecasts of real GDP growth for a large number of industrialized and developing countries for the time period October 1989 to December 1999. The questions addressed are the following:

  • How accurate are private sector forecasts?

  • How does their accuracy compare with that of the IMF’s World Economic Outlook?

  • How well do forecasters predict rare events such as recessions or crises?

  • Is discord among forecasters associated with lower forecast accuracy?

The evidence on such questions is useful for a number of reasons. First, private sector capital flows have supplanted official funds as the dominant form of external financing for many countries. Hence, private-sector assessments of the relative macroeconomic outlook for various countries play a role in guiding the allocation of capital across the globe.

Second, many in the "official" sector are increasingly relying on these forecasts as a summary of the private sector’s assessment of the macroeconomic outlook. In addition to being extensively used in the multilateral institutions for this purpose,1 Consensus Forecasts are used by national government agencies, as revealed for example in the following quote from a speech by New Zealand’s central bank governor Donald Brash (1998):

“We do not ourselves make forecasts of the international economy, but instead use the monthly Consensus Forecasts.… We certainly have no reason to believe that we could produce better forecasts for our overseas markets than can the forecasters’on the ground’ in the countries concerned.”

Third, growth forecasts are an important component for several other forecasts, such as those of the trade balance and government fiscal balance.

Despite the increasing visibility of Consensus Forecasts, there has been very little independent analysis of their accuracy. To our knowledge, the only studies are by Artis (1997), Batchelor (1997), Harvey, Leybourne and Newbold (1999), and Gallo, Granger and Jeon (2002). The first two restrict attention to the G-7 countries, the third to the United Kingdom, and the last to the United States, the United Kingdom, and Japan. 2

I. Data Description and Terminology

Consensus Forecasts has provided macroeconomic forecasts for industrialized countries on a monthly basis since October 1989. Over time, the coverage has expanded to encompass many developing countries; forecasts for these countries are reported in the publication’s off-shoots, namely, Latin American Consensus Forecasts (published bimonthly since 1993), Asia Pacific Consensus Forecasts (monthly since 1995), and Eastern Europe Consensus Forecasts (bi-monthly since 1998). Each of these publications surveys a number of prominent financial and economic analysts and reports their individual forecasts as well as simple statistics summarizing the distribution of forecasts. The focus of this paper is on the mean forecast (the "consensus") and the standard deviation across forecasters.3

Table 1 provides a list of the 63 countries used in the analysis, the sample period over which forecasts are available for each, and whether they are classified as an "industrialized" country or a "developing" country. As noted, for the industrialized countries the forecasts start in October 1989 (with a single exception). For the developing countries, the starting dates are more varied: in about 25 percent of the cases, the starting date is between October 1989 and October 1993; in the remaining cases, the starting date is January 1995. Most of results in this paper are based on an "unbalanced" panel (i.e., countries enter the sample at different dates) in order to make use of all available information.

Table 1.

List of Countries

article image

The "event" being forecast is annual average real GDP growth. Every month (or, as noted above, every other month in the case of Latin American or East European countries) a new forecast is made of this event. To ensure consistency of treatment across countries, we study the bi-monthly sequence of forecasts for each event. To take a concrete example, suppose that the event is 1999 real GDP growth. The sequence of forecasts that we study for this event are the twelve forecasts made between February 1998 and December 1999. The first six forecasts, the ones made during 1998, are referred to as year-ahead forecasts; the six forecasts made during 1999 are called current-year forecasts.

In discussing the results, it will sometimes be convenient to index the forecasts by the forecasting horizon, the number of months before the terminal date. So, to continue the example, the February 1998 forecast is considered as made at a 23- month horizon and the December 1999 forecast as made at a one-month horizon.

The comparison with IMF forecasts is made on the basis of the April and October forecasts, both year-ahead and current-year. In terms of the notation just discussed, these are the forecasts made at horizons 21, 15, 9, and 3 months.

II. Accuracy of Private Sector Forecasts

The forecast errors are given by:

e(t)=A(t)−F(t),(1)

where A(t) is a vector of growth outcomes (the "actuals"), and F(t) is the corresponding vector of forecasts. A perennial issue in the forecasting literature is whether the "actual" value should correspond to the early releases of the data or later revisions. Using a very early release may not be satisfactory as the number may be highly preliminary. But using the final release may not be satisfactory either as it may incorporate information (such as revisions of weights, changes in methods of construction, etc.) that forecasters simply could not have been aware of at the time of the forecast. Given our large sample of countries, there will likely be quite a few data revisions of this kind. Our compromise is to use the real GDP data as reported in the May WEO of the following year—this likely falls in between a highly preliminary number and later revised versions.4

Three measures of forecast accuracy are used. The first is the Mean Absolute Error (MAE), which is the average across all countries and over all years of the differences between actual and forecast values, disregarding the sign of the error. The second is the root mean square error (RMSE). To compute RMSE, the forecast errors are squared and averaged over the sample to get the mean squared error (MSE); RMSE is the square root of MSE. The third measure, Theil’s U-Statistic (TU), is defined as follows:

TU(t)=RMSE(t)/[(A(t)−A(t−1)2/n)](2)

TU accomplishes two things. It scales RMSE by the variability of the underlying data, and it offers a way of evaluating forecasting performance relative to a "naive" forecast of no change in the growth rate between t—1 and t. TUs of less than 1 are said to beat the naive forecast.

The results are reported in Table 2, for the full sample and also for the industrialized and developing countries separately. There are two clear findings. First, as one would expect, the magnitude of the forecast error declines as the forecast horizon gets shorter. For instance, MAE for the full sample is 2 percent at a 23- month horizon and declines to just under 1 percent at the one-month horizon. Similar patterns are evident for RMSE and TU.

Table 2.

Accuracy of Private Sector Forecasts

article image

Second, while in absolute terms errors are larger for developing countries than for the industrialized countries (as shown by the MAE and RMSE columns), taking account of the variability of the underlying data reverses this conclusion. Values of TU are always a bit higher for the industrialized country sample than for developing countries. Another way of thinking about this result is that the year-ahead forecasts for industrialized countries either do not beat the naive forecast of an unchanged growth rate or just barely beat it, whereas for developing countries they do notably better than a naive forecast.

How accurate are the private sector’s output forecasts? One way to try to answer this question to note that real GDP growth averaged about 3 percent a year over this period (2.3 percent for industrialized countries; 3.6 percent for developing). Against this context, the accuracy of the forecasts, particularly of the yearahead forecasts, is not particularly impressive.

A preferable, and less subjective, attempt to evaluate accuracy is to compare the private sector’s forecast against those of forecasts available from other sources. The next section compares private sector’s forecasts against another prominent source of cross-country forecasts, the IMF’s World Economic Outlook (WEO).

III. Comparison of Private Sector and IMF Forecasts

The IMF’s World Economic Outlook (WEO), published in May and October of each year, reports current-year and year-ahead forecasts for member countries. For about 50 of the largest countries accounting for 90 percent of world output the forecasts are updated for each WEO exercise; these countries are referred to as “Group A” countries. For the other countries, the WEO forecasts are from the most recent Article IV consultation or IMF program document, but they are “incrementally adjusted to reflect changes in assumptions and global economic conditions.”5 The May WEO forecasts are compared with the April private sector Consensus Forecasts, and the October WEO forecasts with the October consensus. 6

Table 3 presents measures of accuracy for the two sources of forecasts. It is evident that the accuracy of the private sector’s forecasts is in almost every case a little bit better than that of the WEO forecasts.7 The differences are marginal in the case of forecasts for industrialized countries (particularly current-year forecasts), but rather more substantial in the case of developing countries. So, assessed against the performance of perhaps the single best-known alternative source of crosscountry forecasts, the private sector forecasts perform well in terms of accuracy.8

Table 3.

Comparison of Private Sector and WEO Accuracy

article image

As Diebold and Mariano (1995, p. 262) have emphasized, the superiority of one source of forecasts in terms of forecast accuracy does not necessarily imply that forecasts from other sources contain no additional information. To test if WEO forecasts add information to the Consensus forecasts, we estimate the following regression:

A(t)=cons⁢tant+b1CONSENSUS(t)+b2WEO(t)(3)

where, A(t) is the outcome, and CONSENSUS(t) and WEO(t) are the forecasts from the two sources. Equations of the kind shown above are based on the literature on forecast encompassing (see Fair and Shiller (1990), and Holden and Thompson (1997)). 9 If we cannot reject the null hypothesis that the estimated b2 is zero, the WEO forecasts are encompassed by the Consensus forecast. We report regressions with and without the constant term to see if the results of this exercise are sensitive to assumptions about the unbiasedness of the Consensus forecast.

The regression results are reported in Table 4. Results are reported for all horizons and countries pooled together and for each of the four horizons and two country types separately. This gives a total of 30 regressions. The range of estimates for b1 is between 0.7 and 1.3, with a substantial number of estimates very close to 1. The range of estimates for b2 is -0.2 to 0.4, with a majority of the estimates very close to zero. To the extent that the WEO forecasts are not encompassed by the Consensus forecasts, that tends to be the case for industrialized countries, and even then for current-year forecasts only.

Table 4.

Forecast Encompassing Tests

article image

Another dimension on which forecasts can be compared is directional accuracy. We compute whether the forecasted change in growth is in the same direction as the actual change in growth. The fraction of forecasts that get the direction right is shown in Table 5. For both sources of forecasts the fraction increases as the horizon gets shorter and is higher for industrialized than for developing countries in the case of current-year forecasts. The relative performance does not produce a clear winner: the WEO does better at a three-month horizon but generally a bit worse at the other horizons.

Table 5.

Directional Accuracy of Consensus and WEO Forecasts

article image

To summarize the results of this section, the results for directional accuracy are a statistical “dead heat,” whereas the evidence tends to favor the private sector forecasts as being a little more accurate than, and encompassing, the WEO forecasts. The dominance of private sector forecasts is not wholly unexpected. The Consensus forecasts are updated in a far more timely manner and with a shorter "production lag" than the WEO forecasts. The consensus is also an average of several individual forecasts which—even allowing for "herding" tendencies—should tend to produce a more accurate forecast. 10

It should also be noted that forecasts are only one component of the WEO; far more attention is devoted to analyzing the global outlook, risks to it, and special topics of policy interest. For Consensus Forecasts, however, forecasts are the raison d’etre.

IV. Forecasting Recessions

Fintzen and Stekler (1999, p. 309) note that “one of the most disturbing findings of forecast evaluations is that, in the United States, recessions have generally not been predicted prior to their occurrence.” As one country’s macroeconomic history only yields a few observations of recessions, the cross-country sample used here affords an opportunity for testing if the failure to forecast recessions is an ubiquitous feature of growth forecasts. 11 One limitation is that the term recession—that is, a year in which real G D P declined—has to be interpreted rather broadly to encompass cyclical downturns (as in the case of the United States in 1991), declines in output associated with transition from planned economies to market economies (as in the case of Hungary and Poland), and declines associated with crises of various kinds (e.g., the E R M crisis of 1992-93, the Mexican crisis (1995), and the global financial crises of 1997-99). There were a total of 72 episodes of recessions in our sample (Table 6).

The properties of forecasts during recession years are summarized in Table 7. As shown in the first column, very few recessions are predicted a year in advance; for instance, at a 13-month horizon negative growth was forecast in only 8 of the 72 episodes. There is then a discrete jump in the number of forecast of recessions; at a 11-month horizon the number of cases of negative growth forecasts has risen to 23 out of the possible 72 cases. 12 The number of cases of forecasts of recessions then rises steadily, reaching 54 at a one-month horizon.

While forecasters increasingly start to recognize recessions in the year in which they occur, the results in the second column show that the magnitude of the downturn is underpredicted in the vast majority of cases. For instance, even as late as December of the year of the recession, the forecast is more optimistic than the outcome in 40 cases.

Table 6.

List of 72 Episodes of Recessions

(Countries that experienced recessions over the sample period are shown in bold.)

article image
Table 7.

Forecast Performance During Recession Episodes

(Total number of episodes = 72)

article image

The final columns show the average forecast error at the different forecast horizons over all 72 episodes and also for the industrialized countries and developing countries separately. Note that average forecast errors continue to be quite substantial even for forecasts made fairly late in the year of the recession. For instance, at the three-month horizon, the average forecast error is 0.5 percentage points for industrialized countries and nearly 1.5 percentage points for developing countries; at the onemonth horizon the corresponding numbers are 0.2 and 0.8 percentage points.

A Goldman Sachs report (2001) 13 contains an extended discussion of why private sector economists tend to avoid forecasting recessions. 14 First, recessions are relatively rare events, which makes forecasting positive growth a pretty good bet in most years. (In the sample used in this paper, years marked by recessions comprise about 15 percent of the total number of country years.) Second, as a related point, macroeconomic models are built to capture normal relationships among variables and are hence not always suited to predicting recessions unless there is a clear large exogenous shock. Third, there is a “herding” tendency among forecasters. The report notes (p. 4) that,

“to forecast a recession substantially ahead of the pack, a forecaster must be willing to deviate from the consensus in an extremely transparent manner, knowing very clearly that in any given year a recession is a low probability outcome.”

V. Forecaster Discord and Forecaster Uncertainty

In addition to the mean forecast, users of Consensus Forecasts often look at the degree of discord across forecasters, as measured by the standard deviation of the individual forecasts. For instance, the high standard deviation of forecasts in the case of Japan over the last few years is taken to be a signal that developments in Japan have been particularly difficult to forecast in recent times. The high standard deviation could serve as a warning that the forecast accuracy for Japan’s growth may be low. Conversely, a low level of forecaster discord might suggest that the country’s growth prospects are relatively easy to forecast, and hence that one should expect that the forecast error will be low.

How reliable is forecaster discord as a predictor of accuracy? To investigate this issue, we estimate a regression of the (absolute value of the) forecast error on the standard deviation of the forecast and other variables. 15 The sample consists of 26 countries, listed in Table 8, for which individual forecasts are available. The data are pooled for these countries for four years (1995 to 1998) and for two forecast horizons (the current-year April forecast and the current-year October forecast); 16 this yields a total of 208 observations.

Table 8.

List of Countries Used in Forecaster Discord Regressions

article image

The results of the estimation are given in Table 9. In addition to the standard error, the following explanatory variables are included: (1) dummy variables for each region and each year; (2) dummy variables to test whether the results are due to outliers; and (3) adummy variable that takes the value 1 if the forecast was made in April, and zero if the forecast was made in October. Since, as was shown above, the forecast error is higher earlier on in the year than later, the expected sign of the coefficient on this dummy variable is positive.

Table 9.

Forecast Errors and Forecaster Discord

(Dependent variable: current-year forecast error, absolute value (AFE))

article image
Note: Numbers in parentheses are standard errors

The initial regression in column (1) shows that higher forecaster discord is indeed associated with higher forecast errors, controlling for the month of the forecast (April or October). The coefficient estimate is positive and significantly different from zero.

Adding on region-specific fixed effects (column (2)) and year-specific fixed effects does not materially affect the strength of the positive correlation between (absolute) forecast error and forecaster discord. The inclusion of d u m m y variables to pick up the effects of outliers (columns (4) and (5)) attenuates the correlation, but it remains positive and significantly different from zero.

Overall, these results provide some support for the common practice of using the standard deviation of the forecast as an rough indication of the difficulty of the forecasting "terrain," and consequently as one determinant of the magnitude of the forecast error.17

VI. Conclusions

This paper has assembled further evidence on the properties of private sector growth forecasts for a large sample of countries. The main questions addressed, and the answers suggested by the evidence, are as follows:

  • How accurate are private sector forecasts ?

  • In absolute terms, the magnitude of the errors tends to be larger for developing than for industrialized countries. However, growth is more variable for developing countries; if one adjusts for this fact, say by scaling the forecast error for a country by the variability of its growth, the errors are a bit smaller for developing countries.

  • How does private sector accuracy compare with that of the IMF’s World Economic Outlook?

  • Private sector and WEO forecasts seem to be quite similar. Measures of accuracy such as mean absolute error and root mean square error are better for Consensus forecasts than for WEO forecasts, but the differences are not overwhelming. Tests of directional accuracy do not reveal substantial differences between the two sources either. Nevertheless, the WEO forecasts are encompassed by the Consensus forecasts, suggesting that they do not add explanatory power to the private sector’s forecasts.

  • How well do forecasters predict recessions ?

  • Updating earlier work, we show that very few of the 72 recessions that occurred over the sample were predicted a year in advance and two-thirds remained undetected by the April of the year in which the recession occurred. In over half the cases, the forecast made in December of the year of the recession underestimated its extent. This predictive failure could arise either because forecasters lack the requisite information (in terms of reliable real-time data or reliable models) or because they lack the incentives to predict recessions; further work would be needed to discriminate between these two classes of theories.

  • Is forecaster discord a reliable predictor of forecast accuracy ?

  • There is a positive relationship between the two: when there is greater discord across forecasters, the forecast error tends to be larger, on average. At the same time, the relationship is not overwhelmingly strong. This means that forecast discord can be used as one element in trying to gauge the likely magnitude of the forecast error, but it cannot be used as the only element.

References

  • Alizadeh, S., Brandt, M., and Diebold, F.X., 2002, “Range-Based Estimation of Stochastic Volatility Models,” Journal of Finance, forth coming.

    • Search Google Scholar
    • Export Citation

  • Artis, Michael, 1997, “How Accurate Are the IMF’s Short-Term Forecasts? An Examination of the World Economic Outlook,” Staff Studies for the WEO (Washington, DC: International Monetary Fund).

    • Search Google Scholar
    • Export Citation

  • Batchelor, Roy, 1997, “How Useful Are the Forecasts of Intergovernmental Agencies?” (unpublished; London: City University Business School).

    • Search Google Scholar
    • Export Citation

  • Bates, J.M., and Clive Granger, 1969, “The Combination of Forecasts,” Operational Research Quarterly, Vol. 20, pp.513 –22.

  • Brash, Donald T., 1998, “Reserve Bank Forecasting: Should We Feel Guilty?” An address to the New Zealand Society of Actuaries. May be obtained on the Worldwide Web at http://www.rbnz.govt.nz/speeches/sp981021.htm.

    • Search Google Scholar
    • Export Citation

  • Diebold, Francis X., 1989, “Forecast Combination and Encompassing: Reconciling Two Divergent Literatures,” International Journal of Forecasting, Vol. 5, pp.589 –92.

    • Search Google Scholar
    • Export Citation

  • Diebold, Francis X., 2001, Elements of Forecasting (Thousand Oaks, California: Southwestern Publishing).

  • Diebold, Francis X., and Roberto Mariano, 1995, “Comparing Predictive Accuracy,” Journal of Business and Economic Statistics, Vol. 13 (July), pp.253 –63.

    • Search Google Scholar
    • Export Citation

  • Fair, R.C., and R.J. Shiller, 1990, “Comparing Information in Forecasts from Econometric Models,” American Economic Review, Vol. 80, pp.375 –89.

    • Search Google Scholar
    • Export Citation

  • Fintzen, David, and H.O. Stekler, 1999, “Why Did Forecasters Fail to Predict the 1990 Recession?” International Journal of Forecasting, Vol. 15, pp.309 –23.

    • Search Google Scholar
    • Export Citation

  • Gallo, Giampiero M., Clive W.J. Granger and Yongil Jeon, 2002, “Copycats and Common Swings: The Impact of the Use of Forecasts in Information Sets,” IMF Staff Papers, Vol. 49, No. 1, pp.4 –21.

    • Search Google Scholar
    • Export Citation

  • Goldman Sachs, 2001, Global Economics Weekly Issue No. 01/10 March 14

  • Granger, Clive W.J., and R. Ramanathan, 1984, “Improved Methods of Combining Forecasts,” Journal of Forecasting, Vol. 3, pp.197 –204.

    • Search Google Scholar
    • Export Citation

  • Harvey, David I., Stephen J. Leybourne, and Paul Newbold, 1999, “Analysis of a Panel of UK Macroeconomic Forecasts,” Research Paper 99/8 (Loughboro, UK: Loughborough University, Department of Economics).

    • Search Google Scholar
    • Export Citation

  • Holden, Ken, and John Thompson, 1997, “Combining Forecasts, Encompassing, and the Properties of UK Macroeconomic Forecasts,” Applied Economics, Vol. 29, pp.1447 –58.

    • Search Google Scholar
    • Export Citation

  • Loungani, Prakash, 2000, “The Arcane Art of Predicting Recessions,” Financial Times, December 18.

  • Loungani, Prakash, and 2001a, “How Clear Is the Crystal Ball? Reflections on the Accuracy of Growth Forecasts,” World Economics, Vol. 2 (January-March), pp.1 –8.

    • Search Google Scholar
    • Export Citation

  • Loungani, Prakash, 2001b, “How Accurate are Private Sector Forecasts? Cross-Country Evidence from Consensus Forecasts of Output Growth,” International Journal of Forecasting (July), pp.419 –32.

    • Search Google Scholar
    • Export Citation

  • Loungani, Prakash, 2002, “There Will Be Growth in the Spring: How Credible Are Forecasts of Recovery?” World Economics. Vol. 3 (January-March), spp.1 –6.

    • Search Google Scholar
    • Export Citation

  • Musso, Alberto, and Steven Phillips, 2002, “Comparing Projections and Outcomes of IMF Supported Programs,” IMF Staff Papers, Vol. 49, No. 1, pp.22 –48.

    • Search Google Scholar
    • Export Citation

  • Zarnowitz, Victor, 1986, “The Record and Improvability of Economic Forecasting,” NBER Working Paper 2099 (Cambridge, Massachusetts: National Bureau of Economic Research).

    • Search Google Scholar
    • Export Citation

  • Zarnowitz, Victor, and Louis A. Lambros, 1985, “Consensus and Uncertainty in Economic Prediction,” Journal of Political Economy, Vol. 95, No. 3, pp.591 –621.

    • Search Google Scholar
    • Export Citation
*

Grace Juhn is a student at Harvard University’s Kennedy School of Government, and Prakash Loungani is Assistant to the Director of External Relations at the International Monetary Fund. Work on this paper was completed while Juhn was a Research Assistant in the IMF’s Research Department. We acknowledge useful comments from Frank Diebold.

1

Publications such as the IMF’s World Economic Outlook (WEO), the World Bank’s Global Economic Prospects (GEP), and the OECD’s Economic Outlook (EO) contain references to the Consensus forecasts. See, for instance, WEO: Interim Assessment (December 1997, pp. 34-36), Staff Studies for the WEO (December 1997, pp. 23-25), and GEP (1999, p. 9).

2

Earlier work by Loungani (2001a and b) also contains an evaluation of Consensus Forecasts of output growth. This paper builds on that work in five ways: (1) the entire sequence of bi-monthly forecasts is studied, instead of just the April and October forecasts; (2) forecast encompassing tests are presented to test more formally for the relative information content of private and official sector forecasts, instead of the scatter plots presented in the earlier work; (3) evidence is presented on directional accuracy of consensus and WEO forecasts; (4) the relationship between forecaster discord and forecast accuracy is studied; and (5) the sample period is extended by a year, not a trivial increase when the sample period is as short as it is here. The additional year is particularly useful in updating the evidence on forecasting recessions that was presented in the earlier work.

3

In future work, it would be interesting to examine the properties of the median forecast as well.

4

For example, the 1990 forecast was compared to the realization as reported in the M a y 1991 W E O. In cases where this was not possible, because the data were not reported, w e used the first available realization reported in the W E O

5

Preface to October 1998 WEO.

6

The correlation between Consensus Forecasts for any two adjacent months is very high, 0.95 or better. This suggests that our results are not likely to have been much affected by using the May forecasts instead of the April forecasts.

7

One interesting extension to pursue would be to see if forecast accuracy in the case of countries with IMF-supported programs differs from that in other cases. On the one hand, forecasts for program countries are subject to greater scrutiny, which may lead to greater accuracy. On the other hand, forecasts for program countries are often arrived at after negotiations with the country’s authorities and may not represent true forecasts. See Musso and Phillips (2002) for a further discussion and evidence on the accuracy of projections made as part of IMF-supported programs.

8

We carried out a test, based on Diebold and Mariano (1995) and Diebold (2001, pp. 293-94), of whether the better performance of the Consensus relative to the WEO is statistically significant. Our preliminary results suggest that it is, but this result will need to be tested more rigorously in future work. One reason is that the test is intended for a time series rather than a panel data context; we used fixed effects to control for the panel nature of our data, but this may not be an adequate control.

9

Equations of this kind can also be motivated on the basis of an older literature on combining forecasts (Bates and Granger (1969), and Granger and Ramanathan (1984)), where the focus is on finding the optimal linear combination of available forecasts of an event. Diebold (1989) discusses the links between the forecast combination and forecast encompassing literatures.

10

0See Gallo, Granger, and Jeon (2002) for evidence on copycat behavior among the individual forecasters included in the Consensus Forecasts.

11

For evidence on how well recoveries are forecast, see Loungani (2002).

12

That this jump coincides with the arrival of a new year suggests that there is a heightened focus by both forecasters and their clients in the growth outcomes for the current year and perhaps lesser interest in outcomes for the following year.

13

Cited with permission from Goldman Sachs.

14

In a related discussion, Loungani (2000, 2001a) discusses two classes of theories for why recessions might not be forecast. The first is that the information needed is lacking: forecasters either do not have access to reliable real-time information or lack reliable models for translating available information into predictions of a recession. The second is that the incentives for producing an "outlier" forecast (a recession or a strong boom) are lacking.

15

Recent work (e.g., Alizadeh, Brandt, and Diebold, 2002) makes it clear that the range of forecasts (that is, max.-min.) can be a very informative volatility measure. It would be interesting to use the range instead of the standard deviation in regressions of the sort reported in Table 8.

16

In principle, one could carry out a similar analysis for the year-ahead standard deviation as well.

17

Zarnowitz and Lambros (1985) and Gallo, Granger, and Jeon (2002) caution against using the standard deviation (across analysts) as a measure of the standard deviation of the consensus.

Same Series

Other IMF Content

Other Publishers

Cover IMF Staff Papers
IMF Staff Papers, Volume 49, No. 1
Author: Mr. Robert P Flood
Volume/Issue:
Volume 2002: Issue 002
Publisher:
International Monetary Fund
ISBN:
9781589060975
ISSN:
1020-7635
Pages:
164
DOI:
https://doi.org/10.5089/9781589060975.024