Optimism Bias in Growth Forecasts-The Role of Planned Policy Adjustments

Kareem Ismail; Roberto Perrelli; Jessie Yang

doi:10.5089/9781513560373.001.A001

Jump to Content Jump to Main Navigation

Optimism Bias in Growth Forecasts-The Role of Planned Policy Adjustments

Author: Kareem Ismail¹, Mr. Roberto Perrelli², and Jessie Yang

¹ 0000000404811396https://isni.org/isni/0000000404811396International Monetary Fund
| ² 0000000404811396https://isni.org/isni/0000000404811396International Monetary Fund

Contributor Notes

Author’s E-Mail Address: KIsmail@imf.org; RPerrelli@imf.org; YYang2@imf.org

Publication Date:: 08 Nov 2020

eISBN:: 9781513560373

Language:: English

Keywords:: Fiscal adjustment; IMF-supported programs; forecast error; optimism bias.; WP; optimism bias; IMF program document; IMF team; policy adjustment; IMF mission; program forecast; relevant IMF program document

Download PDF (1.8 MB)

Are IMF growth forecasts systematically optimistic? And if so, what is the role of planned policy adjustments on this outcome? Are program forecasts as biased as surveillance forecasts? We try to answer these questions using a comprehensive database on IMF forecasts of economic growth in surveillance and program cases during 2003–2017. We find that large planned fiscal and external adjustments are associated with optimistic growth projections, with significant non-linearities for both program and surveillance cases. Specifically, we find evidence that larger planned fiscal adjustment is associated with higher growth optimism in IMF non-concessional, non-precautionary financial arrangements. Our results show the tendency for optimism has persisted in the period after the Global Financial Crisis. Moreover, the strong correlation between the magnitude of the optimism and expected fiscal consolidation provides a cautionary signal for the post-COVID IMF projections as countries embark on a path of fiscal adjustment.

Abstract

I. Introduction

Are International Monetary Fund (IMF or the “Fund”) output growth forecasts systematically optimistic? And if so, what is the role of planned policy adjustments on this outcome? Are program-specific forecasts as optimistic as forecasts for countries under regular surveillance?

In this paper we attempt to shed light on these and related questions using a comprehensive database of real GDP growth forecast errors during 2003–2017. Our study covers IMF forecasts both for countries with and without a financial program with the Fund. For the former, we use the macroeconomic projections prepared by the IMF for each review of non-concessional, non-precautionary financial arrangements (henceforth simply called programs) approved by the IMF Executive Board over the past two decades. We compile these program-specific forecasts from the IMF’s Monitoring of Fund Arrangements (MONA) database—a comprehensive database on the design, targets, compliance, and economic performance (projected and observed) of IMF-supported financial arrangements. For the non-program observations, we use the forecast errors recorded in each vintage of the IMF’s World Economic Outlook (WEO). The use of vintage-specific data is warranted to control for information not available at the time the forecasts were made. What differentiate this work from others in this area as we explain below is the wide scope of coverage of Fund forecasts, as well as the close investigation of the association between growth optimism and planned policy adjustments, including evidence of non-linearity.

Our goal is to better understand the role of planned policy adjustments on unveiling systematic errors in IMF forecasts. To attenuate potential endogeneity problems, we distinguish between planned policy adjustments—i.e., those under the baseline macroeconomic scenario projected for each country, including those with an IMF program— and the actual policy adjustment that took place during the forecast horizon. We control for several exogenous factors that may also have affected growth outcomes. In addition, we examine whether planned policy adjustments have non-linear effects on growth forecast errors, and whether the latter increase with the length of the projection horizon. Our paper supplements the work of Blanchard and Leigh (2013, 2014), which estimate the size of fiscal multipliers in a sample of advanced European economies during 2010–2011 using a regression of growth forecast errors on planned fiscal adjustments. Their work, however, does not attempt to answer the interesting questions we raise in this paper.

Our results suggest that planned policy adjustments are associated with optimism in both program and non-program observations, with significant non-linearities: larger planned policy adjustments (defined in our analysis as episodes where planned adjustment exceeded half a standard deviation above the historical average actual adjustment for each country) tend to display stronger association with optimism bias. We also find that optimism on trading partners’ growth is positively associated with a country’s own growth optimism. Moreover, bullish projections on oil prices are associated with sanguine real GDP growth rate projections in both oil importers and exporters. Conversely, we find no evidence that planned structural adjustment, as proxied by the number of structural benchmarks agreed under an IMF program, plays a statistically significant role. Finally, the results suggest that IMF forecasts are associated with an optimism bias of around 0.2 percentage points of a country’s real GDP growth rate each year into the projection horizon resulting in an optimism drift. Over a five-year horizon, the cumulative impact of IMF’s optimism bias adds up to 1 percentage point of a country’s real GDP growth rate in addition to optimism associated from planned policy adjustment.

In understanding the basis of these findings, we also investigate the relationship between planned and actual fiscal adjustment. We find strong positive association, which allows us to analyze the significance of the association of actual fiscal adjustment in growth optimism in an instrumental variable analysis. We find that indeed actual fiscal adjustment carries a strong positive association with growth optimism, advancing the proposition that the underestimation of fiscal multipliers may be playing a role in the observed association between planned fiscal adjustment and growth optimism, particularly in programs. The nonlinear association between planned fiscal adjustment and growth optimism may reflect an increasing cost to growth from higher adjustment, as policy tightening hits public sector activities with increasing value added to output. This makes high fiscal adjustment programs associated with disproportionately optimistic growth projections.

Our results suggest that despite previous studies revealing optimism in growth projections, the tendency for optimism has persisted in the period after the Global Financial Crisis (GFC). Moreover, the strong correlation between the magnitude of the optimism and expected fiscal consolidation provides a cautionary signal for the post-COVID economic projections for IMF member countries as these countries embark on a path of fiscal adjustment.

In the remainder of this paper we provide a brief review of the literature (Section II), explain how we measure optimism and construct the forecast database (Section III), present the econometric strategy (Section IV), discuss the results (Section V), and investigate how actual fiscal adjustment fits in the results (Section VI). Our final remarks and suggestions for future research are presented in Section VII. Robustness tests are available in the Annexes.

II. Review of the Literature

Optimism bias in IMF forecasts has long been explored in the economic literature, both by external researchers and IMF staff. Early papers focused on forecast accuracy in surveillance (i.e. non-program) economies (e.g., Arora and Smyth (1990), Barrionuevo (1992), Artis (1996)). The emphasis on program cases became more prominent with Mussa and Phillips (2002). These authors analyze a sample of 69 non-precautionary, non-concessional financial arrangements approved by the IMF Executive Board during 1993–1997, involving 47 emerging and transition economies, and find evidence of optimism bias in (unconditional) growth forecasts in programs that they identified as large, but no statistically significant bias in the rest of the sample. Despite the broad range of peer-reviewed studies covering this topic, none has yet to show evidence of systematic pessimism in Fund program forecasts.

Systematic reviews of forecast accuracy in program and surveillance cases have also been regularly conducted by the IMF’s Independent Evaluation Office since its inception in 2001. In one of its earliest publications, IEO (2003) extends Mussa and Phillips’ sample to 133 programs in 70 countries during 1993–2001 and unveils a link between fiscal revisions to program targets, fiscal underperformance, and optimism bias in the short-term recovery of private investment and growth. Among other findings, the IEO paper identifies revenue shortfalls (rather than excess expenditure) as the main culprit of fiscal underperformance in programs with large planned fiscal adjustment (i.e., those that targeted fiscal adjustment of more than 3 percentage points of GDP over a two-year horizon).

The consequences of ambitious policy adjustments were further explored by Baqir et al. (2005). Using a sample of 94 IMF-supported programs during 1989–2002, the authors find that, while growth forecasts were indeed optimistic, programs with more ambitious fiscal targets led to better growth performance. The opposite was found regarding monetary policy targets. The political economy of forecast accuracy has been explored by Bird (2005). The author discusses political motivations—and some of their downsides, including lack of program credibility—for over-optimism in the design of IMF programs. The literature has evolved substantially since then, reflecting the post-global financial crisis experience of IMF financial programs, as we will discuss later.

An interesting strand of the literature assesses the accuracy of IMF forecasts relative to market estimates. For instance, Timmerman (2007) evaluates the accuracy of IMF’s World Economic Outlook (WEO) forecasts for 178 countries during 1990–2003, contrasting them with consensus forecasts compiled from the private sector. The author finds that WEO forecasts significantly overpredicted one-year ahead growth outturn in about 20 percent of the advanced economies, 25 percent of the countries in the Western Hemisphere, and 35 percent of the countries in Africa. The performance of these WEO projections, however, was comparable to those from the consensus forecasts—a finding anticipated by Juhn and Loungani (2002).

The perils of optimism bias have also been explored. Frenkel (2011) examines growth forecasts made by 33 official government agencies and finds that optimism bias contributes to excessive deficits even in countries with fiscal rules as it discourages policymakers from enacting precautionary fiscal policies that could prevent episodes of debt distress. In a similar vein, Easterly (2013) shows the association between debt crises and medium- to long-run growth slowdowns is often unanticipated by forecasters. More recently, Beaudry and Willems (2018) find that past optimism in real GDP growth forecasts increase the likelihood of future recessions and fiscal crises in a sample of 189 countries. Interestingly, the authors construct a database on IMF mission chief assignments and use an econometric approach based on instrumental variables to find that the degree of optimism bias is partially explained by IMF mission chief assignments.

Measurement errors and time-dependence have been identified as important factors in the analysis of growth optimism. The contribution of measurement errors to forecast bias is discussed in Atoyan and Conway (2011). The authors inspect the accuracy of macroeconomic projections in a sample of 291 IMF-supported programs during 1993–2009 and find some evidence of optimism bias in the programs approved in the 1990s but not later.² Incomplete information on initial conditions and country-specific fixed effects are identified as the largest contributors to growth forecast errors. The impact of time-dependence on forecast accuracy is explored in Ho and Mauro (2014). Using data from the IMF and World Bank debt sustainability analysis, they show that baseline forecasts are more optimistic than warranted by past growth experience, and that optimism bias is greater the longer the forecast horizon—an econometric result corroborated in our paper. The authors find some evidence that this bias is more pronounced in growth forecasts made prior to a country entering an IMF-supported program. In a more recent study, Eicher et al (2018) find that real GDP growth forecasts for the year when the IMF program is approved are unbiased but inefficient, with the latter being driven by the group of low-income countries. Their study covers 156 programs in 84 countries during 2002–2015.

The link between fiscal multipliers and growth optimism has been widely covered in the academic literature and extensively discussed in policy arenas. Some of the most influential studies in this area have been produced by IMF staff. For instance, using a sample of 26 European advanced economies, Blanchard and Leigh (2013, 2014) find a negative relationship between growth forecast errors and planned fiscal consolidation during 2010– 2011. The authors attribute such finding to an underestimation of fiscal multipliers by IMF forecasters, possibly explained by the binding zero-lower bound on nominal interest rates, impaired balance sheets of financial institutions, and overall economic slack.

The post-global financial crisis performance of IMF forecasts is object of investigation of a series of papers prepared by the IMF’s Independent Evaluation Office. IEO (2014a) provides an analysis of current-year and one-year ahead forecasts of real GDP growth published by the WEO. For these shorter prediction horizons, the evidence of optimism bias is weak. For instance, when recession years are excluded from the sample, the evidence of optimism bias vanishes. This empirical regularity highlights how difficult it is for IMF forecasters to predict turning points and recessions. IEO (2014b) focuses on medium-term forecasts on real GDP growth (i.e. for a horizon 3 to 5 years ahead of the prediction date), considering WEO data published during 1990–2012. In that study, the IEO finds “no evidence of a systemic or built-in organizational bias caused by the approach used by the IMF to produce medium-term forecasts”. Nevertheless, the study confirms some degree of optimism in two-thirds to three-quarters of the countries, with the average error within the range of 0.44–0.76 percentage points and the median error between 0.14 and 0.41 percentage points, albeit the bias being statistically significant only in a small fraction of the cases. The study suggests that there is no statistically significantly difference in the optimism bias detected in program and non-program observations; however, after controlling for whether each country has ever been in an IMF-supported program during the sample period, the study finds that countries that never had an IMF program exhibit optimism bias of 0.15–0.30 percentage points while the bias in program countries varied from 0.27 to 0.88 percentage points, depending on the semi-annual WEO vintage (Spring versus Fall) and on the forecast horizon (3, 4, or 5 years ahead). Interestingly, within the group of countries with a program history, being in an IMF program at the time the forecast was produced attenuates the amount of optimism compared to program countries without an IMF program at the time the forecast was made.

Using a different data source—the IMF’s Monitoring Fund Arrangements (MONA) database—IEO (2014c) examines real GDP growth projections in 103 IMF-supported programs approved during 2002–2011. The study scrutinizes projections for two forecast horizons—the current year and one-year ahead—recorded at the time the program was requested and at the first review of the program. The authors find a significant degree of optimism in the projections of real GDP growth for programs with exceptional access—i.e., those receiving financial support above the normal annual and cumulative access limits established by the IMF lending policies.³ Meanwhile, programs within normal access limits exhibit only a small degree of optimism in growth forecasts. Importantly, the study finds evidence that forecasts prepared for the first review of the programs, which usually takes place three to six months after the program begins, tend to correct the optimism bias recorded at time of the program request.

As one can see, the literature on the accuracy of IMF forecasts is already quite rich albeit the findings are highly influenced by the sample coverage of each study. For this reason, an update on the econometric results is periodically warranted to account for the latest empirical developments. In this paper, we aim to deliver more than a simple econometric update. We focus on the non-linear impact of planned policy adjustments on growth forecast errors and on which lessons one can infer from observed optimism bias for the design of future IMF programs. For this purpose, we examine the universe of WEO and MONA forecasts of annual real GDP growth recorded during 2003–2017, which include some of the most ambitious policy adjustments in the IMF history. Our analysis cover surveillance and program countries, and for the latter it encompasses data on program requests and all program reviews. We consider time horizons from one up to five years ahead the projection date, and test for duration dependence on growth optimism. Our study also informs about possible structural breaks, e.g. differences in the sign and degree of forecast bias before and after the global financial crisis (GFC). Finally, we test whether IMF forecasters learn from forecasts errors over the lifetime of an IMF program.

III. Measuring Optimism Bias in Growth Projections

To measure the degree of optimism bias in IMF forecasts, we rely on two data sources. For non-program observations (henceforth named surveillance cases), we use the semi-annual vintages of the IMF’s World Economic Outlook (WEO) medium-term projections during 2003–2017.⁴ Using vintage-specific data helps us to control for unintended consequences of forecast revisions on the measurement of forecast errors. For program-specific observations, we use the IMF’s Monitoring of Fund Arrangements (MONA) database and relevant IMF program documents.⁵

Our dataset covers 34 advanced, 82 emerging, and 54 low-income economies. It includes information on each country’s actual and projected real GDP growth performance, planned policy adjustments, changes in the monetary base, exchange rate regime, public indebtedness, commodity prices shocks, trading partners’ projected and actual real GDP growth rate, regional affiliation, and dummy variables controlling for other country-specific and program-specific characteristics. We also gathered information on structural benchmarks in the program-specific observations. Details on individual variables are provided in Annex I.

We study the role of planned policy adjustments on growth optimism bias using a panel data regression framework. The dependent variable in our econometric study is the forecast error for country i at year t for the projection year k as recorded in they j – th data vintage, namely $F E_{i t j}^{k}$ , with the index k ranging from 0 (designating a projection made for the same year as the year when the forecast was recorded) up to 5 (indicating a projection made five years ahead of the year when the forecast was recorded).⁶

For intuitive reasons, in this paper we label the optimism bias in growth forecasts with a positive sign. As such, we define the dependent variable in the panel data regression models as follows:

\begin{matrix} F E_{i t j}^{k} = E_{i t j}^{k} (g_{i}^{k}) - g_{i}^{k} & (1) \end{matrix}

where $E_{i t j}^{k} (g_{i}^{k})$ is country i’s real GDP growth rate expected as of year t and recorded in the data vintage j for the period k-years ahead, and $g_{i}^{k}$ is country i’s actual real GDP growth rate for that respective period taken from the latest official data release (the most recent WEO publication). The distinction between forecast errors in program-specific observations and non-program observations (i.e. surveillance cases) is useful to test the hypothesis of optimism bias in IMF forecasts and, if confirmed, to understand in which cases it is more prominent and why. Within the universe of IMF programs, we focus on non-concessional, non-precautionary IMF-supported financial arrangements approved during the sample period. This consists of 42 IMF’s Stand-by Arrangements (SBAs) and 13 arrangements under the IMF’s Extended Fund Facility (EFF). These programs last from 6 months up to 4 years (in the case of extended arrangements under the EFF) and usually require ex-ante conditionality in the form of performance targets agreed with country authorities. Financial disbursements under these arrangements are provided at regular program reviews and are conditional on borrowing countries complying with quantitative program targets set by IMF teams under the baseline scenario at the time of program approval and at subsequent review completions.⁷

All other country forecasts over that period, including those with concessional IMF lending, are treated as non-program forecasts for two main reasons: i) weakness in the MONA data for PRGT programs over that period at the time of the analysis; ii) concessional Fund lending tend to be less focused on consolidation or policy tightening (average primary deficit in this sample increases over the program period) and more focused on capacity strengthening, structural reforms, and mobilizing donor support. Excluding the sample of low-income countries with concessional programs from the sample does not alter the thrust of the results.

Figures 1–3 summarize the main features of our dependent variable in the full sample. Average growth forecast errors and the 5^th-95^th percentile ranges for surveillance and program-specific observations are displayed in Figure 1. Unconditionally, program forecasts tend to be marginally less optimistic than surveillance projections, both in short-term (one-year ahead) projections and in medium-term projections (two to five-years ahead). Figure 2 shows that growth forecast errors were larger during 2008–2010 than in the rest of the sample, reflecting the uncertainty around the global financial crisis period. However, post-crisis forecasts remain more optimistic than in pre-crisis years.

In Figure 3, we focus on program observations and search for common patterns in growth forecast errors across program reviews. We find that forecasts were more pessimistic in program requests (R0), first (R1), second (R2), and remaining reviews before the global financial crisis. During 2008–2010, forecasts tended to show optimism bias at the time of the program request, with knee-jerk corrections in the first and, sometimes, in the second reviews.⁸ However, growth optimism returned in subsequent reviews, both during the global financial crisis years and the following period.

IV. Econometric Strategy

We estimate the direction and magnitude of growth forecast errors using panel data regressions. Fixed effects are allowed for each single country forecast observation, thus nullifying the need for country-or vintage-specific control variables. The main specification is as follows:

\begin{matrix} F E_{i t j}^{k} = Σ_{i = 1}^{N} Σ_{j = 1}^{J} I_{i j} + β_{p} P_{i t j} + β_{s} S_{i t j} + ε_{i t j} & (2) \end{matrix}

Following the notation put forward in equation (1), $F E_{i t j}^{k}$ is the forecast error on the real GDP growth rate of country i at year t and recorded in the data vintage j for the period k-years ahead. In the panel data regression model of equation (2), I_ij is a vector of observation-specific fixed effects; P_itJ is a matrix of planned policy adjustments (further discussed below), and S_itJ is a matrix with other control variables, including: forecast errors in oil and non-oil commodities prices, as well as their interactions with dummy variables for oil and commodity exporters, respectively; a proxy for optimism drift over the length of the projection horizon, based on the time of the forecast within the projection horizon; the interaction of the time of the forecast with the number of structural benchmarks when an IMF-supported financial arrangement was approved (aiming to control for optimism related to expected dividends from structural reforms); and a dummy variable for the global financial crisis (2008–2009).

We also estimate model (2) using panel regressions with random effects and Hausman-Taylor (1981) panel instrumental variables approach. The Hausman-Taylor (H-T) estimator allows for correlation between some of key regressors, including the planned policy variables, and the fixed effects to estimate horizon-invariant coefficients such as on the dummies controlling for program and exceptional access program forecasts. In this H-T specification, the variables assumed to be exogenous are the forecast errors for growth in trading partners, oil and commodity prices and their interaction dummies, the time of the forecast, the oil exporter indicator, and the global financial crisis dummy. All other variables are assumed to be endogenous.

A. Assessing the Impact of Planned Policy Adjustments

We face several challenges as we try to estimate the impact of planned policy adjustments on growth forecast errors. Non-concessional, non-precautionary IMF programs tend to be associated with contractionary policies aimed at closing the balance of payments needs that led to program requests in the first place. Such policy adjustments tend to take place through a combination of fiscal and monetary tightening together with structural reforms aimed at increasing efficiency of public policies and/or strengthening productivity in specific sectors of the economy. The channels through which these adjustments take place are heterogeneous and have differing implications to short- and medium-term economic growth.

A similar economic rationale applies to surveillance cases. For instance, fiscal multipliers can vary from one economic setting to another and from one method of adjustment (e.g. expenditure cuts versus tax increases) to another. Likewise, monetary policy tightening can have a stronger impact in more financially developed countries relative to credit-shallow economies. Structural adjustment is even more difficult to measure as conditionality on structural reforms vary significantly in terms of quality, depth, and pace of implementation. All things considered, we expect a high degree of heterogeneity in the impact of policy adjustments on growth.

With these caveats in mind, we focus on three complementary areas in an attempt to provide a consistent gauge of planned policy efforts across countries: fiscal, external, and structural. We rely on the projected changes in the primary fiscal balance of a given country as a consistent measure of its intended fiscal adjustment.⁹ Likewise, we gauge the aimed external adjustment by the projected change in the current account balance as a proxy for the overall underlying adjustment that addresses balance of payments needs.

It is important to highlight that planned policy adjustments may capture other sources of policy contraction beyond their respective domains, such as those related to monetary tightening. However, monetary tightening could have an impact in countries with access to international financial markets through the external financial account as well the external current account. Structural conditionality may operate through the same channel, for example in the case of privatization or measures to attract foreign direct investment. That said, in most cases it seems that it is the current account adjustment that gives a close proxy to the forecasted demand management resulting from the overall policy adjustment package, including on the fiscal side.

To allow for non-linearities in the impact of planned policy adjustments on growth forecast errors, the matrix of planned policy adjustments P_itJ is constructed as follows:

\begin{matrix} P_{i j t} = β_{F} F_{i j t} + β_{c} C_{i j t} & (3) \end{matrix}

F_{i j t} = \begin{matrix} [\begin{matrix} Δ p B \\ i_{i p} Δ p B \\ i_{i f h} Δ p B \end{matrix}] & (4) \end{matrix}

C_{i j t} = \begin{matrix} [\begin{matrix} Δ C A \\ i_{i p} Δ C A \\ i_{i c h} Δ C A \end{matrix}] & (5) \end{matrix}

where i_ip = 1 if country i is in an IMF-supported financial arrangement at the time the forecast is made and zero otherwise, and i_ifh = 1 if the planned contemporaneous annual primary fiscal balance adjustment ΔpB is higher than the country i’s average actual primary fiscal balance adjustment by half of one standard deviation of the distribution of actual primary fiscal balance adjustments for that specific country during the sample period. A similar decomposition is made for external adjustments using the planned contemporaneous current account adjustment ΔCA and the respective indicator i_ich. The choice of half a standard deviation is the result of a calibration to ensure a sufficiently large sample of high adjustment events given that the distributions of fiscal and current account adjustment are very concentrated around the means (Table 1). This approach allows, inter alia, for heterogeneity in multipliers based on the scale of a planned adjustment relative to country i’s own experience. β_F and β_c are estimated by plugging equation (3) in the econometric model (2).

^{Table 1.}

Incidence of Large Planned Fiscal and External Adjustments, 2003–17

^{Table 1.}

Incidence of Large Planned Fiscal and External Adjustments, 2003–17

	IMF-supported financial arrangements	Surveillance cases
Large planned fiscal adjustment (number of forecasts with at least one “large” adjustment year forecasted in the first three years)	258 out of 901 observations (165 out of 263 forecasts; 63 percent)	2,453 out of 19,062 observations (1,517 out of 3,989 forecasts; 38 percent)
Large planned external adjustment (number of forecasts with at least one “large” adjustment year forecasted in the first three years)	107 out of 901 observations (101 out of 263 forecasts; 38 percent)	1,611 out of 19,062 observations (994 out of 3,989 forecasts; 25 percent)

^{Table 1.}

Incidence of Large Planned Fiscal and External Adjustments, 2003–17

	IMF-supported financial arrangements	Surveillance cases
Large planned fiscal adjustment (number of forecasts with at least one “large” adjustment year forecasted in the first three years)	258 out of 901 observations (165 out of 263 forecasts; 63 percent)	2,453 out of 19,062 observations (1,517 out of 3,989 forecasts; 38 percent)
Large planned external adjustment (number of forecasts with at least one “large” adjustment year forecasted in the first three years)	107 out of 901 observations (101 out of 263 forecasts; 38 percent)	1,611 out of 19,062 observations (994 out of 3,989 forecasts; 25 percent)

The rationale for testing the existence of a non-linear impact of planned fiscal adjustments on output is twofold: (i) as lower-hanging fruits from reducing inefficient spending and investment are depleted, further tightening may interrupt economic activities with high returns to growth, resulting in actual losses to output from additional policy tightening; and (ii) empirical evidences show that fiscal policies may have non-linear effects in the presence of large and persistent fiscal impulses, with the no-linearity being stronger in cases of fiscal contractions (e.g., Giavazzi et al, 2000). This approach is also consistent with the hypothesis of a non-linear relationship between fiscal adjustment and debt burden in program cases, as evidenced in recent research (IMF, 2015). The incidence of large planned fiscal and external adjustments is summarized in Table 1.

Given that the right-hand side of model (2) includes only variables determined before the actual real GDP growth of a given country was observed (e.g. planned policy adjustment) and variables expected to be exogenous for most economies (e.g., forecast errors in trading partners’ growth and in international oil and commodity prices), this approach helps to attenuate potential endogeneity problems arising from simultaneity.

V. Optimism Bias in Surveillance and Program-Specific Forecasts

The econometric results (Figure 4 and Table 2) suggest that planned policy adjustments are associated with optimism in growth forecasts, varying with the size of the former. More specifically, large planned fiscal and external adjustments are associated with a greater degree of optimism across the sample, pointing to potential non-linearities in this relation. In the set of program-specific observations, episodes of large planned fiscal adjustments are associated with more optimism than those with smaller planned fiscal adjustments. Meanwhile, large planned external adjustments are associated with a higher degree of optimism in growth forecasts across the sample but less so in program-specific observations than in surveillance cases.

^{Table 2.}

Impact of Planned Policy Adjustments on Growth Forecast Errors: Surveillance and Non-Concessional, Non-Precautionary Program Forecasts, 2003–16

*** p<0.01, ** p<0.05, * p<0.1

^{Table 2.}

Impact of Planned Policy Adjustments on Growth Forecast Errors: Surveillance and Non-Concessional, Non-Precautionary Program Forecasts, 2003–16


Growth_error = (Forecast Growth‐ Actual Growth)
		(1)	Main Specification	Over Forecast vintage subsamples exc GFC
VARIABLES		Fixed effects	Hausman‐Taylor Est	FE Est over 2003‐8	FE Est over 2009‐16
External Factors
	partnergrowth_error	0.441***	0.441***	0.00962	0.219***
	oil_notexp_error	0.895***	0.913***	-2.079***	-0.872***
	oil_exp_error	1.836***	1.912***	-5.293***	2.358***
	commodity_notexp_error	1.480***	1.452***	3.590***	2.212***
	commodity_exp_error	0.300	0.313	3.876***	9.043***
Optimism drift over horizon
	time_to_forecast (days*365)	0.225***	0.224***	0.268***	0.407***
Forecasted Structural adjustment
	time_sb	0.00444	0.00448	-0.00399	0.00562
Forecasted Fiscal adjustmet
	pb_adjustment	-0.0157	-0.0157	-0.0471**	0.0195
	prog_pb_adjustment	0.157*	0.156**	0.106	-0.0180
	high_pb_adjustment	0.122***	0.122***	0.0693*	0.240***
Forecasted External adjustment
	ca_adjustment	0.0166**	0.0165**	0.0280***	0.00126
	prog_ca_adjustment	-0.165***	-0.163***	-0.426**	0.160***
	high_ca_adjustment	0.416***	0.416***	0.479***	0.227***
GFC (2008,9=1)		0.663***	0.662***
oilexporter			0.979
commexporter			1.011
program_country			-20.06
exceptional_d			52.02
Hausman‐test (prob>chi2)			0.97
Baltagi Wu/ Bhargava et al
Observations		19,963	19,963	6,861	9,863
R‐squared		0.247		0.295	0.127
Number of forecasts		4,252	4,252	1,686	2,562

*** p<0.01, ** p<0.05, * p<0.1

^{Table 2.}

Impact of Planned Policy Adjustments on Growth Forecast Errors: Surveillance and Non-Concessional, Non-Precautionary Program Forecasts, 2003–16


Growth_error = (Forecast Growth‐ Actual Growth)
		(1)	Main Specification	Over Forecast vintage subsamples exc GFC
VARIABLES		Fixed effects	Hausman‐Taylor Est	FE Est over 2003‐8	FE Est over 2009‐16
External Factors
	partnergrowth_error	0.441***	0.441***	0.00962	0.219***
	oil_notexp_error	0.895***	0.913***	-2.079***	-0.872***
	oil_exp_error	1.836***	1.912***	-5.293***	2.358***
	commodity_notexp_error	1.480***	1.452***	3.590***	2.212***
	commodity_exp_error	0.300	0.313	3.876***	9.043***
Optimism drift over horizon
	time_to_forecast (days*365)	0.225***	0.224***	0.268***	0.407***
Forecasted Structural adjustment
	time_sb	0.00444	0.00448	-0.00399	0.00562
Forecasted Fiscal adjustmet
	pb_adjustment	-0.0157	-0.0157	-0.0471**	0.0195
	prog_pb_adjustment	0.157*	0.156**	0.106	-0.0180
	high_pb_adjustment	0.122***	0.122***	0.0693*	0.240***
Forecasted External adjustment
	ca_adjustment	0.0166**	0.0165**	0.0280***	0.00126
	prog_ca_adjustment	-0.165***	-0.163***	-0.426**	0.160***
	high_ca_adjustment	0.416***	0.416***	0.479***	0.227***
GFC (2008,9=1)		0.663***	0.662***
oilexporter			0.979
commexporter			1.011
program_country			-20.06
exceptional_d			52.02
Hausman‐test (prob>chi2)			0.97
Baltagi Wu/ Bhargava et al
Observations		19,963	19,963	6,861	9,863
R‐squared		0.247		0.295	0.127
Number of forecasts		4,252	4,252	1,686	2,562

*** p<0.01, ** p<0.05, * p<0.1

We also find statistically significant evidence of optimism drift over the projection horizon. After controlling for planned policies, structural and exogenous variables, we find that IMF forecasts are associated with an optimism bias of around 0.2 percentage points of a country’s real GDP growth rate each year into the projection horizon. Over a five-year horizon, the cumulative impact of IMF’s optimism bias adds up to 1 percentage point of a country’s real GDP growth rate. As we segment the sample into pre and post GFC subsamples, we find that associations of growth optimism with planned policy adjustment are consistent with the direction observed in the aggregate sample. At the same time, the results indicate that optimism drift in growth forecasts over the horizon increased post-GFC, possibly reflecting lower potential growth after the crisis (Chen, Mrkaic, and Nabar (2019), Kose and Ohnsorge et al. (2019)).

Truncating the sample down to the first three years of the forecast horizon (a typical length of IMF-supported financial arrangements), we find that planned fiscal and external adjustments are associated with growth optimism in program-specific and in surveillance forecasts. However, planned fiscal adjustments contribute more (at the margin) to the bias in program-specific observations than to the surveillance cases. This suggests that much of the optimism associated with IMF programs and highlighted in previous studies may originate in the fiscal sector.

The external sector is also important. Forecast errors in trading partners’ growth as well as in oil price projections have a strong impact on growth forecast errors, with optimism in trade partners’ performance being associated with growth optimism in any given forecast.

Sanguine oil price projections tend to be associated with growth optimism in the forecasts of both oil importers and exporters, and more so in the latter. Once price optimism is controlled for, being an oil importer or an exporter is not significantly associated with growth optimism bias.

Recent research documents increasing delays in the implementation of structural conditionality in IMF-supported financial arrangements, which may potentially contribute to growth forecast errors (IMF, 2019).¹⁰ However, in program-specific observations, our proxy for planned structural adjustments (number of structural benchmarks adopted at program approval) does not seem to be significantly associated with growth forecast errors. A disclaimer, as explained earlier, is that the number of structural benchmarks alone may not be a good proxy for the strength, pace, and depth of structural reforms in IMF-supported financial arrangements.

We supplement the econometric analysis by running OLS regressions of the residuals from model (2) on several cyclical and structural variables. This includes initial public debt, output gap, regional affiliation, and exchange rate regime. In addition, we control for whether a country has had an IMF-supported financial arrangement up to the time the forecast was made. The results (Tables 3 and 4) suggest that the residuals from model (2) are mostly idiosyncratic: none of these controls are significantly associated with growth forecast errors in the set of program-specific observations. We also find no evidence that fixed effects are correlated with having a program in the past. Based on these statistical results, we find little support that learning from past programs has helped bringing down optimism bias in IMF growth forecasts.

^{Table 3.}

Assessing the Role of Other Cyclical and Structural Factors in Growth Forecast Bias: Non-Concessional, Non-Precautionary Program Forecasts, 2003–16