Leading Indicators of Fiscal Distress
Evidence from the Extreme Bound Analysis1
Author:
Martin Bruns https://isni.org/isni/0000000404811396 International Monetary Fund

Search for other papers by Martin Bruns in
Current site
Google Scholar
Close
and
Mr. Tigran Poghosyan
Search for other papers by Mr. Tigran Poghosyan in
Current site
Google Scholar
Close

Contributor Notes

Author’s E-Mail Address: MBruns@diw.de, TPoghosyan@imf.org.

Early warning systems (EWS) are widely used for assessing countries’ vulnerability to fiscal distress. Most EWS employ a specific set of only fiscal leading indicators predetermined by the researchers, which casts doubt on their robustness. We revisit this issue by using the Extreme Bound Analysis, which allows identifying robust leading indicators of fiscal distress from a large set. Consistent with the theoretical predictions of latest generation crisis models, we find that both fiscal (e.g., fiscal balance, foreign exchange debt) and non-fiscal leading indicators (e.g., output, FX reserves, current account balance, and openness) are robust. In addition, we find that a fiscal vulnerability indicator based on fiscal and non-fiscal leading indicators offers a 29% gain in predictive power compared to a traditional one based on fiscal leading indicators only. It also has good predictive power out of sample, with 78 percent of crises predicted correctly and only 34 percent false alarms issued for the period 2008–15. This suggests that both fiscal and non-fiscal leading indicators should be taken into account when assessing country’s vulnerability to fiscal distress.

Abstract

Early warning systems (EWS) are widely used for assessing countries’ vulnerability to fiscal distress. Most EWS employ a specific set of only fiscal leading indicators predetermined by the researchers, which casts doubt on their robustness. We revisit this issue by using the Extreme Bound Analysis, which allows identifying robust leading indicators of fiscal distress from a large set. Consistent with the theoretical predictions of latest generation crisis models, we find that both fiscal (e.g., fiscal balance, foreign exchange debt) and non-fiscal leading indicators (e.g., output, FX reserves, current account balance, and openness) are robust. In addition, we find that a fiscal vulnerability indicator based on fiscal and non-fiscal leading indicators offers a 29% gain in predictive power compared to a traditional one based on fiscal leading indicators only. It also has good predictive power out of sample, with 78 percent of crises predicted correctly and only 34 percent false alarms issued for the period 2008–15. This suggests that both fiscal and non-fiscal leading indicators should be taken into account when assessing country’s vulnerability to fiscal distress.

I. Introduction

The global financial crisis and the subsequent weakening of fiscal positions in advanced and emerging economies once again underscored the importance of monitoring country’s vulnerability to fiscal distress. What variables should policymakers watch to assess fiscal risks? There is a large empirical literature by now attempting to answer this question using Early Warning Systems (EWS). International financial organizations, central banks, rating agencies and other organizations draw on these studies to develop indicators of vulnerability to fiscal distress.

Most of the existing EWS have the following characteristics. First, they use a predetermined set of leading indicators to assess country’s vulnerability to fiscal distress, which are typically based on economic reasoning. Second, the list of these variables varies widely across studies, in part driven by the preference of the researchers for parsimonious models with a large number of significant leading indicators. Finally, the results from the studies are mixed, with no agreement as to which leading indicators are most robustly associated with fiscal distress.

The purpose of this paper is to revisit the issue of leading indicators of fiscal distress by using the Extreme Bound Analysis (EBA). The main advantage of this methodology is that it takes an agnostic approach with respect to leading indicators of fiscal distress and does not require the researcher to predetermine the set of explanatory variables. Instead, it “allows the data to speak” and ranks a set of possible leading indicators based on the “robustness” of their association with fiscal distress. This methodology was used in other fields of economics, notably, growth theory (see for example Sala-i-Martín, 1997) and only recently was applied to analyze financial crises, for example by Ho (2010). However, to our best knowledge we are the first to apply it for analyzing fiscal distress.

Our analysis leads to the following conclusions. First, both fiscal and non-fiscal leading indicators are robustly associated with fiscal distress. This is in contrast to traditional measures of fiscal stress based on fiscal leading indicators (Baldacci et al., 2011). Second, a vulnerability index based on these robust indicators has a performance comparable to the average of performance of other EWS (for fiscal, currency, banking, and other types of crises) in the literature. However, it has better predictive power than the EWS of Baldacci et al. (2011), which is based on only fiscal leading indicators. Finally, the main result on the importance of both fiscal and non-fiscal leading indicators for fiscal distress is robust to various model specifications and sample compositions. The policy implication is that policymakers should not restrict their attention to fiscal indicators only when assessing country’s vulnerability to fiscal distress.

The remainder of the paper is structured as follows. Section II surveys the related literature. Section III describes the empirical methodology and data. Section IV presents the estimation results using EBA and conducts robustness checks. Section V develops an indicator of fiscal distress based on most robust leading indicators identified using EBA and assesses its predictive power. The final section concludes.

II. Related Literature

A. Theories of Crisis Determinants

When modeling crises, it is important to draw distinction between underlying vulnerabilities and crisis risks. The presence of underlying vulnerabilities is a necessary precondition for a crisis to occur, but not sufficient. Crises tend to be triggered by external or domestic shocks, which are highly uncertain. The underlying vulnerability helps to identify countries that are prone to crisis should even a moderate shock occur.

The early theoretical literature emphasized the role of fundamentals in measuring underlying vulnerability. The classic reference is Krugman (1979), which predicts that weak fundamentals, in part driven by unsustainable fiscal and monetary policies, make countries vulnerable to a balance of payments crisis. For fiscal crises, Detragiache and Spilimbergo (2001) show that sovereign default is the only equilibrium response to a large negative shock to fundamentals.

The following generation of theoretical literature of crises emphasizes the role of self-fulfilling expectations and non-fiscal fundamentals. The self-fulfilling crisis literature (Obstfeld, 1984; Calvo, 1988; Alesina et al., 1989; Cole and Kehoe, 1994; Jeanne, 1997; and Masson, 1999) was inspired by the fact that while some crises were preceded by deterioration in fundamentals, some speculative attacks have taken place without apparent monetary and fiscal imbalances. These studies developed multiple-equilibrium models, which allow better capturing the complex interaction between underlying vulnerabilities and speculative attacks (Eichengreen et al., 1995 provide a review). In these models, countries can jump from good to bad equilibrium for a certain range of values of economic fundamentals. For example, Calvo (1988) shows that if a government is unable to commit to repay its debt, multiple equilibria, including repudiation and inflation, can arise.

The main takeaway from the review of theoretical literature is that there is no a priori reason to restrict leading indicators of fiscal distress to fiscal fundamentals.

B. Empirical Evidence on Early Warning Indicators of Fiscal Distress

The empirical literature on early warning indicators of fiscal distress was developed as part of a larger literature on early warning indicators of financial crises.2 As discussed above, the main objective of these studies is to identify leading indicators making countries vulnerable to a crisis, rather than prediction of the timing of the crisis. The empirical techniques used in this literature could be grouped into two main categories: non-parametric and parametric (Hemming et al., 2003; Baldacci et al., 2011; Frankel and Saravelos, 2012; and Comelli, 2013).3

The most popular non-parametric EWS is the “signals” approach popularized by Kaminsky et al. (1998) in the context of currency crises. This approach selects a number of variables as leading indicators of crises and determines threshold values beyond which a crisis signal is considered to have been given. The main drawback of this approach is that it only focuses on bivariate association between an early warning indicator and crises and does not allow controlling for other factors. Also, the statistical significance of the early warning indicators cannot be determined directly, although out-of-sample performance could be assessed. Studies using these techniques in the context of fiscal crises include Reinhart (2002), Hemming et al. (2003), Baldacci et al (2011), Berti et al. (2012) and de Cos et al. (2014). These studies suggest that fiscal fundamentals, such as the level of public debt and its composition, fiscal deficit, fiscal financing needs, are important leading indicators of fiscal distress.

The parametric EWS models draw on limited dependent variable techniques (multivariate logit, probit). These methods allow testing the significance of various leading indicators in determining the likelihood of crisis occurring in the near future, while accounting for their correlation. However, these methods require long-time series of leading indicators and low degrees of freedom that may prevent the use of multivariate approach when the number of predictors is large. Studies that applied parametric methods to analyze fiscal crises include Marashaden (1997), Peter (2002), Manasse et al. (2003), Kraay and Nehru (2006), and Gourinchas and Obstfeld (2012). These studies confirm the importance of fiscal fundamentals, but suggest that macroeconomic developments in general, especially in the external sector, also play a role.

Both groups of studies share a common characteristic – the set of leading indicators of fiscal distress is predetermined by the researcher. This selection is typically done with the benefit of hindsight, with the significance of the leading indicator typically playing a role in the selection process.4 To overcome this issue, some recent studies applied the extreme bound methodology to study leading indicators of financial crises.

C. Extreme Bound Methodology

The extreme bound methodology was developed and applied to study determinants of growth by Leamer (1978), and was later on extended and popularized by Sala-i-Martin (1997). The methodology does not rely on a predetermined set of explanatory variables and “lets the data speak” by examining all possible combinations of explanatory variables. The focus is on the change in signs of explanatory variables. If a sign change is observed relatively frequently, the explanatory variable is said not robustly related to the dependent variable.

Early warning studies using the extreme bound methodology (or its Bayesian equivalent) include Chakrabarti and Zeaiter (2014), Ho (2014), and Christofides et al. (2015).5 However, to our best knowledge no study used this approach to analyze determinants of fiscal distress, which is the gap we would like to fill in our paper. The closest paper to ours is Chakrabarti and Zeaiter (2014), which analyzes determinants of external debt arrears using a linear regression model. The results suggest that a range of fiscal and non-fiscal indicators, including growth, inflation, trade deficit, foreign reserves, and exchange rate, are robust predictors of external debt arrears.

III. Empirical Methodology and Data

A. Empirical Methodology: Extreme Bound Analysis

As discussed above, Extreme Bound Analysis (EBA) is an agnostic approach to identify explanatory variables that are robustly associated with an outcome variable. Instead of preselecting a small set of explanatory variables, EBA explores a large number of combinations from a pool of variables used in the previous literature and based on theoretical considerations.

In the context of an Early Warning System (EWS) for fiscal distress events, the following logit specification is estimated:

log ( P [ Y = 1 | z , x ] P [ Y = 0 | z , x ] ) = β z j z + β x j x j + ɛ

Where Y is a binary outcome variable indicating a fiscal distress event, z is the leading indicator whose robustness is to be assessed and xj are other leading indicators from the pool of all leading indicators excluding z. Assume there are n elements in this pool. Then, for each leading indicator z, one has to estimate a regression with k additional controls xj, which gives M=(n1k)!k! combinations per leading indicator z and a total of nM regressions to be estimated.

For each leading indicator z, we follow the following procedure:

  • Estimate each of the M regressions and store the estimated coefficient of z, β^zj.

  • Weigh coefficients β^zj by the relative likelihood of the model, which they are a part of, so that coefficients stemming from a very unlikely model receive less weight than others. The weight is:

    ω z j = L z j i = 1 M L z i
  • Calculate the weighted average of coefficients across regressions to obtain the coefficient of leading indicator z:

    β z ^ = j = 1 M ω z j β z j
  • Calculate a likelihood-weighted average standard error in a similar fashion.

    Assess the robustness of βz^. Assuming that βz^ is normally distributed across regressions, calculate the proportion of its distribution to one side of zero, and consider βz^ to be robust if this proportion exceeds a certain confidence level (e.g. 90 percent).

Following these steps for each indicator results in a set of indicators which can be ranked according to their respective robustness.

B. Data

Our dataset covers 29 advanced and 52 emerging economies (81 countries in total) over the period 1970–2015. It builds on the original dataset of fiscal distress developed by Baldacci et al. (2011), extending the data through 2015 and expanding the set of leading indicators.

Fiscal distress

A fiscal distress episode is identified as a period when government experiences extreme funding difficulties. Based on the literature survey, Baldacci et al. (2011) identify four types of criteria to capture fiscal distress episodes: (i) debt default or restructuring; (ii) sovereign bond yield pressure; (iii) large IMF-supported program; and (iv) excessive inflation. We adopt the same definition of fiscal distress (see Table 1) and expand the series through 2015.

Table 1.

Definition of Fiscal Distress

article image
Source: Baldacci et al. (2011).

Table 2 presents the distribution of fiscal distress episodes across their types and country groups. Several observations emerge. First, comparison across country groups suggests that advanced economies have experienced fewer fiscal distress episodes compared to emerging economies. The low unconditional likelihood of fiscal distress for advanced economies will have implications for the conditional analysis conducted below. Second, the comparison across types of distress events suggests that advanced economies experienced almost no outright default episodes and most fiscal distress episodes were manifested in the form of bond yield pressures. By contrast, emerging economies experienced a large number of fiscal distress episodes and were frequent recipients of large IMF-supported bailouts. Finally, the total number of distress episodes is lower than the sum of individual episodes. This is due to the fact that some countries have experienced several types of fiscal distress events simultaneously.

Table 2.

Fiscal Distress Episodes

article image
Source: Baldacci et al. (2011) and authors’ calculations for 2011–2015. Note: The numbers in brackets represent fiscal distress episodes in the original Baldacci et al. (2011) dataset running through 2010.

Figure 1 shows the share of fiscal distress episodes that coincided with currency and banking crises using Reinhart and Rogoff’s dataset.6 We find that currency crises overlap with 26 percent of fiscal distress episodes, while banking crises overlap with 24 percent of fiscal distress episodes for the sample spanning through 2010 (the last year of Reinhart and Rogoff’s dataset). The high share of overlaps suggest that some of the fiscal distress events may have originated outside of the fiscal sector, which is consistent with the “twin crises” narrative proposed by Kaminsky and Reinhart (1999).

Figure 1.
Figure 1.

Banking, Currency and Fiscal Crises

Citation: IMF Working Papers 2016, 028; 10.5089/9781475594799.001.A001

Note: Reported is the overlap between fiscal, banking and currency crises for the period 1970–2010. Data on banking and currency crisis episodes are taken from Reinhart and Rogoff (2009). Data on the first year of a fiscal crisis are from Baldacci et al. (2011).

Leading indicators

We identify 37 variables that were used as leading indicators for fiscal distress in the literature. Table 3 lists these variables as well as their sources. The indicators cover not only the fiscal sector, but also macro, monetary, and external sectors. Some of the indicators reflect the impact of contagion effects and global factors, which is consistent with predictions of the theoretical literature. All variables are measured in percentage points.

Table 3.

List of Leading Indicators of Fiscal Distress

article image
Source: Survey of the literature by the authors.

Table 4 presents descriptive statistics, while Table 5 presents bivariate correlations. As shown in the latter table, most of the variables are not highly correlated with the exception of some debt-related indicators. This should limit the issue of multicollinearity.

Table 4.

Descriptive Statistics

article image
Source: IMF WEO, World Development Indicators, International Finance Statistics, Government Finance Statistics.
Table 5.

Correlation Matrix

article image
article image

IV. Estimation Results

In this section, we present estimation results for the baseline specification covering the whole sample and based on the pooled logit specification. We also check whether results are sensitive to changes in the sample and estimation methods. Leading indicators are assessed to be robust if the probability of not switching the sign exceeds 90 percent, as approximated by the normal distribution. We use a forecasting horizon of 1 year throughout the estimation.7

A. Baseline Extreme Bound Analysis Model

Table 6 presents baseline EBA estimation results. Estimations are performed using the pooled logit model. We use various combinations of 2 additional controls per regression, which leads to 630 regressions per leading indicator (23,410 regressions in total). The main reason for using two additional controls as a baseline is computational efficiency and a higher number of observations per regression. The variables are ranked according to their robustness, with most robust indicators placed on top.

Table 6.

EBA Results: Baseline Specification

article image
Note: Reported are estimation results from the EBA regression. The dependent variable is Baldacci et al. (2011) definition of fiscal distress: (i) debt default or restructuring; (ii) sovereign bond yield pressure; (iii) large IMF-supported program; and (iv) excessive inflation. Estimations are performed using the pooled logit model, with 2 additional controls per specification. The variables are ranked according to their robustness, with most robust indicators placed on top. The sample covers 29 advanced and 52 emerging economies for the period 1970–2015.

The estimation results suggest that both fiscal and non-fiscal variables are robust leading indicators of fiscal distress. We find that fiscal distress tends to follow a period of overheating in the real sector (widening of output gap). This is consistent with Alberola et al. (2013) finding that growth, if it is not driven by sound economic fundamentals, can be detrimental to systemic stability. A related explanation could be that a large positive output gap, even if it is non-inflationary, can be associated with a buildup of financial imbalances, which in turn can indirectly lead to fiscal distress (see Borio et al. 2013).

In addition, adverse developments in the external sector (high current account deficit, low level of FX reserves/GDP ratio, slowdown in FX reserves growth, and higher openness) tend to precede fiscal distress episodes. These results are consistent with Chakrabarti and Zeaiter (2014)’s analysis for external debt arrears and also Gourinchas and Obstfeld’s (2011) analysis of the causes of sovereign default. A negative effect of openness is also found in Manasse et al. (2003).

Among fiscal leading indicators, we found that foreign exchange debt to GDP ratio, primary fiscal balance gap, as well as primary and overall fiscal balance to GDP ratios, are robust indicators, which is in line with most fiscal EWS such as Baldacci et al. (2011).

One potential reason for the robustness of non-fiscal indicators could be a high correlation with fiscal indicators. Table 5 shows that this is not the case: for example, the correlation between FX reserves (% of GDP) and various robust fiscal indicators, such as primary balance, overall balance or primary balance gap does not exceed 0.35 in absolute terms. The same is true for the output gap. This indicates that non-fiscal indicators do not only pick up the effects of fiscal indicators but matter on their own.

An illustration for why external factors perform better than fiscal factors as leading indicators can be observed in Figure 2. It shows that debt to GDP ratio before the crisis is not significantly different from the levels observed in tranquil times, but increases rapidly after the crisis incident. This indicates that debt to GDP ratio – one of the key fiscal indicators used in previous studies (see, e.g., Kraay and Nehru, 2006) – is more of an “ex post” indicator of fiscal distress rather than a leading indicator. The current account balance, on the other hand, is significantly lower relative to tranquil times before the crisis, suggesting that it can be used as a leading indicator. This result supports the hypothesis that fiscal vulnerabilities can be built up outside the public sector.

Figure 2.
Figure 2.

Event Study Analysis

Citation: IMF Working Papers 2016, 028; 10.5089/9781475594799.001.A001

Note: Both graphs as generated using the Gourinchas and Obstfeld (2012) event study methodology. It shows the percent deviation of the debt to GDP ratio and the current account balance to GDP ratio from their respective “tranquil” averages. This is defined as the average level of the respective variable outside the 3-year window around the crisis incident. Dotted lines denote the 10% confidence intervals.

Overall, the baseline results suggest that limiting leading indicators to fiscal variables, like in Baldacci et al. (2011), may result in a loss of important information regarding vulnerability to fiscal distress.8 Consistent with predictions of most recent theories of crises, information from other sectors should also be monitored.

B. Robustness Checks

We run several additional EBA regressions to check the results for sensitivity to changes in the sample and estimation methods.

“Narrow” definition of fiscal distress

The robustness of non-fiscal leading indicators, especially those pertaining to the external sector, may be driven by the fact that some of the large scale IMF programs (third definition of fiscal distress) are triggered by balance of payment or other crises not purely linked to fiscal distress. Also, high inflationary pressures (fourth definition of fiscal distress) may be caused by large devaluations following currency crises. To check whether robustness of non-fiscal variables still holds for “narrow” fiscal distress episodes, we re-estimate the model using a more restrictive sample of fiscal distress episodes that did not overlap with banking and currency crises.

Table 7 presents estimation results from a specification that restricts the dependent variable to “narrow” fiscal distress episodes. The set of robust determinants still includes non-fiscal leading indicators, confirming the baseline results. Gross financing needs and interest costs also turn robust in this specification, suggesting that borrowing cost pressures are important leading indicators for “narrow” fiscal distress episodes. The main drawback of this specification is that the sample has to be restricted through 2010, which is the last year for which banking and currency crisis variables are available in Reinhart and Rogoff’s database.

Table 7.

Robustness Check 1: “Narrow” Fiscal Distress Episodes

article image
Note: Reported are estimation results from the EBA regression. The dependent variable is the narrower fiscal stress definition, which excludes Currency and Banking crises as identifies in Reinhart and Rogoff (2009). Estimations are performed using the pooled logit model, with 2 additional controls per specification. The variables are ranked according to their robustness, with most robust indicators placed on top. The sample covers 29 advanced and 52 emerging economies for the period 1970–2010.

Emerging economies

As indicated in the previous section, most fiscal crisis events took place in emerging economies, which could explain the relatively high proportion of external leading indicators that are mostly relevant for emerging economies. For example, difficulties to finance a persistent current account deficit and the associated decline in FX reserves have historically been associated with fiscal distress in emerging markets. Advanced economies on the other hand, face fewer financing problems of this sort as they are often able to borrow in their own currency and generally rely less on external financing (IMF, 2010). To assess whether our baseline results are mostly driven by emerging economies, we redo the analysis by restricting the sample to emerging economies.9

Table 8 presents estimation results from the sample of emerging economies. The set of robust indicators is very similar to the one for the total sample, confirming the influence of emerging economies in driving the main results. Out of 10 robust leading indicators found in the total sample, 8 remain significant in emerging economies. One additional variable that becomes significant for emerging economies is the unemployment rate. The latter is consistent with the robust result on the output gap, given that unemployment is its mirror image.

Table 8.

Robustness Check 2: Emerging Economies

article image
Note: Reported are estimation results from the EBA regression. The dependent variable is Baldacci et al. (2011) definition of fiscal distress: (i) debt default or restructuring; (ii) sovereign bond yield pressure; (iii) large IMF-supported program; and (iv) excessive inflation. Estimations are performed using the pooled logit model, with two additional controls per specification. The variables are ranked according to their robustness, with most robust indicators placed on top. The sample covers 52 emerging economies for the period 1970–2015.

Number of control variables

We also check the robustness of results to the larger number of control variables. Table 9 presents estimation results from a specification that increases the number of control variables from two to three. This results in substantially higher time for estimating the model, as the number of regressions per leading indicator increases to 7140 (264,180 regressions in total). Nevertheless, the main results remain unchanged, with 8 out of 10 robust indicators from baseline regression remaining robust. As before, the set of robust leading indicators includes both fiscal and non-fiscal variables.

Table 9.

Robustness Check 3: Number of Control Variables

article image
Note: Reported are estimation results from the EBA regression. The dependent variable is Baldacci et al. (2011) definition of fiscal distress: (i) debt default or restructuring; (ii) sovereign bond yield pressure; (iii) large IMF-supported program; and (iv) excessive inflation. Estimations are performed using the pooled logit model, with 3 additional controls per specification. The variables are ranked according to their robustness, with most robust indicators placed on top. The sample covers 29 advanced and 52 emerging economies for the period 1970–2015.

Random effects logit and pooled probit models

We also check robustness of main results to types of discrete choice models.

First, we control for unobserved country-specific heterogeneity by using a random effects logit model.10 Table 10 presents estimation results, showing that the set of robust indicators remains qualitatively unchanged, with 8 out of 10 robust indicators from baseline regression remaining robust.

Table 10.

Robustness Check 4: Random Effects Logit Model

article image
Note: Reported are estimation results from the EBA regression. The dependent variable is Baldacci et al. (2011) definition of fiscal distress: (i) debt default or restructuring, (ii) sovereign bond yield pressure, (iii) large IMF-supported program, and (iv) excessive inflation. Estimations are performed using the random effects logit model, with 2 additional controls per specification. The variables are ranked according to their robustness, with most robust indicators placed on top. The sample covers 29 advanced and 52 emerging economies for the period 1970–2015.

Second, given that logit model has fatter tails than probit model, we also check sensitivity to tail risks by using a pooled probit model. Table 11 presents estimation results, showing that the set of robust indicators remains practically unchanged and suggesting that fat tails are not affecting the results.

Table 11.

Robustness Check 5: Pooled Probit Model

article image
Note: Reported are estimation results from the EBA regression. The dependent variable is Baldacci et al. (2011) definition of fiscal distress: (i) debt default or restructuring; (ii) sovereign bond yield pressure; (iii) large IMF-supported program; and (iv) excessive inflation. Estimations are performed using the pooled probit model, with 2 additional controls per specification. The variables are ranked according to their robustness, with most robust indicators placed on top. The sample covers 29 advanced and 52 emerging economies for the period 1970–2015.

Using alternative filtering technique for output gap

Throughout the analysis, we find a persistent evidence of robustness of the output gap variable, which comes on top of the list in all tables. This could be due to the fact that that the two-sided HP filter used to estimate the output gap could potentially bias the results prior to fiscal crises as one uses future values of output, already affected by the crisis, to smooth past series. To check the sensitivity of results to the smoothing technique, we use one-sided filter of Christiano and Fitzgerald (2003). Table 11 presents estimation results using this filter. The results remain unchanged suggesting that measurement of the output gap does not constitute a problem and reinforcing the importance of output cycles as leading indicators of fiscal crises.

V. Fiscal Distress Index and Its Performance

In this section, we present an alternative fiscal distress index based on the most robust leading indicators identified using the EBA methodology (Table 13).11 We then measure its performance for in-sample and out-of-sample and make a comparison with other EWS in the literature.

Table 12.

Robustness Check 6: Using New Measure of Output Gap

article image
Note: Reported are estimation results from the EBA regression. The dependent variable is Baldacci et al. (2011) definition of fiscal distress: (i) debt default or restructuring; (ii) sovereign bond yield pressure; (iii) large IMF-supported program; and (iv) excessive inflation. Estimations are performed using the pooled Logit model, with 2 additional controls per specification. The output gap is generated using a one-sided Christiano-Fitzgerald filter. The variables are ranked according to their robustness, with most robust indicators placed on top. The sample covers 29 advanced and 52 emerging economies for the period 1970–2015.
Table 13.

Early Warning System Based on Most Robust Leading Indicators

article image
Note: Reported are estimation results from the pooled logit model using most robust leading indicators of fiscal distress. The dependent variable is Baldacci et al. (2011) definition of fiscal distress: (i) debt default or restructuring; (ii) sovereign bond yield pressure; (iii) large IMF-supported program; and (iv) excessive inflation. The sample covers 29 advanced and 52 emerging economies for the period 1970–2015. AUROC is the area under the receiver operating curve. ***, **, and * denote significance at 1, 5, and 10 percent levels, respectively.

A natural question which arises is which indicators have the largest contribution to the fiscal distress index. In order to assess this, Figure 8 shows the impact of changes in robust leading indicators between the 75th and the 25th percentile on the logarithm of the odds ratio of the fiscal crisis. The figure shows that non-fiscal indicators have a larger relative impact on the fiscal distress index than fiscal indicators reflecting their importance in assessing underlying vulnerabilities.

Figure 3.
Figure 3.

Fiscal Distress Index Based on Robust Leading Indicators

Citation: IMF Working Papers 2016, 028; 10.5089/9781475594799.001.A001

Note: The fiscal distress index is generated by using predicted values from the logit model that includes most robust leading indicators. The most robust leading indicators are defined as those with at least 90 percent probability of not switching signs using the EBA results. Robust leading indicators include (i) Output Gap; (ii) Current Account Balance (% of GDP); (iii) FX Reserves Growth; (iv) Openness: (M+X)/GDP; (v) Real GDP Growth, (vi) Overall Fiscal Balance (% of GDP); (vii) Primary Balance (% of GDP). Blue line indicates the Fiscal Distress Index (left axis). Dotted lines indicate 90 percent confidence bands around predicted values. Orange line represents the absolute number of distress events in a given year (right axis). Data are missing for the pre-1980 period so that no predictive values can be generated.
Figure 4.
Figure 4.

In-sample Performance: EBA-based Fiscal Distress Index versus Baldacci et al. (2011) Index

Citation: IMF Working Papers 2016, 028; 10.5089/9781475594799.001.A001

Note: Reported is the Total Misclassification Error (TME) for different threshold levels. The blue line represents the baseline model based on robust leading indicators: (i) output gap; (ii) current account balance (% of GDP); (iii) FX reserves growth; (iv) openness: (M+X)/GDP; (v) real GDP growth; (vi) overall fiscal balance (% of GDP); (vii) primary balance (% of GDP). The vertical line marks the threshold level (0.07) that minimizes the TME. The orange line represents TME from the model based on variables used by Baldacci et al. (2011).
Figure 5.
Figure 5.

In-sample Versus Out-of-sample Performance of the EBA-based Fiscal Distress Index

Citation: IMF Working Papers 2016, 028; 10.5089/9781475594799.001.A001

Note: Reported is The Total Misclassification Error (TME) for different levels of the threshold. The fiscal vulnerability index is based on robust leading indicators: (i) output gap; (ii) current account balance (% of GDP), (iii) FX reserves growth; (iv) openness: (M+X)/GDP; (v) real GDP growth; (vi) overall fiscal balance (% of GDP); (vii) primary balance (% of GDP). The blue line represents the in-sample TME based on the years 1970–2007. The orange line represents the out-of-sample TME for the years 2008–2015. The vertical line marks the threshold level (0.04) that minimizes the in-sample TME.
Figure 6.
Figure 6.

Classification Table for Out-of-sample Predictions of the EBA-based Fiscal Distress Indicator

Citation: IMF Working Papers 2016, 028; 10.5089/9781475594799.001.A001

Note: Reported is the model classification for the years 2008–2015. A signal is issued if fiscal distress index exceeds the threshold derived from the in-sample minimization of the TME.
Figure 7.
Figure 7.

Comparison with Other Studies

Citation: IMF Working Papers 2016, 028; 10.5089/9781475594799.001.A001

Note: Reported is the in-sample total misclassification error (TME), Type1 and Type2 errors of 8 Early Warning Systems for currency, banking and sovereign crises reported in the literature. The values represent the minimum, the 25th percentile, the mean, the 75th percentile and the maximum.
Figure 8.
Figure 8.

Economic Significance of Robust Leading Indicators

Citation: IMF Working Papers 2016, 028; 10.5089/9781475594799.001.A001

Note: Reported is the difference in the logarithm of the odds ratio calculated for 75th and 25th percentile of each leading indicator (components of the fiscal distress index). A larger absolute value indicates a larger economic impact of a change in that indicator on the log odds ratio of a fiscal distress.

A. Fiscal Distress Index

Figure 3 presents the evolution of the fiscal distress index based on most robust leading indicators for advanced and emerging economies. Two observations are worth noting. First, the level of fiscal stress tends to be lower in advanced economies compared to emerging economies, which is consistent with the fact that advanced economies have a lower propensity to distress on average. Second, there are spikes in the level of fiscal distress around periods of notable financial crises (late 1990s, the global financial crisis). This is consistent with the empirical regularity of clustering across different types of crises due to contagion effects.

B. Measure of Performance

As pointed out by Ho (2015), it is not obvious that the most robust leading indicators are also those that yield the best predictive performance of an EWS. This is why, following Baldacci et al. (2011), we assess the performance of our model using the total misclassification error:

T M E ( t ) = T y p e 1 ( t ) + T y p e 2 ( t )

where Type1 indicates missed crises as a share of all crises, Type2 indicates false alarms as a share of all non-crisis periods, and t is the threshold level. The choice of the threshold level is made by the policymaker who faces a trade-off. If t is set too high, the index will cross it in very few cases and many crises will be missed resulting in a large Type1 error. Alternatively, if t is set too low, the index will cross the threshold very often so that many alarms will be falsely issued resulting in a large Type2 error. We assume that the utility of the policy maker can be represented by using TME as a loss function, which is minimized over t.

Figure 4 presents in-sample performance for different threshold values. It shows that the TME is minimized at 0.52 when the level of threshold is 0.7. We also plot in the same chart the TME for the Baldacci et al. (2011) fiscal distress index, which obtains a minimum of 0.73 when the level of the threshold is 0.38. The TME is thus reduced by 29 percent, suggesting that our model has a better fit.

Figure 5 presents in-sample and out-of-sample performance for different threshold values of the EBA-based fiscal distress indicator. We use the years 1970 through 2007 to fit the model, compute the coefficients and obtain the threshold. The model is then applied to the years 2008 through 2015. As expected, the in-sample performance outperforms out-of-sample performance. Nevertheless, the minimum TME for the out-of-sample is not much higher than in-sample TME suggesting that our model would have done well predicting the post–2007 sample using information up to 2007.

Figure 6 presents classification table for out-of-sample predictions made by the EBA-based fiscal distress indicator. Out of the 32 crises which were identified between 2008 and 2015, the model predicts 25 correctly (78 percent). A false alarm is issued for 118 out of 344 non-crisis years (34 percent).

Figure 7 presents comparison with other studies. It shows that our model is within the 1st and 3rd quartile of the TME when compared to other studies on EWS. It performs slightly worse than average regarding the type2 errors and slightly better regarding the type1 errors. As mentioned in Berg and Pattillo (2005), a different trade-off between the two error types could be achieved using an asymmetrically weighed loss function. Our model is conservative, issuing false alarms relatively more often than an average EWS study, which seems reasonable from a policymaker’s perspective as a missed crisis is likely to be more costly than a false alarm.

VI. Conclusions

Reviewing the large literature on EWS for fiscal distress, we have assessed the robustness of the leading indicators employed in these studies using the Extreme Bound Analysis. We find both fiscal and non-fiscal leading indicators to be robust, consistent with the prediction of recent theoretical studies emphasizing the role of multiple equilibria and spillovers across different sectors of the economy.

In a second step, we build a vulnerability index using the most robust leading indicators and find that its predictive properties are close to the average found in other EWS (for fiscal, currency, banking, and other types of crises) in the literature. However, the use of robust fiscal and non-fiscal leading indicators helps us to improve performance relative to Baldacci et al. (2011) EWS based on fiscal indicators only. From a policy perspective, these results suggest that some non fiscal leading indicators should be monitored as closely as fiscal leading indicators to assess a country’s vulnerability to fiscal distress.

Designing a EWS based on robust leading indicators poses several problems, which could be addressed in the future. First, the estimation sample using a logit model is constrained by the shortest time series, which can significantly reduce the sample size and thus precision. Second, while using a pooled sample of both advanced and emerging economies does increase the sample size, it constrains the leading indicators and their relative effects to be the same across the two subsamples. However, separate analysis for advanced economies only is complicated given that most fiscal crises have occurred in emerging economies. Lastly, identifying “narrow” fiscal distress events is challenging due to the frequent occurrence of twin or multiple crises and the spillovers among them.

References

  • Abiad, A., 2003, “Early Warning Systems: A Survey and a Regime-Switching Approach,IMF Working Papers 03/32 (Washington, D.C.: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • Alberola, E., Á. Estrada, and D. Santabárbara, 2013, “Growth Beyond Imbalances: Sustainable growth Rates and Output Gap Reassessment.

    • Search Google Scholar
    • Export Citation
  • Alesina, A., A. Prati, and G. Tabellini, 1989, “Public Confidence and Debt Management: A Model and a Case Study of Italy, No. 3135 (Cambridge, Massachusetts: National Bureau of Economic Research).

    • Search Google Scholar
    • Export Citation
  • Berg, A., E. Borensztein, and C. Pattillo, 2005, “Assessing Early Warning Systems: How Have They Worked in Practice?IMF Staff Papers, Vol. 52(3): pp. 462502.

    • Search Google Scholar
    • Export Citation
  • Baldacci, E., I. Petrova, N. Belhocine, G. Dobrescu, and S. Mazraani, 2011, “Assessing Fiscal Stress,IMF Working Paper No. 11/100 (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • Berti, K., M. Salto, and M. Lequien, 2012, “An Early-Detection Index of Fiscal Stress for EU Countries,DG ECFIN Economic Papers No. 475 (Brussels: European Commission).

    • Search Google Scholar
    • Export Citation
  • Bussière, M., 2013, “Balance of Payment Crises in Emerging Markets: How Early Were the ‘Early’ Warning Signals?Applied Economics, 45.12: pp. 16011623.

    • Search Google Scholar
    • Export Citation
  • Borio, C., P. Disyatat, and M. Juselius, 2013, “Rethinking Potential Output: Embedding Information about the Financial Cycle,BIS Working Paper No. 404 (Basel: Bank for International Settlements).

    • Search Google Scholar
    • Export Citation
  • Calvo, G., 1988, “Servicing the Public Debt: The Role of Expectations,American Economic Review, Vol. 78(4): pp. 64761.

  • Chakrabarti, A., and H. Zeaiter, 2014, “The Determinants of Sovereign Default: A Sensitivity Analysis,International Review of Economics and Finance, 33: pp. 300318.

    • Search Google Scholar
    • Export Citation
  • Christiano, Lawrence J., and T. J. Fitzgerald, 2013, “The Band Pass Filter,” International Economic Review 44.2: 435465.

  • Christofides, C., T., Eicher, and C. Papgeorgiou, 2015, “Did Established Early Warning Signals Predict the 2008 Crises?Mimeo (Washington, D.C.: George Washington University).

    • Search Google Scholar
    • Export Citation
  • Cole, Harold L., and T. J. Kehoe, 1996, “A Self-fulfilling Model of Mexico’s 1994–1995 Debt Crisis.Journal of international Economics 41.3: 309330.

    • Search Google Scholar
    • Export Citation
  • Comelli, F., 2013, “Comparing Parametric and Non-parametric Early Warning Systems for Currency Crises in Emerging Market Economies,IMF Working Paper No. 13/134 (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • De Cos, P., G. Koester, E. Moral-Benito, and C. Nickel, 2014, “Signaling Fiscal Stress in the Euro Area: A Country-Specific Early Warning System,ECB Working Paper No. 1712 (Frankfurt: European Central Bank).

    • Search Google Scholar
    • Export Citation
  • Detragiache, E., and A. Spilimbergo, 2001, ”Crises and Liquidity: Evidence and Interpretation,IMF Working Papers 01/2 (Washington, D.C.: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • Eichengreen, B., A. Rose, and C. Wyplosz, 1995, “Exchange Market Mayhem: The Antecedents and Aftermath of Speculative Attacks,Economic Policy, 10 (21): 249312.

    • Search Google Scholar
    • Export Citation
  • Frankel, J., and G. Saravelos, 2012, “Can Leading Indicators Assess Country Vulnerability? Evidence from the 2008–09 Global Financial Crisis,Journal of International Economics, 87: 216231.

    • Search Google Scholar
    • Export Citation
  • Ghosh, A., J. Kim, and M. Qureshi, 2010, “Fiscal Space,Research Department, International Monetary Fund (Washington, D.C.: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • Gourinchas, P-O., and M. Obstfeld, 2011, “Stories of the Twentieth Century for the Twenty-first. No. 17252 (Cambridge, Massachusetts: National Bureau of Economic Research).

    • Search Google Scholar
    • Export Citation
  • Hawkins, J., and M. Klau, 2000, “Measuring Potential Vulnerabilities in Emerging Market Economies,BIS Working Papers No. 91 (Basel: Bank for International Settlements).

    • Search Google Scholar
    • Export Citation
  • Ho, T-K., 2010, “Looking for a Needle in a Haystack: Revisiting the Cross-Country Causes of the 2008–09 CrisisEconomica (forthcoming).

    • Search Google Scholar
    • Export Citation
  • IMF, 2010, “The IMF-FSB Early Warning Exercise – Design and Methodological Toolkit,” (Washington, D.C.: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • Jeanne, O., 1997, “Are Currency Crises Self-Fulfilling? A Test,Journal of International Economics, pp. 26386.

  • Kaminsky, G., S. Lizondo and C. Reinhart, 1998, “Leading Indicators of Currency Crisis,IMF Staff Papers, Vol. 45, 141.

  • Kaminsky, G., and C. Reinhart, 1999, “The Twin Crises: The Causes of Banking and Balance-of-Payments Problems.The American Economic Review, 89 (3): 473500.

    • Search Google Scholar
    • Export Citation
  • Kraay, A., and V. Nehru, 2006, “When is External Debt Sustainable?The World Bank Economic Review, 20(3): pp. 341365.

  • Krugman, P., 1979, “A Model of Balance-of-Payments Crises,Journal of Money, Credit and Banking, 11: pp. 31125.

  • Leamer, E., 1985, “Sensitivity Analyses Would Help,The American Economic Review, 75 (3): 308313.

  • Masson, P., 1999, “Multiple Equilibria, Contagion, and the Emerging Market Crises,IMF Working Paper No. 99/164 (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • Manasse, P., N. Roubini, and A. Schimmelpfennig, 2003, “Predicting Sovereign Debt Crises,IMF Working Paper No. 03/221 (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • Marashaden, O., 1997, “A Logit Model to Predict Debt Rescheduling by Less Developed Countries,Asian Economies 26: pp. 2534.

  • Obstfeld, M., 1984, “Balance-of-Payments Crises and Devaluation,“ Journal of Money, Credit and Banking, 16(2): pp. 20817.

  • Peter, M., 2002, “Estimating Default Probabilities of Emerging Market Sovereigns: A New Look at a Not-So-New Literature.Available at SSRN 1017468.

    • Search Google Scholar
    • Export Citation
  • Reinhart, C., 2002, “Default, Currency Crises and Sovereign Credit Ratings,NBER Working Paper No. 8738 (Cambridge, Massachusetts: National Bureau of Economic Research).

    • Search Google Scholar
    • Export Citation
  • Reinhart, C., and K. Rogoff, 2009, This Time is Different: Eight Centuries of Financial Folly, (Princeton: Princeton University Press).

  • Rothstein, H., A. J. Sutton, and M. Borenstein, 2006, Publication Bias in Meta-analysis: Prevention, Assessment and Adjustments, (John Wiley & Sons).

    • Search Google Scholar
    • Export Citation
  • Sala-i-Martin, X., 1997, “I Just Ran Two Million Regressions,The American Economic Review, pp. 178183.

1

The authors thank Bernardin Akitoby, Suman Basu, Julio Escolano, Greetje Everaert, Davide Furceri, Vitor Gaspar, Jun Kim, Helmut Lütkepohl, Paulo Medas, Marialuz Moreno Badia, Dieter Nautz, Manrique Saenz, Julian Schreiber, Abdelhak Senhadji, and seminar participants at the IMF’s Fiscal Affairs Department and the Freie Universität Empirical Macroeconomics Workshop for useful comments and suggestions. Jonathan Lee and Juan Farah Yacoub provided excellent research assistance. Macarena Torres Girao provided excellent editorial assistance. The usual disclaimer applies.

2

Comprehensive surveys of EWS for banking, currency, sovereign debt, equity, and inflation crises can be found in Kaminsky et al. (1998), Hawkins and Klaw (2000), Abiad (2003), Berg et al. (2005), and Frankel and Saravelos (2012).

3

Some studies have also applied relatively less standard techniques, such as binary recursive tress, artificial neutral networks, and Markov switching models.

4

In the meta-analysis literature, this phenomenon is often referred to as “publication bias”.

5

Similar to the extreme bound analysis, the Bayesian model averaging identifies robust leading indicators by “letting the data speak” and using Bayesian methodology. However, it has even stricter data requirements and works only on a balanced panel.

6

The dataset is available at: http://www.reinhartandrogoff.com/data/.

7

The results do not change qualitatively when using a forecasting horizon of two years.

8

Our results remain qualitatively unchanged when the sample is restricted to the period 1970–2010 as in Baldacci et al. (2011).

9

Unfortunately, we could not replicate the analysis for advanced economies only due to the limited number of fiscal distress events.

10

We have also tried a fixed effects logit specification and the results remain similar. The main drawback of the fixed effect model is that it drops countries that never experienced a fiscal distress (see Bussière, 2013 for a discussion).

11

The index does not include primary balance gap variable (because of low numbers of observations), the foreign exchange debt ratio (because the series end in 2012), and the FX reserves to GDP ratio (because we already control for the FX reserves growth variable).

  • Collapse
  • Expand
Leading Indicators of Fiscal Distress: Evidence from the Extreme Bound Analysis
Author:
Martin Bruns
and
Mr. Tigran Poghosyan