Leading Indicators of Fiscal Distress
Evidence from the Extreme Bound Analysis1
Author:
Martin Bruns https://isni.org/isni/0000000404811396 International Monetary Fund

Search for other papers by Martin Bruns in
Current site
Google Scholar
Close
and
Mr. Tigran Poghosyan
Search for other papers by Mr. Tigran Poghosyan in
Current site
Google Scholar
Close

Contributor Notes

Author’s E-Mail Address: MBruns@diw.de, TPoghosyan@imf.org.

Early warning systems (EWS) are widely used for assessing countries’ vulnerability to fiscal distress. Most EWS employ a specific set of only fiscal leading indicators predetermined by the researchers, which casts doubt on their robustness. We revisit this issue by using the Extreme Bound Analysis, which allows identifying robust leading indicators of fiscal distress from a large set. Consistent with the theoretical predictions of latest generation crisis models, we find that both fiscal (e.g., fiscal balance, foreign exchange debt) and non-fiscal leading indicators (e.g., output, FX reserves, current account balance, and openness) are robust. In addition, we find that a fiscal vulnerability indicator based on fiscal and non-fiscal leading indicators offers a 29% gain in predictive power compared to a traditional one based on fiscal leading indicators only. It also has good predictive power out of sample, with 78 percent of crises predicted correctly and only 34 percent false alarms issued for the period 2008–15. This suggests that both fiscal and non-fiscal leading indicators should be taken into account when assessing country’s vulnerability to fiscal distress.

Abstract

Early warning systems (EWS) are widely used for assessing countries’ vulnerability to fiscal distress. Most EWS employ a specific set of only fiscal leading indicators predetermined by the researchers, which casts doubt on their robustness. We revisit this issue by using the Extreme Bound Analysis, which allows identifying robust leading indicators of fiscal distress from a large set. Consistent with the theoretical predictions of latest generation crisis models, we find that both fiscal (e.g., fiscal balance, foreign exchange debt) and non-fiscal leading indicators (e.g., output, FX reserves, current account balance, and openness) are robust. In addition, we find that a fiscal vulnerability indicator based on fiscal and non-fiscal leading indicators offers a 29% gain in predictive power compared to a traditional one based on fiscal leading indicators only. It also has good predictive power out of sample, with 78 percent of crises predicted correctly and only 34 percent false alarms issued for the period 2008–15. This suggests that both fiscal and non-fiscal leading indicators should be taken into account when assessing country’s vulnerability to fiscal distress.

I. Introduction

The global financial crisis and the subsequent weakening of fiscal positions in advanced and emerging economies once again underscored the importance of monitoring country’s vulnerability to fiscal distress. What variables should policymakers watch to assess fiscal risks? There is a large empirical literature by now attempting to answer this question using Early Warning Systems (EWS). International financial organizations, central banks, rating agencies and other organizations draw on these studies to develop indicators of vulnerability to fiscal distress.

Most of the existing EWS have the following characteristics. First, they use a predetermined set of leading indicators to assess country’s vulnerability to fiscal distress, which are typically based on economic reasoning. Second, the list of these variables varies widely across studies, in part driven by the preference of the researchers for parsimonious models with a large number of significant leading indicators. Finally, the results from the studies are mixed, with no agreement as to which leading indicators are most robustly associated with fiscal distress.

The purpose of this paper is to revisit the issue of leading indicators of fiscal distress by using the Extreme Bound Analysis (EBA). The main advantage of this methodology is that it takes an agnostic approach with respect to leading indicators of fiscal distress and does not require the researcher to predetermine the set of explanatory variables. Instead, it “allows the data to speak” and ranks a set of possible leading indicators based on the “robustness” of their association with fiscal distress. This methodology was used in other fields of economics, notably, growth theory (see for example Sala-i-Martín, 1997) and only recently was applied to analyze financial crises, for example by Ho (2010). However, to our best knowledge we are the first to apply it for analyzing fiscal distress.

Our analysis leads to the following conclusions. First, both fiscal and non-fiscal leading indicators are robustly associated with fiscal distress. This is in contrast to traditional measures of fiscal stress based on fiscal leading indicators (Baldacci et al., 2011). Second, a vulnerability index based on these robust indicators has a performance comparable to the average of performance of other EWS (for fiscal, currency, banking, and other types of crises) in the literature. However, it has better predictive power than the EWS of Baldacci et al. (2011), which is based on only fiscal leading indicators. Finally, the main result on the importance of both fiscal and non-fiscal leading indicators for fiscal distress is robust to various model specifications and sample compositions. The policy implication is that policymakers should not restrict their attention to fiscal indicators only when assessing country’s vulnerability to fiscal distress.

The remainder of the paper is structured as follows. Section II surveys the related literature. Section III describes the empirical methodology and data. Section IV presents the estimation results using EBA and conducts robustness checks. Section V develops an indicator of fiscal distress based on most robust leading indicators identified using EBA and assesses its predictive power. The final section concludes.

II. Related Literature

A. Theories of Crisis Determinants

When modeling crises, it is important to draw distinction between underlying vulnerabilities and crisis risks. The presence of underlying vulnerabilities is a necessary precondition for a crisis to occur, but not sufficient. Crises tend to be triggered by external or domestic shocks, which are highly uncertain. The underlying vulnerability helps to identify countries that are prone to crisis should even a moderate shock occur.

The early theoretical literature emphasized the role of fundamentals in measuring underlying vulnerability. The classic reference is Krugman (1979), which predicts that weak fundamentals, in part driven by unsustainable fiscal and monetary policies, make countries vulnerable to a balance of payments crisis. For fiscal crises, Detragiache and Spilimbergo (2001) show that sovereign default is the only equilibrium response to a large negative shock to fundamentals.

The following generation of theoretical literature of crises emphasizes the role of self-fulfilling expectations and non-fiscal fundamentals. The self-fulfilling crisis literature (Obstfeld, 1984; Calvo, 1988; Alesina et al., 1989; Cole and Kehoe, 1994; Jeanne, 1997; and Masson, 1999) was inspired by the fact that while some crises were preceded by deterioration in fundamentals, some speculative attacks have taken place without apparent monetary and fiscal imbalances. These studies developed multiple-equilibrium models, which allow better capturing the complex interaction between underlying vulnerabilities and speculative attacks (Eichengreen et al., 1995 provide a review). In these models, countries can jump from good to bad equilibrium for a certain range of values of economic fundamentals. For example, Calvo (1988) shows that if a government is unable to commit to repay its debt, multiple equilibria, including repudiation and inflation, can arise.

The main takeaway from the review of theoretical literature is that there is no a priori reason to restrict leading indicators of fiscal distress to fiscal fundamentals.

B. Empirical Evidence on Early Warning Indicators of Fiscal Distress

The empirical literature on early warning indicators of fiscal distress was developed as part of a larger literature on early warning indicators of financial crises.2 As discussed above, the main objective of these studies is to identify leading indicators making countries vulnerable to a crisis, rather than prediction of the timing of the crisis. The empirical techniques used in this literature could be grouped into two main categories: non-parametric and parametric (Hemming et al., 2003; Baldacci et al., 2011; Frankel and Saravelos, 2012; and Comelli, 2013).3

The most popular non-parametric EWS is the “signals” approach popularized by Kaminsky et al. (1998) in the context of currency crises. This approach selects a number of variables as leading indicators of crises and determines threshold values beyond which a crisis signal is considered to have been given. The main drawback of this approach is that it only focuses on bivariate association between an early warning indicator and crises and does not allow controlling for other factors. Also, the statistical significance of the early warning indicators cannot be determined directly, although out-of-sample performance could be assessed. Studies using these techniques in the context of fiscal crises include Reinhart (2002), Hemming et al. (2003), Baldacci et al (2011), Berti et al. (2012) and de Cos et al. (2014). These studies suggest that fiscal fundamentals, such as the level of public debt and its composition, fiscal deficit, fiscal financing needs, are important leading indicators of fiscal distress.

The parametric EWS models draw on limited dependent variable techniques (multivariate logit, probit). These methods allow testing the significance of various leading indicators in determining the likelihood of crisis occurring in the near future, while accounting for their correlation. However, these methods require long-time series of leading indicators and low degrees of freedom that may prevent the use of multivariate approach when the number of predictors is large. Studies that applied parametric methods to analyze fiscal crises include Marashaden (1997), Peter (2002), Manasse et al. (2003), Kraay and Nehru (2006), and Gourinchas and Obstfeld (2012). These studies confirm the importance of fiscal fundamentals, but suggest that macroeconomic developments in general, especially in the external sector, also play a role.

Both groups of studies share a common characteristic – the set of leading indicators of fiscal distress is predetermined by the researcher. This selection is typically done with the benefit of hindsight, with the significance of the leading indicator typically playing a role in the selection process.4 To overcome this issue, some recent studies applied the extreme bound methodology to study leading indicators of financial crises.

C. Extreme Bound Methodology

The extreme bound methodology was developed and applied to study determinants of growth by Leamer (1978), and was later on extended and popularized by Sala-i-Martin (1997). The methodology does not rely on a predetermined set of explanatory variables and “lets the data speak” by examining all possible combinations of explanatory variables. The focus is on the change in signs of explanatory variables. If a sign change is observed relatively frequently, the explanatory variable is said not robustly related to the dependent variable.

Early warning studies using the extreme bound methodology (or its Bayesian equivalent) include Chakrabarti and Zeaiter (2014), Ho (2014), and Christofides et al. (2015).5 However, to our best knowledge no study used this approach to analyze determinants of fiscal distress, which is the gap we would like to fill in our paper. The closest paper to ours is Chakrabarti and Zeaiter (2014), which analyzes determinants of external debt arrears using a linear regression model. The results suggest that a range of fiscal and non-fiscal indicators, including growth, inflation, trade deficit, foreign reserves, and exchange rate, are robust predictors of external debt arrears.

III. Empirical Methodology and Data

A. Empirical Methodology: Extreme Bound Analysis

As discussed above, Extreme Bound Analysis (EBA) is an agnostic approach to identify explanatory variables that are robustly associated with an outcome variable. Instead of preselecting a small set of explanatory variables, EBA explores a large number of combinations from a pool of variables used in the previous literature and based on theoretical considerations.

In the context of an Early Warning System (EWS) for fiscal distress events, the following logit specification is estimated:

log ( P [ Y = 1 | z , x ] P [ Y = 0 | z , x ] ) = β z j z + β x j x j + ɛ

Where Y is a binary outcome variable indicating a fiscal distress event, z is the leading indicator whose robustness is to be assessed and xj are other leading indicators from the pool of all leading indicators excluding z. Assume there are n elements in this pool. Then, for each leading indicator z, one has to estimate a regression with k additional controls xj, which gives M=(n1k)!k! combinations per leading indicator z and a total of nM regressions to be estimated.

For each leading indicator z, we follow the following procedure:

  • Estimate each of the M regressions and store the estimated coefficient of z, β^zj.

  • Weigh coefficients β^zj by the relative likelihood of the model, which they are a part of, so that coefficients stemming from a very unlikely model receive less weight than others. The weight is:

    ω z j = L z j i = 1 M L z i
  • Calculate the weighted average of coefficients across regressions to obtain the coefficient of leading indicator z:

    β z ^ = j = 1 M ω z j β z j
  • Calculate a likelihood-weighted average standard error in a similar fashion.

    Assess the robustness of βz^. Assuming that βz^ is normally distributed across regressions, calculate the proportion of its distribution to one side of zero, and consider βz^ to be robust if this proportion exceeds a certain confidence level (e.g. 90 percent).

Following these steps for each indicator results in a set of indicators which can be ranked according to their respective robustness.

B. Data

Our dataset covers 29 advanced and 52 emerging economies (81 countries in total) over the period 1970–2015. It builds on the original dataset of fiscal distress developed by Baldacci et al. (2011), extending the data through 2015 and expanding the set of leading indicators.

Fiscal distress

A fiscal distress episode is identified as a period when government experiences extreme funding difficulties. Based on the literature survey, Baldacci et al. (2011) identify four types of criteria to capture fiscal distress episodes: (i) debt default or restructuring; (ii) sovereign bond yield pressure; (iii) large IMF-supported program; and (iv) excessive inflation. We adopt the same definition of fiscal distress (see Table 1) and expand the series through 2015.

Table 1.

Definition of Fiscal Distress

article image
Source: Baldacci et al. (2011).

Table 2 presents the distribution of fiscal distress episodes across their types and country groups. Several observations emerge. First, comparison across country groups suggests that advanced economies have experienced fewer fiscal distress episodes compared to emerging economies. The low unconditional likelihood of fiscal distress for advanced economies will have implications for the conditional analysis conducted below. Second, the comparison across types of distress events suggests that advanced economies experienced almost no outright default episodes and most fiscal distress episodes were manifested in the form of bond yield pressures. By contrast, emerging economies experienced a large number of fiscal distress episodes and were frequent recipients of large IMF-supported bailouts. Finally, the total number of distress episodes is lower than the sum of individual episodes. This is due to the fact that some countries have experienced several types of fiscal distress events simultaneously.

Table 2.

Fiscal Distress Episodes

article image
Source: Baldacci et al. (2011) and authors’ calculations for 2011–2015. Note: The numbers in brackets represent fiscal distress episodes in the original Baldacci et al. (2011) dataset running through 2010.

Figure 1 shows the share of fiscal distress episodes that coincided with currency and banking crises using Reinhart and Rogoff’s dataset.6 We find that currency crises overlap with 26 percent of fiscal distress episodes, while banking crises overlap with 24 percent of fiscal distress episodes for the sample spanning through 2010 (the last year of Reinhart and Rogoff’s dataset). The high share of overlaps suggest that some of the fiscal distress events may have originated outside of the fiscal sector, which is consistent with the “twin crises” narrative proposed by Kaminsky and Reinhart (1999).

Figure 1.
Figure 1.

Banking, Currency and Fiscal Crises

Citation: IMF Working Papers 2016, 028; 10.5089/9781475594799.001.A001

Note: Reported is the overlap between fiscal, banking and currency crises for the period 1970–2010. Data on banking and currency crisis episodes are taken from Reinhart and Rogoff (2009). Data on the first year of a fiscal crisis are from Baldacci et al. (2011).

Leading indicators

We identify 37 variables that were used as leading indicators for fiscal distress in the literature. Table 3 lists these variables as well as their sources. The indicators cover not only the fiscal sector, but also macro, monetary, and external sectors. Some of the indicators reflect the impact of contagion effects and global factors, which is consistent with predictions of the theoretical literature. All variables are measured in percentage points.

Table 3.

List of Leading Indicators of Fiscal Distress

article image
Source: Survey of the literature by the authors.

Table 4 presents descriptive statistics, while Table 5 presents bivariate correlations. As shown in the latter table, most of the variables are not highly correlated with the exception of some debt-related indicators. This should limit the issue of multicollinearity.

Table 4.

Descriptive Statistics

article image
Source: IMF WEO, World Development Indicators, International Finance Statistics, Government Finance Statistics.
Table 5.

Correlation Matrix

article image
article image

IV. Estimation Results

In this section, we present estimation results for the baseline specification covering the whole sample and based on the pooled logit specification. We also check whether results are sensitive to changes in the sample and estimation methods. Leading indicators are assessed to be robust if the probability of not switching the sign exceeds 90 percent, as approximated by the normal distribution. We use a forecasting horizon of 1 year throughout the estimation.7

A. Baseline Extreme Bound Analysis Model

Table 6 presents baseline EBA estimation results. Estimations are performed using the pooled logit model. We use various combinations of 2 additional controls per regression, which leads to 630 regressions per leading indicator (23,410 regressions in total). The main reason for using two additional controls as a baseline is computational efficiency and a higher number of observations per regression. The variables are ranked according to their robustness, with most robust indicators placed on top.

Table 6.

EBA Results: Baseline Specification

article image
Note: Reported are estimation results from the EBA regression. The dependent variable is Baldacci et al. (2011) definition of fiscal distress: (i) debt default or restructuring; (ii) sovereign bond yield pressure; (iii) large IMF-supported program; and (iv) excessive inflation. Estimations are performed using the pooled logit model, with 2 additional controls per specification. The variables are ranked according to their robustness, with most robust indicators placed on top. The sample covers 29 advanced and 52 emerging economies for the period 1970–2015.

The estimation results suggest that both fiscal and non-fiscal variables are robust leading indicators of fiscal distress. We find that fiscal distress tends to follow a period of overheating in the real sector (widening of output gap). This is consistent with Alberola et al. (2013) finding that growth, if it is not driven by sound economic fundamentals, can be detrimental to systemic stability. A related explanation could be that a large positive output gap, even if it is non-inflationary, can be associated with a buildup of financial imbalances, which in turn can indirectly lead to fiscal distress (see Borio et al. 2013).

In addition, adverse developments in the external sector (high current account deficit, low level of FX reserves/GDP ratio, slowdown in FX reserves growth, and higher openness) tend to precede fiscal distress episodes. These results are consistent with Chakrabarti and Zeaiter (2014)’s analysis for external debt arrears and also Gourinchas and Obstfeld’s (2011) analysis of the causes of sovereign default. A negative effect of openness is also found in Manasse et al. (2003).

Among fiscal leading indicators, we found that foreign exchange debt to GDP ratio, primary fiscal balance gap, as well as primary and overall fiscal balance to GDP ratios, are robust indicators, which is in line with most fiscal EWS such as Baldacci et al. (2011).

One potential reason for the robustness of non-fiscal indicators could be a high correlation with fiscal indicators. Table 5 shows that this is not the case: for example, the correlation between FX reserves (% of GDP) and various robust fiscal indicators, such as primary balance, overall balance or primary balance gap does not exceed 0.35 in absolute terms. The same is true for the output gap. This indicates that non-fiscal indicators do not only pick up the effects of fiscal indicators but matter on their own.

An illustration for why external factors perform better than fiscal factors as leading indicators can be observed in Figure 2. It shows that debt to GDP ratio before the crisis is not significantly different from the levels observed in tranquil times, but increases rapidly after the crisis incident. This indicates that debt to GDP ratio – one of the key fiscal indicators used in previous studies (see, e.g., Kraay and Nehru, 2006) – is more of an “ex post” indicator of fiscal distress rather than a leading indicator. The current account balance, on the other hand, is significantly lower relative to tranquil times before the crisis, suggesting that it can be used as a leading indicator. This result supports the hypothesis that fiscal vulnerabilities can be built up outside the public sector.

Figure 2.
Figure 2.

Event Study Analysis

Citation: IMF Working Papers 2016, 028; 10.5089/9781475594799.001.A001

Note: Both graphs as generated using the Gourinchas and Obstfeld (2012) event study methodology. It shows the percent deviation of the debt to GDP ratio and the current account balance to GDP ratio from their respective “tranquil” averages. This is defined as the average level of the respective variable outside the 3-year window around the crisis incident. Dotted lines denote the 10% confidence intervals.

Overall, the baseline results suggest that limiting leading indicators to fiscal variables, like in Baldacci et al. (2011), may result in a loss of important information regarding vulnerability to fiscal distress.8 Consistent with predictions of most recent theories of crises, information from other sectors should also be monitored.

B. Robustness Checks

We run several additional EBA regressions to check the results for sensitivity to changes in the sample and estimation methods.

“Narrow” definition of fiscal distress

The robustness of non-fiscal leading indicators, especially those pertaining to the external sector, may be driven by the fact that some of the large scale IMF programs (third definition of fiscal distress) are triggered by balance of payment or other crises not purely linked to fiscal distress. Also, high inflationary pressures (fourth definition of fiscal distress) may be caused by large devaluations following currency crises. To check whether robustness of non-fiscal variables still holds for “narrow” fiscal distress episodes, we re-estimate the model using a more restrictive sample of fiscal distress episodes that did not overlap with banking and currency crises.

Table 7 presents estimation results from a specification that restricts the dependent variable to “narrow” fiscal distress episodes. The set of robust determinants still includes non-fiscal leading indicators, confirming the baseline results. Gross financing needs and interest costs also turn robust in this specification, suggesting that borrowing cost pressures are important leading indicators for “narrow” fiscal distress episodes. The main drawback of this specification is that the sample has to be restricted through 2010, which is the last year for which banking and currency crisis variables are available in Reinhart and Rogoff’s database.

Table 7.

Robustness Check 1: “Narrow” Fiscal Distress Episodes

article image
Note: Reported are estimation results from the EBA regression. The dependent variable is the narrower fiscal stress definition, which excludes Currency and Banking crises as identifies in Reinhart and Rogoff (2009). Estimations are performed using the pooled logit model, with 2 additional controls per specification. The variables are ranked according to their robustness, with most robust indicators placed on top. The sample covers 29 advanced and 52 emerging economies for the period 1970–2010.

Emerging economies

As indicated in the previous section, most fiscal crisis events took place in emerging economies, which could explain the relatively high proportion of external leading indicators that are mostly relevant for emerging economies. For example, difficulties to finance a persistent current account deficit and the associated decline in FX reserves have historically been associated with fiscal distress in emerging markets. Advanced economies on the other hand, face fewer financing problems of this sort as they are often able to borrow in their own currency and generally rely less on external financing (IMF, 2010). To assess whether our baseline results are mostly driven by emerging economies, we redo the analysis by restricting the sample to emerging economies.9

Table 8 presents estimation results from the sample of emerging economies. The set of robust indicators is very similar to the one for the total sample, confirming the influence of emerging economies in driving the main results. Out of 10 robust leading indicators found in the total sample, 8 remain significant in emerging economies. One additional variable that becomes significant for emerging economies is the unemployment rate. The latter is consistent with the robust result on the output gap, given that unemployment is its mirror image.

Table 8.

Robustness Check 2: Emerging Economies

article image
Note: Reported are estimation results from the EBA regression. The dependent variable is Baldacci et al. (2011) definition of fiscal distress: (i) debt default or restructuring; (ii) sovereign bond yield pressure; (iii) large IMF-supported program; and (iv) excessive inflation. Estimations are performed using the pooled logit model, with two additional controls per specification. The variables are ranked according to their robustness, with most robust indicators placed on top. The sample covers 52 emerging economies for the period 1970–2015.

Number of control variables

We also check the robustness of results to the larger number of control variables. Table 9 presents estimation results from a specification that increases the number of control variables from two to three. This results in substantially higher time for estimating the model, as the number of regressions per leading indicator increases to 7140 (264,180 regressions in total). Nevertheless, the main results remain unchanged, with 8 out of 10 robust indicators from baseline regression remaining robust. As before, the set of robust leading indicators includes both fiscal and non-fiscal variables.

Table 9.

Robustness Check 3: Number of Control Variables

article image
Note: Reported are estimation results from the EBA regression. The dependent variable is Baldacci et al. (2011) definition of fiscal distress: (i) debt default or restructuring; (ii) sovereign bond yield pressure; (iii) large IMF-supported program; and (iv) excessive inflation. Estimations are performed using the pooled logit model, with 3 additional controls per specification. The variables are ranked according to their robustness, with most robust indicators placed on top. The sample covers 29 advanced and 52 emerging economies for the period 1970–2015.

Random effects logit and pooled probit models

We also check robustness of main results to types of discrete choice models.

First, we control for unobserved country-specific heterogeneity by using a random effects logit model.10 Table 10 presents estimation results, showing that the set of robust indicators remains qualitatively unchanged, with 8 out of 10 robust indicators from baseline regression remaining robust.

Table 10.

Robustness Check 4: Random Effects Logit Model

article image
Note: Reported are estimation results from the EBA regression. The dependent variable is Baldacci et al. (2011) definition of fiscal distress: (i) debt default or restructuring, (ii) sovereign bond yield pressure, (iii) large IMF-supported program, and (iv) excessive inflation. Estimations are performed using the random effects logit model, with 2 additional controls per specification. The variables are ranked according to their robustness, with most robust indicators placed on top. The sample covers 29 advanced and 52 emerging economies for the period 1970–2015.