Predicting Fiscal Crises

This paper identifies leading indicators of fiscal crises based on a large sample of countries at different stages of development over 1970-2015. Our results are robust to different methodologies and sample periods. Previous literature on early warning sistems (EWS) for fiscal crises is scarce and based on small samples of advanced and emerging markets, raising doubts about the robustness of the results. Using a larger sample, our analysis shows that both nonfiscal (external and internal imbalances) and fiscal variables help predict crises among advanced and emerging economies. Our models performed well in out-of-sample forecasting and in predicting the most recent crises, a weakness of EWS in general. We also build EWS for low income countries, which had been overlooked in the literature.

Abstract

This paper identifies leading indicators of fiscal crises based on a large sample of countries at different stages of development over 1970-2015. Our results are robust to different methodologies and sample periods. Previous literature on early warning sistems (EWS) for fiscal crises is scarce and based on small samples of advanced and emerging markets, raising doubts about the robustness of the results. Using a larger sample, our analysis shows that both nonfiscal (external and internal imbalances) and fiscal variables help predict crises among advanced and emerging economies. Our models performed well in out-of-sample forecasting and in predicting the most recent crises, a weakness of EWS in general. We also build EWS for low income countries, which had been overlooked in the literature.

I. Introduction

Two core objectives of fiscal policy are to promote economic stability and ensure provision of public services. Periods of fiscal distress, and ultimately fiscal crises, can undermine those objectives contributing to economic volatility and disruptions in essential public services. Avoiding fiscal crises is also important because it has implications for economic growth and the development agenda. Fatas and Mihov (2013) showed that volatile fiscal policy lowers economic growth, and work by Gerling et al. (2017) suggests fiscal crises can have long-term implications for GDP per capita. As such, it is important to understand what may be causing the crises and how to avoid them.

The literature on fiscal crises and on early warning indicators is limited, although it has expanded in recent years. Most of the past literature focused on sovereign external debt defaults alone, although more recent papers (Gerling et al., 2017) have looked at more comprehensive definitions of fiscal crises, including access to official financing and implicit domestic default (high inflation). There is also a growing interest in leading indicators of fiscal crises (or fiscal distress), partly motivated by the global financial crises. For example, IMF staff has produced some research (e.g., Baldacci et al., 2011, and Bruns and Poghosyan, 2018) and the European Commission developed an early warning system (Berti, Salto, and Lequien, 2013). One limitation of the literature on early warning systems in general is that it relies on relatively small samples of advanced and emerging markets, and, in some cases, is heavily focused on predicting crises during a specific period.

The objective of this paper is to better understand the structural weaknesses that make countries prone to entering a fiscal crisis. Our assumption is that there are vulnerabilities that are systematically relevant across time and groups of countries. The objective is to identify them as they would be useful to signal when there is a higher risk of future crises. Past studies, by focusing on small samples, may be able to explain specific crises better, but their results may not be as useful to detect (and prevent) potential future crises. We take advantage of a new large sample of fiscal crises built by Gerling et al. (2017) to identify more robust macro-fiscal vulnerabilities and triggers that have been important across different fiscal crises.

Figure 1.1.
Figure 1.1.

Probability of Starting a Fiscal Crisis

(percentage, 1970-2015)

Citation: IMF Working Papers 2018, 181; 10.5089/9781484372555.001.A001

Sources: Gerling and others (2017) and authors’ calculations

We also pay closer attention to the drivers of fiscal crises in low income countries (LICs)—which have been largely overlooked in the literature. This is surprising as fiscal crises are most frequent in LICs. They are six times more likely to enter a crisis than an advanced economy and almost twice as likely as an emerging market. Not surprisingly, there have been several initiatives to provide debt relief to LICs to help alleviate the effects of the crises (e.g., IMF, 2011a). However, efforts to reduce the frequency of crises have not been successful. As these countries have unique characteristics, we investigate separately the potential leading indicators of crises and whether they are different from advanced and emerging economies.

We use two of the more common approaches to build early warning systems (EWS) for fiscal crises: the signal approach and logit model. Using two methodologies provides useful insights and allows us to compare predictive power and test the robustness of indicators across methodologies. As Berg et al. (2005) stressed, a key focus should be on the ability to forecast future crises. The preferred models are those that have stronger out-of-sample performance than models that may explain well past crises (overfitting), but are poor at predicting future ones.

Our results show that there is a small set of robust leading indicators (both fiscal and non-fiscal) that help assess the probability of a fiscal crisis. This is especially the case for advanced and emerging markets. For these countries, we find that domestic imbalances (large output or credit gaps), external imbalances (current account deficit), and rising public expenditures increase the probability of a crisis. We also tested how the early warning systems would perform out of sample, especially how well they would have predicated the fiscal distress episodes during the latest global financial crisis (2007-15). Importantly, the models would be able to predict accurately around 75 percent of the crises for these countries.

We find that the leading indicators of fiscal crises vary depending on the level of development. While there are some common drivers among all economies, some vulnerabilities are specific to LICs. These countries are highly vulnerable to changes in external aid, reflecting the high budget dependence on these flows, and food prices (increases pressure for subsidies).

The remainder of the paper is organized as follows. Section II presents a literature review of past work. The next section describes the definition of fiscal crises and examines the behavior of key macro-fiscal variables around crises, using event studies. Section IV presents the methodology used to build the EWS models. This is followed by a section with the main results of the early warning exercise. Section VI presents the conclusions.

II. Previous Literature on Early Warning Systems

There is ample empirical literature on Early Warning System (EWS) models, analyzing currency, banking, and sovereign debt crises. These studies differ not only by the type of crises, but also by the methodology and set of indicators used. In most cases, the data coverage tends to be limited, focusing on samples of advanced and emerging markets. In the fiscal area, attention has been mainly on sovereign debt crises (e.g., Detragiache and Spilimbergo, 2001; Chakrabarti and Zeaiter, 2014),2 but there is a nascent literature on identifying early warning indicators for episodes of fiscal distress more broadly defined. These include Baldacci et al. (2011) and Bruns and Poghosyan (2018), which identify variables that help predict periods of fiscal stress for advanced economies and emerging markets. There has also been recent work focused on European countries (e.g., Sumner and Berti, 2017).

One of the most used methodologies is the signals approach popularized by Kaminsky, Lizondo, and Reinhart (1998) for currency crises.3 This approach selects a number of variables as leading indicators of crises and determines threshold values for each variable beyond which signals are issued indicating that a crisis is likely to happen in the near future. This approach has been used in the context of fiscal crises more recently. For example, Baldacci et al (2011) looked at a sample of emerging and advanced economies. They focused on a parsimonious set of fiscal leading indicators (e.g., fiscal balances and debt (size, composition, and maturity)) to help signal possibility of fiscal distress. Berti, Salto, and Lequien (2013) estimated a EWS focused on European Union countries. They find that macro-financial variables seem to be more relevant than fiscal variables to assess countries’ vulnerabilities to fiscal distress.

The other frequently used approach draws on limited dependent variable techniques (multivariate logit or probit). The most common tool is a panel regression with a binary dependent variable equal to one if a crisis occurs and zero otherwise. The impact of a set of determinants on the crisis probability is then derived by estimating the model and testing the significance of various leading indicators. Literature using this methodology to analyze sovereign debt crises includes Manasse et al. (2003), Gourinchas and Obstfeld (2012), and Dawood, Horsewood, and Strobel (2017). Sumner and Berti (2017) proposed a logit model to complement the signal approach used by the European Commission to identify periods of fiscal distress. They confirm the importance of macro-financial indicators and find some evidence that increases in public debt can be a predictor of distress periods. Bruns and Poghosyan (2018) use extreme bound analysis to identify leading indicators for crises. They find that both fiscal and non-fiscal leading indicators (e.g., output gap and current account balance) should be considered when assessing a country’s vulnerability to fiscal distress.

III. Fiscal Crises Episodes

We start by defining fiscal crises and analyzing the behavior of fiscal and macro variables around them. This will help identify potential candidates for early warning indicators.

A. Definition of Fiscal Crises

We use the term fiscal crisis to describe a period of heightened budgetary distress, resulting in the sovereign taking exceptional measures. A country may experience fiscal distress when large imbalances emerge between inflows and outflows. These imbalances may lead to a fiscal crisis if the country is not able to respond by sufficiently adjusting its fiscal position. As Bordo and Meissner (2016) note, the canonical fiscal crisis is a debt crisis, when the government is unable to service the interest and or principle as scheduled. Indeed, there has been significant attention in the literature to crises triggered by external default episodes (e.g., Detragiache and Spilimbergo, 2001; Chakrabarti and Zeaiter, 2014). It is important to note, however, that fiscal crises may not necessarily be associated with external debt defaults. They can be associated with other forms of expropriation, including domestic arrears and high inflation that erodes the value of some types of debt (Reinhart and Rogoff, 20094 and 2011). In addition, countries that face severe financial conditions may opt to ask for official creditors’ assistance (e.g., the IMF) instead of defaulting (Manasse and others, 2003).

Our analysis is based on the fiscal crisis episodes identified by Gerling et al. (2017). One key advantage of this database is that is covers a large sample of countries (188), including low income countries, from 1970 to 2015. Another advantage is that it includes episodes of broadly defined budgetary distress and not only outright debt default. Specifically, a fiscal crisis is identified when one or more of the following distinct criteria are satisfied:

  • Credit events associated with sovereign debt (e.g., outright defaults and restructuring).

  • Recourse to large-scale IMF financial support. Countries under distress may opt to request support from international institutions instead of defaulting. This criterion captures any year under an IMF financial arrangement with access above 100 percent of quota and fiscal adjustment as a program objective.

  • Implicit domestic public default (e.g., via high inflation rates). This reflects periods where governments have difficulty meeting their obligations and resort either to running domestic payment arrears or printing money to finance the budget. These episodes are identified by looking at periods of very high inflation and/or accumulation of domestic arrears when data are available.

  • Loss of market confidence in the sovereign. This criterion captures any year with extreme market pressures. One sub-criterion is loss of market access: when sovereigns default or stop issuing bonds, controlling for financing needs and previous patterns of issuance. The second sub-criterion is price of market access: there is a threshold for spreads (1,000 basis points).

The database contains 439 fiscal crisis episodes, implying that countries faced on average two crises since 1970 (Table 3.1). They occurred most often in low income developing countries (LICs, an average of about 3 crises per country) and least often in AMs.

Table 3.1.

Number of Identified Fiscal Crisis Episodes (1970-2015)

article image

B. Examining Behavior of Key Economic Variables Around Fiscal Crises

We now turn to study the behavior of fiscal and macro variables around fiscal crises. The aim is to observe how these variables change between crisis periods and tranquil (non-crisis) periods. Following the literature, we apply an event study to analyze the behavior of key variables during an 11-year window around the start of the crisis, by comparing the dynamics of variables within this window with that of an out-of-window tranquil period. Following closely Gourinchas and Obstfeld (2012), we specify fixed-effects panel regressions for each variable:

yi,t=αi+j=55βjDt+j+εi,t

where y is a variable of interest, αi the fixed-effect, Dt+j the 11 dummy variables taking the value of 1 in period t+j (if period t is a crisis start year), and βj the conditional effect of a crisis in period t+j of the crisis window relative to tranquil times. We set the event window around crisis episodes to 11 years to observe the buildup of imbalances before the crisis and time for adjustment once the crisis starts. The error term ε captures all the remaining variation in the realization of the variable under study.

Our analysis will focus on the conditional effect of a crisis, βj, on the key fiscal and macro variables. This allows us to observe the effect of the crisis relative to tranquil times. For example, if the output gap tends to be higher (or lower) than normal times in the years before the crisis starts and the years immediately after.

C. Advanced and Emerging Economies

The event studies indicate that a fiscal crisis tends to be preceded by loose fiscal policy (Figure 3.1). In the run-up to a crisis, there is robust real expenditure growth. The overall balance also tends to deteriorate sharply before the crisis. Once the crisis begins, governments contain expenditure growth aggressively, suggesting fiscal policy is procyclical as economic conditions are weaker during this period. At the crisis onset, public debt ratios rise substantially, especially in AMs and EMs, and only fall very gradually several years after.

Figure 3.1.
Figure 3.1.

Event Studies. Advanced and Emerging Economies

Citation: IMF Working Papers 2018, 181; 10.5089/9781484372555.001.A001

Note: The Figure plots the estimates of βj for each variable during the 11 -year time window (solid line), together with the 95 percent confidence interval (dotted lines). This is the event study approach in Gourinchas and Obstfeld (2012) and measures the difference between values during the 11-year time window and “normal” period average. The x-axis is the time distance to the start of fiscal crises.

Economic growth falls sharply at the onset of the crisis. In the crisis run-up, economic growth is generally higher than in normal times. As the crisis starts, it declines sharply. AMs and EMs experience the largest fall in real growth in the first two years of the crisis. Private credit growth, robust before the crisis, tends to decelerate just before the crisis and fall sharply in the first two years.

The event studies also show a worsening of the twin deficits, fiscal balance and external current account, in the crisis run-up. More generally, the evidence suggests fiscal crises start when there are several domestic and external imbalances, it does not appear to be driven only by “fiscal” factors.

D. Low Income Countries

External Variables

The event studies suggest external factors play a significant role in understanding fiscal crises in low income countries (Figure 3.3). Crises on average are preceded by periods of sharply rising food prices which can have a large direct impact on households, but also the governments’ budgets. In many cases, governments have large food subsidies or take measures to counteract rises in food prices, including other safety net expenditure measures as well as tax breaks.5 Not surprisingly, LICs also seem vulnerable to slower world economic growth. Declining foreign direct investment (FDI) and lower FX reserve coverage before fiscal crises also suggest external vulnerabilities may be a driver of the crises.

Figure 3.2.
Figure 3.2.

Ratio of Budget Grants to Current Expenditure

(average 2010-16, in percent)

Citation: IMF Working Papers 2018, 181; 10.5089/9781484372555.001.A001

Sources: Authors’ calculations
Figure 3.3.
Figure 3.3.

Event Studies. Low Income Countries

Citation: IMF Working Papers 2018, 181; 10.5089/9781484372555.001.A001

Note: The Figure plots the estimates of βj for each variable during the 11 -year time window (solid line), together with the 95 percent confidence interval (dotted lines). This is the event study approach in Gourinchas and Obstfeld (2012) and measures the difference between values during the 11-year time window and “normal” period average. The x-axis is the time distance to the start of fiscal crises.

Data also indicate that official aid (grants and concessional loans) tends to fall around the start of the crisis. This is important as aid is a key source of fiscal revenue in many cases (IMF, 2009). For example, in about one third of LIC countries, the ratio of grants to current spending exceeds 20 percent, and in 8 countries, this proportion surpasses 50 percent (Figure 3.2).

Domestic Macro-Fiscal Variables

The domestic economy tends to “overheat” before fiscal crises. Economic growth peaks just before the onset, falling afterwards and remaining below the average of tranquil times for some time. This finding is consistent with Gerling et al. (2017). A similar pattern is observed in growth of private sector credit.

Fiscal and debt indicators show a mixed picture around crises. Public debt tends to be significantly higher than in normal times, but is on a downward trend even before the onset of the fiscal crises. Decomposing debt prior to crises reveals that the share of concessional debt in total external debt is lower than normal times, implying that countries have shifted towards non-concessional sources of financing prior to crises. The composition of external debt shifts back towards concessional sources once the crisis begins.

The overall fiscal balance does deteriorate somewhat just before the crisis, but remains close to its level in tranquil times and quickly recovers as the crisis starts. Because the dynamics of the fiscal balance can be influenced by many factors it does not give a clear view of the policy stance. However, real expenditure dynamics suggest countries start tightening a few years before the onset of the crisis—possibly reflecting mounting vulnerabilities—as real primary expenditure growth declines significantly.

IV. Estimation Strategy and Data

A. Alternative Approaches to Predict Crises

In order to construct early warning systems for fiscal crises, we adopt two alternative approaches that have been used in the literature. We first use the signal approach, followed by multivariate logit models. Past studies have compared the performance of the different methods to predict crises, without definitive conclusions (Berg and Pattillo, 1999; Berg et al., 2005; and Baldacci et al., 2011). Using both methods will allow greater insight into the different drivers of fiscal crises and prevent our conclusions being driven by the limitations of one approach.

Signal approach

The signals approach involves monitoring the developments of economic variables that tend to behave differently prior to a crisis. Once they cross a specific threshold this gives a warning signal for a possible fiscal crisis in the next 1-2 years. These thresholds, as discussed below, are derived to balance between the risk of having many false signals and the risk of missing the crisis altogether. An advantage of this approach is that it assesses the relative power of individual variables as predictors of fiscal crisis. This is useful as it increases understanding of the sources of vulnerabilities and policy actions that contribute to a crisis. Another advantage is that it is easier to use with an unbalanced panel. If some data are missing for a variable, but there are observations around crisis periods, this will just make the estimation of the threshold less precise. In addition, this will not affect the estimation of thresholds for other variables where more data are available.

For each explanatory variable xi, we define an indicator variable

dti={1ifxi>criti0ifotherwise

where criti is an indicator variable threshold. There is a ‘signal’ of an approaching crisis if dti = 1.

The threshold criti for an explanatory variable will be a value specific to each country, corresponding to a percentile of values (e.g., 10th percentile) taken by the explanatory variable over the sample period for that country.6 The percentile will be common across countries in the sample. For example, the criti for the exchange rate can be the 10th percentile observed over the sample period for each country. The use of percentiles to define thresholds, instead of absolute values, takes into consideration structural differences across countries (e.g., quality of institutions). For example, some countries may be able to withstand higher debt levels than others without risk of distress.

For each explanatory variable, there are the following possibilities in each year:

Table 4.1.

Occurrence of Crisis—True versus Predicted

article image

A low criti would help detect the largest number of crises and reduce the probability of missing a crisis (type II error). While this is the main objective, setting the threshold too low would undermine the credibility of the EWS as it would increase the probability of false alarms (type I error). Following the literature, the criti is chosen to balance these two considerations. Specifically, the value of criti used to compute the indicator variable dti for each country is the value corresponding to the percentile that maximizes the signal-to noise-ratio (SNR). The SNR is defined as the ratio of correct signals (as a percentage of crises in sample) to false alarms (as a percentage of tranquil periods in sample).

While individual variables contain important information on vulnerabilities, a crisis is more likely to happen if several of these indicators are producing signals. As such, in addition to examining individual variables, we construct a composite early warning indicator:

CIt=iwidti

which is a weighted average of the indicator variables. For each indicator variable dti, the corresponding weight wi is given by the measure of signaling power (1 – TME) for the relevant explanatory variable.7

Logit model

The early warning systems under this approach draw on standard panel regression (multivariate probit or logit models) with a binary dependent variable equal to one when a crisis begins (or when there is a crisis). The impact of a set of explanatory variables on the crisis probability is derived by estimating the model, through maximum likelihood estimation (e.g., 2016, Catao et al., 2013, and Gourinchas and Obstfeld, 2012). The main advantage of this approach is that it allows testing for the statistical significance of the different leading indicators and takes into account their correlation.

We estimate a pooled logit model. Once a crisis starts, the next two years (if still crisis years) are removed from the sample to avoid a bias. The years after the onset of the crisis tend to have different behavior than other years and could bias the results. The probability of a positive outcome is assumed to be determined by the logistic cumulative distribution function.

p(startfiscalcrisisit=1)=F(Xi,thβ)

For each regression specification, we calculate fitted values (probabilities of crisis for each sample observation). We then search over potential cut-off probabilities (from 1% to 35%) and select the optimal cut-off probability that minimizes the TME. The optimal cut-off probability can be used to generate early warning signals for each regression model.

B. Data

The analysis uses annual data for 188 countries—including advanced, emerging, and low income—for the period 1970-2015. However, the availability and quality of data varies significantly (see Annex I for more details). To test for indicators that could help predict crises we looked at a variety of variables following the literature on fiscal and sovereign debt crises. We also included data that are particularly relevant for low income countries, such as aid flows and concessional debt. Variables fall into the following categories:

  • Fiscal and public debt. These include primary and overall balances, expenditure growth, gross financing needs, and measures of public (domestic and external) debt.

  • Economic activity and financial. These include economic growth, real time output gap, unemployment rates, credit growth and credit gap, interest rates.8

  • External. These include variables such as the current account, foreign aid flows (which also have a fiscal impact), exchange rates, terms of trade, international prices of key commodities (food, oil), global growth, and remittances.

V. Early Warning Systems

A. Advanced and Emerging Economies

The choice of variables

We first estimate early warning systems for advanced economies and emerging markets. This relies on a sample of 118 countries. The advantage of merging the two groups of countries is that we have a larger set of crises, which is a significant limitation when only analyzing advanced economies. In addition, the classification of some countries has changed during the period of the sample—this is especially the case for emerging markets that become advanced economies (under the IMF’s World Economic Outlook classification). Furthermore, the event studies indicate that economic variables tended to behave in a similar fashion around crises for EMs and AEs.

The selection of variables needs to consider that we want to test the robustness of the EWS across different sample periods. One concern in the literature is the risk of overfitting a specific sample, at the cost of reduced ability to predict future crises. As Berg et al. (2005) stressed, the real test is whether the EWS can predict future crises (out of sample forecasts). As such, we build our EWS using a parsimonious set of variables to reduce the risk that by trying to achieve a strong performance in sample, we end up undermining out of sample forecasts. In addition, we select potential leading indicators based on their individual signal power for the “in sample” period using data up to 2006. We then test the robustness of the EWS in the “out of sample” period, that is 2007-15 period. This provides a test of how well the EWS would have helped detect fiscal crises during the turbulent years around the global financial crises. As noted by Christofides, Eicher, and Papageorgiou (2016), EWS have in general performed badly in predicting the 2008 global crisis.

Signal Approach

We first assess a large set of possible leading indicators individually. As discussed in the previous section, we derive the “optimal” threshold for each individual indicator. Appendix Tables A.2A.3 show the results for the 1- and 2-years lag approaches for both the in-sample and full sample.9 The tables show the threshold percentile, the signaling power, and type I and II errors for the best performing indicators. For example, the 1-year lag exercise indicates that the threshold for the current account surplus is the 38th percentile for the 1970-2006 sample. If the current account balance is below, there is a higher risk of a fiscal crisis. This indicator alone would have signaled correctly 55 percent of the crises (or 1 - type II error) over the next year within the 1970-2006 sample. The results in Tables A.2-A.3 show the individual indicators with stronger signaling power remained broadly the same across the two samples (1970-2006 and 1970-2015)—suggesting the drivers of fiscal crises are similar across samples.

The best individual performers in-sample (1970-2006) are chosen based on the tables and then used to build the composite indicators, with 1- and 2-year lags, to assess the probability of starting a crisis. The strategy is to be parsimonious, so we focus on a small set of indicators that have the strongest signaling power. Also, we use only indicators for which a significant number of observations are available. This implies some of the best individual indicators are not used due to data limitations—this is particularly the case for gross financing needs, some debt indicators, and a measure of budget rigidities (size of the wage bill). Table 5.1 shows the performance and weights on individual indicators for the composite indicator constructed using data in-sample (1970-2006). We also present the weights and results for the composite indicator constructed with the same variables but using the full sample (1970-2015), where we have considerably more observations. Using as much data as are available over the full sample (1970-2015), the compositor indicator constructed covers a period containing up to 112 crises.

Table 5.1.

Early Warning System for AEs and EMs: Signals Approach

article image

Early Warning System estimated using an unbalanced panel 1970-2006 for the in-sample and 1970-2015 for the full sample.

Number of crises in the period for which data are available on variables used to predict crises.

Number of non-crisis years in the period for which data are available, plus crisis years if 3 or more years from the beginning of a crisis.

Number of countries for which data are available on all variables used to predict crisis.

Type I error, type II error and the signal to noise ratio describe performance over the period of the sample.

The results show that variables linked to domestic economic activity, fiscal policy, and external imbalances matter (Table 5.1). Some of the key indicators are relevant at both one- or two-year lags—suggesting there may be a buildup of vulnerabilities over time. This is the case for the current account deficit, degree of openness, use of central bank credit to finance the deficit, size of the fiscal (overall or primary) deficit and pace of expansion in public expenditures—all these increase the probability of a future crisis. The relevance of the current account deficit as a leading indicator confirms that twin deficits arise before the crises, as shown by the event studies. The 1 -year lag approach also suggests a few other indicators could be relevant, including economic growth, and reserve coverage. A large output gap is an important signal 2-years ahead. Credit gaps also matter (1-year ahead), which likely reflects imbalances in the real economy.10

The performance of the composite indicators is similar for both the in sample and out-of-sample forecasts—suggesting our choice of indicators is robust. The models can identify half of the crises (Table A.4.) either one or two years ahead. However, in the out of sample—we estimate the weights for the composite indicator using the data up to 2006 and use it to predict crises in the 2007-15 period—predictive power is somewhat superior for the two-years lag. This reflects a lower proportion of false alarms. We can also see the tradeoff between false alarms and missed crises in Figure 5.1. Our strategy is to maximize the signal to noise ratio, which leads to a lower type 1 error. Trying to get a lower type 2 error would require a large number of false alarms—especially as non-crisis years are by far the most common—undermining the credibility of the early warning system.

Figure 5.1.
Figure 5.1.

Signals Composite Indicator: Setting the Cut-off Threshold

Tradeoff between false alarms (type 1 error) and missed crises (type 2 error)

Citation: IMF Working Papers 2018, 181; 10.5089/9781484372555.001.A001

Logit Approach

We now turn to the logit approach. Despite data constraints, the number of crises covered (up to 94 in the full sample) is still relatively large, although smaller than for the signals approach. The focus is primarily on trying to improve the overall performance of the EWS relative to the signal approach. To assess the importance of each explanatory variable, we focus on the average marginal effects, which take into account that the impact of a given variable will depend on the values taken by other variables. We also report the pseudo r-square and the AUROC measure—as well as type 1 and type 2 errors—to assess the fit and predictive power of the models.11 The type I and II errors are computed based on the early warning signals generated by fitted values exceeding the optimal cutoff fitted value (chosen to minimize the TME). As for the signal approach, we selected the variables based on the in sample (up to 2006) performance of the model, but we also show the results for the full sample.

Table 5.2.

Advanced and Emerging Economies Logit Model

article image
Note: Reported are marginal effects, p-values in parentheses (*p<0.1, **p<0.05,***p<0.01). The dependent variable is binary (1 for the first year of fiscal crisis; 0 otherwise). The type 2 error corresponds to the portion of missed crises and the type 1 error to the portion of false alarms. The sample covers the period 1970-2015. The in-sample is 1970-2006.

The results, by and large, highlight similar leading indicators as the signals approach (Table 5.2). The probability of entering a crisis increases with growing macroeconomic imbalances due to large output gaps and deteriorating external imbalances. The results also indicate a role for fiscal policy, via public expenditures growth. Figure 5.2 shows the marginal effects for the key indicators (scaled by standard deviation). Current account deficits, high output gaps, and declines in growth tend to have the largest impact on the probability of a crisis. All these factors can be interrelated. For example, high expenditure growth could contribute to a deterioration in the current account and a large output gap, making the fiscal position vulnerable to changes in the economic cycle. The models also show some evidence that the degree of indebtedness and cost of debt matters, as the probability of a crisis increases if interest expenses and debt (both as a share of revenue) rise. Finally, the results for the full sample are similar with the variables showing even higher statistical significance.12

Figure 5.2.
Figure 5.2.

Average Marginal Effects for AMs and EMs

(mean and 95 percent confidence interval)

Citation: IMF Working Papers 2018, 181; 10.5089/9781484372555.001.A001

*Marginal effects are scaled by the sample standard deviation of the explanatory variable. The marginal effects show the percentage point change in the probability of crisis given a one standard deviation change in the explanatory variable, on average across all possible values of other explanatory variables.

The logit model exhibits stronger performance in predicting fiscal crises. This likely reflects higher degrees of freedom: using different lags for different explanatory variables and taking into account the joint impact of all variables. In some of the specifications (Table A.5.), the models can predict around 70 percent of the crises in sample, with the type 1 error (false alarms) around 34-38 percent in the pooled regressions. The predictive power is marginally better for the out of sample forecasts (predicting crises for 2007-15). The model accurately predicts around 75 percent of these crises, with similar type I errors. These results indicate our choice of indicators is robust to different samples.

Comparison with other studies

Our analysis relies on a larger sample of countries and longer time span than most past studies. When comparing results across studies, it is important to note that our models focus on a set of variables that are relevant across a larger number of crises—while other papers can get better fit in-sample for a smaller number of crises. The advantage of the large sample is that it allows us to assess which leading indicators are more robust. In addition, to ensure our results are robust across samples, we only use early warning indicators that we find relevant in the early years (in sample 1970-2006) and then test predictive power on the out of sample period. Several of the past papers used all the information available to select indicators, which prevents a meaningful test of whether their models are robust out of sample. Furthermore, to estimate the output and credit gaps at any point in time, we used the information available at the time. This is particularly relevant for the output gap, as the “real time” output gap can vary significantly from ex-post calculations.

The predictive power in sample of our models is similar to those in past studies, but our results are also robust out of sample. We predict the onset of a crisis in sample with about the same accuracy as in other papers on average (Table A.6). However, our parsimonious approach, based on a relatively small set of variables and the pooled logit, also produces reasonably accurate out of sample forecasts.13 Importantly, some of the out of sample forecasts generated in past studies are not robust tests as the leading indicators in their EWS are chosen based on information from the entire sample.14 The type I errors from our EWS tend to be somewhat higher than in other studies with smaller samples. For policymakers, it may be preferable to have a somewhat higher type I error as the cost of missing a crisis is much larger than the cost of a false signal.

Our results confirm more recent research that stresses non-fiscal variables as crucial when assessing vulnerabilities to fiscal crises. Baldacci and others (2011) relied only on fiscal variables. More recent work has moved away from such a limited focus. Namely, the European Commission (EC) EWS (Berti and el., 2012), which uses a large set of both fiscal and macro-financial leading indicators.15 Their approach is European-centered, heavily influenced by the recent crises (post-2007), and demanding on data requirements. Other papers, like us, focus on a smaller set of non-fiscal variables including external current account, and credit and output gaps.16

Our analysis also sheds some light on the debate about whether fiscal and debt variables are robust leading indicators.17 Our results suggest that indeed fiscal variables matter. Strong expenditure growth and financing pressures (e.g., need for central bank financing) can help predict crises. For debt, there is mixed evidence in the literature on whether the size of public debt is a reliable leading indicator. Some past studies found that the size of FX debt and short-term debt can be good predictors for sovereign debt crises. Sumner and Berti (2017) find that the change in public debt may be a useful indicator for a group of European countries. We found evidence that the size and cost of debt appear to be good leading indicators.

B. Low Income Countries

Contrary to advanced and emerging economies, there is no literature on EWS for LICs that we can build on. We analyze a sample of 70 low income countries. We start by testing the same set of variables for advanced and emerging economies and add others that may be more relevant for LICs. For example, LICs rely much less on market financing and much more on international support via grants and concessional loans. The high dependence on aid makes LICs more vulnerable to volatile aid flows—which impact both the external current account and public finances. Other possible factors include commodity prices and the global environment in general.

The Signals Approach

As for AEs and EMs, we estimated the “optimal” threshold for each individual indicator (Appendix Tables A.7A.8). Again, we did not use some of the best individual indicators to construct the composite indicator due to data limitations—this is particularly the case for gross financing needs.

Global factors and external vulnerabilities appear among the main determinants of fiscal crises, but fiscal variables and credit conditions are also important leading indicators (Table 5.3). For the 1-year lag, the main indicators signaling a crisis are: the current account deficit, deteriorating fiscal balance, falling world GDP per capita growth, and high private credit gap. The role of the credit gap could be indirect—signaling an overheating in the economy that eventually leads to economic deterioration—or direct, if problems in banks eventually require government support. For the 2-years lag approach, the most significant variables were also external, namely rising world food prices, declining terms of trade, and low reserve coverage. The composition and maturity of debt are also among the more relevant indicators, as a higher share of concessional debt (in total external debt) and longer maturity of new external debt reduce the probability of entering a crisis.

Table 5.3.

Early Warning System for LICs (all countries): Signals Approach

article image

Early Warning System estimated using an unbalanced panel 1970-2006 for the in-sample and 1970-2015 for the full sample.

Number of crises in the period for which data are available on variables used to predict crises.

Number of non-crisis years in the period for which data are available, plus crisis years if 3 or more years from the beginning of a crisis.

Number of countries for which data are available on all variables used to predict crisis.

Type I error, type II error and the signal to noise ratio describe performance over the period of the sample.

The performance of the composite indicators varies somewhat between the in sample and out-of-sample forecasts.18 The composite indicator can identify slightly less than half of the crises in sample (Table A.9). The 1-year lag has a higher signal to noise ratio largely reflecting the lower percentage of false alarms. The 2-year lag version can predict crises better out of sample, identifying 55 percent of the crises, but the type I error is higher (false alarms) than for the 1-year lag version. The performance may be different when we analyze separate groups of LICs—we explore this further below.

We analyze separately commodity versus diversified exporters (see Appendix I). In principle, these two groups of countries may face very different vulnerabilities. For example, commodity exporters will be more exposed to falls in commodity prices (oil, metals), while the opposite will be true for diversified exporters.

The performance of the composite indicator, for the commodity exporters, is better than for all LICs (Appendix Tables A.10 and A.11). Based on the results in sample, the composite indicator is able to identify around 60-65 percent of the crises. The performance of out-of-sample forecasting is similar, as we can predict 60-70 percent of the crises. In terms of individual leading indicators, the results suggest that external imbalances and fiscal variables are important. Among the external variables are world real growth, the external current account, volatility in foreign aid, FDI (reduces risk of crisis) and world food prices. A large credit gap (to a lesser degree), also provides a significant signal—indicating that the risk of a crisis increases the more the economy is overheating. Fiscal variables matter too, especially large expenditures or a deteriorating primary balance.

The in-sample performance for diversified exporters is only marginally worse, with the model being able to predict close to 60 percent of crises with the 1-year lag. The performance out-of-sample is stronger for the 2-year lags, as the model can predict a larger share of the crises, almost 70 percent, but with a high type I error. External, financial, and fiscal variables matter (Appendix Table A.10). For the 1-year lag, the most significant indicators are the fiscal balance, current account, and oil prices (higher increases risk). For the 2-years lag, the most significant variables were related to domestic imbalances, size of private credit and fast economic growth (relative to average of past 5 years). Other relevant indicators, include composition of the debt (the more multilateral debt the better) and terms of trade.

Logit Approach

The logit-based EWS performs significantly better than the signal approach both in and out of sample. This suggests that the interaction of several indicators is important in trying to predict crises. For the total LICs sample, the models can predict accurately almost 75 percent of the crises in sample (Tables 5.4 and A.12). The type 1 error (false alarms) is around 35-40 percent. The out-of-sample forecasts show a somewhat weaker performance.

Table 5.4.

Low Income Countries Logit Model

article image
Note: Reported are marginal effects, p-values in parentheses (*p<0.1, **p<0.05,***p<0.01). The dependent variable is binary (1 for the first year of fiscal crisis; 0 otherwise). The type 2 error corresponds to the share of missed crises and the type 1 error to the share of false alarms. The sample covers the period 1970-2015. The In-sample is 1970-2006.

As in the signals approach, external factors do appear to be a key element (Table 5.4). The most significant in helping predict a crisis are increases in global food prices and decline in FDI inflows and, to a lesser degree, declines in official aid and lower reserve coverage.19 Another robust predictor is whether the economy is growing at a faster pace relative to past years.20 We find weaker evidence of an impact of traditional fiscal variables, although rising public expenditures do help improve overall predictive power. The fiscal balance, however, does not seem relevant on its own. One possibility is that some countries tighten the fiscal balance when encountering budget pressures, but not enough to prevent the start of a crisis. It could also reflect that the budget is heavily affected by changes in external aid in some countries, implying collinearity between aid and fiscal variables.

The results improve when looking at commodity exporters separately. In sample, we can predict accurately almost 80 percent of the crises (Tables A.13A.14). Importantly, type 1 errors are lower than for the larger sample. The prediction power is weaker out of sample, but the model can still predict up to 67 percent of the crises. The most significant variables are external, although indicators on domestic activity also matter. Reserve coverage, external aid, global food prices, and “overheating” are important in predicting crises. Somewhat surprisingly, commodity prices do not seem relevant.21 This could be because their impact is felt via other activities—namely, commodity booms may lead to overheating in the domestic economy, which is a strong signal of a crisis. In addition, many of these commodity exporters are poor and heavily dependent on foreign aid. Fiscal vulnerabilities are high in LICs where domestic revenue mobilization has not kept pace with rising public spending. These countries have relatively small revenue bases, which limits their ability to increase tax collections in the short run to offset declines in aid flows.

The results for the diversified exporters show important differences to the commodity exporters (Tables A.15A.16). Some external factors remain important, namely global food prices, but external aid is no longer significant. The in-sample performance of the pooled logit approach is mixed compared to the total LICs sample. It can predict more crises, close to 80 percent, but with a larger frequency of false alarms. The prediction power is similar out of sample.

VI. Conclusion

Our analysis identifies robust indicators of vulnerabilities that can help signal a high probability of the onset of a crisis in the near future. Building early warning indicators that help predict future fiscal crises is inherently difficult, including because countries may take mitigating action as they see the growing vulnerabilities. However, we find that some types of vulnerabilities are consistently relevant to explain fiscal crises. This raises the question why governments do not act as they see signals. In large measure they do, as crises among advanced economies are rare. Still, the occurrence of crises may reflect overly optimistic projections about the future (e.g., economic growth, cost of debt), and as such governments underestimate the risks and fail to take mitigating measures. Another possibility could be that other shocks or crisis (e.g., banking) could lead to fiscal pressures.22

Our results show that a relatively small set of robust leading indicators can help assess the probability of a fiscal crisis in advanced and emerging markets with high accuracy. Past studies focused on small samples, which can bias the results towards a specific crisis or type of country (e.g., European countries). Using a larger sample, we find that both fiscal and non-fiscal variables send robust signals that a crisis is probable in the next 1-2 years. Domestic imbalances (large output or credit gaps), external imbalances, and rising public expenditures increase the probability of a crisis. Encouragingly, the performance of the EWS is robust to testing out of sample. The models could have predicted 75 percent of the crises in the years around the global financial crises (2007-15).

There are also important differences in the early warning indicators between LICs and other economies. While some vulnerabilities are common, LICs face unique challenges that need to be considered to monitor effectively for signals of future crises. First, global variables are an important factor. LICs are vulnerable to changes in global economic growth and food prices. In addition, deterioration in official aid or FDI, and low FX reserve coverage also help predict future crises. Second, crises tend to be preceded by overheating of the domestic economy. When growth is significantly larger than the average in previous years, a fiscal crisis tends to follow the next year (as growth falls). Finally, the evidence also indicates fiscal and debt-related indicators matter. In particular, high expenditure growth and less concessional debt structure do provide some signal on the risk of a future crisis. The predictive power of the models tends to be similar as for advanced and emerging markets. For all LICs, we can predict about 75 percent of the crises in sample. The prediction power is somewhat higher when analyzing separately commodity and diversified exporters.

The analysis highlights that countries can reduce the frequency of fiscal crises by adopting prudent policies and strengthening risk management. Fiscal crises are more likely when economies build domestic and external imbalances. This calls for avoiding excessively loose polices when domestic growth is above average. For fiscal policy, this means avoiding pro-cyclical increases in expenditures that would need to be sharply reversed when the cycle turns. The analysis also points towards building buffers to protect from external shocks. For LICs, the results suggest even bigger challenges. The crises are much more frequent and the leading indicators reflect structural vulnerabilities that will take time to address. For example, the dependence on foreign aid will require continued efforts to enhance own sources of domestic revenue.

References

  • Baldacci, E., I. Petrova, N. Belhocine, G. Dobrescu, and S. Mazraani, 2011, “Assessing Fiscal Stress,” IMF Working Paper No. 11/100 (International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • Berg A. and C. Pattillo, 1999, “Predicting currency crises: the indicators approach and an alternativeJournal of International Money and Finance, 18, 56186.

    • Search Google Scholar
    • Export Citation
  • Berg, A., E. Borensztein and C. Pattillo, 2005, “Assessing Early Warning Systems: How Have They Worked in Practice?IMF Staff Papers, 52(3): 462502.

    • Search Google Scholar
    • Export Citation
  • Berti, K., M. Salto, M. Lequien (2013) “An early-detection index of fiscal stress for EU countries”, European Economy. Economic Papers. 475. Brussels.

    • Search Google Scholar
    • Export Citation
  • Borio, C., P. Disyatat, and M. Juselius, 2013, “Rethinking potential output: Embedding information about the financial cycle,” BIS Working Paper No. 404.

    • Search Google Scholar
    • Export Citation
  • Bruns, M. and T. Poghosyan, 2018, “Leading Indicators of Fiscal Distress: Evidence from the Extreme Bound Analysis,” Applied Economics, 50(13), pp. 145478.

    • Search Google Scholar
    • Export Citation
  • Catão, L. and G. M. Milesi-Ferretti, 2014, “External Liabilities and Crises,” Journal of International Economics, 94, pp. 1932.

  • Chakrabarti, A., and H. Zeaiter, 2014, “The Determinants of Sovereign Default: A Sensitivity Analysis,” International Review of Economics and Finance, 33, pp. 300318.

    • Search Google Scholar
    • Export Citation
  • Christofides, C., T. Eicher, and C. Papageorgiou, 2016, “Did established Early Warning Signals predict the 2008 crises?,” European Economic review, 81, pp. 103114.

    • Search Google Scholar
    • Export Citation
  • Ciarlone, A., and G. Trebeschi, 2005. “Designing an early warning system for debt crises.Emerging Markets Review 6: 376395.

  • Dawood, M., Horsewood, N., and F. Strobel, 2017, “Predicting sovereign debt crises: An Early Warning System approach,” Journal of Financial Stability, 28: 1628.

    • Search Google Scholar
    • Export Citation
  • Detragiache, E. and A. Spilimbergo, 2001, “Crises and Liquidity: Evidence and Interpretation,” IMF Working Paper No. 01/02 (International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • European Commission, 2017. “Debt Sustainability Monitor 2016.European Economy Institutional Paper, no. 047.

  • Fatas, A., and I. Mihov. 2013. “Policy Volatility, Institutions, and Economic Growth.The Review of Economics and Statistics 95 (2): 362376.

    • Search Google Scholar
    • Export Citation
  • Fuertes, A., and E. Kalotychou, 2006, “Early warning systems for sovereign debt crises: The role of heterogeneity.Computational Statistics & Data Analysis 51: 14201441.

    • Search Google Scholar
    • Export Citation
  • Gerling, K., P. Medas, T. Poghosyan, J. Farah-Yacoub, and Y. Xu, 2017, “Fiscal Crises,” IMF Working Paper No. 17/86 (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • Gourinchas, P.-O. and M. Obstfeld, 2012, “Stories of the Twentieth Century for the Twenty-First,” American Economic Journal: Macroeconomics, 4(1), pp. 226265.

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund, 2011. “Heavily Indebted Poor Countries (HIPC) Initiative and Multilateral Debt Relief Initiative (MDRI) - Status of Implementation and Proposals for the Future of the HIPC Initiative,” IMF Policy Paper, Washington, DC.

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund, 2011b. “Managing Global Growth Risks and Commodity Price Shocks - Vulnerabilities and Policy Challenges for Low-Income Countries.” IMF Policy Paper, Washington, DC.

    • Search Google Scholar
    • Export Citation
  • Kaminsky, G., S. Lizondo, and C. Reinhart, 1998, “Leading Indicators of Currency Crises,” IMF Staff Papers, Vol. 45, No. 1, pp. 148.

    • Search Google Scholar
    • Export Citation
  • Laeven, L. and F. Valencia, 2012, “Systemic Banking Crises Database; An Update,” IMF Working Paper No. 12/163.

  • Louviere, J.J., Hensher, D. and Swait, D., 2000, Stated Choice Methods: Analysis and Application. Cambridge University Press, Cambridge

  • Manasse, P., N. Roubini, and A. Schimmelpfennig, 2003, “Predicting Sovereign Debt Crises,” IMF Working Paper No. 03/221 (International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • Reinhart, C. and K. Rogoff, 2011. “The Forgotten History of Domestic Debt.Economic Journal 121 (552), pp. 319350.

  • Reinhart, C. and K. Rogoff, 2009, This time is different Eight centuries of financial folly (Princeton, NJ: Princeton University Press).

  • Sumner, S. and K. Berti, 2017, “A Complementary Tool to Monitor Fiscal Stress in European Economies,” EC Discussion Paper, 49 (June).

    • Search Google Scholar
    • Export Citation

Appendix I. Data

We use data for 188 countries for the period 1970-2015. Countries are split into groups of advanced and emerging economies (AEs and EMs, 118) and low-income countries (LICs, 70). For analytical purposes, we also divide LICs into two groups: commodity exporters (28), and diversified exporters (42). There are, however, large differences in data availability among variables.

We use the database of fiscal crises and their duration developed by Gerling et al. (2017). The rest of the variables mostly come from the IMF World Economic Outlook (WEO) database. We also use BIS Securities Data, OECD Quarterly Debt Statistics, and Baldacci (2011) data to expand the general government short-term debt time series.

Financial data (credit to the private sector, central bank claims on the government) are from IMF International Financial Statistics (IFS).

We use the World Bank WDI database for the following variables: concessional debt; external debt stock-public and publicly guaranteed; interest payments on external debt - public and publicly guaranteed; average maturity on new external debt; and average interest on new external debt. The database however does not cover AEs for those variables. We used the same database for remittances and net official development assistance and official aid.

For advanced economies and emerging markets, the output gap was derived as deviation of real GDP from its trend, using HP filter. However, to avoid biasing the results we use a measure of the real-time output gap based on the IMF’s World Economic Outlook vintages (given output gaps based on all data will already incorporate information on future crises). That is, the output gap estimated at any given year is based on the information known at that time. For low-income countries we use deviation of real GDP growth from the average growth in the previous five years.

The credit gap is defined as the difference between the ratio of total credit relative to GDP, and its long-run statistical trend derived using the HP filter. We use a one-sided filtering approach, based only on data available up to the relevant time period, analogous to when forecasting in real time.

Table A.1.

Sample Countries

article image
Source: AMs are defined by the IMF WEO, LICs are defined by the PRGT-eligible IMF members adding Zimbabwe.
Table A.2.

A.2. (1970-2015). Leading Indicators (1 Lag); Advanced and Emerging Economies

article image
Source: Authors’ calculations.Note: The type 2 error corresponds to the portion of missed crises (i.e. false negative or p(no signal of a crisis|crisis=1)) and the type 1 error to the portion of false alarms (i.e. false positive or p(signal of a crisis|crisis=0)). The signal-to-noise ratio (SNR) is calaculated as (1-type 2 error)/(type 1 error), and the signaling power as 1-(type 1 error+type 2 error).
Table A.3.

(1970-2015): Leading Indicators (2 Lags); Advanced and Emerging Economies

article image
Source: Authors’ calculations.Note: The type 2 error corresponds to the portion of missed crises (i.e. false negative or p(no signal of a crisis|crisis = 1)) and the type 1 error to the portion of false alarms (i.e. false positive or p(signal of a crisis|crisis=0)). The signal-to-noise ratio (SNR) is calaculated as (1-type 2 error)/(type 1 error), and the signaling power as 1-(type 1 error+type 2 error).