A. Definition of Fiscal Crises

We use the term fiscal crisis to describe a period of heightened budgetary distress, resulting in the sovereign taking exceptional measures. A country may experience fiscal distress when large imbalances emerge between inflows and outflows. These imbalances may lead to a fiscal crisis if the country is not able to respond by sufficiently adjusting its fiscal position. As Bordo and Meissner (2016) note, the canonical fiscal crisis is a debt crisis, when the government is unable to service the interest and or principle as scheduled. Indeed, there has been significant attention in the literature to crises triggered by external default episodes (e.g., Detragiache and Spilimbergo, 2001; Chakrabarti and Zeaiter, 2014). It is important to note, however, that fiscal crises may not necessarily be associated with external debt defaults. They can be associated with other forms of expropriation, including domestic arrears and high inflation that erodes the value of some types of debt (Reinhart and Rogoff, 2009⁴ and 2011). In addition, countries that face severe financial conditions may opt to ask for official creditors’ assistance (e.g., the IMF) instead of defaulting (Manasse and others, 2003).

Our analysis is based on the fiscal crisis episodes identified by Gerling et al. (2017). One key advantage of this database is that is covers a large sample of countries (188), including low income countries, from 1970 to 2015. Another advantage is that it includes episodes of broadly defined budgetary distress and not only outright debt default. Specifically, a fiscal crisis is identified when one or more of the following distinct criteria are satisfied:

• Credit events associated with sovereign debt (e.g., outright defaults and restructuring).

• Recourse to large-scale IMF financial support. Countries under distress may opt to request support from international institutions instead of defaulting. This criterion captures any year under an IMF financial arrangement with access above 100 percent of quota and fiscal adjustment as a program objective.

• Implicit domestic public default (e.g., via high inflation rates). This reflects periods where governments have difficulty meeting their obligations and resort either to running domestic payment arrears or printing money to finance the budget. These episodes are identified by looking at periods of very high inflation and/or accumulation of domestic arrears when data are available.

• Loss of market confidence in the sovereign. This criterion captures any year with extreme market pressures. One sub-criterion is loss of market access: when sovereigns default or stop issuing bonds, controlling for financing needs and previous patterns of issuance. The second sub-criterion is price of market access: there is a threshold for spreads (1,000 basis points).

The database contains 439 fiscal crisis episodes, implying that countries faced on average two crises since 1970 (Table 3.1). They occurred most often in low income developing countries (LICs, an average of about 3 crises per country) and least often in AMs.

Table 3.1.

Number of Identified Fiscal Crisis Episodes (1970-2015)

Table 3.1.

Number of Identified Fiscal Crisis Episodes (1970-2015)

		Total	AM	EM	LIC
Total number of start years		439	25	188	226
	Average per country	2.3	0.7	2.3	3.2
Average duration per episode		5.6	3.8	5.6	5.8

View Table

Table 3.1.

Number of Identified Fiscal Crisis Episodes (1970-2015)

		Total	AM	EM	LIC
Total number of start years		439	25	188	226
	Average per country	2.3	0.7	2.3	3.2
Average duration per episode		5.6	3.8	5.6	5.8

B. Examining Behavior of Key Economic Variables Around Fiscal Crises

We now turn to study the behavior of fiscal and macro variables around fiscal crises. The aim is to observe how these variables change between crisis periods and tranquil (non-crisis) periods. Following the literature, we apply an event study to analyze the behavior of key variables during an 11-year window around the start of the crisis, by comparing the dynamics of variables within this window with that of an out-of-window tranquil period. Following closely Gourinchas and Obstfeld (2012), we specify fixed-effects panel regressions for each variable:

where y is a variable of interest, α_i the fixed-effect, D_t+j the 11 dummy variables taking the value of 1 in period t+j (if period t is a crisis start year), and β_j the conditional effect of a crisis in period t+j of the crisis window relative to tranquil times. We set the event window around crisis episodes to 11 years to observe the buildup of imbalances before the crisis and time for adjustment once the crisis starts. The error term ε captures all the remaining variation in the realization of the variable under study.

Our analysis will focus on the conditional effect of a crisis, β_j, on the key fiscal and macro variables. This allows us to observe the effect of the crisis relative to tranquil times. For example, if the output gap tends to be higher (or lower) than normal times in the years before the crisis starts and the years immediately after.

C. Advanced and Emerging Economies

The event studies indicate that a fiscal crisis tends to be preceded by loose fiscal policy (Figure 3.1). In the run-up to a crisis, there is robust real expenditure growth. The overall balance also tends to deteriorate sharply before the crisis. Once the crisis begins, governments contain expenditure growth aggressively, suggesting fiscal policy is procyclical as economic conditions are weaker during this period. At the crisis onset, public debt ratios rise substantially, especially in AMs and EMs, and only fall very gradually several years after.

View Full Size

Event Studies. Advanced and Emerging Economies

Citation: IMF Working Papers 2018, 181; 10.5089/9781484372555.001.A001

Note: The Figure plots the estimates of β_j for each variable during the 11 -year time window (solid line), together with the 95 percent confidence interval (dotted lines). This is the event study approach in Gourinchas and Obstfeld (2012) and measures the difference between values during the 11-year time window and “normal” period average. The x-axis is the time distance to the start of fiscal crises.

• Download Figure
• Download figure as PowerPoint slide

Economic growth falls sharply at the onset of the crisis. In the crisis run-up, economic growth is generally higher than in normal times. As the crisis starts, it declines sharply. AMs and EMs experience the largest fall in real growth in the first two years of the crisis. Private credit growth, robust before the crisis, tends to decelerate just before the crisis and fall sharply in the first two years.

The event studies also show a worsening of the twin deficits, fiscal balance and external current account, in the crisis run-up. More generally, the evidence suggests fiscal crises start when there are several domestic and external imbalances, it does not appear to be driven only by “fiscal” factors.

D. Low Income Countries

External Variables

The event studies suggest external factors play a significant role in understanding fiscal crises in low income countries (Figure 3.3). Crises on average are preceded by periods of sharply rising food prices which can have a large direct impact on households, but also the governments’ budgets. In many cases, governments have large food subsidies or take measures to counteract rises in food prices, including other safety net expenditure measures as well as tax breaks.⁵ Not surprisingly, LICs also seem vulnerable to slower world economic growth. Declining foreign direct investment (FDI) and lower FX reserve coverage before fiscal crises also suggest external vulnerabilities may be a driver of the crises.

View Full Size

Ratio of Budget Grants to Current Expenditure

(average 2010-16, in percent)

Citation: IMF Working Papers 2018, 181; 10.5089/9781484372555.001.A001

Sources: Authors’ calculations

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Event Studies. Low Income Countries

Citation: IMF Working Papers 2018, 181; 10.5089/9781484372555.001.A001

Note: The Figure plots the estimates of β_j for each variable during the 11 -year time window (solid line), together with the 95 percent confidence interval (dotted lines). This is the event study approach in Gourinchas and Obstfeld (2012) and measures the difference between values during the 11-year time window and “normal” period average. The x-axis is the time distance to the start of fiscal crises.

• Download Figure
• Download figure as PowerPoint slide

Data also indicate that official aid (grants and concessional loans) tends to fall around the start of the crisis. This is important as aid is a key source of fiscal revenue in many cases (IMF, 2009). For example, in about one third of LIC countries, the ratio of grants to current spending exceeds 20 percent, and in 8 countries, this proportion surpasses 50 percent (Figure 3.2).

Domestic Macro-Fiscal Variables

The domestic economy tends to “overheat” before fiscal crises. Economic growth peaks just before the onset, falling afterwards and remaining below the average of tranquil times for some time. This finding is consistent with Gerling et al. (2017). A similar pattern is observed in growth of private sector credit.

Fiscal and debt indicators show a mixed picture around crises. Public debt tends to be significantly higher than in normal times, but is on a downward trend even before the onset of the fiscal crises. Decomposing debt prior to crises reveals that the share of concessional debt in total external debt is lower than normal times, implying that countries have shifted towards non-concessional sources of financing prior to crises. The composition of external debt shifts back towards concessional sources once the crisis begins.

The overall fiscal balance does deteriorate somewhat just before the crisis, but remains close to its level in tranquil times and quickly recovers as the crisis starts. Because the dynamics of the fiscal balance can be influenced by many factors it does not give a clear view of the policy stance. However, real expenditure dynamics suggest countries start tightening a few years before the onset of the crisis—possibly reflecting mounting vulnerabilities—as real primary expenditure growth declines significantly.

A. Alternative Approaches to Predict Crises

In order to construct early warning systems for fiscal crises, we adopt two alternative approaches that have been used in the literature. We first use the signal approach, followed by multivariate logit models. Past studies have compared the performance of the different methods to predict crises, without definitive conclusions (Berg and Pattillo, 1999; Berg et al., 2005; and Baldacci et al., 2011). Using both methods will allow greater insight into the different drivers of fiscal crises and prevent our conclusions being driven by the limitations of one approach.

Signal approach

The signals approach involves monitoring the developments of economic variables that tend to behave differently prior to a crisis. Once they cross a specific threshold this gives a warning signal for a possible fiscal crisis in the next 1-2 years. These thresholds, as discussed below, are derived to balance between the risk of having many false signals and the risk of missing the crisis altogether. An advantage of this approach is that it assesses the relative power of individual variables as predictors of fiscal crisis. This is useful as it increases understanding of the sources of vulnerabilities and policy actions that contribute to a crisis. Another advantage is that it is easier to use with an unbalanced panel. If some data are missing for a variable, but there are observations around crisis periods, this will just make the estimation of the threshold less precise. In addition, this will not affect the estimation of thresholds for other variables where more data are available.

For each explanatory variable x_i, we define an indicator variable

where critⁱ is an indicator variable threshold. There is a ‘signal’ of an approaching crisis if $d_t^i$ = 1.

The threshold critⁱ for an explanatory variable will be a value specific to each country, corresponding to a percentile of values (e.g., 10th percentile) taken by the explanatory variable over the sample period for that country.⁶ The percentile will be common across countries in the sample. For example, the critⁱ for the exchange rate can be the 10th percentile observed over the sample period for each country. The use of percentiles to define thresholds, instead of absolute values, takes into consideration structural differences across countries (e.g., quality of institutions). For example, some countries may be able to withstand higher debt levels than others without risk of distress.

For each explanatory variable, there are the following possibilities in each year:

Table 4.1.

Occurrence of Crisis—True versus Predicted

Table 4.1.

Occurrence of Crisis—True versus Predicted

		State of the World
		Crisis next year	no crisis next year
Predicted result	signal	true positive	false alarm (type I error)
	no signal	miss (type II error)	true negative

View Table

Table 4.1.

Occurrence of Crisis—True versus Predicted

		State of the World
		Crisis next year	no crisis next year
Predicted result	signal	true positive	false alarm (type I error)
	no signal	miss (type II error)	true negative

A low critⁱ would help detect the largest number of crises and reduce the probability of missing a crisis (type II error). While this is the main objective, setting the threshold too low would undermine the credibility of the EWS as it would increase the probability of false alarms (type I error). Following the literature, the critⁱ is chosen to balance these two considerations. Specifically, the value of critⁱ used to compute the indicator variable $d_t^i$ for each country is the value corresponding to the percentile that maximizes the signal-to noise-ratio (SNR). The SNR is defined as the ratio of correct signals (as a percentage of crises in sample) to false alarms (as a percentage of tranquil periods in sample).

While individual variables contain important information on vulnerabilities, a crisis is more likely to happen if several of these indicators are producing signals. As such, in addition to examining individual variables, we construct a composite early warning indicator:

which is a weighted average of the indicator variables. For each indicator variable $d_t^i$, the corresponding weight w_i is given by the measure of signaling power (1 – TME) for the relevant explanatory variable.⁷

Logit model

The early warning systems under this approach draw on standard panel regression (multivariate probit or logit models) with a binary dependent variable equal to one when a crisis begins (or when there is a crisis). The impact of a set of explanatory variables on the crisis probability is derived by estimating the model, through maximum likelihood estimation (e.g., 2016, Catao et al., 2013, and Gourinchas and Obstfeld, 2012). The main advantage of this approach is that it allows testing for the statistical significance of the different leading indicators and takes into account their correlation.

We estimate a pooled logit model. Once a crisis starts, the next two years (if still crisis years) are removed from the sample to avoid a bias. The years after the onset of the crisis tend to have different behavior than other years and could bias the results. The probability of a positive outcome is assumed to be determined by the logistic cumulative distribution function.

For each regression specification, we calculate fitted values (probabilities of crisis for each sample observation). We then search over potential cut-off probabilities (from 1% to 35%) and select the optimal cut-off probability that minimizes the TME. The optimal cut-off probability can be used to generate early warning signals for each regression model.

B. Data

The analysis uses annual data for 188 countries—including advanced, emerging, and low income—for the period 1970-2015. However, the availability and quality of data varies significantly (see Annex I for more details). To test for indicators that could help predict crises we looked at a variety of variables following the literature on fiscal and sovereign debt crises. We also included data that are particularly relevant for low income countries, such as aid flows and concessional debt. Variables fall into the following categories:

• Fiscal and public debt. These include primary and overall balances, expenditure growth, gross financing needs, and measures of public (domestic and external) debt.

• Economic activity and financial. These include economic growth, real time output gap, unemployment rates, credit growth and credit gap, interest rates.⁸

• External. These include variables such as the current account, foreign aid flows (which also have a fiscal impact), exchange rates, terms of trade, international prices of key commodities (food, oil), global growth, and remittances.

A. Advanced and Emerging Economies

The choice of variables

We first estimate early warning systems for advanced economies and emerging markets. This relies on a sample of 118 countries. The advantage of merging the two groups of countries is that we have a larger set of crises, which is a significant limitation when only analyzing advanced economies. In addition, the classification of some countries has changed during the period of the sample—this is especially the case for emerging markets that become advanced economies (under the IMF’s World Economic Outlook classification). Furthermore, the event studies indicate that economic variables tended to behave in a similar fashion around crises for EMs and AEs.

The selection of variables needs to consider that we want to test the robustness of the EWS across different sample periods. One concern in the literature is the risk of overfitting a specific sample, at the cost of reduced ability to predict future crises. As Berg et al. (2005) stressed, the real test is whether the EWS can predict future crises (out of sample forecasts). As such, we build our EWS using a parsimonious set of variables to reduce the risk that by trying to achieve a strong performance in sample, we end up undermining out of sample forecasts. In addition, we select potential leading indicators based on their individual signal power for the “in sample” period using data up to 2006. We then test the robustness of the EWS in the “out of sample” period, that is 2007-15 period. This provides a test of how well the EWS would have helped detect fiscal crises during the turbulent years around the global financial crises. As noted by Christofides, Eicher, and Papageorgiou (2016), EWS have in general performed badly in predicting the 2008 global crisis.

Signal Approach

We first assess a large set of possible leading indicators individually. As discussed in the previous section, we derive the “optimal” threshold for each individual indicator. Appendix Tables A.2–A.3 show the results for the 1- and 2-years lag approaches for both the in-sample and full sample.⁹ The tables show the threshold percentile, the signaling power, and type I and II errors for the best performing indicators. For example, the 1-year lag exercise indicates that the threshold for the current account surplus is the 38th percentile for the 1970-2006 sample. If the current account balance is below, there is a higher risk of a fiscal crisis. This indicator alone would have signaled correctly 55 percent of the crises (or 1 - type II error) over the next year within the 1970-2006 sample. The results in Tables A.2-A.3 show the individual indicators with stronger signaling power remained broadly the same across the two samples (1970-2006 and 1970-2015)—suggesting the drivers of fiscal crises are similar across samples.

The best individual performers in-sample (1970-2006) are chosen based on the tables and then used to build the composite indicators, with 1- and 2-year lags, to assess the probability of starting a crisis. The strategy is to be parsimonious, so we focus on a small set of indicators that have the strongest signaling power. Also, we use only indicators for which a significant number of observations are available. This implies some of the best individual indicators are not used due to data limitations—this is particularly the case for gross financing needs, some debt indicators, and a measure of budget rigidities (size of the wage bill). Table 5.1 shows the performance and weights on individual indicators for the composite indicator constructed using data in-sample (1970-2006). We also present the weights and results for the composite indicator constructed with the same variables but using the full sample (1970-2015), where we have considerably more observations. Using as much data as are available over the full sample (1970-2015), the compositor indicator constructed covers a period containing up to 112 crises.

Table 5.1.

Early Warning System for AEs and EMs: Signals Approach

¹Early Warning System estimated using an unbalanced panel 1970-2006 for the in-sample and 1970-2015 for the full sample.

²Number of crises in the period for which data are available on variables used to predict crises.

³Number of non-crisis years in the period for which data are available, plus crisis years if 3 or more years from the beginning of a crisis.

⁴Number of countries for which data are available on all variables used to predict crisis.

⁵Type I error, type II error and the signal to noise ratio describe performance over the period of the sample.

Table 5.1.

Early Warning System for AEs and EMs: Signals Approach

	1 year ahead	2 years ahead	1 year ahead	2 years ahead
	In-Sample1		Full Sample1
Number of Crises2	70	43	112	82
Number of Non-Crisis Years3	1330	824	2369	1798
Number of Countries4	69	69	105	103
Type I Error5	0.34	0.31	0.33	0.42
Type II Error	0.44	0.49	0.41	0.33
Signal to Noise Ratio	1.63	1.64	1.78	1.60
Variable weights for the composite indicator
Private credit gap (one sided) (% potential credit)	0.09		0.06
Real time output gap (% of potential GDP)		0.14		0.14
Current Account (% GDP)	0.23	0.17	0.26	0.24
Real GDP per capita (% Ch.)	0.10		0.08
Openness (Exports and Imports as % of GDP)	0.10	0.14	0.07	0.16
Reserve Coverage (months of imports)	0.09		0.08
Central Bank Claims on Government (% GDP, 1st diff.)	0.12		0.19
Central Bank Claims on Government (% GDP)		0.11		0.09
Overall Balance (% GDP, 1st diff.)	0.16		0.11
Primary Balance (% GDP, 1st diff.)		0.22		0.14
Expenditure (% GDP, 1st diff.)	0.12		0.15
Real Primary Expenditure (% Ch.)		0.23		0.22

¹Early Warning System estimated using an unbalanced panel 1970-2006 for the in-sample and 1970-2015 for the full sample.

²Number of crises in the period for which data are available on variables used to predict crises.

³Number of non-crisis years in the period for which data are available, plus crisis years if 3 or more years from the beginning of a crisis.

⁴Number of countries for which data are available on all variables used to predict crisis.

⁵Type I error, type II error and the signal to noise ratio describe performance over the period of the sample.

View Table

Table 5.1.

Early Warning System for AEs and EMs: Signals Approach

	1 year ahead	2 years ahead	1 year ahead	2 years ahead
	In-Sample1		Full Sample1
Number of Crises2	70	43	112	82
Number of Non-Crisis Years3	1330	824	2369	1798
Number of Countries4	69	69	105	103
Type I Error5	0.34	0.31	0.33	0.42
Type II Error	0.44	0.49	0.41	0.33
Signal to Noise Ratio	1.63	1.64	1.78	1.60
Variable weights for the composite indicator
Private credit gap (one sided) (% potential credit)	0.09		0.06
Real time output gap (% of potential GDP)		0.14		0.14
Current Account (% GDP)	0.23	0.17	0.26	0.24
Real GDP per capita (% Ch.)	0.10		0.08
Openness (Exports and Imports as % of GDP)	0.10	0.14	0.07	0.16
Reserve Coverage (months of imports)	0.09		0.08
Central Bank Claims on Government (% GDP, 1st diff.)	0.12		0.19
Central Bank Claims on Government (% GDP)		0.11		0.09
Overall Balance (% GDP, 1st diff.)	0.16		0.11
Primary Balance (% GDP, 1st diff.)		0.22		0.14
Expenditure (% GDP, 1st diff.)	0.12		0.15
Real Primary Expenditure (% Ch.)		0.23		0.22

¹Early Warning System estimated using an unbalanced panel 1970-2006 for the in-sample and 1970-2015 for the full sample.

²Number of crises in the period for which data are available on variables used to predict crises.

³Number of non-crisis years in the period for which data are available, plus crisis years if 3 or more years from the beginning of a crisis.

⁴Number of countries for which data are available on all variables used to predict crisis.

⁵Type I error, type II error and the signal to noise ratio describe performance over the period of the sample.

The results show that variables linked to domestic economic activity, fiscal policy, and external imbalances matter (Table 5.1). Some of the key indicators are relevant at both one- or two-year lags—suggesting there may be a buildup of vulnerabilities over time. This is the case for the current account deficit, degree of openness, use of central bank credit to finance the deficit, size of the fiscal (overall or primary) deficit and pace of expansion in public expenditures—all these increase the probability of a future crisis. The relevance of the current account deficit as a leading indicator confirms that twin deficits arise before the crises, as shown by the event studies. The 1 -year lag approach also suggests a few other indicators could be relevant, including economic growth, and reserve coverage. A large output gap is an important signal 2-years ahead. Credit gaps also matter (1-year ahead), which likely reflects imbalances in the real economy.¹⁰

The performance of the composite indicators is similar for both the in sample and out-of-sample forecasts—suggesting our choice of indicators is robust. The models can identify half of the crises (Table A.4.) either one or two years ahead. However, in the out of sample—we estimate the weights for the composite indicator using the data up to 2006 and use it to predict crises in the 2007-15 period—predictive power is somewhat superior for the two-years lag. This reflects a lower proportion of false alarms. We can also see the tradeoff between false alarms and missed crises in Figure 5.1. Our strategy is to maximize the signal to noise ratio, which leads to a lower type 1 error. Trying to get a lower type 2 error would require a large number of false alarms—especially as non-crisis years are by far the most common—undermining the credibility of the early warning system.

View Full Size

Signals Composite Indicator: Setting the Cut-off Threshold

Tradeoff between false alarms (type 1 error) and missed crises (type 2 error)

Citation: IMF Working Papers 2018, 181; 10.5089/9781484372555.001.A001

• Download Figure
• Download figure as PowerPoint slide

Logit Approach

We now turn to the logit approach. Despite data constraints, the number of crises covered (up to 94 in the full sample) is still relatively large, although smaller than for the signals approach. The focus is primarily on trying to improve the overall performance of the EWS relative to the signal approach. To assess the importance of each explanatory variable, we focus on the average marginal effects, which take into account that the impact of a given variable will depend on the values taken by other variables. We also report the pseudo r-square and the AUROC measure—as well as type 1 and type 2 errors—to assess the fit and predictive power of the models.¹¹ The type I and II errors are computed based on the early warning signals generated by fitted values exceeding the optimal cutoff fitted value (chosen to minimize the TME). As for the signal approach, we selected the variables based on the in sample (up to 2006) performance of the model, but we also show the results for the full sample.

Table 5.2.

Advanced and Emerging Economies Logit Model

Note: Reported are marginal effects, p-values in parentheses (*p<0.1, **p<0.05,***p<0.01). The dependent variable is binary (1 for the first year of fiscal crisis; 0 otherwise). The type 2 error corresponds to the portion of missed crises and the type 1 error to the portion of false alarms. The sample covers the period 1970-2015. The in-sample is 1970-2006.

Table 5.2.

Advanced and Emerging Economies Logit Model

	In-Sample			Full sample
Dependent variable: first year of crisis	Model 1	Model 2	Model 3	Model 1	Model 2	Model 3
L2. Growth (%)	-0.000785	-0.000864	-0.000871	-0.00133**	-0.00143**	-0.00144**
	(0.236)	(0.212)	(0.236)	(0.017)	(0.014)	(0.012)
L1. Current Account (% of GDP)	-0.00104**	-0.00111**	-0.000838**	-0.00174***	-0.00183***	-0.00171***
	(0.019)	(0.018)	(0.043)	(0.000)	(0.000)	(0.000)
L2. Real Primary Expenditures (%Ch.)	0.000291*	0.000301*	0.000274*	0.000433***	0.000450***	0.000446***
	(0.067)	(0.064)	(0.094)	(0.005)	(0.005)	(0.005)
L1. World GDP growth	-0.00776	-0.00808	-0.00413	-0.00518	-0.00542	-0.00387
	(0.193)	(0.183)	(0.458)	(0.125)	(0.112)	(0.257)
L1. Output gap	0.00345*	0.00380*	0.00298	0.00631***	0.00677***	0.00630***
	(0.085)	(0.078)	(0.116)	(0.000)	(0.000)	(0.001)
L1. Interest expenses (% of revenue)	0.000892**	0.000877**		0.000600*	0.000583*
	(0.022)	(0.027)		(0.083)	(0.099)
L3. Credit Gap		0.00000405			0.00000806*
		(0.416)			(0.077)
Debt (% of revenue)			0.0000964***			0.0000476*
			(0.002)			(0.071)
Observations	1331	1306	1206	2304	2259	2167
pseudo R sq.	0.04	0.04	0.05	0.05	0.05	0.05
No. crises	56	56	47	94	94	85
Type 1 error (%)	34	33	38	39	41	29
Type 2 error (%)	38	39	28	29	27	36
Threshold prob.(%)	4.3	4.4	3.5	3.9	3.9	4.1
AUROC	0.66	0.66	0.69	0.71	0.71	0.72

Note: Reported are marginal effects, p-values in parentheses (*p<0.1, **p<0.05,***p<0.01). The dependent variable is binary (1 for the first year of fiscal crisis; 0 otherwise). The type 2 error corresponds to the portion of missed crises and the type 1 error to the portion of false alarms. The sample covers the period 1970-2015. The in-sample is 1970-2006.

View Table

Table 5.2.

Advanced and Emerging Economies Logit Model

	In-Sample			Full sample
Dependent variable: first year of crisis	Model 1	Model 2	Model 3	Model 1	Model 2	Model 3
L2. Growth (%)	-0.000785	-0.000864	-0.000871	-0.00133**	-0.00143**	-0.00144**
	(0.236)	(0.212)	(0.236)	(0.017)	(0.014)	(0.012)
L1. Current Account (% of GDP)	-0.00104**	-0.00111**	-0.000838**	-0.00174***	-0.00183***	-0.00171***
	(0.019)	(0.018)	(0.043)	(0.000)	(0.000)	(0.000)
L2. Real Primary Expenditures (%Ch.)	0.000291*	0.000301*	0.000274*	0.000433***	0.000450***	0.000446***
	(0.067)	(0.064)	(0.094)	(0.005)	(0.005)	(0.005)
L1. World GDP growth	-0.00776	-0.00808	-0.00413	-0.00518	-0.00542	-0.00387
	(0.193)	(0.183)	(0.458)	(0.125)	(0.112)	(0.257)
L1. Output gap	0.00345*	0.00380*	0.00298	0.00631***	0.00677***	0.00630***
	(0.085)	(0.078)	(0.116)	(0.000)	(0.000)	(0.001)
L1. Interest expenses (% of revenue)	0.000892**	0.000877**		0.000600*	0.000583*
	(0.022)	(0.027)		(0.083)	(0.099)
L3. Credit Gap		0.00000405			0.00000806*
		(0.416)			(0.077)
Debt (% of revenue)			0.0000964***			0.0000476*
			(0.002)			(0.071)
Observations	1331	1306	1206	2304	2259	2167
pseudo R sq.	0.04	0.04	0.05	0.05	0.05	0.05
No. crises	56	56	47	94	94	85
Type 1 error (%)	34	33	38	39	41	29
Type 2 error (%)	38	39	28	29	27	36
Threshold prob.(%)	4.3	4.4	3.5	3.9	3.9	4.1
AUROC	0.66	0.66	0.69	0.71	0.71	0.72

Note: Reported are marginal effects, p-values in parentheses (*p<0.1, **p<0.05,***p<0.01). The dependent variable is binary (1 for the first year of fiscal crisis; 0 otherwise). The type 2 error corresponds to the portion of missed crises and the type 1 error to the portion of false alarms. The sample covers the period 1970-2015. The in-sample is 1970-2006.

The results, by and large, highlight similar leading indicators as the signals approach (Table 5.2). The probability of entering a crisis increases with growing macroeconomic imbalances due to large output gaps and deteriorating external imbalances. The results also indicate a role for fiscal policy, via public expenditures growth. Figure 5.2 shows the marginal effects for the key indicators (scaled by standard deviation). Current account deficits, high output gaps, and declines in growth tend to have the largest impact on the probability of a crisis. All these factors can be interrelated. For example, high expenditure growth could contribute to a deterioration in the current account and a large output gap, making the fiscal position vulnerable to changes in the economic cycle. The models also show some evidence that the degree of indebtedness and cost of debt matters, as the probability of a crisis increases if interest expenses and debt (both as a share of revenue) rise. Finally, the results for the full sample are similar with the variables showing even higher statistical significance.¹²

View Full Size

Average Marginal Effects for AMs and EMs

(mean and 95 percent confidence interval)

Citation: IMF Working Papers 2018, 181; 10.5089/9781484372555.001.A001

*Marginal effects are scaled by the sample standard deviation of the explanatory variable. The marginal effects show the percentage point change in the probability of crisis given a one standard deviation change in the explanatory variable, on average across all possible values of other explanatory variables.

• Download Figure
• Download figure as PowerPoint slide

The logit model exhibits stronger performance in predicting fiscal crises. This likely reflects higher degrees of freedom: using different lags for different explanatory variables and taking into account the joint impact of all variables. In some of the specifications (Table A.5.), the models can predict around 70 percent of the crises in sample, with the type 1 error (false alarms) around 34-38 percent in the pooled regressions. The predictive power is marginally better for the out of sample forecasts (predicting crises for 2007-15). The model accurately predicts around 75 percent of these crises, with similar type I errors. These results indicate our choice of indicators is robust to different samples.

Comparison with other studies

Our analysis relies on a larger sample of countries and longer time span than most past studies. When comparing results across studies, it is important to note that our models focus on a set of variables that are relevant across a larger number of crises—while other papers can get better fit in-sample for a smaller number of crises. The advantage of the large sample is that it allows us to assess which leading indicators are more robust. In addition, to ensure our results are robust across samples, we only use early warning indicators that we find relevant in the early years (in sample 1970-2006) and then test predictive power on the out of sample period. Several of the past papers used all the information available to select indicators, which prevents a meaningful test of whether their models are robust out of sample. Furthermore, to estimate the output and credit gaps at any point in time, we used the information available at the time. This is particularly relevant for the output gap, as the “real time” output gap can vary significantly from ex-post calculations.

The predictive power in sample of our models is similar to those in past studies, but our results are also robust out of sample. We predict the onset of a crisis in sample with about the same accuracy as in other papers on average (Table A.6). However, our parsimonious approach, based on a relatively small set of variables and the pooled logit, also produces reasonably accurate out of sample forecasts.¹³ Importantly, some of the out of sample forecasts generated in past studies are not robust tests as the leading indicators in their EWS are chosen based on information from the entire sample.¹⁴ The type I errors from our EWS tend to be somewhat higher than in other studies with smaller samples. For policymakers, it may be preferable to have a somewhat higher type I error as the cost of missing a crisis is much larger than the cost of a false signal.

Our results confirm more recent research that stresses non-fiscal variables as crucial when assessing vulnerabilities to fiscal crises. Baldacci and others (2011) relied only on fiscal variables. More recent work has moved away from such a limited focus. Namely, the European Commission (EC) EWS (Berti and el., 2012), which uses a large set of both fiscal and macro-financial leading indicators.¹⁵ Their approach is European-centered, heavily influenced by the recent crises (post-2007), and demanding on data requirements. Other papers, like us, focus on a smaller set of non-fiscal variables including external current account, and credit and output gaps.¹⁶

Our analysis also sheds some light on the debate about whether fiscal and debt variables are robust leading indicators.¹⁷ Our results suggest that indeed fiscal variables matter. Strong expenditure growth and financing pressures (e.g., need for central bank financing) can help predict crises. For debt, there is mixed evidence in the literature on whether the size of public debt is a reliable leading indicator. Some past studies found that the size of FX debt and short-term debt can be good predictors for sovereign debt crises. Sumner and Berti (2017) find that the change in public debt may be a useful indicator for a group of European countries. We found evidence that the size and cost of debt appear to be good leading indicators.

B. Low Income Countries

Contrary to advanced and emerging economies, there is no literature on EWS for LICs that we can build on. We analyze a sample of 70 low income countries. We start by testing the same set of variables for advanced and emerging economies and add others that may be more relevant for LICs. For example, LICs rely much less on market financing and much more on international support via grants and concessional loans. The high dependence on aid makes LICs more vulnerable to volatile aid flows—which impact both the external current account and public finances. Other possible factors include commodity prices and the global environment in general.

The Signals Approach

As for AEs and EMs, we estimated the “optimal” threshold for each individual indicator (Appendix Tables A.7–A.8). Again, we did not use some of the best individual indicators to construct the composite indicator due to data limitations—this is particularly the case for gross financing needs.

Global factors and external vulnerabilities appear among the main determinants of fiscal crises, but fiscal variables and credit conditions are also important leading indicators (Table 5.3). For the 1-year lag, the main indicators signaling a crisis are: the current account deficit, deteriorating fiscal balance, falling world GDP per capita growth, and high private credit gap. The role of the credit gap could be indirect—signaling an overheating in the economy that eventually leads to economic deterioration—or direct, if problems in banks eventually require government support. For the 2-years lag approach, the most significant variables were also external, namely rising world food prices, declining terms of trade, and low reserve coverage. The composition and maturity of debt are also among the more relevant indicators, as a higher share of concessional debt (in total external debt) and longer maturity of new external debt reduce the probability of entering a crisis.

Table 5.3.

Early Warning System for LICs (all countries): Signals Approach

¹Early Warning System estimated using an unbalanced panel 1970-2006 for the in-sample and 1970-2015 for the full sample.

²Number of crises in the period for which data are available on variables used to predict crises.

³Number of non-crisis years in the period for which data are available, plus crisis years if 3 or more years from the beginning of a crisis.

⁴Number of countries for which data are available on all variables used to predict crisis.

⁵Type I error, type II error and the signal to noise ratio describe performance over the period of the sample.

Table 5.3.

Early Warning System for LICs (all countries): Signals Approach

	1 year ahead	2 years ahead	1 year ahead	2 years ahead
	In-Sample1		Full-Sample1
Number of Crises2	84	115	142	161
Number of Non-Crisis Years3	679	916	1167	1268
Number of Countries4	40	57	62	61
Type I Error5	0.27	0.29	0.27	0.29
Type II Error	0.54	0.54	0.57	0.53
Signal to Noise Ratio	1.69	1.56	1.61	1.63
Variable weights for the composite indicator
Private Credit (Gap %)	0.17		0.19
Overall Balance (% Change)	0.36		0.31
Current Account (% GDP)	0.35		0.28
World Real GDP per capita (% Change)	0.13		0.22
Avge. Maturity on New PPG External Debt Disbursements (%, 1st diff.)		0.20		0.19
Concessional Debt (% of PPG External Debt, 1st diff.)		0.03		0.12
Reserve Coverage (months of imports, 1st diff.)		0.17		0.23
World Food Prices (% Change)		0.40		0.23
Terms of Trade (% Change)		0.20		0.23

¹Early Warning System estimated using an unbalanced panel 1970-2006 for the in-sample and 1970-2015 for the full sample.

²Number of crises in the period for which data are available on variables used to predict crises.

³Number of non-crisis years in the period for which data are available, plus crisis years if 3 or more years from the beginning of a crisis.

⁴Number of countries for which data are available on all variables used to predict crisis.

⁵Type I error, type II error and the signal to noise ratio describe performance over the period of the sample.

View Table

Table 5.3.

Early Warning System for LICs (all countries): Signals Approach

	1 year ahead	2 years ahead	1 year ahead	2 years ahead
	In-Sample1		Full-Sample1
Number of Crises2	84	115	142	161
Number of Non-Crisis Years3	679	916	1167	1268
Number of Countries4	40	57	62	61
Type I Error5	0.27	0.29	0.27	0.29
Type II Error	0.54	0.54	0.57	0.53
Signal to Noise Ratio	1.69	1.56	1.61	1.63
Variable weights for the composite indicator
Private Credit (Gap %)	0.17		0.19
Overall Balance (% Change)	0.36		0.31
Current Account (% GDP)	0.35		0.28
World Real GDP per capita (% Change)	0.13		0.22
Avge. Maturity on New PPG External Debt Disbursements (%, 1st diff.)		0.20		0.19
Concessional Debt (% of PPG External Debt, 1st diff.)		0.03		0.12
Reserve Coverage (months of imports, 1st diff.)		0.17		0.23
World Food Prices (% Change)		0.40		0.23
Terms of Trade (% Change)		0.20		0.23

¹Early Warning System estimated using an unbalanced panel 1970-2006 for the in-sample and 1970-2015 for the full sample.

²Number of crises in the period for which data are available on variables used to predict crises.

³Number of non-crisis years in the period for which data are available, plus crisis years if 3 or more years from the beginning of a crisis.

⁴Number of countries for which data are available on all variables used to predict crisis.

⁵Type I error, type II error and the signal to noise ratio describe performance over the period of the sample.

The performance of the composite indicators varies somewhat between the in sample and out-of-sample forecasts.¹⁸ The composite indicator can identify slightly less than half of the crises in sample (Table A.9). The 1-year lag has a higher signal to noise ratio largely reflecting the lower percentage of false alarms. The 2-year lag version can predict crises better out of sample, identifying 55 percent of the crises, but the type I error is higher (false alarms) than for the 1-year lag version. The performance may be different when we analyze separate groups of LICs—we explore this further below.

We analyze separately commodity versus diversified exporters (see Appendix I). In principle, these two groups of countries may face very different vulnerabilities. For example, commodity exporters will be more exposed to falls in commodity prices (oil, metals), while the opposite will be true for diversified exporters.

The performance of the composite indicator, for the commodity exporters, is better than for all LICs (Appendix Tables A.10 and A.11). Based on the results in sample, the composite indicator is able to identify around 60-65 percent of the crises. The performance of out-of-sample forecasting is similar, as we can predict 60-70 percent of the crises. In terms of individual leading indicators, the results suggest that external imbalances and fiscal variables are important. Among the external variables are world real growth, the external current account, volatility in foreign aid, FDI (reduces risk of crisis) and world food prices. A large credit gap (to a lesser degree), also provides a significant signal—indicating that the risk of a crisis increases the more the economy is overheating. Fiscal variables matter too, especially large expenditures or a deteriorating primary balance.

The in-sample performance for diversified exporters is only marginally worse, with the model being able to predict close to 60 percent of crises with the 1-year lag. The performance out-of-sample is stronger for the 2-year lags, as the model can predict a larger share of the crises, almost 70 percent, but with a high type I error. External, financial, and fiscal variables matter (Appendix Table A.10). For the 1-year lag, the most significant indicators are the fiscal balance, current account, and oil prices (higher increases risk). For the 2-years lag, the most significant variables were related to domestic imbalances, size of private credit and fast economic growth (relative to average of past 5 years). Other relevant indicators, include composition of the debt (the more multilateral debt the better) and terms of trade.

Logit Approach

The logit-based EWS performs significantly better than the signal approach both in and out of sample. This suggests that the interaction of several indicators is important in trying to predict crises. For the total LICs sample, the models can predict accurately almost 75 percent of the crises in sample (Tables 5.4 and A.12). The type 1 error (false alarms) is around 35-40 percent. The out-of-sample forecasts show a somewhat weaker performance.

Table 5.4.

Low Income Countries Logit Model

Note: Reported are marginal effects, p-values in parentheses (*p<0.1, **p<0.05,***p<0.01). The dependent variable is binary (1 for the first year of fiscal crisis; 0 otherwise). The type 2 error corresponds to the share of missed crises and the type 1 error to the share of false alarms. The sample covers the period 1970-2015. The In-sample is 1970-2006.

Table 5.4.

Low Income Countries Logit Model

Dependent variable: first year of crisis	In-sample			Full sample
Model 1	Model 2	Model 3	Model 1	Model 2	Model 3
L1. Current Account (% of GDP)	0.000255	0.000226		0.000285	0.000843
	(0.762)	(0.831)		(0.697)	(0.345)
L2. Reserve Coverage (months)	-0.00454	-0.00552	-0.00652	-0.00474	-0.00565	-0.00576
	(0.287)	(0.264)	(0.190)	(0.166)	(0.131)	(0.121)
L1. Overall Balance (% of GDP)	0.00244			0.000117
	(0.239)			(0.925)
L2. FDI (% of GDP)	-0.00524**	-0.00532**	-0.00518**	-0.00189	-0.00224	-0.00166
	(0.027)	(0.040)	(0.033)	(0.192)	(0.129)	(0.238)
L1. Official aid	-0.000212	-0.000157	-0.000161	-0.000368**	-0.000359*	-0.000353*
	(0.121)	(0.217)	(0.214)	(0.045)	(0.064)	(0.069)
L2. World food prices	0.00399**	0.00526***	0.00495***	0.00299***	0.00379***	0.00345***
	(0.011)	(0.004)	(0.006)	(0.000)	(0.000)	(0.000)
L1. GDP growth (dev.from average)	0.00281	0.00594**	0.00497**	0.00257	0.00458**	0.00413*
	(0.173)	(0.010)	(0.042)	(0.178)	(0.031)	(0.067)
L3. Real Primary Expenditures (%Ch.)		0.000174	0.000145		0.0000333	0.0000669
		(0.428)	(0.440)		(0.805)	(0.630)
L1. Concessional debt (% of GDP)		-0.000162			0.000288
		(0.615)			(0.236)
Observations	923	735	744	1424	1208	1226
pseudo R sq.	0.04	0.06	0.05	0.03	0.04	0.04
No. crises	95	77	78	139	121	122
Type 1 error (%)	36	39	36	32	41	34
Type 2 error (%)	40	27	29	48	33	41
Threshold prob.(%)	10.6	10.2	10.5	10.5	9.8	10.5
AUROC	0.64	0.69	0.68	0.62	0.65	0.65

Note: Reported are marginal effects, p-values in parentheses (*p<0.1, **p<0.05,***p<0.01). The dependent variable is binary (1 for the first year of fiscal crisis; 0 otherwise). The type 2 error corresponds to the share of missed crises and the type 1 error to the share of false alarms. The sample covers the period 1970-2015. The In-sample is 1970-2006.

View Table

Table 5.4.

Low Income Countries Logit Model

Dependent variable: first year of crisis	In-sample			Full sample
Model 1	Model 2	Model 3	Model 1	Model 2	Model 3
L1. Current Account (% of GDP)	0.000255	0.000226		0.000285	0.000843
	(0.762)	(0.831)		(0.697)	(0.345)
L2. Reserve Coverage (months)	-0.00454	-0.00552	-0.00652	-0.00474	-0.00565	-0.00576
	(0.287)	(0.264)	(0.190)	(0.166)	(0.131)	(0.121)
L1. Overall Balance (% of GDP)	0.00244			0.000117
	(0.239)			(0.925)
L2. FDI (% of GDP)	-0.00524**	-0.00532**	-0.00518**	-0.00189	-0.00224	-0.00166
	(0.027)	(0.040)	(0.033)	(0.192)	(0.129)	(0.238)
L1. Official aid	-0.000212	-0.000157	-0.000161	-0.000368**	-0.000359*	-0.000353*
	(0.121)	(0.217)	(0.214)	(0.045)	(0.064)	(0.069)
L2. World food prices	0.00399**	0.00526***	0.00495***	0.00299***	0.00379***	0.00345***
	(0.011)	(0.004)	(0.006)	(0.000)	(0.000)	(0.000)
L1. GDP growth (dev.from average)	0.00281	0.00594**	0.00497**	0.00257	0.00458**	0.00413*
	(0.173)	(0.010)	(0.042)	(0.178)	(0.031)	(0.067)
L3. Real Primary Expenditures (%Ch.)		0.000174	0.000145		0.0000333	0.0000669
		(0.428)	(0.440)		(0.805)	(0.630)
L1. Concessional debt (% of GDP)		-0.000162			0.000288
		(0.615)			(0.236)
Observations	923	735	744	1424	1208	1226
pseudo R sq.	0.04	0.06	0.05	0.03	0.04	0.04
No. crises	95	77	78	139	121	122
Type 1 error (%)	36	39	36	32	41	34
Type 2 error (%)	40	27	29	48	33	41
Threshold prob.(%)	10.6	10.2	10.5	10.5	9.8	10.5
AUROC	0.64	0.69	0.68	0.62	0.65	0.65

Note: Reported are marginal effects, p-values in parentheses (*p<0.1, **p<0.05,***p<0.01). The dependent variable is binary (1 for the first year of fiscal crisis; 0 otherwise). The type 2 error corresponds to the share of missed crises and the type 1 error to the share of false alarms. The sample covers the period 1970-2015. The In-sample is 1970-2006.

As in the signals approach, external factors do appear to be a key element (Table 5.4). The most significant in helping predict a crisis are increases in global food prices and decline in FDI inflows and, to a lesser degree, declines in official aid and lower reserve coverage.¹⁹ Another robust predictor is whether the economy is growing at a faster pace relative to past years.²⁰ We find weaker evidence of an impact of traditional fiscal variables, although rising public expenditures do help improve overall predictive power. The fiscal balance, however, does not seem relevant on its own. One possibility is that some countries tighten the fiscal balance when encountering budget pressures, but not enough to prevent the start of a crisis. It could also reflect that the budget is heavily affected by changes in external aid in some countries, implying collinearity between aid and fiscal variables.

The results improve when looking at commodity exporters separately. In sample, we can predict accurately almost 80 percent of the crises (Tables A.13–A.14). Importantly, type 1 errors are lower than for the larger sample. The prediction power is weaker out of sample, but the model can still predict up to 67 percent of the crises. The most significant variables are external, although indicators on domestic activity also matter. Reserve coverage, external aid, global food prices, and “overheating” are important in predicting crises. Somewhat surprisingly, commodity prices do not seem relevant.²¹ This could be because their impact is felt via other activities—namely, commodity booms may lead to overheating in the domestic economy, which is a strong signal of a crisis. In addition, many of these commodity exporters are poor and heavily dependent on foreign aid. Fiscal vulnerabilities are high in LICs where domestic revenue mobilization has not kept pace with rising public spending. These countries have relatively small revenue bases, which limits their ability to increase tax collections in the short run to offset declines in aid flows.

The results for the diversified exporters show important differences to the commodity exporters (Tables A.15–A.16). Some external factors remain important, namely global food prices, but external aid is no longer significant. The in-sample performance of the pooled logit approach is mixed compared to the total LICs sample. It can predict more crises, close to 80 percent, but with a larger frequency of false alarms. The prediction power is similar out of sample.