Predicting Macroeconomic and Macrofinancial Stress in Low-Income Countries
  • 1 0000000404811396https://isni.org/isni/0000000404811396International Monetary Fund
  • | 2 0000000404811396https://isni.org/isni/0000000404811396International Monetary Fund
  • | 3 0000000404811396https://isni.org/isni/0000000404811396International Monetary Fund

Contributor Notes

Author’s E-Mail Address: HWeisfeld@imf.org

In recent years, Fund staff has prepared cross-country analyses of macroeconomic vulnerabilities in low-income countries, focusing on the risk of sharp declines in economic growth and of debt distress. We discuss routes to broadening this focus by adding several macroeconomic and macrofinancial vulnerability concepts. The associated early warning systems draw on advances in predictive modeling.

Abstract

In recent years, Fund staff has prepared cross-country analyses of macroeconomic vulnerabilities in low-income countries, focusing on the risk of sharp declines in economic growth and of debt distress. We discuss routes to broadening this focus by adding several macroeconomic and macrofinancial vulnerability concepts. The associated early warning systems draw on advances in predictive modeling.

I. Introduction and Overview

A. Background and Goal

1. Low-income countries (LICs) have shown improved macroeconomic stability since the early 2000s but continue to face volatility and potential for severe stress events. For example, the end of the commodity price super-cycle in 2014, in combination with expansive policy stances, has contributed to strains on growth, fiscal balances, and external positions of numerous LICs. As a result, several LICs, particularly commodity exporters, have seen sharp depreciations and quick increases in external debt that, if continued, could put debt sustainability at risk (IMF, 2018).

2. Consequently, LIC governments have a strong interest in tools for predicting economic stress, as do institutions such as the IMF advising them. Based on predictions of stress, governments may be able to implement strategies for reducing its likelihood of crises materializing and their severity when they do happen, for example by building adequate buffers in good times.1

3. There are numerous frameworks for predicting stress in advanced and emerging economies, which use a range of crisis concepts and empirical approaches. Crisis concepts used include external sector/balance of payments crises, sovereign crises/debt distress, financial sector/banking crises, and growth decelerations/recessions. Traditional empirical approaches to predicting crises include notably the “signal extraction” approach developed by Kaminsky, Lizondo and Reinhart (1998), which issues a crisis warning if an indicator exceeds a specified threshold; and logit or probit estimation based crisis warnings, while more recently, machine-learning techniques such as classification trees have started to be used. Berg, Borensztein, and Pattillo (2005) and IMF (2007) assess the performance of a number of early warning systems developed in the aftermath of the Asian crisis of 1999. Alessi and others (2015) and Holopainen and Sarlin (2017) survey additional work and compare the predictive powers of a range of empirical approaches. Ahuja, Syed, and Wiseman (2017) describe crisis concepts and empirical approaches in use at the IMF until recently, and European Commission, European Stability Mechanism, and OECD staff have also presented empirical frameworks for vulnerability/crisis prediction (Berti, Salto and Lequien 2012; Lenkh, Moshammer and Valenta 2017; and Hermansen and Röhn 2017).

4. In contrast, there are only few frameworks for predicting stress events in LICs, and they focus on only a narrow set of crisis concepts and empirical approaches. The limited extent of work on LICs likely reflects in part the fact that identifying the drivers of stress in LICs is particularly difficult, given the high level of macroeconomic volatility seen in these countries (see IMF 2011b) and the much weaker data available for them. Efforts at the IMF have until recently focused on risks of growth decelerations and debt distress:

  • Dabla-Norris and Bal Gündüz (2012) introduced a methodology for predicting sharp declines in GDP growth, an approach used in the IMF’s Vulnerability Exercise for LICs (IMF 2014, 2015, 2016, and 2018). Their Growth Decline Vulnerability Index classifies countries based on their estimated likelihood of seeing a sharp drop in growth should they be hit by a large adverse shock. The variables entering the index were determined using the signal extraction approach of Kaminski, Lizondo, and Reinhart (1998).

  • The IMF and World Bank joint debt sustainability framework for LICs has aimed to assess the risk of debt distress events. Introduced in 2005 and recently strengthened, this framework has been an important element of IMF and World Bank country work on LICs. Empirical implementation relies on logit regressions as well (IMF and World Bank 2017).

5. This paper proposes early-warning systems for additional crisis concepts for LICs that rely on machine-learning approaches where they perform better than traditional econometric ones.

  • Crisis concepts: With growth declines and debt distress having been tackled previously, the new concepts are balance of payments crises, inflation crises, and financial sector crises in LICs. Such crises are relevant for LICs as well, notwithstanding the facts that (i) sudden stops or reversals of capital inflows remain a less important contributor to balance of payments crises for many LICs given their as yet limited degree of de facto integration into global financial markets; (ii) the impact of financial sector crises on growth may be smaller in LICs than in more developed countries given their smaller financial sectors – the fiscal costs of banking sector repair can be sizeable in LICs as well.2

  • Prediction: As traditional econometric methods struggle in predicting crises in LICs, we rely on machine learning approaches where they perform better. Some of these approaches are particularly well equipped to capture non-linearities. Non-linearity is a key feature of crises, which occur when several factors interact, such as one or more shocks affecting an economy that suffers from perhaps several pre-existing weaknesses (see Holopainen and Sarlin, 2017). A drawback of machine learning approaches is that they can be difficult to interpret and hence intransparent. For this reason, before using machine learning for prediction, we always provide not only a thorough discussion of stylized facts but also use traditional econometrics to build understanding and intuition.

6. Three technical elements merit being highlighted:

  • The proposed crisis concepts are operationalized at an intermediate level of severity. Predictions will be most useful if they signal not only risks of experiencing tail events but also somewhat less severe difficulties, which are still important for macroeconomic performance and policies.

  • Predictions aim to identify the unconditional likelihood that a crisis event will occur in the following year, as is conventional in the literature, rather than the conditional probability proposed by Dabla-Norris and Gunduz, 2012. Users such as country authorities are likely interested more in absolute risk than in conditional risk. Relatedly, we use the terms “vulnerability” and “risk” synonymously.

  • Predictive performance is assessed based primarily on out of sample prediction. A prediction is out of sample if the period for which prediction is being made was not used to help specify or estimate the empirical model. Approaches that have been developed by maximizing in-sample predictive performance are vulnerable to “overfitting”, resulting in a substantial deterioration of predictive performance when confronted with new data. We see the focus on out-of-sample performance as a key strength of our approach relative to some previous work.

7. The paper is structured as follows: In the remainder of this introduction and overview, subsection B summarizes the new crisis concepts, including crisis definition, relevance for growth, key stylized facts, and early warning systems; and subsection C discusses the temporal incidence and overlaps of the crises. Appendices then offer some factual information and technical background discussions: Appendix I.1 lists the LICs included in this study; Appendix I.2 provides explanations on empirical methods for predicting economic stress, including a very brief introduction into the machine learning techniques used; and Appendix I.3 discusses two implementation issues that can arise in the context of vulnerability analysis: moving from binary early warning signals to more finely graduated ones, and aggregating sectoral crisis signals to an overall rating, should aggregation be desired. Sections II-V then present the crisis concepts in a largely nontechnical manner, with brief discussions of key technical elements in appendices. Companion papers on external crises, inflation crises, and financial sector crises in LICs provide in-depth discussions, including all technical detail.

B. Overview of the New Crisis Concepts

8. A crisis prediction framework for LICs may wish to capture vulnerability to three types of crises that have been largely neglected in crisis prediction work for LICs so far: balance of payments crises, inflation crises, and financial sector crises.3 Summarizing key elements of sections II-IV, in the following we briefly present these new concepts, discuss the relevance of such crises and their stylized facts, and describe the associated early warning systems.4

External Sector/Balance of Payments Crises

  • Crisis concept: The concept relies on sharp nominal exchange rate depreciation, drops in reserves to a very low level, and contractions in import volumes that reflect inability to maintain imports at levels observed in the recent past. This concept modifies existing definitions of BoP crises to address the LIC-specific challenges, in particular data quality issues. Also, capital account related crisis concepts that are appropriate for more developed countries and are the heart of the VEE, are not yet very relevant for the majority of LICs that continue to have limited access to global capital markets.

  • Relevance: BoP crises appear to have large output costs, notwithstanding the fact that growth typically already starts falling before crisis onset, suggesting some degree of feedback between growth and BoP crises. Specifically, a comparison of GDP growth during the year of the crisis and the two following years with growth in quiet times suggests that at the end of this period, output is about 7 percent lower than it would have been otherwise. BoP crises also result in inflation spikes when they are associated with depreciations.

  • Stylized facts: High fiscal deficits, sustained credit growth, and real effective exchange rate overvaluation often precede BoP crises, and a range of shocks can trigger them. The sizes of both shocks and buffers matter: for given shock sizes, countries with strong buffers are less likely to experience an external sector crisis than countries with weak ones; and for a given strength of buffers, countries affected by a smaller shock are less likely to fall into crisis than countries hit by a large shock.

  • Performance of early warning system: Traditional econometric approaches struggle to predict BoP crises. Thus, we explored machine learning approaches. The Random Forest classification technique, which is well suited to capturing non-linearities and interactions between explanatory variables, provides acceptable out-of-sample predictions: over the period 2010–15, it misses 25 percent of crises and sounds false alarms 39 percent of the time (and retains this performance reliably when the sample changes).

Inflation Crises

  • Crisis concept: Inflation crises/stress episodes are defined as times when inflation exceeds 15 percent per year. This level of inflation is known to adversely affect growth (Khan and Senhadji, 2001) and is not as high as to flag only very rare events, in keeping with the goal of including stress episodes of intermediate severity.

  • Relevance: High inflation has been shown to adversely affect growth, income distribution, and poverty levels. In the sample under study, a comparison of GDP growth during the year of the crisis and the two following years with growth in quiet times suggests that at the end of this period output is about 2½ percent below the level that would have prevailed otherwise.

  • Stylized facts: Episodes of high inflation in LICs often result from both shocks such as increases in world fuel and food prices, and government policies, particularly expansive fiscal and monetary policies. They can also result from exchange rate depreciation, e.g. in the context of external crises.

  • Performance of early warning system: The early warning system developed for inflation crises relies on machine learning approaches after traditional econometric approaches failed to outperform desk forecasts. The Random Forests classification method delivers strong out of sample prediction performance, with rates of missed crises and false alarms of about 15 percent each.

Financial Sector Crises and Stress Episodes

  • Crisis concept: The main definition is that of banking crises as proposed by Laeven and Valencia (2013). It identifies a crisis by the occurrence of one or both of the following: (a) significant signs of financial distress in the banking system-as indicated by significant bank runs, losses in the banking system, and/or bank liquidations; and/or (b) significant banking policy intervention measures in response losses in the banking system. In addition, given the limited frequency of systemic banking crises in LICs in recent years, an alternative notion of banking sector stress as assessed by country teams is used to capture less severe stress events.

  • Relevance: An extensive literature documents the output and fiscal costs of banking crises in advanced economies and emerging markets. In LICs, however, large output costs of financial sector crises have not been documented and are also not apparent in the sample under study. The apparently limited impact on growth may reflect the much smaller size of LICs’ financial sectors relative to GDP. Nevertheless, it appears that public debt rises by several percent of GDP in the context of financial sector crises, possibly reflecting in part the fiscal costs of banking sector repair.

  • Stylized facts: As concerns the factors preceding and likely contributing to financial crises, we find that banking crises in LICs often follow on a decline in commodity terms of trade. In contrast to patterns found in more developed countries, high credit growth does not help predict financial sector crises in LICs.

  • Performance of early warning system: Using a recently updated sample of banking crises in LICs since the 1980s, and a random effects logit estimator employed in the recent literature, (in-sample) crisis predictions are fairly accurate, with 14 percent of missed crises and 25 percent of false alarms. The use of FSIs to predict less severe financial stress episodes shows a 40 percent rate of missed crises and a false alarm rate of 16 percent, a result that would benefit from being confirmed based on a larger data set in the future.

Appendix I.1. List of Low-Income Countries Included

A total of 73 countries were included in the sample. These are the 70 countries that are presently eligible for borrowing from the Poverty Reduction and Growth Trust (PRGT), except for Somalia (excluded due to missing data) plus Bolivia, Mongolia, Nigeria, and Vietnam, which graduated from PRGT eligibility in 2015. With this, the set of countries includes the following:

Afghanistan, Bangladesh, Benin, Bhutan, Bolivia, Burkina Faso, Burundi, Cambodia, Cameroon, Cape Verde, Central African Republic, Chad, Comoros, Republic of Congo, Democratic Republic of Congo, Côte d’Ivoire, Djibouti, Dominica, Eritrea, Ethiopia, The Gambia, Ghana, Grenada, Guinea-Bissau, Guinea, Guyana, Haiti, Honduras, Kenya, Kiribati, Kyrgyz Republic, Lao P.D.R., Lesotho, Liberia, Madagascar, Malawi, Maldives, Mali, Marshall Islands, Mauritania, Micronesia, Moldova, Mongolia, Mozambique, Myanmar, Nepal, Nicaragua, Niger, Nigeria, Papua New Guinea, Rwanda, St. Lucia, St. Vincent and the Grenadines, Samoa, São Tomé and Príncipe, Senegal, Sierra Leone, Solomon Islands, South Sudan, Sudan, Tajikistan, Tanzania, Timor Leste, Togo, Tonga, Tuvalu, Uganda, Uzbekistan, Vanuatu, Vietnam, Yemen, Zambia and Zimbabwe.

Appendix I.2. Some Explanations on Empirical Methods to Predict Stress Events

A. Assessing Predictive Performance

A traditional approach to assessing an empirical approach’s predictive performance relies on loss functions. In a first step, probit or logit regressions or other approaches are used to estimate the likelihood of a crisis occurring in the following year. In a second step, estimated probabilities are then identified as crisis or non-crisis predictions by determining a threshold probability such that it minimizes a loss function over a weighted sum of the shares of missed crises in all crises and of false alarms in all calm times. In a third and final step, an empirical approach’s performance in issuing crisis predictions is then also often evaluated using loss functions.

Loss functions are, however, subjective in that they reflect the researcher’s preferences regarding the trade-off between missing crises and issuing false alarms. Different authors use loss functions with different functional forms and different weights on missed crises and false alarms (Text Table below).

Preferences for Type I and Type II Errors in the Literature

article image

The Receiver Operating Characteristic Curve (ROC) provides an objective alternative to loss functions. It plots a system’s true positive rate (equal to 1 minus the rate of false negatives or missed crises) on the y-axis against its false positive rate (false alarms) on the x-axis.5 All crisis predictions that are not informed by the data, such as coin flips, lie on the 45-degree line between points (0,0) and (1,1). The improvement in prediction made possible by an empirical model is larger the more its ROC curve bends up and away from the 45-degree line. The stronger the curvature, the larger is the rate of true negatives that can be obtained for any given rate of true positives. A model’s area under the ROC curve (AUC) can thus be used as a measure of predictive performance.

A graphical presentation illustrates the implications of using different loss functions on crisis predictions (Figure below):

  • The functional form of loss functions and the weights on missed crises and false alarms determine the shape and slope of indifference curves. Linear loss functions are represented by linear indifference curves, and loss functions based on a noise-to-signal ratio would be represented by strictly convex curves. The higher is the weight put on missed crises, the flatter the slope of the indifference curve.

  • The optimal probability threshold used in an early warning system is the one where the ROC curve is tangent to an indifference curve. If the ROC curve is concave or the indifference curves are strictly convex, then the optimal threshold is between 0 and 1. For linear indifference curves, however, the optimal threshold can also reach 0 or 1 or very close to these extremes. Such corner solutions or near corner solutions happen when the slope of indifference curves is sufficiently different from 1 and the ROC curve has a low degree of concavity.

  • In or near corner solutions, the system makes no or only little use of the predictive potential of the underlying empirical model’s ability to predict crises. This can be seen in the fact that the ROC is very close to the 45-degree line there. While choosing such solutions is still optimal given the researcher’s preferences, more balanced preferences over false alarms and missed crises better exploits the underlying model’s ability to predict crises.

In-sample ROC Curve of a Logit Model

Citation: IMF Working Papers 2020, 289; 10.5089/9781513564609.001.A001

We use both approaches (loss functions and ROCs) for evaluating alternative empirical approaches’ predictive performance.

B. Machine Learning Approaches to Prediction

Machine learning techniques are well suited to predictive modeling, as they lower the variance of parameters estimated on different samples. Traditional econometric techniques seek to identify parameters within samples while avoiding estimation bias. In small samples, this comes at a cost: parameter estimates are sensitive to the estimation sample, resulting in an out-of-sample fit that is typically substantially weaker than the in-sample fit. This trade-off between parameter bias and variance is well known to any empiricist. Researchers take it into consideration implicitly when limiting the set of explanatory variables to increase the degrees of freedom and reduce the risk of overfitting. By making the trade-off explicit and addressing it squarely, machine learning techniques can nevertheless achieve often achieve more stable predictive performance. Potential drawbacks of at least some machine learning approaches are that they can be non-transparent and rely on implausible explanatory variables. The researcher can mitigate these issues to some extent by selecting suitable explanatory variables.

Ridge” and “Lasso” Approaches

The “Ridge” and “Lasso” linear methods take the logit as a starting point and “shrink” the estimated coefficients towards zero (Tibshirani, 1996). The shrinking is achieved by adding a constraint to the objective function that bounds the sum of squared coefficients (Ridge) or the sum of absolute coefficients (Lasso). If the constraint is set generously, the coefficients will be close to logit results; and if the constraint is tight, they will be close to zero. (This method of reducing the complexity of a model is called regularization.)

The rationale of these approaches relates to the trade-off between bias and variance. In the unconstrained logit, the model fits the sample on which it is estimated well, but parameters can change drastically when a different sample is used. On the other extreme, biasing all parameters to zero reduces explanatory power to that of a coin toss. An intermediate constraint would address this trade-off by limiting the in-sample fit while achieving a degree of robustness across samples.

Sometimes, a combination of Lasso and Ridge in so-called elastic nets results in an improvement of predictive quality.

Classification Trees

Classification trees sequentially split the sample into branches along the criteria that are the most informative about future crises. By adding more variables and more splits, a better in-sample fit can be achieved. Unlike logit, a tree model does not impose a linear structure on the data, giving it an advantage when non-linearities are important, as is surely the case of crisis outbreaks, given interaction between shocks with macroeconomic buffers in shaping the likelihood of a crisis. Another strength of trees is their low sensitivity to outliers.

As with linear models, classification trees can become complex and overfit the data. Predictive performance benefits from constraints on the algorithm that grows the tree. In its most constrained form, a tree has just two branches, thus using only one variable to divide the sample into high and low crisis probability cases (as in the signal extraction model).

In practice, finding the tree that achieves a good fit with a minimal degree of complexity is challenging. However, “ensembles” of trees that complement each other can be powerful. One popular method for creating ensembles is Random Forest. This method randomizes over both the samples and the variables available to the tree growing algorithm (Breiman 1996, 2001). Random Forests outperform other methods in many applications, even though the constituent trees are often severely overfitted. In other words, if the ensemble is diverse enough, it benefits from “crowd wisdom”.

Appendix I.3: Implementation Issues

A. From Binary Early Warning Signals to Three-Tiered Signals

Risks analyses may sometimes benefit from a gradation of risk estimates that is finer than the binary risk signals provided by typical early warning systems. Early warning systems typically provide a binary signal by distinguishing countries that face elevated risks from those that do not. (This is usually done by choosing a threshold crisis probability beyond which estimated probabilities are classified as crisis predictions, such that the threshold minimizes a loss function defined over shares of missed crises and false alarms.)

There are several options for providing three-tiered signals. These include:

  • With the high-risk rating determined by the threshold probability that minimizes a loss function with chosen weights on missed crises and false alarms, one can set the moderate risk rating in line with a high-risk threshold corresponding to a loss function with a higher weight on missed crises.

  • Determination of elevated risk and medium risk thresholds can also proceed by reference to probabilities perceived as meaningful by the user. For example, a crisis risk of 15 percent could serve as a high-risk threshold and a probability of 10 percent as a moderate risk threshold. (This approach would thus do away with early warning system’s typical approach of setting only one probability threshold.) To ensure that crisis warnings are operationally useful, these thresholds would need to be determined such that neither too many nor too few crises are signaled. This implies that probabilities would generally need to be set at diverse levels for different crises types, depending on the respective different crisis frequencies.

  • There are also hybrid options that rely on setting the high-risk threshold such that it minimizes a loss function over missed crises and false alarms and setting the moderate risk threshold such that it corresponds to a certain crisis probability. This approach was pursued by the VELIC.

  • The LIC DSF pursues yet a different approach involving stress tests. It identifies risk levels based on whether projected debt ratios exceed levels that have been identified as indicating a heightened risk of debt distress. If debt ratios exceed thresholds under baseline macroeconomic projections, the model classifies the country as facing a high risk of debt distress. If this is not the case but ratios exceed thresholds under alternative stress tests macroeconomic projections, countries are said to face a moderate risk of debt distress. This approach can be used only for early warning systems that comprise stress tests.

B. Aggregating Estimated Sectoral Crisis Risks

If desired, the estimated sectoral crisis risks could be aggregated into an overall rating to provide a sense of countries’ total risks. The VEA assesses that its “collection of models does not lend itself to aggregation and serves to provide a more generalized sectoral perspective on risks” (Ahuja, Syed, and Wiseman 2017). Accordingly, the VEA downplays aggregation. When required, sectors are aggregated according to a simple average or maximum of the flags produced by constituent modules. The overall level of vulnerability is taken from the distribution of the average of sectoral flags with thresholds set atthe mean and one standard deviation above the mean.

II. External Sector Crises in LICS6

This section proposes a concept and an early warning system for external sector crises in LICs. The concept relies on sharp drops in exchange rates, reserve levels, and imports. Analysis of stylized facts and econometric work suggests that external sector crises are associated with large output losses, often preceded by expansive policies, and triggered by a range of shocks, with crisis probability depending on both the size of shocks and the strength of macroeconomic buffers. While econometrics struggle to predict external crises, a machine learning technique provides reasonably accurate and stable predictions.

A. Introduction and Definition of External Sector Crises

1. External sector crises are events that involve a sharp depreciation of a currency’s exchange rate or a sharp decline in foreign exchange reserves (and possibly other factors). Such crises, also called external, currency, or balance of payments (BoP) crises, can be very destructive to welfare, as evidenced by large drops in GDP growth.

2. External sector crises occurring in emerging market economies have been discussed extensively, but two challenges arise when applying existing concepts to LICs: First, while the literature on emerging markets (and the VEE) emphasize the dynamics of short-term capital flows, such flows still play an only limited role in most LICs, with the exception perhaps of the frontier countries. Thus, models developed to understand and predict external crises in emerging markets may not be appropriate for most LICs. Second, data are much more limited for LICs than EMs, and data coverage and quality vary greatly. Any crisis definition and prediction methodology thus need to work with a smaller set of variables, be robust to data gaps, and make do with annual rather than quarterly or higher frequency data.

3. The literature relating to EMs usually defines external sector crises with respect to a sharp depreciation of a currency’s exchange rate or decline in foreign exchange reserves. Our definition for LICs adds a criterion on a drop of imports. This criterion is meant to capture crises that data on movements in reserves fail to indicate (data on reserves is patchy, particularly for earlier years and small states). The additional criterion could also be useful in situations where currency depreciations or reserve losses may not fully reflect external pressures, as may be the case in dollarized economies, currency unions, or where the reported official exchange rate deviates considerably from the parallel or interbank market rate. Imports may also decline in non-crisis circumstances, including declines in the prices of imports, the winding down of large investment projects, and domestic demand shocks. Therefore, to avoid casting the net too widely, we only consider cases of non-oil import contractions where there are additional signs of significant external pressures.

4. Specifically, we say that a country is suffering an external crisis in any given year if it meets one or more of the following three criteria:

  • i. The average official exchange rate depreciates against the U.S. dollar by at least 30 percent relative to the preceding year and the rate of depreciation increases by at least 10 percentage points relative to the preceding year. (This criterion is adopted from Laeven and Valencia, 2013.) Information on parallel market exchange rates is limited and therefore excluded.

  • ii. Reserve holdings, measured in U.S. dollars, drop by at least 30 percent relative to the preceding year and to a level below 1½ months of imports. If, in the preceding 10 years, a country has had a median reserve coverage of less than 1½ months of imports, however, this low level of reserve coverage may not be a signal of a crisis, and the country in question is excluded from application of the reserves criterion.

  • iii. The U.S. dollar value of non-oil imports falls by at least 15 percent relative to the preceding year and declines over a two-year period. In addition, at least one of the four following conditions need to be met:

    • The country experiences a substantial negative current account shock (e.g., a decline in exports or transfers) in the current or the preceding year. This shock is measured as a deterioration of the current account balance net of imports by at least 2 percentage points of GDP.

    • Reserves at the end of the current year cover less than 1½ months of imports.

    • External debt arrears grow by at least 3 percent of GDP in the current year.

    • The country enters an IMF-supported program in the current or the following year.

    At the same time, to rule out cases of orderly (unforced) rather than disorderly (crisis-related) external adjustment, we exclude cases of import compressions in which any of the above criteria are met but reserve coverage in the current year and the preceding year exceed 5 months of imports and reserve holdings increased by more than 5 percent in the current year.7

5. Whenever a country meets the above definition for two or more years in a row, we refer to this as a sustained crisis episode rather than two separate crises. Also, when two crises are separated by only one tranquil year, this year is integrated with the preceding and following years into one crisis episode.

6. The remainder of this section is structured as follows: subsection B reviews stylized facts around external crises including their relationship with the other types of crises discussed in this paper; and subsection C turns to predicting crises (using both traditional and machine learning approaches) and evaluates predictive performance. The machine learning approaches used were briefly described in Appendix I.2 above.

B. Stylized Facts

Incidence and Duration of Crises

7. External crises as defined above are moderately frequent events, whose incidence has fallen sharply since the turn of the century. On average, LICs moved into an external crisis once every 13 years and spent 16 percent of their time in crises.8 The incidence dropped sharply around the turn of the century. During the 1980s and 1990s, difficult and turbulent decades for LICs, countries were in external crises some 21 percent of the time, and thereafter only 9 percent of the time. A drop in the incidence of sharp depreciations contributed most to the improvement seen in the 2000s, followed by fewer cases of reserve depletion. The driving forces underlying this shift remain a subject for future research.

8. The incidence of crises varies greatly across country groups (Text Table below): frontier markets have spent a limited amount of time in external sector crises, while fragile states have seen significantly more such crises than the average LIC. Commodity exporters, particularly fuel exporters, have also spent a larger share of the time in crises than other countries (many of them are fragile countries).9

9. The typical external crisis is short-lived. Nearly two thirds of crisis episodes last only one year, and most crises are over after three years. Note however that economic impacts may linger after the crisis has come to an end. For example, the recession that often follows external crises typically lasts longer than the crisis episode itself (see below).

Share of Years Spent in Crisis by Type of Countries

(In percent)

article image
Note: Country classification based on most recent groupings. Since the status of frontier markets and fragile states has changed markedly over time, summary statistics for these two groups are not calculated for the pre-2000 period.

Dynamics of Macro-Financial Variables Around External Crises

10. Using a simple linear regression to describe the behavior of macro-financial variables around external crises,10 we find that the run-up to external crises is often marked by expansionary policies:

  • There is a significant deterioration in the fiscal stance, as revenue declines while the spending remains broadly unchanged. In line with the fiscal expansion and indications of real exchange rate overvaluation, the current account balance weakens.

  • Monetary and financial sector policies are expansive as well. Domestic credit as a share of GDP is substantially higher than during tranquil times, and net credit to the public sector is rising. As a result, the money supply grows and inflation rises.

  • These expansionary policies do not coincide with, or lead to, higher real growth. On the contrary, growth is below average and falling.

11. Once the crisis erupts, economic performance deteriorates sharply and typically takes several years to recover, while external debt rises to a sustained higher level:

  • Growth is severely hit by an external crisis and recovers to pre-crisis levels only over several years. Comparison of GDP growth during the year of the crisis and the two following years with growth in quiet times suggests that at the end of this period, output is about 7 percent lower than it would have been in the absence of a crisis. This said, as noted earlier, growth tends to slow already prior to the crisis. There may thus be feedback between growth and crises: slowing growth may contribute to the outbreak of crises, including perhaps by motivating governments to pursue imprudent policies; and in turn, crises slow growth.

  • The fiscal balance deteriorates sharply after the onset of a crisis, as GDP growth and fiscal revenue contract. Subsequently, the fiscal position improves gradually, following some expenditure tightening and gradually recovering revenue and growth.

  • Monetary typically tightens once a crisis breaks, with credit and money supply on a declining trend. However, credit to the public sector keeps rising to help finance the deteriorated fiscal balance, suggesting that credit to the private sector is being squeezed. Inflation spikes following the onset of external crises, but only for a brief time, allowing the real exchange rate to fall and converge to equilibrium.

  • The current account improves gradually after the crisis outbreak, at least initially mainly because of import compression. Less favorably, external debt rises to a sustained higher level than before the crisis, reflecting both exchange rate depreciation and additional crisis-related borrowing.

12. There are substantial differences between country groups regarding the behavior of key variables around external crises. Fuel-exporting LICs suffer the most. Growth drops by 6 percentage points into negative territory, inflation jumps to high levels, and fiscal deficits more than double. And countries with fixed exchange rate regimes suffer larger impacts than countries with flexible regimes.

13. External crises are almost always triggered by shocks interacting with domestic macroeconomic weaknesses. Where domestic buffers are weak, even small shocks can trigger crises.

  • The larger is a shock, the higher is the likelihood that a country with a given strength of preexisting fundamentals/buffers experiences a crisis. For example, considering countries with intermediate buffers, 18 percent of countries experiencing a large shock fell into crisis in the same or the next year, while only 10 percent of countries hit by a small shock did.

  • The stronger a country’s fundamentals are, the better it can withstand shocks of a given size. For example, for countries hit by large shocks, those with strong buffers fell into a crisis into a crisis only 6 percent of the time while countries with weak buffers did so in 33 percent of the cases.

C. Predicting Crises

14. We explore a range of empirical specifications, including both traditional econometrics and machine learning approaches. The latter help address the challenges of model selection and provide systematic approaches to dealing with overfitting, which undermines out-of-sample predictive performance.

Traditional Econometrics

15. We estimate a simple logit model of the probability of an external crisis (see Appendix II.1 for detail). Explanatory variables include both global and country-specific variables, chosen based on economic theory, stylized facts, and the experience of external crises in emerging markets. Variables enter in a linear fashion without interactions. Given the highly non-linear nature of crises, such a regression can only be a start.

16. Results broadly confirm the findings of the stylized facts investigation above, while also suggesting a role for global factors.

  • Most of the country-specific macro-economic indicators are significant across specifications, and with the expected signs: a weaker fiscal balance, higher inflation, higher or overvalued real exchange rate, low levels of foreign exchange reserves, and deteriorating terms of trade are all associated with a higher crisis probability.

  • Among the global variables, the VIX index is statistically significant across all specifications, with the expected sign, while world real GDP growth, 10-year US interest rates, global liquidity and changes in commodity prices are not significant. These findings suggest that global financial conditions/perceptions of uncertainty have some impact on the likelihood of LICs moving into an external crisis (while global commodity prices has no impact beyond that working through terms of trade.

17. Adding emerging market countries to the sample improves the fit for LICs but also substantially changes the coefficient estimates, suggesting that determinants of external crises in LICs differ noticeably from those in emerging markets. Only the VIX and the change in the terms of trade survive as significant variables.

18. Regression analysis confirms and further quantifies the finding that the probability of moving into an external crisis is shaped by the interaction of shocks and macroeconomic buffers (Appendix II.2).

Performance in Predicting Crises

19. The logit regressions’ out-of-sample prediction performance is weak. Crisis probabilities were estimated for a test sample spanning the period 1980–2009, using the baseline regression. Threshold probabilities were determined such that they minimize a loss function over the sum of the shares of missed crises and false alarms with equal weights. Over the forecasting period 2010–16, the resulting rate of false alarms was a low 15 percent but the share of missed a very high at 78 percent. Adding EMs to the sample lowered the rate of missed LIC crises enhanced predictive accuracy only to a limited extent, as did varying the weights in the loss function. A signal extraction approach to predicting crises (Kaminski and Reinhart, 1999) was tried as well and found to deliver even weaker results.

Machine Learning Techniques

20. Applying Lasso and Ridge approaches to external crises in LICs, we start with an initial set of 16 candidate explanatory variables, while also investigating predictive performance resulting from a larger set of variables. We also determine the optimal tightness parameter. Assessment of predictive performance will be provided below.

21. We also grow a Random Forest using the same sets of explanatory variables as for the Lasso and Ridge approaches. Assessment of predictive performance will be provided next.

22. As done above using the traditional econometrics, we separate predictions into high and low crisis probability groups and assess predictive performance. To obtain crisis predictions from the continuous probabilities, we again find the threshold probability that minimizes a loss function with equal weights on missed crises and false alarms. Prediction performance can be assessed using the shares of missed crises and false alarms, as well as the value of the loss function. As any loss function is a subjective one, we also measure predictive performance using the (preference-free) area under the Receiver Operating Characteristic Curve (a larger area indicates better ability of the empirical method’s ability to extract useful information from the data).

23. Machine learning techniques show predictive performance that is somewhat better on average and much more consistent than that of traditional approaches when using data exclusively on LICs.

  • Across samples, the predictive performance of logit models is weak on average and volatile. Machine-learning models have somewhat better performance, but only Random Forest provides a combination of missed crises and false alarms that approaches acceptable levels (30 percent share of missed crises and 50 percent share of missed alarms). Random Forests also provides the highest AUC. Further, machine learning approaches achieve much more stable performance across different samples (as seen in their much lower variability of binomial deviance and the AUC compared with logit). Low variability implies a high likelihood that predictive performance for a new sample will be close to those determined for past samples.

  • To probe predictive quality further, we divide the sample into a training sample spanning the more distant past (1990–2009) and a test sample covering more recent years (2010–15).11 Results highlight the main advantage of machine learning: the robustness of models across samples, as seen in much lower variability than in logit regressions. As a result, the test sample performance is close to the training sample performance. Even so, the level of performance is not great.

24. Prediction benefits from drawing on the experience of emerging market economies. Logit estimations and machine learning techniques were applied to pooled data covering both LICs and EMs. Across methodologies, the additional training data help lower variability of results, implying a higher likelihood for out-of-sample prediction performance to resemble in-sample performance. In addition, there is improvement in mean prediction performance for some specifications. Across performance measures, Random Forest is the preferred methodology. For the prediction sample 2010–15, this method misses only 25 percent of crises and generates 39 percent of false alarms (in the approach using the initial limited range of 16 variables).

Appendix II.1. Logit Regressions for Probability of Moving into External Crises

We run a simple logit model of the probability of an external crisis.12 Right-hand side variables enter in a linear fashion without interactions. Given the likely highly non-linear nature of crises, such a regression can only be a start.

Explanatory variables include both global and country-specific variables, chosen based on economic theory, stylized facts, and the experience of external crises in emerging markets:

  • Global variables include world real GDP growth, the VIX Index, a global liquidity indicator, the ten-year U.S. interest rate, and the year-to-year changes in the global commodity price index.13

  • Country-specific variables include variables representing the real, fiscal, external, financial and institutional sectors. Real sector indicators: real GDP growth and inflation. Fiscal indicators: overall fiscal balance and government grants. External sector indicators: long-term external debt level, short-term external debt service, a measure of real effective exchange rate misalignment,14 remittances, changes in the terms of trade, and foreign exchange reserve adequacy indicators (ratio between short-term external debt and foreign exchange reserves, ratio between M2 and foreign exchange reserves and the ratio between foreign exchange reserves and imports). Financial sector indicators: total credit to the economy. Institutional sector indicators: the public-sector management and institutions cluster of the World Bank’s Country Policy and Institutional Assessment.

Restrictions were imposed to address endogeneity and ensure focus on transitions into crises. To address endogeneity, all independent variables were lagged by at least one period. To focus on only those observations where a country moves from tranquil times into a crisis, we require that the lagged value of the crisis indicator be zero.

Results broadly confirm the findings of the stylized facts investigation, while also suggesting a role for global factors. A baseline regression was identified by starting with a general specification and gradually removing insignificant variables except for those that economic theory or empirical studies for EMs suggest are highly likely to play a role in predicting external crises in LICs, such as domestic GDP growth, real exchange rate levels, and changes in the terms of trade (Table A.II.1.1, column “Baseline”). The regression was then subjected to robustness checks by adding more global and country-specific indicators (Table A.II.1.1, other columns):

  • Most of the country-specific macro-economic indicators are significant across specifications, and with the expected signs: a weaker fiscal balance, higher inflation, higher or overvalued real exchange rate, low levels of foreign exchange reserves, and deteriorating terms of trade are all associated with a higher crisis probability.

  • Among the global variables, the VIX index is statistically significant across all specifications, with the expected sign, while world real GDP growth, 10-year US interest rates, global liquidity and changes in commodity prices are not significant. These findings suggest that global financial conditions/perceptions of uncertainty have some impact on the likelihood of LICs moving into an external crisis (while global commodity prices has no impact beyond that working through terms of trade).

Table A.II.1.1.

Logit Regressions, LICs, 1980–2016

article image
*** p<0.01, ** p<0.05, * p<0.1

Adding emerging market countries to the sample improves the fit for LICs but also substantially changes the coefficient estimates, suggesting that determinants of external crises in LICs differ from those in emerging markets. Only the VIX and the change in the terms of trade survive as significant variables.

Appendix II.2: Quantification of Interactions between Shocks and Fundamentals

Logit regressions were estimated to quantify the extent to which the impact of shocks depends on macroeconomic fundamentals.15,16 By repeating the logit regressions on different subsamples, we confirm that for a given shock size, countries with strong buffers are less likely to experience an external crisis than countries with weak ones (Table A.II.2.1).17 And the importance of buffers increases with the size of the shock.18 For example, for countries with weak fundamentals, both small and large shocks significantly increase external crisis probability; while for countries with strong fundamentals, only large shocks have a statistically significant impact on crisis probability (Table A.II.2.2).

Table II.2.1.

Impact of Macroeconomic Fundamentals on Crisis Probability Given Size of Shocks, 1980–2016

article image

*** p<0.01, ** p<0.05, * p<0.1

Table II.2.2.

Impact of Shocks on Crisis Probability Given Macroeconomic Fundamentals, 1980–2016

article image
*** p<0.01, ** p<0.05, * p<0.1

Appendix II.3: Machine Learning – Performance in Predicting Crises

Machine learning techniques show predictive performance that is somewhat better on average and much more consistent than that of traditional approaches when using data exclusively on LICs.

  • Table A.II.3.1a shows findings on predictive performance. The first column shows the binomial deviance of the crisis probabilities predicted by the models studied so far;1 the three middle columns show the performance of the resulting crisis predictions (ones or zeroes), and the last column shows the AUC. Standard deviations of binomial deviances and the AUC across 100 different samples (cross-validation with N=20 and 5 resamples each) are reported in parentheses. 2 Across samples, the predictive performance of logit models is weak on average and volatile. Machine-learning models have somewhat better performance, but only Random Forest provides a combination of missed crises and false alarms that approaches acceptable levels (30 percent share of missed crises and 50 percent share of false alarms). Random Forests also provides the highest AUC. Further, machine learning approaches achieve much more stable performance across different samples (as seen in their much lower variability of binomial deviance and the AUC compared with logit). Low variability implies a high likelihood that predictive performance for a new sample will be close to those determined for past samples.

  • To probe predictive quality further, we divide the sample into a training sample spanning the more distant past (1990–2009) and a test sample covering more recent years (2010–15).3 Out-of-sample prediction performance is reported in Table A.II.3.1b. Results highlight the main advantage of machine learning: the robustness of models across samples, as can be seen from the lower variability. As a result, the test sample performance is close to the training sample performance. Even so, the level of performance is not great. In fact, the test-sample AUC is highest in the logit with fewer variables.

Table A.II.3.1.

Predictive Performance Using Training Data for LICs Only

article image

Performance improves from drawing on the experience of emerging market economies, resulting in prediction showing acceptable consistency and accuracy. Logit estimations and machine learning techniques were applied to pooled data covering both LICs and EMs. Table A.II.3.2 shows the performance of predictions for LICs only. Across methodologies, the additional training data help lower variability of results, implying a higher likelihood for out-of-sample prediction performance to resemble in-sample performance. In addition, there is improvement in mean prediction performance for some specifications. Across performance measures, Random Forest is the preferred methodology. For the prediction sample 2010–15, this method misses only 25 percent of crises and generates 39 percent of false alarms (version using a limited set of variables).

Table A.II.3.2.

Predictive Performance Using Training Data for Both LICs and EMs

article image

III. Inflation Crises in LICS4

This section proposes a concept of, and an early warning system for, inflation crises in LICs. We define such crises as inflation exceeding 15 percent per year. Analysis of stylized facts confirms the literature’s finding that high inflation lowers growth. Analysis of stylized facts and econometric work further suggest that high inflation in LICs often results from expansive fiscal and monetary policies, increases in world fuel and food prices, and depreciation of the domestic currency. After assessing that traditional econometrics struggle to predict inflation crises, we find that machine learning techniques provide fairly accurate predictions.

A. Introduction and Definition of Inflation Crises

1. Research for countries at various levels of income finds that high and variable inflation has negative macroeconomic and social consequences.

  • It increases the cost of holding liquid assets that facilitate transactions; it increases uncertainty, thereby reducing investment and growth;5 and it may cause financial sector strains whenever the values of assets and liabilities respond differently to inflation.

  • In fixed exchange rate regimes, it erodes price competitiveness, contributing to external sector crises.

  • It hurts the poor in particular, who often lack access to inflation hedges.6 Surveys of attitudes towards inflation show that the poor are more concerned about it, which is consistent with inflation being negatively correlated with improvements in their well-being (Easterly and Fischer, 2001). In LICs, poor people’s vulnerability to inflation can be very high given that their incomes barely cover basic needs.

2. We propose a definition of inflation crises/episodes that is pitched at an intermediate degree of severity. In a seminal paper, Khan and Senhadji (2001) found that inflation higher than 12 percent per year has a negative impact on growth among developing countries.7 While the identification of a threshold above which inflation becomes particularly harmful depends on estimation methods, sample and modeling choices, Khan and Senhadji’s result is widely cited and arguably matches the views of policy-makers. Aiming at capturing risks that are neither trivial nor in the extreme tails of distributions, we define inflation crises/episodes as inflation rates exceeding 15 percent.

3. The section is structured as follows: Subsection B reviews the stylized facts on inflation and inflation crises in LICs. Subsection C turns to predicting inflation crises. In a first step towards this, we present a panel regression-based forecasting model of inflation. Then we develop and compare several possible early warning systems for inflation crises based on country team forecasts, the panel regression and others on machine learning approaches. The “Random forest” machine learning technique performs best, providing strong out-of-sample prediction.

B. Stylized Facts

4. LICs have on average higher inflation than advanced and emerging economies. This does not necessarily imply poor policies or inefficiencies, as the Balassa-Samuelson effect ensures that higher growth in low income countries implies a trend real exchange rate appreciation relative to advanced economies (e.g., Edwards and Savastano, 2001). For low income countries with fixed exchange rates, this means higher inflation than in advanced economies. However, persistent large differentials in inflation cannot be explained by Balassa-Samuelson mechanisms. These are best explained by weaknesses in monetary and fiscal frameworks that ought to be remedied.

5. Over recent decades, inflation in the typical LIC has followed a downward trend. Median inflation across LICs sharply fell during the 1990s and early 2000s, in part thanks due to the gradual disinflation and stabilization in transition economies. LIC inflation trends did not differ significantly across regions except for the higher inflation levels in transition countries in the 1990s. They also did not differ much across classifications of LICs in terms of export structures. LIC inflation spiked in 2008 due mainly to higher world food prices (Habermeier et al., 2009). In recent years, inflation has fallen further, in line with lower global inflation. More recently, however, some countries (e.g., Ghana, Nigeria and Zambia) that experienced exchange rate depreciation in the wake of the 2014 commodity price decline have seen inflation pressures.

LIDCs Inflation 1995–2016 median and interquartile range

Citation: IMF Working Papers 2020, 289; 10.5089/9781513564609.001.A001

6. Inflation crises/episodes have become less frequent. Using the definition described above, there were 255 inflation crisis episodes out of 1264 country-year pairs (25 percent of the sample) during the period 1990–2015, with a trend reduction in the incidence of inflation crises/episodes: during the 1990s, there were inflation crises in 37 percent of the country-year pairs; since 2000, this incidence has dropped to 12 percent.8

Proportion of LICs with inflation crisis

Citation: IMF Working Papers 2020, 289; 10.5089/9781513564609.001.A001

7. Using a simple linear regression to describe the behavior of macroeconomic variables around crises, we find that fiscal pressures, loose monetary policies, depreciation of the domestic currency, and increases in world fuel and food prices precede inflation crises:

  • Fiscal crises (as defined in Gerling et al., 2017) are 10 percentage points more likely to occur three years before an inflation episode than in other times (Figure 1, upper left). The intuition is that high fiscal deficits drive monetary financing.

  • Broad money and credit to the private sector grow faster prior to inflation episodes than in tranquil times (Figure 1, upper right and second row right).9

  • Periods of strong real effective exchange rates, which may result in overvaluation, tend to precede inflation episodes (Figure 1, second row left).

  • Quarters with inflation crises/episodes are associated with nominal depreciations (Figure 1, third row left).

  • Oil and food price increases tend to precede years with inflation episodes (Figure 1, bottom row).

8. Confirming the insights of the literature, we find that inflation crises are associated with a permanent output loss. A comparison of GDP growth during the year of the crisis and the two following years with growth in quiet times suggests that at the end of this period, output is about 2½ percent below the level that would have prevailed otherwise.

C. Predicting Crises

A Panel Regression-Based Model for Forecasting Inflation in LICs

9. The model forecasts both inflation and inflation variance in the subsequent year, with inflation a reduced form function of plausible explanatory factors. The choice of explanatory variables reflects theoretical considerations, data availability, and the aim to avoid overfitting. Appendix III.1 provides detail.

10. Estimation results for next year’s inflation broadly conform to priors from economic theory. In our preferred specification, as concerns the domestic variables, we find that inflation in the following year tends to be higher when current inflation, money growth, and exchange rate depreciation are higher; and when reserve coverage is lower. As concerns external and structural variables, we find that inflation in the following year tends to be higher when global food and fuel price inflation is higher, and when a country suffers under droughts and political instability. Estimates for inflation variance also conform to priors, with variance a monotonically rising function of inflation levels.

11. Comparison of panel regression-based forecasts with IMF desk economists’ forecasts shows that desk projections are somewhat more precise. Comparing forecasts over the period 2006–16 using the median absolute deviation of forecasts from outturns, we find that desk (“WEO”) forecasts are more precise (with an average median absolute deviation of about 3 percentage points for the model and 2.6 points for the WEO forecasts). This may reflect in part desks’ use of additional country-specific information.

Early Warning Systems

12. We now describe a suite of possible early warning systems for inflation crises in LICs and assess their out-of-sample prediction performance (Table III.1). The systems build alternatively on the regression-based model presented above, desk forecasts, a logit regression for inflation crises, and two machine learning approaches (the Lasso model and the Random Forest classification approach). Indicated out-of-sample prediction performance refers to the period 2008–16 conditional on data available at the time the forecast would be made. For instance, to forecast inflation crises in 2008, we estimate the models based on data available in late summer 2007 (and feeding into the October 2007 WEO), including the desk inflation forecast to bring to bear all relevant available information.10 All systems use the same data and sample as the panel regression presented above.

  • A “naïve” desk forecast-based early warning system performs poorly. This approach issues signals when desks forecast inflation above 15 percent in the subsequent year. This approach misses most inflation crises (79.5 percent) while sending few false alarms (2.6 percent), reaching a loss function value of -0.345.

  • A more sophisticated desk forecast-based approach that takes account of forecast errors performs much better. The root mean square error of WEO forecasts can be estimated from the difference between observed and forecast inflation. If we assume that the forecast error has a normal distribution with a standard deviation equal to this root mean square error, we can estimate the implied probability of an inflation crisis given the WEO forecast. The loss minimizing threshold is a 16 percent probability that there will be an inflation crisis in the next year. This yielded a missed crises rate of 7 percent and a false alarm rate of 38.7 percent, with a value of the loss function of -0.809.

  • The regression-based early warning system shows intermediate predictive performance. Based on a comparatively low loss-minimizing cutoff probability of 5 percent that there will be an inflation crisis/episode next year, it yields a missed crises rate of 5.3 percent and a false alarm rate of 54.2 percent. The realized value of the loss function is -0.743.

  • Logit performs relatively poorly. Once the WEO inflation forecast is included, the only remaining variable that helps forecast inflation crises/episodes is domestic GDP growth. The loss minimizing cutoff probability is 19 percent, which yielded a missed crises rate of 54.1 percent and a rate of false alarms of 11.5 percent. This resulted in a realized value of loss function of -0.572.

  • Lasso performs quite well. The loss minimizing cutoff probability for the Lasso specification is 8 percent, which yielded a missed crises rate of 26.8 percent and a rate of false alarms of 14.4 percent. This results in a realized value of the loss function of -0.771, ranking between the regression model and the sophisticated WEO.

  • The Random Forests classification model outperforms all other approaches and is the recommended basis for an early warning system for inflation crises in LICs. The loss minimizing cutoff probability is 29 percent, which yielded a missed crises rate of 3.6 percent and a rate of false alarms of 34.8 percent, and at -0.846 the lowest value of the loss function among all early warning systems under consideration. This is not surprising as the random forest model also shows the highest area under the ROC curve. The superior performance of the Random Forests classification model is apparent also in out-of-sample forecasts using data available in late summer 2009 and covering the period 2010–17, with rates of missed crises and false alarms of less than 15 percent each.

Table III.1.

Predictive Performance of Various Early Warning Systems

article image
Note: All results presented above are based on out-of-sample forecasts, except for the calculation of the model specific cutoff rates which are based on a 2008–2014 training sample.I. “Naïve WEO” stands for an early warning system whereby alarms are raised when the Fall WEO inflation forecast for t+1 is above 15 percent.II. “Sophisticated WEO” assumes that the WEO forecast errors are normally distributed.III. “Panel Regression Model” is the forecast regression described above and in Appendix III.1.IV. “Logit” is a binary regression model to predict inflation crises.V. “Lasso” refers to Least Absolute Shrinkage and Selection Operator.VI. “Random forest” stands for the random forest classification model in Breiman (2001).

Appendix III.1. A Panel Regression-Based Model for Forecasting Inflation in LICs

The model forecasts both inflation and inflation variance in the subsequent year, with inflation a reduced form function of plausible explanatory factors. The choice of specification reflects the fact that the workhorse Phillips curve inflation forecast model commonly used for advanced and emerging countries is not suitable for LICs, including because it is difficult to distinguish between trend and cycle in LICs, and because of a lack of timely data on output and unemployment gaps.

The choice of explanatory variables, grouped into external and domestic factors, reflects theoretical considerations, data availability, and the aim to avoid overfitting. The model is therefore relatively parsimonious. (The model uses data available in late summer; small modifications would allow using data available at any other time of the year.)

Domestic macroeconomic factors are chosen such as to avoid including variables that are themselves functions of inflation forecasts, as we wish to benchmark the model’s forecasts against desk forecasts. Our preferred specification includes the following domestic factors:

  • Current inflation carry-over (calculated from the IFS variable PCPI_IX). Specifically, we consider the carry-over of inflation up to June of the present year in our forecasts for inflation in the next calendar year.

  • Current inflation interacted with dummy for currency union: It Is expected that inflation persistence is lessened in currency unions that maintain a peg to an anchor currency.

  • Current money growth (WEO variable bmf).

  • Current exchange rate depreciation and its carry-over.

  • Current reserve coverage (WEO variable iar_bmgs): Low reserve coverage is often a harbinger of exchange rate depreciation, making it a leading indicator also for inflation increases.

  • Current GDP growth (calculated from WEO variable ngdp_r): The effect of GDP growth on inflation could be positive or negative depending on what type of shock drives growth.

External and structural explanatory factors are included contemporaneously without concerns of endogeneity because they are not affected by a LIC’s own inflation. Our preferred specification includes the following external factors:

  • WEO projections for world market price of food commodities in dollars (GAS variable food).

  • WEO projections for world market oil prices in dollars (GAS variable crude) interacted with dummy for oil importer.

  • Incidence of droughts.

  • Political instability.

We also consider interaction terms to consider non-linearities and try to control for structural breaks. Specifically, we include interaction terms between current inflation, money growth and exchange rate depreciation. In general, those interaction terms are significant. We also attempt to account for structural breaks associated with exit from hyperinflation, using dummy variables.

We obtain the following system of two simultaneous equations, estimated using feasible generalized least squares (FGLS):

cpii,t+1=αi+β0cpii,tWEO+γ1foodt+1+γ2(oilt+1×oilImporter)+β1a×πcarryJune,tt+β1b×πcarryJune,tt+1+β1cucpii,tWEO×currencyUnioni+β2DeprcarryAug,tt+1+β3moneyi,t+β4gdpi,t+β5reservesi,t+β6droughti,t+β7polInstabilityi,t+λ1×πcarryJune,tt×DeprcarryAug,tt+1+λ2×πcarryJune,tt×moneyi,t+λ3×DeprcarryAug,tt+1×moneyi,t+ΣiωiI(ti>Ti)+ϵi,t+1(2)
E[ϵi,t+12]=Φ(Infli,t)(3)

where cpii,tWEO is the current year forecast for inflation in the WEO, πcarryJune,ttandπcarryJune,tt+1 are the inflation carry-over from the June price level for the present year and one year ahead, food is the change in the commodity price index in dollars, oil is the change in oil price index in dollars, oilImporter is a dummy for a country’s status as oil importer,1 DeprcarryAug,tt+1 is the depreciation carry-over for the next year based on the level of the exchange rate in August of the present year, money is the rate of growth in a broad monetary aggregate, gdp is either a measure of output gap or the growth rate of real GDP, reserves is the reserve coverage measured as months of imports, the □ are coefficients on the interactions between inflation, depreciation and money growth, I is an indicator function equal to 1, Ti marks the end of hyperinflation spells, and ε is an error term. Equation (2), the “level equation” provides the forecast of the inflation level, and equation (3), the “conditional variance equation,” provides an estimate of the inflation forecast squared error (which is also the variance of inflation if equation (2) correctly describes the determinants of inflation) as a non-parametric function of the current level of inflation. Thus, the preferred specification allows for non-constant forecast errors (“conditional heteroscedasticity”).

Estimation results for next year’s inflation broadly conform to priors from economic theory (Table A.III.1.1). In our preferred specification, as concerns the domestic variables, we find that inflation in the following year tends to be higher when current inflation, money growth, and exchange rate depreciation are higher; and when reserve coverage is lower. As concerns external and structural variables, we find that inflation in the following year tends to be higher when global food and fuel price inflation is higher, and when a country suffers under droughts and political instability.

Table A.III.1.1.

Results of a Panel Regression of Inflation on Domestic and External Variables

Column I and II estimates assume a constant variance of inflation; column III refers to the heteroscedastic model described in equations (2) and (3). This is our preferred specification.

article image
Note: ‘+’ denotes that inflation forecast is positively associated with this variable at the 95 percent confidence level; ‘-’ denotes negative association, “0” denotes this variable is not a statistically significant forecaster for future inflation.

Estimates for inflation variance also conform to priors, with variance a monotonically rising function of inflation levels (Figure A.III.1.1).

Figure A.III.1.1:
Figure A.III.1.1:

Estimate of the Inflation Forecast Standard Error

Citation: IMF Working Papers 2020, 289; 10.5089/9781513564609.001.A001

IV. Financial Sector Crises and Stress Episodes in LICS2

This section describes a concept of and early warning system for financial sector crises in LICs. The concept, proposed by Laeven and Valencia (2013), involves significant signs of systemic financial distress and policy intervention. Analysis of stylized facts does not find evidence for adverse growth impacts of financial sector crises but suggests that such crises may contribute to rising public debt. Econometric analysis provides satisfactory in-sample prediction of financial sector crises. As there have been very few full-blown financial sector crises in LICs in recent years, we also propose a method for predicting less severe financial sector stress.

A. Introduction and Definition of Financial Sector Crises and Stress

1. In a setting of overall financial sector stability, several low-income countries (LICs) have seen financial sector stress in recent years. Only a handful of LICs have experienced full-fledged banking crises over the past two decades, in sharp contrast to pronounced financial sector instability in the 1980s and 1990s. Several factors likely contributed to this overall benign state of affairs, including an extended period of sustained growth and favorable external conditions, including high and stable commodity prices. Since 2014, however, several LICs have seen financial stress emerge (IMF 2017).

2. Banking crises are often the result of external shocks interacting with domestic vulnerabilities (Demirguc-Kunt and Detragiache 1998; Kaminsky and Reinhart 1999; Reinhart and Rogoff 2014). Specifically, a sharp and persistent decline in commodity prices is a common element behind recent episodes of financial stress in LICs. For example, the decline in copper prices during the global financial crisis, following on a credit boom and deteriorating capital adequacy levels and asset quality, contributed to a banking crisis in Mongolia in 2008–09 (see IMF 2009 and IMF 2010).

3. This section develops a two-tiered concept of financial sector crises/stress episodes and an early warning system (EWS) that flags the risks of such episodes occurring. The crisis concept comprises both severe banking sector crises and less severe stress episodes. The effort to predict financial sector stress rather than full-blown banking crises was made necessary by the rarity of full-blown crises in recent years.

4. The analysis of the determinants of banking crises finds that both domestic and external factors play important roles. The analysis covers 60 LICs over 1981–2015. Regarding domestic factors, we find that banking crises occur more frequently in countries that have elevated inflation, high public debt, low reserve coverage, factors that may capture the adverse impact on financial sector stability of weak policies and the resulting weak macro fundamentals/buffers more generally. Regarding external factors, banking crises tend to follow on deterioration of the commodity terms of trade. Finally, there is evidence that financial sector crises are more likely in periods of tight global monetary conditions.

5. The analysis of the determinants of less severe financial sector stress identifies FSIs relating to asset quality and bank profitability as important leading indicators. This analysis compares FSIs over 2011–16 between the ten countries identified by desks as financially stressed in 2016 and other LICs. Descriptive analysis, statistical tests, and noise-to-signal computations indicate that 3 FSIs act as leading indicators of financial stress: the ratio of non-performing loans to total gross loans, return on equity, and return on assets.

6. We posit that a LIC’s financial sector experiences a crisis/stress episode when it is undergoing either a severe crisis in the definition of Leaven and Valencia (2013) or less severe stress episode as assessed by IMF country teams:

  • Severe crisis: the banking sector crisis definition of Laeven and Valencia (2013), which focuses on systemic events with significant signs of financial distress and policy interventions (detail in section III).

  • Less severe stress: IMF desks assess a country as presently experiencing financial sector stress, taking account of a range of financial soundness indicators (FSIs).

7. Both the crisis and stress components comprise judgment and are not fully quantifiable, in contrast to other crisis definitions such as balance of payments crises. However, this component is important to ensure that the early warning system can flag stress situations that do not (or not yet) amount to systemic banking sector crises, enabling policymakers to take corrective action.

8. This section is structured as follows: Subsection B presents the stylized facts around financial sector crises, subsection C turns to predicting such crises; and subsection D to predicting financial sector stress.

B. Stylized Facts of Financial Sector Crises

9. The sample includes 60 LICs over the period 1981–2015. A total of 43 banking crises took place in this sample, but due to data availability for the control variables our regressions only capture 35 of these. After cleaning the data for outliers and missing values, we obtain a sample of 1,588 observations, with an average period covered of 26.5 years per country.

10. We use three main sources of data on banking crises and their likely drivers.

  • First, we use an update of the Laeven and Valencia (2013) database to identify banking sector crises. This dataset identifies systemic banking crises when either of the following conditions is met: (a) significant signs of financial distress in the banking system—as indicated by significant bank runs, losses in the banking system, and/or bank liquidations; and/or (b) significant banking policy intervention measures in response losses in the banking system.

  • Second, to capture the role of commodity prices on the occurrence of banking crises we use monthly primary commodity price data from the IMF for 44 fuel and non-fuel commodities. We construct a country-specific aggregate commodity price growth index and a measure of the volatility of this index.

  • Third, informed by the existing literature on banking crises3, we collate a set of possible drivers and control variables organized into categories of: (i) macroeconomic fundamentals (GDP growth, growth of real credit/GDP, inflation, public debt/GDP, short-term debt to total external debt, net capital inflows/GDP, M2/reserves), (ii) measures of banking system structure (leverage, liquidity, size, deposit insurance), (iii) a global economic indicator (the 10-year US Treasury Constant Maturity Rate), and (iv) a set of indicators for periods of currency and fiscal crises and armed conflicts.

11. Banking crises in low-income countries were primarily a feature of the 1980s and 1990s. Since then, only two countries (Nigeria and Mongolia) out of 60 LICs experienced a banking crisis during the recent Global Financial Crisis (GFC). It is also notable that 39 of the 43 LIC crises took place during a narrow 15 year-window between 1982 and 1996—an average of almost three crises per annum. Financial crises in LICs frequently last several years, with the median crisis in our set of countries lasting two years.4

12. In contrast, many high-income countries suffered banking crises as part of the GFC, while only few suffered banking crises in the 1980s or 1990s. Out of 35 high-income countries in the Laeven and Valencia (2013) dataset, 19 suffered banking crises as part of the GFC (2007–08), and only 12 (half of which were transition economies) experienced crisis events in the 1980s or 1990s.

13. Banking crises in advanced and emerging market countries have large and long-lasting output costs, and there is a concern that costs could be substantial in LICs as well, particularly as their banking sectors grow. A large body of evidence—mostly based on advanced and emerging economies—indicates that financial crises have more severe consequences than other recessions () and that the disruption of credit intermediation that follows a banking crisis leads to a sharp contraction of real GDP, a significant increase in unemployment, and large fiscal costs (). Focusing on developing countries, Reinhart and Rogoff (2013) notice that GDP growth shows a steeper decline during and immediately after a banking crisis but a somewhat faster comeback in emerging markets than in the advanced economies. In addition, as found by Dell’Ariccia et al. 2008 (p. 100), “banking crises are likely to have relatively larger real effects in developing countries where bond and equity markets are less developed and where governments may find it more difficult to provide support for troubled banks.”

14. An analysis of the path of key variables around the onset of banking crises highlights the specificity of banking crises in low-income countries. We use the approach of Gourinchas and Obstfeld 2012 and Anundsen et al. 2016 to shed light on variable behavior around banking crises. Commodity terms of trade emerge is a key indicator, while some of the crisis predictors that have been found to matter greatly in the advanced economy context—such as high credit and GDP growth—do not seem to matter in low-income countries. Specifically:

  • Commodity terms of trade fall in the lead-up to crises, and the volatility of commodity terms of trade is higher than in tranquil periods as well.

  • Credit growth is depressed prior to a crisis, in contrast to the credit booms that have been identified as key drivers of banking crises in advanced countries (Kaminsky and Reinhart 1999; Jorda et al. 2011; Jorda et al. 2015; Schularick and Taylor 2012). At the same time, M2 relative to foreign exchange reserves is high, confirming that the extent to which liabilities of the banking sector are backed by international reserves is a leading indicator of banking crises. (Kaminsky and Reinhart 1999).

  • GDP growth is lower than in tranquil times prior to crisis events, in line with falling commodity prices and low credit growth, but there is no evidence for financial crises lowering growth. Rather, growth recovers fairly quickly after crisis onset. This pattern contrasts with that observed in more advanced countries, where GDP growth tends to be high prior to crises, typically driven by high credit growth, and where growth tends to fall sharply after crisis onset, remaining depressed for extended periods (see, e.g., Cerra and Saxena 2008; Cecchetti et al. 2009, Hoggarth et al. 2002; Hutchinson and Noy 2005; Jorda et al. 2013, Reinhart and Rogoff 2013).

  • Inflation is higher prior to crises than during normal times. This finding is not easily reconciled with the findings of declining commodity prices and low growth, but it could be the result of a devaluation following adverse external shocks. More generally, it suggests that more than one macroeconomic pattern may result in banking crises.

  • The public debt-to-GDP ratio starts increasing some time prior to crises, stabilizing one year after the crisis. This suggests that banking crises in LICs may contribute to adverse debt developments through the fiscal costs of banking sector repair (as they do in more developed countries, see Reinhart and Rogoff 2013), even if the fact that debt starts rising before crises shows that other drivers are at work, too.

15. These differences in the distribution of banking crises across decades and in the patterns of crisis predictors highlight the need to develop an early warning system specific to LICs.

C. Predicting Crises

16. As do many recent studies in the financial crises literature, we estimate a logit model for the probability of entering a banking crisis (see Appendix IV.1). To avoid limiting the regression sample to countries that experienced a crisis at some point during the sample period, we follow Caballero (2016) and use the so-called RE-Mundlak logit estimator.

17. We find that

  • Changes in commodity prices play a key role in triggering banking crises, confirming insights of the stylized facts analysis. Specifically, there is a negative association between the growth rate of commodity terms of trade and the likelihood of a banking crisis, and a positive association between the volatility of commodity terms of trade and the probability of crises.

  • There is no evidence that credit growth or other standard banking system variables matter for the occurrence of banking crises, except for banking system liabilities not backed by international reserves.

  • Crisis episodes are more likely to follow periods of high inflation, high public debt, and reliance on short-term borrowing

  • Crises are more likely in periods of tight global monetary conditions. There is no evidence, however, that high net private capital inflows contribute to banking crises.

18. Given the very low number of banking crises in LICs in recent years, out-of-sample prediction covering recent years is not a meaningful option for evaluating the predictive power.

19. The empirical model provides good in-sample predictive power. Taking the sample’s unconditional probability of a banking crisis (2.2 percent, based on 35 crisis events) as a threshold such that predicted probabilities exceeding this threshold are interpreted as signaling a coming crisis, in-sample prediction misses 14 percent of crises and results in 25 percent of false alarms.

D. Predicting Stress Episodes

20. The second exercise concerns episodes of financial stress, specifically financial stress in 2016, based on a desk survey conducted for IMF (2017). Desks identified ten countries (Burundi, Central African Republic, Chad, Liberia, Moldova, São Tomé and Príncipe, Sierra Leone, South Sudan, Tajikistan, and Zimbabwe) as having been under stress in 2016.

21. Our analysis focuses on key financial soundness indicators (FSIs) as explanatory variables. The investigation of FSIs as potential leading indicators of financial stress is motivated by evidence showing a correlation between some FSIs—including NPLs and bank profitability—and banking crises in a large set of countries (Navajas and Thegeya 2013). Also, the macro variables used in the crisis prediction logit analysis do not exhibit significantly different behavior between the groups of stressed and non-stressed countries.

22. In a preliminary step, we compare FSIs in the years preceding the designation of countries as stressed and find that the means of certain indicators are quite consistently statistically different from those in other countries (Appendix Table A.IV.1.2). Robust statistical differences between means of stressed and non-stressed countries are observed for non-performing loans (higher for stressed countries), return on equity (lower for stressed countries), and regulatory capital to risk weighted assets ratio (higher for stressed countries). Return on assets (lower for stressed countries), along with ratio of liquid assets to total assets (usually lower for stressed countries) and provisioning rate (higher for stressed countries) exhibit differences that are less consistently statistically different from zero.

Table A.IV.1.1.

Predicting Banking Crises, Baseline Results

article image
Notes: Sample of 35 banking crises in 60 countries, with 1,557 observations. In the last column the number of countries is 29 and the observations are 804. Absolute t-ratios in parentheses based on standard errors clustered at the country-level. *, ** and *** indicate statistical significance at the 10%, 5% and 1% level, respectively. Columns (1) -(4) present results of the RE-Mundlak Logit model; column (5) results of a pooled logit regression; in column (6) we estimate logit models with country fixed effects. The latter only includes countries with at least one banking crisis. The makeup of the additional covariate groups is as follows (all in percent except for the dummies): “Crisis & Conflict Dummies”: fiscal crisis dummy, currency crisis dummy, major conflict dummy; “Macro Fundamentals”: real GDP growth; “Banking System” leverage, liquidity, size; “Major Donor Aid Flows” share of ODA by top-3 donors. “ROC Comparison” compares the predictive power (via AUROC) of the two models indicated, with the null hypothesis of identical predictive power.
Table A.IV.1.2.

Predicting Banking Crises, Economic Magnitudes

article image
Notes: All results here are the economic magnitudes for a one standard deviation increase in the explanatory variable, expressed in percent based on the results in Table A.IV.1.1; additional covariates are included as indicated in that table. Absolute t-ratios in parentheses are based on standard errors computed via the Delta method. See Table A.IV.1.1 for all other details.

23. A more formal noise-to-signal ratio analysis suggests three FSIs as leading indicators of financial stress: the ratio of non-performing loans (NPLs) to total gross loans, return on equity, and return on assets. We carry out a noise-to-signal computation exercise for each of the FSIs, for every year, using the following procedure: (i) we construct a grid of 10,000 values between the 75th and 90th percentile of a FSI (between the 10th and 25th percentile for FSIs where lower values signal stress); (ii) for each of these values we compute the noise-to-signal ratio as in Kaminsky et al. (1998); and (iii) choose the value that minimizes the noise-to-signal ratio as the threshold value for that FSI. The lower the noise-to-signal ratio, the higher the informational content of a signal; and a noise-to-signal ratio of less than 1 indicates that the signal has some information. We find that the ratio of non-performing loans to total gross loans, return on equity, and return on assets have the lowest noise-to-signal ratios. This largely confirms the results of the test of means exercise, except for return on assets, which lacked consistent statistical significance in the test of means. These results are consistent with evidence on OECD countries (Kasselaki and Tagkalakis 2014).

24. The signal extraction approach appears to provide acceptable prediction quality, but this finding should be used with caution given the small sample it is based on. The signals from the three key variables combined ---the value of at least one of the three FSIs breaches its threshold value---to predict financial stress in 2016 yields a missed crisis rate of 40 percent and a false alarm rate of 16 percent. The exercise could usefully be firmed up in future years based on additional desk surveys.

Appendix IV.1. The Empirical Model for Predicting Financial Sector Crises

As do many recent studies in the financial crises literature, we estimate a logit model for the probability of entering a banking crisis. To avoid limiting the regression sample to countries that experienced a crisis at some point during the sample period (in our case, this amounts to 29 economies), we follow Caballero (2016) and use the so-called RE-Mundlak logit estimator. This model builds on a random effects logit model, where the assumption of no correlation between the individual (in our case country-specific) effects and the covariates is relaxed by separately including estimates of the country-specific means of each covariate. The coefficients capture the within-country dimension.

Results confirm the finding of the stylized facts analysis that changes in commodity prices play a key role in triggering banking crises. Table A.IV.1.1 reports selected coefficients. In the first specification (column 1), we include the commodity terms of trade growth and volatility, controlling only for the presence of deposit insurance, dummies for conflicts and fiscal and external crises, and the US interest rate. We find a negative association between the growth rate of commodity terms of trade and the likelihood of a banking crisis, and a positive association between the volatility of commodity terms of trade and the probability of crises. These results are robust to the inclusion of a set of bank, macro, and external sector controls (columns 2 to 4).

There is no evidence that credit growth or other standard banking system variables matter for the occurrence of banking crises, except for banking system liabilities not backed by international reserves. Consistent with the stylized facts analysis, we do not find any indication that credit growth matters for the occurrence of banking crises. Similarly, banking system variables on leverage and size do not show a robust correlation with the occurrence of crises, and only weak indications that crises are less likely when banks are more liquid (column 2). In contrast, high growth of banking system liabilities not backed by international reserves do tend to precede crises (column 3).

Crisis episodes are more likely to follow periods of high inflation, high public debt, and reliance on short-term borrowing (column 3). Including macroeconomic variables increases the predictive power of the model (the area under the ROC curve is statistically greater than in the reduced model with only banking system variables).

There is evidence that crises are more likely in periods of tight global monetary conditions. This finding could also partially explain the limited number of crisis events during recent years, characterized by low global interest rates. There is no evidence, however, that high net private capital inflows contribute to banking crises.

To gauge economic impacts, Table A.IV.1.2 reports the effects associated with a one standard deviation increase in explanatory variables on the probability of banking crises. Based on the preferred model in column 4, an increase in commodity terms of trade by one standard deviation is associated with about a 1 percent decline in the propensity of a crisis, which unconditionally is 2.2 percent in our sample, indicating that this effect is economically substantial. High terms of trade volatility are associated with an even larger increase in crisis propensity. Fiscal variables such as public debt and in particular short-term debt have high economic magnitudes of 1 percent and 2.2 percent respectively, while the magnitudes of low reserve coverage and inflation range between 1.1 percent and 1.4 percent.

Appendix Table A.IV.1.2.

Test of Means---Key Financial Soundness Indicators (FSI)

article image

References for Section I

  • Ahuja, A., M. Syed, and K. Wiseman, 2017, “Assessing Country Risk—Selected Approaches—Reference Note,” Technical Notes and Manuals 17/08 (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • Alessi, L., A. Antunes, J. Babecky, S. Baltussen, M. Behn, D. Bonfim, O. Bush, C. Detken, J. Frost, R. Guimaraes, T. Havranek, M. Joy, K. Kauko, J. Mateju, N. Monteiro, B. Neudorfer, T. Peltonen, P. Rodrigues, M. Rusnák, W. Schudel, M. Sigmund, H. Stremmel, K. Smidkova, R. van Tilburg, B. Vasicek, and D. Zigraiova, 2015, “Comparing Different Early Warning Systems: Results from a Horse Race Competition among Members of the Macro-Prudential Research Network,” Research Network, ECB Mimeo.

    • Search Google Scholar
    • Export Citation
  • Berg, A., E. Borensztein, and C. Pattillo, 2005, “Assessing Early Warning Systems. How Have they Worked in Practice?”, IMF Staff Papers, Vol. 52, No. 3, pp. 462502.