A. Background and Goal

1. Low-income countries (LICs) have shown improved macroeconomic stability since the early 2000s but continue to face volatility and potential for severe stress events. For example, the end of the commodity price super-cycle in 2014, in combination with expansive policy stances, has contributed to strains on growth, fiscal balances, and external positions of numerous LICs. As a result, several LICs, particularly commodity exporters, have seen sharp depreciations and quick increases in external debt that, if continued, could put debt sustainability at risk (IMF, 2018).

2. Consequently, LIC governments have a strong interest in tools for predicting economic stress, as do institutions such as the IMF advising them. Based on predictions of stress, governments may be able to implement strategies for reducing its likelihood of crises materializing and their severity when they do happen, for example by building adequate buffers in good times.¹

3. There are numerous frameworks for predicting stress in advanced and emerging economies, which use a range of crisis concepts and empirical approaches. Crisis concepts used include external sector/balance of payments crises, sovereign crises/debt distress, financial sector/banking crises, and growth decelerations/recessions. Traditional empirical approaches to predicting crises include notably the “signal extraction” approach developed by Kaminsky, Lizondo and Reinhart (1998), which issues a crisis warning if an indicator exceeds a specified threshold; and logit or probit estimation based crisis warnings, while more recently, machine-learning techniques such as classification trees have started to be used. Berg, Borensztein, and Pattillo (2005) and IMF (2007) assess the performance of a number of early warning systems developed in the aftermath of the Asian crisis of 1999. Alessi and others (2015) and Holopainen and Sarlin (2017) survey additional work and compare the predictive powers of a range of empirical approaches. Ahuja, Syed, and Wiseman (2017) describe crisis concepts and empirical approaches in use at the IMF until recently, and European Commission, European Stability Mechanism, and OECD staff have also presented empirical frameworks for vulnerability/crisis prediction (Berti, Salto and Lequien 2012; Lenkh, Moshammer and Valenta 2017; and Hermansen and Röhn 2017).

4. In contrast, there are only few frameworks for predicting stress events in LICs, and they focus on only a narrow set of crisis concepts and empirical approaches. The limited extent of work on LICs likely reflects in part the fact that identifying the drivers of stress in LICs is particularly difficult, given the high level of macroeconomic volatility seen in these countries (see IMF 2011b) and the much weaker data available for them. Efforts at the IMF have until recently focused on risks of growth decelerations and debt distress:

• Dabla-Norris and Bal Gündüz (2012) introduced a methodology for predicting sharp declines in GDP growth, an approach used in the IMF’s Vulnerability Exercise for LICs (IMF 2014, 2015, 2016, and 2018). Their Growth Decline Vulnerability Index classifies countries based on their estimated likelihood of seeing a sharp drop in growth should they be hit by a large adverse shock. The variables entering the index were determined using the signal extraction approach of Kaminski, Lizondo, and Reinhart (1998).

• The IMF and World Bank joint debt sustainability framework for LICs has aimed to assess the risk of debt distress events. Introduced in 2005 and recently strengthened, this framework has been an important element of IMF and World Bank country work on LICs. Empirical implementation relies on logit regressions as well (IMF and World Bank 2017).

5. This paper proposes early-warning systems for additional crisis concepts for LICs that rely on machine-learning approaches where they perform better than traditional econometric ones.

• Crisis concepts: With growth declines and debt distress having been tackled previously, the new concepts are balance of payments crises, inflation crises, and financial sector crises in LICs. Such crises are relevant for LICs as well, notwithstanding the facts that (i) sudden stops or reversals of capital inflows remain a less important contributor to balance of payments crises for many LICs given their as yet limited degree of de facto integration into global financial markets; (ii) the impact of financial sector crises on growth may be smaller in LICs than in more developed countries given their smaller financial sectors – the fiscal costs of banking sector repair can be sizeable in LICs as well.²

• Prediction: As traditional econometric methods struggle in predicting crises in LICs, we rely on machine learning approaches where they perform better. Some of these approaches are particularly well equipped to capture non-linearities. Non-linearity is a key feature of crises, which occur when several factors interact, such as one or more shocks affecting an economy that suffers from perhaps several pre-existing weaknesses (see Holopainen and Sarlin, 2017). A drawback of machine learning approaches is that they can be difficult to interpret and hence intransparent. For this reason, before using machine learning for prediction, we always provide not only a thorough discussion of stylized facts but also use traditional econometrics to build understanding and intuition.

6. Three technical elements merit being highlighted:

• The proposed crisis concepts are operationalized at an intermediate level of severity. Predictions will be most useful if they signal not only risks of experiencing tail events but also somewhat less severe difficulties, which are still important for macroeconomic performance and policies.

• Predictions aim to identify the unconditional likelihood that a crisis event will occur in the following year, as is conventional in the literature, rather than the conditional probability proposed by Dabla-Norris and Gunduz, 2012. Users such as country authorities are likely interested more in absolute risk than in conditional risk. Relatedly, we use the terms “vulnerability” and “risk” synonymously.

• Predictive performance is assessed based primarily on out of sample prediction. A prediction is out of sample if the period for which prediction is being made was not used to help specify or estimate the empirical model. Approaches that have been developed by maximizing in-sample predictive performance are vulnerable to “overfitting”, resulting in a substantial deterioration of predictive performance when confronted with new data. We see the focus on out-of-sample performance as a key strength of our approach relative to some previous work.

7. The paper is structured as follows: In the remainder of this introduction and overview, subsection B summarizes the new crisis concepts, including crisis definition, relevance for growth, key stylized facts, and early warning systems; and subsection C discusses the temporal incidence and overlaps of the crises. Appendices then offer some factual information and technical background discussions: Appendix I.1 lists the LICs included in this study; Appendix I.2 provides explanations on empirical methods for predicting economic stress, including a very brief introduction into the machine learning techniques used; and Appendix I.3 discusses two implementation issues that can arise in the context of vulnerability analysis: moving from binary early warning signals to more finely graduated ones, and aggregating sectoral crisis signals to an overall rating, should aggregation be desired. Sections II-V then present the crisis concepts in a largely nontechnical manner, with brief discussions of key technical elements in appendices. Companion papers on external crises, inflation crises, and financial sector crises in LICs provide in-depth discussions, including all technical detail.

B. Overview of the New Crisis Concepts

8. A crisis prediction framework for LICs may wish to capture vulnerability to three types of crises that have been largely neglected in crisis prediction work for LICs so far: balance of payments crises, inflation crises, and financial sector crises.³ Summarizing key elements of sections II-IV, in the following we briefly present these new concepts, discuss the relevance of such crises and their stylized facts, and describe the associated early warning systems.⁴

External Sector/Balance of Payments Crises

• Crisis concept: The concept relies on sharp nominal exchange rate depreciation, drops in reserves to a very low level, and contractions in import volumes that reflect inability to maintain imports at levels observed in the recent past. This concept modifies existing definitions of BoP crises to address the LIC-specific challenges, in particular data quality issues. Also, capital account related crisis concepts that are appropriate for more developed countries and are the heart of the VEE, are not yet very relevant for the majority of LICs that continue to have limited access to global capital markets.

• Relevance: BoP crises appear to have large output costs, notwithstanding the fact that growth typically already starts falling before crisis onset, suggesting some degree of feedback between growth and BoP crises. Specifically, a comparison of GDP growth during the year of the crisis and the two following years with growth in quiet times suggests that at the end of this period, output is about 7 percent lower than it would have been otherwise. BoP crises also result in inflation spikes when they are associated with depreciations.

• Stylized facts: High fiscal deficits, sustained credit growth, and real effective exchange rate overvaluation often precede BoP crises, and a range of shocks can trigger them. The sizes of both shocks and buffers matter: for given shock sizes, countries with strong buffers are less likely to experience an external sector crisis than countries with weak ones; and for a given strength of buffers, countries affected by a smaller shock are less likely to fall into crisis than countries hit by a large shock.

• Performance of early warning system: Traditional econometric approaches struggle to predict BoP crises. Thus, we explored machine learning approaches. The Random Forest classification technique, which is well suited to capturing non-linearities and interactions between explanatory variables, provides acceptable out-of-sample predictions: over the period 2010–15, it misses 25 percent of crises and sounds false alarms 39 percent of the time (and retains this performance reliably when the sample changes).

Inflation Crises

• Crisis concept: Inflation crises/stress episodes are defined as times when inflation exceeds 15 percent per year. This level of inflation is known to adversely affect growth (Khan and Senhadji, 2001) and is not as high as to flag only very rare events, in keeping with the goal of including stress episodes of intermediate severity.

• Relevance: High inflation has been shown to adversely affect growth, income distribution, and poverty levels. In the sample under study, a comparison of GDP growth during the year of the crisis and the two following years with growth in quiet times suggests that at the end of this period output is about 2½ percent below the level that would have prevailed otherwise.

• Stylized facts: Episodes of high inflation in LICs often result from both shocks such as increases in world fuel and food prices, and government policies, particularly expansive fiscal and monetary policies. They can also result from exchange rate depreciation, e.g. in the context of external crises.

• Performance of early warning system: The early warning system developed for inflation crises relies on machine learning approaches after traditional econometric approaches failed to outperform desk forecasts. The Random Forests classification method delivers strong out of sample prediction performance, with rates of missed crises and false alarms of about 15 percent each.

Financial Sector Crises and Stress Episodes

• Crisis concept: The main definition is that of banking crises as proposed by Laeven and Valencia (2013). It identifies a crisis by the occurrence of one or both of the following: (a) significant signs of financial distress in the banking system-as indicated by significant bank runs, losses in the banking system, and/or bank liquidations; and/or (b) significant banking policy intervention measures in response losses in the banking system. In addition, given the limited frequency of systemic banking crises in LICs in recent years, an alternative notion of banking sector stress as assessed by country teams is used to capture less severe stress events.

• Relevance: An extensive literature documents the output and fiscal costs of banking crises in advanced economies and emerging markets. In LICs, however, large output costs of financial sector crises have not been documented and are also not apparent in the sample under study. The apparently limited impact on growth may reflect the much smaller size of LICs’ financial sectors relative to GDP. Nevertheless, it appears that public debt rises by several percent of GDP in the context of financial sector crises, possibly reflecting in part the fiscal costs of banking sector repair.

• Stylized facts: As concerns the factors preceding and likely contributing to financial crises, we find that banking crises in LICs often follow on a decline in commodity terms of trade. In contrast to patterns found in more developed countries, high credit growth does not help predict financial sector crises in LICs.

• Performance of early warning system: Using a recently updated sample of banking crises in LICs since the 1980s, and a random effects logit estimator employed in the recent literature, (in-sample) crisis predictions are fairly accurate, with 14 percent of missed crises and 25 percent of false alarms. The use of FSIs to predict less severe financial stress episodes shows a 40 percent rate of missed crises and a false alarm rate of 16 percent, a result that would benefit from being confirmed based on a larger data set in the future.

A. Assessing Predictive Performance

A traditional approach to assessing an empirical approach’s predictive performance relies on loss functions. In a first step, probit or logit regressions or other approaches are used to estimate the likelihood of a crisis occurring in the following year. In a second step, estimated probabilities are then identified as crisis or non-crisis predictions by determining a threshold probability such that it minimizes a loss function over a weighted sum of the shares of missed crises in all crises and of false alarms in all calm times. In a third and final step, an empirical approach’s performance in issuing crisis predictions is then also often evaluated using loss functions.

Loss functions are, however, subjective in that they reflect the researcher’s preferences regarding the trade-off between missing crises and issuing false alarms. Different authors use loss functions with different functional forms and different weights on missed crises and false alarms (Text Table below).

Preferences for Type I and Type II Errors in the Literature

Preferences for Type I and Type II Errors in the Literature

	weight on missed crises (e (0, 1))	aggregation method
LIC DSF	0.66	linear
VEE	0.5	linear
Kaminski and Reinhart (1999)	0.5	log-linear

View Table

Preferences for Type I and Type II Errors in the Literature

	weight on missed crises (e (0, 1))	aggregation method
LIC DSF	0.66	linear
VEE	0.5	linear
Kaminski and Reinhart (1999)	0.5	log-linear

The Receiver Operating Characteristic Curve (ROC) provides an objective alternative to loss functions. It plots a system’s true positive rate (equal to 1 minus the rate of false negatives or missed crises) on the y-axis against its false positive rate (false alarms) on the x-axis.⁵ All crisis predictions that are not informed by the data, such as coin flips, lie on the 45-degree line between points (0,0) and (1,1). The improvement in prediction made possible by an empirical model is larger the more its ROC curve bends up and away from the 45-degree line. The stronger the curvature, the larger is the rate of true negatives that can be obtained for any given rate of true positives. A model’s area under the ROC curve (AUC) can thus be used as a measure of predictive performance.

A graphical presentation illustrates the implications of using different loss functions on crisis predictions (Figure below):

• The functional form of loss functions and the weights on missed crises and false alarms determine the shape and slope of indifference curves. Linear loss functions are represented by linear indifference curves, and loss functions based on a noise-to-signal ratio would be represented by strictly convex curves. The higher is the weight put on missed crises, the flatter the slope of the indifference curve.

• The optimal probability threshold used in an early warning system is the one where the ROC curve is tangent to an indifference curve. If the ROC curve is concave or the indifference curves are strictly convex, then the optimal threshold is between 0 and 1. For linear indifference curves, however, the optimal threshold can also reach 0 or 1 or very close to these extremes. Such corner solutions or near corner solutions happen when the slope of indifference curves is sufficiently different from 1 and the ROC curve has a low degree of concavity.

• In or near corner solutions, the system makes no or only little use of the predictive potential of the underlying empirical model’s ability to predict crises. This can be seen in the fact that the ROC is very close to the 45-degree line there. While choosing such solutions is still optimal given the researcher’s preferences, more balanced preferences over false alarms and missed crises better exploits the underlying model’s ability to predict crises.

In-sample ROC Curve of a Logit Model

Citation: IMF Working Papers 2020, 289; 10.5089/9781513564609.001.A001

• Download Figure
• Download figure as PowerPoint slide

View Full Size

In-sample ROC Curve of a Logit Model

Citation: IMF Working Papers 2020, 289; 10.5089/9781513564609.001.A001

• Download Figure
• Download figure as PowerPoint slide

In-sample ROC Curve of a Logit Model

Citation: IMF Working Papers 2020, 289; 10.5089/9781513564609.001.A001

• Download Figure
• Download figure as PowerPoint slide

We use both approaches (loss functions and ROCs) for evaluating alternative empirical approaches’ predictive performance.

B. Machine Learning Approaches to Prediction

Machine learning techniques are well suited to predictive modeling, as they lower the variance of parameters estimated on different samples. Traditional econometric techniques seek to identify parameters within samples while avoiding estimation bias. In small samples, this comes at a cost: parameter estimates are sensitive to the estimation sample, resulting in an out-of-sample fit that is typically substantially weaker than the in-sample fit. This trade-off between parameter bias and variance is well known to any empiricist. Researchers take it into consideration implicitly when limiting the set of explanatory variables to increase the degrees of freedom and reduce the risk of overfitting. By making the trade-off explicit and addressing it squarely, machine learning techniques can nevertheless achieve often achieve more stable predictive performance. Potential drawbacks of at least some machine learning approaches are that they can be non-transparent and rely on implausible explanatory variables. The researcher can mitigate these issues to some extent by selecting suitable explanatory variables.

A. From Binary Early Warning Signals to Three-Tiered Signals

Risks analyses may sometimes benefit from a gradation of risk estimates that is finer than the binary risk signals provided by typical early warning systems. Early warning systems typically provide a binary signal by distinguishing countries that face elevated risks from those that do not. (This is usually done by choosing a threshold crisis probability beyond which estimated probabilities are classified as crisis predictions, such that the threshold minimizes a loss function defined over shares of missed crises and false alarms.)

There are several options for providing three-tiered signals. These include:

• With the high-risk rating determined by the threshold probability that minimizes a loss function with chosen weights on missed crises and false alarms, one can set the moderate risk rating in line with a high-risk threshold corresponding to a loss function with a higher weight on missed crises.

• Determination of elevated risk and medium risk thresholds can also proceed by reference to probabilities perceived as meaningful by the user. For example, a crisis risk of 15 percent could serve as a high-risk threshold and a probability of 10 percent as a moderate risk threshold. (This approach would thus do away with early warning system’s typical approach of setting only one probability threshold.) To ensure that crisis warnings are operationally useful, these thresholds would need to be determined such that neither too many nor too few crises are signaled. This implies that probabilities would generally need to be set at diverse levels for different crises types, depending on the respective different crisis frequencies.

• There are also hybrid options that rely on setting the high-risk threshold such that it minimizes a loss function over missed crises and false alarms and setting the moderate risk threshold such that it corresponds to a certain crisis probability. This approach was pursued by the VELIC.

• The LIC DSF pursues yet a different approach involving stress tests. It identifies risk levels based on whether projected debt ratios exceed levels that have been identified as indicating a heightened risk of debt distress. If debt ratios exceed thresholds under baseline macroeconomic projections, the model classifies the country as facing a high risk of debt distress. If this is not the case but ratios exceed thresholds under alternative stress tests macroeconomic projections, countries are said to face a moderate risk of debt distress. This approach can be used only for early warning systems that comprise stress tests.

B. Aggregating Estimated Sectoral Crisis Risks

If desired, the estimated sectoral crisis risks could be aggregated into an overall rating to provide a sense of countries’ total risks. The VEA assesses that its “collection of models does not lend itself to aggregation and serves to provide a more generalized sectoral perspective on risks” (Ahuja, Syed, and Wiseman 2017). Accordingly, the VEA downplays aggregation. When required, sectors are aggregated according to a simple average or maximum of the flags produced by constituent modules. The overall level of vulnerability is taken from the distribution of the average of sectoral flags with thresholds set atthe mean and one standard deviation above the mean.

A. Introduction and Definition of External Sector Crises

1. External sector crises are events that involve a sharp depreciation of a currency’s exchange rate or a sharp decline in foreign exchange reserves (and possibly other factors). Such crises, also called external, currency, or balance of payments (BoP) crises, can be very destructive to welfare, as evidenced by large drops in GDP growth.

2. External sector crises occurring in emerging market economies have been discussed extensively, but two challenges arise when applying existing concepts to LICs: First, while the literature on emerging markets (and the VEE) emphasize the dynamics of short-term capital flows, such flows still play an only limited role in most LICs, with the exception perhaps of the frontier countries. Thus, models developed to understand and predict external crises in emerging markets may not be appropriate for most LICs. Second, data are much more limited for LICs than EMs, and data coverage and quality vary greatly. Any crisis definition and prediction methodology thus need to work with a smaller set of variables, be robust to data gaps, and make do with annual rather than quarterly or higher frequency data.

3. The literature relating to EMs usually defines external sector crises with respect to a sharp depreciation of a currency’s exchange rate or decline in foreign exchange reserves. Our definition for LICs adds a criterion on a drop of imports. This criterion is meant to capture crises that data on movements in reserves fail to indicate (data on reserves is patchy, particularly for earlier years and small states). The additional criterion could also be useful in situations where currency depreciations or reserve losses may not fully reflect external pressures, as may be the case in dollarized economies, currency unions, or where the reported official exchange rate deviates considerably from the parallel or interbank market rate. Imports may also decline in non-crisis circumstances, including declines in the prices of imports, the winding down of large investment projects, and domestic demand shocks. Therefore, to avoid casting the net too widely, we only consider cases of non-oil import contractions where there are additional signs of significant external pressures.

4. Specifically, we say that a country is suffering an external crisis in any given year if it meets one or more of the following three criteria:

• i. The average official exchange rate depreciates against the U.S. dollar by at least 30 percent relative to the preceding year and the rate of depreciation increases by at least 10 percentage points relative to the preceding year. (This criterion is adopted from Laeven and Valencia, 2013.) Information on parallel market exchange rates is limited and therefore excluded.

• ii. Reserve holdings, measured in U.S. dollars, drop by at least 30 percent relative to the preceding year and to a level below 1½ months of imports. If, in the preceding 10 years, a country has had a median reserve coverage of less than 1½ months of imports, however, this low level of reserve coverage may not be a signal of a crisis, and the country in question is excluded from application of the reserves criterion.

• iii. The U.S. dollar value of non-oil imports falls by at least 15 percent relative to the preceding year and declines over a two-year period. In addition, at least one of the four following conditions need to be met:

  • The country experiences a substantial negative current account shock (e.g., a decline in exports or transfers) in the current or the preceding year. This shock is measured as a deterioration of the current account balance net of imports by at least 2 percentage points of GDP.

  • Reserves at the end of the current year cover less than 1½ months of imports.

  • External debt arrears grow by at least 3 percent of GDP in the current year.

  • The country enters an IMF-supported program in the current or the following year.

  At the same time, to rule out cases of orderly (unforced) rather than disorderly (crisis-related) external adjustment, we exclude cases of import compressions in which any of the above criteria are met but reserve coverage in the current year and the preceding year exceed 5 months of imports and reserve holdings increased by more than 5 percent in the current year.⁷

5. Whenever a country meets the above definition for two or more years in a row, we refer to this as a sustained crisis episode rather than two separate crises. Also, when two crises are separated by only one tranquil year, this year is integrated with the preceding and following years into one crisis episode.

6. The remainder of this section is structured as follows: subsection B reviews stylized facts around external crises including their relationship with the other types of crises discussed in this paper; and subsection C turns to predicting crises (using both traditional and machine learning approaches) and evaluates predictive performance. The machine learning approaches used were briefly described in Appendix I.2 above.

B. Stylized Facts

Incidence and Duration of Crises

7. External crises as defined above are moderately frequent events, whose incidence has fallen sharply since the turn of the century. On average, LICs moved into an external crisis once every 13 years and spent 16 percent of their time in crises.⁸ The incidence dropped sharply around the turn of the century. During the 1980s and 1990s, difficult and turbulent decades for LICs, countries were in external crises some 21 percent of the time, and thereafter only 9 percent of the time. A drop in the incidence of sharp depreciations contributed most to the improvement seen in the 2000s, followed by fewer cases of reserve depletion. The driving forces underlying this shift remain a subject for future research.

8. The incidence of crises varies greatly across country groups (Text Table below): frontier markets have spent a limited amount of time in external sector crises, while fragile states have seen significantly more such crises than the average LIC. Commodity exporters, particularly fuel exporters, have also spent a larger share of the time in crises than other countries (many of them are fragile countries).⁹

9. The typical external crisis is short-lived. Nearly two thirds of crisis episodes last only one year, and most crises are over after three years. Note however that economic impacts may linger after the crisis has come to an end. For example, the recession that often follows external crises typically lasts longer than the crisis episode itself (see below).

Share of Years Spent in Crisis by Type of Countries

(In percent)

Note: Country classification based on most recent groupings. Since the status of frontier markets and fragile states has changed markedly over time, summary statistics for these two groups are not calculated for the pre-2000 period.

Share of Years Spent in Crisis by Type of Countries

(In percent)

		pre-2000	post-2000
by somes of export earnings
	Diversified	18.2	7.3
	Non-fuel commodity	0.3	11.5
	Fuel	15.7	7.6
by economic development
	Frontisr marksts		7.4
	Xon-frontier marks ti		9.4
by fragility
	Fragile states		14.2
	Non-fragile states		5.0

Note: Country classification based on most recent groupings. Since the status of frontier markets and fragile states has changed markedly over time, summary statistics for these two groups are not calculated for the pre-2000 period.

View Table

Share of Years Spent in Crisis by Type of Countries

(In percent)

		pre-2000	post-2000
by somes of export earnings
	Diversified	18.2	7.3
	Non-fuel commodity	0.3	11.5
	Fuel	15.7	7.6
by economic development
	Frontisr marksts		7.4
	Xon-frontier marks ti		9.4
by fragility
	Fragile states		14.2
	Non-fragile states		5.0

Note: Country classification based on most recent groupings. Since the status of frontier markets and fragile states has changed markedly over time, summary statistics for these two groups are not calculated for the pre-2000 period.

C. Predicting Crises

14. We explore a range of empirical specifications, including both traditional econometrics and machine learning approaches. The latter help address the challenges of model selection and provide systematic approaches to dealing with overfitting, which undermines out-of-sample predictive performance.

A. Introduction and Definition of Inflation Crises

1. Research for countries at various levels of income finds that high and variable inflation has negative macroeconomic and social consequences.

• It increases the cost of holding liquid assets that facilitate transactions; it increases uncertainty, thereby reducing investment and growth;⁵ and it may cause financial sector strains whenever the values of assets and liabilities respond differently to inflation.

• In fixed exchange rate regimes, it erodes price competitiveness, contributing to external sector crises.

• It hurts the poor in particular, who often lack access to inflation hedges.⁶ Surveys of attitudes towards inflation show that the poor are more concerned about it, which is consistent with inflation being negatively correlated with improvements in their well-being (Easterly and Fischer, 2001). In LICs, poor people’s vulnerability to inflation can be very high given that their incomes barely cover basic needs.

2. We propose a definition of inflation crises/episodes that is pitched at an intermediate degree of severity. In a seminal paper, Khan and Senhadji (2001) found that inflation higher than 12 percent per year has a negative impact on growth among developing countries.⁷ While the identification of a threshold above which inflation becomes particularly harmful depends on estimation methods, sample and modeling choices, Khan and Senhadji’s result is widely cited and arguably matches the views of policy-makers. Aiming at capturing risks that are neither trivial nor in the extreme tails of distributions, we define inflation crises/episodes as inflation rates exceeding 15 percent.

3. The section is structured as follows: Subsection B reviews the stylized facts on inflation and inflation crises in LICs. Subsection C turns to predicting inflation crises. In a first step towards this, we present a panel regression-based forecasting model of inflation. Then we develop and compare several possible early warning systems for inflation crises based on country team forecasts, the panel regression and others on machine learning approaches. The “Random forest” machine learning technique performs best, providing strong out-of-sample prediction.

B. Stylized Facts

4. LICs have on average higher inflation than advanced and emerging economies. This does not necessarily imply poor policies or inefficiencies, as the Balassa-Samuelson effect ensures that higher growth in low income countries implies a trend real exchange rate appreciation relative to advanced economies (e.g., Edwards and Savastano, 2001). For low income countries with fixed exchange rates, this means higher inflation than in advanced economies. However, persistent large differentials in inflation cannot be explained by Balassa-Samuelson mechanisms. These are best explained by weaknesses in monetary and fiscal frameworks that ought to be remedied.

5. Over recent decades, inflation in the typical LIC has followed a downward trend. Median inflation across LICs sharply fell during the 1990s and early 2000s, in part thanks due to the gradual disinflation and stabilization in transition economies. LIC inflation trends did not differ significantly across regions except for the higher inflation levels in transition countries in the 1990s. They also did not differ much across classifications of LICs in terms of export structures. LIC inflation spiked in 2008 due mainly to higher world food prices (Habermeier et al., 2009). In recent years, inflation has fallen further, in line with lower global inflation. More recently, however, some countries (e.g., Ghana, Nigeria and Zambia) that experienced exchange rate depreciation in the wake of the 2014 commodity price decline have seen inflation pressures.

LIDCs Inflation 1995–2016 median and interquartile range

Citation: IMF Working Papers 2020, 289; 10.5089/9781513564609.001.A001

• Download Figure
• Download figure as PowerPoint slide

View Full Size

LIDCs Inflation 1995–2016 median and interquartile range

Citation: IMF Working Papers 2020, 289; 10.5089/9781513564609.001.A001

• Download Figure
• Download figure as PowerPoint slide

LIDCs Inflation 1995–2016 median and interquartile range

Citation: IMF Working Papers 2020, 289; 10.5089/9781513564609.001.A001

• Download Figure
• Download figure as PowerPoint slide

6. Inflation crises/episodes have become less frequent. Using the definition described above, there were 255 inflation crisis episodes out of 1264 country-year pairs (25 percent of the sample) during the period 1990–2015, with a trend reduction in the incidence of inflation crises/episodes: during the 1990s, there were inflation crises in 37 percent of the country-year pairs; since 2000, this incidence has dropped to 12 percent.⁸

Proportion of LICs with inflation crisis

Citation: IMF Working Papers 2020, 289; 10.5089/9781513564609.001.A001

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Proportion of LICs with inflation crisis

Citation: IMF Working Papers 2020, 289; 10.5089/9781513564609.001.A001

• Download Figure
• Download figure as PowerPoint slide

Proportion of LICs with inflation crisis

Citation: IMF Working Papers 2020, 289; 10.5089/9781513564609.001.A001

• Download Figure
• Download figure as PowerPoint slide

7. Using a simple linear regression to describe the behavior of macroeconomic variables around crises, we find that fiscal pressures, loose monetary policies, depreciation of the domestic currency, and increases in world fuel and food prices precede inflation crises:

• Fiscal crises (as defined in Gerling et al., 2017) are 10 percentage points more likely to occur three years before an inflation episode than in other times (Figure 1, upper left). The intuition is that high fiscal deficits drive monetary financing.

• Broad money and credit to the private sector grow faster prior to inflation episodes than in tranquil times (Figure 1, upper right and second row right).⁹

• Periods of strong real effective exchange rates, which may result in overvaluation, tend to precede inflation episodes (Figure 1, second row left).

• Quarters with inflation crises/episodes are associated with nominal depreciations (Figure 1, third row left).

• Oil and food price increases tend to precede years with inflation episodes (Figure 1, bottom row).

8. Confirming the insights of the literature, we find that inflation crises are associated with a permanent output loss. A comparison of GDP growth during the year of the crisis and the two following years with growth in quiet times suggests that at the end of this period, output is about 2½ percent below the level that would have prevailed otherwise.

C. Predicting Crises

Early Warning Systems

12. We now describe a suite of possible early warning systems for inflation crises in LICs and assess their out-of-sample prediction performance (Table III.1). The systems build alternatively on the regression-based model presented above, desk forecasts, a logit regression for inflation crises, and two machine learning approaches (the Lasso model and the Random Forest classification approach). Indicated out-of-sample prediction performance refers to the period 2008–16 conditional on data available at the time the forecast would be made. For instance, to forecast inflation crises in 2008, we estimate the models based on data available in late summer 2007 (and feeding into the October 2007 WEO), including the desk inflation forecast to bring to bear all relevant available information.¹⁰ All systems use the same data and sample as the panel regression presented above.

• A “naïve” desk forecast-based early warning system performs poorly. This approach issues signals when desks forecast inflation above 15 percent in the subsequent year. This approach misses most inflation crises (79.5 percent) while sending few false alarms (2.6 percent), reaching a loss function value of -0.345.

• A more sophisticated desk forecast-based approach that takes account of forecast errors performs much better. The root mean square error of WEO forecasts can be estimated from the difference between observed and forecast inflation. If we assume that the forecast error has a normal distribution with a standard deviation equal to this root mean square error, we can estimate the implied probability of an inflation crisis given the WEO forecast. The loss minimizing threshold is a 16 percent probability that there will be an inflation crisis in the next year. This yielded a missed crises rate of 7 percent and a false alarm rate of 38.7 percent, with a value of the loss function of -0.809.

• The regression-based early warning system shows intermediate predictive performance. Based on a comparatively low loss-minimizing cutoff probability of 5 percent that there will be an inflation crisis/episode next year, it yields a missed crises rate of 5.3 percent and a false alarm rate of 54.2 percent. The realized value of the loss function is -0.743.

• Logit performs relatively poorly. Once the WEO inflation forecast is included, the only remaining variable that helps forecast inflation crises/episodes is domestic GDP growth. The loss minimizing cutoff probability is 19 percent, which yielded a missed crises rate of 54.1 percent and a rate of false alarms of 11.5 percent. This resulted in a realized value of loss function of -0.572.

• Lasso performs quite well. The loss minimizing cutoff probability for the Lasso specification is 8 percent, which yielded a missed crises rate of 26.8 percent and a rate of false alarms of 14.4 percent. This results in a realized value of the loss function of -0.771, ranking between the regression model and the sophisticated WEO.

• The Random Forests classification model outperforms all other approaches and is the recommended basis for an early warning system for inflation crises in LICs. The loss minimizing cutoff probability is 29 percent, which yielded a missed crises rate of 3.6 percent and a rate of false alarms of 34.8 percent, and at -0.846 the lowest value of the loss function among all early warning systems under consideration. This is not surprising as the random forest model also shows the highest area under the ROC curve. The superior performance of the Random Forests classification model is apparent also in out-of-sample forecasts using data available in late summer 2009 and covering the period 2010–17, with rates of missed crises and false alarms of less than 15 percent each.

^{Table III.1.}

Predictive Performance of Various Early Warning Systems

Note: All results presented above are based on out-of-sample forecasts, except for the calculation of the model specific cutoff rates which are based on a 2008–2014 training sample.I. “Naïve WEO” stands for an early warning system whereby alarms are raised when the Fall WEO inflation forecast for t+1 is above 15 percent.II. “Sophisticated WEO” assumes that the WEO forecast errors are normally distributed.III. “Panel Regression Model” is the forecast regression described above and in Appendix III.1.IV. “Logit” is a binary regression model to predict inflation crises.V. “Lasso” refers to Least Absolute Shrinkage and Selection Operator.VI. “Random forest” stands for the random forest classification model in Breiman (2001).

^{Table III.1.}

Predictive Performance of Various Early Warning Systems

			I			II			III			IV			V			VI
			Naïve WEO			Sophisticated WEO			Panel Regression Model			Logit			LASSO			Random forests
Year	# obs.	Number of crises	# early warnings	P(misse d crises)	P(false alarms)	# early warnings	P(missed crises)	P(false alarms)	# early warnings	P(missed crises)	P(false alarms)	# early warnings	P(missed crises)	P(false alarms)	# early warnings	P(missed crises)	P(false alarms)	# early warnings	P(missed crises)	P(false alarms)
2008	52	20	2	0.90	0.00	38	0.00	0.56	35	0.00	0.47				33	0.05	0.47	47	0.00	0.90
2009	53	2	2	1.00	0.04	34	0.00	0.63	32	0.00	0.59	0	1.00	0.00	0	1.00	0.00	35	0.00	0.69
2010	52	3	6	1.00	0.14	24	0.33	0.45	47	0.00	0.90	9	1.00	0.18	0	1.00	0.00	39	0.00	0.75
2011	53	8	0	1.00	0.00	32	0.00	0.53	53	0.00	1.00	31	0.13	0.53	31	0.00	0.52	21	0.13	0.32
2012	51	6	1	0.83	0.00	11	0.17	0.13	18	0.00	0.27	5	0.67	0.07	4	0.50	0.02	12	0.17	0.16
2013	51	2	4	0.50	0.06	4	0.00	0.04	30	0.00	0.57	7	0.00	0.10	18	0.00	0.33	8	0.00	0.12
2014	52	4	2	0.50	0.00	19	0.00	0.31	37	0.00	0.69	2	0.50	0.00	1	0.75	0.00	9	0.00	0.10
2015	53	5	1	0.75	0.00	40	0.00	0.73	10	0.40	0.15	2	0.80	0.02	3	0.40	0.00	12	0.00	0.15
2016	53	7	2	0.71	0.00	11	0.29	0.13	16	0.14	0.22	5	0.57	0.04	9	0.17	0.09	11	0.00	0.12
2008–16	470	57	20	0.795	0.026	213	0.070	0.387	278	0.053	0.542	61	0.541	0.115	99	0.268	0.144	194	0.036	0.348
2015–16	106	12	0	0.731	-0.345	51	0.167	0.436	26	0.250	0.181	7	0.667	0.032	12	0.264	0.044	23	0.000	0.133
Cutoff probability				n.a.			16%			5%			19%			8%			29%
Loss function 2008–16				-0.345			-0.809			-0.743			-0.572			-0.771			-0.846

Note: All results presented above are based on out-of-sample forecasts, except for the calculation of the model specific cutoff rates which are based on a 2008–2014 training sample.I. “Naïve WEO” stands for an early warning system whereby alarms are raised when the Fall WEO inflation forecast for t+1 is above 15 percent.II. “Sophisticated WEO” assumes that the WEO forecast errors are normally distributed.III. “Panel Regression Model” is the forecast regression described above and in Appendix III.1.IV. “Logit” is a binary regression model to predict inflation crises.V. “Lasso” refers to Least Absolute Shrinkage and Selection Operator.VI. “Random forest” stands for the random forest classification model in Breiman (2001).

View Table

^{Table III.1.}

Predictive Performance of Various Early Warning Systems

			I			II			III			IV			V			VI
			Naïve WEO			Sophisticated WEO			Panel Regression Model			Logit			LASSO			Random forests
Year	# obs.	Number of crises	# early warnings	P(misse d crises)	P(false alarms)	# early warnings	P(missed crises)	P(false alarms)	# early warnings	P(missed crises)	P(false alarms)	# early warnings	P(missed crises)	P(false alarms)	# early warnings	P(missed crises)	P(false alarms)	# early warnings	P(missed crises)	P(false alarms)
2008	52	20	2	0.90	0.00	38	0.00	0.56	35	0.00	0.47				33	0.05	0.47	47	0.00	0.90
2009	53	2	2	1.00	0.04	34	0.00	0.63	32	0.00	0.59	0	1.00	0.00	0	1.00	0.00	35	0.00	0.69
2010	52	3	6	1.00	0.14	24	0.33	0.45	47	0.00	0.90	9	1.00	0.18	0	1.00	0.00	39	0.00	0.75
2011	53	8	0	1.00	0.00	32	0.00	0.53	53	0.00	1.00	31	0.13	0.53	31	0.00	0.52	21	0.13	0.32
2012	51	6	1	0.83	0.00	11	0.17	0.13	18	0.00	0.27	5	0.67	0.07	4	0.50	0.02	12	0.17	0.16
2013	51	2	4	0.50	0.06	4	0.00	0.04	30	0.00	0.57	7	0.00	0.10	18	0.00	0.33	8	0.00	0.12
2014	52	4	2	0.50	0.00	19	0.00	0.31	37	0.00	0.69	2	0.50	0.00	1	0.75	0.00	9	0.00	0.10
2015	53	5	1	0.75	0.00	40	0.00	0.73	10	0.40	0.15	2	0.80	0.02	3	0.40	0.00	12	0.00	0.15
2016	53	7	2	0.71	0.00	11	0.29	0.13	16	0.14	0.22	5	0.57	0.04	9	0.17	0.09	11	0.00	0.12
2008–16	470	57	20	0.795	0.026	213	0.070	0.387	278	0.053	0.542	61	0.541	0.115	99	0.268	0.144	194	0.036	0.348
2015–16	106	12	0	0.731	-0.345	51	0.167	0.436	26	0.250	0.181	7	0.667	0.032	12	0.264	0.044	23	0.000	0.133
Cutoff probability				n.a.			16%			5%			19%			8%			29%
Loss function 2008–16				-0.345			-0.809			-0.743			-0.572			-0.771			-0.846

Note: All results presented above are based on out-of-sample forecasts, except for the calculation of the model specific cutoff rates which are based on a 2008–2014 training sample.I. “Naïve WEO” stands for an early warning system whereby alarms are raised when the Fall WEO inflation forecast for t+1 is above 15 percent.II. “Sophisticated WEO” assumes that the WEO forecast errors are normally distributed.III. “Panel Regression Model” is the forecast regression described above and in Appendix III.1.IV. “Logit” is a binary regression model to predict inflation crises.V. “Lasso” refers to Least Absolute Shrinkage and Selection Operator.VI. “Random forest” stands for the random forest classification model in Breiman (2001).