Abstract
The purpose of this paper is to compare in-sample and out-of-sample performances of three parametric and non-parametric early warning systems (EWS) for currency crises in emerging market economies (EMs). The parametric EWS achieves superior out-of-sample results compared to the non-parametric EWS, as the total misclassification error of the former is lower than that of the latter. In addition, we find that the performances of the parametric and non-parametric EWS do not improve if the policymaker becomes more prudent. From a policy perspective, the policymaker faces the standard trade-off when using EWS. Greater prudence allows the policymaker to correctly call more crisis episodes, but this comes at the cost of issuing more false alarms. The benefit of correctly calling more currency crises needs to be traded off against the cost of issuing more false alarms and of implementing corrective macroeconomic policies prematurely.
I. Introduction
The goal of this study is to compare how parametric and non-parametric early warning systems (EWS) predict in-sample and out-of-sample currency crises in emerging market economies (EMs). We look at episodes of currency crises that took place in selected EMs between January 1995 and December 2011. We define currency crises as large depreciations of the nominal exchange rate and/or extensive losses of foreign exchange reserves over a 24-month forecast horizon. In this context, a crisis occurs when the exchange market pressure index - a weighted average of one-month changes in the exchange rate and foreign exchange reserves - is more than three (country-specific) standard deviations above the country average value.2
We build parametric and non-parametric EWS. In the parametric EWS a binary crisis variable is regressed on a set of explanatory macroeconomic indicators and an indicator of political risk, using a fixed effects logit estimator, to estimate the probability of experiencing a currency crisis. We build the non-parametric EWS following IMF (2013) and Dabla-Norris and Bal Gündüz (2012). In the non-parametric EWS the crisis probability has been derived as a weighted average of crisis signals issued by a set of indicators. In both approaches, we use a panel dataset which includes macroeconomic and political risk indicators for 28 EMs, with monthly data between January 1995 and December 2011. As is standard in the EWS literature, we are interested in assessing the in-sample and out-of-sample performances of the parametric and non-parametric EWS.
This study contributes to the EWS existing literature in the following ways. First, we assess in-sample and out-of sample performances of parametric and non-parametric EWS by calculating optimal cut-off values for the crisis probabilities, while in most studies those cut-off values are selected arbitrarily. As noted by Candelon and others (2012), this matters because the cut-off value for the crisis probability determines the total misclassification error of an EWS.3 Selecting cut-off values arbitrarily implies that the quantification of the total misclassification error is also arbitrary. Second, we allow for the possibility that the policymaker’s policy preferences change. Specifically, when selecting the cut-off values for the crisis probability, we first assume that the policymaker is equally concerned about the risk of missing a crisis and that of issuing a false alarm, therefore she will assign the same weight to both risks. Then, we assume that the policymaker is relatively more concerned about the risk of missing a crisis episode than issuing a false alarm. Therefore she will err on the side of caution and attach a larger weight to the risk of missing crises than to the risk of issuing a false alarm. We do this because we are interested to assess how changing the policymaker’s preferences affects in-sample and out-of-sample performances of the parametric and non-parametric EWS. Furthermore, from an empirical point of view, we include in the EWS a specific measure of political risk which quantifies the degree of government instability in a given country, to check whether government instability is significant or not to explain the crisis incidence.
We find that in the parametric EWS, real GDP growth, the ratio between foreign exchange reserves and short term external debt, the growth rate in the stock of foreign exchange reserves, and the current account balance are all significant and negatively related with the crisis incidence. A political risk explanatory variable measuring the degree of government instability and the ratio of domestic money stock expressed in U.S. dollars and foreign exchange reserves are significant and positively related with crisis incidence. By contrast, monthly changes in real effective exchange rates and a measure of real effective exchange rate misalignment were not significant. Similarly, neither credit to the government, nor the level of foreign exchange reserves were significantly associated with crisis incidence. A measure of political risk assessing the presence of corruption in the political system was not significant to explain crisis incidence.
In the non-parametric EWS, the current account balance and the ratio between the stock of foreign exchange reserves and short-term external debt are the two most reliable indicators in issuing crisis signals. Real GDP growth, the change in foreign exchange reserves, the ratio between the domestic money stock expressed in U.S. dollars and foreign exchange reserves, and government instability appear less reliable. All in all, the results obtained with the parametric and non-parametric EWS suggest that monetary expansions, which may reflect rapid increases in credit growth, are expected to increase crisis incidence. Finally, government instability plays is significant in the parametric EWS, but does not play an important role not in the non-parametric EWS.
In terms of performance, the parametric EWS achieves superior out-of-sample results compared to the non-parametric EWS. We also find that the percentages of correctly predicted out-of-sample crisis episodes by two of the three EWS during the period January 2009-December 2011 increase substantially compared to out-of-sample periods that include the year 2008. One possible interpretation of this result is that after 2008, the most turbulent year of the global financial crisis, international investors may have been assigning greater importance to macroeconomic indicators when assessing their exposure toward EMs assets. We also find that the performance of the EWS does not improve when the policymaker is relatively more concerned about the risk of missing a crisis than the risk of issuing a false alarm.
This paper is organized as follows. Section 2 reviews the literature, while section 3 discusses the methodology used in this study. Section 4 presents the results obtained with the parametric and non parametric EWS, while in section 5 we discuss and compare the in-sample and out-of-sample performances of the parametric and non-parametric EWS. Section 6 concludes.
II. Related Literature
Following the episodes of severe financial distress in Mexico (1994-95) and Asia (1997-98), economists became interested in thinking about frameworks that could help policymakers anticipating episodes of financial crises, whose economic costs are well documented (Cerra and Saxena, 2008). We divide the EWS literature contributions relevant for this study in two groups. The first group includes those studies that propose parametric (i.e. regression-based) and non-parametric (i.e. crisis signal extraction) EWS and assess in-sample and out-of-sample performances of different EWS. Kaminsky, Lizondo and Reinhart (KLR) look at the evolution of those indicators which exhibit an unusual behavior in periods preceding financial crises. When the indicator exceeds a given threshold then that indicator is issuing a signal that a crisis could take place within the next 24 months. They find that exports, measures of real exchange rate overvaluation, GDP growth, the ratio between the money stock and foreign exchange reserves and equity prices have the best track record in terms of issuing reliable crisis signals. Berg and Pattillo (1999) test the KLR model out-of-sample and show that their regression-based approach tends to produce better forecasts compared to the KLR model. Bussiere and Fratzscher (2006) develop a multinomial logit regression-based EWS, which allows distinguishing between tranquil periods, crisis periods and post-crisis periods. They show that the multinomial logit model tends to predict better than a binomial logit model episodes of financial crisis in emerging market economies. Beckmann and others (2007) compare parametric and non-parametric EWS using a sample of 20 countries during the period included between January 1970 and April 1995. They find that the parametric EWS tends to perform better than non-parametric EWS in correctly calling financial crisis episodes. However, as noted by Candelon and others (2012), in these studies the choice of the crisis probability cut-off value is arbitrarily made and not optimally derived.
The second group of relevant EWS literature contributions for this study includes recent studies that discuss the significance of the various macroeconomic indicators to explain crisis incidence. Berkmen and others (2012) looked at the change in growth forecasts by professional economists before and after the global financial crisis. They found that countries with more leveraged domestic financial systems and rapid credit growth tended to suffer larger downward revisions to their growth forecasts, while international reserves did not play a significant role. Similarly, Blanchard and others (2010) do not find a significant role played by reserves in explaining in unexpected growth, which is defined as the forecast error for output growth in the semester from October 2008 until March 2009. Rose and Spiegel (2012) find that the only robust predictor of crisis incidence in the 2008 global financial crisis is the size of the equity market prior to the crisis. They are unable to link most of the other commonly cited causes of the global financial crisis to its incidence across countries. By contrast, Gourinchas and Obstfeld (2011) look at financial crisis episodes in advanced and emerging economies from 1973 until 2010. They find that for both advanced and emerging market economies, the two most robust predictors are domestic credit growth and real currency appreciation. In addition, they find that in emerging market economies the country’s level of foreign exchange reserves is a significant factor in determining the probability of future crises. Llaudes and others (2010) find that foreign exchange reserve holdings helped to mitigate the growth collapse in EMs provoked by the global financial crisis. Frankel and Saravelos (2012) estimate the crisis incidence of the 2008-2009 global financial crisis. They surveyed the existing literature on early warning indicators to see which leading indicators were the most reliable in explaining the crisis incidence. They find that foreign exchange reserves, the real exchange rate, credit growth, real GDP growth and the current account balance as a percentage of GDP are the most reliable indicators to explain crisis incidence and conclude that the large accumulation of foreign exchange reserves has played an important role in reducing countries’ vulnerability during the global financial crisis. The results obtained in this study are in line with the notion that the stock of foreign exchange reserves is significantly negatively related with out measure of crisis incidence.
Against this background, the contribution of this study to the EWS literature is twofold. First, we assess in-sample and out-of sample performances of parametric and non-parametric EWS by calculating optimal cut-off values for the crisis probabilities, while in most of the existing studies those cut-off values are selected arbitrarily. As noted by Candelon and others (2012), this matters because the cut-off value for the crisis probability determines the total misclassification error of an EWS.4 Selecting cut-off values arbitrarily implies that the quantification of the total misclassification error is also arbitrary. Second, we enrich the EWS literature by letting the policymaker having different policy preferences about the risk of missing a crisis and issuing a false alarm. The role of the policymaker in an EWS is to select the cut-off values for the crisis probability. In this context, we are interested to assess how changing the policymaker’s policy preferences affects the EWS performance.
III. Methodology
We build competing EWS to compare their ability to correctly predict in-sample and out-of-sample episodes of currency crises in EMs. We focus on those EMs which had at least once experienced an episode of currency crisis between January 1995 and December 2011.5 We proceed as follows. We build an exchange rate pressure index from which we derive a crisis variable that identifies episodes of currency crisis in EMs. The crisis variable is binary, as it assumes the value of one if a currency crisis takes place within the next 24 months, and 0 otherwise. Once defined the crisis variable, we construct the parametric and non-parametric EWS. For each EWS, the objective is to construct a crisis probability.
A. Parametric EWS
The parametric EWS is regression-based, where the crisis variable (or crisis incidence) is regressed on a set of selected macroeconomic indicators of emerging market economies, using a fixed effects logit estimator. A crisis probability is then calculated with the coefficient estimates obtained from the regression. Following Bussiere and Fratzscher (2006), we assume that there are N countries, i = 1, 2, …, N, that we observe during T periods t = 1, 2, …, T. For each country and month, we observe a forward-looking crisis variable Yit that can assume as values only 0 (non-crisis) or 1 (crisis). To derive the crisis binary variable, we follow Kaminsky and others (1998) and build an exchange rate pressure index.6 The exchange rate pressure index for country i at time t (ERPIi,t) is defined as a weighted average between the monthly change in the nominal exchange rate and that in the stock of foreign exchange reserves.
where ei,t denotes the nominal exchange rate of country i’s currency against the U.S. dollar at time t, while fxri,t denotes the stock of foreign exchange reserves of country i at time t. Finally, σei and σfxri are the standard deviations of the nominal exchange rate and foreign exchange reserves in country i, respectively.
As a next step, we define a currency crisis hitting country i at time t, CCi,t, as a binary variable that can assume either 1 (when the ERPI is above its mean by a number of standard deviations) or 0 (otherwise):
where ϕ is arbitrarily set equal to 3, and σERPIi,t is the standard deviation of the exchange rate pressure index of country i.7 Next, the variable CCi,t is converted into the forward-looking crisis variable Yi,t which is defined as follows
The forward-looking crisis variable Yi,t is equal to 1 if within the next 24 months a currency crisis is observed in country i, and to 0 otherwise. As in Bussiere and Fratzscher (2006), the crisis definition adopted in this study allows to capture both successful and non-successful speculative attacks to a given currency.
We define Pr(Yi,t = 1) as the probability of country i to experience a currency crisis at time t. We estimate the probability of a currency crisis with a fixed effects logit model whereby the probability of a currency crisis is a non-linear function of the macroeconomic indicators X:
Condition (4) expresses the unconditional probability that country i experiences a currency crisis at time t as a function of the macroeconomic indicators. To obtain the estimates of β, we regress the crisis binary variable Yi,t on the macroeconomic indicators in the period between January 1995 and December 2007. Then, based on (4), we derive currency crises probabilities.
B. Non-parametric EWS
Following Dabla-Norris and Bal Gündüz (2012) and IMF (2013), in the non-parametric EWS the crisis probability Pt is calculated as a weighted average of crisis signals issued by a set of selected EMs macroeconomic indicators. To establish when an indicator is issuing a crisis signal, we need to choose a threshold. In most of the EWS literature, thresholds are arbitrarily chosen. Instead, like Candelon and others (2012) and IMF (2013), for each indicator we determine an optimal threshold because we want to reduce as much as possible the forecasting error. For instance, suppose that the threshold has been arbitrarily set very high. Suppose also that the indicator assumes a value which is lower than the arbitrarily chosen threshold and that a crisis occurs. In that case, the indicator does not issue a crisis signal, yet a crisis occurs. We refer to this kind of forecasting error as type 1 error (missing a crisis because the threshold has been set too high). Alternatively, suppose that the indicator assumes values which are higher than the threshold, which has been arbitrarily set too low and suppose that a tranquil period follows. In that case, the indicator issues a crisis signal, yet no crisis has materialized. We refer to this forecasting error as type 2 error (issuing a false alarm). In order to minimize the sum of errors associated to a given threshold, for each macroeconomic indicator we choose an optimal threshold. In correspondence of the optimal threshold, the sum between type 1 and type 2 errors (total misclassification error) is minimized. Hence, the threshold is optimal in the sense that it discriminates best between crisis and non-crisis signals. There is no other threshold that, for a given macroeconomic indicator, separates the crisis observations from non-crisis ones better than the optimal threshold. Summing up, choosing an optimal threshold allows minimizing the forecasting errors; this is not possible if instead the threshold is arbitrarily chosen.
To obtain the crisis probability with the non-parametric EWS, we proceed in several steps. First, for each macroeconomic indicator Xi, we derive an optimal threshold that minimizes the total misclassification error. Specifically, for each indicator Xi we separate the observations assumed by the indicator in the crisis periods from those assumed in the non-crisis periods, into two subsamples. For both subsamples we calculate the cumulative density functions, and the associated total misclassification error, which is given by the sum between type 1 error (missing a crisis) and type 2 error (issuing a false alarm). The optimal threshold
The total misclassification error is defined as follows:
where
Second, we need to map indicator values into zero-one scores. To do that, we convert each indicator Xi,t into a binary variable fi,t: when Xi,t assumes values equal to or higher than its optimal threshold
As a next step, we need to choose weights to aggregate crisis signals issued by the indicators into a crisis probability. Specifically, when fi,t = 1, the indicator Xi is issuing a crisis signal, and the binary variable fi,t is multiplied by the weight assigned to the indicator Xi. The weight of the indicator Xi is a function of the total misclassification error zi and is defined as:
Intuitively, condition (6) implies that the higher the weight of an indicator Xi, the lower its total misclassification error zi, hence the higher its reliability in issuing crisis signals. It follows that the more reliable an indicator is, the higher its contribution in calculating the crisis probability should be.
Next, we use the weight wi to construct the crisis probability for each sector of the economy Î j,t:
where the subscript j = 1, 2, …, J denotes the sectors of the economy and where each crisis signal fi,t is multiplied by the weight wi. Put differently, Πj,t is a weighted average of the crisis signals fi,t issued by the macroeconomic indicators of a given sector. The higher Πj,t, the higher the probability that a crisis will originate from sector j, hence the higher the contribution of sector j to the likelihood of experiencing a currency crisis. The sectors of the economy that we consider are the external sector, the domestic sector and the financial sector.
Finally, we aggregate the sectoral crisis probability Î j,t into a measure of crisis probability Pt for the economy as a whole:
where σj is the weight assigned to sector j of the economy.9 The crisis probability Pt is calculated as a weighted average of crisis probabilities associated to each sector of the economy Πj,t. The higher Pt, the higher the probability to experience a currency crisis.
IV. Results
A. Parametric EWS
We begin estimating a fixed effects logit model where the dependent variable, or crisis incidence, Yit defined in (3) is regressed on a set of macroeconomic indicators, which are believed to be relevant in anticipating currency crises in EMs. For the choice of the explanatory variables, we follow a general-to-specific approach to obtain a parsimonious specification of the model, where the explanatory variables have the desired sign and are significant in explaining crisis incidence. The explanatory variables are real GDP growth (ΔRGDP), the growth rate in the stock of foreign exchange reserves (ΔFXR), the ratio between the current account balance and nominal GDP (CAB/Y), the ratio between the stock of foreign exchange reserves (henceforth reserves) and short-term external debt (e.g. maturing within one year, FXR/STED), the ratio between M2 expressed in U.S. dollars and the stock of reserves (M2/FXR), and on a political risk variable which measures the degree of government instability (GOVT. INST). We consider three separate estimation periods, as we are interested to check how the coefficient estimates of the explanatory variables change if the global financial crisis of 2008-2009 is included or not in the estimation period. The three estimation periods are: January 1995-December 2006, January 1995-December 2007 and January 1995-December 2008.
Table 1 reports the coefficient estimates obtained with the fixed effects logit model.10 All the coefficient estimates are significant in explaining crisis incidence and have the expected sign.11
Parametric EWS: Fixed Effects Logit Model
p < 0.01
Parametric EWS: Fixed Effects Logit Model
| Jan. 95 - Dec. 06 | Jan. 95 - Dec. 07 | Jan. 95 - Dec. 08 | |
|---|---|---|---|
| Dependent variable: Yit | |||
| ΔRGDP | −0.120 | −0.061 | −0.058 |
| (0.021)* | (0.018)* | (0.017)* | |
| ΔFXR | −0.021 | −0.040 | −0.023 |
| (0.008)* | (0.007)* | (0.006)* | |
| CAB/Y | −0.348 | −0.267 | −0.232 |
| (0.024)* | (0.019)* | (0.016)* | |
| FXR/STED | −0.012 | −0.008 | −0.008 |
| (0.001)* | (0.001)* | (0.001)* | |
| GOVT. INST. | 0.112 | 0.119 | 0.125 |
| (0.038)* | (0.033)* | (0.029)* | |
| M2/FXR | 0.129 | 0.159 | 0.185 |
| (0.039)* | (0.031)* | (0.030)* | |
| Observations | 3113 | 3568 | 3880 |
| Log-Likelihood | −889 | −1232 | −1449 |
p < 0.01
Parametric EWS: Fixed Effects Logit Model
| Jan. 95 - Dec. 06 | Jan. 95 - Dec. 07 | Jan. 95 - Dec. 08 | |
|---|---|---|---|
| Dependent variable: Yit | |||
| ΔRGDP | −0.120 | −0.061 | −0.058 |
| (0.021)* | (0.018)* | (0.017)* | |
| ΔFXR | −0.021 | −0.040 | −0.023 |
| (0.008)* | (0.007)* | (0.006)* | |
| CAB/Y | −0.348 | −0.267 | −0.232 |
| (0.024)* | (0.019)* | (0.016)* | |
| FXR/STED | −0.012 | −0.008 | −0.008 |
| (0.001)* | (0.001)* | (0.001)* | |
| GOVT. INST. | 0.112 | 0.119 | 0.125 |
| (0.038)* | (0.033)* | (0.029)* | |
| M2/FXR | 0.129 | 0.159 | 0.185 |
| (0.039)* | (0.031)* | (0.030)* | |
| Observations | 3113 | 3568 | 3880 |
| Log-Likelihood | −889 | −1232 | −1449 |
p < 0.01
In all the specifications, real GDP growth, the ratio between reserves and short term external debt, the growth rate in the stock of reserves and the current account balance are all significant and negatively related with crisis incidence. By contrast, a political risk explanatory variable measuring the degree of government instability is significant and positively related with crisis incidence.12 Intuitively, doubts about government stability may create uncertainty about future macroeconomic policy, trigger portfolio outflows and currency depreciation.13 The ratio between the domestic money stock expressed in U.S. dollars and the stock of reserves is also significant and positively related with crisis incidence. This result is in line with the work of Calvo and Mendoza (1996), who looked at the 1994 Mexican financial crisis and observed that in Mexico the domestic money stock expressed in U.S. dollars increased much faster than that of gross foreign exchange reserves during in the five years before the crisis. A persistently rising ratio between M2 and reserves indicates that a credit expansion is taking place, which is incompatible with a fixed exchange rate regime.14
In other (not reported) regressions, we replaced among the explanatory variables the ratio between M2 and reserves with private credit as a percentage of nominal GDP. Private credit as a percentage of nominal GDP, expressed in differences, is significant and positively associated with the crisis incidence and have the expected sign. By contrast, the level of private credit as a percentage of nominal GDP does not have the expected sign when the estimation sample is set between January 1995 and December 2006.
Other explanatory variables were not significant and are not presented in the table 1. Monthly changes in real effective exchange rates and a measure of real effective exchange rate misalignment were not significant.15 Similarly, neither credit to the government as a percentage of nominal GDP (expressed in both levels and differences), nor the level of reserves were significant. A measure of political risk assessing the presence of corruption in the political system was not found to be significant to explain crisis incidence. All in all, the coefficient estimates obtained with all specifications suggest that monetary expansions, which may reflect rapid increases in credit growth, are expected to increase crisis incidence, hence the likelihood of exchange rate depreciation. Summing up, the results obtained with the fixed effect logit regression appear to be in line with some of the empirical studies on currency crises. Frankel and Saravelos (2012), Gourinchas and Obstfeld (2011), and Llaudes and others (2010) find that reserves play an important role in reducing the likelihood of experiencing currency crises.
B. Non-parametric EWS
In this section, we calculate optimal thresholds and weights of each macroeconomic indicator that we use to construct the crisis probability with the non-parametric EWS. We group the indicators into three different groups: the domestic sector (which includes the real GDP growth and the measure of government instability), the external sector (which includes the growth rate of the stock of foreign exchange reserves, the ratio between the stock of foreign reserves and the stock of short -term external debt, the current account balance in percent of nominal GDP, and the monthly differences in the real effective exchange rate), and the financial sector (which includes the ratio between the money stock expressed in U.S. dollars and the stock of foreign reserves, credit to the private sector as a percentage of nominal GDP in levels and differences, and the level of credit to the government as a percentage of nominal GDP).
We are interested to check whether optimal thresholds and weights of each indicator change if the global financial crisis of 2008-2009 is included or not in the estimation period. We consider three different estimation periods: January 1995-December 2006, January 1995-December 2007 and January 1995-December 2008. For each indicator, tables 2-4 report optimal thresholds, weights, as well as type 1 and type 2 errors calculated in correspondence of the optimal threshold.
Non-parametric EWS: January 1995-December 2006
Non-parametric EWS: January 1995-December 2006
| Estimation period: January 1995-December 2006 | |||||
|---|---|---|---|---|---|
| Direction to be safe | Type 1 error | Type 2 error | Threshold | Weight | |
| ΔRGDP | > | 0.72 | 0.10 | 0.45 | 0.20 |
| ΔFXR | > | 0.65 | 0.16 | −1.80 | 0.23 |
| CAB/Y | > | 0.28 | 0.45 | −1.48 | 0.36 |
| FXR/STED(in%) | > | 0.59 | 0.17 | 90.74 | 0.32 |
| GOVT. INST. | < | 0.27 | 0.65 | 2.00 | 0.05 |
| M2/FXR | < | 0.58 | 0.30 | 3.43 | 0.14 |
| Other indicators | |||||
| CPR/Y | < | 0.00 | 0.86 | 16.77 | 0.15 |
| ΔCPR/Y | < | 0.88 | 0.05 | 0.26 | 0.07 |
| ΔREER | < | 0.31 | 0.56 | 0.06 | 0.14 |
| ΔCPB/Y | < | 0.35 | 0.56 | −0.01 | 0.08 |
Non-parametric EWS: January 1995-December 2006
| Estimation period: January 1995-December 2006 | |||||
|---|---|---|---|---|---|
| Direction to be safe | Type 1 error | Type 2 error | Threshold | Weight | |
| ΔRGDP | > | 0.72 | 0.10 | 0.45 | 0.20 |
| ΔFXR | > | 0.65 | 0.16 | −1.80 | 0.23 |
| CAB/Y | > | 0.28 | 0.45 | −1.48 | 0.36 |
| FXR/STED(in%) | > | 0.59 | 0.17 | 90.74 | 0.32 |
| GOVT. INST. | < | 0.27 | 0.65 | 2.00 | 0.05 |
| M2/FXR | < | 0.58 | 0.30 | 3.43 | 0.14 |
| Other indicators | |||||
| CPR/Y | < | 0.00 | 0.86 | 16.77 | 0.15 |
| ΔCPR/Y | < | 0.88 | 0.05 | 0.26 | 0.07 |
| ΔREER | < | 0.31 | 0.56 | 0.06 | 0.14 |
| ΔCPB/Y | < | 0.35 | 0.56 | −0.01 | 0.08 |
Non-parametric EWS: January 1995 December 2007
Non-parametric EWS: January 1995 December 2007
| Estimation period: January 1995-December 2007 | |||||
|---|---|---|---|---|---|
| Direction to be safe | Type 1 error | Type 2 error | Threshold | Weight | |
| ΔRGDP | > | 0.61 | 0.29 | 0.45 | 0.15 |
| ΔFXR | > | 0.70 | 0.15 | −1.93 | 0.17 |
| CAB/Y | > | 0.29 | 0.44 | −1.48 | 0.35 |
| FXR/STED(in%) | > | 0.56 | 0.22 | 108.31 | 0.26 |
| GOVT. INST. | < | 0.42 | 0.59 | 2.50 | 0.05 |
| M2/FXR | < | 0.46 | 0.42 | 2.83 | 0.14 |
| Other indicators | |||||
| CPR/Y | < | 0.00 | 0.87 | 16.3 | 0.14 |
| ΔCPR/Y | < | 0.78 | 0.12 | 0.19 | 0.11 |
| ΔREER | < | 0.37 | 0.55 | 0.06 | 0.08 |
| ΔCPB/Y | < | 0.27 | 0.66 | 0.00 | 0.06 |
Non-parametric EWS: January 1995 December 2007
| Estimation period: January 1995-December 2007 | |||||
|---|---|---|---|---|---|
| Direction to be safe | Type 1 error | Type 2 error | Threshold | Weight | |
| ΔRGDP | > | 0.61 | 0.29 | 0.45 | 0.15 |
| ΔFXR | > | 0.70 | 0.15 | −1.93 | 0.17 |
| CAB/Y | > | 0.29 | 0.44 | −1.48 | 0.35 |
| FXR/STED(in%) | > | 0.56 | 0.22 | 108.31 | 0.26 |
| GOVT. INST. | < | 0.42 | 0.59 | 2.50 | 0.05 |
| M2/FXR | < | 0.46 | 0.42 | 2.83 | 0.14 |
| Other indicators | |||||
| CPR/Y | < | 0.00 | 0.87 | 16.3 | 0.14 |
| ΔCPR/Y | < | 0.78 | 0.12 | 0.19 | 0.11 |
| ΔREER | < | 0.37 | 0.55 | 0.06 | 0.08 |
| ΔCPB/Y | < | 0.27 | 0.66 | 0.00 | 0.06 |
Non-parametric EWS: January 1995-December 2008
Non-parametric EWS: January 1995-December 2008
| Estimation period: January 1995-December 2008 | |||||
|---|---|---|---|---|---|
| Direction to be safe | Type 1 error | Type 2 error | Threshold | Weight | |
| ΔRGDP | > | 0.78 | 0.10 | 0.59 | 0.13 |
| ΔFXR | > | 0.69 | 0.16 | −1.80 | 0.17 |
| CAB/Y | > | 0.28 | 0.46 | −1.36 | 0.36 |
| FXR/STED(in%) | > | 0.58 | 0.22 | 108.29 | 0.24 |
| GOVT. INST. | < | 0.71 | 0.26 | 5.00 | 0.03 |
| M2/FXR | < | 0.46 | 0.42 | 2.86 | 0.14 |
| Other indicators | |||||
| CPR/Y | < | 0.45 | 0.43 | 39.3 | 0.14 |
| ΔCPR/Y | < | 0.73 | 0.16 | 0.14 | 0.11 |
| ΔREER | < | 0.12 | 0.85 | −0.01 | 0.04 |
| ΔCPB/Y | < | 0.21 | 0.74 | −0.03 | 0.05 |
Non-parametric EWS: January 1995-December 2008
| Estimation period: January 1995-December 2008 | |||||
|---|---|---|---|---|---|
| Direction to be safe | Type 1 error | Type 2 error | Threshold | Weight | |
| ΔRGDP | > | 0.78 | 0.10 | 0.59 | 0.13 |
| ΔFXR | > | 0.69 | 0.16 | −1.80 | 0.17 |
| CAB/Y | > | 0.28 | 0.46 | −1.36 | 0.36 |
| FXR/STED(in%) | > | 0.58 | 0.22 | 108.29 | 0.24 |
| GOVT. INST. | < | 0.71 | 0.26 | 5.00 | 0.03 |
| M2/FXR | < | 0.46 | 0.42 | 2.86 | 0.14 |
| Other indicators | |||||
| CPR/Y | < | 0.45 | 0.43 | 39.3 | 0.14 |
| ΔCPR/Y | < | 0.73 | 0.16 | 0.14 | 0.11 |
| ΔREER | < | 0.12 | 0.85 | −0.01 | 0.04 |
| ΔCPB/Y | < | 0.21 | 0.74 | −0.03 | 0.05 |
Tables 2-4 show that across periods, the current account balance as a percentage of nominal GDP and the ratio between the stock of foreign exchange reserves and short-term external debt are the two most reliable indicators in issuing crisis signals as they have the largest weights. The real GDP growth rate, the change in foreign exchange reserves, and the ratio between the domestic money stock expressed in U.S. dollars and foreign exchange reserves also contribute to the crisis probability, but carry lower weights. The lowest weight is assigned to government instability. The weights of all the indicators used (except those of the level of current account balance as a percentage of nominal GDP and the difference in credit to the government as a percentage of nominal GDP) decline when the estimation period includes the 2008-09 global financial crisis. By construction, the lower the weight of a given indicator, the higher its total misclassification error, the lower its contribution to calculate the crisis probability and its reliability in issuing crisis signals.16 The results in tables 2-4 show that the reliability of the indicators in issuing crisis signals declines when the estimation period includes the global financial crisis of 2008-09. The declining reliability in issuing crisis signals may reflect the fact that the EWS presented in this paper are based mainly on ‘traditional’ macroeconomic indicators, and do not include those factors that are believed to have played a role in the development of the global financial crisis. For instance, neither of the EWS presented in this paper include measures of financial vulnerabilities,17 investors’ preferences,18 cross-border financial linkages, and financial contagion.19 In addition, the declining ability of the EMs macroeconomic indicators in issuing crisis signals may also reflect that fact that for a number of EMs, the global financial crisis of 2008-09 was an externally driven event.20
Tables 2-4 also report optimal threshold values for each indicator. When the global financial crisis is included in the in-sample period, the threshold of the ratio between foreign exchange reserves and short-term external debt is higher as it raises from 90 percent to 108 percent. Intuitively, a higher threshold for the ratio between foreign exchange reserves and short-terms external debt means that after the beginning of the global financial crisis, EMs needed more reserves relatively to their short-term external debt to be better insured against the risk of experiencing disruptive capital outflows.
Summing up, both the parametric and non-parametric EWS assign an important role to the external sector indicators. Government instability is significant and positively associated with crisis incidence only in the parametric EWS. From a policy perspective, the results imply that those EMs with good macroeconomic indicators and that had improved their policy fundamentals in the pre-crisis period, are likely to have suffered less from the impact of the global financial crisis.21
V. Comparing Performances of EWS
We now turn to assess which of the EWS used in this study has the highest ability in correctly predicting in-sample and out-of-sample crisis and tranquil periods. We proceed as follows. First, we derive in-sample and out-of-sample crisis probabilities for the parametric and the non-parametric EWS. Second, we determine optimal cut-off values for the crisis probabilities in the same way we determined optimal thresholds for the individual indicators.22 Then, in correspondence of the optimal cut-off values, we check how the parametric and non-parametric EWS perform in correctly predicting in-sample and out-of-sample crisis episodes and tranquil periods. We compare the performance of three different EWS:
At this stage, to assess the performance of the non-parametric EWS, we introduce an important assumption about the policymaker’s policy preferences. Specifically, we will initially assume that the policymaker assigns the same weight to type 1 and type 2 errors. Put differently, the policymaker dislikes in equal way the risk of missing a crisis and that of issuing a false alarm. The total misclassification error is expressed as follows:
where
In a second stage, we assume that the policymaker becomes more cautious, in the sense that she dislikes more missing a crisis episodes than issuing a false alarm. This can be motivated by the consideration that the policymaker thinks that missing a crisis can potentially be much costlier than issuing a false alarm in terms of foregone output. Therefore, compared to (10), the total misclassification error is amended as follows:
We make these assumptions about the policymaker’s policy preferences because we are interested to check how the non-parametric EWS performance changes when the policymaker’s preferences change as well.
A. The Policymaker Assigns Equal Weights To Type 1 and Type 2 Errors
We derive optimal cut-off values for the crisis probabilities of the parametric and non-parametric EWS. For each crisis probability, crisis period observations have been collected in a subsample and separated from tranquil period observations. For each subsample (e.g. crisis and tranquil periods), cumulative density functions, as well as type 1 and type 2 errors have been calculated. An optimal cut-off value for the crisis probability has been chosen such that, in its correspondence, the sum between type 1 and type 2 errors is minimized. Put differently, there is no other cut-off value that separates better the crisis period subsample from the tranquil period subsample than the optimal cut-off value. The policymaker assigns the same weight to the risk of missing crises and to that of issuing false alarms.
In table 5 we report the in-sample and out-of-sample performances of the parametric and non-parametric EWS when the policymaker assigns the same weight to the risk of missing crises and to that of issuing false alarms. For each EWS we report:
The percentage of crisis episodes correctly called, calculated as the ratio between the number of correctly predicted crisis episodes and the total number of crisis episodes observed;
The percentage of tranquil periods correctly called, calculated as the ratio between correctly predicted non-crisis episodes and the total number of non-crisis episodes observed;
The probability of crisis given an alarm issued;
The probability of crisis given no alarm issued;
The total misclassification error, calculated as the sum of the ratio between the number of missed crises over the number of observed crisis episodes and the ratio between the number of false alarms over the number of observed tranquil periods;
The Policymaker Assigns Equal Weights To Type 1 and Type 2 Errors
The Policymaker Assigns Equal Weights To Type 1 and Type 2 Errors
| Jan 95 - Dec 06 | Jan 95 - Dec 07 | Jan 95 - Dec 08 | |||||||
|---|---|---|---|---|---|---|---|---|---|
| In-sample | P | NP1 | NP2 | P | NP1 | NP2 | P | NP1 | NP2 |
| Probability critical threshold | 0.25 | 0.26 | 0.35 | 0.39 | 0.26 | 0.31 | 0.44 | 0.27 | 0.27 |
| % of crises correctly called | 59.1 | 52.0 | 46.0 | 56.5 | 48.9 | 48.5 | 53.6 | 49.9 | 60.0 |
| % of tranquil periods correctly called | 65.6 | 79.4 | 81.0 | 67.1 | 81.3 | 79.9 | 70.1 | 79.6 | 72.5 |
| Probability of crisis given alarm | 45.0 | 54.6 | 53.6 | 47.0 | 57.4 | 55.5 | 48.3 | 56.1 | 53.4 |
| Probability of crisis given no alarm | 22.9 | 22.4 | 24.1 | 25.1 | 24.5 | 25.0 | 25.7 | 24.8 | 22.4 |
| Total misclassification error | 75.3 | 68.3 | 73.0 | 76.4 | 69.8 | 71.6 | 76.3 | 70.6 | 67.5 |
| Jan 07 - Dec 11 | Jan 08 - Dec 11 | Jan 09 - Dec 11 | |||||||
| Out-of-sample | P | NP1 | NP2 | P | NP1 | NP2 | P | NP1 | NP2 |
| Probability critical threshold | 0.25 | 0.26 | 0.35 | 0.39 | 0.26 | 0.31 | 0.44 | 0.27 | 0.27 |
| % of crises correctly called | 61.7 | 44.5 | 57.3 | 44.1 | 42.6 | 60.8 | 95.2 | 41.3 | 93.7 |
| % of tranquil periods correctly called | 74.4 | 71.5 | 69.6 | 85.0 | 71.2 | 65.4 | 72.5 | 72.0 | 51.7 |
| Probability of crisis given alarm | 37.7 | 28.1 | 32.1 | 34.5 | 21.0 | 23.9 | 18.8 | 8.9 | 11.5 |
| Probability of crisis given no alarm | 11.4 | 16.3 | 13.3 | 10.5 | 12.6 | 9.7 | 0.4 | 5.2 | 0.8 |
| Total misclassification error | 63.9 | 84.0 | 73.1 | 70.9 | 86.1 | 73.9 | 32.3 | 86.8 | 54.6 |
The Policymaker Assigns Equal Weights To Type 1 and Type 2 Errors
| Jan 95 - Dec 06 | Jan 95 - Dec 07 | Jan 95 - Dec 08 | |||||||
|---|---|---|---|---|---|---|---|---|---|
| In-sample | P | NP1 | NP2 | P | NP1 | NP2 | P | NP1 | NP2 |
| Probability critical threshold | 0.25 | 0.26 | 0.35 | 0.39 | 0.26 | 0.31 | 0.44 | 0.27 | 0.27 |
| % of crises correctly called | 59.1 | 52.0 | 46.0 | 56.5 | 48.9 | 48.5 | 53.6 | 49.9 | 60.0 |
| % of tranquil periods correctly called | 65.6 | 79.4 | 81.0 | 67.1 | 81.3 | 79.9 | 70.1 | 79.6 | 72.5 |
| Probability of crisis given alarm | 45.0 | 54.6 | 53.6 | 47.0 | 57.4 | 55.5 | 48.3 | 56.1 | 53.4 |
| Probability of crisis given no alarm | 22.9 | 22.4 | 24.1 | 25.1 | 24.5 | 25.0 | 25.7 | 24.8 | 22.4 |
| Total misclassification error | 75.3 | 68.3 | 73.0 | 76.4 | 69.8 | 71.6 | 76.3 | 70.6 | 67.5 |
| Jan 07 - Dec 11 | Jan 08 - Dec 11 | Jan 09 - Dec 11 | |||||||
| Out-of-sample | P | NP1 | NP2 | P | NP1 | NP2 | P | NP1 | NP2 |
| Probability critical threshold | 0.25 | 0.26 | 0.35 | 0.39 | 0.26 | 0.31 | 0.44 | 0.27 | 0.27 |
| % of crises correctly called | 61.7 | 44.5 | 57.3 | 44.1 | 42.6 | 60.8 | 95.2 | 41.3 | 93.7 |
| % of tranquil periods correctly called | 74.4 | 71.5 | 69.6 | 85.0 | 71.2 | 65.4 | 72.5 | 72.0 | 51.7 |
| Probability of crisis given alarm | 37.7 | 28.1 | 32.1 | 34.5 | 21.0 | 23.9 | 18.8 | 8.9 | 11.5 |
| Probability of crisis given no alarm | 11.4 | 16.3 | 13.3 | 10.5 | 12.6 | 9.7 | 0.4 | 5.2 | 0.8 |
| Total misclassification error | 63.9 | 84.0 | 73.1 | 70.9 | 86.1 | 73.9 | 32.3 | 86.8 | 54.6 |
The main point is that not always in-sample performances are superior to out-of-sample performances. The in-sample total misclassification error of the parametric EWS ranges between 75.3% and 76.4%, while the out-of-sample error ranges between 32.3% and 70.9%. The out-of-sample error of the parametric EWS declines because the percentages of correctly called crisis episodes and tranquil periods rise compared to the in-sample period. In this context, it is particularly striking to observe that in two cases out of three (P and NP2), when the out-of-sample period is restricted to the period between January 2009 and December 2011, the ability to correctly call out-of-sample crisis episodes is very high (P and NP2). This is because the number of missed out-of-sample crisis is very low.23
The parametric EWS is more reliable in correctly predicting out-of-sample crisis episodes than the non-parametric EWS. The probability of an out-of-sample crisis episode when the parametric EWS issues an alarm is higher than the probability of an out-of-sample crisis when the non-parametric EWS issue an alarm. The parametric EWS tends to have lower probabilities than the non-parametric EWS of missing crisis episodes when the EWS fails to issue an alarm.
Overall, when the policymaker assigns the same weight to the risk of missing crises and to that of issuing false alarms, the results show that the parametric EWS achieves superior out-of-sample results compared to the non-parametric EWS, as the total misclassification error of the former is lower than that of the latter. In addition, the striking increase in the percentages of correctly called out-of-sample crisis episodes by two of the three EWS considered during the period January 2009 - December 2011 (95.2% and 93.7% respectively) shows that EWS mainly relying on traditional macroeconomic indicators perform quite well. Indeed, following the most turbulent year of the global financial crisis, 2008, the two EWS improve markedly their out-of-sample performance, since during the periods January 2007-December 2011 and January 2008-December 2011 the percentages of correctly called out-of-sample crisis range between 44% and 61%. One possible interpretation of this result is that following 2008, international investors may have been assigning more importance to macroeconomic indicators when assessing their exposure toward EMs assets.
B. A More Cautious Policymaker
We now turn to assume that the policymaker assigns a relatively higher weight to the risk of missing crisis episodes than to the risk of issuing false alarms. This will induce the policymaker to be more cautious and select a lower critical value for the crisis probability compared to the previous case when she assigned the same weight to type 1 and type 2 errors. The main point is that the performances of the parametric and non-parametric EWS do not improve if the policymaker becomes more cautious. Table 6 shows that the total misclassification error tends to increase when the policymaker assigns a larger weight to the risk of missing a crisis than to that of issuing a false alarm. This implies that if the policymaker becomes more cautious, hence more averse to missing crisis episodes, the benefit of correctly calling more in-sample and out-of-sample crisis episodes tends to be more than offset by the cost of issuing more in-sample and out-of-sample false alarms.
A More Cautious Policymaker
A More Cautious Policymaker
| Jan 95 - Dec 06 | Jan 95 - Dec 07 | Jan 95 - Dec 08 | |||||||
|---|---|---|---|---|---|---|---|---|---|
| In-sample | P | NP1 | NP2 | P | NP1 | NP2 | P | NP1 | NP2 |
| Probability critical threshold | 0.03 | 0.03 | 0.21 | 0.11 | 0.21 | 0.17 | 0.12 | 0.19 | 0.22 |
| % of crises correctly called | 87.4 | 82.3 | 79.0 | 86.8 | 74.3 | 78.9 | 87.7 | 77.3 | 74.5 |
| % of tranquil periods correctly called | 36.2 | 39.8 | 46.0 | 37.2 | 54.7 | 45.7 | 34.4 | 52.4 | 53.6 |
| Probability of crisis given alarm | 39.5 | 39.4 | 41.1 | 41.7 | 45.9 | 42.9 | 41.2 | 46.0 | 45.7 |
| Probability of crisis given no alarm | 14.2 | 17.5 | 17.9 | 15.5 | 19.5 | 19.3 | 15.8 | 18.5 | 20.0 |
| Total misclassification error | 76.4 | 77.9 | 75.0 | 76.0 | 70.9 | 75.4 | 77.9 | 70.3 | 72.0 |
| Jan 07 - Dec 11 | Jan 08 - Dec 11 | Jan 09 - Dec 11 | |||||||
| Out-of-sample | P | NP1 | NP2 | P | NP1 | NP2 | P | NP1 | NP2 |
| Probability critical threshold | 0.03 | 0.03 | 0.21 | 0.11 | 0.21 | 0.17 | 0.12 | 0.19 | 0.22 |
| % of crises correctly called | 85.2 | 76.9 | 76.6 | 76.5 | 76.0 | 79.9 | 96.8 | 96.8 | 96.8 |
| % of tranquil periods correctly called | 30.8 | 37.7 | 44.9 | 52.9 | 48.6 | 40.4 | 26.1 | 46.1 | 42.6 |
| Probability of crisis given alarm | 23.6 | 23.6 | 25.9 | 22.5 | 20.9 | 19.3 | 8.0 | 10.7 | 10.1 |
| Probability of crisis given no alarm | 10.8 | 13.4 | 11.6 | 7.4 | 8.1 | 8.2 | 0.8 | 0.5 | 0.5 |
| Total misclassification error | 84.1 | 85.5 | 78.5 | 70.6 | 75.4 | 79.7 | 77.0 | 57.0 | 60.5 |
A More Cautious Policymaker
| Jan 95 - Dec 06 | Jan 95 - Dec 07 | Jan 95 - Dec 08 | |||||||
|---|---|---|---|---|---|---|---|---|---|
| In-sample | P | NP1 | NP2 | P | NP1 | NP2 | P | NP1 | NP2 |
| Probability critical threshold | 0.03 | 0.03 | 0.21 | 0.11 | 0.21 | 0.17 | 0.12 | 0.19 | 0.22 |
| % of crises correctly called | 87.4 | 82.3 | 79.0 | 86.8 | 74.3 | 78.9 | 87.7 | 77.3 | 74.5 |
| % of tranquil periods correctly called | 36.2 | 39.8 | 46.0 | 37.2 | 54.7 | 45.7 | 34.4 | 52.4 | 53.6 |
| Probability of crisis given alarm | 39.5 | 39.4 | 41.1 | 41.7 | 45.9 | 42.9 | 41.2 | 46.0 | 45.7 |
| Probability of crisis given no alarm | 14.2 | 17.5 | 17.9 | 15.5 | 19.5 | 19.3 | 15.8 | 18.5 | 20.0 |
| Total misclassification error | 76.4 | 77.9 | 75.0 | 76.0 | 70.9 | 75.4 | 77.9 | 70.3 | 72.0 |
| Jan 07 - Dec 11 | Jan 08 - Dec 11 | Jan 09 - Dec 11 | |||||||
| Out-of-sample | P | NP1 | NP2 | P | NP1 | NP2 | P | NP1 | NP2 |
| Probability critical threshold | 0.03 | 0.03 | 0.21 | 0.11 | 0.21 | 0.17 | 0.12 | 0.19 | 0.22 |
| % of crises correctly called | 85.2 | 76.9 | 76.6 | 76.5 | 76.0 | 79.9 | 96.8 | 96.8 | 96.8 |
| % of tranquil periods correctly called | 30.8 | 37.7 | 44.9 | 52.9 | 48.6 | 40.4 | 26.1 | 46.1 | 42.6 |
| Probability of crisis given alarm | 23.6 | 23.6 | 25.9 | 22.5 | 20.9 | 19.3 | 8.0 | 10.7 | 10.1 |
| Probability of crisis given no alarm | 10.8 | 13.4 | 11.6 | 7.4 | 8.1 | 8.2 | 0.8 | 0.5 | 0.5 |
| Total misclassification error | 84.1 | 85.5 | 78.5 | 70.6 | 75.4 | 79.7 | 77.0 | 57.0 | 60.5 |
From a macroeconomic policy perspective, the policymaker faces then the standard trade-off when using EWS. On the one hand, greater aversion to missing crisis episodes allows the policymaker to correctly predict more crises. But, on the other hand, more cautiousness implies that the policymaker is likely to issue more false alarms compared to a situation of more balanced policy preferences. The higher the number of false alarms, the higher the risk of implementing corrective macroeconomic policies when a crisis does not happen. More generally, the benefit of correctly calling more crisis episodes needs to be weighed against the risk of issuing many false alarms and the costs of implementing corrective macroeconomic policies prematurely.
VI. Conclusions
In this study we compared the performance of parametric and non-parametric EWS in correctly predicting in-sample and out-of sample currency crisis in EMs.
We find that both the parametric and non-parametric EWS assign an important role to ‘traditional’ macroeconomic indicators, such as the real GDP growth rate, the current account balance as a percentage of nominal GDP, the monthly growth rate in foreign exchange reserves, the ratio between the money stock and foreign exchange reserves, and the ratio between foreign exchange reserves and short-term external debt. In the parametric EWS, the measure of government instability is significant and positively associated with crisis incidence, while in the non-parametric EWS it turns out to be a much less reliable indicator than the macroeconomic indicators. From a macroeconomic policy perspective, the results obtained with the parametric and non-parametric EWS imply that those emerging market economies with good macroeconomic indicators and that had improved their policy fundamentals in the pre-crisis period, are likely to have suffered less from the impact of the global financial crisis.24
In terms of performance, we find that when the policymaker equally dislikes missing crisis episodes and issuing false alarms, the parametric EWS achieves superior out-of-sample results compared to the non-parametric EWS, as the total misclassification error of the former is lower than that of the latter. We also find that the percentages of correctly called out-of-sample crisis episodes by two of the three EWS during the period January 2009-December 2011 increase substantially compared to those out-of-sample periods which included 2007 and/or 2008. One possible interpretation of this result is that after 2008, the most turbulent year of the global financial crisis, international investors may have been paying more attention to macroeconomic indicators when assessing their exposure toward EMs assets.
The performance of the parametric and non-parametric EWS does not improve if the policymaker becomes more averse to the risk of missing crisis episodes, as the total misclassification error tends to increase. While a more cautious policymaker correctly calls more crisis episodes compared to a policymaker with more balanced policy preferences, the former issues more false alarms compared to the latter. This implies that the benefit of correctly calling more in-sample and out-of-sample crisis episodes is more than offset by the cost of issuing more in-sample and out-of-sample false alarms. From a macroeconomic policy perspective, the policymaker faces the standard trade-off when using EWS. On the one hand, a greater degree of caution allows the policymaker to correctly predict more crises. But, on the other hand, more prudence leads the policymaker to issue more false alarms. The higher the number of false alarms, the higher the risk of implementing corrective macroeconomic policies when a crisis does not happen. More generally, the benefit of correctly calling more crisis episodes needs to be weighed against the risk of issuing many false alarms and the costs of implementing corrective macroeconomic policies prematurely.
The analysis in this study can be extended in several ways. First, the EWS used in this study rely mainly on the information conveyed by standard macroeconomic indicators. It would be interesting to assess if and how in-sample and out-of-sample performances change if indicators quantifying contagion, spillover effects or cross-border financial linkages are included in the EWS. Second, it would be interesting to include in the panel also those emerging and developing economies that never experienced a currency crisis before, assuming that sufficient data is available, to see how the estimates and EWS performance change. Finally, it would be interesting to check whether there are more elaborated measures of real effective exchange rate misalignment that could be significant in explaining crisis incidence in the parametric EWS.