Monitoring Systemic Risk Basedon Dynamic Thresholds1
  • 1 0000000404811396https://isni.org/isni/0000000404811396International Monetary Fund

Contributor Notes

Author’s E-Mail Address: Kasper.lund.jensen@economics.ox.ac.uk.

Successful implementation of macroprudential policy is contingent on the ability to identify and estimate systemic risk in real time. In this paper, systemic risk is defined as the conditional probability of a systemic banking crisis and this conditional probability is modeled in a fixed effect binary response model framework. The model structure is dynamic and is designed for monitoring as the systemic risk forecasts only depend on data that are available in real time. Several risk factors are identified and it is hereby shown that the level of systemic risk contains a predictable component which varies through time. Furthermore, it is shown how the systemic risk forecasts map into crisis signals and how policy thresholds are derived in this framework. Finally, in an out-of-sample exercise, it is shown that the systemic risk estimates provided reliable early warning signals ahead of the recent financial crisis for several economies.

Abstract

Successful implementation of macroprudential policy is contingent on the ability to identify and estimate systemic risk in real time. In this paper, systemic risk is defined as the conditional probability of a systemic banking crisis and this conditional probability is modeled in a fixed effect binary response model framework. The model structure is dynamic and is designed for monitoring as the systemic risk forecasts only depend on data that are available in real time. Several risk factors are identified and it is hereby shown that the level of systemic risk contains a predictable component which varies through time. Furthermore, it is shown how the systemic risk forecasts map into crisis signals and how policy thresholds are derived in this framework. Finally, in an out-of-sample exercise, it is shown that the systemic risk estimates provided reliable early warning signals ahead of the recent financial crisis for several economies.

I. Introduction

The financial crisis in 2007-09, and the following global economic recession, has highlighted the importance of a macroprudential policy framework which seeks to limit systemic financial risk. While there is still no consensus on how to implement macroprudential policy it is clear that successful implementation is contingent on establishing robust methods for monitoring systemic risk.3 This current paper makes a step towards achieving this goal. Systemic risk assessment in real time is a challenging task due to the intrinsically unpredictable nature of systemic financial risk. However, this study shows, in a fixed effect binary response model framework, that systemic risk does contain a component which varies in a predictable way through time and that modeling this component can potentially improve policy decisions.

In this paper, systemic risk is defined as the conditional probability of a systemic banking crisis and I am interested in modeling and forecasting this (potentially) time varying probability. If different systemic banking crises differ completely in terms of underlying causes, triggers, and economic impact the conditional crisis probability will be unpredictable. However, as illustrated in section IV, systemic banking crises appear to share many commonalities. For example, banking crises are often preceded by prolonged periods of high credit growth and tend to occur when the banking sector is highly leveraged.

Systemic risk can be characterized by both cross-sectional and time-related dimensions (e.g. Hartmann, de Bandt, and Alcalde, 2009). The cross-sectional dimension concerns how risks are correlated across financial institutions at a given point in time due to direct and indirect linkages across institutions and prevailing default conditions. The time series dimension concerns the evolution of systemic risk over time due to changes in the macroeconomic environment. This includes changes in the default cycle, changes in financial market conditions, and the potential build-up of financial imbalances such as asset and credit market bubbles. The focus in this paper is on the time dimension of systemic risk although the empirical analysis includes a variable that proxies for the strength of interconnectedness between financial institutions.

This paper makes the following contributions to the literature on systemic risk assessment: Firstly, it employs a dynamic binary response model, based on a large panel of 68 advanced and emerging economies, to identify leading indicators of systemic risk. While Demirgüç-Kunt and Detragiache (1998a) study the determinants of banking crises the purpose of this paper is to evaluate whether systemic risk can be monitored in real time. Consequently, it employs a purely dynamic model structure such that the systemic risk forecasts are based solely on information available in real time. Furthermore, the estimation strategy employed in this paper is consistent under more general conditions than a random effect estimator used in other studies (e.g. Demirgüç-Kunt and Detragiache (1998a) and Wong, Wong and Leung (2010)). Secondly, this paper shows how to derive risk factor thresholds in the binary response model framework. The threshold of a single risk factor is dynamic in the sense that it depends on the value of the other risk factors and it is argued that this approach has some advantages relative to static thresholds based on the signal extraction approach.4 Finally, I perform a pseudo out-of-sample analysis for the period 2001-2010 in order to assess whether the risk factors provided early-warning signals ahead of the recent financial crisis.

Based on the empirical analysis, I reach the following main conclusions:

The rest of the paper is organized as follows. Section II contains a brief literature overview. Section III presents the econometric methodology and the model specification. Section IV presents the empirical results. Section V illustrates how the estimated models can be used for monitoring purposes and how to derive optimal risk factor thresholds. Finally, Section VI concludes.

II. Related Literature

The purpose of this section is to provide a brief overview of the literature. A more comprehensive review can be found in Bell and Pain (2000), Gáytan and Johnson (2002) and Demirgüç-Kunt and Detragiache (2005). Understanding the causes and triggers of systemic banking crisis has long been a core interest of regulators, central bankers and academics and there is a vast literature on the subject.5 There are generally two approaches in the empirical literature: (i) the signal extraction approach and (ii) the econometric approach.

The signaling approach was popularized in this field by Kaminsky and Reinhart (1999) who focused on the phenomenon of the “twin crises,” namely the simultaneous occurrence of currency and banking crises. They document the incidence of currency, banking, and twin crises in a sample of twenty industrial and emerging countries during 1970-95. The paper looked at 16 potential indicators and found that the real exchange rate, stock prices and the ratio of public sector deficits to GDP were the most useful indicators. Later, Borio and Lowe (2002) and Borio and Drehmann (2009), among others, employed the same approach to identify leading indicators of systemic banking crises. They found that the credit-to-GDP gap was the most useful indicator of systemic risk.

Demirgüç-Kunt and Detragiache (1998a) used an econometric approach to study the determinants of systemic banking crisis over the period 1980-1994. Their empirical results indicated that systemic banking distress was associated with a weak macroeconomic environment of low economic growth, high inflation, and high real interest rates. In addition, they found that balance of payments crisis was also associated with banking crises. Other studies such as Demirgüç-Kunt and Detragiache (1998a, 1998b, 2000, 2002 and 2005), Hutchison and McDill (1999), Domac and Martinex-Peria (2000) and Wong, Wong and Leung (2010) followed a similar econometric approach.

This paper employs an econometric approach but it differentiates itself from the literature in the following dimensions. First, the purpose of this study is to evaluate whether systemic risk can be monitored in real time rather than identifying the determinants of systemic banking crisis. In some sense this is a less ambitious goal, because I do not necessarily need to worry about interpreting coefficients as representing causal effects, but it forces me to employ a purely dynamic model specification such that the systemic risk forecasts are solely based on information available in real time. The model specification in Demirgüç-Kunt and Detragiache (1998a) is not appropriate for systemic risk monitoring since it utilizes contemporaneous economic variables which are not available before a crisis occurs. In addition, contrary to the studies mentioned, this study employs a fixed effect estimation approach to estimate the model parameters. This estimator is consistent under weaker conditions than a random effects estimator since it does not require that the risk factors are independent of the unobserved country fixed effects. If the independence assumption is true, the two estimators should converge towards the same (true) parameter value as the number of observations goes to infinity. Interestingly, I find that the fixed effect estimator and the random effects estimator generate quite different parameter estimates. Since both estimators are consistent under the independence assumption, this suggests that the risk factors and the unobserved country fixed effects are not independent and therefore that the random effects estimator is inconsistent in the model specification.6 Second, the popularity of the signal extraction approach is partly due to its easily interpretable thresholds. One potential drawback of this approach is that the thresholds are static. This implies that if the threshold of credit growth is 5 percent, a crisis signal is issued if credit growth exceeds this level independent of other factors that might impact the level of systemic risk. This is problematic as one could argue that a boom in credit can also reflect a healthy market response to expected future productivity gains as a result of new technology, new resources or institutional improvements (IMF, 2011). This paper derives dynamic risk factor thresholds, based on a binary response model framework, which allow for a more realistic environment where appropriate policy thresholds depend on the state of the economy via several other risk factors.

III. Econometric Methodology and Model Specification

The empirical analysis is based on the assumption that the conditional probability of a systemic banking crisis varies over time in a systematic way. More specifically, I will assume that the binary crisis variable, yi,t, is drawn from a Bernoulli distribution that depends on k systemic risk factors, Xi,t-h, such that the probability of a systemic banking crisis, in country i, can be written as:

Pr(yi,t=1|Xi,th;αi,β)=G(αi+xi,thβ),(1)

where G is a known continuous link function and (αi, β)T is a (k+1) dimensional column vector of unknown parameters to be estimated. A plausible requirement of the link function is that limx→∞G(x) = 1 and limx→−∞G(x) = 0 such that the probability is bounded between 0 and 1. A link function that satisfies this requirement is the cumulative distribution function of a standard normal distribution Φ(x)=x12πexp{0.5z2}dz,, and this choice leads to the probit model. Another link function that is bounded between 0 and 1 is the standard logistic distribution function, A(x) = ex/(1 + ex), and this leads to the logit model. Alternatively, by assuming that the link function is simply given by, f(x) = x, one gets the linear probability model (LPM): Pr(yi,t=1|Xi,th;αi,β)=αi+Xi,thβ. A drawback of this model is that the probability is not bounded between 0 and 1.

Since the main purpose of this paper is to monitor systemic risk, rather than to identify the determinants of systemic banking crises, the model structure is dynamic such that all the risk factors are known h periods (years) in advance. This implies that I am estimating the crisis probability in period t conditional on information known at time t-h. This is crucial since it gives a policy maker time to react to a crisis signal.

Contrary to a standard linear regression model, the marginal effects are generally not constant in a binary response model. The marginal effect of an incremental increase in Xij,t-h, an element of Xi,t-h, is given by: Pr(yi,t=1|xi,th;αi,β)xij,th=G(αi+xi,thβ)βj where G’(.) denotes the first derivative of the link function. The standardized marginal effect is here defined as Pr(yi,t=1|xi,th;αi,β)xij,th×Var[Xj,t]=G(αi+xi,thβ)βj×Var[Xj,t]. It measures the approximate increase in systemic risk due to a standard deviation increase in the risk indicator.

It is clear that the marginal effect depends on the value of all the risk indicators and on the value of the country fixed effect, αi. Therefore, the model structure implies that there is an implicit interaction effect between all the risk indicators. The marginal effect of an increase in a risk indicator is higher, all else equal, if the other risk indicators are high.7 This is illustrated in Figure 1a. According to the theoretical model predictions in Allen and Gale (2000a), the marginal increase in systemic risk following an asset price bubble is higher if it is accompanied by a credit expansion. The binary response model structure is consistent with this dynamic.

It is also worth pointing out that introducing an explicit interaction term in the model, Xi,t-hZi,t-h, will generate some perverse model dynamics as illustrate in Figure 1b.

Figure 1.
Figure 1.

Binary Response Model Structure

Citation: IMF Working Papers 2012, 159; 10.5089/9781475504576.001.A001

Source: Author’s calculations.

For example, if ‘x’ takes a negative value the marginal effect of ‘z’ can change sign. This is usually not an appealing property and I will avoid this model structure in the paper.

The model parameters, (α, β), are estimated via maximum likelihood by maximizing the log likelihood function, Σi=1NΣt=1Tiyi,tlog[G(αi+xi,thβ)]+(1yi,t)log[1G(αi+xi,thβ)], where N denotes the number of countries and Ti denotes the number of observations for country i. Note that I do not impose any restrictions on the relationship between the country fixed effects, the αi’s, and the risk factors, xi,t-h, since I treat the country fixed effects as parameters to be estimated. This approach is more robust than a random effects estimator where consistency requires that the risk indicators and the country fixed effects are independent.

A. Model Specification

The empirical analysis is based on an unbalanced annual panel of 68 advanced and emerging economies over the time period 1970-2010. Table 1 contains an overview of the economies. Due to the rare nature of systemic banking crisis it is advantageous to use a panel data approach. In this way one is able to exploit information from several time series and get more reliable and precise parameter estimates of β. On the other hand, the drawback is that one is imposing the potential false restriction that β is common for all the countries.

Table 1.

Countries in Data Sample

article image
Source: The selection of countries is based on data availability.

For the dependent variable, yi,t, I adopt the definition of a systemic banking crisis from Reinhart and Rogoff (2010). They define a banking crisis to be systemic if one of the following conditions is satisfied:

  • Bank runs that lead to closure, merger, or takeover by the public sector of one or more financial institutions.

  • Or if there are no runs, the closure, merger, takeover, or large-scale government assistance of an important financial institution that marks the start of a string of similar outcomes for other financial institutions.

Laeven and Valencia (2008, 2010) employ a similar definition as illustrated in Figure 2a. Interestingly, a larger number of countries were experiencing a systemic banking crisis in the mid-90’s comparing to the recent 2007-09 financial crises. This is a bit surprising at first glance. However, after correcting for the relative sizes of the economies, the recent financial crisis is more severe as indicated by Figure 2b.

Figure 2.
Figure 2.

Systemic Banking Crises, 1970-2010

Citation: IMF Working Papers 2012, 159; 10.5089/9781475504576.001.A001

Notes: Details of the crises dates can be found in Table 9 in Appendix III. For further details regarding the definitions of systemic banking crises see Reinhart and Rogoff (2010) and Laeven and Valencia (2008, 2010). The graph in figure (b) is computed as Σi=1Nwiyi,t, where the weights, {wi}i=1N, are based on GDP in 2005, wi=GDPi2005(1NΣj=1NGDPj2005)1.Source: Reinhart and Rogoff (2010), Laeven and Valencia (2008, 2010) and author’s own calculations.

As potential risk factors, I focus on private credit by deposit money banks as a percentage of GDP, asset prices, the banks’ lending premium, the real effective exchange rate, banking sector leverage and bank interconnectedness as proxied by the ratio of non-core to core bank liabilities. I will motivate this choice in the following.

The real effective exchange rate is a macroeconomic variable which is related to a country’s international trade competiveness. The idea behind including this variable as a risk factor is that a deteriorating international trade competitiveness would be more likely to amplify an initial shock and create a systemic banking crisis. Therefore, I conjecture that real effective exchange rate appreciation will have a positive impact on a country’s systemic risk.

Asset prices are represented by equity and house prices. These prices tend to move cyclically around a long run trend over time. Therefore, high asset price inflation might indicate a higher possibility that prices may fall back to the trend level in the future and price corrections like this are likely to happen when adverse economic shocks occur. Furthermore, banks are vulnerable to such asset price declines since sharp declines in collateral value and rising defaults on loans will deteriorate a bank’s balance sheets.

According to the theoretical model in Allen and Gale (2000a), credit growth amplifies asset price inflation, thus generating a higher systemic risk. Empirically, Borio and Lowe (2002) and Borio and Drehmann (2009) also find that credit-to-GDP growth and the credit-to-GDP gap is associated with systemic banking crises. The credit-to-GDP gap is the difference between the current level and the long-term trend. The (equilibrium) trend level is estimated by a backward-looking Hodrick-Prescott filter which is estimated recursively for each time period. When using the HP-filter one has to choose the smoothing parameter, λ. Following Drehmann, Borio and Tsatsaronis (2011), I set λ=1600 reflecting that financial cycles are approximately four times longer than standard business cycles.8

The banking sector is generally more vulnerable to adverse shocks when it is highly leveraged. Therefore, I also include a banking sector leverage variable as a potential risk indicator. I define leverage as private credit by deposit money banks as a percentage of demand, time and saving deposits in deposit money banks.9

Theoretically, even a single bank default can pose a threat to the financial system via its dependence to other financial institutions (Diamond and Dybvig, 1983, and Allen and Gale, 2000b). In order to capture this effect I also include a measure of the degree of interconnectedness in the financial sector. All else equal, if banks are highly interconnected a single bank default is more likely to trigger a systemic banking crisis. Hahm, Shin and Shin (2011) argue that the ratio of non-core to core bank liabilities is related to the degree of financial interconnectedness and is signaling financial vulnerability. A bank’s core liabilities consist of retail deposits from the household sector. This form of funding does not increase systemic risk as it does not increase the dependence between the banks. Non-core liabilities, on the other hand, consist of funding from other financial institutions. This type of funding is ‘bad’ in the sense that it increases the banks’ interconnectedness and hereby the systemic risk in the economy. Therefore, I propose to use non-core liabilities as a percentage of core liabilities as a potential risk indicator. Following Hahm, Shin and Shin (2011), I adopt two alternative measures of non-core bank liabilities:

NonCore1=Liabilityofbankstotheforeignsector+Liabilitytothenonbankingfinancialsector(2)
NonCore2=Liabilityofbankstotheforeignsector+(M3M2)(3)

Both measures include bank liabilities to the foreign sector, which constitutes an important source of non-deposit wholesale funding for banks in emerging and developing economies. In addition, Non-Core 1 definition adds bank liabilities to non-bank financial institutions such as insurance companies and pension funds, and Non-Core 2 definition adds M3 - M2 as an additional component. I use broad money, M2, to measure core liabilities.

In order to evaluate whether there exists a contagion effect between the economies I also try to include a contagion variable which is defined as:

Contagioni,t=Σjiωj,tyj,t(4)

where yj;t is the binary systemic banking crisis variable for country j at time ‘t’ and ω j,t ∊ [0:1] is country j’s market capitalization, at time t, as a percentage of the world’s market capitalization. If a systemic banking crisis in a large economy increases the level of systemic risk in other countries one should observe a significant positive coefficient estimate for this variable.

Finally, I also include the change in the banks’ lending premium. The lending premium is defined as the difference between the interest rate charged by banks on loans to prime private sector customers minus the “risk free” treasury bill interest rate at which short-term government securities are issued or traded. The purpose of including this variable is to capture time variation in conditional bank returns.

IV. Estimation Results

This section presents the estimation results based on the binary response model framework discussed in the previous section. Detailed definitions and sources of data are given in Appendix I.

To get an overview of the proposed risk indicators’ ability to detect systemic risk I initially estimated single variable model specifications. As a robustness check, I considered three different models: logit, probit and the linear probability model (LMP). The results are presented in Table 6 and 7 in Appendix II.

Based on these tables it is clear that the β coefficients vary between the three model specifications. This is not surprising as the link functions are different. What matters are the marginal effects and they are a function of both the model parameters and the link function. Figure 3 illustrates the standardized marginal effects, i.e. the marginal effects multiplied by the standard deviation of the risk indicator, for the three model specifications. Based on this figure it is clear that the results are consistent across the three model specifications. This is comforting as it reveals that the results are not sensitive to the choice of link function. In the following, I will focus on the logit model specification where the link function is given by Λ(x) = ex/(1 + ex). Figure 4 illustrates the β estimates and their corresponding confidence intervals for the logit specification.

Figure 3.
Figure 3.

Standardized Marginal Effects

Citation: IMF Working Papers 2012, 159; 10.5089/9781475504576.001.A001

Notes: The marginal effects for the logit and the probit model specifications are evaluated at the median country fixed effect and the median value of the risk indicator. The (standardized) marginal effect is defined as: G(αi+xi,thβ)βj×Var[xj,t]. It approximates the marginal increase in systemic risk due to one standard deviation increase in a risk indicator. Models with different lags (h) are estimated using the same data sample. See Table 6 and 7 for detailed estimation results.Source: Author’s estimates.
Figure 4.
Figure 4.

Systemic Risk Factors

Citation: IMF Working Papers 2012, 159; 10.5089/9781475504576.001.A001

Notes: All the estimations are based on a single factor logit model with country fixed effects. Models with different lags are estimated using the same data sample. The red triangles denote the parameter estimate when only using advanced economies in the estimation and the green squares denote the parameter estimate when only using emerging economies. See Table 6 and 7 in Appendix II for estimation details.Source: Author’s estimates.

Based on Figure 4a and 4b, it is clear that credit-to-GDP growth and the credit-to-GDP gap have a positive significant effect on the level of systemic risk up to three years in advance. This is consistent with the findings in Borio and Lowe (2002) and Borio and Drehmann (2009) who also find that these variables are useful leading indicators of systemic risk. It is also interesting to note that the parameter estimates are similar for advanced and emerging economies. One advantage of the binary response model approach, relative to the signaling extraction method, is that one is able to quantify the increase in systemic risk following an increase in a risk indicator. The results suggest that a one standard deviation increase in the credit-to-GDP gap, evaluated at the median country fixed effect and median risk factor value, increases the systemic risk by around 3.3 percent, 2.6 percent and 1.3 percent the following three years for the logit model. This is illustrated in Figure 3.

Based on Figure 4c it is also clear that a leveraged banking sector is associated with higher systemic risk. This effect is significant up to two years in advance. For a median risk country, at a one year forecast horizon, a one standard deviation increase in banking leverage increases systemic risk by 2.5 percent based on the logit specification, cf. Figure 3.

Bank interconnectedness, as proxied by non-core to core bank liabilities, is also found to have a significant impact on the level of systemic risk at a one year forecast horizon, cf. Figure 4d. This is consistent with the theoretical model in Hahm, Shin and Shin (2011) where non-core funding increases the banks’ interconnectedness and hereby increases the likelihood of a systemic banking crisis if a single bank defaults.

High asset price inflation is also found to be systematically associated with systemic banking crises. For the single predictor model specification, equity price inflation has a significant positive impact on systemic risk, three to four years ahead as seen in Figure 4e. House price inflation also appears to increase systemic risk at a two year forecast horizon although the effect is not statistically significant at a five percent significance level. This is illustrated in Figure 4f.

Deteriorating trade competitiveness, measured by a real effective exchange rate appreciation, is also found to increase systemic risk (see Figure 4g). Not surprisingly, emerging economies appear to be more sensitive to this effect.

Real GDP growth does not appear to have any impact on systemic risk based on the single factor logit model. However, it appears that the impact is asymmetric on emerging and advanced economies. Emerging economies tend to experience negative GDP growth leading up to a systemic banking crisis while advanced economies tend to experience positive GDP growth as seen in Figure 4h.

After the initial single factor analysis I now turn to a multivariate analysis where I allow the risk indicators to interact with each other (implicitly). I focus again on the logit model specification but the results are similar for the probit model specification as indicated by Figure 3. Table 2 presents the results for various model specifications.

Table 2.

Systemic Risk Factors based on Dynamic Logit Model, 1970-2010

article image
Notes: The model parameters are estimated by maximum likelihood based on an unbalanced annual panel for the period 1970-2010. ***, ** and * indicate statistically significant parameters at a 1 percent, 5 percent and 10 percent significance level respectively (against a double sided alternative). Standard errors are written below the parameter estimates. The forecast horizon (in years) is denoted by h. Coefficient estimates and standard errors are in percent.

The contagion variable for country i, at time t+1, is defined as Σj#iωj,tyj,t where Yj,t denote the binary systemic banking crisis for country j at time t and ωj,t is country j’s market capitalization, at time t, as a percentage of the world’s market capitalization.

Bad Credit Premium is defined as “Credit-to-GDP Gap”*”1{Previous two years equity inflation>20pct.}” where 1 {.} denote an indicator function which takes the value unity if the condition is true and zero otherwise. Source: Author’s estimates.

The key findings are summarized as follows:

Based on the estimation results in this section I find strong evidence that the level of systemic risk contains a predictable time-varying component. Note that the systemic risk estimates at time t are based solely on information available at time t-h. The next section evaluates how to use this information to monitor systemic risk in real time.

V. Monitoring Systemic Risk

The empirical analysis revealed that banking sector leverage, the credit-to-GDP gap, equity price inflation, real effective exchange rate appreciation and the ratio of non-core to core bank liabilities all have a significant impact on the level of systemic risk in an economy. Furthermore, as all the model specifications provide an estimate of the level of systemic risk in the future, based on information today, the binary response model approach is potentially useful for a policy maker. The purpose of this section is to construct crisis signals, based on the binary response model approach, to evaluate the models ability to monitor systemic risk. Since the focus is on monitoring systemic risk, rather than on how to implement macroprudential policy, I will not take a stand on which macroprudential tools to employ but simply assume that the policy decision is binary: the policymaker acts or he does not. This assumption also allows me to compare the binary response model approach with the signal extraction approach.

A. The Signal Extraction Approach

The signal extraction approach was originally developed to identify turning points in business cycles and was first applied to banking crises by Kaminsky and Reinhart (1999). The methodology is straightforward. For each period a signal is computed. The signal takes either the value 1 (is “on”), or “0” (is “off”). The signal is “on” if one or several risk indicators cross a certain threshold and “off” otherwise. Once a crisis occurs, it makes no sense to predict another crisis: the indicator has already done its job. Therefore, I do not consider any signals in the two years after the beginning of a crisis. How to choose this threshold? A popular assessment methodology distinguishes between two types of forecast errors: Type I error, when no signal is issued and a crisis occurs, and Type II error, when a signal is issued but no crisis occurs. The different signal classifications are illustrated in Table 3.

Table 3.

Signal Classification

article image

The optimal thresholds are determined by minimizing a loss function defined over the fraction of Type I and Type II errors:

t*=argmintk{L(TypeI(t).TypeII(t))},(5)

where t=(t1,…,tk)T denote the thresholds, TypeI(t) = B (t)/(B(t) +D(t)) and TypeII(t) = C(t)/(A(t)+ C(t)). A common choice of loss function, e.g. Borio and Lowe (2002), is the noise-to-signal ratio: L(TypeI(t),TypeII(t)) = TypeII(t)/(1 – TypeI(t)).

B. Crisis Signals Based on Binary Response Model

How does one construct crisis signals based on the binary response model framework? In this paper, I suggest that a crisis signal is ‘On’ if the estimated level of systemic risk, not individual risk indicators, crosses a certain threshold, λ*. The optimal threshold level, λ*, depends again on the policy maker’s preferences over these two types of errors as represented by a loss function, L(TypeI,TypeII):

λ*=argminλ[0,1]{L(TypeI(λ),TypeII(λ))}.(6)

In the following, when calibrating both t* and λ*, I will assume that the policy maker’s preferences can be approximated by L(TypeI,TypeII) = TypeI + TypeII.11

When choosing the optimal threshold there is a trade-off between type I and type II errors. This is illustrated in Figure 5 for a binary response model with banking sector leverage and the credit-to-GDP gap. The trade-off is clear: when a variable captures a lot of crisis (few Type I errors), due to a low threshold, it tends to overpredict their number (i.e. issue false signals and exhibit a high number of Type II errors). Figure 5 also illustrates that the optimal threshold depends on the policymakers preferences. The optimal threshold is 7.9 if policy maker’s preferences can be approximated by the following loss function, L (TypeI, TypeII) = TypeI + TypeII, and 4.9 if the loss function can be approximated by L(TypeI, TypeII) = 2* TypeI + TypeII.

Figure 5.
Figure 5.

Optimal Threshold

Citation: IMF Working Papers 2012, 159; 10.5089/9781475504576.001.A001

Notes: The figure illustrates the fraction of type I and type II errors as a function of the systemic risk threshold (based on a binary response model with banking sector leverage and the credit-to-GDP gap). The estimated conditional crisis probability for a given year is formed in the preceding year. Type I Errors = B/(B+D), Type II Errors = C/(A+C), Predicted Crises = D/(B+D) and Loss Function = γ*Type I Errors + Type II Errors, where γ={1,2}.Source: Author’s calculations.

One might ask whether the binary response model approach has any advantages relative to the signal extraction approach. If the binary response model provides a good approximation of the true conditional crisis probability this approach should lead to more accurate crisis probability forecasts. In addition, the thresholds in the binary response model are dynamic in the sense that the threshold of a single risk factor depends on the values of the other risk factors. This allows for a more realistic environment where appropriate policy thresholds depend on the state of the economy via several factors.

Table 4 illustrates the signaling performance of different model specifications based on both the signal extraction method and the binary response model approach. Interestingly, the binary response model approach outperforms the signal extraction approach for all model specifications. For example, based on a model specification with credit-to-GDP growth, banking sector leverage and equity price growth the policymaker’s “loss” (the value of the loss function) is 29.9 percent, whereas the signal extraction approach based on Credit-to-GDP gap is 63.1 percent and 47.0 percent when using the same data sample.

Table 4.

Monitoring Systemic Risk, 1970-2010

article image
Notes: The parameters in the binary response models are estimated by maximum likelihood using country fixed effects. The estimation is based on an unbalanced annual panel over the period 1970-2010. A type I error is when a systemic banking crisis occurs but no crisis signal was issued (in the current or two previous periods) and a type II error is when a signal is issued but no crisis occurs (in the current or the next two periods). Once a crisis occurs it makes no sense to predict another crisis since the indicator already has done its job. Therefore, any signals in the two years after the beginning of a crisis are ignored. The calibration of the thresholds are based on the assumption that the policymaker’s preferences over type I and type II errors can be approximated by the following loss function: L(TypeI, TypeII) = TypeI + TypeII. Loss denotes the value of the loss function (in percent) for the optimal calibration. The thresholds for the signal extraction approach are risk indicator thresholds while thresholds for the binary response models refer to the level of systemic risk. NTS denotes the noise-to-signal ratio and is given by Type II/(1-Type I).

For comparison, I also compute the optimal threshold based on the data sample for the model specification with credit-to-GDP growth, banking sector leverage and equity price growth. Source: Author’s calculations.

For the model specification with credit-to-GDP growth, banking sector leverage and equity price growth, the optimal systemic risk threshold is determined to be around 13 percent. How does this relate to the threshold levels for the underlying risk factors? I will explore this question in the following section.

C. Risk Factor Thresholds

In the previous section I identified several systemic risk factors that affect the conditional crisis probability and I have shown that they provide accurate crisis signals in terms of type I and type II errors. A crisis signal is issued when the estimated systemic risk reaches a certain threshold. This section looks at how the optimal threshold is related to the levels of the underlying systemic risk factors. I focus on model specification (1) from Table 8, with credit-to-GDP growth, banking sector leverage and equity price growth, due to its good signaling properties as shown in Table 4.

When should a policy maker be concerned about the level of systemic risk? If her preferences over forecasting errors can be described by, L(TypeI(λ),TypeII(λ)) = TypeI(λ) +TypeII(λ), she should react when the conditional crisis probability reaches around 13 percent. Figure 6 illustrates the systemic risk estimates for different values of the risk indicators and the corresponding crisis signal regions. Based on this figure it is clear that all the risk indicators affect each other such that the threshold of one indicator depends on the level of the other indicators. For example, if equity prices have grown by 20 percent and the banking sector leverage is around 160 percent, a crisis signal is issued if the credit-to-GDP growth is above 0 percent. On the other hand, if equity prices have decreased by 10 percent and the banking sector leverage is around 130 percent a crisis signal is only issued if the credit-to-GDP gap is above 10 percent points.

Figure 6.
Figure 6.

Systemic Risk Estimates and Crises Signals

Citation: IMF Working Papers 2012, 159; 10.5089/9781475504576.001.A001

Notes: The systemic risk estimates are based on a fixed effect logit model with banking sector leverage, the credit-to-GDP growth and equity price growth. The probabilities are evaluated at the median country fixed effect. For more details on estimation results see column (1) in Table 8 in Appendix II.Source: Author’s estimates.

Figure 6 also illustrates the crisis signal region for a signal extraction model with credit-to-GDP growth. A crisis signal is simply issued when the growth rate is above 4.9 percentage points.12 The analyses in Tables 2 and 4 indicate that a policymaker might gain from combining several systemic risk indicators. Rather than just looking at the credit-to-GDP growth, for example, it might also be useful to examine the amount of leverage in the banking sector. A crisis signal rule based solely on whether the credit-to-GDP gap is above 4.9 percent or not might be too simplistic. A credit boom does not necessarily increase systemic risk if it reflects a healthy market response to expected future productivity gains as a result of new technology, new resources or institutional improvements. Indeed, many episodes of credit booms were not followed by a systemic banking crisis or any other material instability. The binary response model framework allows for a more realistic environment where the risk indicator threshold depends on the level of the other risk indicators. In general, the optimal threshold, for risk factor j, is given by

RiskFactorj*=1βj{Λ1(λ*)αiΣijβixi},(7)

where xi denotes the value of risk factor i, αi denotes the country fixed effect, βi denotes the coefficient of risk factor i, Λ1(x)=log(x1x)andλ* denotes the optimal systemic risk threshold from equation (6). For model specification (1) from Table 8, with credit-to-GDP growth, banking sector leverage and equity price growth, the optimal systemic risk threshold is 13.0 percent (see table 4). The corresponding optimal credit-to-GDP growth threshold is given by the following equation:

CtGGrowth*=122.91%{Λ1(13.0%)(10.96)5.32%×BankLeverage1.70%×EquityGrowth}=39.523.2%×BankLeverage7.4%×EquityGrowth,(8)

where CtG–Growth* denotes the optimal credit-to-GDP growth threshold in percentage points and -10.96 denotes the median country fixed effect. Figure 7 illustrates the credit-to-GDP growth thresholds for different values of the other risk indicators.

Figure 7.
Figure 7.

Credit-to-GDP Growth Threshold

Citation: IMF Working Papers 2012, 159; 10.5089/9781475504576.001.A001

Notes: The thresholds are based on a binary response model with banking sector leverage, equity price growth and credit-to-GDP growth evaluated at the median country fixed effect. The threshold calibration is based on the assumption that the policy maker’s preferences can be approximated by L = Type I + Type II.Source: Author’s estimates.

Based on Figure 6 and 7, it is clear that if equity price growth has increased dramatically the optimal credit-to-GDP threshold decreases consequently. This dynamic is consistent with the theoretical model in Allen and Gale (2000a) where asset price inflation and credit growth amplify each other.

D. Out-of-Sample Analysis

As a robustness check, I also evaluate the models’ performance to provide early-warning-signals, out-of-sample. Specifically, I use data from 1970-2000 to estimate the model parameters and use these to construct early warning signals, out-of-sample, for the period 2001-2010. The optimal thresholds are calibrated based on data from 1970-2000. Table 5 contains the results for various logit model specifications.

Table 5.

Monitoring Systemic Risk - Out-of-Sample Analysis, 2001-2010

article image
Notes: The out-of-sample crisis signals for 2001-2010 are based on a dynamic logit model, with country fixed effects, estimated over the period 1970-2000. The parameters are estimated by maximum likelihood. A type I error is when a systemic banking crisis occurs but no crisis signal was issued (in the current or two previous periods) and a type II error is when a signal is issued but no crisis occurs (in the current or the next two periods). Once a crisis occurs it makes no sense to predict another crisis since the indicator already has done its job. Therefore, any signals in the two years after the beginning of a crisis are ignored. The calibration of the thresholds are based on the assumption that the policymaker’s preferences over type I and type II errors can be approximated by the following loss function: L(TypeI, TypeII) = TypeI + TypeII. The calibration is based on the in-sample period, 1970-2010. NTS denote the noise-to-signal ratio and is given by Type II/(1-Type I).Source: Author’s calculations

Not surprisingly, the policy maker’s loss is generally larger than in the in-sample analysis. That being said, however, the model specifications still perform well. All the models predict around 50-80 percent of the systemic banking crises, out-of-sample, without issuing unreasonably many false signals. Figure 8 illustrates the estimated systemic risk for the United States based on a binary response model with credit-to-GDP growth and banking sector leverage. Interestingly, the systemic risk estimates increase quite dramatically as the subprime mortgage crisis approaches in 2007.

Figure 8.
Figure 8.

Systemic Risk Estimates for the United States

Citation: IMF Working Papers 2012, 159; 10.5089/9781475504576.001.A001

Notes: The forecast of systemic risk for a given year is formed in the preceding year. Systemic risk estimates are based on a binary response panel data model with credit-to-GDP growth, banking sector leverage and country fixed effects for 1970-2000. The dashed lines show the out-of-sample probabilities for 2001-2009 and the horizontal red line depicts the optimal threshold level, 7.5, see also Table 5.Source: Author’s estimates.

VI. Concluding Remarks

In this paper, I used a dynamic binary response model with country fixed effects to model the time varying conditional probability of a systemic banking crisis. I found that the level of systemic risk depends significantly on several risk factors: banking sector leverage, credit-to-GDP growth, changes in banks’ lending premium, equity price growth, increasing interconnectedness in the financial sector and real effective exchange rate appreciation. I showed how to translate the systemic risk estimates into crisis signals and that this method provided accurate crisis signals in terms of type I and type II errors.

I discussed the implications for economic policy and threshold determination. In the signal extraction method the thresholds are constant: a crisis signal is issued if a variable is above a specific threshold, independent of the value of other systemic risk indicators. This is problematic since a credit boom does not necessarily increase the level of systemic risk. For example, the credit-to-GDP ratio could increase as a consequence of a healthy market response to expected future productivity gains as a result of new technology, new resources or institutional improvements. The thresholds, based on the binary response model framework, allow for a more realistic environment where the appropriate threshold depends on the state of the economy as measured by a combination of other risk factors.

Finally, a word of warning is warranted. This paper has shown that there exist several risk factors that can help forecast the level of systemic risk in an economy. Although the choices of risk factors generally were inspired by the theoretical literature it is still a bit unclear whether they represent a causal relationship or not. In particular, more theoretical research is needed in order to evaluate whether credit-to-GDP growth has a causal impact on the level of systemic risk or if it is simply related to another economic variable that do. For monitoring purposes it does not matter whether credit-to-GDP growth has a causal impact or not. However, macroprudential policy that attempts to reduce credit-to-GDP growth will only be successful if there is a causal relationship. It is beneficial to consider a simple analogy in order to illustrate this point. Suppose that two individuals are accused of having committed a crime and you want to identify the offender. Ideally, you would like to observe the individuals propensity to commit crime. This information is unobservable but you do observe that one of them have a tattoo on his arm. The tattoo is useful for predicting the offender if having a tattoo is related to an individual’s propensity to commit crime. However, there is clearly no causal relationship between having a tattoo and an individual’s propensity to commit crime. Therefore, although the tattoo variable is useful for forecasting, a policy that prohibits tattoos will have no impact on the crime level. To reiterate, based on the empirical results in this paper, it is yet unclear whether macroprudential policy should aim to reduce, or mitigate, any of the identified risk factors used in this model. More research is needed in order to answer this question.

Appendix I: Data Sources and Description

Binary Systemic Crisis Variable:

I adopt the definition of a systemic banking crisis from Reinhart and Rogoff (2010). They define a banking crisis to be systemic if two conditions are satisfied:

  • Bank runs that lead to closure, merger, or takeover by the public sector of one or more financial institutions.

  • Or if there are no runs, the closure, merger, takeover, or large-scale government assistance of an important financial institution that mark the start of a string of similar outcomes for other financial institutions

Only the first crisis year is used in the estimation. The exact dates can be found in Appendix III. All data are annual.

Systemic Risk Factors:

  • Credit-to-GDPt: (Private credit by deposit money bankst / GDPt)*100. Source: IMF’s International Financial Statistics (line 22d and 99b).

    Credit-to-GDP Growtht (in percentage points) = (Credit-to-GDPt - Credit-to-GDPt-2)/2

    Credit-to-GDP Gapt = Credit-to-GDPt – CtG_Trendt where CtG_Trendt is computed by the Hodrick-Prescott Filter (1981) with smoothing parameter, λ=1600.

  • Banking Sector Leverage (in percent): (Private credit by deposit money banks)/(demand, time and saving deposits in deposit money banks) Source: IMF’s International Financial Statistics (line 22d, line 24 and 25).

  • Equity Prices are from Bloomberg.

    Equity Price Growtht = ln(Equity Pricet/Equity Pricet-2)/2

  • House Prices are from Bloomberg.

    House Price Growtht = ln(House Pricet/House Pricet-2)/2

  • REER: Real Effective Exchange rate. Source: IMF Information Notice System (INS) database.

    REER Growtht = ln(REERt/REERt-1)

  • Non-core liabilities Proxy 1: The sum of banks foreign liabilities (line 26C) and bank liabilities to other financial institutions (line 36J). Source: IMF’s International Financial Statistics.

  • Non-core liabilities proxy 2: The sum of banks foreign liabilities (line 26C) and M3-M2. Source: IMF’s International Financial Statistics.

  • Core Liabilities: Broad money, M2. Source: IMF’s World Economic Outlook.

  • Real GDP (RGDP) are from IMF’s World Economic Outlook.

    Real GDP Growtht: ln(RGDPt/RGDPt-2)/2

  • The contagion variable for country i is defined as Contagioni,t+1=Σjiωj,tyj,t where yj,t is a binary systemic banking crisis variable for country j at time ‘t’ and ωj,t=MarketCapitalizationj,ΣiMarketCapitalizationi,. Source: World Bank.

  • Change in Lending Premium: Risk premium on lending (the interest rate charged by banks on loans to prime private sector customers minus the “risk free” treasury bill interest rate at which short-term government securities are issued or traded in the market). Source: World Bank.

    ΔLending Premiumt = (Lending Premiumt - Lending Premiumt-2)/2

Appendix II. Binary Response Model Estimation Results

Table 6.

Systemic Risk Factors (1/2), 1970-2010

article image
Notes: The model parameters are estimated by maximum likelihood based on a country fixed effect model specification. The estimation is based on an unbalanced annual panel for the period 1970-2010. Models with different lags are estimated using the same data sample. Beta refers to the risk factor parameter estimate, SE refers to the standard error, McF.s R2 refers to McFadden’s R2, R2 refers to the coefficient of determination, ME LR refers to the Marginal effect for a low risk country (20th percentile country fixed effect) and ME HR refers to the Marginal effect for a high risk country (80th percetile country fixed effect). The marginal effects are evaluated at the median value of the risk factor. Lag refers to ‘h’ in Pr(yi,t=1)=G(αi+βxXi,t-h). The Credit-to-GDP gap is based on the devation from the HP-filter trend with smoothing parameter λ=1600.Source: Author’s estimates
Table 7.

Systemic Risk Factors (2/2), 1970-2010

article image
Notes: The model parameters are estimated by maximum likelihood based on a country fixed effect model specification. The estimation is based on an unbalanced annual panel for the period 1970-2010. Models with different lags are estimated using the same data sample. Beta refers to the risk factor parameter estimate, SE refers to the standard error, McF.s R2 refers to McFadden’s R2, R2 refers to the coefficient of determination, ME LR refers to the Marginal effect for a low risk country (20th percentile country fixed effect) and ME HR refers to the Marginal effect for a high risk country (80th percetile country fixed effect). The marginal effects are evaluated at the median value of the risk factor. Lag refers to ‘h’ in Pr(yi,t=1)=G(αi+αxXi,t-h).Source: Author’s estimates
Table 8.

Systemic Risk Factors based on Dynamic Logit Model (Credit-to-GDP Growth), 1970-2010

article image
Notes: The model parameters are estimated by maximum likelihood based on an unbalanced annual panel for the period 1970-2010. ***, ** and * indicate statistically significant parameters at a 1%, 5% and 10% significance level respectively (against a double sided alternative). Standard errors are written below the parameter estimates. The forecast horizon (in years) is denoted by h. Coefficient estimates and standard errors are in percent.

The contagion variable for country i, at time t+1, is defined as Σj≠iωj,tyj,t where Yj,t denote the binary systemic banking crisis for country j at time t and ωj,t is country j’s market capitalization, at time t, as a percentage of the world’s market capitalization.

Bad Credit Premium is defined as “Credit-to-GDP Growth”*’1{Previous two years equity inflation<20pct}” where 1{.} denote an indicator function which takes the value unity if the condition is true and zero otherwise.

Source: Author’s estimates.

Appendix III. Systemic Banking Crises Dates

Table 9.

Systemic Banking Crises Dates

article image
Notes: The definition of a systemic banking crisis follows Reinhart and Rogoff (2010). For further details see the original paper.Source: Reinhart and Rogoff (2010).