Stress Testing at the IMF
Author:
Mr. Tobias Adrian
Search for other papers by Mr. Tobias Adrian in
Current site
Google Scholar
PubMedClose
https://orcid.org/0000-0001-9379-9592
,
Mr. James Morsink
Search for other papers by Mr. James Morsink in
Current site
Google Scholar
PubMedClose
, and
Miss Liliana B Schumacher
Search for other papers by Miss Liliana B Schumacher in
Current site
Google Scholar
PubMedClose
Publication Date:
05 Feb 2020
ISBN:
9781513520742
Language:
English
Keywords:
DPPP; DP; bank; IMF staff; stress tests; bank exposure; cash flow; funding cost; share price; defaults affect bank asset value; IMF solvency stress tests; optimization process; bank capital; IMF team; counterbalancing capacity; Stress testing; Financial Sector Assessment Program; Systemic risk; Solvency; Solvency stress testing; Global
Download PDF (521.2 KB)

This paper explains specifics of stress testing at the IMF. After a brief section on the evolution of stress tests at the IMF, the paper presents the key steps of an IMF staff stress test. They are followed by a discussion on how IMF staff uses stress tests results for policy advice. The paper concludes by identifying remaining challenges to make stress tests more useful for the monitoring of financial stability and an overview of IMF staff work program in that direction. Stress tests help assess the resilience of financial systems in IMF member countries and underpin policy advice to preserve or restore financial stability. This assessment and advice are mainly provided through the Financial Sector Assessment Program (FSAP). IMF staff also provide technical assistance in stress testing to many its member countries. An IMF macroprudential stress test is a methodology to assess financial vulnerabilities that can trigger systemic risk and the need of systemwide mitigating measures. The definition of systemic risk as used by the IMF is relevant to understanding the role of its stress tests as tools for financial surveillance and the IMF’s current work program. IMF stress tests primarily apply to depository intermediaries, and, systemically important banks.

Abstract

This paper explains specifics of stress testing at the IMF. After a brief section on the evolution of stress tests at the IMF, the paper presents the key steps of an IMF staff stress test. They are followed by a discussion on how IMF staff uses stress tests results for policy advice. The paper concludes by identifying remaining challenges to make stress tests more useful for the monitoring of financial stability and an overview of IMF staff work program in that direction. Stress tests help assess the resilience of financial systems in IMF member countries and underpin policy advice to preserve or restore financial stability. This assessment and advice are mainly provided through the Financial Sector Assessment Program (FSAP). IMF staff also provide technical assistance in stress testing to many its member countries. An IMF macroprudential stress test is a methodology to assess financial vulnerabilities that can trigger systemic risk and the need of systemwide mitigating measures. The definition of systemic risk as used by the IMF is relevant to understanding the role of its stress tests as tools for financial surveillance and the IMF’s current work program. IMF stress tests primarily apply to depository intermediaries, and, systemically important banks.

Chapter 1 Introduction

IMF staff use macroprudential stress tests to assess systemic risk as part of the IMF’s mandate to monitor global financial stability. Stress tests help assess the resilience of financial systems in IMF member countries and underpin policy advice to preserve or restore financial stability. This assessment and advice are mainly provided through the Financial Sector Assessment Program (FSAP). IMF staff also provide technical assistance in stress testing to a large number of its member countries.

An IMF macroprudential stress test is a methodology to assess financial vulnerabilities that can trigger systemic risk and the need of systemwide mitigating measures. The difference between a macroprudential and a supervisory stress test lies in the nature of the assessment and the consequences of the results:

  • A microprudential stress test is a forward-looking supervisory tool that assesses the adequacy of individual banks’ capital (or liquidity) conditional on their portfolio risks. Key to the supervisory purpose is the ability of the bank “to pass or not to pass the test” as well as the subsequent supervisory measures that may be needed to beef up cushions when the bank does not pass the test.

  • A macroprudential stress test instead focuses on financial vulnerabilities that can trigger systemic risk. Financial vulnerabilities are imbalances and other financial characteristics of the financial environment (such as high leverage, mispricing, concentration of risk, liquidity mismanagement, and others) that amplify adverse shocks. While an important part of IMF stress testing involves assessing the health of individual financial institutions, the final objective is not to determine whether individual banks are adequately capitalized based on a hurdle rate but to assess whether the identified vulnerabilities can compromise financial stability for the whole economy. Results by institution are not published; instead, they are discussed with the authorities and used to support the financial stability assessment and the recommendations that are at the core of the IMF FSAP reports.1 Although recommendations could include the need to boost capital cushions, they can also include the adoption of other macroprudential measures, such as measures targeting credit demand (debt to income and loan to value ratios), surcharges (countercyclical or risk specific surcharges), or liquidity requirements.

The definition of systemic risk as used by the IMF is relevant to understanding the role of its stress tests as tools for financial surveillance and the IMF’s current work program. IMF, FSB, and BIS (2009) defines systemic risk at the onset of the global financial crisis as the risk of disruptions to the provision of financial services caused by an impairment of all or parts of the financial system, that had the potential to cause serious negative consequences for the real economy. Most of the vulnerability analysis conducted by the IMF until recently as part of its stress testing exercises has been related to vulnerabilities that could lead to financial crisis. More recently, however, IMF work on financial stability also includes identifying financial vulnerabilities that may not necessarily lead to a financial crisis but, through the operation of the financial system, could create downside risks to growth. Both systemic financial disruptions, as well as milder reversals of financial vulnerabilities, could create downside risks to growth (“growth at risk”) (IMF 2017a; see also Adrian, Boyarchenko, and Giannone 2019). Consistently, the goal of the IMF financial surveillance function and stress testing at present is to not only assess the risk of systemic failures of significant financial institutions, but also identify financial vulnerabilities that can create risks for sustainable economic growth, even if these vulnerabilities may not lead to a financial crisis.

IMF stress tests primarily apply to depository intermediaries, and in particular, systemically important banks (whether globally or domestically systemic). Banks are more prone to engage in behavior that can lead to systemic risk, either through the maturity and liquidity transformation or through the credit risk channel. However, in many cases, following the identification of specific sources of systemic risk, IMF staff have also included in the FSAPs stress tests of nonbanks, such as insurance and asset management companies and nonfinancial firms, as well as estimates of stress for households.2

Stress testing at the IMF is adapted to the diversity of its member countries. The IMF membership is diverse, which presents challenges for stress testing. By contrast with national agencies that typically focus on one or a limited number of national financial sectors over time, the IMF uses stress tests as part of financial stability assessments in 12–14 different financial systems each year. (In addition, the IMF helps develop country authorities’ capacity in stress testing—through so-called Financial Sector Stability Reviews and other technical assistance missions—in about 18 different financial systems each year.) While this schedule helps countries to gain experience in understanding sources of vulnerabilities, it also imposes the need to adapt to differ-ent types of threats to financial stability, uneven data availability, and diverse complexity of financial systems. To benefit from local knowledge, stress testing at the IMF usually combines top-down stress tests (conducted by IMF staff, sometimes in collaboration with national supervisors) and bottom-up stress tests (produced by financial institutions), based on agreed-on methodology and scenarios with IMF staff.

The plan of this paper is as follows: After a brief section on the evolution of stress tests at the IMF, the paper presents the key steps of an IMF staff stress test. They are followed by a discussion on how IMF staff uses stress tests results for policy advice. The paper concludes by identifying remaining challenges to make stress tests more useful for the monitoring of financial stability and an overview of IMF staff work program in that direction.

Chapter 2 Stress Testing at the IMF: From the Beginnings to now

Risk management is a relatively recent field within finance. Its origins can be traced to the interplay between the wave of financial innovation triggered by the path-breaking advances in option pricing in the 1970s and the increasingly volatile financial environment of the 1980s and 1990s, which included shifts in the policy framework of major central banks, failure or almost failure of large investment banks, the 1987 stock market crash, the savings and loans crisis in the US, and financial crisis in major emerging market econo-mies.1 Early risk management techniques focused on the assessment of one risk factor at a time and the adoption of risk-based performance measures. These approaches changed radically after the release of JPMorgan’s portfolio Value-at-Risk (VaR) frameworks for market risk (Risk Metrics) in 1994 and for credit risk (credit metrics) in 1996 that became the standard in the field and started to be used extensively in practice and academic studies. The portfolio VaR measure was subsequently adopted in the so-called market risk amendment of 1996, and later in the passage of Basel II in 2006.2

It did not take long for stress testing to emerge as a more-articulated version of risk management techniques. IMF staff were among the first to adopt stress testing for banks. The Asian crisis of 1997–98 was a wakeup call for IMF staff, who had hitherto not placed much emphasis on the macroeconomic impact of a bank’s performance. The FSAP, inaugurated by the IMF and the World Bank in 1999, was an effort to respond to the lessons from the Asian crisis (as well as to bring in World Bank’s expertise on financial sector development in emerging market and developing economies). An FSAP for a country included a supervisory component, centered on the assessment of supervisory principles, done jointly by the IMF and the Bank, an IMF-led quantitative component centered on stress tests, and a World Bank-led development component. Stress tests were adopted as the key tool of assessment because of their forward-looking dimension, as opposed to historical balance sheet-based indicators used at the time (for example, C.A.M.E.L).

Given the novelty of the approach, initial IMF staff stress tests lacked a uniform methodology. Many of them were based on a spreadsheet (Cihak 2007) that estimated interest rate risk in the banking book, credit, and market risks independently with losses being added up at the end. The risk horizon was one year for credit risk while market risks were based on instantaneous shocks. Scenarios were simple and based on the worst historical development (for example, the country’s worst historical recession). Akin to what was the norm in those times among banks, IMF staff also developed portfolio models to obtain estimates of credit risk under stress:

  • The 1999 South African FSAP (Barnhill, Papapanagiotou, and Schumacher 2002) used Monte Carlo simulation to project the multivariate distribution of macro-financial variables under stress and modeled banks’ market and credit risks in a correlated fashion. The resulting stress test result was a distribution of bank losses for each bank, enabling to assess the probability that each bank’s equity would fall below a certain equity threshold.

  • Some FSAP teams used Credit Risk Plus, an actuarial approach developed by Credit Suisse in 1997, to obtain the distribution of bank credit losses in a large number of stress tests (Avezani and others 2006).

  • Starting with the 2006 Denmark FSAP, several stress tests followed a portfolio approach based on entropy risk measures, allowing for non-parametric approaches to be used in risk assessments (Segoviano 2006).

With the introduction of Basel II, as the Basel formula became prevalent for the calculation of capital requirements for banks under the Internal Risk-Based (IRB) approach, IMF staff also adopted this approach, making the stress testing methodology more uniform (Schmieder, Hasan, and Puhr 2011). Staff began using the Basel formula to estimate capital requirements under the adverse scenario, as authorities demanded that stress tests evaluate banks under the regulatory approach rather than under models developed by staff. Consequently, most portfolio models developed by IMF staff were left in the shelves and instead FSAP stress testers concentrated on estimating inputs under stress that were needed for the calculation of capital requirements under the adverse scenario using the Basel formula (for example, Probabilities of Default [PDs], Loss-Given-Default [LGDs], and Exposure at Default [EaDs] for IRB exposures).

The global financial crisis (GFC) and European crisis showed the limitations of the microprudential approach to stress testing. Until these crises, stress tests had focused on the assessment of individual institutions, and results were communicated in terms of the number of banks that had passed the test (and/or the share of banking assets). The crises—which affected entire asset classes and markets across many types of financial intermediaries, including nonbanks—propagated rapidly in ways that stress tests had not anticipated. Also, losses were much larger than stress tests had estimated in previous years, exposing the limitations of the bank-by-bank analysis.

As a result, the focus turned to stress testing methodologies that could capture systemic risk. Since then, IMF staff stress tests have emphasized the need to assess systemic risk, rather than the risk of individual institutions. Clear conceptual and functional separation was achieved between supervisory and macroprudential stress tests. Staff have also improved their tools for macro-financial analysis and scenario modeling, extended the stress testing framework to cover forms of risk that had received less attention before (such as sovereign, funding, and market liquidity risks), introduced contagion models to assess negative externalities from interconnectedness, developed stress tests for nonbanks, and prioritized the development of methodologies that can capture systemic losses from amplification mechanisms, including the interaction between solvency and liquidity risks, and between financial vulnerabilities and the real economy.

Importantly, following the crises, the IMF decided that financial stability assessments—including stress tests—would be mandatory for jurisdictions whose financial systems were determined to be systemically important. From its inception to the time of the crisis, the FSAP program was voluntary. In particular, the US did not volunteer and was not assessed by the IMF before the crisis. In 2010, the IMF decided that henceforth financial stability assessments under the FSAP would be mandatory for jurisdictions with systemically important financial sectors. At the time of the 2010 decision, 25 jurisdictions were deemed to have systemically important financial sectors. Following a review of the decision in 2013, the number of jurisdictions subject to mandatory financial stability assessments increased to 29.3

The postcrisis era saw the incorporation of stress testing into the supervisory toolkits of major countries around the world. Following a stint at the IMF, Timothy Geithner—first as president of the US Federal Reserve Bank of New York and later as Secretary of the US Department of Treasury—proposed and helped operationalize in 2009 a new idea for deploying public funds from the Troubled Asset Relief Program (TARP) program to recapitalize banks.4 The plan was to use stress tests as a forward-looking exercise to help markets distinguish between viable banks that were temporarily illiquid and weak banks that were undercapitalized. The success of the 2009 stress tests, backed up with the US Treasury Capital Assistance Program to recapitalize weak banks was influential in the adoption of stress tests as a permanent supervisory tool by the US Federal Reserve in subsequent years.

European countries subsequently adopted bank stress testing starting in 2010, in the face of widespread banking distress. Initial stress tests by the Committee of European Banking Supervisors (CEBS) did not enjoy the same immediate credibility and success as the US stress tests for several reasons: 1) markets viewed adverse scenarios as too mild; 2) concerns emerged about current bank asset quality, including the value of sovereign securities, which created the market perception that results overestimated the value of banks’ capital; and 3) no backup measures were in place that could provide capital support if needed.5 The introduction of the Transparency Exercise in 2013, with its focus on current asset quality, was a useful complement to enhance the credibility of the European stress tests. Since 2014, stress tests have been conducted by the European Banking Authority (EBA) in cooperation with the European Central Bank (ECB) becoming a well-established tool in European banking supervisions. These tests were useful to identify solvency issues as well as to help banks regain market confidence.

Chapter 3 Key Steps of an IMF Staff Stress Test

The IMF staff’s approach to stress testing broadly consists of the following steps:

  • Initial assessment of vulnerabilities (Chapter 4): This is conducted using a range of financial indicators and more recently using the growth at risk (GaR) methodology. The preliminary identification of vulnerabilities helps guide the stress test scenario and its severity, as well as the channels of risk amplification that the stress tests need to explore. The first step also involves the definition of the perimeter of stress testing, discussed in Annexes 1 and 2 together with data issues.

  • Scenario design (Chapter 5): The scenario design module is responsible for the choice of shocks and the calibration of the macro-financial variables that will characterize the adverse scenario. Shocks are usually based on an FSAP Risk Assessment Matrix (RAM). The RAM is an organizing framework guiding the FSAP work and connecting potential shocks to vulnerabilities. It helps articulate scenarios by providing a “tail risk story.” Potential shocks can be country-specific or of a global nature. The latter are taken from the Global RAM, a core risk assessment framework prepared by the IMF each quarter, assessing risks in the global economy consistently across IMF departments. In addition, the GaR framework is used to provide a consistent metric of scenario severity across countries by enabling the calibration of tail scenarios with similar probability of occurrence across countries conditional on each country’s position in its financial cycle.

  • Stress tests of solvency, liquidity, and contagion (Chapter 6): These comprise multiple satellite models to translate adverse scenario variables as well as other features of the country’s financial cycle (such as corporate and household leverage) into balance sheet items and profits and losses that affect financial institutions capital and capital requirements; liquidity inflows and outflows are also stressed to assess the sufficiency of banks’ counter-balancing capacity (liquid assets) in the presence of shocks; the contagion analysis assesses, among others, risks from different networks to propagate shocks within and across countries

  • Risk amplification mechanisms (Chapter 7): The different stress tests are integrated to obtain an overall picture of systemic risk, including by assessing the interaction among types of risks, between solvency and liquidity factors, and among real and financial effects and other forms of risk amplification. Most of this module is still work in progress focusing on how to better calibrate the financial system’s power to amplify shocks.

Chapter 4 Assessment of Vulnerabilities

As part of the preparatory work, IMF staff start by identifying systemic vulnerabilities that can threaten financial stability. Systemic vulnerabilities are imbalances and other characteristics and financial conditions that can magnify shocks. They emerge from market failures that can lead to leverage, excessive maturity transformation, interconnectedness, and complexity. In the context of adverse shocks, they can lead to negative feedback loops, asset fire-sale dynamics, and reduced credit supply; their monitoring comes from the recognition that shocks are inherently hard to predict; therefore, financial stability assessments should be based on those vulnerabilities that build over time creating threats to bank resilience and/or downside risks to GPD growth1

Staff use a range of vulnerability indicators, balance sheet analyses (including of banks, households, and nonfinancial firms), and more recently GaR tools.

  • GaR and stress testing are forward-looking methodologies for the assessment of systemic risk. Both are based on the identification of vulnerabilities that can trigger systemic risk. Vulnerabilities are then mapped into metrics of financial distress (stress testing) and downside risks to growth (GaR). Both measures have implications for the real economy and the financial system and, given that both methodologies measure developments in the tail, these implications are expected to be consistent across the two approaches. Figure 1 presents a graphical comparison of how both methodologies relate.

  • GaR also helps detect the key vulnerabilities that stress tests need to focus on. The GaR framework estimates downside risk of future GDP growth as a function of financial vulnerabilities and in this way, it helps teams identify the key factors that need to be explored further, in a more granular setting, through stress testing.

  • Finally, the term structure of GaR provides information on whether some financial characteristics and conditions could mitigate macro-financial risks at some horizons and exacerbate them at others. Adrian and others (2018) show, using panels of 11 advanced and 10 emerging market economies, that loose financial conditions mitigate downside risks to growth at short-term horizons, but then forecast higher risks at medium-term horizons. That is, the term structure of GaR features an intertemporal trade-off; this knowledge can be used for the choice of the risk horizon of the stress tests.

Figure 1.
Figure 1.

Stress Testing and GaR

Citation: Departmental Papers 2020, 001; 10.5089/9781513520742.087.A001

Source: IMF staff.Note: C/A = capital-to-asset ratio; GaR = growth-at-risk; ST = stress testing.

Staff distinguishes among cyclical, structural, and institutional vulnerabilities

  • Cyclical vulnerabilities stem from a financial system’s position in the financial cycle. During boom cycles, a general optimism tends to prevail, leading to a low price of risk and correspondingly high-asset valuations. Funding constraints are relaxed for individuals and companies, credit expands, and leverage builds up. When the general optimism starts to dissipate, possibly owing to the realization of adverse shocks (for example, owing to an economic contraction or a change in investors’ sentiment), lenders become more risk averse, the price of risk increases, and asset values fall. As pointed by many authors, no single indicators can provide a perfect guide to the assessment of systemic risks; therefore, IMF staff look at a range of signals.2

  • Leverage is used as a key indicator of cyclical vulnerabilities. In line with a large literature showing that financial crises tend to be preceded by credit booms, staff start by looking at credit growth, in domestic and foreign currency, by sectors and by type of loans; staff also check other measures such as the credit-to-GDP gap that was proposed by the Basel Committee on Bank Supervision (Drehmann, Borio, and Tsatsaronis 2011; Drehman and Tsatsaronis 2014) as an indicator of the financial cycle for the implementation of the countercyclical capital; the leverage of nonfinancial firms and households; banks’ exposures to maturity transformation such as the loan-to-deposit ratio; and prevalence of types of volatile funding (such as wholesale funding in foreign currency).

  • The price of risk is also used as a cyclical indicator. Brunnermeier and Sannikov (2014) offers a rationale for the idea that the financial system is prone to crises even if measured risk is low. Similarly, Gilchrist and Zakrajšek (2012) and Stein (2014) argue that when the risk premium is low, there is a greater probability of a subsequent upward spike in credit spreads. To help address the volatility paradox, the term structure of GaR enables the IMF analyst to approximate the time horizon in which vulnerabilities would bring about a realization of risks, enabling in this way to address vulnerabilities with concrete policy recommendations. Staff also look at deviation of real estate and other asset prices from long-term trends and fundamentals, market volatility, and market-based measures of risk

  • Structural vulnerabilities also amplify shocks. Some structural vulnerabilities are particularly prevalent in developing economies and emerging markets. For example, international capital inflows may play a stabilizing role in the short term, but they carry the risk of the future reversal. High dollarization of financial intermediation, typical of many emerging market and developing economies, also acts as an amplifier following shocks that require exchange rate adjustments, by increasing the debt service of unhedged borrowers. Other vulnerabilities are present in advanced economies such as high interconnectedness across firms which exposes entities to counterparty credit risk or concentration of funding sources, instruments, or products that increase the likelihood that a substantial portion of funding will be withdrawn at the same time. It also increases the correlation between sources of funds and market conditions. Risk concentration on financial institutions’ balance sheets define another source of structural vulnerabilities, such as excessive exposures to particular type of assets, markets, and high share of nonperforming loans (NPLs), among others.

  • Institutional vulnerabilities are also part of staff analysis. In some cases, their impact may be quantified and included in an adverse scenario when they represent a threat to financial stability (for example, weak framework for anti-money laundering and countering financing of terrorism [AML-CFT] that could trigger blacklisting, loss of correspondent services, or rating downgrades).

An example of how GaR and stress testing interact in an IMF staff stress test is available in the technical note prepared for the Peru FSSA (IMF 2018a). The team used a set of macro-financial variables representing financial conditions such as leverage, the domestic price of risk and structural factors to forecast the probability distribution of future GDP growth at horizons of up to three years through quantile regressions. The separation of a large set of financial indicators into these three predetermined categories was made as a reasonable compromise between maintaining parsimony, allowing various classes of indicators to provide separate signals about risks to growth at different horizons, and being able to provide a more direct economic interpretation to the various sub-indexes. The analysis concluded that external conditions are the main drivers of Peru’s growth over short-term horizons, both on the baseline and also on the tails; their impact is twice as large as for other variables. This was consistent with the fact that Peru is a small open dollarized economy, with a strong impact from China, and exchange rate movements. The impact of external conditions on growth was also found to be mostly short-term (up to one year), which is consistent with commodity exporters facing volatile markets. This information was then used to map vulnerabilities into an adverse scenario including the scenario path.

Chapter 5 Scenario Design

IMF stress tests are based on at least two scenarios. These are a baseline scenario using the World Economic Outlook (WEO) projections, and at least one adverse scenario. The risk-horizon spans three to five years, with the choice of length and shape based on the nature of vulnerabilities characterizing the country as well as the ability of staff (based on data and type of vulnerabilities) to perform realistic projections beyond the first three years.

A scenario describes forward-looking, severe, consistent, and robust trajectories for a comprehensive set of macro-financial variables that react following the materialization of shocks. A forward-looking approach is necessary because stressful scenarios may respond to new triggers. Scenarios need to be severe enough given current vulnerabilities. Consistency across countries is important for the IMF because of the need for evenhanded treatment of member countries. Robustness poses the question of the ability of models to capture features of tail events and the number of scenarios that may be needed to assess resilience. This section presents the steps followed in scenario design and illustrates how these requirements are met in practice.

Scenario design is divided in three phases:

  • Selection of shocks that can exacerbate identified financial vulnerabilities.

  • Assessment of sufficient severity.

  • Simulation of the complete set of macro-financial variables that are consistent with the shock.

Scenario design starts with a narrative about how the realization of tail risks could interact with financial vulnerabilities to generate severe but plausible macro-financial impact. In modeling terms, this step comprises the choice of one or more shocks. The related domestic and external financial stability episodes that could trigger the shocks are drawn from the RAM, discussed previously. This initial narrative represents the forward-looking aspect of stress testing. A stylized representation of how an FSAP RAM is organized is included in Figure 2.

Figure 2.
Figure 2.

Stylized Representation of an FSAP RAM

Citation: Departmental Papers 2020, 001; 10.5089/9781513520742.087.A001

Source: IMF staff.Note: FSAP = Financial Sector Assessment Program; RAM = risk assessment matrix.

The choice of the shocks helps align the scenario with vulnerabilities, as well as changes in institutions’ business models, policies, and the financial system structure. A truly consistent scenario cannot be defined independently of financial sector vulnerabilities and conditions—both initial conditions and behavioral responses. The whole idea of systemic risk (and the use of GaR as a measure of financial sector impact on the real economy) hinges on the notion that the financial sector is macro relevant. Hence, a properly defined scenario must have an impact on the underlying conditions and behavior of the financial sector.

Scenario severity is typically measured in terms of the fall in the level of real GDP below baseline or equivalently the cumulative fall in real GDP growth. Real GDP is the anchor variable of the scenario because a recession typically defines the worst macro-financial environment for most financial institutions. Consideration is also given to other variables that can be used to anchor severity. In some countries, scenario severity could be better captured by variables other than GDP growth. Examples are the US, where unemployment is generally considered as the best anchor variable for stress, Gulf region countries where the evolution of oil prices has been used to proxy severity, or countries with material foreign currency loans where foreign exchange shocks are key drivers of default risk.

Choosing the right severity is challenging. For years, IMF staff used a rule of thumb developed over time by which shocks to GDP growth should represent a deviation from the IMF baseline projection over the first two to three years of the scenario of at least two historical standard deviations from the mean, that is, the key benchmark was provided by the unconditional historical distribution of GDP growth. At present the choice is more complex. It starts by using GaR as a minimum severity related to cyclical vulnerabilities. This benchmark is chosen consistently across countries as it is derived conditional on each country’s cyclical phase.1

Consistency is needed across countries. Consistency across countries does not mean identical decline in GDP growth. The scenario and its calibration should reflect the principle that systemic risk depends, in part, on the phase of the financial cycle in which the financial system is operating. This approach uses the intuitive understanding that the larger the vulnerabilities, the larger could be the amplification mechanisms; and therefore, the assessment of vulnerabilities should be used to calibrate the type and size of shock that will drive the scenario. Among other drivers, these could include a fall of asset prices, a shock to interest rates, a reassessment of risk premiums or a large depreciation to correct an external imbalance. For example, in the Sweden FSAP (IMF 2016c, 2017b), against the background of rising household debt and a large stock of interest-only mortgage loans, shocks were represented by a reassessment of global and Swedish-specific risks that raised interest rates by about 250–500 basis points, and a large fall in real estate prices.

Many countries are also exposed to vulnerabilities that do not have a clear cyclical evolution. Therefore, the final severity may be stronger in countries with significant structural and/or institutional features. Spain FSAP’s (IMF 2017c) core shocks were related to international disruption, given the important role the Spanish banks’ international operations play to compensate declining domestic profitability and domestic NPLs. More importantly, legacy issues appeared still powerful to impair financial stability and therefore the FSAP mostly concentrated on the recognition of losses generated during the 2012 crisis. Many developing and small emerging market economies are characterized by dependence on one or a few export products, low credit to GDP ratios, high concentration of banks’ loan portfolios due to a concentrated economy, significant lack of household and small and medium enterprises access to financial services, and significant lending to the government. For these countries, vulnerabilities would mostly be structural and scenarios will be motivated by capital flow reversals, terms of trade shocks or other global financial conditions. For example, in Armenia, which is a small, open, highly dollarized economy, the recent FSAP’s adverse scenario (IMF 2018b) focused on a terms of trade shock and a market reassessment of sovereign and private risks.

Once the severity of the scenario is chosen, a set of consistent macro-financial variables paths (expressed as deviations from the WEO baseline) are simulated using macro-financial models and targeting the chosen severity benchmark (Figure 3). For major advanced and emerging market economies, scenarios are simulated using either the Global Macrofinancial Model (GFM) or the Flexible System of Global Models (FSGM). The GFM is a structural macro econometric model of the world economy, covering 40 economies at the quarterly frequency, documented in Vitek (2018). This New Keynesian dynamic stochastic general equilibrium (DSGE) model features a range of nominal and real rigidities, extensive macro-financial linkages with both bank and capital market based financial intermediation, and diverse spillover transmission channels. The FSGM is a semi-structural macro-econometric model of the world economy, covering 24 economies at the annual frequency, documented in Andrle and others (2015). For simulating stress test scenarios in which macro-financial linkages or international financial spillovers are important, the use of the GFM is preferable, as the FSGM lacks a banking sector and international financial linkages. In contrast, the use of the FSGM is preferable for simulating FSAP stress test scenarios in which structural shifts are important, as the GFM lacks permanent shocks. For developing economies, IMF staff generate their own scenarios using structural vector autoregressive models. On occasions, staff uses the authorities’ macro-economic models.

Figure 3.
Figure 3.

From Severity to a Fully-Fledged Scenario

Citation: Departmental Papers 2020, 001; 10.5089/9781513520742.087.A001

Source: IMF staff.Note: DSGE = dynamic stochastic general equilibrium models; FSAP = Financial Sector Assessment Program; GaR = growth-at-risk; SVAR = structural vector autoregression.

Given computational feasibility considerations, the GFM and the FSGM are linear. This is a disadvantage shared by all main structural macro econometric models used by major central banks to help inform their policy analysis and is necessitated by the high computational cost of solving, estimating and simulating nonlinear models, which increases exponentially with their scale. In turn, their advantage is that they can generate consistent macro-financial variables for a large number of countries, which is particularly useful for international banks operating under different macro-financial conditions. In addition to recommendations to the FSAP country, this kind of setup can also help identify spillovers and support policies of macroprudential coordination. Nevertheless, some important nonlinearities are captured in FSAP stress test scenarios simulated using linear structural macro econometric models through the scenario design. These include incorporating discrete asset price adjustments calibrated based on empirical analysis, asymmetric default rate adjustments calibrated based on debt at risk indicators, and effective lower bound restrictions imposed on monetary policy.

Robustness may require the use of several adverse scenarios. This is the case when there are several potential threats, or a combination of cyclical and structural vulnerabilities. More than one scenario is sometimes used when banks have different business models or geographical coverage of operations (for example, as for Spanish banks as some of them operate mainly internationally while others operate mainly domestically). In other occasions, a layer-of-shocks approach is used whereby all material geographies to individual banks are subject to simultaneous shocks (for example, 2018 Euro Area FSAP). Typically, teams use sensitivity stress tests to complement scenario stress tests to explore resilience to wider shifts to risk factors. Sensitivity stress tests can also include tests of failure of large or concentrated exposures.

IMF staff has recently added to their tools an analytical framework that allows simulating full distributions of financial institutions’ capital position conditional on a range of scenarios (Gross, Leika, and Valderrama 2018). The Macro-Financial System Simulator (MASS) allows them to identify ex post (rather than ex ante as currently done) those scenarios that are associated with downside risks to banks. The main reason to use MASS: it dispels an overly strong focus on point estimates of banks’ capital ratios in a stress testing context, on the back of uncertainty in scenario generation as well as model uncertainty in satellite model calibration. The semi-structural model framework allows capturing macro-financial feedback as well as state-dependent (nonlinear) economic cycle dynamics. A worst-case scenario search methodology is embedded in the framework, which connects naturally to the framework whose focus lies on distributions, thereby also reflecting scenario uncertainty.

Chapter 6 Stress Testing Banks: The Basic Framework

We define basic framework as the set of modules for solvency, liquidity, and contagion. The advanced framework comprises the gradual integration of all sources of risk (including solvency, liquidity, and contagion) in one single exercise (Figure 4).

Figure 4.
Figure 4.

The Basic Stress Testing Framework

Citation: Departmental Papers 2020, 001; 10.5089/9781513520742.087.A001

Source: IMF staff.Note: GaR = growth-at-risk.

Solvency Stress Testing

Solvency stress tests measure institutions’ resilience to adverse scenarios and identify those vulnerabilities that may be responsible for weaknesses. Resilience is assessed by the adequacy of bank capital under stress. Adequacy depends on a comparison between actual capital under stress and required capital under stress.

  • Actual capital under stress is the sum of the initial capital and net losses that are expected to materialize under the adverse scenario.

  • Stress tests also estimate capital requirements under the adverse scenario. For example, for credit risk, the calculation of capital required under stress would reflect the evolution of the portfolio size as well as changes in the credit risk parameters under the scenario.

Although IMF stress tests focus on losses due to different sources of risk, all elements of the income statement need to be forecasted in the adverse scenario to arrive at a new measure of actual capital under stress. This is comprised of the net interest income (including the impact of interest rate risk and funding shocks), provisions for credit risk, trading losses, fee income, operational expenses, and taxes. Staff estimate “satellite” models to measure how changes in the macro-financial variables would impact the different components of the income statement. The final stress test results consist of projections of complete income statements, balance sheets, and capital ratios under the adverse scenario for all years included in the risk horizon.

A hurdle or a passing rate is used as the first step to identify the drivers and not as a goal in itself. These hurdle rates based on the regulatory approach chosen by the country, that is, would be based on some measure of capital to risk-weighted assets ratio. For banks following Basel III, hurdle rates include the minimum capital requirement (Pillar 1), and supervisory review to address idiosyncratic risk not included in Pillar 1 (Pillar II). Capital conservation buffers are extended by an institution-specific countercyclical capital buffer, a buffer for systemically important banks, and a systemic risk buffer if applicable. Only the countercyclical buffer may be excluded from the hurdle rates as banks may be allowed to use this capital to pay for cyclical losses. Leverage ratios are being increasingly used by the IMF as a supplementary hurdle rates for global banks. On occasions, FSAPs have used the asset maintenance ratio for systemically important branches. The asset maintenance ratio is the amount of local assets that could be available to pay for local deposits (IMF 2013).

The presentation of results intends to maximize the information on potential vulnerabilities while preserving confidentiality. The IMF framework produces stress tests for each individual bank as shown in Table 1 and Figure 5. However, this information on individual banks is not released to the public. Instead, for public release, results are presented in terms of vulnerabilities unveiled by the analysis as well as recommendations to mitigate risks. Table 1 shows that for each bank, staff can identify the source of risk that was responsible for the reduced profits or capital losses. Since the losses are related to actual bank exposures and client data, the stress tester can identify drivers and suggest policy solutions to mitigate this potential outcome, as described in Chapter 6.

Table 1.

Presentation of Results of IMF Solvency Stress Tests

article image
article image
Figure 5.
Figure 5.

Presentation of Results of IMF Solvency Stress Tests

Citation: Departmental Papers 2020, 001; 10.5089/9781513520742.087.A001

Source: IMF staff.Note: LGD = loss-given defauly; PD = probability of default.

Liquidity Stress Testing

Liquidity stress tests assess the capacity of individual banks and the banking system to withstand extreme but plausible funding shocks (funding liquidity), and/or significant decline in the value of banks’ assets (market liquidity). The tests are a means to explore weaknesses in banks’ management of liquidity risk, assess their preparedness to deal with financial stress, and help the authorities identify priorities for policy actions—including those aimed at reducing specific exposure or building buffers. While not fully integrated in a single exercise with solvency stress tests, liquidity stress tests benefit from inputs from the solvency stress tests (see Chapter 7).

Liquidity stress testing usually is comprised of an assessment of compliance with current liquidity standards as well as a cash-flow analysis that goes beyond the minimum resilience required by current standards. There are both similarities and significant differences between the IMF staff’s cash-flow stress test and the liquidity coverage ratio (LCR). The LCR is designed to ensure that banks have a sufficient stock of high-quality liquid assets (HQLA) that can be easily converted into cash to meet their liquidity needs in a 30-day risk horizon. Historical experience in emerging market economies and the global crises demonstrate that illiquidity can last for a period of more than 30 days. Moreover, IMF staff experience is that the management of liquidity risk suffers from cliff effects at the end of the 30 days and can reveal mismatch risk for time horizons shorter than 30 days. This is why the cash-flow test is performed over a wide range of time buckets. For the LCR, the stress scenario is defined by internationally “harmonized” and predetermined parameters, including run-off rates on funding sources, roll-over rates on assets and haircuts that simulate reduced market liquidity. Instead, parameters in the cash-flow analysis are customized to reflect the characteristics of funding markets and include all unencumbered marketable securities (rather than focusing on HQLA), acknowledging the fact that all marketable assets can be used to raise funding (at appropriate haircuts) to address negative funding gaps.

The cash-flow stress test simulates the behavior of inflows and outflows in a baseline and in one or more adverse scenarios. In a baseline scenario, there are no funding shocks, asset markets operate with normal depth and banks may benefit from net behavioral cash inflows. In contrast, an adverse scenario is characterized by funding shocks (proxied by run-off and roll-over rates), reduced liquidity in asset markets (proxied by haircuts) and the absence or reduction of net new behavioral cash inflows. The test includes explicit assumption on the evolution of the exchange rate.

The data for a cash-flow-based stress test are more granular than those for solvency stress tests and need to be collected at higher frequency. The template is consistent with Basel liquidity monitoring tools for contractual maturity mismatches. It requires contractual and behavioral assumption for all types of inflows and outflows in all material currencies and a detailed breakdown of available unencumbered assets. Inflows and outflows are classified in time buckets, the first and second ones comprising all inflows and outflows that are expected to materialize in the next 24 hours and between 2 to 7 days respectively. Typical time buckets are up to 30 days, 31–60 days, 61–90 days, 91–120 days, 121–150 days, 151–180 days, 181 days–1 year, 1–3 years, and beyond 3 years. Some of these inflows and outflows are determined by contracts, but others require behavioral assumptions such as demand deposits and callable bonds. There are also contingent flows that depend on changes in prices of financial instruments or on a downgrade in the bank’s credit rating, therefore behavioral assumptions need to be included.

The calibration of parameters in the adverse scenario is challenging. Parameters are comprised of run-off rates, rollover rates, and haircuts. Ideally, the calibration should use the results of the solvency stress tests that could trigger liquidity dry-outs as well as past information related to crises episodes in the FSAP country. However, this information is usually not available. Instead, stress testers use a combination of information on the characteristics of the FSAP country, and information available in the literature on the behavior of inflows and outflows including their behavior during the crisis. To address uncertainty, teams use multiple scenarios with various levels of parameter severity. A good guidance for liquidity scenario design is the literature review in BCBS (2013) on factors related to liquidity stress, which suggests that core deposits are more stable and that the deposit status (insured vs. non-insured) as well as the bank-depositor relationship matter during times of stress. It also points to the procyclicality of secured funding with regard to haircuts, valuation, tenors, and counterparty credit risk limits, highlighting that even for the relatively stable tri-party repo market, asset credit quality is important; regarding wholesale funding, evidence from the GFC reveals that it did not entirely freeze; instead, tenors shortened and rates became more sensitive to borrower characteristics. Recent FSAPs where contractual cash flow data were available (Japan [IMF 2017d], euro area [IMF 2018c], Switzerland [IMF 2019]) simulated multiple scenarios to find the level of liquidity risk tolerance (that is, survival horizon under different stress scenarios) of various banks. The metrics for scenario severity were based on the episodes of the GFC, as well as linked to asset price shocks derived from solvency scenarios.

A bank fails the test when only emergency liquidity assistance (ELA) from the central bank would allow the bank to continue fulfilling its obligations. Banks can cover negative balances of cash inflows relative to cash outflows by using their counterbalancing capacity to obtain liquidity in secondary markets or through standard central bank facilities. At this point, the bank has depleted all its cash items, its eligible collateral to access the standard central bank facilities is insufficient to offset negative funding gaps and only access to ELA can keep it open.

Contagion Analysis

IMF teams assess contagion risks at three levels: interbank, cross-sectoral (that is, among different types of financial intermediaries) and cross-border.1 The analysis is conducted using exposure data and market-based indicators. Exposure data are typically used to assess domino effects. To complement the analysis, a range of measures of codependence embedded in asset prices is used by staff to assess potential contagion between banks and other financial intermediaries, and among banks and sovereigns. The use of market-based methodologies is important because of limited availability of exposure data and because they capture forms of contagion related to indirect linkages (such as exposures to common risk factors) other than domino effects based only on direct claims.2 Methodologies combining exposure and market-based data have also been used (Cortes and others 2018).

For domestic interbank exposures, exercises based on potential domino effects are usually undertaken by the authorities at the IMF’s request using agreed-on scenarios and methodologies. Results (not the underlying data) are shared with IMF teams, as the underlying counter-party exposure data are typically confidential. The Espinosa-Vega and Solé (2010) approach is often used for interbank analysis, capturing both credit and funding shocks. The credit shock arises from the failure of a client. The funding shock arises from the failure of a funding source. Whenever relevant (IMF 2013, page 14), IMF teams request that bank exposures include derivatives that are valued at the prices prevailing in the adverse scenario and therefore results would capture correlated market, credit, and contagion risks.

Systemic risk resulting from cross-sectoral exposures is comprised of the interaction between banks and nonbanks. These can take place because of different channels, including the group’s structure, the funding channel, the exposure channel, and the common exposure channel. The conglomerate structure may affect banks mainly through reputational risks when other intermediaries in the conglomerate are in distress. The funding channel can affect banks funding and liquidity positions through the response of nonbanks to the risk of banks in which they invest; the exposure channel is mainly credit risk from bank investments in other nonbanks; finally, the failure or downgrading of common exposures could trigger contagion among bank and nonbanks. The assessment of conglomerate risks is mainly assessed through common adverse scenarios for banks and nonbanks. Staff also assess regulations to ensure that bank capital is not exposed to risks arising from other parts of the conglomerate. Funding and exposure data are mainly used when available for interconnectedness analysis. The analysis from interaction from common exposures is usually jeopardized by the lack of sufficient data to assess potential contagion.

The analysis is usually more limited for cross-sectoral exposures than for the interbank market because of data constraints. Data on bilateral claims between a bank and a nonbank financial institution are often not collected by countries, including advanced economies. At the best, countries may have fragmented data or data which are not of easy use because it is not specifically reported for this type of analysis; therefore, data preparation may take too long to be useful within the timeframe of the IMF assessment. As a substitute, teams use flow of funds data or aggregate data for intersectoral exposures, which give a bird’s eye view of interconnectedness based on exposure and funding data among types of financial intermediaries (that is, not at individual institution level). The use of individual data, when available, can shed important information among potential sources of systemic risk. For example, in Sweden (2016), the mission had access to information on holdings of covered bonds issued by some banks and held by other banks or by insurance companies which lead to a fruitful exchange with the authorities on potential financial stability issues associated with cross-sectoral interconnectedness. In the 2017 Japan FSAP (IMF 2017d), the team obtained granular exposure data from banks, insurers, pension and mutual funds, and therefore, it was able to expand the interbank network to capture risks from nonbank financial institutions; the recent Euro Area FSAP (IMF 2018c) analyzed two complementary networks: one focused exclusively on large institutions, the other on the most material extra-euro area exposures of each of those institutions.

Cross-border network effects are particularly important for the IMF given its mandate to monitor global financial stability. Cross-border network effects could emerge from exposure to other financial intermediaries or intragroup exposures that put pressures on the parent bank to support entities abroad. Similarly, subsidiaries or branches can be affected by problems in the parent bank located in another country.

Cross-border interconnectedness is assessed using both exposure and market data. Typically, staff use the framework developed in Espinosa-Vega and Solé (2010) for cross-border exposures, enabling the simulation of both credit and funding shocks. While data on intragroup exposures are usually not publicly available at individual institution level, the IMF performs network analysis of (aggregate) national banking systems using Bank for International Settlements locational or consolidated data. Locational data classifies funds by country based on the “resident” criteria, while consolidated data are based on country of incorporation. The initial negative credit or funding shock is propagated through the network of bilateral claims across countries. If any banking system incurs losses larger than their total Tier 1 or regulatory capital, the system “fails.” This failure can subsequently cause some other banking system to fail, triggering domino effects, where a failure of a banking system in a network transmits to other banking systems.

The exposure-based approach is often complemented by market-based methodologies. These studies usually examine indirect linkages among banks or across countries that may not be properly captured by domino effects. For example, banks or countries exposed to the same type of assets (for example, bonds issued by a sovereign in distress) or associated to a bank or country in distress by investors because of lack of specific knowledge and/or asymmetric information. Furthermore, in the recent Spain FSAP (2017), the mission applied the Global VAR methodology (Dees and others 2007) to examine the outward spillover of credit shocks originated in Spain on banking systems in Europe and Latin America.3

For the final assessment of whether cross-border banking promotes or threatens financial stability, the quantitative cross border analysis is complemented by qualitative information. This information includes the core characteristics of cross-border relations including whether funding flows from or to parent banks, the home country’s rules regarding the provision of liquidity, existing (and possible) liquidity requirements and ringfencing imposed by home and host supervisors (for instance, the application of LCR at group and/or subsidiary/branch levels), and resolution modalities (single vs. multiple point of entry), among others.

Chapter 7 Risk Amplification

IMF staff currently focus on better integrating the different elements of their basic framework through the identification of channels of risk amplification. IMF staff have been at the forefront of introducing mechanisms in stress tests to capture how the financial system amplifies shocks. A first stylized but integrated approach was presented in the Global Financial Stability Report (April 2011, page 106)1 in which systemic risk was simulated through a combination of runs, asset fire sales, and solvency deterioration. Using publicly available information and a forward-looking simulation of integrated market and credit risks applied to stylized balance sheets of a set of US banks, the paper estimated the probability that multiple banks would fail or experience liquidity runs simultaneously. Liquidity runs were modeled as a response to elevated solvency risk and uncertainties and were shown to increase the probability of correlated bank failures. Under this setting, lower funding liquidity arose from increased uncertainty over counterparty risk and lower asset valuations which induced banks and investors to hoard liquidity, leading to systemic liquidity shortfalls. The simulation featured fire sale of assets as stressed banks sought to meet their cash flow obligations. Lower asset prices affected asset valuations and margin requirements for all banks in the system, and these in turn affected funding costs and profitability, and generated further systemic solvency concerns.

Following this initial mapping, several amplification channels have been analyzed in FSAPs (Figure 6 and Table 2), including:

  • Amplification effects from interconnectedness in the interbank market, across sectors and across borders. Although some of this work was already in progress before the GFC, it has become a more important feature of FSAP since then.

  • The solvency and liquidity interaction. This channel covers the solvency consequences of liquidity risk as well as liquidity risks that emerge owing to solvency and asset-quality deterioration The bank-sovereign nexus is included here.

  • The multiple feed-back effects between the real economy and the financial sector. This covers the behavioral response of financial intermediaries to stress conditions including the pass-through of funding costs to borrowing rates; and the reallocation of bank assets, including the impact of stress on the credit supply and other forms of deleveraging.

Figure 6.
Figure 6.

Key Risk Amplification Mechanisms Captured in FSAP Stress Tests

Citation: Departmental Papers 2020, 001; 10.5089/9781513520742.087.A001

Source: IMF staff.
Table 2.

Mechanisms of Risk Amplification

article image

Liquidity Factors Included in Solvency Stress Tests

Current efforts are addressed at obtaining robust estimates of the interaction between liquidity and solvency risk. Cont, Kotlicki, and Valderrama (2019) propose a structural framework for the joint stress testing of solvency and liquidity. These relations are then used to model liquidity and solvency risk in a coherent framework, involving external shocks to solvency and endogenous liquidity shocks. This allows for the introduction of solvency-liquidity diagrams to analyze the resilience of banks. This framework was used in the Switzerland FSAP (2019).

Most FSAPs capture the interaction between solvency and liquidity following a simplified approach pioneered by the National Bank of Austria (Puhr and Schmitz 2013). The simultaneous interdependence of solvency and liquidity risks and cross-bank contagion is difficult to model with sufficient granularity. Therefore, the IMF staff approach follows a shortcut by which, while solvency and liquidity stress tests are still run independently, solvency stress tests incorporate the impact of liquidity factors, and liquidity stress tests incorporate the impact of solvency factors. Key liquidity factors that affect the outcome of solvency stress tests are the funding cost-solvency loop, including the impact of the sovereign-bank nexus, and the market liquidity channel. Key solvency factors that affect the outcome of liquidity stress tests are changes in the risk characteristics of assets used by banks as collateral for central bank lending (“haircuts”) and on assets held as counterbalancing capacity, as well as the impact of NPLs on the flow of interest earnings and the shut-of of funding markets owing to solvency concerns or uncertainty about the quality of bank assets.

Funding Cost and Solvency

As a bank needs to tap funding markets to fund its balance sheet, funding cost can affect bank solvency. This could lead to additional asset encumbrance, deleveraging with further impact on the economy and asset liquidation, which could further reduce asset prices—increasing mark-to-market losses, reducing banks’ value of counterbalancing capacity (CBC), and impairing capital positions. These negative feedback loops could significantly amplify the effects of stress scenarios (Adrian and Shin 2010).

Several recent papers have addressed the empirical problem of linking funding costs with banks solvency position, especially the endogeneity of funding costs and capital of banks. The recent literature suggests that the interaction between solvency and funding cost is indeed statistically significant and it might be economically relevant, in particular during period of stress.2 Aymanns and others (2016), using publicly available information for the largest US banks, find that a decline in solvency ratios leads to higher unsecured funding costs, as well as a decline in net interest margin. Schmitz, Sigmund, and Valderrama (2017) analyze change in bank funding costs for 54 global banks. Their finding suggests that a 100 basis point increase in regulatory capital ratios is associated with a decrease of bank funding costs of about 105 basis points. Higher funding costs also reduce capital ratio (100 basis point increase of funding costs reduces CAR by 32 basis points on average). These papers also report empirical evidence of non-linearities in funding costs. A key finding: during periods of stress interbank funding cost is more sensitive to solvency than in normal times and the relationship between funding cost and solvency appears more sensitive at lower levels of solvency.

Recent IMF staff stress tests (Japan [IMF 2017d], New Zealand [IMF 2016f], United Kingdom [IMF 2016f], Poland [IMF 2018e], euro area [IMF 2018c], and Switzerland [IMF 2019])2 focused on integrating the solvency and funding feedback loop. This was conducted via a sequential step procedure by which capital ratios resulting from the first round of solvency test are used to update results for the next (year) round’s funding costs, and this iteration continues during the risk horizon. The process is repeated until convergence in funding costs is achieved. Funding costs are projected at the bank, instrument, and contract level. For Japan, IMF staff stress tests used the ratio of interest expenses to interest earning liabilities; the final equation of bank funding costs was based on individual bank data and included a bank-specific term that related funding costs to individual capital ratios as well as a systemic component based on the behavior of base rates that were controlled by a Markov regime-switching regressor; the latter captured the typical overreaction (nonlinear effect) that impacts markets in times of stress. The team also modelled funding costs in foreign currency using a Markov regime-switching model to analyze the liquidity risk premium in the USD/ JPY swap market.

Solvency, Equity Valuation, and Funding Cost

Although the solvency stress test focuses on regulatory capital ratios, the risk of insolvency can be amplified by a market equity correction which triggers a bank credit rating downgrade. The Euro Area FSAP estimated the ampli-fcation channel from stressed bank equity valuations and solvency risk to funding costs using a Merton-based approach. The analysis was based on Bloomberg’s Default Risk (DRSK) module that calculates the implied default probability, implied rating, and implied bank credit default swap conditional on bank financials and aggregate market variables. The key inputs of the model were drawn from the worst period in the FSAP solvency stress test, which included stressed effective short-term debt, effective long-term debt, loan loss reserves, NPLs, and net income. The market category variables, including bank share price, market capitalization, and share price volatility, were projected using a simple model linking the beta of individual banks’ share price and volatility to the path of the market share price and volatility calibrated in the scenario.

The Sovereign-Bank nexus

The credit quality of government securities can be an important source of uncertainty of bank credit quality that affects the funding constraint. National approaches usually have benign rules for provisions and capital requirements for government bonds that are held to maturity (that is, will not be traded) under the assumption that losses will not take place since full payment will be made at maturity.3 However, investors are more interested in the value of bank equity. As such, the impact of sovereign risk on funding costs may be larger for investors than for capital ratios measured by regulators.

For this reason, IMF stress tests may assume, on occasions, losses for sovereign bonds held to maturity when calculating the capital ratio under stress. For stress testing, IMF staff—while still allowing for low or zero risk-weights—typically estimate potential losses due to credit/sovereign risk by means of a market valuation under stress; these losses will reduce the numerator of the capital ratio. The decision to incorporate market valuation losses depends in part on IMF staff’s assessment of the sustainability of sovereign debt.

Solvency and Asset Sales

Abrupt and large asset fire sales amplify capital losses. Liquidity shortages may force solvent financial institutions to liquidate assets under fire-sale prices, face losses, and become insolvent. Because of common exposures among banks, the fire-sale channel also captures the contagion effects of reduced market liquidity, impacts the values of banks’ counterbalancing capacity (CBC), and leads to marked-to-market losses.

This channel is also relevant in the presence of central bank support. After the GFC, central banks granted a massive liquidity support to banks via quantitative easing programs and refinancing operations with full allotment. This provided banks with an alternative source of funding instead of selling securities in the market; however, selling prices and repo haircuts applied by central banks are still based on market values of securities, thus exposing banks to market liquidity risks.

The applied literature has suggested different approaches to model this channel. The Bank of England approach incorporates changes in market liquidity premia under stress scenarios. In particular, Baranova and others (2017) calibrates a partial-equilibrium model of dealer intermediation in bond markets, subject to model uncertainty. Han and Leika (2019) takes an empirical approach to estimate the changes in market liquidity premiums by using Markov regime-switching models for market liquidity.4 In their approach, asset prices are affected as market liquidity premiums change from one regime to another (non-linearly), which in turn affects the value of banks’ CBC either through the existing stock of encumbered assets (margin calls), additional asset encumbrance (in case of negative funding gaps), or asset fire sales (in worst-case scenarios), leaving banks’ liquidity endogenous to asset prices. As an application to euro area banks, they estimated the market liquidity premiums of key European sovereign bonds in both normal and stress regimes using transaction-level data. The analysis showed that liquidity premia in a stress regime, compared to that in normal times, could have a much larger impact on banks’ capital ratios, and the severity of the impact depends on systemwide funding shortages. Similarly, in the 2017 Japan FSAP (IMF 2017d), they calculated the Amihud (2002) measure of stock market liquidity—that is, the ratio of price change to the trading volume—to capture the price impact of sales in equities for banks that face liquidity shortfall.

Solvency Factors Included in Liquidity Stress Tests

IMF staff also incorporates solvency factors affecting outcomes of liquidity stress tests. Staff use results from the solvency stress tests to calibrate some of the liquidity parameters in the liquidity stress test. For example, solvency factors affect liquidity collateral requirements; PDs estimated for the assessment of credit risk and the scenario variables are used for haircuts set by the central bank and for the revaluation of counterbalancing capacity; counterparty credit risk plays an important role for secured funding; therefore, solvency deterioration of a bank (implying a downgrade) would lead to an increase in collateral requirements and potential reduction in funding. Liquidity inflows, from interest income, are reduced by the amount of interest payments on NPLs.

Liquidity Dry-out Due to Solvency Concerns

Concerns about bank insolvency could lead to the inability to raise funding at any cost (“liquidity dry-out”). Gorton and Metrick (2009) characterized the global financial crisis as a systemwide “run” in the securitized banking system—more precisely a “run on the repo market”—similar to the banking panics of the 19th century. Both episodes, according to this view, were triggered by insolvency problems. They find that during 2007–08, changes in the LIBOR-OIS spread, a proxy for counterparty risk in the interbank market, was strongly correlated with changes in credit spreads and repo rates for securitized bonds. These changes implied higher uncertainty about bank solvency and lower values for repo collateral. They conclude that the market slowly became aware of the risks associated with the subprime market, which then led to doubts about repo collateral and bank solvency. At some point—August 2007—a critical mass of such fears led to the first run on repo, with lenders no longer willing to provide short-term finance at historical spreads and haircuts.

In general, building a fully integrated solvency-liquidity dry-out model is difficult. Episodes like the deposit run suffered by Northern Rock in 2007, the failure of Lehman Brothers in 2008, and/or the evaporation of liquidity in the US securitization markets are rare episodes; data for estimates are needed with high granularity (for example, by type of liabilities and maturity), as well as with high frequency as these episodes develop very fast, and circumstances can be very specific making it difficult to find a model that can capture observed regularities. Moreover, liquidity in the banking system is more of a systemic phenomenon that requires modeling the behavior of different types of financial intermediaries at the same time.

IMF staff liquidity stress tests use several ad hoc scenarios to reflect asymmetric information. For example, asymmetric information translates into lower funding at longer horizons (for example, one year) from all or from specific sources (for example, wholesale funding, funding in foreign currency, funding from nonresidents and others depending on countries’ idiosyncratic funding structure). The type of scenario will depend on system-specific factors. For example, given the large liquidity support from the European Central Bank, the IMF liquidity stress tests for Spanish banks, used assumptions on the reduction of central banks funding to assess the sufficiency of banks’ liquidity buffers and the quality of banks’ liquidity management. IMF stress tests for Sweden assumed disruptions in the wholesale market given the reliance of Swedish banks on short-term funding (covered bonds) to finance long-term loans. In the most severe scenario, euro area stress test assumed that nonmar-ketable assets, such as credit claims, would not be included into counterbalancing capacity. There are also some ad hoc versions of this type of scenarios such as assuming that all banks are shut off from funding over a year for one quarter; or all banks are deprived from a significant fraction of their wholesale funding.

Banks’ Reaction to Stress: Feedback Effects between the Financial Sector and the Real Economy

The IMF staff stress-testing framework allows bank reactions to stress in a limited manner. Typically, staff calibrate the impact of funding costs on lending rates and in turn, these new rates affect credit demand and GDP growth. Lending rates are split into a reference rate and a client spread. The calibration of lending rates is based on the country’s market structure as well as historical pass-through rates of higher funding costs. For example, staff used euro area countries’ data on maturing loans and other assets to estimate the pass-through effect on the flow of different types of loans and securities.

Most IMF staff stress tests assume that the banking system maintains the supply of credit in the adverse scenario. In line with other stress test methodologies used by major central banks such as the Federal Reserve and the Bank of England (see Bank of England 2015, Box 1; Federal Reserve Board 2014), this feature is incorporated in IMF staff stress scenarios by specifying paths for aggregate bank lending to the economy, and it has been adopted for two reasons. First, it prevents institutions from achieving higher capital ratios by deleveraging; second, it allows the use of stress test for policy purposes since the test assesses not only the banks’ ability to remain resilient, but also their capacity to preserve economic growth at the same time by maintaining the needed supply of credit.

From a forecasting perspective, the assumption of no change to credit supply in the adverse scenario is not a realistic assumption. One important feature of a financial crisis is the procyclicality of leverage (Adrian and Shin 2010, 2014) as financial intermediaries adjust their balance sheets to changes in net worth. Moreover, the GFC showed that banks’ optimization process is constrained by capital requirements imposed by regulators and by a market funding constraint. When banks hit minimum regulatory ratios, they are likely to stop funding new operations leading to a credit freeze; if they lose market access, they are likely to deleverage their positions leading to a credit contraction and fire sales. This has the potential to unleash self-fulfilling adverse dynamics, whereby banks’ reaction under stress leads the system toward a new equilibrium characterized by lower credit growth, underpricing of assets, and subdued economic growth. Therefore, the current framework underestimates a risk amplification element that comes from banks’ optimization response to market funding constraints.

IMF staff are beginning to incorporate banks’ responses to shocks and the associated externalities triggered through macro-financial feedback effects:

  • Valderrama (forthcoming) developed an agent-based dynamic portfolio optimization wherein banks rebalance their portfolio subject to regulatory and market funding constraints, enabling not only a full integration of solvency and liquidity stresses but also financial-real feedbacks. The provision of credit is endogenous to the model and fluctuates with banks’ solvency and the underlying economic environment. The outcome of the model allows to examine the aggregate impact of banks’ unwinding of portfolios on financial markets through asset price dynamics, credit growth developments, banking system resilience, and the trajectory of key macroeconomic variables under different scenarios.

  • Krznar and Matheson (2017) developed a modeling framework that facilitates the analysis of both the direct effects of macroeconomic shocks on the solvency of individual banks and feedback effects that allow for the amplification and propagation of shocks that result from bank deleveraging and credit crunches. At the same time, the framework ensures consistency in the key relationships between macroeconomic and financial variables, and banks’ balance sheets. This is accomplished by embedding a standard stress-testing framework based on individual banks’ data in a semi-structural macroeconomic model. The paper also provides an avenue for many extensions to address the challenges of incorporating other second-round effects important for comprehensive systemic risk analysis, such as interactions between solvency, liquidity and contagion risks.

  • Gross and others (2018) developed a macro-financial, micro-founded Agent-Based Model (ABM), for the purpose of conducting macroprudential policy analyses with capital- and borrower-based policy instruments. The model generates endogenous, self-evolving business and financial cycles. The essential features of the model for policy analysis purpose pertain to a housing market, in an advanced bank balance sheet and loan supply process, with a detailed structural model of bank funding cost dynamics and pass-through of funding cost changes to loan prices. A countercyclical capital buffer-based policy, for example, renders bank defaults less frequent and reduces the extreme positive and negative moves in housing prices, while it compresses the business and financial cycle variance.

  • Catalán and Hoffmaister (forthcoming) incorporated a disaggregated banking sector into an otherwise standard macroeconomic structural vector autoregression (SVAR) framework. Following an initial round of bank losses, second-round effects from banks’ lending decisions collectively affect macroeconomic developments, leading to subsequent adjustments in banks’ profits and capitalization levels. As banks and their lending responses are heterogeneous, accounting for feedback loops has differentiated impact across banks. The model can also be used to quantify individual banks’ contributions to systemic risk by assessing the dynamic effects of shocks on macroeconomic outcomes under counterfactual assumptions about the initial capitalization of individual banks.

Chapter 8 Using Stress Test Results for Policy Advice

IMF staff are increasingly using stress test results to support macroprudential policy advice. Stress tests can support recommendations of microprudential measures, such as higher provisions or enhanced rules to related party transactions. Increasingly, recommendations of macroprudential nature to mitigate systemic risk are also made. They include measures such as additional capital cushions, limits on credit demand (for example, debt service to income [DSTI] ratios), or floors to capital ratios, among others. For example:

  • The 2019 Switzerland FSAP (IMF 2019) advocated the introduction of borrower-based measures to strengthen banks’ resilience to losses in the residential property segment (owner occupied and investment-led) to tighten lending criteria. The recommendation was supported by the amount of mortgage losses exposed by the stress test exercise.

  • In the context of the 2017 Netherlands (IMF 2017f) and the 2016 Ireland FSAPs (IMF 2016d), staff used micro-level household survey data to assess households’ debt repayment capacity and default rates on mortgage loans under adverse scenarios, which provided the basic framework for a macro-prudential limit on loan-to-value (LTV) ratios. To examine the feedback effects from the financial sector to the real economy, the Netherland FSAP team also used a DSGE model to simulate effects of house price shocks on consumption and investment under higher LTV limits through higher mortgage burden, less credit demand, and banks’ deleveraging. For the calibration of specific limits on LTV ratios, granular micro-level data on household income, characteristics of households and mortgage loans, and collateral values are needed. As these data are usually confidential, IMF staff collaborated closely with country authorities to design and conduct the macroprudential stress test framework.

  • Based on data from the Household Finance and Consumption survey used for the calibration of PDs, the 2016 Finland FSAP (IMF 2016b) found weaknesses in banks’ estimates of risk parameters that fed into their risk-weights and recommended the introduction of regulatory floors for internal models.

  • In the 2016 Sweden FSAP, staff recommended adding a cap on the debt-to-income (DTI) ratio to the macroprudential policy toolkit, as a tool to help contain the risks from high household indebtedness (IMF 2016c).

IMF staff have also calibrated macroprudential measures using stress tests in Austria and Romania.

  • The Austria FSAP (forthcoming), used a semi-structural model to project losses on banks’ mortgage portfolios and to analyze the potential mitigating role of different macroprudential policies, including LTV and DSTI. The model was modified to incorporate Austria-specific characteristics and used to simulate mortgage default rates and losses in the stress test scenarios. The drivers of borrowers’ debt servicing capacity included changes in house prices, income, unemployment rate, and mortgage interest rates.

  • The 2018 Romania FSAP (IMF 2018f) proposed a systemic risk buffer (SRB) to address risks stemming from the sovereign-bank nexus, and proposed a calibration based on the stress test results. The Romanian banking system is characterized by exceptionally concentrated and large exposures toward the domestic sovereign. The risk analysis found that these exposures posed significant interest rate risk, contributing to a large part of capital losses in the baseline and adverse scenario, as rates increase. As such, the team proposed an SRB calibrated to absorb the losses that a bank would face, should interest rates rise by as much as anticipated in the baseline scenario.

  • The 2018 Romania FSAP also used a loan-level model to calibrate a limit on DSTI ratio for household lending. The FSAP identified household borrowing as a growing vulnerability, particularly because loans are extended at variable rates leaving the borrowers vulnerable to interest rate increases in an environment of accelerating growth. The policy recommendation was thus to build resilience at the borrower level by imposing a maximum stressed DSTI limit for households. A loan-level probability of default (PD) model for mortgage and consumer loans was estimated based on data from the Central Credit Register. The results suggest that the PD of a borrower is highly sensitive to any changes in DSTI at DSTI ratios around 50 percent, particularly for mortgage loans. Therefore, it was recommended to set the limit such that loans do not exceed this sensitivity threshold.

Staff have also developed a general equilibrium model to signal when to loosen or tighten macroprudential policy measures (Lipinsky and Miescu, forthcoming). The model uses macroeconomic and financial time series to derive risk measures and to assess fluctuations in default risk. It differentiates between two sources of risk, idiosyncratic and aggregate risk. Idiosyncratic risk is measured by the probability of default of nonfinancial firms. Aggregate risk is measured by the risk of systemic failure (financial intermediary probability of default), for example, the risk that the aggregate capital ratio of the banking sector falls below the hurdle rate. The dynamics of the model indicate that bank capital is undershooting during downturns and overshooting during booms, leading to overshooting of risks and risk premiums during downturns and undershooting of risk and risk premia during booms. The model can be used to estimate nonfinancial firm and financial intermediary PDs and quantify its deviation from the optimum. If risks are too high, macroprudential policies should be tightened. On the contrary, if risks are too low, macroprudential policies should be loosened.

Chapter 9 A new Stress Testing Frontier: Climate Risk

Given the likely massive financial stability challenges due to climate change, IMF staff are prioritizing the assessment of the macrofinancial transmission of climate risk. There are two main channels of risk transmission that can affect the financial sector1 : physical risks arising from damage to property; and transition risks, arising from changes in policies and technologies. For financial institutions, physical risks can materialize directly, through their exposures to corporations, households, and countries that experience climate shocks, or indirectly, through the effects of climate change on the wider economy and feedback effects within the financial system. One early manifestation of the physical risks are the annual global weather-related insured losses, which increased from about US$10 billion in the 1980s to about US$50 billion in the last decade and US$138 billion in 2017—the highest since 1980. Exposures also manifest themselves through increased default risk of loan portfolios and lower values of assets.

FSAP stress tests have often captured the physical risks, such as insurance losses and nonperforming loans associated with storms, floods, and droughts. These types of stress tests have become common in FSAPs for small island states (such as the Bahamas, Jamaica, and Samoa) and other countries prone to natural disasters. FSAPs for major economies with systemically important financial sectors (such as Belgium, Denmark, France, Sweden, and the United States) have also typically covered natural catastrophe risks as part of insurance stress testing. Obtaining comparable data on financial costs of natural disasters is often a challenge (information is more readily available on numbers of displaced persons and deaths than on financial costs), but IMF staff have been working with counterparts and experts to improve the understanding of the effects of more frequent and more damaging natural disasters. This has allowed to broaden the stress tests for natural disasters from narrow calculations focused on soundness of nonlife insurance to more integrated exercises capturing macro-financial effects of more frequent and larger natural disasters. FSAPs for the Bahamas and Jamaica are recent published examples in this regard.

Work is ongoing to examine, on a pilot basis, financial stability risks associated with the transition to a low-carbon economy. The transition risks are potentially relevant for all countries, with many authorities recognizing that energy transition is unlikely to be smooth, and that abrupt changes in policies or technological breakthroughs may change asset valuations. The financial system can exacerbate shocks through leverage and interconnectedness, amplifying instability. Transition risks are challenging to quantify, but recent data show how disruptive changes cause sharp changes in valuations, such as drops in the values of “stranded assets.”2 To deepen our understanding of these risks and get a better sense of both the data needs and analytical limitations, IMF staff are analyzing financial system exposure to transition risks in an FSAP for an oil-producing advanced economy.

The transition risks are multifaceted and inherently hard to model. Climate change and the adjustment to a low-carbon economy are subject to fundamental uncertainty. The risks affect a broad range of geographies, sectors, and business models. They are large, nonlinear, and irreversible. They are only partly foreseeable, they depend on short-term actions, and their time horizon is long. A key next step will be to capture second-round effects, in which a decline in asset prices leads to fire sales that further depress asset prices, generating a vicious cycle and an amplifying mechanism for the initial shock. Preliminary attempts at quantifying second-round effects suggest that they can be sizable.

An essential element in the assessment of climate risk is the availability of sufficiently detailed information. The IMF supports public and private sector efforts to adopt climate risk disclosures across markets and jurisdictions, particularly by following the recommendations of the Task Force on Climate related Financial Disclosures (2017). A well-defined, internationally comparable taxonomy of green assets, as well as disclosure standards, would help incentivize market participants to reflect climate risks in prices. Unfortunately, disclosures are still uneven across asset classes and jurisdictions.3 Comprehensive climate stress testing would require improved provision and accessibility of high-quality data.

Going forward, IMF staff plan to expand and deepen the coverage of climate-related risk in assessments under the FSAP. Better understanding the macro-financial transmission of climate risks is where the IMF staff can build on their comparative advantages, help country authorities strengthen their policy frameworks, and contribute to the global debate. On physical risks, staff plan to move towards broader stress tests that explicitly examine the effects of increased frequency and impact of natural disasters not only on nonlife insurance, but also on the rest of the financial system and the economy, and are more integrated with the rest of the financial stability assessment. On transition risks, IMF staff will build on the lessons from the ongoing pilot exercise and conduct assessments for a larger number of countries (in collaboration with World Bank staff for jurisdictions where the FSAP is done jointly by the two organizations). IMF staff are engaging on these topics with experts from central banks and supervisory agencies, think tanks, and academia. The IMF has joined the Central Banks and Supervisors Network for Greening the Financial System as an observer and is collaborating with its members on developing an analytical framework for assessing climate-related risks.

Chapter 10 IMF Stress Tests: What’s next?

Although stress testing by IMF staff has made significant progress in recent years, important challenges remain. These have to do with improving scenario models to capture nonlinearities, including by using the GaR framework, incorporating additional channels of risk amplification to better identify vulnerabilities, better integrating its different modules; further use of stress test results to adopt and calibrate macroprudential measures; and upgrading the stress testing framework to assess new risks and take advantage of improved inputs.

Work is in progress to define a methodological approach to link GaR to the set of macro-financial variables needed for a complete stress scenario. GaR captures at a high level many of the relationships that stress tests attempt to capture in a granular way. It provides a reduced form model that helps to pin down the financial stability consequences of imbalances and other frictions in the financial system. GaR also provides a benchmark for scenario severity so that severity is at the same time consistent across countries (as the same methodology is applied) and consistent with the severity of the country’s financial imbalances (given the conditional nature of its forecasts). However, a methodology to link GaR with the broader set of variables required for the definition of a complete adverse scenario is still challenging work in progress. Relatedly, work is also in progress to improve the ability of approaches (such as DSGE models currently used by staff) to capture the behavior of macro-financial variables at low probability events to reconcile their use with the evidence that the impact of financial conditions on GDP growth exhibits significant non-linearities

The IMF staff sees many benefits from further integrating and expanding its different modules and using stress tests for policy advice. Progress has been made over time by integrating credit and market risk, solvency and liquidity feedback, as well as interaction among banks, between banks and other financial intermediaries, and between domestic and international financial systems. However, there is still room to make stress testing exercises more realistic by integrating the real economy and regulatory responses and by modeling the reaction of financial institutions to stress. Very importantly, stress tests need to become suitable tools to mitigate systemic risks by using their results to adopt and calibrate macroprudential measures.

Finally, climate change, fintech, and big data techniques create new challenges and opportunities for stress testing. IMF staff are further developing approaches to assess the physical and transition risks of climate change. Fintech may bring new channels of risk transmission, including interactions with banks in terms of credit provisions to the economy, and interconnect-edness via similar trading patterns. In turn, stress testing methodology and models will benefit from big data analytics including artificial intelligence, and machine-learning tools and techniques. This approach would include the use of detailed big data-based analysis that would employ predictive analytics rather than historical econometric-based relationships.

Same Series

Other IMF Content

Other Publishers

  • Collapse
  • Expand
Cover Departmental Papers
Stress Testing at the IMF
Author:
Mr. Tobias Adrian
,
Mr. James Morsink
, and
Miss Liliana B Schumacher
Volume/Issue:
Volume 2020: Issue 001
Publisher:
International Monetary Fund
ISBN:
9781513520742
ISSN:
2616-5333
Pages:
72
DOI:
https://doi.org/10.5089/9781513520742.087
Figure 1.

Stress Testing and GaR
Figure 2.

Stylized Representation of an FSAP RAM
Figure 3.

From Severity to a Fully-Fledged Scenario
Figure 4.

The Basic Stress Testing Framework
Figure 5.

Presentation of Results of IMF Solvency Stress Tests
Figure 6.

Key Risk Amplification Mechanisms Captured in FSAP Stress Tests