Macroprudential Stress Tests and Policies: Searching for Robust and Implementable Frameworks1
Author:
Ron Anderson https://isni.org/isni/0000000404811396 International Monetary Fund

Search for other papers by Ron Anderson in
Current site
Google Scholar
Close
,
Jon Danielsson https://isni.org/isni/0000000404811396 International Monetary Fund

Search for other papers by Jon Danielsson in
Current site
Google Scholar
Close
,
Chikako Baba
Search for other papers by Chikako Baba in
Current site
Google Scholar
Close
,
Mr. Udaibir S Das
Search for other papers by Mr. Udaibir S Das in
Current site
Google Scholar
Close
,
Mr. Heedon Kang
Search for other papers by Mr. Heedon Kang in
Current site
Google Scholar
Close
https://orcid.org/0000-0003-0743-9094
, and
Miguel A. Segoviano
Search for other papers by Miguel A. Segoviano in
Current site
Google Scholar
Close

Macroprudential stress testing (MaPST) is becoming firmly embedded in the post-crisis policy-frameworks of financial-sectors around the world. MaPSTs can offer quantitative, forward-looking assessments of the resilience of financial systems as a whole, to particularly adverse shocks. Therefore, they are well suited to support the surveillance of macrofinancial vulnerabilities and to inform the use of macroprudential policy-instruments. This report summarizes the findings of a joint-research effort by MCM and the Systemic-Risk-Centre, which aimed at (i) presenting state-of-the-art approaches on MaPST, including modeling and implementation-challenges; (ii) providing a roadmap for future-research, and; (iii) discussing the potential uses of MaPST to support policy.

Abstract

Macroprudential stress testing (MaPST) is becoming firmly embedded in the post-crisis policy-frameworks of financial-sectors around the world. MaPSTs can offer quantitative, forward-looking assessments of the resilience of financial systems as a whole, to particularly adverse shocks. Therefore, they are well suited to support the surveillance of macrofinancial vulnerabilities and to inform the use of macroprudential policy-instruments. This report summarizes the findings of a joint-research effort by MCM and the Systemic-Risk-Centre, which aimed at (i) presenting state-of-the-art approaches on MaPST, including modeling and implementation-challenges; (ii) providing a roadmap for future-research, and; (iii) discussing the potential uses of MaPST to support policy.

I. Introduction

Macroprudential stress tests (MaPSTs) are beginning to play an increasingly major role in financial sector policymaking. The global financial crisis in 2008 showed us that relatively small initial losses in the financial system can be magnified to systemic dimensions. Since then, the financial stability authorities have prioritized the development of stress scenarios and tests that attempt to quantify losses from systemic risk amplification (SRA) mechanisms. These tests capture those losses that can be endogenously amplified through macrofinancial feedback effects, contagion across financial entities, and markets that have the potential to magnify moderate exogenous shocks via endogenous feedbacks into substantial negative financial outcomes with significant welfare losses. Thus far, MaPSTs conducted by national authorities, with a few notable exceptions, have remained diagnostic tools, used to sense sources of risk and vulnerabilities while remaining independent of the calibration of macroprudential tools. As leading central banks in the United States and the United Kingdom, however, have begun to think how to link the calibration of their policy tools to stress testing results, the debate surrounding the extent to which MaPSTs should inform macroprudential policy is coming to the forefront.

A properly designed MaPST can generate valuable information for policymakers. MaPSTs have the potential to offer a quantitative, forward-looking assessment of the resilience of individual banks, and of the financial system as a whole, to particularly adverse shocks. Therefore, they are well suited to support the surveillance of macrofinancial vulnerabilities and to inform the use of relevant macroprudential policy (MaPP) instruments. MaPSTs generate useful information for risk management and decision-making processes in periods of financial distress, and contribute to the design of recovery and resolution frameworks. They also benefit financial institutions, such as banks, pension funds, and insurance companies, concerned about tail risk.

The note seeks to bring together ideas and technical approaches for developing robust MaPST frameworks, and to discuss potential uses of macroprudential stress tests to support macroprudential policy. The modeling of losses from SRA is challenging. Amplification mechanisms are diverse and complex, and can vary in structure and magnitude at different points in time. The relevant data are usually scarce, and models constrained by data availability are often subject to model error. Given the complexity of modeling and implementing stress tests that capture SRA mechanisms, it is unlikely that a single integrated model can be used to capture the whole range of possible amplification effects. Therefore, we propose the development of “encompassing frameworks” (EF) aimed at integrating a diverse collection of data and of modeling frameworks with different characteristics as a way to maximize the information content of heterogeneous data sources and minimize potential model error. We further present the various EFs being developed by authorities around the world that capture (at various levels of comprehensiveness) systemic risk amplification mechanisms under the data available in their respective countries. Therefore, an objective of this paper is to identify major trends in theoretical and empirical modeling and provide guidance to policymakers and academics for implementation. Moreover, since MaPSTs are just beginning to be implemented, we discuss how they can be used for the calibration of MaPP instruments and how stress testing fits into the overall macroprudential agenda.

The paper proceeds as follows: Section II discusses the interaction of stress testing and financial policy, and identifies key principles for developing robust MaPSTs. Section III defines a conceptual taxonomy to analyze systemic risk, and discusses empirical methods that measure systemic risk and the implementation challenges faced when using such methods. Section IV presents MaPST frameworks currently being developed. Section V examines how the calibration of specific policy instruments can be improved by information produced from MaPST. The role of communication, governance frameworks, and accountability in ensuring the integrity of MaPST is discussed in Section VI. Section VII concludes the paper.

II. Stress Testing and Financial Policy

A. Stress Testing: Evolution

Stress testing emerged in the 1990s as a tool employed by financial institutions to assess their exposure to large risks.2 Supervisors of banks and other financial institutions quickly recognized the usefulness of stress tests for the purposes of microprudential regulation. Starting from the basic premise that banks play a central role in assuring the efficient functioning of the economy and that the failure of a bank poses a significant threat to economic growth, microprudential regulation aims to strictly control the risk of bank failure. For this reason, microprudential stress tests (MiPSTs) have been used as a tool to assess the risk of failure of a single institution. Implicitly, the dictum that has served to justify this approach, which was enshrined in the Basel capital standard, is that “financial stability is ensured as long as each and every institution is sound.”

However, the soundness of particular individual banks is not necessarily critical to overall financial stability, especially when it comes to smaller and non-SIFI banks. The Asian financial crisis showed that shocks that initially appeared to be limited to a few institutions or a single country were able to set off chain reactions that were transmitted rapidly and widely throughout Asia, and ultimately, the global financial system. These events led to the development in the late 1990s of the Financial Sector Assessment Program (FSAP) at the International Monetary Fund (IMF) and the World Bank, and motivated a generation of economists’ work, raising doubt as to the sufficiency of focusing exclusively on the soundness of individual entities for the purposes of assuring financial stability. For example, Crockett (2000) argues that it is decisions taken a long time before a crisis event that precipitated the crisis, and Danielsson and Shin (2003) conclude that all serious financial risk is endogenously created by market participants.

The FSAP introduced the use of stress testing into the policy toolkit.3 The stress tests conducted within the FSAP provide local policymakers with quantitative measures of macrofinancial vulnerabilities that complement other lessons learned from the financial sector assessment as a whole. Additionally, stress tests aim to achieve tractable, qualitative results that allow for a thorough analysis of the regulatory and crisis management frameworks in place. Exposure to the FSAP has encouraged national authorities to develop their own methods and tools for stress testing banks. These frameworks are now established as a key part of the policy toolkit available to financial authorities.

The stress testing of banks is intended to test the resiliency of an individual bank or an entire financial system against exogenous and endogenous economic shocks. Policymakers and analysts alike postulate hypothetical paths for macroeconomic variables and impose tough macrofinancial scenarios on institutions in order to test their ability to withstand these shocks in real crises and economic downturns. Initially, the stress tests focused on the resiliency of individual institutions in the face of exogenous shocks. However, in the years following the global financial crisis, the use and prominence of stress tests has increased substantially and a broader view of stress tests has developed. This “macro” perspective aims to incorporate the interaction of financial institutions in times of stress as well as the mechanisms that have the potential to endogenously amplify exogenous shocks into large losses. For this reason, stress testing authorities can pursue both microprudential and macroprudential objectives.

B. Stress Testing: Micro- and Macroprudential Objectives

Microprudential objectives aim to prevent the failure of individual financial entities. MiPSTs encompass an examination of banks’ balance sheets with a focus on capital and regulatory ratios, and increasingly on assessments of risk management practices. Subsequently, where deficiencies are identified, remedial efforts by banks, including additional safety buffers in the form of bank capital may be warranted.

MaPSTs assess the impact of an adverse scenario on the financial system’s capital, profitability, and ability to support activity in the economy as a whole. Since the global financial crisis, authorities have been increasingly focused on maintaining a “macro” perspective on the risk assessment of financial systems.4 By simultaneously subjecting a number of institutions to the same scenario, stress tests allow for an assessment of the system as a whole after losses from SRA have materialized. SRA can endogenously magnify losses through macrofinancial feedback effects and contagion across financial entities and markets (beyond the banking sector). It is worth noting that in addition to the quantitative information extracted, MaPSTs provide qualitative information for assessing “reactions of the system” in periods of stress.

C. Macroprudential Stress Testing and Financial Sector Policies

Therefore, MaPSTs are an important element of the MaPP toolkit. MaPSTs are used to obtain quantitative and qualitative information that can be useful for setting policy focused on the smooth functioning of systems as a whole: that is, MaPP. The objective of MaPP is to lower the likelihood of systemic risk, or contain it, once underway. Most definitions of systemic risk emphasize that such risk originates within the financial system and then has a real impact on the economy. The authoritative view put forward by the Bank for International Settlements (BIS), Financial Stability Board (FSB), and IMF in 2009 defines systemic risk as: “the disruption to the flow of financial services that is caused by an impairment of all or parts of the financial system; and has the potential to have serious negative consequences for the real economy” (BIS and others 2009). Hence, MaPP instruments attempt to curtail systemic risk amplification mechanisms and improve the resilience of the system to shocks. Box 1 presents a summary of macroprudential policy instruments used around the world. Country-by-country experience suggests that policymakers increasingly rely on quantitative analysis for the calibration of MaPP instruments. Supervisory judgment, however, retains an overriding role. In this way, appropriately designed MaPSTs can support the calibration processes. Table 1 summarizes current calibration practices.

Table 1.

Macroprudential Policy Tools42, 43

article image
article image
Sources: Dent and others (2016) and IMF staff.

MaPSTs can also inform the design of recovery and resolution frameworks and systemic crisis management. Microprudential policies are bottom up while macroprudential policies are top down. Microprudential policy hopes to limit the risk of failure of individual institutions; however, it does not necessarily target a zero probability of failure. The failure of a relatively small institution can be tolerated because its adverse effects on the economy can usually be contained. However, the failure of a very large institution with deep and broad connections to the rest of the financial system, a so-called systemically important financial institution, can pose a major threat to the health of the economy as a whole. For such institutions, special resolution regimes are necessary because ordinary bankruptcy procedures do not work. These aim to ensure the functioning of the financial system as a whole, and therefore, recovery and resolution are complimentary to MaPP. In this respect, macroprudential stress testing can be useful in addressing thresholds and scenarios in which the socially optimal path forward for financial institutions is recovery instead of resolution (Goodhart and Avgouleas 2014 and Goodhart and Segoviano 2015).5 This is a case of the interaction that exists between MaPP and crisis management and resolution, another policy that has a bearing on systemic risk (IMF 2013).

MaPSTs require simultaneous coordination with a range of other policy tools in order to properly address issues of systemic risk. These include microprudential aspects of supervision and regulation, as well as monetary and fiscal policy (IMF 2013). Interactions between policies are complex and can give rise to both complementarities and tensions, which may need to be resolved in order to ensure the appropriate use of instruments and policy mix (Viñals 2011) and (IMF 2013). In general, the task of attaining financial stability is too challenging to be left to one policy area alone.

D. Current and Future Stress Testing Developments

Over the past 15 years stress tests have moved from being an isolated risk management tool used by banks, to becoming a core part of the policy toolkit. However, while the sophistication of stress tests has increased substantially in recent years, they are not without their limitations, and there are a number of areas in which further enhancements can be made to support their use in policymaking for micro- and macroprudential purposes (Dent and others 2016). Table 1 presents a summary of stress testing frameworks currently being implemented in key financial markets by authorities. The priorities identified for the improvement of stress tests include:

  • Integration of liquidity and solvency stress tests: As opposed to standalone liquidity and solvency exercises, integrating liquidity and solvency allows policymakers to model the feedback from risks that can provoke significant losses in stressed situations.

  • Further development of dynamic balance sheet stress testing: Models need to capture behavioral responses of banks, such as changes in business strategies and portfolio compositions directed at coping with external shocks. The ability to improve the modeling of bank responses should ultimately allow us to have a more realistic view of the potential impact of shocks on solvency, liquidity, and profitability.

While these developments are useful from a microprudential perspective, they are not sufficient from a macroprudential point of view.6

The successful development of macroprudential stress testing depends on the following elements:

  • Further incorporation of SRA mechanisms: A conceptual taxonomy of these mechanisms includes macrofinancial feedback effects, and contagion stemming from direct and indirect interconnectedness across financial entities and markets. The following chapter presents a conceptual framework that is useful for characterizing SRA mechanisms and identifying methodologies that have been implemented in attempts to capture the losses brought about by these mechanisms.

  • Expanding the scope of MaPSTs to nonbank financial entities: MaPP hopes to contain risks across the financial system as a whole (BIS and others 2016). Since banks are usually the key providers of credit to the economy, MaPP will typically apply its policy levers to the banking system. MaPP also needs to consider the systemic risk that can build up from activities outside the banking system and develop policy responses to contain such risk (FSB 2011) and (IMF 2013). Banks interact with other entities, including pension funds, insurance companies, asset managers, hedge funds, and sovereign wealth funds. All of these entities react to stress in different ways, acting as amplifiers and dampeners, depending on the state of their cycles. Duffie (2011) proposes that the monitoring of systemic risk focus on the largest banks, the largest asset classes, and the largest counter-parties. Cortes and others (forthcoming) propose the Systemic Risk and Interconnectedness framework (SyRIN), to measure systemic risk accounting for interconnectedness across banks and nonbanks, including insurance companies, pension funds, investment funds, and hedge funds.

While there have been notable improvements in the development of MaPSTs, data and model constraints represent significant challenges that warrant a stimulating research and policy agenda. The objective is to improve the understanding and quantification of SRA mechanisms in order to make the best use of MaPSTs for policy purposes.

Macroprudential Instruments

A range of instruments has been used to contain systemic risk and address the procyclical build-up of both excessive leverage and volatile funding (IMF 2014 and ESRB 2014). These instruments are designed to strengthen and complement each other by addressing the buildup of systemic risk through time. They can be split into three groups:

  • Broad-based buffers/capital tools. Risks from a broad-based credit boom can be addressed through a variety of capital tools. These include dynamic provisioning requirements (DPRs), countercyclical capital buffers (CCyBs), and countercyclical leverage ratio caps. These tools work to increase the resilience of institutions to aggregate shocks, and to maintain the supply of credit through periods of adverse conditions. They are usually uniformly applied to all exposures.

  • Sectoral tools. When vulnerabilities from deterioration in lending standards for loans originating from specific sectors arise, sectoral tools (for example, sectoral capital requirements, limits on loan-to-value [LTV], debt-service-to-income [DSTI], loan-to income [LTI] ratios, and caps on the share of foreign currency loans) can help maintain the resilience of lenders and/or borrowers. Although they are usually applied to mortgages (residential and commercial), they can also be used in other market segments, including consumer and some corporate credit.

  • Liquidity tools. A range of prudential tools aim to contain the build-up of liquidity risks associated with credit booms. These prudential tools are meant to ensure that financial institutions avoid fire sales that are triggered by disruptions in funding markets. Tools include differentiated reserve requirements, the liquidity coverage ratio (LCR, potentially calibrated by currency), and the net stable funding ratio; caps on the loan-to-deposit ratio; and price-based tools (such as liquidity charges on non-core funding).

Countries are also gradually implementing tools to contain systemic risk emanating from contagion within the financial system. In order to improve the resilience and resolvability of financial institutions whose failures pose risks to the system, authorities may impose capital surcharges on all institutions deemed systemically important, independent of the specific circumstances of these institutions. In Europe, the Capital Requirements Directive IV (CRD IV) allows the authorities to introduce a systemic risk buffer to complement the surcharges (e.g., G-SII and O-SII buffers). These tools are starting to be phased in from 2016 to 2019 in equal steps of 25 percent, and are joined by measures, such as total loss-absorbing capacity (TLAC), to increase the resolvability of these institutions (BCBS, 2013; FSB, 2015).

Calibration of Macroprudential Instruments

Operationalizing MaPP involves assessing systemic risks and selecting and calibrating tools in order to target well-identified risks. Using a range of data sources, policymakers assess: (i) economy-wide vulnerabilities from excessive leverage and growth in total credit or asset prices; (ii) sectoral vulnerabilities arising, for example, from growing credit exposure in specific sectors; (iii) vulnerabilities from a build-up of maturity and foreign currency mismatches; and (iv) the structure of the financial system (for example, concentration) and the level of inter-linkages within and across key classes of intermediaries. Because the signaling performance of any single indicator is often imperfect, multiple indicators and various analyses are used to assess the extent of each type of vulnerability and to choose appropriate tools. Different policy tools and descriptions of current calibration procedures are laid out in Table 1.

In practice, policymakers rely on “guided discretion” when calibrating their instruments. Country experience suggests that macroprudential policymaking increasingly relies on quantitative analysis, but judgment retains an overriding role given the difficulties and data limitations faced when trying to assess systemic risk. Existing tools provide only partial coverage of potential risks, and tentative signals on the likelihood of systemic risk events provide limited information about the need for macroprudential actions. Moreover, the influence of policy actions on market participants’ behavior and expectations is an area in which quantitative approaches, so far, only offer limited guidance (CGFS 2016).

III. Amplification: Conceptual and Empirical Frameworks

A. Systemic Risk Amplification: Conceptual Frameworks

A major consideration in the design of stress tests is the potential for a small triggering element to result in a systemic crisis. The same trigger can result in a crisis one day, and have a negligible impact on another. Consequently, it is more important to understand the deep mechanisms that propel the amplification of a shock into a crisis. While banks are an important channel for amplification, other financial intermediation is increasingly relevant. The portfolio holdings of large entities, such as mutual funds, hedge funds, pension funds, insurance companies, central counterparties (CCPs), and sovereign wealth funds, all interact. Furthermore, the dependence on bank financing for small and medium enterprises (SME) credit directly affects the degree to which a financial crisis impacts the real economy. Each institution has its own cyclicality and constraints: one may ride out the cycle, another may buy, and a third could sell.

It is therefore important to study the system in its entirety and to look at how institutions and markets interact within it. This means that the financial grid structure matters (Glasserman and Young 2016). Cross (direct) holdings and indirect inter-linkages can work as absorbers and amplifiers at different times, and various categories of entities will naturally amplify or absorb risk. This dynamic implies that vulnerabilities embedded in the macroeconomy and financial system, including leverage, lack of liquidity, and information asymmetries can interact to magnify and accelerate amplification. From a modeling perspective, this has several implications:

  • Amplification mechanisms are caused by macrofinancial feedback effects and direct and indirect contagion.7

  • Amplification mechanisms can change across time and at different points of the financial and economic cycles.

  • MaPSTs need to look at the financial system in its entirety in order to see how it absorbs and amplifies shocks.

The modeling of SRA mechanisms requires conceptual frameworks that guide the development of empirical models. One contributing factor to the slow rate at which macroprudential principles have been introduced into practice is that the economics profession has been slow to develop conceptual frameworks, let alone operational models, that can convincingly and comprehensively capture the phenomenon of systemic risk amplification. Initial conceptions of systemic risk focused on the source of risks, implicitly assuming that the risks were exogenous and not endogenous. Eventually, the focus shifted to direct contagion and ultimately to indirect contagion. In the following paragraphs, we summarize some important contributions to each development.

Systemic risk: Initial interpretations

Systemic risk analysis formerly focused on the source of risk. In the early literature, the focus was more on the source of a shock and on identifying the origin of financial stress, rather than on the way stress was transmitted throughout the financial system. The goal was to identify the “big shock” that affected more than just a few financial institutions. Bartholomew and Whalen (1995) write that “Systemic refers to an event having effects on the entire banking, financial, or economic system, rather than just one or a few institutions.” This approach to understanding systemic financial risk pushed research toward identifying shocks that had particularly large effects on the financial system and its ability to allocate resources optimally in the economy. This is essentially an exogenous risk view.

Systemic risk was often directly related to the relationship between the financial system and the real economy. Mishkin (1995) defined systemic risk as the “likelihood of a sudden, usually unexpected, event that disrupts information in financial markets, making them unable to effectively channel funds to those parties with the most productive investment opportunities.” Although the literature continues to acknowledge the relationship between the financial sector and the real economy as being relevant for policymaking, conceptual thinking surrounding systemic financial issues has become less explicitly focused on the real economy and more focused on financial market structures and networks.

Systemic risk: Direct contagion

Domino effects constituted the first characterization of systemic risk that accounted for contagion. This depiction of contagion focused on transmission mechanisms that operate through direct contractual obligations between counterparties. The underlying understanding of the financial system that defines this conceptualization of systemic risk is laid out in Kaufman (1995). Systemic risk is here defined as the “probability that cumulative losses will accrue from an event that sets in motion a series of successive losses along a chain of institutions or markets comprising a system. That is, systemic risk is the risk of a chain reaction of falling interconnected dominos.” At the heart of this analysis lie the direct lending relationships among banks. A default in one bank can lead to losses on that bank’s debt held by other banks, which are then more likely to default on their own debt, and so on. This domino effect described in Kaufman (1995) is echoed in BIS (1994): “Systemic risk is the risk that the failure of a participant to meet its contractual obligations may in turn cause other participants to default with a chain reaction leading to broader financial difficulties.”8

A domino approach to thinking about the spread of losses throughout the system through a web of connected institutions does not seem to provide a full account of risk amplification. Adrian and Shin (2008) argue that a focus on direct interbank linkages misses large amounts of systemic risk created through spillovers, and that this focus is inconsistent with the experience of the global financial crisis.9 The basic problem with the domino model is the mechanistic manner in which losses are treated. A loss in one bank that triggers default is passed on to its counterparties in proportion to the size of their contractual obligations without any spillover (indirect contagion) onto values of the counterparties’ other assets. But in attempting to mitigate the consequences of the loss, the counterparties may undertake actions that threaten changes in value of assets held by other counterparties. In other words, there can be important risk spillovers. Therefore, the key to having a more realistic understanding of risk amplification is to capture some aspect of the behavior of the institutions in the network.

Systemic risk: Indirect contagion

By addressing the weaknesses of what can be thought of as the first generation of contagion models, researchers have explored a wide variety of frameworks. These models provide hypothetical new channels for risks to spread throughout the financial system and try to generate contagion effects that are more realistic, and also capture the approximate scale of risk amplification observed in past crises.

Systemic risk is endogenous. Based on the definition in Danielsson and Shin (2002), financial risk can be classified as endogenous or exogenous. For exogenous risk, shocks arrive from outside the financial system, then economic agents react to shocks but don’t influence them. Endogenous risk emphasizes the importance of how economic agents, who all have their individual objectives, resources, abilities, and constraints, react to shocks. Most financial risk, and all systemic risk, is endogenous, and endogenous risk lies at the heart of SRA mechanisms. Baranova and others (2017) model endogenous risk in corporate bond markets. They focus on how the various types of institutions can act as amplifiers of stress.

Financial networks. The nature and extent of the interconnectedness of markets and market participants influences the manner by which positive feedback loops and cascades grip the entire system. This makes it tempting to apply network analysis to the financial network in order to understand the strategic interaction of financial entities and the systemic implication of individual entity behavior. Systemic risk, for example, is not necessarily driven by the largest borrowers, or most likely to default, and the topology plays a major part; see, for instance, Allen and Gale (2000), Acemoglu and others (2015), and Cabrales and others (2016).

The impact of constraints. Constraints that have primarily microprudential justifications can serve as SRA mechanisms. These include margins, mark-to-market transactions, leverage constraints, liquidity constraints, and capital constraints. These regulatory requirements tend to bite in times of financial extremis and can impose a predictable pattern of influencing what to buy and sell, leading to a vicious feedback loop. Danielsson and others (2012) propose a general equilibrium framework, whereby low risk induces economic agents to take on more risk, which then endogenously affects the likelihood of future shocks and crises. This happens because of binding constraints on risk-taking.

The time between risk-taking and crises. Hyman Minsky, famously having said that “stability is destabilizing,” argued that economic agents react to a low-risk environment by taking on more risk, which eventually culminates in a crisis. Danielsson and others (2018) empirically verify this by showing that low volatility predicts crises up to a decade in the future. This suggests that a considerable time lag between observed risk and eventual adverse outcomes needs to be incorporated in any modeling.

A taxonomy for indirect contagion. Three broad channels of indirect contagion have been identified by the frameworks put forward in the literature.10 These channels can be broadly grouped into three categories: “fire sales,” “information asymmetry,” and “strategic complementarities.”

Fire sale channel

Adverse feedback loops and loss amplification can arise from fire sales in financial markets. Consider two firms, A and B, which both hold an illiquid asset x. While not directly connected, firm A can affect firm B by selling x at a below market price, therefore affecting the value of x in B’s balance sheet when marked to market. An additional obstacle banks face is the procyclicality of liquidity that leads to fire sales, and illiquidity occurring in financial downturns, thereby negatively affecting already suffering balance sheet positions. Shleifer and Vishny (2011) argue that the most common mechanism for fire sales in financial markets is collateralized lending. Entities for which a lot of funding comes from short-term collateralized lending agreements can become forced sellers if there are falls in the value of the collateral they are posting.11 Investment funds can exacerbate fire sales in financial systems with significant negative effects on the balance sheets of other actors in markets in three ways: (i) by reducing collateral values; (ii) by reducing credit financing for banks, firms, and governments (Cortes and others 2016); and (iii) when their own financiers or investors pull out (including lending by prime brokers that creates a bank-funds loop). Hence, under certain conditions, investment funds can act as systemic risk amplifiers. During the 2008 crisis, the run on money market funds contributed to fears of a systemic collapse, and in 2011 and during the Greek crisis, the withdrawal of U.S. money market funds from Europe led to funding shortages for European banks. Other examples include the bailout of investment funds by banks during the global financial crisis.

Fire sales interact with illiquidity spirals. Funding liquidity concerns are closely related to issues of market liquidity and therefore affect the risk of fire sales. Brunnermeier and Pedersen (2009) find that traders’ ability to provide market liquidity depends on their ease of funding, and vice versa. This mutually reinforcing relationship can lead to liquidity spirals in which the risk of fire sales, and concomitant discounts on assets, increases.12

Deleveraging of financial institutions often leads to high levels of discounts in asset prices. Recent models capture the characterization of the mechanism that would trigger a levered investor to shed assets and incur a price impact sufficient to provoke a fire sale and margin spiral. One approach, for example Greenwood and others (2015) employs leverage targeting. The authors examine the deleveraging of European banks during the sovereign debt crisis of 2011–2012 and propose a model that analyzes the effect of deleveraging on asset prices and the level of contagion as a result of depressed asset values.13

Deleveraging by levered entities. While hedge funds, for example, are relatively small in global terms (with US$3 trillion in assets under management), their use of derivatives, active portfolio management, quantitative trading, high frequency trading, and leverage increases their risk potential and makes their importance larger than suggested by their size. Leverage in hedge funds differs significantly across funds. Although the average hedge fund has much lower leverage than an average bank, some hedge funds’ business models involve high leverage and a large market share, which can lead to potential problems, as seen in the failure of Long Term Capital Management in 1998.

Adrian and Shin (2010) find evidence that marked-to-market leverage in financial intermediaries is strongly procyclical, implying that these financial institutions actively manage their balance sheets to get high leverage during booms and low during downturns. Adrian and Shin (2014) further investigate these results and find that banks aim to keep value-at-risk relatively constant throughout the cycle, implying that financial intermediaries de-risk in downturns and load up on risk in good times, acting procyclically.14

Information asymmetry channel

Information asymmetry has been recognized as a key cause of bank runs. In the face of a shock, information about the causes and magnitude of the shock and the risky exposures of each bank are often not easily available, because all involved parties have a strong incentive for guarded self-interest. Since information processing and analysis is often difficult and costly, participants generally require time and resources to paint a misleadingly optimistic picture of the situation, and consequently no hastily produced optimistic statements from them are likely to be believed. This makes it hard to distinguish solvent parties from insolvent in periods of high uncertainty, and market participants increasingly tend to make their portfolio adjustments based on rather crude or simplistic assumptions. Such adjustments are often implemented in an unsophisticated or ill-thought-out manner—for example, an extreme flight to quality, ceasing to lend at all, or doing so only to a small subset of clients. Because these runs are concurrent and widespread, they are likely to exert strong downward pressure on the prices (upward pressures on interest rates) of securities held by affected financial institutions and markets. Any resulting liquidity problems are likely to spill over to banks not directly affected by the initial shock.

The extent of information asymmetry in financial contagion has been the subject of extensive academic research. Informational frameworks that may drive the manner in which banking crises develop have been fairly extensively analyzed. Jacklin and Bhattacharya’s15 research analyzed the problem of two-sided asymmetric information, in which banks do not know the liquidity needs of depositors and depositors do not know the quality of bank assets. Other analyses of information-influenced contagion are similarly based on information asymmetries. As pointed out by Khandani and Lo (2011), in summer 2007, financial markets began to notice some disturbing irregularities in market behavior. The U.S. subprime mortgage market seemed to be transmitting risk to other parts of the market for reasons that were hard to understand. The authors provide a possible explanation: that the initial shock caused by the subprime market changed the structure of information, which resulted in contagion through a chain of fire sales.16, 17

In periods of high uncertainty, the impact of information asymmetry becomes more severe. Informational frictions render fundamental solvency less relevant during crises and can lead banks to asset fire sales aimed at addressing cash flow and liquidity concerns. Kapadia and others (2012) argue that banks in “defensive positions” during crises make decisions with systemic implications in order to address funding constraints and to guarantee incoming cash flows. To do this, banks can cut maturities on their interbank loans and sell assets below market value. Kapadia and others write: “In such funding crises, the stock solvency constraint no longer fully determines survival; what matters is whether banks have sufficient cash inflows, including income from asset sales and new borrowing, to cover all cash outflows.” The actions taken by banks to shore up their cash flow in order to reassure markets about their solvency may have systemic consequences.

Strategic complementarity channel

Strategic complementarities can lead to firms taking similar actions at the same time, amplifying financial losses. Bebchuk and Goldstein (2011) present a model where operating firms are interdependent, with their wellbeing dependent upon the ability of other operating firms to obtain financing. In such an economy, an inefficient credit market freeze may arise in which banks abstain from lending to operating firms with good projects because of their self-fulfilling expectations that other banks will not be making such loans. Similar scenarios can also arise in other situations, for example, lending in the interbank market.

Complementarities affect both investors in and creditors of any institution, and have perhaps been most thoroughly analyzed for mutual funds. Chen and others (2010) analyze the emergence of financial fragility in the mutual fund sector where funds hold more illiquid assets, noting that these funds suffer greater redemptions following bad performance. Goldstein (2017) reached similar conclusions regarding corporate-bond funds. During the LTCM episode as the market became aware that LTCM was losing money, investors became unwilling to buy or hold assets for which large further forced sales could be anticipated. More generally, during crises, asset return correlations are often primarily driven by the presence or absence of distressed sellers rather than by asset properties. Morris and others (2017) discuss asset managers’ liquidation decisions. Faced with a requirement to sell some number of holdings they might choose to preferentially sell more liquid holdings, or by contrast, to sell a greater proportion than required, and hoard cash. Both the decision and the assumptions made by investors about that decision have systemic implications.

B. Systemic Risk Amplification: Empirical Methods

There are considerable challenges to developing operational models that can convincingly and comprehensively capture systemic risk amplification. Data constraints, the range and variety of amplification mechanisms, the difficulty of modeling them, and how they interact in complex financial systems all impose significant impediments. Moreover, authorities face important challenges in the calibration of scenarios to be used in stress tests.

The decisions that lead to a crisis take years before the crisis event. In the words of Andrew Crockett (2000): “The received wisdom is that risk increases in recessions and falls in booms. In contrast, it may be more helpful to think of risk as increasing during upswings, as financial imbalances build up, and materializing in recessions.” This is in line with Minsky’ s notion of destabilizing stability and is verified empirically in Danielsson and others (2018), who find that low volatility predicts crises five to 10 years into the future. Policymakers therefore need to be able to identify risk-taking in as close to real time as possible, and take corrective actions. Otherwise, there is the risk of the policy initiative acting procyclically (see Danielsson and others 2016).

Data

A major hindrance for stress testing is the availability of appropriate data. This is a common issue for both micro- and MaPSTs. However, since MaPSTs aim to capture direct and indirect sources of systemic risk amplification and how such amplification mechanisms change their structure, as well as the speed of risk propagation under shocks, data constraints are more restrictive for MaPSTs.

  • Direct contagion. A thorough analysis of direct contagion requires data on the exposure networks of financial institutions. The lack of such data—even for the core banking sector—has led to the use of alternative techniques such as network reconstruction (Upper and Worms 2004) or analyses of payments data (Furfine 2001). Some policy institutions have recently gained access to data on interbank loan exposures; however, data on other assets’ cross-exposures is usually non-existent or very costly to obtain. This is especially problematic when it comes to cross-border exposures, since data is collected nationally and often not made available for international analyses, thus preventing the mapping out of both sides of exposures. In most cases, only aggregate data on the interbank exposures of each bank to banks located in select countries are available for research on MaPSTs. Moreover, even data on interbank loan exposures have typically only covered the core domestic banking network. No satisfactory database has been developed to capture cross-exposures and the risky influence of nonbanking institutions.

  • Indirect contagion. Reliable data to model amplification mechanisms such as fire sales and information asymmetries are even harder to obtain. The analysis of fire sales requires data on the holdings and transactions of securities for the whole financial system, involving asset managers, insurers, other nonbanks, and banks. Data on the interconnections of different parts of the financial system are particularly patchy, and the picture is constantly evolving. It seems likely, for example, that new regulatory constraints imposed on banks have led to nonbank entities taking on intermediation involving risk that was formerly conducted by banks, and this “shadow banking” has in turn created new channels of contagion between banks and shadow banks.

Even for existing data, deficiencies remain. Although large amounts of publicly available and confidential supervisory-level data are available, the most promising sources of data are typically inconsistent, fragmented, geographically restricted, costly, and therefore hard to assimilate. Determining the relative importance of each data set is, consequently, a difficult conceptual problem.

Accounting data. Analysts, policymakers, and researchers have been able to use increasing amounts of publicly available accounting data, gathering complex and fairly detailed accounting data on banks as well as other markets, including real estate investment trusts, asset management firms, and insurance companies. Financial institutions publish annual reports and financial results in which balance sheet data are available. While this type of data targets the fundamental values of entities’ assets and liabilities, several aspects merit caution:

  • Accounting values can be inconsistent with market values. Hence, they might not reflect values that investors would actually pay for assets and liabilities. As informational frictions render fundamental solvency less relevant during crises, market participants tend not to believe reported numbers unless they can be verified—and book values typically cannot be usually verified. To make matters worse, the gap between accounting data and market values inevitably widens during periods of distress, as liquidity becomes more important and long-term equilibrium value relatively less so. Finally, accounting data tends to miss risk exposures related to both off-balance sheet items and complex derivative instruments. Stressed entities have a particularly strong incentive to hide exposures in this manner and so the discrepancies will be widest for precisely the entities that should be of most concern.

  • Accounting data are backward-looking and usually updated infrequently. Reported data can change from one year to the next, solely because of adjustments resulting from a financial institution’s recalculation.

  • These data usually lack granularity. Financial reports do not often detail the data at the branch or subsidiary level, or at the national or international level. This impedes the assessment of potential common risk and asset exposures of subsidiaries in different countries, and prevents analysis of the flows among subsidiaries, and between the subsidiaries and other financial institutions. Moreover, granularity frequently falters at the balance sheet level and therefore does not properly reveal, for example, the level of risk on the asset-and-liability side of the financial institution, but only indicates aggregate numbers with very low levels of detail. Furthermore, some vendors have created useful databases18, but these databases can suffer from the same issues of data granularity and data quality as their sources. The BIS provides statistics on international banking activities and banks’ foreign positions. However, these statistics largely remain aggregated, do not provide sufficient detail on banks’ assets and liabilities and, therefore, may not be optimal for a detailed analysis of banks’ activities.

Supervisory-level data. While regulators usually have access to more granular data for banks operating in their own jurisdiction, most of the reported supervisory data are accounting data. Hence, concerns related to consistency with market values, the data’s backward-looking nature, lags, and slow updates remain. New databases, for example on OTC derivative trading, are now available for research. Supervisory-level data are usually inaccessible for institutions located in other jurisdictions. Data collection processes can vary across jurisdictions, and in some cases, data might become irrelevant for systemic risk measurement, as, for example, if data are published on an aggregate level. More important, data collection is always a sensitive process and the results are highly confidential, further impeding cooperation among authorities across jurisdictions. There are, however, several initiatives attempting to overcome these issues (Section IV). In addition, academic access to such data is steadily increasing in many jurisdictions.

Market data. The advantages of market data are that they are readily available in many developed and large emerging markets; they are usually updated daily or even more frequently, and they are more trusted than other data sources by practitioners, and so can be expected to have substantial impact on crisis behavior. In principle, they also allow high-frequency monitoring of risks. The public availability of market data also means that market price-based models can be used, and indeed are being used, for stress tests by private sector analysts, academics, and policymakers who have difficulty in accessing sufficiently complete supervisory data. However, there are some issues to consider:

  • Market data can be noisy. They may overestimate or underestimate risks, even without relation to financial entities’ fundamentals. However, as Demekas (2015) argues, market data can often reflect information not yet known to (or fully understood by) supervisors. In addition, regardless of their relationship with “fundamentals” (or absence thereof), market trends are often self-fulfilling because, as argued earlier, market perceptions can precipitate herding in periods of stress. Therefore, a blanket dismissal of all such signals as “noise” may be a mistake, especially as stress in markets increases.19

  • Data may not be available for all institutions in all countries.

  • Shallow or illiquid markets may render prices uninformative.

  • Market data cannot predict market movements from one day to the next. Market prices, in theory, embed forward-looking information on market expectations that can change in a day. However, market data primarily reflect the outcome of behavior by market participants, not the information that influences them when they made their market decisions. The time lag between the decision to take on risk and the eventual outcome of the decision could be many years or even decades. Market data is more likely to focus on ex-post outcomes rather than ex-ante information. Therefore, any signal may come too late for policymakers to react. It is best to think of market data as a “thermometer.”20

Flow data. A growing trend in recent years has been to collect flow data, in order to determine how fast and to what extent investors can move from one asset class to another, or from one country to another. Measuring the intensity and speed of change in international financial positions, and more important, understanding the causes and consequences of such moves, requires several elements: (i) the timing of inflows and outflows to or from a country or an asset class; (ii) the geographic allocation of an investor or financial institution; (iii) country and sector flows; and (iv) overall indicators of investors’ and financial institutions’ risk appetite dictated by their cash or reserve positions. The availability of these data is not uniform across countries, markets, and asset classes. However, some commercial providers have begun to make such data available.21 Nevertheless, flow data remain relatively scarce and are not systematic at the international level. Some initiatives have aimed at documenting flows between asset classes, and if made available to macroprudential policymakers, could be valuable.

Methods

SRA mechanisms are diverse, complex, and can change their structure and magnitude at different points in time. Policymakers must consider these characteristics so that models are sensitive to these changes and provide a valuable assessment of risk during a crisis, when strictly historical information becomes an especially poor guide for the future. This raises many challenges for modelers seeking to capture systemic risk.

  • Crisis-consistent estimations. Models must be designed to provide information conditional to a current crisis, even though crises in general are infrequent and therefore contribute little to statistical relationships drawn from historical data.

  • Structural changes. Financial systems have experienced significant structural changes over the past few years due to technology, market shocks, regulatory changes and the changing relevance of existing and new players in systems (for example, central counterparty (CCP) clearinghouses). Regulators and participants are all highly aware of past crises, and it can be assumed that most participants will be well hedged against them. Consequently, past relationships will never be a sufficient or reliable benchmark for estimating or calibrating models.

  • Non-linear changes. Quantification of SRA is difficult because of the non-linear increases in the magnitude and speed of loss propagation observed during financial crises. The potential for model error is considerable. Indeed, crises only emerge when some important part of the market finds that they were in error in some key assumption. Furthermore, it is during crises that models often fail.

  • Interpretable metrics. Models must provide an evaluation of systemic risk that is interpretable by policymakers, and thus useful for their policy decision-making regarding systemic risk.

  • Model risk. “Model uncertainty” has been defined as model misspecification at the level of the individual financial decision maker. “Risk of model uncertainty” in the macroprudential context refers not only to the possibility that any one agent’s model is wrong and can lead to unfavorable outcomes for the agent, but more crucially, to the resulting risk of financial instabilities at the system level arising from the use of such wrong financial models, and from the way the resulting decisions interact in the aggregate (Cont 2006). Danielsson and others (2016) find that model risk increases with market uncertainty.

Alternative frameworks developed by authorities around the world attempt to capture amplification mechanisms in stress testing. While these developments are welcome, the coverage of amplification mechanisms in models currently implemented is, in most cases, restrictive. The taxonomy discussed in the previous section allows us to classify these models according to the amplification mechanisms that they aim to quantify. With some exceptions, analyses focus mainly on direct contagion (through limited coverage of contractual obligations), and, to a lesser extent, on macrofinancial feedbacks, and to a very limited degree on indirect contagion. Major central banks are already developing a modeling agenda that attempts to cover any revealed gaps. Tables 310 summarize approaches to measuring amplification mechanisms within stress testing frameworks that are currently implemented by authorities in key financial markets. The most common approaches that endeavor to capture SRA mechanisms in stress tests include:22

  • Models for macrofinancial feedbacks. Some implementations include acceleration mechanisms stemming from liquidity constraints, which lead to frictions between the financial market and the real economy. Initial efforts to capture macrofinancial feedback effects have focused on linear VAR models, which have been useful for understanding the basic properties of data with respect to serial correlation, breaks, and endogeneity. There is, however, a growing understanding that the relationship between the real economy and the financial system displays non-linear properties. Periods of sustained financial fragility do not merely display greater volatility; there is a discontinuity in the fundamental relationship between the financial system and the macroeconomy. Moreover, a common feature in this literature is the reliance on ad-hoc measurements of financial stability that typically feature a weighted average of various spreads and interest rates. Ideally, macrofinancial feedbacks could be assessed using theoretically sound measures of financial stability that also incorporate the non¬linear dependence and contagion among financial institutions during times of distress. This means that linear models are generally unsuitable. The literature on empirical, non-linear models of macrofinancial linkages is still rather sparse, but an increasing appreciation for the changing relationships across time is forming in the academic community. Thus, efforts have been made to incorporate non-linearities observed in macrofinancial feedbacks through non-linear econometric methods.23 Examples include Markov-switching vector autoregression (MS-VAR) that allow for non-linear relationships among the endogenous variables modeled. However, the state-switching process is assumed exogenous, and usually volatilities in different regimes are assumed fixed.

  • Models capturing direct contagion through contractual obligations. These models usually include cross exposures of interbank loans, but very rarely include cross exposures of other bank-to-bank assets/liabilities or cross exposures of bank-to-nonbanks’ assets/liabilities. The most widely used approach to capturing cross-exposure is through network models. These models try to quantify contagion losses through the propagation of credit and funding risk in a system given contractual obligations (exposures) across participants. A loss in a bank’s counterparty triggers default that is passed on to the bank in proportion to the size of their contractual obligations. This usually occurs without any spillover because of indirect contagion to the value of counterparties’ other non-contractual assets. Network models are often added as an additional module to MiPST frameworks that are built on entities’ balance sheets and income statement models. This literature dates back to Eisenberg and Noe (2001), Furfine (2003), and Cifuentes and others (2005). Gai and others (2011) developed a network model to study how repo market activity, haircuts, and liquidity hoarding can amplify overall risks in the interbank market. In a similar way, Chen (2014) presents a model in which the liquidity and network channels interact to spread the impact of an individual default among members of the network. A key disadvantage of these models is the extreme level of data granularity required to map interbank lending and identify additional cross exposures. In practice, regulatory fragmentation means that sufficient data to support an international analysis is simply not available to any single entity. Another drawback is that these models usually capture a low amount of systemic risk (see BCBS 2015). A reason for this might be that most applications often only capture unsecured lending between banks and fail to include other positions such as derivatives, cross-holdings of securities, and so on.24 However, while there is the possibility to expand the coverage of these models, though possibly at a high cost, it is questionable whether the cost of such expansion can justify the benefits, given that a basic problem with these models is the mechanistic manner in which losses are treated.

  • Models capturing indirect contagion from fire sales and information asymmetry. These models include contagion from the interaction of various markets, including interbank loans, other assets/liabilities within the banking sector and assets/liabilities within the nonbank sector. While some central banks are developing datasets to measure these effects and are making important advances in this area, there is not yet a set of established analytical methodologies that can both capture indirect contagion in a comprehensive manner and that can be widely implemented in stress testing frameworks for policy purposes. However, progress is ongoing. Some authorities are focusing on agent-based model (ABM) which try to explain behavioral responses among agents in the system (BCBS 2015). ABMs build on the contributions of behavioral economics in order to better explain microeconomic behavior of agents in financial markets.25 These models include a heterogeneous set of agents, as well as a topology that describes their methods of interaction within an environment. They therefore attempt to go further than network models by incorporating the heterogeneity of agents and banks and their behavior. Criticisms of these models often question the assumption that all topologies describing significant interactions across agents can be understood, foreseen, and embedded into the models. Moreover, there is an implicit assumption that topologies included in models remain stable aftershocks to the system. There are also issues related to the calibration of models, and the detailed information necessary for the calibration and the models’ computational feasibility. Although, from a conceptual perspective, ABMs represent an attractive approach to incorporating amplification mechanisms in stress tests, it is difficult to judge their tractability because of their modeling and computational complexity (Demekas 2015). In addition to ABMs, other approaches are under development. In the next section, we describe approaches being developed by some authorities that, in our view, represent viable options for countries subjected to diverse data constraints.

Table 2.

International Stress Testing Frameworks

article image
Sources: Dent and others (2016) and IMF staff.
Table 3.

SRA Modeling: BoE

article image
Sources: BoE and IMF staff.
Table 4.

SRA Modeling: ECB

article image
article image
Sources: ECB and IMF staff.
Table 5.

SRA Modeling: Bank of Japan

article image
Sources: BoJ and IMF staff.
Table 6.

SRA Modeling: HKMA

article image
article image
Sources: HKMA and IMF staff.
Table 7.

SRA Modeling: Reserve Bank of India

article image
Sources: RBI and IMF staff.
Table 8.

SRA Modeling: Banco de México

article image
Source: Banco de México and IMF Staff
Table 9.

SRA Modeling: BoC

article image
Sources: BoC and IMF staff.
Table 10.

SRA Modeling: U.S. Federal Reserve

article image
article image
Sources: U.S. Federal Reserve and IMF staff.

Complexity can quickly become a problem, resulting in intricate frameworks. This is especially the case when a single analytical approach, like a balance sheet model designed to assess vulnerabilities of individual entities, is boosted with “appended satellite models” that make use of supervisory information and try to capture SRA. It is often difficult to expand frameworks if additional data or amplification mechanisms are to be explored. Moreover, many of these approaches rely on defined structures that assume fixed structures for amplification mechanisms. Therefore, their results can only capture effects that are explicitly modeled. This could be problematic given that structures of these mechanisms can change during crises. Finally, while approaches of this type might work for precise data specifications and assumptions regarding financial system structures, they are usually non-replicable by arm’s-length institutions, academics, or market analysts. This makes it difficult to develop a broad-based approach that would allow implementation of consistent stress tests and vulnerability analysis across jurisdictions. Therefore, rather than expanding single modeling approaches into complex frameworks, we support the development of EFs, as described in the next section.

Scenarios

The proper calibration of scenarios is essential for the usefulness of MaPSTs. The approaches proposed by some authorities include countercyclical scenarios for stress test evaluation (BoE 2015 and Tarullo 2016). A severe scenario in financial upturns helps ensure that banks build buffers against increasing systemic vulnerabilities. Likewise, a less severe scenario in financial downturns would help release buffers and avoid tightening financial conditions, which can worsen real economic activity. Moreover, different degrees of scenario severity might also provide information to authorities about the readiness of financial entities’ management, and the effectiveness of systems at hand to confront stressed situations. Last, the proper calibration of scenarios can also be useful for fine-tuning crisis resolution frameworks, for example, by testing the effectiveness of responses, by testing communication and coordination systems, and so on, according to different degrees of severity of crises.

Picking the proper severity level for a scenario is a difficult problem. For example, the BoE (2015) proposes an approach in which both advanced empirical analysis with a focus on tail risk and regulatory discretion are used to define scenarios. Yet, the success or failure of the stress test hinges on regulators being able to identify and test the correct scenarios. Figure 1 illustrates a well-documented phenomenon, namely that contagion and spillovers are often non-linear and lead to a sudden jump in system-wide losses if the severity of a stress scenario hits a certain threshold (Elsinger and others 2006). This non-linear behavior has implications for scenario selection, as it makes it less obvious which scenarios cause significant systemic risk amplification and which ones do not.

Figure 1.
Figure 1.

Pattern of Systemic Losses with and without the Effect of Amplification Mechanisms

Citation: IMF Working Papers 2018, 197; 10.5089/9781484375839.001.A001

Source: Elsinger and others (2006).Note: Scenario B is marginally worse than Scenario A; however, the losses under scenario B are significantly higher than under scenario A.

This is compounded by the fact that whether or not systemic risk amplification occurs is not just a function of the severity of the initial shocks; but of the resilience of financial institutions’ balance sheets at a given point in time and so their ability to absorb rather than amplify those shocks. Considering Figure 1, if ‘severity’ refers to prescribed shocks to asset prices for example, at two different time periods the point at which contagion kicks in could be at very different places along this line—a banking system with plenty of excess capital will be able to absorb much larger shocks than one with very thin buffers. Since systemic risk arises endogenously, the severity of the first-round impact, and the ability of institutions to absorb it, must be considered to understand how and whether significant contagion is likely to occur.

Nevertheless, for a particular conjuncture, this tipping point effect will hold. By selecting scenarios just short of the threshold, regulators may build a sophisticated model that underestimates systemic risk. Selecting a slightly more severe scenario might cause such severe losses that, were regulators and policymakers to use them to mechanically impose large increases in capital requirements, they could endanger the legitimacy of the stress testing exercise and cause political backlash.

Using the results of the MaPST in such a mechanical way however may not be appropriate. Systemic risk amplification mechanisms often kick in beyond certain tipping points—low levels of capital or liquidity adequacy at which institutions start to default, or deleverage rapidly, or start withdrawing from funding markets, for example.

When modelling systemic risk amplification mechanisms, appropriate policy responses could be calibrated to prevent institutions from reaching potential tipping points, rather than to absorb the full extent of losses were they to occur. To take a simple example: if one bank’s default leads to additional losses for other banks; and if the existing policy response is to require that bank to hold sufficient capital so that it would survive the scenario; then those other banks would not in fact face those additional losses in a real stress, and should not be capitalized against them (as this would constitute double-counting).

Challenges doubtless remain for regulators where there are time lags involved in banks raising extra capital against these risks. But the point remains that a qualitatively different approach is required when thinking about policy responses to losses caused by amplification mechanisms, due to their intrinsic interdependence with institutional resilience.

An additional challenge comes from the fact that it may become necessary to incorporate political factors and consequences into the design of stress test scenarios. There is increasing awareness of the interplay among the political system, financial markets, and financial regulations. If this political dimension is not taken into account, as argued by Danielsson and Macrae (2016) the credibility of a MaPP may be weakened, and where, in addition, the macroprudential authority may find it difficult to respond to political events because it could be seen as intervening in the democratic process. It therefore seems necessary to have strategies for dealing with political risk in MaPST design.

IV. Macroprudential Stress Tests: Going Forward

Given the current limitations to modeling SRA mechanisms, we support the use of a variety of data and methods under EFs. No data or modeling approaches are sufficient for capturing SRA mechanisms; however, many of these approaches have positive features that could, in principle, complement each other, and in turn provide policy makers with an enhanced perspective, as opposed to the narrow analysis obtained from only one data source or model. The challenge is to combine the results of different methods into a coherent set of information in order to obtain tractable results. This can be addressed through the development of EFs. These are organized yet flexible operational entities that combine elements of separate analyses in order to provide for a more integrated and comprehensive understanding than a single path analysis can deliver.

EF should be designed to allow for the analysis of vulnerabilities due to systemic risk through alternative modules incorporating diverse data and methods. Once adequacy of results is verified, there should be a defined procedure that allows the analyst to enrich her views by drawing information from a variety of approaches in a consistent manner. For example, this could be done by designing a framework that defines “connecting metrics” obtained from, and used in, different modules. EF could be used to connect models (whereby each model produces an input for the next model) to generate some output that is more than the sum of its inputs, and potentially also more robust. We believe that this represents a pragmatic approach to integrating diverse types of data and modeling methods in a consistent way, providing policymakers with three key benefits:

  • Improved insights by taking advantage of complementary methods.

  • Different perspectives on risks.

  • Reduced exposure to the risks and limitations of a single model or single analytical framework, due to the use of a diverse range of data sources and approaches.

EF can be particularly useful to national regulatory authorities. In many cases, national authorities are developing frameworks that take into account data specific to their countries (we discuss some examples below); however, frameworks can also be constructed with publicly available data that is accessible in numerous countries (we refer to an example of this type below, developed by the IMF). The latter can be useful in advancing analyses cooperatively with other authorities (for example, monetary authorities, bank supervisors, insurance supervisors, and so on) who employ their own analytical tools and data. This EF can serve as a high-level tool that can draw upon transferable inputs from cooperating entities in addition to other information in order to provide a system-wide analysis. These frameworks can play a similar role in international efforts (either bilateral or multilateral) to achieve an assessment of systemic risks at the regional or global level.

Despite data constraints and the complexity of modeling SRA mechanisms, encouraging developments are underway. Many authorities have prioritized the improvement of data collection and development of frameworks that incorporate amplification mechanisms in stress tests. Although this work is underway in many countries, in many cases, models have not been officially vetted; hence, they have not been publicly disclosed. Yet, in some other cases, country authorities have publicly announced strategies for the development of frameworks that rely on a variety of models, and have made it clear that they will develop frameworks that combine the information produced by different models in a consistent manner, thereby taking a similar approach as the one proposed in this paper.

Data. Since the global financial crisis, important progress in data enhancement relative to crisis mitigation has been made through the IMF/FSB/G20 Data Gap Initiative and other data initiatives, such as the FSB’s work on monitoring shadow-banking risks. However, the development of databases that allow for a comprehensive analysis of contagion will be difficult to achieve in the foreseeable future. Nevertheless, Banco de México offers a good example of how to enhance databases that assess direct contagion by expanding their databases from interbank loan exposures to other assets, including securities, derivatives, and foreign exchange exposures. The set of variables, data templates, IT systems, and legal provisions under which Banco de México collects this data could be a useful example for other countries looking to enhance their collection of direct exposure data. While this is encouraging, attention should also be given to the development of databases that improve the measurement of indirect contagion. As we explain below, some efforts are already making strides on this front.

  • One example of promising supervisory-level data is credit registers. These have recently been made available for research in a number of countries, and such data have the potential to provide early indicators of crisis along the mechanism, see Schularick and Taylor (2012).

  • Trade repository data and records of OTC derivatives trading is increasingly available for macroprudential research, as EMIR in the European Union. Such data have the potential to elucidate how derivatives link financial institutions and identify channels of contagion. However, the use of such data in Europe is currently hampered by governance restrictions, whereby researchers may only see one side of cross-border trades.

  • There are a number of commercial databases that are actively used to explore interconnections between financial institutions and channels of contagion. An example is given by the use of securities lending data, which captures how market participants lend financial securities to other market participants for a fee and against collateral, (see, for example, Adrian and others 2013). This creates exposures that might create vulnerabilities in times of stress. While aggregate securities lending data is available commercially, information with counterparty identities could be used by researchers in supervisory institutions.

  • Financial institutions trading in derivatives validate their internal pricing models through a commercial service called TOTEM. Such data have become available to researchers and commercial entities and can be used to identify and model risk, the term structure of risk and transmission channels between risk across maturities, instruments, and countries.

Macrofinancial feedbacks. More recent applications of macrofinancial feedback methodologies have made use of time-varying parameter structural VAR models, in which coefficients are allowed to break continuously. These approaches hope to improve the measurement of nonlinear relationships between macro and financial linkages, and usually involve a different coefficient and covariance matrix for each time period in addition to featuring stochastic volatility. The Bank of Japan has been a pioneer in the development of this type of model and offers a good reference for how these approaches could be made implementable in stress tests (see Table 4).26 The Bank of Canada (BoC) has developed a Bayesian Threshold VAR to generate quantitative macrofinancial risk scenarios where the switching process between low- and high-financial stress regimes is endogenous and depends on initial conditions and shocks. The IMF is also currently developing a non-parametric Bayesian VAR to measure time-varying macrofinancial linkages (Bazinas and Segoviano 2017).27

Losses due to systemic risk amplification. Despite the complexity of SRA loss modeling, some authorities have made significant strides in this area and have developed frameworks that combine different types of modeling approaches (structured and closed form), as well as different types of data (supervisory and market based). Examples include:

  • Bank of Canada. The BoC has developed an approach for modeling fire sales in interbank loans. The model captures contagion driven by bank deleveraging when banks are leverage-ratio constrained. The model quantifies negative externalities for other banks via mark-to-market effects on securities portfolios, which in turn might generate additional leverage-ratio constrained effects, and hence additional fire sales. Table 8 provides details of the model.

  • Bank of England. The main features of the BoE’ s stress-testing framework strategy for 2018 represents a good example of a comprehensive EF.28 Its framework aims to use “a suite of models,” and the strategy places significant importance on the development of methods that quantify losses from systemic risk amplification. As the BoE takes up its supervisory powers through the establishment of the Prudential Regulatory Authority (PRA), it gains access to highly granular data. Table 2 provides details of the models that are currently under development to quantify direct and indirect contagion.29

  • European Central Bank. The ECB framework, named “Stamp€” (Stress Test Analytics for Macroprudential Purposes in the € area) was first introduced in Henry and Kok (2013) and consists of four pillars: (i) The macrofinancial scenario design; (ii) models to translate scenarios into impacts on banks; (iii) the solvency calculation module; and (iv) the module for contagion and feedback analysis. Here we focus on the last pillar. The framework is based on a number of different building blocks and models that are linked together consistently and dynamically to provide a flexible tool for assessing banking sector resilience against identified systemic risks. The ECB’s EF has been developed to support its contribution to safeguarding financial stability and its financial sector-related work in the context of EU-IMF Financial Assistance Programs. It is also used to challenge results from bottom-up stress tests conducted by banks and their supervisors. As the ECB takes up its supervisory powers in the context of the establishment of the Single Supervisory Mechanism, it has also gained access to highly granular data. Table 4 provides details of the models that are currently under development by the ECB to quantify direct and indirect contagion.

The IMF is developing EF aimed at integrating diverse types of data and modeling frameworks. As opposed to authorities with supervisory powers, the IMF operates under highly restrictive data limitations, especially for the estimation of contagion losses. Moreover, the IMF is tasked to analyze vulnerabilities of a vast group of financial systems with heterogeneous characteristics, which renders methods that rely on fixed data configurations and granular data unsuitable. Therefore, the IMF must develop general approaches that can be implemented with different types and granularity of data and can incorporate alternative methodologies.

An EF under development by the IMF combines MiPST models with the quantification of SRA losses valued through a reduced-form approach using publicly available data. The proposed framework will use a range of MiPST models to determine the “first order” impact of the stress scenario and then overlay the effects of any risk amplification mechanisms estimated with publicly available data. Box 2 provides further details about this EF.

  • This approach makes use of MiPST frameworks that are already implemented by assessed firms (bottom up), authorities, or IMF staff (top down),30 with proprietary or supervisory data.

  • For the estimation of amplification losses, the EF relies on an estimate of the market perception of financial systems’ distress dependence structures based on observed probabilities of distress. Perceptions of dependence are clearly relevant for crisis-contingent estimates, especially in periods of stress, as market trends can become self-fulfilling. Hence, contagion loss estimates embed realistic market reactions and become computationally simple and relatively light on data requirements. Using market-based estimates is an advantage of the framework, given the data limitations faced by the IMF which make the proper calibration of methods that rely on ex-ante modeled structures very challenging.

  • Because of computational simplicity and ready availability of data to estimate SRA losses, the proposed EF is a cost-efficient approach to implementing MaPST. Importantly, computational simplicity does not come at the expense of analytical rigor (Demekas 2015).31

As publicly available data becomes richer and theoretical models improve, future EF will combine theoretical models with reduced-form approaches. For example, structural general equilibrium macrofinancial frameworks that attempt to capture some of the systemic risk amplification mechanisms described in the previous section are being developed to be implementable with data publicly available in numerous jurisdictions. 32 Given the difficulty of properly calibrating these frameworks, the IMF proposes to use reduced-form approaches to improve their calibration, and thus make the output of theoretical models more consistent with empirically observed outcomes.33 Note that the cost efficiency of the proposed reduced-form approach allows for the parallel running of frameworks that provide policymakers with enhanced understandings, combining the benefits of improved measurement of reduced-form approaches with better insights of theoretical models.

Quantifying SRA Losses Based on a Reduced-Form Approach: An EF Developed by the IMF

This EF supplements loss estimates of individual entities from MiPST modules with contagion losses from SRA mechanisms that are based on publicly available and market-based information and estimated by the “systemic risk amplification loss” (SRA-loss) module put forth in Alla and others (2017). The quantification of SRA losses is based on the valuation of bank assets and the realization of specific events such as the failure of a financial entity, or a scenario in which a group of entities falls into distress. These SRA losses are conditional on stressed macrofinancial scenarios and the realization of specific events that are typically marked by a single financial entity, or a group of financial entities falling into distress. Figure below summarizes the proposed EF.

uA01fig01

Encompassing Framework for Macroprudential Stress Tests

Citation: IMF Working Papers 2018, 197; 10.5089/9781484375839.001.A001

  • 1. The framework takes a macrofinancial scenario as a starting point. Given the assumptions regarding the scenario, risk parameters (probabilities of default, loss given default, and exposures at default for different assets and entities) are individually estimated for each of the financial institutions analyzed (see left hand side of the figure)1.

  • 2. These parameters are used as inputs to estimate losses and profitability for each entity under the MiPST framework and scenario.

  • 3. These parameters are also employed as inputs to estimate the system’s multivariate density (and distress dependence structure) from which SRA losses are quantified (see right hand side of the figure). SRA losses are conditional losses, derived from the multivariate distribution representing the financial system and using an asset pricing model to compute the expected valuation of each firm’s assets under any stress event of interest. Obvious candidates for events that should be checked involve entities failing the MiPST (for example, the entities whose capital adequacy (CA) falls below a predetermined “hurdle rate” after the entity goes through the MiPST). Moreover, the impact of the default of any entity on the system (SRA losses) can also be analyzed in this framework. The approach, therefore, is stochastic since it allows analysts to estimate losses conditional on being on any of the tails of the multivariate density and the probabilities of such events. SRA losses can be added to the losses of individual entities estimated in MiPSTs.

  • 4. The multivariate framework also permits an easy integration of nonbank financial intermediaries into the analysis of systemic risk; thus, interactions between banks, insurance companies, pension funds, investment funds, and hedge funds can be considered for the quantification of losses due to systemic risk amplification mechanisms (Cortes and others 2017).

Therefore, the SRA-loss module permits policymakers to:

  • Quantify losses due to amplification mechanisms in the bank and nonbank sectors;

  • Assess whether specific entities would be able to “survive” (that is, if their CA would be above the hurdle rate) the additional losses brought by SRA; and

  • Calculate the contribution to SRA losses from each “connecting” entity in the system, and incorporate the decomposition of contributions into the likelihood of the event and the intensity (amount) of induced SRA losses.

In order to quantify contagion losses in a financial system, it is necessary to estimate the distress dependence structure across the entities operating in the system. It is necessary to estimate a system’s multivariate density in order to characterize the dependence structure that typifies how values of the entities in the system correlate. Therefore, when an entity suffers a shock, it is possible to quantify the likely losses suffered by another entity whose value is correlated to the entity that initially suffered the shock. We propose a statistical method (Segoviano 2006) to model a financial system’s multivariate distribution, using observed market data on stock returns and probabilities of distress (PoDs) of individual entities. The method constructs a multivariate distribution of asset returns that is consistent (in the sense of the default model proposed by Merton, 1973) with the observed PoDs and is closest to a prior distribution calibrated to match stock returns data. The construction of the multivariate density allows us to infer the (unobserved) distress dependence structure across the entities in the system, that is, the “system’s interconnectedness structure.” As PoDs of individual entities change across time, the distress dependence structure inferred by the method is updated. The method allows us to estimate distress-dependence structures consistent with market perceptions of risks that change as macrofinancial conditions evolve.

The proposed method of estimating SRA contagion losses incorporates useful features. Because market data is rapidly updated and embeds market views of risk spillovers from direct contagion (through contractual obligations across entities) or indirect contagion (through market price channels, including asset fire sales triggered by stressed entities, or asset sell-offs due to information asymmetries), the method allows us to:

  • Incorporate updates in a system’s distress dependence structure based on market perceptions of direct and indirect contagion across financial entities (that can reflect nonlinear increases in periods of high volatility) in a timely manner;

  • Quantify SRA contagion losses without, ex-ante, needing to assume structured agent interactions and behaviors that can change in unknown manners in periods of distress;

  • Estimate SRA contagion losses from readily available market information without the need for highly detailed and granular supervisory information that is not available in many countries, nor to institutions like the IMF; and

  • Compute complementary measures of systemic risk that provide supportive information to various systemic risk policy objectives (Segoviano and Espinoza 2017).

These features allow policymakers to get crisis consistent estimates of contagion losses and complementary measures of systemic risk that are interpretable, that incorporate market-perceived structural changes of agent interactions, and that can change quickly and nonlinearly in periods of distress.

By no means do we consider this EF to be complete or without limitations. As for its quantitative methods, there are intrinsic limitations that apply, as is the case for any quantitative methodology. Moreover, while the nonstructural approach to modeling contagion losses offers advantages by minimizing model error and simplifying implementation, the calculated estimates are generated from a reduced-form statistical model and, therefore, do not provide much insight into specific amplification mechanisms. Nevertheless, the quantification of contagion losses is absolutely necessary for assessing the potential magnitude of SRA losses, which is key knowledge for policymakers. Additionally, insights into transmission channels can be complemented by MiPSTs and some of the contagion approaches discussed above.

1 There is a suite of models that can be used to estimate these parameters, including Merton-type approaches, Value at Risk approaches, non-arbitrage models that rely on credit default swap spreads, bond spreads, and so on.

V. Linking Macroprudential Stress Testing to MaPP

MaPSTs provide information that can be useful for the definition of policies and calibration of instruments that have a bearing on systemic risk. Within the range of MaPP instruments, MaPSTs can most obviously be used to calibrate capital buffer requirements. This section discusses ideas on how to use information produced by MaPSTs to calibrate capital requirements in a way that is consistent with the Basel III framework. Ideas on how to use information provided by MaPSTs might be further developed as models improve and their use extends. Consistent with this, we discuss an alternative view of the implementation of capital buffers and pose questions that we consider relevant for policy makers to debate. We conclude the section by offering up ideas as to how information produced by MaSPTs can inform a wider set of policy tools.34

A. Calibration of Capital Buffer Requirements

The Basel III framework includes multiple layers of capital buffers to ensure the resilience of the financial system as a whole.35 Banks are required to meet the minimum total capital ratio of 8 percent of risk-weighted assets (RWAs) at all times. Additionally, banks are required to maintain the following capital requirements (Figure 2):

  • A capital conservation buffer (CCoB). This buffer consists of 2.5 percent of RWAs in CET1 on top of the minimum capital requirement outside periods of stress; the buffer, however, can be drawn down in stress periods. Drawing down this buffer does, however, impose distribution constraints on banks. Specifically, banks that draw on this buffer but are not yet in violation of minimum capital requirements can continue their operations, but must retain a significant portion of their earnings to rebuild their capital stock.

  • A countercyclical capital buffer (CCyB). The goal of this buffer is to enhance the resilience of the financial system to counter systemic risks emanating from the financial cycle (time dimension) while also reducing the procyclicality of bank lending. The CCyB can vary between zero and 2.5 percent of RWA, and should build up extra capital in boom times in order to absorb potential losses in economic downturns. The CCyB is set by national authorities, based on the prevalent state of the macrofinancial environment. Ideally, authorities should increase the CCyB during a lending boom and reduce capital requirements during a contraction.36

  • Surcharges for systemically important bank (SIB) capital surcharge. The SIB capital surcharges were introduced to protect the system from the structural dimension of systemic risk, therefore requiring an additional buffer commensurate to a bank’s contribution to systemic risk. The FSB sets a series of buffers for globally systemically important banks (G-SIBs) and national authorities can also set buffers for domestically important banks.

  • In addition to these buffers, regulatory authorities are able to set additional bank-specific capital buffers for banks that they regulate. There are a range of approaches taken by different regulators. In general, these buffers are set to ensure the capital adequacy of each individual bank.

Figure 2.
Figure 2.

Basel III Capital Requirements and Buffers1

Citation: IMF Working Papers 2018, 197; 10.5089/9781484375839.001.A001

1/ The above illustrates the minimum requirements presented in the Basel III framework. National authorities may have additional minimum capital requirements or other types of buffer requirements. More precisely, banks are required to meet the minimum CET1 ratio of 4.5 percent of RWA, the minimum Tier 1 ratio of 6 percent, and the minimum total capital ratio of 8 percent.2/ National authorities can impose a capital buffer requirement on SIBs that is higher than 3.5 percent. The Basel framework introduces capital surcharges for G-SIBs ranging from 1 to 3.5 percent. For banks that are systemically important both globally and domestically, the higher of G-SIB and D-SIB capital surcharges applies.3/ National authorities can impose a CCyB higher than 2.5 percent, while the mandatory international reciprocity applies only up to 2.5 percent.

The Basel Committee on Banking Supervision (BCBS) has proposed indicator-based approaches, along with supervisory judgment and prudence, to calibrate capital buffers.

  • For the CCyB, the BCB S provides a reference guide based on the aggregate private sector credit-to-GDP gap (BCBS 2010). This guide was based on an analysis that showed that a credit-to-GDP ratio of 10 percentage points or more above trend issues the strongest signal of an impending crisis (in terms of noise-to-signal ratio). According to the BCBS buffer guide formula, when the credit gap breaches a “lower threshold” of 2 percent, a decision to start increasing the buffer could be merited, if surveillance supports the judgment that systemic risk may be building up, and when it reaches the “upper threshold” of 10 percent, the CCyB should be set at 2.5 percent of RWA. It can also be set higher, based on broader macroprudential considerations (IMF 2014).

  • For the SIB capital surcharges, the BCBS has published a methodology for assessing and identifying global systemically important banks (G-SIBs), (BCBS 2013) and proposed a similar framework for domestic systemically important banks (D-SIBs) (BCBS 2012). The identification of SIBs uses indicators that capture four dimensions of systemic importance: size, interconnectedness, level of substitutability, and complexity. For G-SIBs, there is a fifth indicator: global scope of activities.37 Banks are ranked by their systemic importance based on the indicators and supervisory judgment, and placed in five buckets with a gradual scale of surcharge ranging from 1 to 3.5 percent.

MaPSTs may also contribute to calibrating these capital buffers. Since MaPSTs can give guidance on how much overall capital is desirable in order to withstand contagion losses from SRA mechanisms and assess the economic impact of changing levels of capital, they would be valuable tools in calibrating capital. Stress test scenarios can also be designed to be countercyclical, such that the degree of severity increases as the economy moves up the financial cycle.

Policymakers should ensure that banks are sufficiently capitalized to withstand a stress scenario of appropriate severity for the position in the financial cycle. This, while banks continue supporting the real economy through their banking activities. If a stress test suggests that the banking system as a whole is insufficiently capitalized, then policymakers might want to use system-wide capital buffers to respond. For example, a CCyB or sectoral capital requirements could be implemented. In addition, if individual banks in the system are systemically important and are shown to be insufficiently capitalized so as to support the real economy in a stress period, then a microprudential policymaker might want to increase bank-specific capital buffers for those banks. Similarly, buffers could be adjusted both up and down, depending on economic conditions and stability. The possibility that both system-wide and bank-specific capital buffers might be adjusted in response to stress tests means that some coordination between macro- and microprudential authorities may be required.

  • In the United Kingdom, authorities intend to set capital requirements for the system-wide CCyB and CCoB, as well as for the bank-specific PRA buffer, based in part on stress test results (BoE 2015). The specific sizes of the CCoB and the CCyB are set by the Financial Policy Committee (FPC) and the size of the PRA buffer is set by the PRA, both of which are within the BoE. See Box 3 for a detailed explanation of how buffers are intended to be set.

  • In the United States, one idea is to introduce a bank-specific stress capital buffer (SCB) that can replace the 2.5 percent CCoB of the Basel III framework. The SCB would be set at least as high as the CCoB and would be equivalent to the maximum decline of a bank’s Tier 1 capital ratio under a severe adverse scenario (Tarullo 2016).

  • Bologna and Segura (2016) propose the introduction of an explicit stress test buffer on top of the CCoB and the CCyB in case the losses during the stress test exceed the sum of those buffers. Alternatively, one could see their proposal as forcing banks to hold enough capital to withstand the stress test losses with the CCoB and the CCyB counting as a credit toward that capital requirement. In essence, all three proposals are similar in tying capital requirements to stress test losses and in giving credit for existing buffers.

There are many challenges for calibrating a capital buffer strategy.

Time consistency

  • There can be sizable lags between the publication of information, the start and the finish of a stress test. As a result, it is possible that the stress test is somewhat out of date by the time the results are finalized and capital buffers calibrated. For example, bank balance sheets can change materially during the course of a stress test, as can overall stability and the state of the economy. And the risks captured in the stress test scenario, or the severity of the scenario, may no longer be appropriate some months later. On the other hand, the financial cycle typically moves fairly slowly (Aikman, Haldane, and Nelson 2015), so this may not be a material concern.

  • The asymmetry of CCyB calibration around the (potentially gradual) buildup of risk in a financial upswing versus the (potentially abrupt) crystallization of risks is another challenge.

Optimal level of regulatory discretion versus quantitative calibration

  • Mechanically linking stress tests to the calibration of capital buffer requirements reduces the scope for policymakers to apply their judgment and discretion when setting capital buffers. Stress tests are not the only source of useful information about the appropriate size of capital buffers, and therefore policymakers may also wish to take into account other information when calibrating capital buffers. On the other hand, commitment to a systematic, transparent approach has its benefits—for example, by acting as a form of defined rule.

  • Calibration also requires an assessment of the costs and benefits of changing the CCyB, both for risk and resilience. These costs and benefits are likely to be state contingent and nonlinear, again warranting further research.

Consistency of alternative uses of stress tests

  • There may be a tradeoff between running stress tests that are useful for setting capital requirements versus those that are more useful for shedding light on dark corners. Stress tests that serve the former purpose are likely to require a clear framework and be fairly stable over time. For example, stress tests aiming to calibrate a CCyB should systematically link the severity of the stress scenario to the state of the financial cycle. But there is also value in running stress tests that are more exploratory. Regulators might want to consider adopting a “wide palette,” one that takes into account scenarios that are both novel and orthogonal to historical experience. But, while these sorts of scenarios might be extremely helpful in allowing policymakers to learn about risks, they might not be as suitable for setting capital.

Robustness of methods

  • The measurement of systemic risk is inherently uncertain. Risk assessment is a complex, multivariate problem. The assessment of macrofinancial imbalances requires a notion of financial equilibrium, which is difficult to assess. As methods for stress testing are enriched to incorporate SRA, they may become more prone to data and model uncertainty. As gaps, misclassifications, and omissions in the data are inevitable, it is therefore important to assure the robustness of the techniques employed and to quantify as far as possible the sensitivity of the results to the assumptions that have been made.

B. Informing a Wider Set of Policy Tools

Beyond informing the calibration of capital requirements, MaPSTs could be used to inform a wider set of MaPP tools. These may relate both to risks in the real economy, and to those in the financial system.

  • Identification of firms that could cause the most severe externalities or be most vulnerable to shocks. By identifying those whose actions may most amplify and spread distress (e.g., through their activity in making or trading in markets, or their role in providing funding to non-bank financial institutions, as well as their importance to the real economy), or those firms that might suffer the highest losses as a result of distress of other entities.

  • Lending standards. By developing and exploiting granular, sectoral models of real economy balance sheets, MaPSTs could be used to explore how the distribution of borrower vulnerabilities may evolve under different scenarios—during both booms and busts. This would both add clarity to the risks facing the banking system, and quantify the potential for borrowers to amplify downturns through reduced consumption and investment. Furthermore, it could provide a route to modelling how macroprudential policy tools can impact on the risks of macrofinancial feedbacks, both in terms of how they build up during upswings, and what impact they have in downturns. Tools that target lending standards—such as loan-to-income and loan-to-value limits on mortgage lending, for example—will impact on how the distribution of borrower vulnerabilities evolves during good times; and on the severity of losses and macrofinancial feedbacks during a downturn. For example, Figure 3 shows how the shape of the LTV distribution can lead to a tipping point in the impact of a downturn on mortgage risks. Running a MaPST that incorporates the way that policy can impact on this distribution can provide a forward-looking assessment of how such a policy can dampen the severity of future stress events.

  • MaPP responses targeting systems’ structural features. Quantifying the potential losses arising from systemic risk amplification mechanisms may also motivate other MaPP responses. For example, following the recent financial crisis, large exposure limits, bilateral margining, and central clearing mandates were implemented with the aim of altering structural features of the financial system that amplified losses in the crisis. Using MaPSTs to understand and quantify structural risks in a forward-looking manner could motivate policies that similarly target the structure of the financial system.

  • Improving the design of recovery and resolution frameworks. MaPSTs can provide useful information for designing recovery and resolution frameworks for systemic crisis management. Bankers will often stave off bankruptcy and resolution for too long for a variety of reputational and behavioral reasons. Goodhart and Segoviano (2015) argue that MaPSTs can be useful for defining a ladder of regulatory intervention thresholds in which the socially optimal path for financial institutions is recovery instead of resolution. This gives authorities their best chance for dealing with fragile banks as a going concern (recovery), instead of a potentially contagious failure, which usually requires resolution.

  • Supporting the understanding of the impact of regulatory constraints. Modeling of SRAs in MaPSTs can also provide insights into how existing regulations may influence the actions that institutions might take under severe stress, and how these actions could cause spillovers and amplifications of shocks. MaPSTs can identify whether, for example, risk-based or leverage requirements are (more) likely to bind in a stress scenario. They then can model which actions banks might take to improve their solvency, and, taking into account liquidity constraints, how these actions would impact the wider financial system (see, for example, IMF 2017).38

Figure 3.
Figure 3.

Mortgage Characteristics

Citation: IMF Working Papers 2018, 197; 10.5089/9781484375839.001.A001

Source: Authors’ calculations.Note: Figure 3a shows a hypothetical distribution of loan-to-values on mortgages. The majority of these have LTVs below 80 percent, with a large cohort just below this point. Figure 3b shows the proportion of homes in negative equity for house price shocks of 0 percent to 40 percent. For house price falls of up to 20 percent, relatively few mortgages enter negative equity; beyond 20 percent, there is a rapid increase in the number of mortgages entering negative equity. This nonlinearity would mean that beyond a certain house price shock, borrower distress and credit losses to banks would be expected to increase rapidly for this hypothetical economy. LTV restrictions on new mortgages could be applied to start reshaping the LTV distribution in order to mitigate this risk.

Calibrating the CCyB Rate: The BoE Approach

In the U.K., stress tests are one of many inputs into the setting of the CCyB. This box explains how the Bank of England uses stress tests to calculate an implied CCyB rate, which informs the FPC’s decision over the CCyB. Each year, the BoE subjects the seven largest U.K. banks (covering c.80 percent of lending to the real economy) to the same macroeconomic stress scenario, the annual cyclical scenario (ACS). The severity of the ACS varies according to the FPC’s view of the level of risk in the financial system: as risks build, the severity increases; as risks crystallize or abate, the severity decreases. The BoE extracts an ACS-implied U.K. CCyB rate from the stress test results by treating the seven participants as a single bank, isolating the U.K. cyclical impact at the system-wide CET1 ratio low point. To do this, authorities receive data from participating institutions on the U.K. element of CCyB-relevant impact items (cyclical and relevant to the U.K. banking system as a whole). Examples include firms’ estimates for U.K. income and expenses; U.K. impairment charges; and U.K. credit risk RWA.

For some impact items that are either partly idiosyncratic, or otherwise difficult to allocate between U.K. and non-U.K. impact, the BoE uses its judgment to estimate U.K. impact. Examples include dividend payments; available-for-sale assets; and impact relating to defined benefit pension schemes. Its dataset is sufficiently granular to enable a prudent and consistent U.K. allocation for most of these items. The BoE then applies the equation below to calculate an ACS-implied CCyB rate. If the U.K. cyclical stress is greater than the end-state CCoB rate, the FPC may decide to make up the difference by setting a positive U.K. CCyB rate (or change the rate where already positive).

The ACS is published end-Q1 each year. Firms submit projections and the BoE undertakes its analysis over Q2 and Q3, with the results and decisions disclosed in Q4. Between setting the ACS and taking decisions on the results, significant risks could crystallize or abate. Consequently, the ACS results may be overly severe (where risks abated) or not severe enough (where risks increased). Where this occurs, the FPC and PRA use their judgment to determine the appropriate and coordinated regulatory response to the stress test results. The BoE emphasizes that this calculation helps to inform FPC discussion. The FPC’s ultimate judgment on the appropriate UK CCyB rate to set can take into account a variety of other factors and indicators, including any changes to the risk outlook that could occur after the test was set.

  • The numerator of the fraction represents the total absolute change in capital resources under stress plus the absolute change in capital requirements. This is measured from the starting point to the stress CET1 ratio low point.

  • The impact of the stress (numerator) is converted to an RWA-based measure by dividing by the starting point CCyB-applicable RWAs base as a denominator. Determining the U.K.-relevant portion of RWAs is not straightforward. In practice, a proxy (such as RWAs related to domestic credit exposures) might be used.

  • Where the stress impact exceeds the CCoB, the residual helps to inform the setting of the ACS-implied U.K. CCyB rate. The actual U.K. CCyB rate itself is set in 0.25 percent increments.

VI. Governance for Effective Macroprudential Stress Testing Frameworks

Macroprudential stress testing is an ongoing process and not an event or a one-off, and requires a strong governance framework. A key element of its effectiveness depends on the governance framework. Macroprudential stress testing is not just about models and the mechanics of applying specific tests. It accounts for the wider macrofinancial environment within which the stress tests are applied and used within the decision-making process. Many assumptions are made entailing much uncertainty. Judgments are formed on possible behavioral reactions, systemic interactions, and feedback effects. Based on these, a choice is made of the type of prudential instruments to be deployed, when, on who, and how. Therefore, a strong governance framework for MaPSTs should be fully integrated into the MaPP institutional framework.

Apart from the rigor of the stress tests, ensuring the integrity of MaPSTs becomes a key requirement. Adequate focus needs to be placed on the roles and duties of the various officials responsible for systemic risk assessments. The functions of internal audit, validation, and upkeep of the stress test framework are crucial to the integrity of the process.

A. Governance Framework

In general, any setup that seeks to implement an effective MaPST framework must be guided by six principles:

  • First, maintain stability and consistency across the stress test mandates, process, models and technology, and outcomes.

  • Second, establish a strong end-to-end governance framework with early engagement of senior management.

  • Third, establish risk and controls frameworks to challenge and validate methodologies and assumptions about the financial system’s response to stresses.

  • Fourth, continuously test systemic factors and make preemptive policy adjustments rather than view MST as an annual exam that needs to be passed and then promptly forgotten.

  • Fifth, integrate the flow of data that moves in various key systemic channels into the analytic process, leveraging resources, promoting both top-down and bottom-up flows of information, and focusing on data quality, granularity, and frequency.

  • Sixth, adopt a forward-looking approach that integrates outcomes across different areas of financial stability policymaking, with the goal of broader economic governance.

The appropriate governance framework depends on policymakers’ remits and objectives driven by the structure of the financial system. As these vary across jurisdictions, it is unlikely that there will be one single correct governance framework. Similar to the MaPP institutional framework, an effective governance configuration for MaPSTs is well served by three desirable elements:

  • Providing designated authorities with a clear macroprudential mandate to foster a willingness to assess.

  • Ensuring the designated authorities’ ability to assess by assuring access to relevant information and providing sufficient technical resources.

  • Promoting effective coordination and cooperation in systemic risk assessments while preserving the autonomy of separate policy functions.

Will to assess

A clear MaPST function requires a clear assignment of responsibility for identifying systemic risks and providing input for policy action. Legislation should be clear about who is responsible for MaPP including systemic risk monitoring and should assign specific intermediate objectives. Where a clear assignment is lacking, extempore group actions might lead to underinvestment in systemic risk monitoring. Thus, the perimeter of stress tests may be kept narrow, focusing on individual financial institutions, and failing to consider negative externalities from direct and indirect financial interlinkages.

Such responsibility should be assigned to a national macroprudential authority, an agency, a council, or a committee. In many jurisdictions, the central bank takes on a leading role, given its analytical expertise on macrofinancial risk assessment, practical knowledge about financial markets, and the role as lender of last resort. Some arrangements also involve the relevant authorities with microprudential authorities because of their role in preserving the health of individual financial institutions and the function of financial markets. In some cases, external experts can bring together independent and comprehensive views on systemic risk.

Ability to assess

Assuring timely access to the appropriate data is critical for enabling the authority to properly perform systemic risk assessments. Data gaps can hinder the early detection of systemic risks and increase uncertainty regarding the need for a policy response to identified concerns, while also potentially impeding the choice and enforcement of macroprudential measures. Closing these gaps requires not just new data, but also improvements in the granularity, frequency, and timeliness of existing data.

The designated authority needs to have the power to collect information beyond directly regulated entities. Financial activity can often migrate to unregulated entities in response to regulations in unintended and/or unpredictable ways. In closing any data gaps, consideration must also be given to the cost of data collection imposed on the financial industry. Any governance framework should facilitate the flow of information among agencies and avoid redundant data requests.

Effective coordination and cooperation

Explicit mechanisms are needed to ensure the flow of information and cooperation in risk assessment (IMF 2014). IMF-BIS-FSB (2016) shows that most of the observed MaPP institutional designs belong to one of the three models (Box 4), influencing the flow of data and risk assessments. Full institutional integration (Model 1) facilitates access to the available quantitative as well as qualitative supervisory data, while strong institutional separation across agencies (Model 3) may impede the free flow of data and risk assessments. In the latter case, legal impediments to the sharing of supervisory data for financial stability purposes will need to be resolved through legal obligations (Germany and Turkey) or memoranda of understanding (Australia, Ireland, and Switzerland).

To ensure that the MaPSTs are done across the financial system, all relevant agencies should actively participate in the risk assessment. Reaching a common view on systemic risks based on shared information will reduce incentives for disagreements and uncoordinated policy actions by respective agencies (Osinski and others 2013). Frequent contact, senior-level engagement, open dialogue, and constructive challenge will promote successful coordination across agencies.

Macroprudential stress testing and policy actions should work hand in hand with microprudential oversight. Shared information, joint analysis, and a strong dialogue can reinforce the complementarities between macroprudential and microprudential perspectives. In high stress conditions, tensions can arise, since the macroprudential perspective may call for a relaxation of regulatory requirements (such as capital buffer) that could limit fire sales (as banks do not need to deleverage to maintain a regulatory ratio). At the same time, the traditional microprudential perspective may seek to retain these buffers to protect the interests of clients of individual financial institutions (the U.K. seeks to minimize the possibility of conflicts such that the BoE houses both the microprudential policy via the PRA and macroprudential arm of policy via the FPC). Stress tests provide a framework for governing the interaction between micro- and macroprudential instrument setting and should help facilitate effective policy coordination.

MaPP Institutional Framework Models

Model 1. The main macroprudential mandate is assigned to the central bank, with its Board or Governor making macroprudential decisions. This model is the prevalent choice where the central bank already concentrates the relevant regulatory and supervisory powers. Systemic risk assessment can bring together macro- and microprudential expertise and fully exploit complementarities between top-down and bottom-up risk analyses, for example, in the approach to stress tests.

Model 2. The main macroprudential mandate is assigned to a dedicated committee within the central bank. This setup creates dedicated objectives and decision-making structures for monetary and MaPP, and can help counter the potential risk for multiple mandates affecting decision-making within the central bank “(IMF 2013)”. Unlike Model 1, it can foster an open discussion of systemic risks through participation of separate supervisory agencies and external experts in the committee.

Model 3. The main macroprudential mandate is assigned to an interagency committee outside the central bank, in order to coordinate policy action and facilitate information sharing and discussion of system-wide risk, with the central bank participating on the committee (as in France, Germany, Mexico, and the United States). Identification and mitigation of systemic risk is a multi-agency effort. This model can accommodate a stronger role of the MoF. Participation of the MoF can be useful to create political legitimacy and enable decision-makers to consider policy choices in other fields, for example, when cooperation of the fiscal authority is needed to mitigate systemic risk.

article image
Source: IMF-BIS-FSB (2016). Notes:

Countries with an “*” have an additional council including other supervisors (for example, insurance supervisory authorities and financial market authorities) that play a coordinating role.

The central bank mandate is confined to the CCyB.

“(C)” or “(M)” indicates whether the council is chaired by the central bank or by a government minister (usually the minister of finance), respectively.

B. Accountability and Communication

With responsibility clearly assigned, accountability requirements become the check and balance. A strong accountability mechanism ensures that the authority takes seriously responsibility for stress test results and impacts on policy. The mechanism should be geared toward obliging the authority to allocate sufficient resources for the conduct of risk analysis and stress tests.

Guided discretion should be combined with a proper degree of transparency. Open communication promotes public understanding of the factors affecting systemic risk and the need for a specific tool for promoting financial stability. A range of communication tools is now frequently employed, such as regular testimony to the national parliament, financial stability reports, disclosure of policy statements, and meeting records. In some cases, these tools have been required by law as accountability measures (France, Germany, and the United Kingdom).

There has been a trend toward greater transparency among central banks and regulatory authorities over the past few decades. Transparency and disclosure can be seen as important policy levers. In fact, public communication of stress test results has gained momentum since the onset of the global financial crisis (IMF 2012). Some countries were disseminating the results of stress tests in their financial stability reports even before the crisis, but the aftermath saw an unprecedented degree of disclosure of stress test results in the United States and Europe. This spurred greater public interest in stress tests, which increased pressure for more disclosure.

Publication of stress test results can have benefits. In principle, publication of stress test results should allow external observers to judge the resilience of financial institutions and the financial system to various risks.39

  • Communication can make policy more effective. Disclosure can boost market discipline by providing market participants with information about risks and resilience. Promoting market discipline through an effective communication regime can act as an important complement to direct policy actions undertaken by authorities. The authority can use it as a way to shore up market confidence, promote realistic risk pricing, and (preemptively) raise additional capital from private sources if necessary, thereby reducing the probability of sudden reversals of market sentiment. Even when the results are weak, public communication can have a positive impact if it is accompanied by credible contingency plans for financial institutions that reflect the authorities’ commitment to financial stability.

  • Transparency can boost the credibility of MaPSTs, increasing public confidence in financial stability and the stress testing authority.

  • Communication can improve policymakers’ decisions by providing a credible commitment to explain stress-testing judgments publicly.

  • Publication of stress test results can enhance public accountability.40 Transparency requires regulators to ground their judgments and actions on verifiable information and allow stakeholders and the public at large to hold them to account. It provides legislatures and the public with an effective basis to challenge the authorities’ stress-testing judgments (BOE 2016).

These considerations significantly contributed to the design of the first U.S. macro stress testing exercise carried out in 2009, the Supervisory Capital Assessment Program (SCAP). SCAP had two noteworthy features. First, it was announced, in advance, that any bank failing the stress test would be required to issue new shares in order to bring the bank’s capital ratio up to the regulatory minimum under the stress scenario. Second, the results of the test were announced publicly in considerable detail. The reaction of most commentators and the market was favorable. As pointed out by Bernanke (2013), SCAP was a prime example of the positive effects achieved by providing investors “credible information about prospective losses at banks.” Subsequently, this opinion has been widely accepted as one of the confirmed lessons from the crisis, and later stress testing has been judged to a high degree on the transparency of result reporting that is built into the tests.

However, a number of analysts have expressed doubt that stress tests should be automatically disclosed to the public. Goldstein and Sapra (2013) survey the theoretical literature on mandatory disclosure and identify a number of unintended consequences of disclosure rules that suggest that they need to be structured with care.

  • Disclosure may undermine risk sharing, as has been highlighted by Allen and Gale (2000). For example, exposure to losses revealed by stress testing may undermine the normal provision of liquidity in interbank markets.

  • Disclosure may disincentivize private efforts toward information acquisition and may undermine due diligence.

  • Disclosure of stress test results may entice financial institutions to make portfolio choices to “game” the tests.

  • If contingency plans or credible backstops are not in place, it can undermine market confidence.

  • If stress tests are not severe enough, they can provide a false sense of confidence in the resilience of the financial system.

However, these observations seem to be based on a too hasty reading of the early literature on asymmetric information and bank runs. An announcement of stress test results for banks participating in a macro stress test run by supervisors does not automatically place all participants on an equal level of common knowledge. Information asymmetries may still persist.41 In this context, the question of whether revealing information to the public increases or decreases systemic risk becomes more complex. Therefore, a number of preconditions should be met: stress testing should target all the relevant risks and SRA mechanisms, it should assume several shocks, produce a candid assessment, and be accompanied by a convincing framework for crisis resolution and follow-up action, including government support, if needed.

Hence, the optimal degree of transparency in stress tests is still an open question, and can vary across different dimensions of stress tests. An assessment is required of the trade-offs between the costs and benefits of transparency in scenarios, methodologies and models, results, and policy actions. (Goldstein and Sapra 2013).

  • Scenarios. Most authorities disclose a substantial amount of quantitative information about their chosen stress test scenario. The public can then gauge the severity of the test by, for example, comparing it to previous recessions or financial crises. In this way, the stress test can gain credibility. Further, the stress test scenario allows policymakers to publicly quantify what state of the world they want the banking system to be resilient to, and stakeholders can then hold them to account on that judgment. At the same time, authorities do not typically publish paths for all of the variables that stress test participants require in order to produce stress test projections, possibly because policymakers want stress test participants to develop and improve their own capabilities to model and generate hypothetical adverse scenarios. Providing them with a comprehensive set of scenario variable paths might weaken banks’ own incentives to develop and maintain these capabilities.

  • Methodologies and models. There is typically less transparency over methodologies and models. Disclosure of the details of specific stress testing models may lead to banks simply replicating the regulator’s model rather than developing their own, leading to a “model monoculture” (Bernanke 2013). On the other hand, publishing information about stress testing authorities’ models sends a clear signal to participants about what they regard as best practice. In addition, publishing details about models allows outsiders (for example, in academia) to scrutinize those models, provide constructive feedback that leads to model improvements.

  • Results. Most authorities disclose quantitative stress test results, including for individual institutions. This enhances accountability and market discipline by allowing investors and market participants to form their own assessments of the resilience of banks. But disclosure of stress test results is limited. Not all authorities disclose individual bank results, and those that do tend to disclose only headline metrics (for example, capital ratios or shortfalls). They may do so to avoid triggering an over-reaction in financial markets to the vicissitudes of specific entities.

  • Policy actions. There is wide variation in how authorities use stress tests to inform policy actions; in many cases, this dimension of stress testing is still in its infancy. Nevertheless, authorities have tended to disclose some information about the policy responses to stress tests, such as the hurdle rate, or qualitative information obtained and actions taken. Disclosure in this area has the potential to boost accountability, but disclosing too much may risk revealing commercially sensitive information.

The benefits of transparency can be reinforced by focusing on key features that define the stress tests.

  • Publication of a policy strategy. The framework can encourage the development and announcement of a preferred policy strategy based on assessments of systemic risks and a deployment of specific macroprudential tools. Such a strategy can generate a degree of commitment by spelling out under what conditions these tools would be deployed.

  • Periodic reports of risk assessments and policy actions. It is essential to publish periodic reports on an assessment of risks as well as an ex-post assessment of measures taken. Periodic risk assessments should be comprehensive, include macroprudential stress testing and complementary analyses, and focus on the resilience of the financial system as a whole. Publication of an ex-post assessment of macroprudential measures is useful to create a measure of success to gauge previous actions and which can help build policy credibility. It can also create public support for additional measures when the conclusion is that existing measures have not achieved their objectives. Both risk identification and ex-post assessment may form part of a (semi-annual) Financial Stability Report published by the central bank or a dedicated macroprudential authority.

  • Record of meetings. Publication of a record of meetings should establish transparency on the systemic risk issues discussed and clarity regarding the votes cast by members on policy decisions. Such a record can help the authority establish a narrative that prepares the market and the public for macroprudential action. It can also signal to the market that MaPP will be used unless there is a change in market behavior (expectations channel). When such a threat is credible, it can help change market behavior and reduce the costs associated with the variation of macroprudential tools. To promote accountability, the record should identify the key decisions taken at the meeting. When voting records are published, accountability increases and those opposing actions are more likely to feel a need to justify their decisions, or indecisions.

The way MaPST results and methodologies are communicated is crucial to maintaining a strong mandate. By its very nature, MaPP aims at reducing the probability of crises, which are inherently low-probability events. Therefore, judging the benefits of MaPSTs is a difficult task, and the longer the interim between crises, the stronger will be the voices claiming that the costs of MaPST outweigh the benefits. It is therefore important that the purpose and methods of MaPSTs be communicated clearly. Stressed scenarios need to be credible and the calculated losses need to be accompanied by a convincing narrative about how the losses could arise in practice.

Conclusion

In this report we have studied the development of stress testing as a framework and presented current discussions of its uses for formulating MaPP. In many contexts authorities now regularly undertake stress tests of major banks and other financial institutions in order to assess their vulnerabilities to a variety of risks. A number have taken the lead in organizing the stress testing framework to provide an assessment of risk for the banking sector as a whole. This is a challenging endeavor because the stress testing methodology that underpins it reflects its microprudential origins, in that it provides indicators of the solvency of an individual institution if it is exposed to extreme moves in external risk drivers viewed as exogenous to the system. In order to form a genuine system-wide assessment, the analyst needs to allow for the ways in which risk can be amplified within the system.

Our study has surveyed the current state of play in this programme. First, we reviewed the extensive recent efforts to develop theoretical models that capture risk amplification mechanisms in a financial system, mechanisms that make systemic risk endogenous. Building on the insight that the most important systemic risks are genuinely endogenous in nature, we considered the approaches that incorporate direct means of contagion, for example, through interbank exposures. The early applications of this approach explored the ways in which a loss in one part of a network can spread throughout the system. Treating transmission mechanistically tends to produce a degree of amplification that seems too small to capture the kinds of contagion observed in the crisis of 2008 or in the Asian financial crisis of 1997. Therefore, more recent work in this area has introduced additional elements that may better capture the financial sector’s potential to generate rapid and large amplification of losses. Most of this work develops either or both of two lines of inquiry. One focuses on the modeling of behavior of actors within the system (structural models), while the other explores the structure of information within the system and how that structure may be altered as a result of agents’ behavior as a crisis unfolds (reduced-form approaches).

One specific area where these approaches have been applied is in the understanding of fire sales and how they operate within a system with levered investors. A hit to the value of assets of a levered investor mechanically translates into a large percentage loss of capital. The agent may respond by selling assets to reduce leverage; however, the amount of assets sales may be amplified by precautionary behavior. Investors themselves may seek to restore a capital buffer by holding extra capital beyond the minimum amount needed to support the remaining assets. Or investors’ creditors may reduce the amount of leverage they will accept, for example, by increasing haircuts. In addition to these effects, the fact that many of the assets that investors are forced to sell may be relatively illiquid means that their actions will have a price impact that will be observed and felt by players not directly connected either to the investors exposed to the initial loss or to their creditors. Thus, the loss can spread to other sectors. As information of these losses spreads, an additional amplification effect can emerge in the form of herding behavior. While herding is sometimes treated as a reflection of bounded rationality, recent work has shown that it can emerge naturally by the behavior of rational agents in market structures that create strategic complementarities.

These forces have been explored in a wide variety of modeling frameworks. Compared to theory, the work on empirical implementation of financial models with risk amplification is in its earlier stages. Such empirical work is challenged on a number of fronts. Data is often a problem. The most granular data on exposures in individual institutions is typically observable only by regulators and supervisors, with only limited access given to those implementing stress tests. In addition, supervisory data is often backward-looking accounting data and largely confined to balance sheet information, which may omit off-balance sheet items including derivatives and information of agents’ behavior that can amplify systemic risk. On the modeling front, there is currently no framework that has emerged as the standard workhorse model for the empirical estimation of parameters of amplification mechanisms. Linear models tend to be most tractable, but they must be modified to capture the apparent nonlinearity involved in amplification. How best to do this remains an open question with a number of candidate modeling approaches still in consideration. Furthermore, there may be decreasing returns to building larger more complex models. With enough time, effort, and resources, it may be possible to build a great model to predict the last crisis, but there is a risk that we may have a framework that is too inflexible to allow us to perceive new systemic risks as the financial system evolves, or when we try to translate lessons from one financial system to another.

In the face of these challenges, we have argued that it would be useful for authorities to adopt a structured yet flexible approach to stress testing for macroprudential purposes. We refer to these approaches as “EFs.” An EF can employ a number of separate models as part of the analysis. Some of these can be structural models or bottom up. Some can be reduced-form or top down. They may tap information in alternative data sets; e.g., market data and supervisory information. Some models may be estimated using statistical, econometric or finance techniques. Others may require a calibration approach.

Efforts to develop EF are underway in a number of settings. In our report, we discussed the ongoing efforts of Canada, the EU, Japan, and the United Kingdom, and the IMF. One of the findings from this review is that considerable progress has been made in modeling direct interbank exposures. Most of the work with these effects finds that, when implemented with banks operating at current capital and liquidity standards, the systems seem capable of absorbing and mitigating the effects of even extreme stress scenarios. In contrast, there is currently no settled view on how to incorporate amplification mechanisms that usually arise due to fire sales, herding and information asymmetry. Furthermore, the work on integrating the nonbank financial sector into the analysis is at very early stages. On a more positive front, we discussed how efforts to integrate calibrations of macroprudential tools can be incorporated consistently within the Basel III compliant framework.

Finally, our report examined the governance framework that is desirable for supporting macroprudential stress testing. There needs to be both a willingness to act and an ability to do so. Probably the most important step toward achieving this goal is to give a clear mandate to a recognized institution within the policy making framework. Furthermore, there needs to be accountability for those in charge of formulating macroprudential policy to the broader policymaking authorities. Finally, there needs to be a clear policy on the delicate issue of transparency of stress testing results. There are costs and benefits of communicating stress test results, which involves weighing possible effects on operations of individual institutions versus the system as a whole, perception of risks at different times of the cycle, and consequences for risk sharing within the system.

References

  • Acemoglu, D., A. Ozdaglar, and A. Tahbaz-Salehi, 2015, “Systemic Risk and Stability in Financial Networks.” American Economic Review.

    • Search Google Scholar
    • Export Citation
  • Acharya, V., and L. Pedersen, 2005, “Asset pricing with liquidity risk.” Journal of Financial Economics.

  • Admati, A., and M. Hellwig, 2014, The bankers’ new clothes: What’s wrong with banking and what to do about it. s.l.: Princeton University Press.

    • Search Google Scholar
    • Export Citation
  • Adrian, T., and H.S. Shin, 2008, “Liquidity and financial contagion.” Banque de France Financial Stability Review: Special Issue on Liquidity 11, pp. 17.

    • Search Google Scholar
    • Export Citation
  • Adrian, T., and H.S. Shin, 2010, “Liquidity and Leverage.” Journal of Financial Intermediation, 19(3), pp. 418437.

  • Adrian, T., and H.S. Shin, 2014, “Procyclical leverage and value-at-risk. Review of Financial Studies, 27(2), pp. 373403.

  • Adrian, T., and H.S. Shin, B. Begalle, B., A. Copeland, and A. Martin, 2013, “Repo and Securities Lending,” FRBNY Staff Report.

  • Aikman, D., Haldane, A. G., & Nelson, B. D., 2015, “Curbing the credit cycle.” The Economic Journal, 125 (585), pp.10721109.

  • Alessandri, P., and others, 2009, “Towards a Framework for Quantifying Systemic Stability.” International Journal of Central Banking.

    • Search Google Scholar
    • Export Citation
  • Alla, Z., R. Espinoza, Q.H. Li and M. Segoviano, 2017, “Macroprudential Stress Tests: A Reduced-Form Approach to Quantifying Systemic Risk Losses,” forthcoming IMF Working Paper, Washington DC: International Monetary Fund

    • Search Google Scholar
    • Export Citation
  • Allen, F., and D. Gale, 2000, “Financial Contagion,” Journal of Political Economy, Volume 108, pp. 133.

  • Alvarez, F., and G. Barlevy, 2014, “Mandatory Disclosure and Financial Contagion,” National Bureau of Economic Research Working Paper No. 21328.

    • Search Google Scholar
    • Export Citation
  • Anand, K., G. Bédard-Pagé, and V. Traclet, 2014, “Stress testing the Canadian banking system: A system-wide approach,” Financial System Review, Volume 61.

    • Search Google Scholar
    • Export Citation
  • Anderson, R. W., 2016, “Stress Testing and Macroprudential Regulation: A Transatlantic Assessment.” In Stress Testing and Macroprudential Regulation: A Transatlantic Assessment. R.W. Anderson, ed. London: CEPR Press, pp. 129.

    • Search Google Scholar
    • Export Citation
  • Avery, C., and P. Zemsky, 1998, “Multidimensional Uncertainty and Herd Behavior in Financial Markets.” American Economic Review, Volume 88, pp. 724748.

    • Search Google Scholar
    • Export Citation
  • Bank of England, 2015, The Bank of England’s approach to stress testing the UK banking system, s.l.:s.n.

  • Bank of England, 2016, The Financial Policy Committee’s approach to setting the countercyclical capital buffer, s.l.:s.n.

  • Bank for International Settlements, 1994, 64th Annual Report, Basel, Switzerland: s.n.

  • Bank of Korea, 2012, Financial Stability Report, s.l.: s.n.

  • Baranova, Y., J. Coen, P. Lowe, J. Noss, and L. Silvestri, 2017, “Simulating stress across the financial system: the resilience of corporate bond markets and the role of investment funds”, Bank of England, Financial Stability Paper 42.

    • Search Google Scholar
    • Export Citation
  • Bardoscia, M., Barucca, P., Brinley Codd, A., and Hill, J., 2017, “The decline of solvency contagion risk”, Bank of England Working Paper 662.

    • Search Google Scholar
    • Export Citation
  • Bartholemew, P., and G. Whalen, 1995, “Fundamentals of Systemic Risk.” In Research in Financial Services: Banking, Financial Markets, and Systemic Risk. G.G. Kaufman, ed. Greenwich, CT: JAI, pp. 317.

    • Search Google Scholar
    • Export Citation
  • Basel Committee on Banking Supervision, 2010, Guidance for national authorities operating the countercyclical capital buffer, s.l.: Bank for International Settlements.

    • Search Google Scholar
    • Export Citation
  • Basel Committee on Banking Supervision, 2011, Basel III: A global regulatory framework for more resilient banks and banking systems, s.l.: Bank for International Settlements.

    • Search Google Scholar
    • Export Citation
  • Basel Committee on Banking Supervision, 2012, A framework for dealing with domestic systemically important banks, s.l.: Bank for International Settlements.

    • Search Google Scholar
    • Export Citation
  • Basel Committee on Banking Supervision, 2013, Global systemically important banks: updated assessment methodology and the higher loss absorbency requirement, s.l.: Bank for International Settlements.

    • Search Google Scholar
    • Export Citation
  • Basel Committee on Banking Supervision, 2015. “Making Supervisory Stress Tests More Macroprudential: Considering Liquidity and Solvency Interactions and Systemic Risks.” BCBS Working Papers, Volume 29. November.

    • Search Google Scholar
    • Export Citation
  • Battiston, S., and others, mimeo. “Network based model for systemic risk.”

  • Bazinas, V., and M. Segoviano, 2017, “Assessing Time-varying Macrofinancial Linkages.” IMF Working Paper, forthcoming.

  • Bebchuk, L., and I. Goldstein, 2011, “Self-Fulfilling Credit Market Freezes.” Review of Financial Studies, Volume 22(11), pp. 35193555.