Approach

In FSAPs, surveillance stress testing of banks’ solvency risk usually consists of a “top-down” (TD) approach, which is sometimes combined with a “bottom-up” (BU) approach. These are carried out in the following manner:

• TD tests are conducted by IMF staff or by the authorities or by both, typically in close collaboration with one another. In these exercises, tests are either conducted using the data of individual banks and then aggregated, or on an aggregated group of banks to analyze the impact of pre-defined shocks on the system as a whole. A common macrofinancial environment is assumed and a standardized set of behavioral assumptions (see below) is applied across the board. TD stress tests may be used as a standalone analysis or to complement the BU exercise, if one is conducted.

• The BU approach is used by FSAP teams where authorities are supportive of having individual institutions conduct their own stress tests and banks have sufficient expertise to do so. In the BU approach, individual institutions run the stress tests using their own internal risk models and data. As with the TD approach, common macroeconomic shocks and selected standardized assumptions are prescribed by IMF staff to isolate the impact of shocks on banks’ financial soundness in order to identify specific vulnerabilities.

IMF staff advocates conducting both BU and TD stress tests, as much as possible, to enrich the surveillance analysis in FSAPs. Each approach has its strengths and weaknesses and is considered complementary for cross-validation purposes, rather than as substitutes for one another. The process of reconciling the BU and TD results is usually an important learning process in itself, with any divergence in the results from the two approaches usually traced to differences in either the model design, the scope of the stress testing exercise (including the type of underlying data used), behavioral assumptions and/or modeling of sensitivities. For instance, bank-specific assumptions and the application of internal models based on more granular data can lead to differences in the projection of profits and losses—and consequently the impact on the capital ratios—for individual banks under the various scenarios.

The decision as to whether BU stress tests are conducted to complement TD tests or if TD stress tests are performed by country authorities or by IMF staff, or jointly, is mostly made on an ad-hoc, country-by-country basis. It is usually based on data and resource availability, and on the receptiveness and degree of involvement by authorities. Around half of the FSAPs to the major countries since 2010 have run both BU and TD tests (e.g., Australia, China, France, India, Indonesia, Japan, Mexico, Russia, Turkey and the United Kingdom). TD tests are either conducted by the IMF team only (e.g., Indonesia, Netherlands, Turkey, Saudi Arabia, Sweden, India and Australia) or by the authorities only (e.g., Luxembourg, Russia and Japan), or in some cases, separately by both, using different methods (e.g., China, France, Mexico and the United Kingdom).

The solvency stress testing of the banking sector in the 2011 United Kingdom FSAP Update epitomizes the necessary collaboration among country authorities, the IMF and individual financial institutions (IMF, 2011a). In this instance, both BU and TD solvency stress tests are conducted (together with TD liquidity risk stress tests). BU stress tests are run by the seven major U.K. banks, in close coordination with the FSAP team and the Financial Services Authority (Figure 2). At the same time, TD tests are separately performed by the Bank of England (BoE) using its Risk Assessment Model for Systemic Institutions (RAMSI) and by the FSAP team using the Systemic Contingent Claims Analysis (SCCA) model, applying macroeconomic forecasts and projections from the IMF and FSA, respectively, and satellite model outputs from the BoE.

View Full Size

Example of IMF Stress Testing Exercise: U.K. FSAP Update

Citation: IMF Working Papers 2013, 068; 10.5089/9781616355074.001.A001

Source: IMF (2011a).

• Download Figure
• Download figure as PowerPoint slide

Coverage

The coverage of a stress test is crucial for the usefulness and thus credibility of the exercise. Ideally, surveillance stress testing of the banking sector for macroprudential purposes should include all institutions, if data availability and resources permit. Realistically, all systemically important institutions, as well as second-tier banks which are potentially systemic depending on circumstances, should be covered. Smaller institutions which may be considered at risk could also be included.

FSAPs typically focus on stress testing the major commercial banks in their respective jurisdictions. The market share coverage of the banks included in the various major country stress testing exercises has been 60 percent or more of the total assets of the sector and up to 100 percent in six of the 18 major countries in which FSAPs have been conducted since 2010 (e.g., Brazil, India, Indonesia, Japan, Luxembourg and Russia) usually determined in collaboration with the authorities. Where resource constraints dictate that only a small sample of banks can be considered, especially in the case of BU stress tests, the usual practice is to focus on the obviously systemic institutions.

The identification of systemically important domestic banks is still not clear-cut. While some banks are of obvious systemic importance in their own respective countries and their selection for stress tests is indisputable, the difficulty has been in identifying those that are systemic at the margins, e.g., some of the smaller institutions which may have the potential to become systemic depending on the environment at a particular point in time (IMF/BIS/FSB, 2009). Thus, the definition of what constitutes a systemic bank remains largely ad hoc in IMF-related stress testing exercises, and a more structured approach is desirable. The BCBS methodology for identifying global systemically important banks (G-SIBs) has facilitated this process (BCBS, 2011; FSB, 2011), while the guidelines on the implementation of supervisory measures for domestic systemically important banks (D-SIBs) and the policy recommendations by the Financial Stability Board (FSB, 2012) for their identification represents another positive step in this direction (BCBS, 2012b).

Data

The availability and sufficiency of timely and reliable data underpin the robustness and credibility of the stress test results. The type, quantity and quality of data play a crucial role in determining the stress tests that can be conducted, the risks that are possible to cover and the models that may be applied in the tests (Howard, 2009). As much as possible, FSAP stress tests utilize the latest audited and corresponding supervisory data alongside the latest macroeconomic projections, all of which contribute to the determination of the appropriate cut-off date. In situations where the authorities are less forthcoming, IMF staff relies on publicly available data on individual banks, which may be less granular. Supervisory data have been provided in almost all major country FSAPs to date. In 17 out of 18 cases, supervisory authorities have made available to IMF teams the relevant data from regulatory returns, which are usually supplemented by publicly available information; only in one instance was staff wholly dependent on public information for the stress testing exercise.

One area in which there has been little standardization across FSAPs is the nature of consolidation of bank financial data applied to the stress tests. While about half of the FSAPs to date have used consolidated banking group data for the stress tests (e.g., Australia, China, Brazil, France, Japan, Netherlands, Sweden, the United Kingdom and the United States), most of the others have utilized unconsolidated local entity data (e.g., Germany, India, Luxembourg, Mexico, Russia, Spain and Turkey).

The main focus of bilateral FSAPs is typically on the domestic banking system, which suggests that data of banks’ local businesses should be utilized on a local-consolidated basis. Such data would avoid double counting local business operations. The use of consolidated level data would not allow consideration of issues such as ring-fencing of subsidiary profits, capital and liquidity by host countries, which may be important for large international groups (Cerutti and Schmieder, 2012). That said, the decision as to which type of data to use may sometimes be moot as it could be constrained by the type of data that are collected for supervisory purposes.

The use of forward-looking market data and other variables reflecting point-in-time risks to complement accounting information is growing, especially for data-rich advanced economies. Market data have been found to add value to the analysis insofar as it provides corroborating evidence of market perceptions of what existing book values represent. They can also be used as a benchmark for internal ratings based (IRB) parameters—i.e., those derived from banks’ own credit risk models to quantify required capital—and for other risks, namely, market risk and operational risk.

An important caution with regard to stress testing in general lies in the use and interpretation of the data. Expert judgment is a crucial supplement to the quantitative approach at all times. It should also be emphasized that FSAPs do not conduct audits of banks’ accounts and therefore cannot corroborate the quality of the reported data used in stress tests. In instances where staff may be concerned about the effects of issues such as loan misclassifications and/or lender forbearance on the accuracy of the data, caveats should be explicitly noted (e.g., Spain and the United Kingdom).

Risk horizon

For surveillance purposes, the choice of a risk horizon is important in terms of designing an exercise that would yield valuable information for policy discussions. Covering a longer risk horizon for macro-scenario solvency stress tests offers several benefits, namely: (i) major macrofinancial distress events typically have a lasting impact spanning several years, especially in the case of credit risk; and (ii) regulatory reforms are likely to be protracted and take several years to implement (e.g., the implementation of Basel III). While the degree of uncertainty also increases as the risk horizon lengthens, surveillance stress testing is not a forecasting exercise; rather, the exercise should adequately capture any medium-term effects of shocks. In contrast, sensitivity tests are usually applied to assess instantaneous shocks.

It is important to balance consistency of the risk horizon across countries with the usefulness of the findings for individual country circumstances. As in other aspects of stress testing, expert judgment is crucial—while major country FSAPs typically apply a five-year risk horizon in their macro-scenario design, exceptions may be made in cases where staff is of the view that the application of a longer sample period may be unconstructive. As an example, the FSAP stress test for Spain applies a 2-year risk horizon to accommodate the rapidly changing financial landscape as a result of ongoing restructuring efforts (IMF, 2012b). In the majority of emerging market economies, whose banking systems are less mature (e.g., Indonesia, China, Turkey, Mexico), risk horizons of between 1–3 years have been used.

Stress scenarios

Stress tests are based on scenario shocks and/or sensitivity analysis. In scenario tests, a baseline scenario is first established and post-shock assessments are made relative to the baseline scenario. In FSAPs, the IMF’s World Economic Outlook (WEO) projections are typically used as the baseline for stress tests. Stress scenarios are then defined based on either historical simulation; hypothetical scenarios that have not yet happened but are particularly relevant given specific vulnerabilities in banks’ portfolios; or ad-hoc expert judgment. The stress scenarios are then applied consistently across banks within the same system.

One of two approaches is to construct the appropriate stress scenarios for FSAP solvency tests, depending on the availability of data and the modeling capabilities. Scenarios may reflect a hypothetical state of risk parameters under stress affecting solvency conditions (the “direct approach”), which is often used in the case of ad-hoc scenarios or historical simulation, or be based on adverse macroeconomic scenarios, which need to be translated into financial stress parameters (the “indirect approach”). The latter approach consists of:

• An estimation of economic and financial variables conditional on the macroeconomic scenario. Common methods for predicting economic and financial variables conditional upon certain macroeconomic conditions include: (i) structural econometric models; (ii) vector autoregressive (VAR) methods; and (iii) pure statistical approaches (Foglia, 2008). As a general rule, these macrofinancial linkages would need to be clearly documented and back-tested.

• The translation of these economic and financial variables into financial risk parameters via various types of “satellite” (or auxiliary) models. This step links different macrofinancial shocks, reflected in macroeconomic variables, to the main determinants of bank solvency, i.e., pre-impairment profit, impairments and risk-weighted assets (RWA), since macroeconomic models do not usually include financial balance sheet variables (and credit aggregates in particular). Common explanatory variables include:

  • (i) macroeconomic variables, such as economic growth, unemployment, short- and long-term interest rates, inflation, and exchange rates;

  • (ii) sectoral (asset price) indicators, such as residential and commercial real estate prices and equity market conditions (Figure 3); as well as

  • (iii) micro-level data, such as bank-specific credit growth (e.g., deleveraging under severe stress conditions), which could also be modeled as a macroeconomic variable, operational/financial leverage and funding gaps.

View Full Size

View Full Size

Example of Macro Scenarios for Stress Testing: U.K. FSAP Update

Citation: IMF Working Papers 2013, 068; 10.5089/9781616355074.001.A001

Source: IMF (2011a).Note: BoE fan charts are based on BoE, rather than WEO projections.

• Download Figure
• Download Figure
• Download figure as PowerPoint slide

Recent FSAPs have attempted to introduce similarly severe macro-scenario shocks in the respective solvency stress tests. The aim has been to facilitate the identification of other factors that drive differences across institutions and to facilitate comparisons across peer countries. Growth shocks are defined in terms of standard deviations from long-term historical averages, usually one (mild adverse) and/or two (severe adverse) standard deviations, over varying periods as deemed appropriate. For example, the four standard deviation shock imposed on the Australian banking system is estimated over a 50-year period, whereas the two standard deviation shock applied to several EU countries is calculated over a 30-year period. In about half the exercises, a prolonged slow growth scenario is also included as a separate stress (e.g., Australia, Brazil, China, Germany, Japan, Turkey, Sweden, the United Kingdom and the United States).

Flexibility in the scenario design remains key, even though the application of these shock magnitudes have become more or less a general rule of thumb for recent FSAPs. The prevailing macroeconomic environment and main risks to financial stability should continue to drive the decision as to what constitutes the most appropriate and credible tail shock scenario(s) for a particular financial system. For example, the issue of overheating was a key risk for Turkey at the time of its FSAP and was therefore incorporated into the design of the stress scenario. For Spain, the one standard deviation shock applied takes into account a revised baseline that took into account the rapidly deteriorating economic outlook and a fiscal adjustment.

Nonetheless, there remains significant room for improvement in this area of the IMF’s stress testing work, notably, from a spillover perspective. A current weakness is the inability of IMF staff to extend the shock scenarios for the home country in question to consistently and comprehensively quantify the impact that such scenarios may have on the macro environments of other countries where the international banks in question are active. In such cases, IMF staff sometimes has to rely on the banks themselves to estimate the corresponding scenarios in their footprint countries in BU exercises, potentially giving rise to inconsistencies in projections, and thus the resulting impact on banks’ financial performance and position possibly for the same countries.

FSAP stress scenarios emphasize the importance of tail risks. The tests are aimed at identifying the vulnerabilities of a country’s financial system and the ability of its supervisory and crisis management frameworks to deal with the realization of extreme but plausible risks. In the U.K. FSAP, for instance, capital losses are estimated for a 0.1 percent probability event (IMF, 2011a and 2011b)—the U.K. Financial Services Authority (FSA) had ascribed a two percent probability to a two standard deviation shock to growth materializing (FSA, 2011) and the IMF’s model subsequently calculates capital losses at the 95^th percentile of this scenario (i.e., falling into the 5 percent tail of the scenario distribution), i.e., 0.05 × 0.02 = 0.001. That said, it is sometimes difficult to convince national authorities of the importance of running extreme tail scenarios that would show the demise of their financial institutions or system. A useful way forward may be to also run reverse stress tests, i.e., stress tests that aim to determine scenarios that would cause a bank to become insolvent.

Separately, sensitivity tests provide useful information on the immediate impact of individual shocks. These are usually applied as the only type of stress tests for financial systems with little or poor quality data, or to complement the scenario analyses conducted on more complex financial systems. Several risk factors could also be combined to determine the impact of concurrent multiple shocks to a system. Sensitivity analysis has been conducted in the majority of major country FSAPs stress testing exercises on various market risk factors.

Risk factors

The selection of main risk drivers to incorporate and the choice(s) and manner in which they are integrated (or not) have significant bearing on the stress test results. The focus on risks in FSAP solvency stress tests, and thus the manner of tests conducted, has evolved and expanded over time and indeed sharpened following the global financial crisis. FSAPs attempt to cover all key risks borne by a financial institution and the system as a whole. Prior to the global financial crisis, these tests have focused largely on credit and market risks (e.g., interest rates, exchange rates, equity, credit spreads and commodity prices). While these risks remain the mainstay of FSAP solvency stress tests, lessons learned since the onset of the crisis have also motivated the inclusion of additional types of exposures.

Risks which had previously been in the periphery have taken center stage in the design of FSAP stress tests in the throes of the current crisis. They include:

• Exposures to sovereign and other previously low-default assets. Prior to the global financial crisis, exposures to sovereign debt did not figure prominently in stress tests, if at all. They were considered “risk free” and were typically assigned the lowest (often zero) risk weightings for regulatory capital requirements under the Basel framework. However, recent FSAPs have acknowledged rising sovereign risks by estimating the potential asset price losses for such exposures. The future yield-to-maturity (and the corresponding haircut) of a bond of a given country can be determined based on the impact of changes to the individual sovereign credit risk on its bond price (Jobst and others, forthcoming). The same issue applies to other previously low-default portfolios, such as holdings of bank debt. Shocks to bank holdings of sovereign assets have been incorporated into the recent FSAP stress tests in S-25 EU countries, as well as Japan; and some have also applied the same treatment to portfolios of bank debt.

• Banking and trading books. For securities, stress tests had previously considered shocks to trading books only, largely because longer horizons were not covered. However, some institutions moved their securities to banking books during the crisis, supported, in some cases, by regulatory forbearance, underscoring the need for stress tests to cover securities in all accounts. FSAP stress tests now attempt to estimate valuation losses in both the available-for-sale (AfS) portfolio—which are not included in net income but put through a reserve under shareholders’ equity, unlike those associated with the trading book—and the hold-to-maturity (HtM) portfolio in the banking book (modeled via provisions). However, not all country authorities are receptive to a comprehensive application of shocks to banks’ securities holdings. In the FSAP stress tests for France, Japan, the Netherlands, Sweden and the United Kingdom, valuation haircuts are applied to both portfolios (excluding the “AAA”-rated sovereigns in the HtM portfolio), but only to the AfS portfolio in the case of Russia and Spain.

• Funding costs. The experience from the global financial crisis has emphasized the importance of incorporating the impact of rising funding costs on bank solvency (as part of the simulation of income under stress more generally). Funding costs change disproportionately to changes in solvency conditions, rising sharply as a bank’s capital position worsens (especially for banks with sizeable portions of wholesale funding). Stress test calculations link net funding costs (simulating the impact on both assets and liabilities) to income. The explicit incorporation of funding costs into FSAP solvency stress tests is a nascent practice (e.g., France, Germany, Sweden and the United Kingdom) and is not yet widely implemented even for the major countries.

• Off-balance sheet items. The crisis saw the realization of contingent liabilities arising from explicit and implicit guarantees of investment vehicles that contributed to the sudden realization of large losses. Thus, incorporating off-balance sheet positions that could give rise to such contingent liabilities (such as guarantees, commitments, and derivatives) is important to adequately capture the impact of extreme stress on all relevant exposures. That said, such data are not as readily available especially from public and sometimes even supervisory sources.

• Cross-border exposures. Prior to the crisis, credit risk tests focused largely on banks’ exposures to domestic corporate and households, paying scant attention to their overseas exposures (through branches and subsidiaries). Since then, FSAPs have incorporated spillover risks in the form of network (e.g., Australia, France, Japan and Spain) and ring-fencing (e.g., Spain) analyses as separate modules in the TD approach. In some cases, international banks are required to take into account shocks to the countries in which they are active, in the BU assessments (e.g., the United Kingdom).

The modeling of impairment parameters has also taken on significant import during the crisis. Estimates of credit losses, usually simulated via probabilities of default (PDs) and losses given default (LGDs) and the resulting potential losses under stress should account for differences in banks’ respective business models and/or specific risks. The decision as to whether through-the-cycle (TTC) or point-in-time (PIT) PDs (and LGDs) should be applied at various points in the economic cycle could have significant impact on the stress test results. In order to form a view on current risks, it is desirable to use PIT risk parameters especially during stressed periods. Another key challenge in FSAPs is to ensure the availability of these parameters for the universe (or at least the majority) of banks tested, and if necessary, to proxy by other methods, such as from loan loss provisions (Schmieder and others, 2011).

Factors that management control

Standardized prescriptions which control for strategic decisions and behavioral adjustments are particularly important for surveillance stress tests. Specifically, common assumptions on factors that management control ensure that findings on the capital adequacy of banks under adverse macroeconomic conditions can be analyzed in a consistent and comparable manner. In FSAPs, common assumptions are especially pertinent for BU stress tests which use own internal models, albeit sacrificing flexibility and some degree of realism. Assumptions adopted in FSAPs are also typically (and appropriately) on the conservative side. The main behavioral variables include:

• Balance sheet growth. This assumption determines the trend growth in core items on the assets and liabilities sides of banks’ balance sheets. FSAP stress tests typically assume constant (i.e., growing with nominal GDP or some pre-defined rule) or static balance sheets (possibly in combination with a constant credit portfolio). Indeed, major country FSAPs to date have been split almost evenly on the adoption of either assumption.

• Credit growth. In FSAPs, credit growth assumptions are usually based on models (e.g., Brazil, Spain and Sweden) or on descriptive empirical evidence (e.g., Turkey) and expert judgment. More broadly, it is assumed that banks under stress are likely to reduce lending in line with a slowdown or reversal in balance sheet growth, usually consistent with changes in nominal GDP.

• Dividend payout. The assessment of potential capital shortfalls takes into account assumptions regarding dividend payouts. In most of the major country FSAPs, the dividend payout is assumed to be zero under stress. For the others, assumptions include payouts based on Basel III capital conservation standards (e.g., Sweden) or on historical ratios (e.g., Brazil, France and Japan); the general rule is that dividends are assumed to be paid only by banks that satisfy all three measures of capital adequacy, as relevant (i.e., total capital, Tier 1 and core/common equity Tier 1) after making adequate provisions for asset impairments and transfers of profits to statutory reserves, which banks must keep on hand to meet their obligations to depositors.

• Strategic changes and asset disposal. FSAP stress tests typically do not consider changes to business operations that require managerial involvement, such as plans to increase operational efficiencies. Moreover, non-realized and/or strategic disposals (e.g., loan books in run-off or sales of non-core businesses) or acquisitions (except when there are legally binding commitments under competition rules, e.g., as agreed with European Commission in the case of EU countries) are generally eschewed. Firms are also assumed to replace maturing exposures unless there is a sound basis for assuming that this will not happen (e.g., deleveraging plans for banks in IMF program countries).

There is not necessarily a specific “best practice” associated with each assumption on the factors that bank management controls. However, conservatism should be an important consideration. FSAPs have sought to ensure some uniformity in their application where possible, and to match their specific relevance to the country in question. Detailed guidance on these assumptions is usually provided for FSAP stress tests as relevant (Appendix II).

Macroeconomic and satellite modelling

System-wide stress tests that are informed by adverse macroeconomic conditions affecting the profitability and solvency of banks necessitate the use of satellite models, which help project the impact of key sources of risk. Specifically, satellite models are used to determine credit losses and various components of profit, including funding costs, under various scenarios. Under each stress scenario, macro and financial sector variables are projected as input into the solvency stress tests (Figure 4). These would have a bearing on the net interest income, non-interest income, trading income, credit growth and credit losses of banks. Satellite models can be run at the economy level, sectoral level and also at the level of individual banks or of one of their specific portfolios.

View Full Size

General Representation of Satellite Modeling in Bank Solvency Stress Testing

Citation: IMF Working Papers 2013, 068; 10.5089/9781616355074.001.A001

Source: Authors.

• Download Figure
• Download figure as PowerPoint slide

The construction of satellite models typically comprises three key steps. They are:

(i) the choice of the estimation method;

(ii) the selection of the dependent variable and a set of potential explanatory variables that form the initial model specification; and

(iii) the iterative process of fitting the model (and completing robustness checks).

Various types of modeling may be used. These include time series analysis, regression models (e.g., OLS regression, logistic regression, and panel data analysis) and structural models (Foglia, 2008; Drehmann, 2009). Most major country FSAP stress tests have typically relied on the authorities’ satellite models, on the basis that these models would have undergone repeated calibrations and robustness checks over time. The FSAP team sometimes cross-validates with IMF staff’s own satellite models in parallel TD tests (see Figures 5 and 6 for application in U.K. FSAP Update).

View Full Size

Example of Satellite Model Estimations for Bank Solvency Stress Testing: U.K. FSAP Update

Citation: IMF Working Papers 2013, 068; 10.5089/9781616355074.001.A001

Source: IMF (2011a).

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Example of Application of Satellite Model Outputs to Top-down Bank Solvency Stress Test Models: U.K. FSAP Update

Citation: IMF Working Papers 2013, 068; 10.5089/9781616355074.001.A001

Source: IMF (2011a).

• Download Figure
• Download figure as PowerPoint slide

Stress test models

There is no one specific stress test model that is perfectly suited for a particular financial system. What is important is that the model is able to adequately capture the complexity, uniqueness and idiosyncrasies of that system, subject to data availability. In this context, an important challenge in FSAPs has been to ensure that appropriate stress test model(s) are applied on each occasion. FSAP stress tests for simple financial systems with a predominantly domestic financial sector are normally less resource intensive and require less-sophisticated models. In contrast, stress tests of more complex systems have applied correspondingly more advanced stress testing methods to capture the gamut of risks.

The stress testing methods that are applied to estimate capital adequacy under projected financial stress are based on either a deterministic or stochastic framework. Deterministic approaches are predicated on prudential information in balance sheet based stress test specifications, while stochastic frameworks incorporate uncertainty around these accounting identities using historical volatility and/or market information, usually in the context of portfolio-based models. Both approaches allow for running scenario or sensitivity analysis.

It is important to be aware of the differences between different stress testing methods and their implications for the results. As a general rule, the more sophisticated the model, the higher the chances of estimation uncertainty, an issue which needs to be taken into account when drawing policy conclusions from stress tests. At the same time, simpler methods might be inadequate for highly interconnected and complex banking sectors with large credit and market risk exposures. When different approaches (TD, BU) and models are used in an FSAP, the results are cross-validated and the differences reconciled; discussions on the assumptions and caveats attached to the different models are also included in the write-up.

There is still significant room for improvement in stress test modeling. For instance, existing FSAP stress tests do not adequately capture feedback effects beyond the initial impact of macroeconomic shocks on the banking sector, despite some recent work in this area (Vitek and Bayoumi, 2011). An important reason is that the interaction between adverse macroeconomic scenarios, such as changes in credit aggregates, and firm-level financial soundness complicates the specification of feedback effects (BIS, 2009). The literature and the actual use of stress test models that incorporate feedback effects from the financial sector to the general economy remains very limited to date (Alfaro and Drehmann, 2009).

A suite of stress testing models is currently used by IMF staff for surveillance stress testing. They can be categorized into two broad strands, supplemented by a third, and are discussed below. These approaches are not mutually exclusive in that there are overlaps in the types of data that are utilized (Table 4 and Figure 7). IMF staff is presently cataloguing models developed within the institution to improve transparency in the models used in FSAPs and other areas of IMF work (Čihák and Ong, forthcoming).

Table 4.

Scorecard on Data and IMF Stress Test Models

Source: Čihák and Ong (forthcoming). Note: For descriptions of models, see: Espinosa-Vega and Solé (2011) for the network approach; Chan-Lau and others (2012) for the extreme value theory approach; Jobst and Gray (2013) and Gray and Jobst (2011) for systemic contingent claims analysis; Segoviano and Padilla (2006) for the distress dependence framework.

Table 4.

Scorecard on Data and IMF Stress Test Models

		Data Quality
		Basic Data	Detailed Data
Data Type	Balance sheet	✓	✓	✓	✓		✓	✓
	Macroeconomic	✓	✓				✓	✓	✓
	Supervisory	✓	✓	✓				✓
	Market				✓	✓	✓		✓
	Inter-bank			✓
Stress Test Elements	Methodology	Accounting-based	Accounting-based incorporating macro-financial models	Accounting-based	Market price-based	Market price-based	Accounting-and market price-based incorporating macro-financial models	Accounting-based incorporating macro-financial models	Market price-based incorporating macro-financial models
	Model	Balance sheet approach	Balance sheet approach	Network approach	Extreme-value theory approach	Co-risk	Systemic contingent claims analysis	Satellite models	Distress dependence framework
	Shock	Sensitivity analysis	Macro scenarios	Sensitivity analysis	Sensitivity analysis	Sensitivity analysis	Macro scenarios	Macro scenarios	Macro scenarios

Source: Čihák and Ong (forthcoming). Note: For descriptions of models, see: Espinosa-Vega and Solé (2011) for the network approach; Chan-Lau and others (2012) for the extreme value theory approach; Jobst and Gray (2013) and Gray and Jobst (2011) for systemic contingent claims analysis; Segoviano and Padilla (2006) for the distress dependence framework.

View Table

Table 4.

Scorecard on Data and IMF Stress Test Models

		Data Quality
		Basic Data	Detailed Data
Data Type	Balance sheet	✓	✓	✓	✓		✓	✓
	Macroeconomic	✓	✓				✓	✓	✓
	Supervisory	✓	✓	✓				✓
	Market				✓	✓	✓		✓
	Inter-bank			✓
Stress Test Elements	Methodology	Accounting-based	Accounting-based incorporating macro-financial models	Accounting-based	Market price-based	Market price-based	Accounting-and market price-based incorporating macro-financial models	Accounting-based incorporating macro-financial models	Market price-based incorporating macro-financial models
	Model	Balance sheet approach	Balance sheet approach	Network approach	Extreme-value theory approach	Co-risk	Systemic contingent claims analysis	Satellite models	Distress dependence framework
	Shock	Sensitivity analysis	Macro scenarios	Sensitivity analysis	Sensitivity analysis	Sensitivity analysis	Macro scenarios	Macro scenarios	Macro scenarios

Source: Čihák and Ong (forthcoming). Note: For descriptions of models, see: Espinosa-Vega and Solé (2011) for the network approach; Chan-Lau and others (2012) for the extreme value theory approach; Jobst and Gray (2013) and Gray and Jobst (2011) for systemic contingent claims analysis; Segoviano and Padilla (2006) for the distress dependence framework.

View Full Size

Stress Test Models Developed by IMF Staff

Citation: IMF Working Papers 2013, 068; 10.5089/9781616355074.001.A001

Source: Čihák and Ong (forthcoming).

• Download Figure
• Download figure as PowerPoint slide

Market price-based models

The market price-based models are often built on portfolio risk management techniques and typically derive concise “systemic risk measures” from estimated dependencies among different risk factors. These risks (e.g., sovereign, credit and market) are typically excluded when modeling the default risk of each institution in isolation (Segoviano and Padilla, 2006; Gray and others, 2010; Gray and Jobst, 2011; Jobst and Gray, 2013). Unlike accounting values, risk-based measures of solvency take into account the following considerations to inform the assessment of capital adequacy under stressful conditions (Figure 8):

• The possibility that institutions may fail simultaneously (joint default risk). Most conventional stress tests do not account for default dependencies across institutions, i.e., when one risk factor increases the likelihood of realization of other risk factors (with common shocks affecting multiple firms at the same time), especially under stressful conditions. Further, given that large shocks are transmitted across entities differently from small shocks, measuring non-linear dependence in stress testing can provide important insights into the joint tail risks that arise in extreme loss scenarios. This would also include measuring the differential effects of combinations of risk factors on the realization of joint outcomes, which affects system-wide capital adequacy.

• The sensitivity of stress test results to the historical volatility of risk factors (risk-based capital adequacy). Prudential information based purely on accounting identities observed at a certain point in time reflects the outcome of a stochastic process rather than a discrete value. In contrast, the individual and joint default risks of banks within a system vary over time and depend on the individual bank’s propensity to cause and/or propagate shocks as a result of adverse change in one or more risk factors (a distribution-based approach). Thus, there are clear conceptual differences in loss measurement under balance sheet- and distribution-based approaches affecting the comprehensiveness of the capital assessment. Unlike RWA, risk-based measures of solvency (such as market-implied expected losses and the corresponding capital shortfalls) consider the actual historical dynamics of default risk, such as Value-at-Risk (VaR), or the Expected Shortfall, i.e., the average density of extreme losses beyond VaR at a selected percentile level. Hence, in this distribution-based approach, the capital adequacy assessment takes into account the variability of both assets and liabilities at different levels of statistical confidence.

View Full Size

Key Conceptual Differences in Loss Measurements between the Accounting-based and Market Price-based Approaches

Citation: IMF Working Papers 2013, 068; 10.5089/9781616355074.001.A001

Source: Authors.

• Download Figure
• Download figure as PowerPoint slide

The stress test outcomes of market price-based models are likely to involve valuation methods and tend to be less tractable. They usually do not show direct links to key regulatory ratios, which need to be derived in separate, additional steps. Owing to data limitations—the prices of certain market instruments (e.g., equity prices and CDS spreads) are not always readily available—this approach has so far only been applied in FSAPs to supplement the accounting-based approach. The systemic contingent claims analysis (SCCA) and/or the distress dependence (DiDe) models have been used in only a handful of major country FSAPs, where the necessary data are available for credible implementation (e.g., Germany, Mexico, Spain, Sweden, the United Kingdom and the United States).

Presentation of outputs

Stress tests are aimed at drawing attention to and, if necessary, action of senior supervisors. It is thus important that the results be presented in an accessible manner in order to appropriately convey the findings, namely, by highlighting the relevant risks and vulnerabilities. In FSAPs, stress test results, especially those generated via the BU approach, are often aggregated by the authorities for confidentiality reasons, which means that the design of a meaningful presentation format for analysis by the FSAP team is essential (Figures 9–11). Specifically, the presentation of the aggregated results by the authorities should be:

• consistent with local regulatory requirements and where relevant, any transition to a new regulatory regime (e.g., Basel III); and

• sufficiently granular, such that it:

  • lists the individual institutions or (if constrained by confidentiality) peer groups, at the very least;

  • shows some measure of dispersion, such as the minimum, inter-quartile range (e.g., the 25^th, 50^th and 75^th percentiles of the distribution of capital adequacy levels) and the maximum, if they are not presented by institution;

  • shows the outcome for each year of the risk horizon;

  • shows the amount of capital required in instances where there is a failure to meet the pre-defined hurdle, both in absolute terms and as a percentage of GDP and as a percentage of total sector assets under consideration;

  • details the contributions of different drivers (e.g., profitability, credit/trading losses, RWA) of the results; and

  • clarifies assumptions and key limitations to the stress tests.

The findings of the stress tests are then used for two main purposes, which are to:

• provide quantitative support for the FSAP’s stability risk assessment by estimating the impact from the realization of key tail risks; and

• facilitate policy discussions with the authorities on risk mitigation strategies and crisis preparedness.

View Full Size

Example of Bottom-up Bank Solvency Stress Test Output Template Provided to Banks: U.K. FSAP Update ^1/

Citation: IMF Working Papers 2013, 068; 10.5089/9781616355074.001.A001

Source: Authors.^1/ See Excel attachment for actual template.

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Example of Bottom-up Bank Solvency Stress Test Output Template Provided to Authorities: U.K. FSAP Update ^1/

Citation: IMF Working Papers 2013, 068; 10.5089/9781616355074.001.A001

Source: Authors.^1/ See Excel attachment for actual template.

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Example of Bottom-up Bank Solvency Stress Test Summary Template Provided to Authorities: U.K. FSAP Update ^1/

Citation: IMF Working Papers 2013, 068; 10.5089/9781616355074.001.A001

Source: Authors.^1/ See Excel attachment for actual template.

• Download Figure
• Download figure as PowerPoint slide

Publication

The manner in which FSAP stress test results is conveyed to the public is a critical element of the exercise. In addition to providing a meaningful judgment on the outcome of the test (for instance, the fact that no bank fails a test does not mean that vulnerabilities do not exist), a substantial part of the effort in FSAPs is dedicated to the communication of results. Not surprisingly, disclosure of stress test results is a very sensitive issue, especially for supervisors and the financial institutions they oversee. Thus, the presentation of stress test findings should be appropriately nuanced to ensure that the information does not promote a false sense of security or cause undue alarm:

• The objectives, definitions, assumptions, models and limitations of stress tests are usually written up in detail, either in Technical Notes and/or as supplementary information in the FSSA report. Publication of these documents is voluntary for country authorities.

• More recently, mandatory summaries of the stress testing exercises are also presented in the FSSA in a standard framework format, i.e., the STeM, to improve transparency and facilitate comparisons across countries (Table 5).

• The aggregated results of FSAP stress tests of a particular financial system are always disclosed in the reports. As a minimum, information such as the relevant post-stress ratio(s) and the respective amount(s) of capital shortfall is presented. Rarely do authorities agree to make available the results of individual banks.

Table 5.

Example of Stress Test Matrix (STeM) for Bank Solvency Risk: Spain FSAP Update

Source: IMF (2012b).

Table 5.

Example of Stress Test Matrix (STeM) for Bank Solvency Risk: Spain FSAP Update

Domain	Assumptions
	Top-Down by Authorities	Top-Down by FSAP Team
Institutions included	Commercial banks and intervened savings banks.	All publicly listed banks with sufficient pricing history.
Market share	Over 96 percent of the banking sector, excluding foreign branches.	About 45 percent of the banking sector, excluding foreign branches.
Data and baseline date	Supervisory data as at end-2011. Scope of consolidation: legal entity as at end-2011. Risk horizon of 2 years, under crisis conditions.	Publicly available market and statutory data. Scope of consolidation: legal entity as at end-2011. Risk horizon of 2 years, under crisis conditions.
Methodology (e.g., included in scenario analysis linking solvency and liquidity, separate test using ad hoc model/balance sheet)	BdE macrofinancial panel regression model (estimates capital shortfall) without behavioral adjustments. IMF balance sheet approach (estimates capital shortfall).	Systemic CCA model (estimates expected losses, capital shortfall, and contingent liabilities).
Risks (e.g., funding liquidity shock, market liquidity shock, both)	“Double-dip” recession (severe and short-term) scenario of one standard deviation from the IMF-projected baseline GDP growth trend over a two-year risk horizon—without positive adjustment dynamics towards the end of the (short) risk horizon. The second, more adverse scenario further escalates the macro economic shock by increasing the shock to two-year real GDP growth by another 2.5 percentage points. Sovereign risk reflected in valuation haircut to AfS and trading book debt holdings. Extra provisioning and capital add-on due to regulatory changes.
Regulatory standards	Basel II transitioning to Basel III and Basel III capital requirements slightly exceeded (4 percent CT1 hurdle rate for both years). Basel III capital definition. RWA remain constant but are subject to changes due to deleveraging by banks in both 2012 and 2013.
Results	Post-shock, more than a third of all banks in the system would not be able to comply with Basel III hurdle requirements until end-2013 irrespective of the choice of top-down model. The BdE model reveals projected impairment losses of around € 73 billion under the IMF adverse scenario, which generates capital shortfall about € 18 billion compared with a Core Tier 1 capital hurdle rate of 4 percent. Based on the SCCA results, challenges exist from the realization of low probability tail risk of multiple firms experiencing a dramatic escalation of losses. In the IMF adverse scenario, the largest (and publicly listed) banks would experience a market-implied capital shortfall of more than € 14 billion on average (with a peak in excess of € 21 billion at end-2012) at a statistical probability of five percent or less (expressed as “tail risk”).

Source: IMF (2012b).

View Table

Table 5.

Example of Stress Test Matrix (STeM) for Bank Solvency Risk: Spain FSAP Update

Domain	Assumptions
	Top-Down by Authorities	Top-Down by FSAP Team
Institutions included	Commercial banks and intervened savings banks.	All publicly listed banks with sufficient pricing history.
Market share	Over 96 percent of the banking sector, excluding foreign branches.	About 45 percent of the banking sector, excluding foreign branches.
Data and baseline date	Supervisory data as at end-2011. Scope of consolidation: legal entity as at end-2011. Risk horizon of 2 years, under crisis conditions.	Publicly available market and statutory data. Scope of consolidation: legal entity as at end-2011. Risk horizon of 2 years, under crisis conditions.
Methodology (e.g., included in scenario analysis linking solvency and liquidity, separate test using ad hoc model/balance sheet)	BdE macrofinancial panel regression model (estimates capital shortfall) without behavioral adjustments. IMF balance sheet approach (estimates capital shortfall).	Systemic CCA model (estimates expected losses, capital shortfall, and contingent liabilities).
Risks (e.g., funding liquidity shock, market liquidity shock, both)	“Double-dip” recession (severe and short-term) scenario of one standard deviation from the IMF-projected baseline GDP growth trend over a two-year risk horizon—without positive adjustment dynamics towards the end of the (short) risk horizon. The second, more adverse scenario further escalates the macro economic shock by increasing the shock to two-year real GDP growth by another 2.5 percentage points. Sovereign risk reflected in valuation haircut to AfS and trading book debt holdings. Extra provisioning and capital add-on due to regulatory changes.
Regulatory standards	Basel II transitioning to Basel III and Basel III capital requirements slightly exceeded (4 percent CT1 hurdle rate for both years). Basel III capital definition. RWA remain constant but are subject to changes due to deleveraging by banks in both 2012 and 2013.
Results	Post-shock, more than a third of all banks in the system would not be able to comply with Basel III hurdle requirements until end-2013 irrespective of the choice of top-down model. The BdE model reveals projected impairment losses of around € 73 billion under the IMF adverse scenario, which generates capital shortfall about € 18 billion compared with a Core Tier 1 capital hurdle rate of 4 percent. Based on the SCCA results, challenges exist from the realization of low probability tail risk of multiple firms experiencing a dramatic escalation of losses. In the IMF adverse scenario, the largest (and publicly listed) banks would experience a market-implied capital shortfall of more than € 14 billion on average (with a peak in excess of € 21 billion at end-2012) at a statistical probability of five percent or less (expressed as “tail risk”).

Source: IMF (2012b).

All the countries in our sample have published their FSSAs. In almost all cases, Technical Notes on the respective stress testing exercises have been produced (with the exception of Australia and Spain, where the details are described in appendices to the respective FSSAs, but only a few countries (i.e., Germany, Sweden, the United Kingdom and the United States) have consented to their publication (see IMF, 2010c; 2011c; 2011d, respectively).