Paul Hilbers and Matthew T. Jones
The financial crises of the 1990s underscored the importance of detailed knowledge about the vulnerabilities of the financial sector. In response, the IMF, working closely with the international community, began to develop new tools to better assess the stability of financial systems. One of these tools, stress testing, focuses on how well a country’s financial system can weather a variety of shocks.
What is a stress test? At its simplest, a stress test shows the sensitivity of a portfolio to a particular shock. It measures the change in the value of a portfolio in response to changes in the underlying risk factors. The assumed changes in risk factors are usually made large enough to impose some stress on the portfolio (they are considered exceptional), but not so large as to be considered implausible. Stress tests can be used to assess a variety of risks (see table). They usually produce a numerical estimate of a portfolio’s change in value, often expressed in terms of the impact on some measure of capital, to illustrate the sensitivity of an institution’s net worth to a given risk.
Although stress tests were originally developed for use with trading portfolios, they have now become a widely used risk management tool—including for measuring the sensitivity of a group of institutions (such as commercial banks), or even an entire financial system, to common shocks. Such system-focused tests are not intended to replace the regular stress tests of individual institutions; rather, they provide information on the overall impact of shocks as well as on their distribution throughout the system.
The IMF has increasingly been using these tests as part of the health checks that it conducts with the World Bank through their joint Financial Sector Assessment Program (FSAP), which started in 1999. In fact, more than 90 member countries—representing roughly half of world GDP—have so far participated in this program. This article looks at how these tests are conducted and what the IMF has learned from the process so far.
Conducting a stress test
System-focused stress testing is best seen as a process that begins with identifying specific vulnerabilities. Knowledge of a country’s macroeconomic environment provides a context for the performance of the financial system and indicates potential sources of shocks. Data on ownership and market shares help identify systemically important institutions and sectors. Aggregate balance sheet structures can indicate significant exposure to particular classes of assets and liabilities or income sources. Flow-of-funds accounts can provide insights into major changes in the patterns of intermediation in the economy and into trends in fund-raising in different sectors and through different instruments.
The second step is constructing scenarios. This is done, ideally, with an econometric model that will form the basis for the stress test. The aim should be to provide a forward-looking and internally consistent framework for analyzing key linkages between the financial system and the real economy. In the absence of a formal macroeconomic model, simple sensitivity tests of the effect of varying a single parameter (for example, interest rates) can also provide useful information. Scenarios can be based on historical data, or they can be hypothetical and involve large movements thought to be plausible. Other countries’ experiences can be a useful guide as well.
What might a scenario involve? Suppose that housing prices have risen sharply on the strength of economic growth and low interest rates, fueling a mortgage lending boom. One possible scenario could involve a rise in unemployment, a fall in disposable income, and a sharp rise in interest rates affecting households’ debt servicing ability. The outputs from an econometric model could provide the information on employment, real incomes, prices, and interest rates, which could be used to formulate a specific stress test for bank balance sheets.
The third step is crunching the numbers. This means translating the various outputs of the macroeconomic model into financial institutions’ balance sheets and income statements. In the bottom-up approach, estimates are based on detailed data for individual portfolios, and in the top-down approach, aggregated or macro-level data are used to estimate the impact. A top-down approach provides a useful countercheck of the results derived from the bottom-up approach. It can also be used for financial institutions in countries that are unable to estimate the impact of a given set of shocks on their portfolio.
Stress tests can subject financial systems to a wide range of possible risks and shocks.
|Type of risk||Possible shocks||How applied|
|Interest rate||• Parallel shift in yield curve, such as 100–300 basis-point increase for all maturities.|
• Short, medium, or long spike, changing shape of yield curve.
• Domestic and foreign rates can be shocked.
|• Applied to trading book (mostly fixed income securities), possibly also to banking (credit) book.|
|Exchange rate||• Change in exchange rate, such as 10–20 percent depreciation or appreciation against currency of major trading partner (for example, the U.S. dollar or the euro), depending on the time horizon.|
• Other currencies can be stressed according to importance in portfolios.
|• Applied to trading book mostly, sometimes to banking book if foreign exchange loans are significant. Can be applied to net open positions.|
|Credit||• Increase in probability of default, such as increase across all classes equivalent to one rating notch.|
• Increase in nonperforming loan ratios: for example, higher migration from performing to all nonperforming categories or increase in worst credit quality, with associated increase in provisioning.
|• Applied to banking book. Often calibrated on previous episodes of rising defaults (for example, most recent recession).|
|Equity prices||• Fall in stock market index; for example. 20–30 percent decline in major indices.||• Applied mostly to trading book.|
|Volatility||• Increase in volatilities for interest rates, exchange rates, equity prices, and associated options.||• Applied to options portfolio. Can be calibrated based on previous episodes of increased market volatility (such as at the time of the Russian crisis in 1998).|
|Liquidity||• Decrease in liquidity of trading securities; for example, a haircut in the value of all but the most liquid securities.|
• Increase in depositor withdrawal rate or reduction in interbank credit lines.
|• Applied mostly to trading book. Can be analyzed by looking at liquid asset ratios before and after haircut.|
|Commodity||• Fall in value of key export commodity (for example, a 20 percent drop in oil price).||• Applied mostly to countries with heavy reliance on commodities (such as oil or a primary product).|
Individual institutions should be involved in implementing stress tests as much as possible because they will typically have the best access to data and the greatest familiarity with their own portfolios. Most institutions with sophisticated risk management systems or significant international operations will have stress-testing procedures in place as part of their internal risk monitoring processes. For countries with more rudimentary systems and less expertise in modeling portfolios, it may be necessary for the central bank or supervisory agency to provide guidance or even undertake parts of the empirical analysis.
The fourth step is analyzing second-round effects. Most stress-testing approaches assume that a change in risk factors does not lead to a significant change in the portfolio structure. Stress tests are typically applied to a balance sheet at a specific time or in conjunction with a forecast over a specific horizon, and the impact is calculated as if the shock were valued at market prices. This approach is valid if the time horizon is relatively short, if changes in the underlying portfolio take time to implement, or if an individual institution does not have a large impact on the financial system.
Stress testing in New Zealand and Singapore
In New Zealand, the Reserve Bank of New Zealand (RBNZ), in conjunction with the New Zealand Treasury, conducted a scenario analysis of the likely macroeconomic effects of a limited foot-and-mouth disease outbreak in New Zealand (see Gereben, Woolford, and Black, 2003). The RBNZ used its macroeconometric model (the Forecasting and Policy System) to simulate the impact on a range of macroeconomic variables of shocks to export volumes and prices, to the exchange rate, and to risk premiums on assets denominated in New Zealand dollars. These shocks were thought to be consistent with a moderate outbreak of foot-and-mouth disease. The foot-and-mouth scenario then formed the basis of a larger stress-testing exercise conducted later as part of the FSAP for New Zealand (see Gordon, 2004). As part of this exercise, the RBNZ provided the five largest banks with scenarios and supporting material and asked them to calculate the impact of the scenarios on their balance sheets. The scenarios included a series of sensitivity tests (a sharp fall in the exchange rate and a large increase in interest rates) and two dynamic scenarios (an outbreak of foot-and-mouth disease and a reduction in offshore funding). The results indicated that none of the shocks would cause a serious deterioration in bank asset quality or capital.
In Singapore, the Monetary Authority of Singapore (MAS) coordinated a stress test of systemically important banks and insurance companies as part of the FSAP (see Chan and Lim, 2004). The MAS provided three local banking groups and three systemically important foreign bank branches with a detailed list of variables for two hypothetical scenarios. The first was based on weaknesses in the global economy (particularly for the electronics sector), and the second was based on a global growth recession with additional economic weaknesses from terrorism in the region. The shocks were derived after analyzing historical trends and MAS’s baseline forecasts for the year ahead, obtained from MAS’s macroeconometric model of the Singapore economy. Banks estimated the impact of the scenarios on their balance sheets over a full year and provided their results to the MAS. The MAS also coordinated stress tests for 10 systemically important insurance companies, providing them with information on the two scenarios plus additional shocks to corporate bond prices and receivables. The results indicate that Singapore’s systemically important banks have only modest risk exposures to the scenarios. The capital buffers of the local and foreign banks were more than adequate to absorb the potential losses, and the potential losses in all but one participating bank could be absorbed by 2002 pre-tax profits. For insurance companies, the results indicate that the insurance sector is not a systemic risk to the financial sector, as the non-life sector would remain largely unaffected. The life sector could maintain solvency margins under the less severe scenario, and even under the more severe scenario would not need to raise external capital, provided they took some remedial measures to maintain solvency margins.
One strategy for analyzing second-round effects and linkages between institutions is to use contagion models, which attempt to estimate the impact of the failure of key institutions on other institutions and, hence, on the overall financial system. The exercise typically has two stages: a stress test of individual balance sheets and income statements, followed by an examination of counterparty exposures to the institutions made most vulnerable by the stress test, for example, through interbank loans, cross shareholdings, deposits, or other exposures. By examining the impact of difficulties in one institution on the health of other institutions, we can consider second-round effects in a relatively simple and intuitive manner.
The final step is interpreting and publishing the results. Stress tests enable policymakers to compare the impact of a common set of shocks on different institutions, measure the relative importance of different types of shocks (such as interest rate risk versus credit risk), and gauge the impact of shocks on different types of financial institutions (for example, is the banking system hit harder than the insurance sector, or are state-owned banks affected more than private institutions?). They can also provide information on the evolution of a system’s risk profile over time, particularly if these tests are conducted regularly.
But the use of stress tests does come with some caveats. While stress tests may be useful for evaluating the effects of large movements in key variables, they should not be portrayed as providing a precise measure of the magnitude of losses. Stress tests are also unlikely to capture the full range and interaction of risk exposures (such as operational risk and legal risk) and may give only a partial picture of the participating institutions’ risk taking. For that reason, it is useful to compare the results with other measures of risk exposure, such as financial soundness indicators.
How widely should the results be shared? Policymakers may disclose some summary information to enable financial markets and individual institutions to compare their own results with those of competitors—and many countries have done so. For example, central banks in Austria, Denmark, Hungary, Indonesia, New Zealand, Singapore, and the United Kingdom have all reported on stress test findings (see box). But the flip side of disclosure is that care must be taken to ensure confidentiality and prevent misinterpretation, especially with regard to the results of individual institutions.
Trends in test conducting
After several years of conducting these stress tests as a routine part of the FSAPs, four key trends have emerged in the way the tests are done.
• Country authorities—in particular, central banks and supervisory agencies—and individual financial institutions now play a much greater role in designing and implementing FSAP stress tests. Where possible, banks’ internal models are used to measure the impact of shocks, including on their off-balance-sheet exposures. This outcome has been facilitated by the rapidly increasing familiarity with stress tests in the financial industry.
“In some cases, the ranking of institutions (in terms of vulnerability) provided by a stress test did not come as a surprise to policymakers, but the relative importance of the different shocks provided a new insight.”
• The coverage of other financial institutions has increased, with many FSAPs now including stress tests for large insurance companies. Stress tests have also been applied to the household and corporate sectors, because of their relevance for the underlying quality of bank assets.
• It appears that data availability often drives the approach and the degree of sophistication of the tests. Most analyses are performed individually on a selection of major banks (bottom-up approach). Second-round effects have typically not been addressed in many FSAPs, and the risk of interbank contagion has been examined in only a few cases.
• Increasing use is made of macro-simulation models to calibrate consistent macroeconomic scenarios. Initially, stress testing focused on single factor shocks, and scenarios were often derived from a combination of such shocks. Macro models now allow the stress tester to link a particular set of shocks to key macro and financial variables in a consistent and forward-looking framework. As a result, model builders in supervisory institutions have become more actively involved in this part of the stress-testing exercise.
Definite value added
Policymakers have gained useful insights from stress tests about the vulnerability of their system and have appreciated them as an alternative means of confirming the results of other quantitative and qualitative analyses. In some cases, the ranking of institutions (in terms of vulnerability) provided by a stress test did not come as a surprise to policymakers, but the relative importance of the different shocks provided a new insight. In other instances, the most affected institutions were not those that the supervisor expected, or particular groups of institutions turned out to be more vulnerable to a given set of shocks. In some instances, the stress test results have shown the system to be more robust than expected or commonly reported by private sector analysts.
Many countries have also noted the benefits of the stress-testing process itself. For some countries, it was the first instance where different supervisors worked together or took a systemic view. The lack of suitable data or technical expertise also became apparent in several countries, prompting a reconsideration of the information collected from financial institutions, or a closer look at internal risk management processes. Finally, after the experience of stress testing as part of the FSAP, some countries have begun conducting stress tests on a regular basis, or requiring institutions to periodically report the results of their own tests.
The IMF’s use of stress testing is no longer limited to the FSAP. Increasingly, stress tests are becoming part of its regular surveillance of member countries’ economies. Many countries are now implementing their own stress-testing programs, and the IMF is offering more technical cooperation in this area. Work on assessing the predictive power of stress testing is also being undertaken. These developments reflect that, despite its limitations and technical complexities, stress testing has become an important new tool in assessing the strengths and weaknesses of financial systems.
Paul Hilbers is an Area Chief and Matthew T. Jones is a Senior Economist in the IMF’s Monetary and Financial Systems Department.
This paper is based on “Stress Testing Financial Systems: What to Do When the Governor Calls,” IMF Working Paper 04/127 (Washington: International Monetary Fund) by Matthew T. Jones, Paul Hilbers, and Graham Slack.
BlaschkeWinfridMatthew T.JonesGiovanniMajnoni and SoledadMartinez Peria2001 “Stress Testing of Financial Systems: An Overview of Issues, Methodologies, and FSAP Experiences,” IMF Working Paper 01/88 (Washington: International Monetary Fund).
ChanLily and LimPhang Hong2004 “FSAP Stress Testing: Singapore’s Experience,” MAS Staff Paper 34August (Singapore: Monetary Authority of Singapore).
GerebenAronIanWoolford and MellenyBlack2003 “The macro-economic impacts of a foot-and-mouth disease outbreak: an information paper for Department of the Prime Minister and Cabinet” (Wellington: Reserve Bank of New Zealand) available on the Internet at http://www.rbnz.govt.nz/research/0130346_2.pdf.
GordonMichael2004 “Outcomes of the Financial Sector Assessment Programme for New Zealand,” Reserve Bank of New Zealand Quarterly Bulletin Vol. 67 (June) pp. 49–58.
International Monetary Fund and World Bank2003 “Analytical Tools of the FSAP,” background paper prepared for March 14 2003 IMF Executive Board meeting Washington (Washington: International Monetary Fund) available on the Internet at http://www.imf.org/external/np/fsap/2003/022403a.htm.