Review of the Financial Sector Assessment Program - Further Adaptation to the Post-Crisis Era - Background Studies

The past five years have seen an expansion of the scope of FSAPs to assess countries' macroprudential policy (MaPP) frameworks. This note documents this increase and offers some suggestions on how the treatment of MaPP issues in FSAPs can be further strengthened and better integrated into the overall financial stability assessment.

Abstract

The past five years have seen an expansion of the scope of FSAPs to assess countries' macroprudential policy (MaPP) frameworks. This note documents this increase and offers some suggestions on how the treatment of MaPP issues in FSAPs can be further strengthened and better integrated into the overall financial stability assessment.

Macroprudential Policy Coverage in FSAPS: Progress and Remaining Challenges1

The past five years have seen an expansion of the scope of FSAPs to assess countries’ macroprudential policy (MaPP) frameworks. This note documents this increase and offers some suggestions on how the treatment of MaPP issues in FSAPs can be further strengthened and better integrated into the overall financial stability assessment.

A. Motivation

1. FSAP assessments are being conducted in a much more complex environment compared to that prevailing at the program’s inception. Financial systems and institutions have become increasingly sophisticated, integrated within and across borders, larger, and complex (Otker-Robe et al., 2011). And despite the recent crisis and the extensive reform measures adopted since then, systemically important institutions continue to be perceived as “too important to fail” (IMF, 2014). Economic and financial connectivity has increased countries’ exposure to potentially destabilizing macro-financial feedback loops. This environment has put an even greater premium on FSAP assessments providing a critical review of countries’ ability to manage and respond to systemic risk. A key element in this regard is the capacity to implement MaPPs that are targeted at identifying and managing systemic risk.

2. The 2009 FSAP review represented an important step toward responding to these challenges (IMF, 2009). The review envisaged refining stress-testing methodologies to incorporate macro-financial and cross-border linkages; addressing the regulatory perimeter, procyclicality of prudential norms, and supervision of cross-border entities; enhancing the capacity of national authorities to coordinate with domestic and foreign peers; and strengthening crisis preparedness and management strategies. It also envisaged new methodologies for assessing macroprudential vulnerabilities that could gradually become a part of the FSAP with a view to enriching the policy dialog.

3. This note reviews the coverage of macroprudential issues in FSAPs against this backdrop. It documents the extent to which the coverage of MaPP issues has become an important part of FSAPs since the last review, discusses the areas that have been covered more extensively, and identifies ways where the coverage could be strengthened going forward, drawing on the various building blocks of MaPP identified in the paper on “Key Aspects of Macroprudential Policy” (IMF, 2013).2 The note also examines the extent to which MaPP analyses have been well integrated into financial stability assessments. It acknowledges, however, that in the 2010–14 period, the MaPP guidelines were still being formulated and expertise was still being developed, within and outside the Fund. The coverage of MaPP frameworks in, and their integration to, FSAPs have therefore evolved over time as the policy lines on MaPP have been formulated in parallel with the changing scope of FSAPs. The coverage is expected to further improve in the coming years with the forthcoming Staff Guidance Note on MaPP.

B. Assessing the Macroprudential Policy Coverage of FSAPs

4. This section reviews the coverage of MaPP in FSAPs undertaken since the last FSAP review in late 2009. The analysis is based on a review of mostly published Financial System Stability Assessments (FSSAs), covering a set of 60 FSAPs for which the scoping missions were undertaken during 2010–13. The FSAPs initiated during the calendar years 2009 and 2014 were not included so as to limit the analysis to only those FSAPs carried out after the last review and to those with completed FSSA reports. The review covers countries and jurisdictions from all income groups, including 23 advanced economies (AEs) and 37 emerging market and developing economies (EMDEs), 8 of which are low income countries (LICs) (Table 1).

Table 1.

FSAP Coverage, 2010–131

article image

Uses the IMF World Economic Outlook’s groups and aggregates classification scheme.

For the purposes of this paper, “year” refers to the calendar year in which an FSAP Scoping Note was completed. This calendar year convention is used instead of the FSAP fiscal year convention because it best captures the time period in which FSAP teams finalized the scoping of the topics to be covered in the FSAP.

5. The “Key Aspects” paper provides a useful frame of reference to assess the extent and scope of MaPP coverage in FSAPs (IMF, 2013). The paper provides a framework that lays out the role MaPP play in the overall policy mix and its interactions with other policies, sets out the key steps to operationalize MaPP (to assess systemic risk, assemble and calibrate the toolkit, and close regulatory and data gaps to better manage the risk), and discusses the institutional mechanisms that enable effective implementation. It also discusses the multilateral issues that can arise in its application (e.g., through cross-border spillover of MaPP actions (or lack of action), cross-country differences in the phase of financial cycles, and conflicts between home-host authorities of cross-border banking groups). In what follows, these key aspects are used to assess the MaPP coverage and identify the areas to better integrate MaPP analysis to financial stability assessments.

6. The review suggests that macroprudential issues have indeed become an important feature of FSAPs in recent years. Many FSSAs completed in 2010–14 include explicit discussions of MaPP, some with technical notes dedicated to various aspects of the MaPP framework and systemic risk,3 although the depth and breadth of the coverage differ across countries. In some cases, macroprudential issues are covered but without explicit references to MaPP (e.g., through discussions of system-wide risks associated with growing indebtedness of the corporate or household sectors or interconnectedness and complexity of financial systems; risks and supervisory challenges in oversight of financial conglomerates; or the need for home-host coordination to address systemic risks arising from cross-border bank activities). Overall, advanced economy FSSAs contain a slightly more extensive coverage (Figure 1), but the latter reflects, in general, the prominence of macroprudential issues in a given economy. The depth of coverage also seems to have grown along with the evolution of policy lines on MaPP, with FSAP teams drawing on a wide range of conceptual, analytical and empirical MaPP work within and outside the Fund (Figure 2).

Figure 1.
Figure 1.

Coverage of Various Aspects of MaPP in FSAPs

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Sources: FSSAs completed during 2010–14 and staff computations.
Figure 2.
Figure 2.

Evolution of Coverage of Various Aspects of MaPP in FSAPs

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Sources: FSSAs completed during 2010–14 and staff computations.

C. Operational Aspects of MaPP

7. Systemic risk assessments have become a standard feature of FSAPs—a marked change from earlier stability assessments that focused primarily on risks affecting individual sectors and not the financial system as a whole (IMF, 2012). FSAP teams carry out systemic risk analyses with increasing sophistication, covering, in general, both time and structural dimensions of systemic risk associated with the buildup of systemic vulnerabilities over time or through financial interlinkages. Fairly advanced network analysis, stress-testing, contingent-claims, and distress-dependence tools are used to carry out contagion analyses with a view to assessing the structural dimension of systemic risk arising from growing interconnectedness within and across financial systems (Figure 3 and Box 1). Teams also discuss systemic risk that may arise from linkages between banks and nonbanks (e.g., Brazil, Chad, China, the Czech Republic, Kosovo, Malaysia, Uruguay, among others) and, in a few cases (Colombia, the European Union (EU), France, Italy, Spain, Switzerland), between banks and sovereigns.

Figure 3.
Figure 3.

Coverage of Selected Issues Relevant for Macroprudential Analysis

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Sources: FSSAs completed during 2010–14 and staff computations.

8. FSAP assessments have emphasized the challenges faced in properly identifying and measuring systemic risk, and recommended monitoring and closing information and regulatory gaps that could undermine effective responses to deal with it. Many FSAPs provide explicit recommendations to strengthen systemic risk analysis and monitoring, including through revamped stress-testing exercises, intensive supervisory monitoring of financial linkages and potential contagion channels, more granular supervisory data on exposures of financial institutions, and collection and dissemination of high quality data on financial indicators, such as real estate prices. FSAPs also note that gaps in supervisory-regulatory oversight could undermine the ability to identify and address systemic risk, and discuss the challenges introduced by financial conglomerates. Accordingly, they recommend supervisory authorities to have explicit legal authority to oversee directly all entities within a conglomerate in order to identify possible contagion risks, to subject like-activities to like-regulation, and to coordinate closely the regulation (and taxation) of close substitutes.

9. FSAP assessments often discuss the various measures undertaken by country authorities to address emerging risks and offer recommendations on how to strengthen the toolkit. In some cases, FSAPs recommend expanding the existing MaPP toolkit to mitigate time and structural dimensions of systemic risk, or tightening or eliminating other MaPP tools, making use, in some cases, of country experiences. Less frequently, FSAPs discuss the effectiveness of the macroprudential tools in place, and, in a few cases, assess their costs (such as distortionary effects of the countercyclical tax on banks in Slovenia, or direct lending limits in China) and potential leakages (e.g., Brazil, China, India FSSAs). The Brazil FSSA notes, for instance, that while the financial transaction tax and differentiated capital requirements were generally effective in achieving their objectives, in some cases the effect was temporary, owing to circumvention through other tools.

10. Calibrating MaPP instruments to address changes in systemic risk is a challenging aspect of MaPP frameworks and receives relatively limited coverage in FSAPs. In some cases, FSAPs note that the macroprudential authority calibrates the instruments to target specific risk factors, based on a more systematic approach, including by simulating their impact and taking into account cross-country and past experience and the uncertainty about their transmission to macroeconomic and financial stability (e.g., Singapore). In others, FSAPs, note the authorities’ preference to use judgment, in light of the insufficient evidence on the efficacy of certain MaPP tools from countries with a similar level of financial development and structure (e.g., Australian authorities use judgment about the ability of borrowers to repay or the quality of bank lending standards in adjusting tools such as risk weights or capital requirements, instead of imposing loan-to-value (LTV) and debt-service-to-income (DTI) limits).

11. The discussions of specific MaPP tools in the FSAPs are typically guided by the source of systemic risk that is prevalent or emerging in a given financial system (consistent, in general, with the risks identified in the Risk Assessment Matrix). Some teams focus exclusively on the time dimension of systemic risk arising, for instance, from rapid credit growth or overheating in a particular segment of the market, such as real estate, and recommend introduction or tightening of LTV or DTI limits or countercyclical capital buffers (e.g., Austria, Brazil, Netherlands, Poland, Switzerland). In others, cases, FSAPs highlight risks associated with growing connectivity and complexity of financial institutions, some with large conglomerate structures (e.g., India, Malaysia), and recommend a broad range of tools, including systemically important financial institution (SIFI) capital surcharges or higher capital and liquidity buffers, more intensive supervision, enhanced inter-agency coordination, greater supervisory powers and resources to change the necessary tools, or widening the regulatory perimeter to ensure that like-activities are similarly regulated.

Structural Dimension of Systemic Risk: Selected Cases of Contagion and Spillover Risk Analysis in FSAPs

Czech Republic: Contagion tests were used alongside top-down and bottom-up stress tests to assess the threat of contagion that foreign parent banks posed to the Czech financial system. The contagion tests simulated a partial loss on exposures to parent banks under a severe double dip scenario, which assumed that 40 percent of the gross exposure of banks to their parent would be lost. The tests revealed that the risk of direct contagion from foreign parent banks was significant and that a capital shortfall of as much as two percent of GDP would arise if a severe double dip recession were to occur.

Germany: Bank-by-bank balance sheet and market-based systemic stress testing approaches were used in conjunction to comprehensively assess the effects of spillovers/contagion on the solvency of the German banking system. Basel III requirements, endogenous bank behavior (e.g., dividend payout ratios), and macrofinancial linkages, simulated via satellite models, were also incorporated into the stress-testing scenarios.

Hong Kong SAR: Network analysis was used to illustrate the effects of inward and outward spillovers on Hong Kong. The analysis complemented traditional stress tests and helped to highlight foreign bank branch vulnerability to credit and funding shocks. Using these methods, it was determined that Hong Kong’s banking system, as a whole, was resilient to significant credit shocks, including those stemming from mainland China.

India: Network analysis was used to model the contagion effects in the banking system. The analysis showed that the failure of the two most connected net borrowing banks would give rise to financial contagion in five distinct stages.

Italy: A comprehensive analysis of domestic and foreign bank exposures, intra-group liquidity activity, and banks’ gross foreign claims was conducted to assess the risk of inward/outward spillovers involving the Italian banking system. The analysis indicated that exposure to foreign banks constituted only a limited risk to the Italian financial system, but that Italian bank subsidiaries were systemically important to a large number of Central and Eastern European countries. In addition, extensive top-down and bottom-up liquidity/solvency stress tests were performed to gauge the resilience of Italian banks to shocks. The results suggested that the use of additional buffers above regulatory minima and European Central Bank (ECB) liquidity support would mitigate the risk of Italian banks transmitting shocks to the rest of the euro area.

Korea: Sophisticated stress tests, which took into account the simultaneous occurrence of interbank and macroeconomic shocks, were used to determine that banks would remain solvent in the face of second-round spillover effects which may arise from sudden liquidity shortfalls or asset fire-sales.

Malaysia: An interbank contagion stress test was conducted using a gross interbank exposures model. The model accounted for the effect of local/foreign currency exposures and traced out contagion paths to assess the degree of joint credit and funding shocks arising from various types of defaults. The test results were used to conclude that the simultaneous default of multiple large domestic commercial banks would give rise to severe financial contagion.

Poland: Top-down stress tests were utilized to assess the banking system’s resilience to credit shocks, liquidity shocks, and contagion risk. The tests suggested that up to 30 percent of Polish banks may fail to meet minimum capital adequacy ratios under severe macroeconomic scenarios.

Singapore: Network analysis was used to show that Singapore is relatively more vulnerable to cross-border interbank exposures than to domestic interbank exposures. The analysis showed that the country was most at risk from credit shocks originating in the U.K. Contagion analysis was also used to show that the network effects from foreign to local banks in the domestic interbank market were limited.

Sweden: Contagion tests were employed to evaluate the degree of concentration risk and interbank exposure arising from unsecured lending, securities-trading, and derivative-related activities. The test results indicated that the financial sector could withstand the default of a major bank but not the joint default of the three largest bank exposures.

Switzerland: Top-down and bottom-up stress tests incorporating credit, market, contagion, liquidity, funding, and operational risk factors were performed on the majority of Swiss banks. The stress tests were used together with tests of contagion to conclude that systemic risk arising from domestic banks was increasing, whereas other potential sources of contagion to the Swiss financial system were either decreasing or stable. The tests also showed that the risk of contagion from Swiss banks to other sovereigns had fallen significantly in recent years.

United Kingdom: Nonparametric empirical copula were used to construct market perception indicators for estimating systemic risk in the U.K. banking system. The indicators, which took into account the Joint Probability of Distress (JPoD) and the Conditional Probability of Distress (CoPoD), suggested that the implied spillover threat posed to foreign large complex financial institutions (LCFIs) by U.K. banks had returned to pre-crisis levels after a sharp rise during the course of the crisis.

D. Institutional Framework

12. FSAP assessments increasingly recognize that a strong institutional framework guiding the implementation of MaPPs is essential to ensure that MaPP can work effectively. Most FSAPs provide assessments of the institutional framework for MaPP, recognizing the role of a strong institutional framework to assure willingness to act when needed, limit biases for inaction, as well as to reduce incentives to use MaPP for purposes other than safeguarding financial stability. Extensive discussions also reflect ongoing efforts in many advanced and emerging market economies to set up an effective institutional mechanism that formally assigns a specific institution an explicit mandate for monitoring, assessing, and addressing systemic risk with a view to safeguarding financial stability. Consensus is yet to emerge on whether a single institution, such as the central bank, or a separate financial stability committee (or a systemic risk council), should carry out the task of systemic risk monitoring, regulation, and crisis management4 Accordingly, FSAP recommendations concerning the institutional architecture are not uniform, also taking into consideration factors unique to a country (see Box 2 for selected examples):

  • Several FSAPs (e.g., for Belgium, Czech Republic, Germany, Netherlands) recommend making the central bank the main MaPP authority, expanding its price stability mandate to include systemic risk monitoring and financial stability, and granting the central bank a clear role to implement tools to address emerging systemic risks, with appropriate accountability frameworks to enhance willingness to act.

  • A number of others (e.g., for Austria, Brazil, Chile, China, El Salvador, the EU, Israel, Korea, Malaysia, Mexico, Paraguay, Poland, Sweden, Turkey) recommend assigning the MaPP mandate to an inter-agency body, board, or a committee, some along with the provision of “act or explain” powers to the committee and a leading role to the central bank. In other cases, teams are less prescriptive in recommending a given institutional model (e.g., Canada, Italy, Switzerland).

13. While specific recommendations differ on the various ways to strengthen the institutional framework FSAP assessments underscore the importance of: (i) providing the MaPP authority with appropriate powers to achieve its goal and establishing clear goals, mandate, and accountability mechanisms; (ii) assigning a major role to the central bank, given its expertise in analyzing systemic risk, and protecting its independence when it is assigned the MaPP responsibility; and (iii) ensuring an adequate level of government involvement, including to help garner political support for MaPP actions (see also IMF (2012)). Beyond the discussion of designation of the macroprudential authority, FSAPs do not provide detailed discussions on the modalities of MaPP implementation. There are more limited discussions, for instance, on the nature of the powers assumed by the macroprudential authority (direct control, act-or-explain, or provision of opinion or recommendation), or on the choice between rules vs. discretion in decision making.

Selected FSAP Recommendations on the Institutional Framework for MaPP

article image
article image
article image
article image

E. Interaction with Other Policies

14. FSAPs, acknowledge that MaPP cannot alone be effective in safeguarding financial stability. The interaction of MaPP with other policies is important when assessing the stability of a financial system, because achieving and maintaining financial stability is a shared responsibility. To achieve its goals, MaPP must hence be supported by strong microprudential supervision and enforcement, and complemented by appropriate monetary, fiscal, structural, and other financial sector policies. Effective MaPPs can, in turn, help these policies attain their goals (IMF, 2013). Clear definition and delineation of roles across policies would help limit the risk of MaPP being used (or abused) to achieve non-macroprudential objectives, which other policies would be better placed to address (e.g., fiscal, monetary, or structural policies). In this context, many FSAP teams discuss the need to specify clearly the objectives of MaPPs, as well as bringing out the complementarities and trade-offs their interaction with other policies may present in attaining their respective goals.

15. FSAP assessments have increasingly emphasized the importance of a clear delineation of micro- and macroprudential responsibilities. Missions have stressed (e.g., in Poland) that the authorities need to set clear MaPP objectives that are separate from monetary and microprudential policy objectives, emphasizing that the regulatory agency should not use tools to manage the credit cycle but focus on the soundness and safety of financial institutions, while the MaPP authority should focus on reducing systemic risk. FSAPs have also highlighted (e.g., Germany) the need for free exchange of information between micro- and macro-prudential supervisors, since the macroprudential supervisor, in its role, needs to assess domestic, foreign, and regional risk factors and make recommendations that may affect microprudential supervision.

16. The importance of micro-macroprudential linkages has been reflected in assessments of supervisory architectures. For example, analyses of the Dutch twin-peaks supervisory approach in the Netherlands FSAP underscored the need to take advantage of the synergy between micro-and macroprudential oversight. Where these responsibilities lie in the central bank, teams have noted (e.g., Armenia) that assigning these responsibilities to separate departments may help address potential conflicts arising from the central bank’s multiple mandates (including price stability, financial stability, regulation, supervision, and crisis management).

17. FSAP assessments have also underscored the importance of effective crisis prevention and crisis resolution frameworks to help limit and mitigate systemic risk. Teams had rich discussions on the scope to improve the existing financial safety nets (including emergency liquidity assistance, bank resolution, and deposit insurance) to ensure that they can cope with future shocks and limit moral hazard in increasingly larger and more complex financial systems (e.g., in Brazil). FSAPs have also highlighted (e.g., in the Czech Republic) the importance of close coordination between the MaPP authority and fiscal authorities on crisis management issues, as well as of coordination with peers through participation in cross-border crisis management groups and better access to information on recovery and resolution plans for the banking groups active in the domestic financial system.

18. At the same time, FSAP assessments have also highlighted the potential trade-offs between MaPP and crisis management that could undermine financial stability. FSAPs have noted, for example, that certain tools, such as substantial precautionary buffers of government securities that banks are required to hold in India, may provide liquidity protection to banks during crisis times but leave banks potentially exposed to unmanaged interest rate risk. Where crisis prevention and management responsibilities and MaPP role rest in the same institution (e.g., Armenia), teams have also underlined potential conflicts that may arise, and emphasized the role of formal arrangements to reduce such conflicts.

19. FSAPs have also brought to light the possible interactions between MaPP and monetary policy and the potential complementarities and tradeoffs that may arise under certain circumstances. For example, when interest rate policy may have adverse financial stability implications, teams have noted that central banks may establish a dedicated committee for financial stability issues (e.g., as in Armenia), facilitating the delineation of MaPP from monetary policy. Teams have also stressed that under certain situations, monetary and macroprudential policies may reinforce each other. The India FSSA argued, for example, that the authorities tightened (or loosened) several macroprudential instruments in conjunction with, and as a complement to, tighter (looser) monetary policy, to reinforce its effect in dampening the credit cycle. A number of FSSAs have also signaled that a range of MaPP tools central banks use do not always have a macroprudential intention and recommended to limit their use to reducing systemic risk, while using more conventional policies, such as monetary policy, for macro management.

20. Potential complementarities between MaPP and fiscal policy are also captured in a number of FSAP assessments. FSSAs discuss, for instance, the role of tax instruments in addressing systemic risk (e.g., stamp duty in Hong Kong SAR to cool down the housing market when overheating pressures in the housing market may reflect demand factors) or in contributing to it (e.g., tax deductibility of interest payments in the Netherlands and Switzerland, and the special tax on banks in Slovenia). Less frequently, FSAPs also discuss the role of market infrastructure policies (e.g., central counterparties (CCPs), standardization of over-the-counter (OTC) derivatives) to improve systemic liquidity risk management and limit counterparty risk (e.g., Hong Kong SAR, India, Spain, Singapore). Other structural policies, such as competition policy, feature less in the assessments, though in a few cases, FSAPs highlight its role in facilitating access to alternative sources of funding to reduce systemic risk posed by large complex financial and nonfinancial conglomerates.

F. Multilateral Aspects

21. The multilateral aspects of MaPPs—an area of growing importance in a world of highly interconnected financial systems—have received relatively less attention in FSAPs. The limited coverage of potential spillover effects of failures to take MaPP action where needed, or leakage and boundary effects of unilateral national regulatory actions for other countries (e.g., tighter constraints imposed on globally active cross-border banks, ring-fencing financial systems, or tighter regulatory requirements compared with the international standards) reflects, to some extent, the ongoing international deliberations on these issues. Nonetheless, several FSAPs discuss (e.g., in Poland, Australia, the EU, Malaysia, Mexico, and the U.K.) the need for coordination to limit cross-border arbitrage, with explicit discussions of the reciprocity arrangements across countries.5 FSAPs highlight the importance of close supervisory coordination to address the potential implications of policies imposed by one country on others (e.g., Austria, Australia, the EU, France, Poland, Singapore, Switzerland, and the U.K.), or comment on the (insufficient) evidence of adverse spillover effects of policies to address external sources of systemic risk (e.g., the capital flow management measures targeted at financial stability in Brazil).

22. In this connection, home-host supervisory coordination is one area where FSAPs have devoted most attention—in effect, a manifestation of greater emphasis on cross-border issues since the global financial crisis in 2007. FSSAs have covered in some detail the need for effective coordination between home and host authorities of cross-border banking groups—an area that has gained prominence as financial systems have become tightly interlinked. The importance of formalizing cooperation through Memoranda of Understanding (MOUs), active participation in supervisory colleges and crisis management groups (CMGs), and information—and burden—sharing arrangements has been underscored for host countries with systemically important G-SIFI operations (e.g., Albania, the Czech Republic, Ghana, Poland, Singapore), as well as for home countries of international banks—through sharing of supervisory information and joint on-site inspection programs beyond supervisory colleges (e.g., India, the Netherlands, Sweden, the U.K.). In a regional context, FSAPs have assessed crisis management and supervisory coordination mechanisms in an evolving global regulatory landscape (e.g., the EU FSAP).

23. FSAPs have also delved into the treatment of cross-border banking group affiliates in this context (Box 3). Teams have discussed unilateral national policy actions undertaken by host authorities, such as ring-fencing of local affiliates, close oversight and regulation of their activities, limits imposed on intra-group transactions, and standardized treatment of subsidiaries and branches to limit cross-border contagion risks. FSAPs have not, in general, prescribed particular actions against such measures, while at times noting their merits (e.g., Albania, Australia, the Czech Republic, Finland, Singapore, Luxemburg) given the difficulty of assessing risks associated with branches where mechanisms for information exchange between home-host authorities are missing or weak (Fiechter et al., 2011). FSAPs have underscored the need for group-wide supervision to provide home-host authorities with a better understanding of the soundness of different parts of a group, especially in which local operations are systemically important for host and home jurisdictions. They have stressed the importance of effective supervisory coordination through active participation in supervisory colleges and crisis management groups, regular information exchanges and joint assessments of banking groups and their affiliates, close monitoring of exposures to parents and subs, and cross-border resolution agreements with foreign counterparts of failing cross-border banks.

G. Conclusions and Implications for Future FSAPs

24. The review of 60 FSSAs completed during 2010–14 suggests that assessments of systemic risk, MaPP frameworks to address them, and of their role in financial stability have become an important feature of the FSAP in recent years. Consistent with the call by the 2009 FSAP review, most FSSAs reviewed include explicit discussions of MaPP issues, some with dedicated technical notes on the various aspects of MaPP. FSAP teams conducted extensive systemic risk monitoring and analyses with advanced techniques to capture time and structural dimensions of systemic risk, analyzed the macroprudential frameworks the authorities used to carry out systemic risk assessments and respond to the identified risks, described the institutional frameworks that guide macroprudential policy-making and identified potential areas for improvement, and touched on the interactions of macroprudential oversight with other policies—in particular microprudential oversight and crisis management frameworks.

25. However, the depth and breadth of the coverage and the degree to which MaPP assessments were integrated into the overall stability assessment differ across countries, as well as over time. Overall, the FSSAs for advanced economies and more advanced emerging market economies had more comprehensive coverage, and more recent FSAPs had more systematic treatments of the various aspects of MaPP frameworks, which were also better integrated into the overall stability analysis. The depth and scope of the coverage of MaPP issues, as well as their integration into the financial stability analysis, have by and large evolved along with the development of policy lines on MaPP within and outside the institution and the changing scope of FSAPs as a key tool for bilateral surveillance.

26. The treatment of MaPP issues in FSAPs can be strengthened in various dimensions, and the findings can be more systematically integrated into the overall financial stability analysis. Going forward, future analyses of MaPP frameworks in FSAPs should capitalize on the experience gained in recent years to strengthen the coverage of the various aspects of MaPP. In addition, MaPP analysis needs to be better integrated into the overall assessment of financial stability. After all, FSAPs and MaPP frameworks share precisely the same objective: to assess the stability of the overall financial system and identify and respond to the key threats and weaknesses undermining its stability, with a view to strengthening its resilience to future shocks. In this context, FSAPs need to ensure:

  • Better integration of MaPP analysis into the stability analysis. Such integration will require greater efforts to reflect the key messages in the executive summary and in the FSAP recommendations, and cross-referencing the key findings throughout the FSSA report In some FSSAs reviewed, detailed analyses of systemic risk or the institutional setup and tools to respond to them did not feature prominently in the summary or among the key recommendations, and the vulnerabilities arising from the various components of the stability framework (e.g., crisis management-resolution framework or bank-nonbank financial linkages) were insufficiently articulated in evaluating the risks to the overall financial system. Adequately connecting the dots will ensure that FSAPs provide an integrated assessment of the key risks faced by the financial sector.

  • A more systematic treatment of MaPP frameworks. As the knowledge and understanding of MaPP frameworks deepen, the coverage of MaPP analysis in FSAPs should become more comprehensive and systematic, as the most recent FSAPs suggest. Dissemination of best practices from country experiences, together with the framework for MaPP advice developed in the Key Aspects paper (and operationalized in the forthcoming Staff Guidance Note on Macroprudential Policy), will provide a more systematic treatment of the key dimensions of MaPP frameworks in FSAPs and improve its integration into the overall stability analysis.

27. Using systemic risk—the key focus of MaPP—as the organizing concept could help in ensuring a more systematic treatment of MaPP analysis and better integrating it into the overall assessment of financial stability. Treatment of key MaPP issues in each of the three pillars of stability assessments (i.e., main threats to systemic stability; the monitoring, regulation, and supervision of these risks; and the authorities’ ability to mitigate the risks or to respond to them if the risks could not be contained) will help ensure that systemic risk considerations will guide the stability analysis in FSAPs.

28. FSAPs should also assess the authorities’ capacity to operate their MaPP frameworks effectively. Given the still limited experience with some MaPP tools and their potential leakage and boundary effects within and across jurisdictions, future MaPP work in FSAPs could focus more on the multilateral and calibration aspects of MaPPs. FSAP teams could face growing demand to provide analyses of potential costs and benefits and cross-border spillover effects of MaPP measures, as well as to assess the authorities’ capacity to do such assessments. The latter may become particularly important in an environment where the resort to MaPP measures may increase as AEs start normalizing their unconventional monetary policies.

Selected Treatments of Foreign-Bank Branches and Subsidiaries in FSAPs

Albania: The FSSA highlighted the significant presence of foreign banks in Albania and discussed how financial difficulties in the banks’ parent countries could impact Albania’s financial system via direct contagion, defunding pressures, and increases in non-performing loans (NPLs). The FSSA noted that the decision of the central bank to convert foreign bank branches into subsidiaries (which the FSAP viewed as a MaPP measure) was a timely step that served to limit contagion from foreign parents to the local banking system and preserve liquidity in the banking system, while acknowledging that deleveraging pressures may nevertheless continue. It recommended the authorities to obtain, to the extent possible, support letters from mother banks agreeing to provide liquidity and solvency support to their subsidiaries and formalize cooperation with relevant foreign bank supervisors.

Australia: The FSAP examined the relationship between Australian-based subsidiaries and New Zealand’s banking sector. It concluded that spillover effects from Australia to New Zealand are likely to be limited; because the subsidiaries of major Australian banks—which account for 90 percent of New Zealand banking assets—are ring-fenced, allowing host authorities to require the subsidiary to maintain sufficient capital and liquidity buffers in the country. It noted that the statutory obligations of both supervisors and Trans-Tasman Council on Banking Supervision, comprising key representatives of the two country regulatory agencies, provided a mechanism for cooperation and coordination during financial stress, and encouraged the authorities to enhance cross-border coordination arrangements with other jurisdictions.

Czech Republic: The FSSA discussed how the deteriorating financial condition of foreign-parent banks may have a significant downstream effect on these banks’ Czech subsidiaries. The assessment described a number of ways in which foreign-parent bank actions could disrupt the Czech financial system, including deleveraging at the subsidiary level, moving substantial subsidiary resources upstream, and engaging in fire sales of Czech subsidiary assets in times of stress. It discussed how risks to the reputation of parent banks could also negatively affect Czech bank operations by raising host subsidiaries’ cost of funding. The introduction of intra-group exposure limits and enhanced cross-border cooperation were prescribed to address the risks.

El Salvador: The FSSA noted that some local subsidiaries of international banks hit hard by the global crisis were directed by their headquarters to conserve risk capital and faced reductions in external credit lines after the onset of the global financial crisis. These subsidiaries faced restrictions, despite being well-capitalized and not directly exposed to the crisis. The assessment determined that banks’ direct cross-border exposures were effectively limited by parent-bank policies that require their subsidiaries to fund and lend mostly in the domestic market. Cross-border risks were also contained by regulatory requirements that limit foreign asset exposure to a certain percentage of capital. The FSSA notes that, while parents could extend liquidity to their subsidiaries, this may not always be forthcoming except in certain circumstances; liquid assets provide self-insurance against liquidity shocks—though at a high cost both for banks in terms of bank profitability and for the system in terms of forgone credit.

Finland: The FSSA noted that large foreign ownership brings challenges for home-host coordination and puts a premium on effective cross-border supervision. It noted that plans by two pan-Nordic financial groups to convert their Finnish subsidiaries into branches is likely to limit supervisory effectiveness in Finland since the authorities would have more limited supervisory power over branches. Effective supervision would then require enhanced efforts at home-host cooperation, including timely and relevant information sharing of foreign branch activities and effective cross-border crisis management planning. Pending EU decisions in this area, cooperation will need to include more explicit information exchange; clear understanding of burden sharing; increased harmonization of how international standards are applied to improve joint risk monitoring; common reporting system and supervisory approaches. It recommended enhancing the framework to assess systemic risk, including by using more detailed information on cross-border exposures and cross-border cooperation on systemic risk assessment with other authorities.

Ghana: The FSAP gave an account of the high degree of ownership of Ghanaian bank assets by foreign-bank subsidiaries. The FSSA found that major internationally active banks in Ghana were the only ones that exhibited an active risk management culture and discussed some of the benefits of a strong foreign-bank presence. It highlighted weaknesses in bank oversight by pointing out the failure to conduct due diligence when issuing banking licenses to foreign-bank subsidiaries. It stressed that the systemic importance of foreign banks to the local financial system requires prompt correction of weaknesses in supervision.

Hong Kong SAR: The FSSA noted that foreign-bank branches account for 35 percent of the total local banking system assets and 27 percent of domestic lending, and that the authorities strongly encourage foreign branches to subsidiarize if their local business becomes significant. The FSAP found that the authorities have robust bilateral arrangements in place to exchange confidential supervisory information and to engage in cross-border intervention of foreign-owned institutions. Cross-border efforts were also found to have been initiated between Hong Kong and mainland China to combat the risks related to cross-selling of financial products. The FSSA noted that active engagement with supervisory colleges and crisis management groups of G-SIFIs (banks and insurance companies) plays a key role in managing cross-border risks given the dominant role of G-SIBs locally.

India: The FSAP noted that an upcoming change in the existing entry norms of foreign banks (which permitted only branch presence in India, though with locally assigned capital requirements) under discussion would provide incentives for foreign banks to incorporate as subsidiaries. It also noted that the Reserve Bank of India defers to foreign supervisors of overseas offices and subsidiaries for hands-on supervision of operations subject to their jurisdiction. Given the large and growing overseas activities of Indian banks in many foreign jurisdictions, the absence of formal or informal arrangements for information exchange with from host supervisors is a serious information gap that should be addressed through acceleration of formal MOUs or other means.

Luxembourg: The FSSA noted that the crisis exposed significant vulnerabilities in the financial system due primarily to large cross-border exposures to foreign-parent banks (subsidiaries’ net providers of liquidity to their parents). A decline in cross-border intra-group transactions during the financial crisis had caused balance sheets to severely contract, and a few bank subsidiaries failed on contagion from their parent groups. Stress tests showed that the failure of a counterparty could lead to that of at least one local bank, illustrating how local subsidiaries remain exposed to liquidity and counterparty risk from their parents. Given the contagion risks from home jurisdictions and the pre-crisis weaknesses in the supervision of foreign subsidiaries and in crisis preparedness/resolution (e.g., cross-border communication), the FSSA recommended continuation of monitoring parent-bank exposures and readiness to take further action to limit unsound exposures to parent banks (e.g., with formal sanctions), as well as strengthening supervisory collaboration through active participation in supervisory colleges and by seeking pragmatic solutions to facilitate coordination in cross-border resolution.

Mexico: The FSSA noted that the banking system is dominated by seven large banks, five of which are foreign-owned subsidiaries of major international banks, making home-host cooperation essential. It discussed the possible effects of liquidity pressures facing parent groups on their subsidiaries and local credit growth, given the subsidiaries’ funding structure that relies heavily on a broad and diversified domestic deposit base. It also discussed how G-SIFI surcharges on parents could affect capital allocation in the group, including subsidiaries. It recommended that internationally agreed principles should guide potential conflicts of interest between home-host supervisors for locally systemically important G-SIFI subsidiaries. Coordination in the form of MOUs and the smooth functioning of supervisory/crisis management colleges are essential for a resolution process that balances the interest of home-host jurisdictions. It also stressed the need for crisis-simulation exercises and reciprocal knowledge of parents and subsidiaries’ legal/regulatory frameworks.

Netherlands: The FSAP found that the Dutch central bank frequently relied on supervision performed by host supervisors, although it had the necessary legal and regulatory powers to engage in consolidated supervision of cross-border banking groups. The FSSA noted that the activities of Dutch banks’ foreign subsidiaries were significant enough to threaten the viability of their entire banking groups should they fail. It recommended the authorities to dedicate additional resources for more intrusive monitoring of the activities of Dutch bank subsidiaries located abroad, revamp its supervision and information sharing with foreign supervisors to obtain timely information about the contagion risks that host subsidiaries’ operations pose to their home banks, and participate in supervisory colleges to avoid risk of regulatory arbitrage and a fragmented view of risk.

Poland: The FSAP found a high degree of interconnection between the country’s banking system and the rest of the European financial system. To mitigate the risks posed by foreign-bank branches, Poland was advised to fully utilize the “reciprocity” principle agreed upon by EU institutions (e.g., the ESRB), thereby ensuring that measures taken by Polish authorities would be applied to EU-bank branches operating in Poland. As a general precaution, it was also recommended that the Polish central bank intensify its supervision of foreign-banks branches’ risk management practices.

Singapore: The FSSA noted that 116 of the country’s 122 banks are foreign-bank branches that collectively own 65 percent of banking system assets, and that branches face certain rules in their operations: there are limits to the number of branches with retail access; wholesale branches are allowed to take wholesale domestic funding but not retail; and purely offshore branches are generally not allowed to accept deposits in Singaporean dollars from residents. Branches are subject to the same liquidity requirements as domestic banks, and the Monetary Authority of Singapore (MAS) can impose corrective and remedial action on branches. The FSSA discussed the risks associated with branching including from parent instability and challenges of supervision/resolution, and whether branches pose material risk to financial stability. It noted the authorities move to convert large retail branches into domestically incorporated subsidiaries to address potential spillover risks from parents, and highlighted their efforts in international fora for greater sharing of supervisory information on G-SIFIs. It recommended that asset maintenance ratios imposed by the MAS to limit the risk arising from branches should be complemented with a strict licensing policy, monitoring parent-bank risks and international supervisory coordination.

Spain: The FSAP conducted a cross-border analysis that also analyzed the impact of ring-fencing by host authorities of major Spanish banks, in which host supervisors’ unilateral ring-fencing of bank affiliates may prevent transfer of excess subsidiary resources within a banking group. The FSAP mentioned that countries such as Spain, whose two largest banks have sizeable foreign operations outside of the EU, are particularly susceptible to spillover effects from the ring-fencing of their domestic banking groups’ foreign subsidiaries. The analysis found that the banks could be affected somewhat by such actions under adverse conditions and their capital ratios would be lower but the impact would be limited.

U.K.: The FSSA discussed how domestic authorities’ powers varied greatly depending on whether a financial institution was classified as a branch of an EU-based parent bank or a U.K.-based subsidiary. The FSAP stressed the need to address this disparity by enhancing collaboration between home and host authorities, including joint inspections and decision-making processes, and by increasing the amount of information shared via using supervisory colleges and bilateral contacts. It also discussed the authorities desire to push through financial reforms related to ring-fencing and standalone subsidiarization of EU-bank branches should any such reforms fail to pass at the EU level.

Evenhandedness of FSAPS6

This study was prepared by external consultants, who were asked to evaluate the evenhandedness of FSAP assessments. The authors defined the scope, focus, and methodology of the study. While the results were discussed with staff, the analysis and views expressed here are solely those of the authors, do not represent IMF views or policy, and should not be attributed to the IMF or its Executive Board.

A. Introduction

29. The theme of this paper, evenhandedness, is difficult to define objectively. What someone considers to be fair may not be so if seen by others. In the end, it is in the eye of the beholder, and in the case of FSAP, or any other IMF-related matters, the relevant beholder is member country authorities and Board members.

30. We therefore do not attempt in this paper to deliver any definitive verdict on evenhandedness of FSAPs. What we do instead is to offer benchmarks for assessing FSAP practices which may help Board members reflect on their evenhandedness. We present several benchmarks in this paper, analyze recent FSAPs based on them, and show whether there are notable deviations from what may be considered as “even.” The presence of such deviations does not necessarily imply lack of evenhandedness. And there are potentially numerous benchmarks that one can use in assessing FSAP. But our analysis of these benchmarks does suggest that further examination of FSAP practices may be warranted, in order to ensure that they remain fair and effective.

31. In what follows, we consider three aspects of FSAP; namely, (i) the use of FSAPs by member countries; (ii) the cross-country distribution of Basel Core Principles (BCP) ratings; and (iii) stress test modalities and assumptions. These three have been chosen in light of their importance as well as the ease of benchmarking. Since this paper is written in the context of quinquennial FSAP review, our sample FSAPs are those that have been conducted over the past five years. We consider both published and unpublished FSAPs and BCP Assessments, in order to avoid any bias in the sample that may arise from the authorities’ decision to (or not to) publish. The list of our sample FSAPs, 82 altogether for FY2010–14, can be found in Appendix Table 1, along with some country attributes.

B. Use of FSAPs

32. Access to FSAP was entirely voluntary until financial stability assessments under FSAP became mandatory for some members. In September 2010, the Board adopted the decision that made financial stability assessments under the FSAP mandatory for 25 systemically important jurisdictions, which naturally changed the composition of FSAP-recipient countries toward larger and higher income Fund members.7 This shift was exacerbated with the expansion of the list to 29 jurisdictions in 2013.8

33. The introduction of mandatory financial stability assessments under FSAP should not affect the ability of other members to request an FSAP in principle. We have been informed by staff that when a member formally requests an FSAP, that request is generally met, though sometimes with some delay. Thus, there is apparently no case of “denied access to FSAP,” in spite of the heavy burden being placed on the IMF’s FSAP resources due to the need to carry out financial stability assessments for all systemically important countries every five years.

34. That said, there is still a potential concern about how FSAP resources are being utilized by different groups of member countries. Since FSAP resources are limited, a larger share of their allocation to systemically important jurisdictions does imply that fewer resources are being allocated to the rest of member countries than otherwise. The Board was informed of this implication at the time of the decision to make FSAP assessments mandatory for a subset of the Fund’s membership.9 This section explores whether there are notable patterns in the way these scarce resources are being used that might worry the broader Fund membership.

35. We compare use of FSAPs by member countries between FY2005–09 and FY2010–14, focusing on certain country characteristics. First, we examine distribution across different groups of countries according to the standard country classification; namely, systemically important jurisdictions (SIs), AEs, emerging markets (EMs), and LICs.10 Second, we look at the regional distribution, as represented by area departments’ country coverage. The regional and country classifications used in this paper are shown in Appendix Table 1.

36. Figure 4 shows, for each country group, the share of countries that had an FSAP during the two five-year periods. Not surprisingly, SIs’ FSAP share has jumped from 44.0 percent in FY2005–09 to 100 percent in FY2010–14. On the flip side of this, the shares of non-SI EMs and LICs have plummeted to 41.3 percent and 27.6 percent, respectively. A mechanical calculation shows that with these shares, non-SI EMs would expect to have an FSAP only once every 12.1 years on average, and LICs once every 18.1 years (Table 2).11 These numbers will further rise in the coming years, since the number of SIs has been raised to 29.

Figure 4.
Figure 4.

Distribution of FSAPS across Country Groups; FY2005–09 vs. FY2010–14

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Authors’ estimates.
Table 2.

Average Length of Interval between Two FSAPS

(in years)

article image
Source: Authors’ estimates.

37. The difference in the implied years of FSAP interval is rather stark between SIs on one hand, and non-SI EMs and LICs on the other. During those long interval years, some non-SI EMs and LICs might have multiple home-grown financial crises. It is clear that the Board made a decision in September 2010 that more FSAP resources be allocated toward SIs, and extended this mandate in December 2013. And, at least implicitly, the Board also decided to limit the FSAP resources that would be directed to other jurisdictions. The question now, with the benefit of five years’ experience, is whether the substantial fall in the FSAP frequency for non-SIs is deemed to be extreme, and whether there is a need to take some additional action, such as expanding FSAP resources, reducing SIs’ FSAP frequency, or simplifying SI FSAPs to release resources. We present this as an issue for the Board’s further consideration.12

38. The change in the regional distribution between FY2005–09 and FY2010–14 is shown in Figure 5. The figure indicates the “FSAP share” for each area department, which is defined to be the “number of countries that had an FSAP, divided by the number of countries that belong to the department,” expressed in percent. One can see a fairly sharp contrast between the two five-year periods. The share dropped very substantially in the African and Middle East and Central Asia Departments (AFR and MCD, respectively), while it jumped up in the Asia and Pacific Department (APD) from a low level. The European Department (EUR) has consistently been well-represented, although the Western Hemisphere Department (WHD) has the highest share of all in FY2010–14, close to 60 percent.

Figure 5.
Figure 5.

Regional Distribution of FSAP; FY2005–09 vs. FY2010–14

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Authors’ estimates.

39. The uneven regional distribution shown in Figure 5 does not necessarily imply a lack of evenhandedness. Assuming for now that the actual country group distribution observed during FY2010–14 was fully intended, we give weights to countries under each area department to calculate a hypothetical FSAP share that takes country characteristics into account. The weights used are the FSAP shares of different country groups shown in Figure 4.13 This is a potentially useful exercise because, for example, a particular area department’s high share may be simply due to its having many SIs in its constituency. These hypothetical shares are compared with the actual shares in Figure 6.

Figure 6.
Figure 6.

Actual and Hypothetical FSAP Shares across Regions; FY2010–14

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Authors’ estimates.

40. We can derive two implications from Figure 6. First, the hypothetical distribution shows that if we take the recent FSAP distribution across country groups as given, we should expect a fairly large difference in the use of FSAPs across regions. AFR and MCD countries’ expected FSAP incidence during a five-year period is about 30 percent, whereas that of EUR countries is more than 60 percent.14 Second, the actual distribution across regions in FY2010–14 was even more lopsided than the hypothetical one, since AFR and MCD countries’ actual use was lower than hypothetical. Also, while EUR’s discrepancy between the actual and hypothetical is small, WHD’s over-representation is not removed by weighting. As noted earlier, FSAPs are voluntary for non-SIs, and there is no case of “denial of access.” Hence, any unevenness of regional distribution should be attributed to the choice of individual member countries. But if the IMF considers some degree of geographical evenness as desirable in light of the universal nature of the institution, some attempts could be made to give priority to FSAP requests from countries under AFR and MCD.

C. BCP Assessments

41. An examination of BCP ratings is a useful additional yardstick for our evaluation of evenhandedness. A large majority of FSAPs include assessments of international standards and codes, and among them, BCP is by far the most widely assessed. Standards and codes assessments offer a clearly-stated “grade” on each principle such as “compliant” and “largely compliant,” which makes it easy to carry out cross-country analysis of how member countries are being assessed.

42. However, it is important to recognize that BCP assessments are not exactly comparable across countries. The source document of BCP published by the Basel Committee on Bank Supervision lists several reasons for this, and concludes that “Seeking to compare countries by a simple reference to the number of ‘Compliant’ versus ‘Non-Compliant’ grades they receive is unlikely to be informative.”15 The reasons for non-comparability include the so-called “proportionality principle;” that is, to take into account the specific features and context in a particular jurisdiction which is being assessed (complexity of the system and institutions being assessed), recognizing that supervisory practices should be commensurate with the risk profile and systemic importance of the banks being supervised. Taken at its face value, the Basel Committee’s argument would mean that one should be careful in using BCP assessment grades for cross-country comparisons or as a benchmark for evenhandedness.

43. Yet, it seems wrong to completely foreclose the possibility that these grades can be used to compare different jurisdictions. We all strive to get a good grade when we take a test, especially if our grades are to be published. And an important reason why we strive is because we worry about how we do relative to others. Country authorities are no exception to this rule. Moreover, how they are assessed in BCP and other standards and codes can influence the decision of investors and rating agencies, in which case the grades they receive have a very tangible payoff. Indeed, this was one of the original objectives of the standards and codes initiative.16

44. In light of these arguments, we assume that the assessments made for individual countries are fully comparable. Our intention is not to verify fairness of specific assessments by comparing them across countries. We are not BCP experts, and do not consider ourselves to be capable of evaluating experts’ judgments one by one. What we do instead is to apply some basic statistical analysis to BCP assessments to uncover their underlying patterns. For this, we first convert the assessment for each principle to a number (i.e., compliant = 4, largely compliant = 3, materially non-compliant = 2, and non-compliant = 1). We then calculate a simple average of scores on all principles for each country, and check if we can find any pattern or bias in its distribution across countries. The BCP assessments analyzed in this Section are shown in Appendix Table 2. They include all assessments, published or unpublished, conducted during FY2010–14, altogether 69 assessments.17 For this exercise, we use the following five country groupings; systemically-important AEs (SI/AEs); systemically-important EMs (SI/EMs); non-SI AEs; non-SI EMs; LICs; and OFCs.18

45. Figure 7 offers an overview of BCP assessments in our sample. In all the panels of Figure 7, average scores are on the x-axis, and standard deviations (SDs) on the y-axis. The distribution across country groups (first panel) shows that; (i) SI/AEs and SI/EMs have received high scores; but (ii) LICs have been dispersed in terms of both their average and standard deviation of scores. Also, dividing BCP assessments into published and unpublished ones reveals a pattern that relatively unfavorable assessments tend to remain unpublished (third panel). While this is natural and understandable, we note that such a behavior of member countries gives rise to a bias in the overall message that the public receives from BCP assessments.

Figure 7.
Figure 7.

Average and Standard Deviation of BCP Assessment Scores

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Authors’ estimates.

46. Our next step is to find some benchmarks against which BCP scores (country average) can be evaluated. We tried the usual suspects; namely, per capita income, the Corruption Perception Index (Transparency International), and the Ease of Doing Business Index (World Bank). We did not have high hopes, because there is no logical reason why BCP experts’ judgments should be correlated with any of these indicators at all, except perhaps a likely loose link that a country’s “institutional strength” might be related to all these variables. It turned out that this link was much stronger than we initially thought.19

47. Figure 8 plots the relationship between BCP scores and per capita income, the Corruption Perception Index, and the Ease of Doing Business Index. They all tell a similar story. First, average scores are correlated with the three indicators, with each of AEs (including SI/AEs), EMs (including SI/EMs), and LICs forming a cluster. The dispersion within a cluster is smallest for AEs, and largest for LICs. Second, among these country groups, SI/EMs seem different in that, conditioned on the benchmarks, their scores are better than the others regardless of which indicator is used as benchmark.

Figure 8.
Figure 8.

Average BCP Scores (on y axis) against Benchmarks

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: World Bank (2013 ranking).

48. To go beyond observations from visual inspection, we conduct some elementary regression analysis. Namely, average BCP scores are regressed on each of the three indicators and a constant term to estimate the intercept and the slope, for the overall sample as well as group-by-group sub-samples. For this exercise, our sub-samples are “SI/AEs+AEs,” “SI/EMs+EMs,” and “LICs,” since non-SI AEs and SI/EMs are too few to form an independent regression sample.20 These three sub-samples have roughly 20 observations each.

49. The results, shown in Table 3, confirm that the BCP average scores’ correlation with the three indicators is statistically significant for the overall sample. The link is less clear for sub-samples, with mixed significance of parameter estimates.21 Using the result obtained from the per capita income (PCI) benchmark, one can expect that a country with $20,000 PCI (AE) has an average BCP score higher than a country with $1,000 PCI (LIC) by 0.6, i.e., slightly more than half a notch in the four-notch grading scale. This difference, which seems smaller than one might expect from the income difference, may reflect the “proportionality principle” embedded in BCP (see paragraph 42).

Table 3.

OLS Regression Results (Yi = α + βXi + εi); Overall Sample and Sub-samples

article image
Source: Authors’ estimates.Notes: 1) *, **, and *** implies ten percent, five percent, and one percent significance, respectively.2) Since a smaller number represents a better status in the EoDB ranking, the β coefficient in the EoDB regression is estimated to be negative.

50. In order to check how BCP scores are distributed in each group relative to the benchmarks, we plot z-scores in Figure 9.22 For each group, the individual z-scores are ordered from the smallest to the largest. A positive score means that the country’s BCP is better than is predicted by the benchmark.

Figure 9.
Figure 9.

Z-scores Derived from the Three Benchmarks

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Authors’ estimates.

51. From the figure, we can make the following observations. First, as noted earlier, SI/EMs are rated better, as a group as well as individually. One of them have z-scores for Corruption Perception Index and Ease of Doing Business Index exceeding two, which may be taken as a threshold for abnormal observations. Second, AEs’ and non-SI EMs’ z-scores are well-balanced around zero, with no abnormal observations. Third, LICs’ z-score is slightly biased on the negative side. There is one conspicuously positive observation (per capita income and Corruption Perception Index). On the negative side, four LICs recorded abnormal observations for some or all benchmarks.23

52. These findings require careful interpretation as to what has caused them and what they mean. Any pattern found in the BCP scores’ deviations from our benchmarks does not necessarily imply that assessments lack evenhandedness. In fact, the “model” we use here is very simple, and the indicators used in it have little intuitive appeal as having strong power to explain BCP scores. There can be any number of country-specific or other factors that may account for the observed patterns of z-scores. We leave the task of identifying these factors to a more detailed review of BCP assessments that the Board may wish to initiate. However, we take one more step in this Section to shed some light on what might lie behind the observed patterns.

53. The differences across countries in BCP average scores come from different assessments made for individual principles. It is interesting to check which principles are contributing most to the differences in the average scores. To answer this question, we present the average and standard deviation of BCP scores for each principle (Figure 10).24 A yellow circle is added to highlight those principles whose assessments have high standard deviations across countries. These are the ones that are contributing most to the differences in the country-by-country BCP scores.

Figure 10.
Figure 10.

Average (x axis) and Standard Deviation (y axis) of BCP Scores for Each Principle

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Authors’ estimates.

54. The principles in the yellow circle, the ones that have the highest standard deviations across countries, are as follows, in the descending order in terms of the size of standard deviation;

article image
article image

From some of the items on this list (e.g., Principles 1.5 and 18), one can see a possible link with the institutional strength that some of our benchmark indicators might represent. However, perhaps more important are the supervisory authorities’ and banks’ technical skills in monitoring, assessing, and managing various risks, such as in Principles 12, 16, and 13, that might be correlated with some of our benchmarks. With these observations in mind, the question to be asked is whether those countries with particularly high (or low) z-scores are much stronger (weaker) in these areas than might be expected from their positions in terms of the benchmark indicators.

55. Another aspect of BCP assessment that can be checked objectively is who did the assessment. Each BCP assessment is carried out by two experts in principle, and sometimes by three. The total of 68 experts were involved in the 63 assessments in our sample, with some of them conducting multiple assessments (Table 4).25 The table shows that the assessment work has been widely distributed among assessors, with as many as 40 doing only one assessment over the past five years. Out of the total of 63, four assessments were made by a pair of one-time assessors (that is, zero continuity with any other assessment). The involvement of so many assessors naturally makes comparison of assessments across countries and over time more difficult.

Table 4.

Number of Assessments Carried Out by Individual Assessors (FY2009–14)

article image
Source: Authors’ estimates.Note: The counting is only for FY2009–14, so it is possible that assessors have done more assessments once earlier years are included.

56. An obvious question one can ask is whether the assessment scores have been affected by assessors’ identity. We test this by including an assessor dummy in our simple regression model with the three benchmark indicators. We do this only for four assessors who have done five or more assessments over the past five years. The result, shown in Table 5, indicates that the average assessment scores can indeed be influenced by assessor identity, sometimes as much as by half a notch in the four-notch scale of BCP ratings. This may call for greater efforts to ensure better standardization of assessors’ judgments.

Table 5.

Assessor Dummy

article image
Source: Authors’ estimates.Notes: 1) A-D are assessors, and the number in parenthesis after each assessor refers to the number of assessments he/she has done over the past five years.2) PCI, CPI, and EoDB are “Per capital income,” “Corruption Perception Index,” and “Ease of Doing Business,” respectively, used as benchmark indicator.3) Our regression equation is Yi = a + bXi + cADj + εi, where Yi is Country i’s average BCP score, Xi is the benchmark indicator such as PCI, ADj is a dummy for a particular assessor j, and the estimated value and significance of “c” are shown in the Table.4) ** and *** indicate 5 percent and 1 percent significance, respectively.

D. Stress Testing

Overview

57. Stress testing is one of the core parts of FSAP that helps grasp the risks of the financial systems, especially from the macroprudential viewpoints. The nature, distribution, and extent of financial system risks vary across member countries, but evenhandedness is still needed in assessing these risks while considering country specific situations. To evaluate evenhandedness of stress testing, the seven “best practice” principles which IMF (2012)26 offers as practical guidelines for macrofinancial stress testing are a useful starting point. We select four principles from them which we think are most pertinent to risk propagation, and examine the stress tests in the published FSSAs completed in 2009–14 to see if they are consistent with these Principles.

Principle No. 1 “Define Appropriately the Institutional Perimeter for the Tests”

58. The first step in designing stress testing is to define its institutional perimeter. To determine systemic importance of financial institutions, interconnectedness is an important criterion along with size, substitutability and complexity. Analysis of interconnectedness, such as network analysis, is useful in identifying relevant financial institutions to be covered. Stress testing also requires finding an appropriate perimeter of financial activities. In this regard, the GFC has clearly shown that the financial system outside the banking system (so-called shadow banking institutions and activities) also plays a large role in inducing a crisis.

59. Our review of recent stress tests indicates that the institutional perimeter differs substantially from country to country. For example, analysis of interconnectedness is limited to SIs, AEs and a few from other categories, with LICs entirely missing (Figure 11). Also, stress tests of non-bank financial institutions are concentrated on SIs, AEs, and OFCs, with EMs and LICs almost missing (Figure 12). In addition, most of these non-bank stress tests cover just the insurance companies, and some cover other long-term non-bank financial institutions, leaving aside shadow banking entities like investment funds and finance companies, and shadow banking activities like securities lending and repos of securitized products which function as credit intermediaries.

Figure 11.
Figure 11.

Share of FSAPs that have Analysis of Interconnectedness

(Network or Related Analysis; in percent)

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Authors’ estimates.Notes: 1) SIs include the EU. AEs and EMs here exclude SIs.2) The radar chart illustrates, for each country group, the ratio of the number of FSAPs that contain analysis of interconnectedness to the number of all FSAPs.
Figure 12.
Figure 12.

Share of FSAPs that Covered Non-bank Financial Institutions

(in percent)

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Authors’ estimates.Notes: 1) SIs include the EU. AEs and EMs here exclude SIs.2) The radar chart illustrates, for each country group, the ratio of the number of FSAPs with stress tests of non-banks to the number of all FSAPs.

60. The difference across countries in the institutional perimeter may well be explained by the structure of the recipient country’s financial system or simply by the lack of data. For example, if the country’s non-bank activities are still at their infancy, conducting a stress test on them would be unnecessary. However, there may be some scope for expanding the stress test perimeter in some AEs and EMs, at least. As evidenced by the FSB’s monitoring report,27 not only AEs but also EMs have large asset shares of shadow banking entities. In spite of this, stress testing that covered shadow banking substantively is found only in the United States (U.S.) FSSA.

Principle No. 2 “Identify All Relevant Channels of Risk Propagation”

61. There are a variety of potentially important channels of shock propagation. Among them, as financial globalization deepens, the analysis of cross-border spillover and contagion has become one of key channels of risk propagation worldwide. However, so far such a spillover analysis has been conducted only for a small number of countries, heavily concentrated on SIs and a few from other categories (Figure 13). Spillover risks do not seem to be limited to SIs and a few other countries, as illustrated by the worldwide effects of the recent tapering of the U.S. Federal Reserve’s unconventional monetary policy. In OFCs and LICs there are some cases where only single-factor risk analysis has been conducted (Figure 14).28 Since such analysis cannot deal with synergistic effects of multiple risk factors, it cannot capture complex channels of risk propagation. As all member countries face a variety and combination of risks, the highly selective application of spillover and other macro scenario analysis across countries does not seem appropriate, and may constitute lack of evenhandedness.

Figure 13.
Figure 13.

Share of FSAPS that Contained Analysis of Cross-border Spillover and Contagion

(in percent)

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Authors’ estimates.Notes 1) SIs include the EU. AEs and EMs here exclude SIs.2) The radar chart illustrates, for each country group, the ratio of the number of FSAPs with spillover stress tests to the number of all FSAPs.
Figure 14.
Figure 14.

Share of Countries with Only Single-factor Tests or No Stress Tests

(in percent)

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Authors’ estimates.Notes 1) SIs include the EU. AEs and EMs here exclude SIs.2) The radar chart illustrates, for each country group, the ratio of the number of FSAPs with only single-factor tests or no stress test to the number of all FSAPs.

62. IMF (2012) argues that “policy feedbacks” are one of propagation channels, because the authorities’ policy reaction (or lack thereof) affects the nature and extent of risk propagation. Although the paper does not explain what these “policy feedbacks” are in any detail, this concept could include the authorities’ action after a crisis, such as new and tighter regulations, aimed at preventing its recurrence. In this regard, we have noted a considerable number of survey responses by member country authorities that stability assessments do not take into account effects of such mitigating policy actions and that the perceptions of shocks are exaggerated. For example, one FSSA Report acknowledges that as a result of the ongoing restructuring promoted by the authorities, one third of the banks in the stress test sample no longer exist as stand-alone entities and that the financial strength of the merged entities may be different from the sum of its individual parts, which is not captured by the stress tests. Although it is difficult to find a solution to this situation, a practice like this goes against the spirit of taking account of policy feedbacks.

Principle No. 3 “Include All Material Risks and Buffers”

63. In order to utilize the stress tests effectively to assess risks of financial systems, capturing a variety of risks is key. For this, as is currently discussed in many international fora, it is important to collect necessary financial data. We therefore look into data gaps as a potential obstacle to stress testing’s ability to capture all material risks and buffers.

64. Figure 15 shows the share of FSSAs in which data gaps are explicitly mentioned. One can see that insufficient data are a common problem for AEs, EMs and LICs. The breakdown of data gaps shows that there are cases where only the public data are provided to FSAP teams (Figure 16). Partly as a result of that, the teams could not conduct their “own” stress tests in the end in some cases, which accounts for 12 percent of SIs and 4 percent of AEs among countries for which data gap issues are cited. In some FSAPs it is stated that a data gap hindered the full assessment of the extent of the risks and of how they could permeate through the economic and financial system (see Box 4).

Figure 15.
Figure 15.

Share of FSAPs with Data Gaps

(in percent)

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Authors’ estimates.Notes: 1) SIs include the EU. AEs and EMs here exclude SIs.2) “Data gap” includes the member countries where FSSAs mention the existence of data gap or FSAP team did not conduct the stress test.3) The radar chart illustrates, for each country group, the ratio of the number of FSAPs facing data gap issues to the number of all FSAPs.
Figure 16.
Figure 16.

Share of FSAPs that Used Only Public Data for Stress Testing

(in percent)

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Authors’ estimates.Notes: 1) SIs include the EU. AEs and EMs here exclude SIs.2) The radar chart shows, with each country group, the ratio of the number of FSAPs using only public data or aggregate data by the teams to the number of FSAPs facing the data gap issues.

65. The existence of data gaps implies that although stress test results presented in FSAP Reports appear to be derived from a standardized procedure and of similar quality, they differ very substantially. Indeed, it is possible that the results cannot properly show the underlying risks, depending on the nature and quality of data. This suggests that the authorities’ willingness or ability to offer critical source data to the FSAP team can undermine evenhandedness of stress tests, in the sense that some countries could hide behind the veil of secrecy or data insufficiency.

Data Constraints Affecting the Quality of FSAP

There are some cases in FSAPs that assessment of risks in the financial systems is hindered by the data gap because the authorities cannot collect financial data sufficiently or they cannot provide supervisory data to the FSAP team due to legal or other constraints. These cases are mostly centered on EMs and LICs, but cases originating in legal constraints are also seen among AEs. Such data constraints affect the evenhandedness of stress testing.

A case in point is the FSAP for China that was conducted in 2010. In this FSAP top-down stress tests were conducted by the authorities using the supervisory data, and by the FSAP team using public data only.

The executive summary of the China FSSA published in November 2011 notes the following: “A full assessment of the extent of these risks [multiple shocks occurring at the same time] and how they could permeate through the economic and financial system, however, was hindered by data gaps, the lack of sufficiently long and consistent time series of key financial data, weakness in the informational infrastructure, and constraints on the FSAP team’s access to confidential data.”

More concretely, the FSSA lists the following data gaps as the basis for the caveats attached to the team’s risk assessment: data constraints on the sectoral exposures and types of entities that banks lend to; off-balance sheet positions and operations of banks; short time series of key financial data with structural breaks in the series; and constraints on the FSAP team’s access to confidential data.

Other countries whose FSSAs refer to data gaps include many LICs (for example, Mozambique, Tanzania, El Salvador, Paraguay and Rwanda) and some non-SI EMs (for example, Romania and Serbia).

66. Stress tests’ quality also depends critically on how accurately the financial system’s buffers to shocks are captured. In this context it is especially important to grasp the true level of capital buffer of financial institutions. We find that in some cases, FSAP teams suspected an over-reporting of capital adequacy, and attempted to adjust capital buffer by quantifying the extent of over-reporting (see Box 5). These adjustments are judgmental, and their appropriateness debatable, but simply disregarding suspected under-reporting is not the team’s right response, either. So we are not against making adjustments, but leaving it entirely to individual teams’ discretion is not a good solution. While it must be difficult to establish strict guidelines, we suggest that staff produce a note that presents cases, both actual and hypothetical, of under-reporting and explains possible adjustments to be made, so that the teams’ case-by-case decision can be sourced to some common understanding. A step like this will enhance FSAP’s transparency, and contribute to its evenhandedness.

Principle No. 5 “Focus on Tail Risks”

67. Recent FSAPs are making a systematic effort to determine severe shocks that are comparable across countries. One rule of thumb being used for stress scenarios is a two standard deviation (2SD) shock on real GDP growth over a period of two years applied to the baseline real GDP projection by IMF (WEO or Article IV adjusted by the FSAP teams and the authorities). The standard deviation of real GDP is usually calculated using 20–30 years of past data. However, there is no strict guideline imposed on FSAP teams to follow this rule, and the practice differs from country to country, apparently depending on each country’s specific circumstances. Actual shocks provided are being determined by the FSAP teams and the authorities by adjusting the rule widely.

Case-by-case Adjustment of Capital Buffer by the FSAP Team

In some countries, there are cases where banks are seen to be under-provisioning for loan losses, thereby over-reporting their capital adequacy.

For example, in the FSAP for Russian Federation conducted in 2011, the FSAP team tried to capture the quantitative impact on capital of structurally weak corporate governance of banks and the flaws in the supervisory framework. Specifically, (i) to adjust for the regulatory forbearance introduced during the crisis from late 2008 to early 2009, its estimated impact was added to the required provisions at end-2010, and (ii) to further adjust for the low level of provisions, the required provisions for each loan category were raised to the midpoint of the regulatory range, and collateral of poor quality was assumed to have no value. The team estimated that adjusting for these factors could wipe out as much as one-third of total bank capital. The team admitted that these adjustments were somewhat ad hoc and arguably extreme. However, it was also noted that they did not cover all possible sources of overestimation of capital.

Another example is the FSAP for Rwanda conducted in 2011. The financial sector and the regulatory framework are in the process of transition to a more modern financial system, but the FSAP team judged that banks were still under-provisioning, and adjusted their capital by applying stronger provisioning requirements that were being used in neighboring countries. As of the end of September 2010 the banking industry had 12.8 percent of core capital-to- asset ratio on average, and 20.3 percent of regulatory capital-to-asset ratio. However, by correcting the under-provisioning (such as raising the provisioning against interest rate risk) and by imposing general and watch provisions, the core capital ratio dropped to 11.6 percent and the regulatory capital ratio to 18.5 percent. After making these adjustments, the FSAP team conducted stress testing in cooperation with the National Bank of Rwanda.

Furthermore, in Rwanda, not all collateral held against a loan was eligible for the purpose of provisioning. The stress test made a very conservative assumption that all collateral held against classified loans becomes worthless. As a result, core capitalization declined to 8.3 percent and regulatory capitalization to 13.3 percent. With these adjustments, two banks fell below 6 percent core capitalization with one of them falling below three percent.

68. The ad hoc relationship between the rule of thumb and the actual practice may cast doubt on the way stress scenarios are produced. Unless a very careful specification and standardization are made, different countries will be subjected to different levels of stress, which clearly violates evenhandedness. Country specific adjustments being made quite frequently make us wonder if the resulting stress scenarios have enough uniformity of treatment across countries.

69. The use of 2SD shock in the “rule of thumb” stress testing suggests that it is based on the notion that it sets the likelihood of stress events at a certain low level of probability. This itself is reasonable, but the way standard deviations are calculated and used in stress scenarios gives rise to the following questions.

  • a) Is it appropriate to use the historical mean GDP growth rate over two years as the central value for calculating standard deviations? What if the mean GDP growth rate was shifting over time? Also, what if the mean real GDP growth over two years has a serial correlation? It seems more reasonable to take into account the shift of the mean GDP growth rate and at the same time to take into account serial correlation of GDP movements (i.e., use information about recent GDP developments).

  • b) Is it appropriate to apply the calculated 2SD shocks to where the economy happens to be when an FSAP is conducted? When the economy is immediately after a crisis, is it appropriate to superimpose 2SD stress events on the resulting low level of economic activity? For the sake of standardization across countries (in particular, to minimize the influence of when an FSAP happens to be conducted), it seems more reasonable to use a method of choosing and applying shocks to the economy that does not unduly penalize (favor) the country that is in recession (in a boom) at the time of FSAP.

  • c) What is the right sample period over which to calculate standard deviation? If a country had a tail shock episode in the past, whether or not to include that episode in the sample influences the calculated standard deviations substantially. More generally, there are different ways to define a 2SD event, which might be more reasonable than historical SD, regardless of its sample period for calculation

70. To address some of these questions, we present an alternative method of defining stress events in the next sub-section, and see how they differ from the actual stress events used by FSAP teams.

Alternative method of defining stress events

71. The idea behind our proposal on alternative “rules of thumb” is to standardize macro stress scenarios across countries. The criteria we use for selecting such rules is the following;

(i) they should be simple enough that every FSAP team can easily implement; (ii) they should be based on the notion that stress events occur with a certain low level of probability; (iii) this probability should be broadly uniform across countries, regardless of when an FSAP takes place; (iv) at the same time, some account should be taken of the recent economic conditions because they affect the level of risks the country faces in the short run. There is a tension between (iii) and (iv) above, so the rules need to strike a balance between them.

72. There are different ways to construct macro stress scenarios that meet our criteria. Among them, we consider the following two, which we think are simple enough to be alternative “rules of thumb.” In both cases, we assume that real GDP moves around its stochastic trend which we capture by HP filter. By taking the difference between the actual and trend GDP, we obtain the cyclical component of GDP denoted by Ct. Then, the two methods are:

  • (i) Calculate the standard deviation of Ct (SDC) from past data, and subtract 2SDCs from the trend GDP that is expected to prevail in one to two years from the time of FSAP;

  • (ii) Estimate an autoregressive (AR) model of Ct, and produce a one-year and two-year forecast for Ct (Ct+1e and Ct+2e) using the model. Then, construct stress scenarios by subtracting 2SDs of the error term of the AR model from Ct+1e and Ct+2e.

Some details of these methods are given in the Appendix. They differ primarily with respect to the extent to which the recent economic conditions affect stress scenarios. Under Method (i), their influence is kept minimal,29 whereas under Method (ii), they set the starting point of the projection because the recent realizations of Ct are used for forecasting.

73. The comparison of actual stress scenarios used in the FSAPs and alternative scenarios based on our rules of thumb is shown in Table 6. Six countries are presented for illustration in two groups―”projection of economic downturn” and “projection of economic recovery”―in order to check how the countries’ economic circumstances at the time of FSAP affect stress scenarios. With a proviso in mind that only a relatively small number of country cases have been examined, we note the following from the Table;

  • a) Under the alternative methods, the shocks are smaller than those under the current rule of thumb. This is because of the differences in the way standard deviations are calculated. First, in our methods, fluctuations of trend GDP are removed when estimating real GDP’s volatility that forms the basis of shocks.30 Second, our methods take into account the mean-reverting property of real GDP’s cyclical factor. On both counts, the use of historical standard deviations in the current rule of thumb exaggerates the shocks to be applied, compared with our methods.

  • b) The discrepancy between the shocks under the alternative methods and those under the current rule of thumb tends to be larger when an economic downturn is projected than when an economic recovery is projected. This is because the current rule of thumb applies 2SD shocks regardless of where the country is in its business cycle. Our methods can reduce the tendency of the current rule to amplify the size of shocks at the time of economic downturn, and to possibly underestimate it at the time of economic recovery. This has an especially important implication when the output gap is large and the financial system is struggling to cope with the recession and financial stress right after a crisis (e.g., EUR SI A in the Table). Under the current rule of thumb, a further economic shock is imposed on the economy, and the output gap is projected to widen drastically. In contrast, our methods offer more moderate stress scenarios.

  • c) Shocks actually used by the FSAP teams are generally milder than the current rule of thumb, and closer to our alternative methods. One can think of this as the actual practice being more reasonable than the rule of thumb. However, what worries us is that without an alternative benchmark, any “reasonable” deviation from the rule of thumb has to be made in an entirely case-by-case, ad hoc way. We also note that during and after the period of crisis, FSAP sometimes used severe shocks for stress testing (e.g., EUR SI B, Scenario 3). This prompted the authorities to offer a counter-scenario, which was close to our alternative methods. Another example is APD EM F, which experienced the Asian Crisis in the late 1990s. The FSAP team rejected the authorities’ counter-scenarios, which were very grave but plausible in the authorities’ view, and simply used one third of the GDP shock from the Asian Crisis to capture some flavor of the past tail event. The shocks under our alternative methods are closer to the ones the authorities proposed.

Table 6.

Comparison of Stress Test Scenarios (GDP Shock)

(in percent)

article image
Source: Authors’ estimates.Notes: 1) “Scenario” is the actual shock that the FSAP teams used. “3 years” and “5 years” are shown for some countries because the teams also used longer-term stress scenarios.2) “Rule of thumb” is 2SD shocks, where SD is calculated from 30 years of historical data, applied to the FSAP team’s forecast for real GDP growth for 2 years.3) Sample periods for calculating SDC (Method (i)) and estimating an AR model (Method (ii)) are from 1980 to the year when the FSAP was conducted.

74. Finally, regarding the period over which standard deviations are to be calculated, we have noted that some ad hoc treatment is being made in actual FSAPs. For example, in the case of the above-mentioned APD EM F, including the Asian Crisis episode in the sample makes the estimated SDs unstable depending on the length of sample periods (Figure 17). Therefore, the team did not calculate SDs from historical data. In another case of APD EM not shown in the table above, the team calculated SD over 12 years that do not include the Asian Crisis episode but used a 2.6SD shock, severer than the current rule of thumb. On the other hand, in the case of some of the European Debt Crisis countries (e.g., Europe SI B), the teams used a very long sample period (30 years) for their SD calculation, in order to dilute the impact of the recent major shock. While these compromise solutions make some sense, the fact that different teams are using different solutions is a cause for concern.

Figure 17.
Figure 17.

Comparison of Two Standard Deviations (Asia EM F)

(in percent)

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Authors’ estimates.Note: Standard deviations of “10 years,” “15 years,” and “20 years” are calculated based on the historical data of real GDP growth over two years.

75. In order to extract risks in the financial system, the stress test design should be intrinsically forward-looking. However, given the difficulty to foresee future financial and economic developments, using historical data is inevitable. We should do so in a way that is less affected by the past experience of crises while utilizing the recent information. We believe our alternative “rules of thumb” offer a reasonable uniform benchmark, but do not exclude case-by-case adjustments based on the team’s and the authorities’ forward-looking judgments, if necessary. However, when incorporating such judgments, it is essential that a scenario based on uniform rules is constructed and presented in the FSSA, and any judgmental deviation from it is clearly explained and justified. We do not claim that our methods are the most suitable rules that serve this purpose. However, we believe that their key elements—namely, (a) standardizing the meaning of two standard deviations across all countries using some stochastic forecasting model, and (b) reducing the influence of when an FSAP happens to take place—should be an ingredient of any uniform rules to be introduced.

E. Conclusion

76. In this paper, we analyzed FSAPs that were conducted over the last five years, with an assessment of their “evenhandedness” in mind. Since evenhandedness is difficult to define objectively, we adopted an approach in which we propose several benchmarks, assess FSAPs against them to see if there are any notable patterns across countries, and if we find them, discuss what they mean in the context of equality of treatment among member countries. We did this on three aspects of FSAPs; namely, their use by member countries; BCP assessments; and stress testing.

77. On all the three aspects we analyzed, we found some interesting patterns. Specifically,

  • FSAPs’ use by member countries is evolving, partly because of the Board decision to designate systemically important jurisdictions and make an FSAP mandatory for them every five years. As a result, the average interval between FSAPs is expanding for LICs and non-SI EMs. While giving priority to SIs in FSAP resource allocation was intended, what is the right extent of skewed allocation is debatable, and we wonder if the Board gave enough attention to the flip side of its decision to expand SI designation. We also found a pattern in members’ use of FSAP across geographical areas, with AFR and MCD under-represented and WHD over-represented.

  • BCP assessments were found to be correlated with standard indicators of countries’ technical or institutional strength. This does not mean that these indicators “explain” BCP assessments, but they nevertheless offer themselves to be a potentially useful benchmark. What we found from this benchmarking is that SI EMs appear to be assessed better than might be expected from their relative position with respect to benchmark indicators. There is also some evidence that assessments are influenced by assessor identity, and the difference across assessors can be fairly large.

  • We were surprised that there is at present no common framework by which key macro assumptions that underlie stress testing are determined for each country. There is a “rule of thumb”, but individual teams may deviate significantly from it depending on the circumstances. We therefore proposed two alternative methods for determining macro assumptions that are aimed at standardizing the likelihood of stress events across countries, and compared them with the “rule of thumb” and the actual scenarios used by FSAP teams. We found that the scenarios actually being used are more moderate than the “rule of thumb” and closer to our alternative methods. This was reassuring, but the case-by-case, ad hoc adjustments that are often required for the team to agree with the authorities on actual scenarios makes us wonder if all members are subjected to stress events whose likelihood is broadly the same.

  • We also found that the scope of stress testing differs substantially across countries. As all member countries face a variety and combination of risks and their propagation channels, highly selective application of spillover and other macro scenario analysis does not seem appropriate for FSAP stress testing. Since one reason for selectivity is insufficient data, efforts to fill the data gap are also required, either through an improvement in the coverage and quality of data or the authorities’ better cooperation in data provision within the legal constraints.

78. All of our findings above do not necessarily imply lack of evenhandedness. But we believe that these results suggest room for improvement, or at least, for a further study. In particular, we see the following two areas worth exploring;

  • We stayed away from the substance of BCP assessments, limiting ourselves to the analysis of average scores. Experts could be assigned to look inside of the assessments, in order to examine if any evidence can be found in the assessments’ substance that may corroborate or contradict the seeming bias found from aggregate numbers. Also, since the recently revised BCP introduced macroprudential elements based on the experience of the GFC, examining how they are being assessed would be useful in due course to ensure that this important new aspect is evenly covered in all FSAPs.

  • A more systematic search can be made for simple models to be used by all FSAP teams for constructing stress scenarios. This is not entirely a matter of forecasting accuracy; while a better-performing model is of course more desirable, our emphasis is to enhance evenhandedness by minimizing ad hoc adjustments made for each country. The choice of a model cannot be made without a clear idea about exactly what kind of stress events the Board wants to test. Not just the degree, but also the nature of risks need to be clearly identified in order to select right models.

Appendix. Technical Note on Alternative Methods

The followings are the details of the two alternative methods of rule-of-thumb stress testing.

Alternative Method (i)

Based on the idea that GDP moves back to a stochastic trend, this method assumes that, during the stress period, a 2SD shock of the cyclical component of the historical GDP path hits the trend GDP that is expected to prevail in one to two years. This method does not utilize the information about recent GDP developments.

First, divide the real GDP (Yt) into the trend GDP and the cyclical component. We apply HP filter to real GDP data to estimate trend GDP (Gt). We use the FSAP teams’ GDP forecasts at the time of the FSAP to avoid HP’s end-point problem. The cyclical component of the real GDP, Ct, is derived by subtracting Gt from real GDP.

Ct=YtGt(1)

Second, since Ct’s magnitude tends to expand as the level of GDP rises, we divide this component by the trend GDP, Ct/Gt , and calculate its standard deviation from thirty years of past data.

Then, the shock is calculated by multiplying 2 to this standard deviation, which is subtracted from Gt in two years.

Alternative Method (ii)

Assuming that GDP moves around a stochastic trend, this method forecasts the future GDP by estimating a simple AR model of the cyclical component, and uses the model’s dynamics and error term to construct stress events. It utilizes information contained in the recent GDP developments and the team’s baseline scenario in near future.31

First, divide the real GDP into the trend GDP and the cyclical component, as under Method (i). Then, we estimate the AR (2) model of Ct32 as follows;

Ct=αCt1+βCt2+εt(2)

To calculate 2SD shocks at t+1 and t+2, we estimate the following conditional forecast for Ce and Ye.

Ct+1e=α*Ct+β*Ct1(3)
Ct+2e=α*Ct+1e+β*CtCt+2e=α*Ct+1e+β*Ct(4)
Yt+1e=Gt+1e+Ct+1e(5)
Yt+2e=Gt+2e+CT+2e(6)

where α*,β*are the estimated parameters of α,β.

The stress scenario is derived by subtracting from Ye 2SDs of the error term from the AR (2) model. The standard deviation at t+1 is equal to the standard error of the regression (σ). The standard deviation at t+2 is calculated as follows.

Yt+2=αYt+1+βYt+εt+2=α(αYt+βYt1+εt+1)+βYt+εt+2=Yt+2e+αεt+1+εt+2(7)
Var(Yt+2Yt+2e)=Var(αεt+1+εt+2)=(α2+1)σ2+2αCov(ε,ε1)(8)

Since the Durbin-Watson statistics is close to two in the estimation of each country, we disregard the serial correlation (Cov = 0). Hence, the shock to Y two periods ahead is two times the square root of (α2 + l)σ2, where a is replaced by α* in actual calculation.

Appendix Table 1.

List of FSAPs Covered in this Paper

article image
Appendix Table 2.

List of BCP Assessments Covered in this Paper

article image

Results From the Surveys of National Authorities and IMF Executive Directors33

79. The views of country authorities and IMF Executive Directors on the FSAP program were gathered through two similar surveys. As part of the 2014 FSAP review, surveys were conducted to gather the views of national authorities and Executive Directors on the effect of the measures introduced by the 2009 review, the domestic impact of FSAPs, and on ways to improve the FSAP program going forward. National authorities of the 168 countries, territories, and currency unions that had an FSAP by early 2014 were invited to participate. Responses were received from 55 of them, which yields a response rate of 33 percent, with relatively high completion rates for advanced and emerging countries, and low for low income countries.34 About half of respondents had a second or third FSAP (or stability module) after the 2009 FSAP review, and about a fifth had a first-time FSAP during the same period. Most responses came from central banks and the rest from ministries of finance. A very similar survey was submitted to the 24 IMF Executive Directors, with views in this case based on their experiences with Executive Board discussions of several FSAPs. Eight responses were received, for a 33 percent completion rate.

80. The results of both surveys are broadly similar. They indicate high satisfaction with the FSAP program and broad consensus with the ways in which the program needs to be adjusted to improve its quality and effectiveness. FSAPs are considered very useful and the changes introduced by the 2009 FSAP review (i.e., the definition of main the components of stability assessments, the introduction of the RAM, and the modular approaches) are perceived as having added significant value to assessments and further strengthened the strong points identified in earlier studies. Respondents also pointed to room for improvement in some areas: increased attention to systemic risk and macroprudential issues; strengthened analytical toolkits for stress testing; better coverage of cross-border links and domestic channels of contagion; better tailoring analyses to individual countries; and streamlining FSSAs.

81. Where differences in views between country authorities and Executive Directors exist, they tend to be moderate. The largest differences in views relate to the quality of the analytical treatment of real-financial transmission channels and domestic contagion—with views of Executive Directors being less favorable; reasons for non-implementation of some FSAP recommendations— with country authorities putting more emphasis on disagreement with the recommendations and Executive Directors on recent completion of FSAPs; and on the role of targeted supervisory assessments—where some differences exist in relation to the specific principles that should be assessed.

A. Impact of the 2009 Review

82. FSAPs are adequately structured around the three pillars identified by the 2009 FSAP review (Figure 19). FSAPs are overwhelmingly seen (over 90 percent of country authorities and all Executive Directors) as adequately structured around the three main pillars (vulnerabilities and resilience of the financial sector, oversight policy framework, and safety nets).35 There were similarly high degrees of satisfaction with the treatment of each of the first two pillars. National authorities’ satisfaction with the treatment of safety nets was slightly lower (75 percent) than with the other two pillars, still a remarkably positive result considering that this pillar was not covered systematically prior to 2009. Both groups of respondents showed a somewhat lower, albeit still favorable, level of satisfaction (69 percent for country authorities and 88 percent for Executive Directors) with the way on which FSAPs articulated the main threats to financial stability, an important result in light of the 2014 FSAP Review’s objective of better focusing future FSAPs on systemic risk.

Figure 18.
Figure 18.

Sample Characterization

(Only relevant for the Country Authority Survey)

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Figure 19.
Figure 19.

Post 2009 Review

(Structure and Coverage)

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Survey results.

83. FSAPs addressed the most relevant financial sector issues with sufficient analytical depth, but need to take better account of country-specific institutional features (Figure 20). A large majority of country authorities (76 percent or more) and Executive Directors (88 percent or more) agree that FSAPs covered the most important issues, that FSAPs’ analytical depth was adequate, and that FSAPs discussed policies to mitigate the identified risks. These results show an improvement over the relatively strong level of satisfaction expressed by country authorities in these areas in the 2009 FSAP review. Satisfaction was somewhat lower among country authorities (57 percent) with FSAPs taking into account countries’ unique features, an issued flagged also by country authorities in the 2009 review. However, based on other responses to the survey (see below), it appears that FSAPs are perceived as not paying sufficient attention to countries’ institutional and structural features, rather than to the prevailing macro-financial circumstances, as respondents (76 percent of national authorities and 88 percent of Executive Directors) found FSAPs adequately framed with respect to the latter.

Figure 20.
Figure 20.

Scope

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Survey results.

84. FSAPs’ analytical work on vulnerabilities is seen favorably, but satisfaction with the treatment of feedback effects and contagion is relatively lower (Figure 21). A large majority (over 75 percent) of respondents agreed that the analysis was adequately framed in countries’ prevailing macro-financial conditions, and that the technical work was calibrated to the availability and granularity of the financial sector data. There was somewhat less agreement (71 percent for country authorities and 51 percent for Executive Directors) on whether the models and methodologies used in the stress tests were adequate in light of the sophistication of the financial system. However, approval rates were lower for the analysis of real-financial transmission channels (59 percent for country authorities and 38 percent for Executive Directors), feedback effects (58 percent for country authorities and 25 percent for Executive Directors), and potential contagion among domestic financial institutions (51 percent for country authorities and 38 percent for Executive Directors).

Figure 21.
Figure 21.

Analytical Focus

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

85. Views on the assessment of supervisory standards are positive, but they could be better adapted to country-specific circumstances (Figure 22). About 75 percent of country authorities (Executive Directors were not asked this question) shared the view that these assessments were sufficient to provide a robust evaluation of the regulatory and supervisory framework, that they adequately reflected the main risks threatening the financial system, and that they were well aligned with other parts of the FSAP. A somewhat lower, but still favorable, level of satisfaction (65 percent) was expressed with respect to the assessments being sufficiently tailored to country-specific circumstances.

Figure 22.
Figure 22.

Assessment of Selected Principles from an International Standard

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Survey results.

86. FSAPs can do a better job a explaining the likelihood and impact of shocks described in RAMs (Figure 23). Views on whether RAMs focus on the most relevant risks were positive (67 percent for country authorities and 75 percent for Executive Directors), and only slightly less positive (59 percent for country authorities and 63 percent for Executive Directors) on RAMs helping frame the discussion of risks with national authorities and anchoring the discussion of risks in FSSAs (58 percent for country authorities and 63 percent for Executive Directors). Less favorable views were expressed about RAMs adequately explaining the likelihood of the shocks (53 percent for country authorities and 25 percent for Executive Directors), and about RAMs properly accounting for mitigating policies in the assessment of impact (45 percent for country authorities and 75 percent for Executive Directors).

Figure 23.
Figure 23.

The Risk Assessment Matrix

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Survey results.

87. Views on the modular approach could not be analyzed properly (Figure 24). Although there had been only 5 stability modules at the time the survey of country authorities was launched, there were 38 responses based on experience with them, suggesting that the question may not have been formulated adequately, or misinterpreted. Views of Executive Directors on stability modules were mixed, with 63 percent of them agreeing that they were less demanding on country’s internal resources and 38 percent agreeing that stability modules allowed for better selection of country-specific themes.

Figure 24.
Figure 24.

The Stability Module

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Survey results.

88. FSAP recommendations are candid and clear, but could be prioritized and sequenced better (Figure 25). A very large majority of country authorities (over 80 percent) and Executive Directors (75 to 100 percent) agreed that the main FSAP recommendations summarized in the first table of Aide Mémoires (for country authorities only) and FSSAs were clear and candid. The numbers were somewhat lower, but still at or above 50 percent in both groups, on whether recommendations were actionable with a reasonable effort from national authorities, and whether they were adequately prioritized and sequenced.

Figure 25.
Figure 25.

Recommendations of Assessment

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Survey results.

B. Traction and Domestic Impact

89. The rate of implementation of FSAP recommendations is relatively high (Figure 26 and Figure 27). About a third of all respondents indicated having implemented FSAP recommendations in full, and slightly over 60 percent of them reported having implemented FSAP recommendations partially. Responses by country authorities and Executive Directors were almost identical. However, views differed on the main reason for non- or partial implementation, with nearly half of country authorities choosing disagreement with the recommendations and 60 percent of Executive Directors pointing to the recent completion of the FSAP (country authorities’ second choice). Executive Directors also pointed to limited capacity to implement recommendations (40 percent, 14 percent of country authorities) and lack of financial resources as important reasons, while national authorities singled out limited political support and recommendations not representing priorities for the government.

Figure 26.
Figure 26.

Implementation of Recommendations

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Survey results.
Figure 27.
Figure 27.

Motivations for Not Fully Implementing Recommendations

Citation: Policy Papers 2014, 067; 10.5089/9781498342827.007.A001

Source: Survey results.

90. Views of country authorities and Executive Directors on the main contributions of FSAPs to financial analysis were very similar (Table 7). The surveys asked to rank 10 areas (9 for Executive Directors) and the results of the rankings were very similar with one exception. Among the top contributions, both groups of respondents indicated that FSAPs provided adequate assessments of financial system stability (and identification of main risks), the regulatory and supervisory framework, macrofinancial linkages, and crisis management capabilities. The main difference in views related to FSAPs providing an integrated financial sector analysis, which was ranked second by country authorities but eight by Executive Directors. There was agreement on the areas where FSAP contributions rank lower, including outward spillovers, macroprudential frameworks and tools, and developmental assessments. These results are similar to those of the 2009 review.

Table 7.

Most Valuable FSAP Contribution

article image
Note: Respondents were asked to rank from most (one) to least (ten) valuable the contribution of each of the following FSAP areas to their country. The topic ranking reflects the highest-to-lowest, on average, score (i.e., the lowest mean reflects the highest score).