Review of the Financial Sector Assessment Program-Further Adaptation to the Post-Crisis Era

The Financial Sector Assessment Program (FSAP), established in 1999, is an in-depth assessment of a country’s financial sector. It is an important element of the Fund’s surveillance and provides input to the Article IV consultations. In developing and emerging market countries, FSAP assessments are usually conducted jointly with the World Bank and include two components: a financial stability assessment (the main responsibility of the Fund) and a financial development assessment (the main responsibility of the World Bank). Each FSAP concludes with the preparation of a Financial System Stability Assessment (FSSA), which focuses on issues of relevance to IMF surveillance and is discussed by the IMF Executive Board normally together with the country’s Article IV staff report. Since the program’s inception, 144 member countries have requested and undergone FSAPs, most of them more than once. In recent years, the Fund has been conducting 14–16 FSAPs per year at an annual cost of US$13–15 million. The last review of the FSAP in 2009, in the aftermath of the global financial crisis, introduced a number of far-reaching reforms that have clarified the responsibilities of the Fund and the Bank in developing and emerging market countries, where assessments usually take place jointly, established institutional accountability, strengthened the analytical focus and coverage of FSAPs, and introduced the option of modular assessments that has afforded the Fund and national authorities greater flexibility on the scope and timing of assessments. In 2010, the financial stability assessment under the FSAP in 25 jurisdictions with financial sectors deemed by the Fund to be systemically important became a mandatory part of Article IV surveillance, expected to take place every five years. The list was expanded to 29 jurisdictions in 2013. For all other jurisdictions, FSAP participation continues to be voluntary.In 2010, the financial stability assessment under the FSAP in 25 jurisdictions with financial sectors deemed by the Fund to be systemically important became a mandatory part of Article IV surveillance, expected to take place every five years. The list was expanded to 29 jurisdictions in 2013. For all other jurisdictions, FSAP participation continues to be voluntary.

Abstract

The Financial Sector Assessment Program (FSAP), established in 1999, is an in-depth assessment of a country’s financial sector. It is an important element of the Fund’s surveillance and provides input to the Article IV consultations. In developing and emerging market countries, FSAP assessments are usually conducted jointly with the World Bank and include two components: a financial stability assessment (the main responsibility of the Fund) and a financial development assessment (the main responsibility of the World Bank). Each FSAP concludes with the preparation of a Financial System Stability Assessment (FSSA), which focuses on issues of relevance to IMF surveillance and is discussed by the IMF Executive Board normally together with the country’s Article IV staff report. Since the program’s inception, 144 member countries have requested and undergone FSAPs, most of them more than once. In recent years, the Fund has been conducting 14–16 FSAPs per year at an annual cost of US$13–15 million. The last review of the FSAP in 2009, in the aftermath of the global financial crisis, introduced a number of far-reaching reforms that have clarified the responsibilities of the Fund and the Bank in developing and emerging market countries, where assessments usually take place jointly, established institutional accountability, strengthened the analytical focus and coverage of FSAPs, and introduced the option of modular assessments that has afforded the Fund and national authorities greater flexibility on the scope and timing of assessments. In 2010, the financial stability assessment under the FSAP in 25 jurisdictions with financial sectors deemed by the Fund to be systemically important became a mandatory part of Article IV surveillance, expected to take place every five years. The list was expanded to 29 jurisdictions in 2013. For all other jurisdictions, FSAP participation continues to be voluntary.In 2010, the financial stability assessment under the FSAP in 25 jurisdictions with financial sectors deemed by the Fund to be systemically important became a mandatory part of Article IV surveillance, expected to take place every five years. The list was expanded to 29 jurisdictions in 2013. For all other jurisdictions, FSAP participation continues to be voluntary.

Executive Summary

In the five years since the last FSAP review the program has been transformed, largely in response to the global financial crisis. The 2009 review introduced the most fundamental changes to the program since its inception, aimed at incorporating the emerging lessons from the crisis, increasing the flexibility and effectiveness of the FSAP and, by delineating the main responsibilities of the Fund and the World Bank in developing and emerging market countries, where FSAPs are usually conducted jointly, strengthening institutional accountability. The landmark 2010 decision by the Fund to make financial stability assessments under the FSAP a mandatory part of Article IV surveillance for jurisdictions with systemically important financial sectors was a major step toward integrating the FSAP with Fund surveillance for the most systemic jurisdictions and ensuring a risk-based allocation of FSAP resources globally; but implementing this decision in a resource-constrained environment had implications for the availability of FSAPs to non-systemic countries.

These changes have strengthened the focus, effectiveness, and traction of the FSAP. The 2009 review provided a framework for the financial stability assessment under the FSAP—the Fund’s main responsibility in the context of the program—which has proved effective in disciplining and focusing the assessments. The introduction of the Risk Assessment Matrix (RAM); expansion of stress tests to cover a broader set of risks; increasing—albeit still fledgling—analysis of spillovers; and systematic coverage of macroprudential frameworks and financial safety nets were major improvements. This is corroborated by the results of the surveys undertaken in the context of this review: authorities of countries that underwent FSAPs indicate a high degree of satisfaction with the focus and relevance of the assessment, the clarity and candor of the recommendations, and the overall usefulness and impact of the exercise. FSAPs are seen as making a useful contribution to the policy debate, and there is evidence that they have a market impact. FSAP recommendations have a high rate of implementation. And FSSA reports are published at an increasingly high rate, and are making inroads into broader audiences.

At the same time, the experience of the last five years provides useful lessons to help improve the program. First, success of the FSAP depends on the cooperation of all counterparts in the country, notably policy-makers and supervisors, as well as on the availability of high-quality data that go beyond the requirements of regular surveillance. Second, recent experience underscores that even where such data are made available, FSAP teams are not normally in a position to assess their accuracy or the quality of the underlying assets, which means that the limits to their risk assessment in FSAPs need to be clearly understood and communicated. Third, the analysis of financial interconnectedness and spillovers—issues of increasing importance—remains a challenge, partly dependent on data availability. Fourth, a balance needs to be struck between analyzing risks in a comparable and evenhanded manner on one hand, and the need to tailor the tools and scenarios to the individual circumstances of the country in question on the other. Fifth, it will be important to explore options for resolving the tension between the comprehensive nature of formal assessments of compliance with international standards and the FSAP’s focus on systemic risk.

Going forward, this paper provides a basis for consolidating and extending the gains made by the FSAP in the past five years. This does not require radical reforms but relatively straightforward steps toward (i) continuing to strengthen the systemic risk focus of all components of the FSAP; (ii) maintaining a cutting-edge analytical toolkit for the analysis of vulnerabilities and resilience while, at the same time, being realistic and transparent about the limits of this analysis; and (iii) improving the clarity and quality of the FSSA to ensure continuing traction with external audiences and a better input into Article IV surveillance. Concrete steps are summarized in the Box.

These steps will contribute to the broader agenda laid out in the 2014 TSR to enhance the coverage and integration of macrofinancial issues in surveillance. The 2014 TSR offers proposals for enhancing financial surveillance by Article IV teams. The FSAP’s low frequency and narrower focus on financial stability and risks means that it should be seen as a complement rather than a substitute for regular macrofinancial surveillance under the Article IV, and the steps laid out in this paper will help maximize the integration of these two instruments.

Managing the tradeoffs imposed by resource constraints will continue to be a challenge. Most of these steps should have modest resource implications. In a broader context, however, an unchanged resource envelope means continuing tension between the focus of the program on systemically important countries and the needs of non-systemic countries, especially LICs. For the latter, stepped up technical assistance, targeted to financial stability and closely integrated with surveillance, will help bridge the gap.

2014 FSAP review: the agenda going forward

  • Continue to shift the focus of all components of the financial stability assessment towards systemic risk.

  • Upgrade the analytical underpinnings, transparency, and evenhandedness of the assessment of vulnerabilities and resilience through:

    • Expansion of the coverage of stress testing tools to non-banking financial sectors.

    • Detailed guidance on scenario design and dissemination of standardized quantitative tools.

    • Deeper analytical treatment of interconnectedness and better integration with stress tests.

    • More systematic analysis of cross-border exposure and spillovers, based on data availability.

    • Transparent and detailed disclosure of the limitations of risk assessments.

  • Explore a macrofinancial approach to supervisory standards assessments to enable the assessment effort to focus on the most relevant principles for financial stability and integrate the findings into the FSAP’s overall risk assessment.

  • Ensure greater focus on and more systematic treatment of macroprudential policy issues in all FSAPs.

  • Improve traction and maximize input to Article IV consultations by using macrofinancial relevance as the organizing principle for steamlining and prioritizing FSAP findings and recommendations, and structuring FSSAs so as to feed more directly into the surveillance priorities in subsequent years.

  • Step up targeted Technical Assistance on financial stability issues and, where appropriate, use the World Bank’s development modules to mitigate the impact of the global systemic focus of the FSAP on LICs.

Introduction

1. The global financial crisis triggered major changes to the FSAP program. The crisis underscored the central importance of financial sector issues for domestic and global stability, prompting a careful examination of how these issues were covered in Fund surveillance, including FSAPs. Although at that point the lessons from the crisis were just emerging, the 2009 FSAP review recognized that assessments would henceforth be conducted in a very different environment than when the program had originally been framed a decade earlier. It therefore introduced the most fundamental changes to the program since its inception, aimed at incorporating the emerging lessons from the crisis and at increasing the flexibility and effectiveness of the FSAP. By delineating the main responsibilities of the Fund and the World Bank in FSAP assessments, the 2009 review also established clear institutional accountability for FSAPs in developing and emerging market countries, where they are usually conducted jointly. These changes, and further adaptations to reflect the changing global environment and evolving financial reform agenda, have transformed the way FSAPs are conducted.

2. The landmark 2010 decision by the Fund to make financial stability assessments under the FSAP a mandatory part of Article IV surveillance for jurisdictions with systemically important financial sectors also had a major impact. The 2010 decision built on the new framework established by the 2009 FSAP review, notably the definition of the content of the financial stability assessment in the context of the FSAP and the assignment of responsibility for this assessment to the Fund. It was a major step toward integrating the FSAP with Fund surveillance for the most systemic jurisdictions,1 and ensured the allocation of FSAP resources in a globally risk-based manner, consistent with the Fund’s goal of promoting global financial stability. At the same time, implementing this decision in a resource-constrained environment had implications for the availability of FSAPs to non-systemic countries.

3. Time to take stock. This paper assesses the impact of the 2009 review and subsequent changes on the program. Given the depth and breadth of those changes, the paper focuses on evaluating their impact and deriving lessons to make the program more effective.

  • The first part evaluates the implementation of the reforms of the 2009 review and explores two other questions: the traction and broader impact of FSAPs; and the role of FSAPs in surveillance today, especially in light of the 2010 decision.

  • The second part of the paper discusses ways to incorporate the lessons learned during the last five years in order to strengthen the FSAP program further.

4. The stock-taking draws on a number of different sources. These include an internal staff review of all FSAPs conducted since 2009; staff analytical background notes, summarized in this paper (main text or appendices) or in the accompanying set of background studies; surveys of country authorities and Executive Directors; and a special study by independent external consultants2 on the question of evenhandedness of FSAPs, referred to in the relevant parts of this paper and included in its entirety in the accompanying background document.

5. This paper focuses mainly on the Fund’s role and responsibilities in the FSAP. Consistent with the Fund’s mandate for macrofinancial surveillance and with the framework introduced by the 2009 review, and in close collaboration with World Bank staff, it was decided to limit the scope of this paper to the experience with financial stability assessments in the context of the FSAP. A separate paper by the World Bank focuses on developmental issues in FSAPs in developing and emerging market countries. This approach is appropriate given the different mandates, areas of focus, and priorities of the Fund and the Bank. Nonetheless, staff of the two institutions have worked closely to ensure consistency of the two papers, and certain joint aspects of the program, notably the coordinating role of the inter-institutional Financial Sector Liaison Committee (FSLC), are covered equally in both papers.

6. This paper should be read in conjunction with the 2014 Triennial Surveillance Review. The coincidence of the Triennial Surveillance Review (TSR) with the quinquennial FSAP review this year provides an opportunity to maximize synergies and avoid duplication. The TSR examines all aspects of surveillance and, in that context, outlines an agenda to enhance the coverage and integration of macrofinancial issues in Article IV consultations. This paper focuses on the FSAP program itself, which has a narrower focus, and discusses how the FSAP assessments, and in particular the Financial System Stability Assessment (FSSA) documents, can be adapted to maximize their usefulness for the Article IV process and thus contribute to this agenda.

The Impact of the 2009 Review

7. The 2009 review introduced changes aimed at improving four aspects of the program.

  • The focus and analytical depth of FSAPs—specifically, of the Fund’s main responsibility in the context of FSAPs—was to be sharpened by the definition of the three components of the stability assessment (vulnerabilities and resilience; quality of the financial stability policy framework; and financial safety nets), and the introduction of the Risk Assessment Matrix (RAM)3 and a prioritized Table of Key Recommendations.

  • Accountability was to be strengthened with the definition of the stability and development components of FSAPs and the assignment of institutional responsibilities for each in the case of joint FSAPs to the Fund and the World Bank, respectively.

  • Flexibility—for both country authorities and the two institutions—would be enhanced by the introduction of the option of separate stability and development modules and of targeted standards assessments.

  • Finally, these changes combined were also expected to facilitate the closer integration of FSAPs with Article IV surveillance.

8. In addition, the 2009 review acknowleged the continuing need for FSAPs to incorporate the lessons from the global financial crisis, which would crystallize only gradually over time. Four areas were explicitly identified in this regard: (i) the assessment of macroprudential risks and policies; (ii) the evaluation of crisis preparedness and crisis management frameworks; (iii) the analysis of cross–border linkages and spillovers; and (iv) the incorporation of the emerging global regulatory and supervisory reform agenda in the assessments.

9. So how did we do? The remainder of this section provides a critical evaluation of progress made following the 2009 review in improving focus and analytical depth—including incorporating the lessons from the financial crisis—and of the impact of the reforms aimed at enhancing flexibility and accountability. Issues related to the traction and impact of FSAPs, as well as to the role of FSAPs in surveillance, which go beyond the 2009 review, are discussed in the following two sections.

A. Focus and Analytical Depth

10. The 2009 review defined the three components of stability assessments under the FSAP. These are (i) the identification of key vulnerabilities and assessment of the resilience of the financial sector; (ii) the assessment of the financial stability policy framework, including the micro- and macroprudential regulatory framework and supervision in practice; and (iii) the evaluation of financial safety nets, which reflect the system’s capacity to manage and resolve a financial crisis. Up to that point, the first two topics had always been covered in FSAPs, while coverage of the third one had been less even. Elevating the prominence of financial safety nets was one of the ways in which the 2009 review incorporated the lessons of the global financial crisis.

11. This framework has worked well. Since 2009, FSAPs systematically covered these three core areas, as established by a staff review of FSSAs and confirmed by the survey of national authorities: over 90 percent of respondents agreed that FSSAs were adequately structured around these three core areas and covered most relevant financial sector issues (Figure 1). Moreover, this three-pillar construct has provided a sound yet flexible framework for the assessment: disciplined enough to cover all key areas (over 80 percent of survey respondents agreed that the coverage and depth of analysis were adequate, a higher level of satisfaction than in the 2009 survey) and flexible enough to accommodate new elements, like macroprudential frameworks. The rest of this section delves in more detail into the experience with each of the components of the framework since 2009.

Figure 1.
Figure 1.

Survey Results: Focus and Analytical Depth

Approval Rates (Strongly Agree+ Agree)

Citation: Policy Papers 2014, 066; 10.5089/9781498342841.007.A001

Source: Survey results.Note: Results from the 2009 authorities’ survey shown only for the questions that had been included in that survey.

Assessment of vulnerabilities and financial system resilience

12. The introduction of the Risk Assessment Matrix (RAM)—one of the key innovations of the 2009 review—was a major step toward better and more candid risk assessments, although experience suggests some areas for improvement. The requirement to include RAMs in all FSAPs has provided a valuable disciplining device for FSAP teams to think through plausible risks—probability and possible impact—and prioritize them. Country-specific RAMs—informed by the Global RAM—have helped teams be more candid and provided a useful vehicle for cooperation between FSAP and Article IV teams and engagement with country authorities. Last but not least, they have focused stress tests on assessing the impact of the most significant macrofinancial risks facing the country, rather than the mechanistic “scattergun” approach sometimes taken in FSAPs pre-2009. However, experience has shown that the quality of RAMs varies and their integration with other components of the FSAP beyond stress testing is uneven. Respondents to the surveys of country authorities and Executive Directors expressed a high degree of satisfaction with RAMs’ focus on the most relevant risks but somewhat lower satisfaction with their ability to trace the relevant propagation channels of such risks and to incorporate the role of mitigating policies in the estimation of the impact of shocks (see Figure 1 and detailed survey results in the accompanying background document). And RAMs are not used to guide the focus of the evaluation of supervisory frameworks or financial safety nets, as these continue to be exhaustive rather than targeted to the macrofinancial risks facing the country.

13. Stress tests in FSAPs now cover a wider range of sectors and risks, the findings are better integrated into the overall assessment of vulnerabilities, but challenges remain in a number of areas. Stress tests always cover banking, but in recent FSAPs stress tests have been applied to other sectors, notably insurance. Stress test scenarios almost always cover the key macrofinancial risks identified in RAMs (which, in turn, reflect those in the Global RAM, as relevant), with additional scenarios or single-factor shocks included as necessary. Prior to 2009, stress tests in FSAP tended to focus mainly on solvency risk. This has given way to broader risk coverage that now almost always includes liquidity, contagion risk and, to a lesser extent, spillover analysis—including through the assessment of financial cross-border exposures. The use of a more comprehensive set of stress tests allows for a better quantification of the impact of the macrofinancial risks identified in the RAM and provides a more solid basis for recommendations. At the same time, there are still important gaps in the stress testing toolkit—in the Fund and internationally—notably in incorporating second-round effects and nonlinearities.

14. The reliability of stress test results depends directly on the quality and provision of data by the authorities. Stress tests require data that in many cases are not publicly available, such as granular bank-by-bank supervisory information (loan default probabilities, loss given default, collateral) and inter-institutional exposures. These data go beyond the requirements of regular surveillance. In most cases, country authorities provide these data on a voluntary basis to the FSAP team, subject to strict confidentiality protocols. When this does not happen, reflecting legal constraints or other reasons, stress tests have to rely on much less granular publicly available data, and the reach and depth of the analysis is correspondingly lower.

15. Fund staff has been in the forefront of developing and disseminating new stress testing tools. Several new stress testing models have been created in-house in recent years with varying levels of sophistication, which can be adapted to country-specific circumstances (including data availability and the complexity of the financial system).4 These models are regularly discussed by experts internally, which facilitates further consistency in the use of techniques by FSAP teams, as well as with the global stress testing community, including through the regular Expert Forum on Advanced Stress Testing Techniques.5 Beyond tools, in 2012 staff also put forward for the first time general principles for the design and implementation of stress tests, based on a review of the cumulative experience of stress testing in FSAPs, as well as a survey of practices among central banks and supervisory authorities. MCM staff also deliver extensive Technical Assistance on the use of stress testing tools to national authorities, as well as staff training on stress testing internally through the MCM Toolkit Seminar Series and ICD courses and seminars.

16. Cross-country comparability of stress testing approaches in FSAPs presents difficult tradeoffs. On one hand, there is the understandable desire—among both staff and national authorities—to ensure comparability and an even-handed approach to stress test design and scenario selection across FSAPs. On the other, stress testing approaches in FSAPs need to be flexible and adaptable to country circumstances (data availability, idiosyncratic nature of shocks, and the capacity and sophistication of the authorities’ own stress testing frameworks). For example:

  • Advanced versus developing economies. In advanced countries, stress tests typically incorporate the shocks identified in the RAM in fully-fledged macrofinancial scenarios through macro-econometric models, and are often implemented in cooperation with the national supervisory authorities, leveraging each side’s comparative advantages. This is rarely seen in less developed countries, where lack of reliable data or long time series often limit the stress tests in FSAPs to single-factor shocks.

  • Defining “extreme but plausible” shocks. In recent years, FSAPs relied on a simple rule of thumb calling for a two-standard deviation shock from the long-term growth trend. But strict application of this rule was not appropriate in cases where large deviations from potential output had taken place recently—for instance, in recent assessments of crisis-hit European countries. This has limited the comparability of the scenarios used in different countries.6

17. One size does not fit all. Instead of trying to impose a straightjacket, staff has accepted the inherent limitations in cross-country comparability and opted for ensuring transparency in models and assumptions used in FSAP stress tests through the detailed Stress Testing Matrix (STeM), included in all FSSAs since 2011.

18. FSAPs are increasingly analyzing interconnectedness among financial institutions, but the findings are not always integrated with the rest of the stability analysis. Typically, the focus is on the domestic interbank market, although the analysis sometimes covers also cross-border exposures. The results are normally incorporated in the narrative of vulnerability assessments rather than formally integrated with those of stress tests. In addition, the findings on interconnectedness are not always used to assess the systemic importance of financial institutions or lead to specific policy recommendations (see section on macroprudential policies). Survey results (Figure 1) also suggest that this is an area for improvement.

19. FSAPs have made strides in covering spillovers, but the analytical depth and data availability vary. This fact—also reflected in survey results—is due partly to the lack of a standard, integrated macrofinancial model. When covered, spillovers are typically treated in one or more of the following ways: (i) through a narrative that explains the relevant channels of transmission; (ii) through the quantification of cross-border exposures, including banks’ foreign credit exposures (for outward spillovers) and foreign bank borrowing (for inward spillovers); and (iii) through a more elaborate assessment of contagion from the domestic (foreign) banking sector to foreign (domestic) banking systems. Virtually all FSAPs include a narrative about spillovers, and many quantify direct inward cross-border exposures; but only a handful include a quantitative treatment of outward spillovers7 (recent examples include the 2011 Luxembourg, the 2013 Austria and Italy, and the 2014 Switzerland FSAPs).

20. Despite this considerable deepening of their analytical depth, there are limitations to the vulnerability assessment in FSAPs. It has always been understood that FSAPs do not cover all sources of financial sector risk (e.g., operational risk or the risk of fraud). But even for risks that the FSAPs do cover, there are limitations that are not always appreciated. These are inherent to the FSAP process—and more broadly to the nature of Fund surveillance—and stem from a variety of factors. The most important of these are (i) the varying access of FSAP teams to supervisory bank-by-bank data; (ii) the inability of FSAPs to assess the underlying quality of bank assets and reported components of capital (even when FSAP teams have full access to supervisory data); and (iii) model risk. Box 1 explains these limitations in detail.

The Limits of Risk Assessments in FSAPs1

The standard disclaimer in FSSAs is not an adequate summary of the limitations of the FSAP risk assessment exercise. It has long been recognized that FSAP assessments do not cover all potential sources of risk to financial stability. All FSSAs include on the cover page the following disclaimer:

FSAPs assess the stability of the financial system as a whole and not that of individual institutions. They are intended to help countries identify key sources of systemic risk in the financial sector and implement policies to enhance its resilience to shocks and contagion. Certain categories of risk affecting financial institutions, such as operational or legal risk, or risk related to fraud, are not covered in FSAPs.

This disclaimer is explicit about three types of risk that are not covered in FSAPs: fraud, legal, and operational risk. Although a good supervisor should be able to minimize these risks, and FSAPs do evaluate the strength of supervision, this is not equivalent to a proper assessment of these risks by staff: the disclaimer is thus appropriate and necessary. Experience has shown, however, that there are other important sources of risk that, for various reasons, are not covered in FSAPs which, nonetheless, are not included in the disclaimer.

  • Data gaps. As was pointed by the G-20, serious data gaps affect the ability of supervisors to conduct reliable risk monitoring and financial stability assessment. For example, the international interbank market has not been mapped: it covers only partially some jurisdictions. The lack of international intra-group data for global financial institutions is another important data gap that impairs the assessment of cross-border contagion and spillovers. Beyond this type of gaps, whose existence is recognized and well-understood, there are also data gaps at the individual country level, which may severely impair the capacity of the FSAP team to assess risks and vulnerabilities. These gaps are not always explicitly indicated in FSSAs.

  • Access to supervisory data. Access to bank-by-bank supervisory data may be provided by country authorities to FSAP teams on a voluntary basis and under a strict protocol that protects the confidentiality of the information vis-à-vis the Board and even members of staff outside the FSAP team. While such access is provided to the team in many FSAPs, this practice is not universal: in several cases, especially in advanced economies, the authorities do not share non-public institution-specific data with FSAP teams citing legal constraints or other considerations. This means that the risk analysis and stress testing done in the FSAP have to rely on publicly available information and/or the authorities’ own stress tests, limiting the ability of staff to come to an independent assessment of risks and systemic resilience. Although they have a bearing on the staff assessment, differences in data-sharing across FSAPs are not always flagged in FSSAs.

  • Asset quality. Even in cases where institution-specific supervisory data are shared with staff, FSAP teams are—in most cases—not in a position to assess the accuracy of this information (e.g., whether the loan classification reported by banks reflects the true economic value of the loans; or if various components of capital are properly classified). In a few cases where supervisory forbearance could be documented and its impact estimated, FSAP teams made adjustments to the reported data; but these cases are the exception. In most FSAPs, these data are taken at face value. However, the asset quality review undertaken by the ECB and national EU supervisors underscores the fact that there can be considerable deviations between country practices and “best” practice guidelines. This source of risk is rarely discussed in FSSAs.

  • Model risk. Stress tests are a key tool for the assessment of resilience in FSAPs. However, as indicated in the 2012 paper on Macrofinancial Stress Tests: Principles and Practices, forecasting the impact of tail risks is subject to considerable model risk, reflecting inter alia non-linearities due to risk factor dependence, spillovers, and feed-back effects. More broadly, stress test results are conditional on the simulated shock. But regardless of how extensive the coverage of risk factors, how refined the analytical models, and how severe the shocks, there is always the risk that the “unthinkable” will materialize.

1 Prepared by Liliana Schumacher.

Financial stability policy framework

21. Coverage of financial sector oversight is extensive in FSAPs, with increasing attention to systemic oversight and macroprudential policies. The analysis of financial sector oversight in FSAPs has continued to follow a sector-by-sector approach (banking, insurance, securities). But since 2009, the coverage of regulatory and supervisory issues relevant to the analysis of systemic risk and macroprudential policies has improved significantly. Today, FSSAs almost always include a discussion of mechanisms for monitoring systemic risk, ensuring interagency cooperation and, in countries with significant cross-border banking activity, the cooperation among supervisors in different jurisdictions, although there is still room for improvement (Box 2).

Coverage of Macroprudential Policy (MaPP) Issues in FSAPs1

The 2009 review acknowledged the incipient nature of the work on MaPP and identified four areas of particular interest for FSAPs: systemic risk monitoring tools; the role of systemically important financial institutions; gaps in systemic risk monitoring; and institutional responsibilities of central banks and other supervisory agencies. However, the 2009 FSAP review did not provide a blueprint for the treatment of these issues, as the Fund’s position in these areas would emerge only gradually in the following years (IMF 2011, 2013a, 2013b).

FSAPs have assessed MaPP frameworks regularly since 2010, but the depth and breadth of coverage differed across countries and evolved over time. Overall, the FSSAs for advanced economies and more advanced emerging market economies had more comprehensive coverage. More recent FSAPs had more systematic treatment that were better integrated into the overall stability analysis. The scope of coverage, as well as their integration to the financial stability analysis, has by and large evolved along with the development of policy guidelines on MaPP within and outside the institution.

  • Assessments of systemic risk have become a standard feature of FSAPs, a marked departure from earlier stability assessments that focused on sectoral risks. However, the scope of the analysis largely reflects data availability and many FSAPs highlight data deficiencies as a key obstacle. Some FSAPs recommend collection of more granular data on bank exposures, while others emphasize development of forward–looking frameworks for assessing systemic risk through enhanced group-wide monitoring of financial conglomerates and closer monitoring of exposures to parents or subsidiaries of domestic banks.

  • Almost all recent FSAPs cover the institutional setup for MaPP, which in many countries was in the process of being set up or, in some cases, absent altogether. FSAP recommendations in this area varied, reflecting the different institutional history and traditions; but all FSAPs typically stressed the need for an active central bank role in systemic risk monitoring and the importance of establishing clear goals, mandates, accountability mechanisms, and powers necessary to implement MaPP effectively.

  • The discussion of tools was extensive but uneven, with most FSAPs focusing on cyclical developments (rapid credit growth) more frequently than on structural issues (interconnectedness and complexity). Some FSAPs recommended introducing or tightening of LTV or DTI limits or setting up countercyclical capital buffers, while others highlighted risks associated with growing connectivity and complexity of financial institutions, more intensive supervision.

  • Discussion of multilateral aspects of MaPP was less frequent in FSAPs. Home-host supervisory coordination, including the treatment of systemic subsidiaries and branches in the host country, was discussed in some advanced and emerging market countries. Only a few FSAPs discussed cross-border spillovers associated with uncoordinated actions abroad and/or inaction domestically.

The treatment of MaPP issues in FSAPs can be strengthened in various dimensions: As the knowledge and understanding of MaPP frameworks deepen, the coverage of MaPP analysis should become more comprehensive and systematic. Dissemination of best practices from country experiences, together with the framework developed in the Key Aspects paper should provide a more systematic treatment. Integration of MaPP analysis into the stability analysis requires greater efforts to reflect the key messages in the FSAP recommendations, and cross-referencing the analysis throughout the FSSA. Adequately connecting the dots will ensure that FSAPs provide an integrated assessment of the key risks faced by the financial sector.

Systematic treatment of MaPP issues in FSAPs will also help gradually promote greater international consistency and coordination. FSAPs provide a vehicle for the discussion of MaPP frameworks both with individual country authorities and as an input into in a multilateral context, leveraging the Fund’s unique perspective and institutional position. Over time, this may help engender a more consistent approach to macroprudential policies globally.

1 Prepared by Inci Otker-Robe and Ben Huston.

22. There are different tools for the evaluation of financial sector oversight in FSAPs.

  • This evaluation can take place through a formal assessment of compliance with international standards, summarized in a Detailed Assessment Report (DAR) and a shorter Report on Observance of Standards and Codes (ROSC) for each standard assessed, issued to the Board alongside the FSSA. This is a thorough and time- and resource-intensive process for both the authorities and the FSAP team. Like the standards themselves, the methodology for these assessments is prescribed by the standard-setters. Formal compliance assessments are voluntary for the country (even in cases where the financial stability assessment under the FSAP is mandatory).

  • In cases where a full compliance assessment is not necessary (or not requested by the authorities), the evaluation of oversight takes place through a more focused, targeted assessment of the regulatory and supervisory framework. Although in practice these assessments are guided by the same methodology as the formal assessments, they are more limited, concentrating on the areas that are most relevant, and do not generate a DAR with assessment grades but are summarized in an FSAP Technical Note.8

23. Prior to the global financial crisis, most evaluations of financial sector oversight in FSAPs took place through the latter, more informal process. The average number of formal assessments of all financial sector standards in FSAPs up to 2008 was less than one (Table 1).9 Following the financial crisis and widespread revisions in international standards by standard-setters, as well as in response to various FSB initiatives (Box 3), the average number of ROSCs per FSAP tripled to two-and-a-half, peaking in FY12 before starting to decline again, as demand started returning to more normal levels. This had a major cost impact on FSAPs. The three supervisory standards for banking (Basel Core Principles—BCP), insurance (Insurance Core Principles—ICP), and securities (Principles on Securities Regulation—IOSCO) are by far the most frequently assessed; other (non-supervisory) standards are rarely assessed formally. And within the three supervisory standards, the BCP is the most prominent (Figure 2).

Table 1.

Average Number of ROSCs Conducted During FSAP Missions by Standard

article image
Source: FSAP Tracking System.Note: Figures exclude stand-alone ROSCs and—prior to 2008—OFC assessments. CPSS and CPSS/IOSCO are now part of a new standard on Financial Market Infrastructures (FMI).

The Financial Stability Board (FSB) and FSAP Assessments

The FSB and the G-20 processes have helped cement the role of the FSAP and standards assessments. Following the 2008 G-20 commitment to undertake FSAP assessments, in 2010, FSB members committed to undergoing FSAP assessments every five years and to publishing the detailed assessment of supervisory standards. The FSB also conducts peer reviews of its members 2-3 years after each FSAP assessment, including assessing members’ progress in addressing FSAP recommendations.

Cooperation between the FSB and the IMF has been effective, and has helped ensure a more continuous review of FSB members. Through their membership in the FSB, the Fund is able to coordinate the timing of the peer reviews with the FSAP schedule and help define the thematic focus of the FSB’s work. Just as the FSAP assessments provide a foundation for the peer reviews, peer reviews have also provided valuable input to Fund surveillance and subsequent FSAP assessments (e.g., the 2014 Article IV report for Germany and the 2014 FSAP assessment for South Africa).

The FSB initiative to promote information-sharing and international cooperation has increased demand for standards assessments. This initiative has defined a subset of principles from the three key supervisory standards (BCP, IOSCO, IAIS) deemed by the FSB to be relevant for international cooperation and information-sharing, and is actively promoting compliance with these principles among FSB members and non-members alike. The FSB initially defined a list of some 60 jurisdictions (including FSB members) with large financial sectors and asked them to submit to assessments of their compliance with the subset of international standards; this list is being gradually expanded. Jurisdictions whose compliance—based exclusively on the assessment “grades”—falls short are required to engage in a dialogue with the FSB to identify remedial measures. Jurisdictions not showing sufficient progress or refusing to participate are labeled “non compliant.” Although the Fund (and the World Bank) do not participate in this process, given its coercive nature, the effect has been to increase demand for FSAP and standards assessments by the two institutions.

Although the FSB/G-20 processes have helped boost the profile and impact of the FSAP, they pose challenges. The first is the danger of “assessment fatigue”—the volume of reviews by the FSB and the associated SSBs, as well as FSAP assessments of G-20 members, have increased significantly at a time when regulatory authorities are also in the midst of significant reform. The capacity and willingness of these authorities to engage, and subsequently respond, to these assessments may be strained as a result. Second, this increased volume also requires the assessing institutions to coordinate carefully to avoid conflicting messages or unnecessary duplication of effort.

Figure 2.
Figure 2.

Recent Trends in Standards Assessments

Citation: Policy Papers 2014, 066; 10.5089/9781498342841.007.A001

Source: Staff calculations.

24. Formal standards assessments have important benefits for the recipient country, although comparability across countries and over time is not easy. Formal standards assessments provide an exhaustive mapping of the quality of regulation, infrastructure, or safety nets. As importantly, they provide insights on the authorities’ capacity to use these systems effectively. In some cases, they reveal gaps that had not been previously identified, while in others, they help mobilize support for changes that are planned or under consideration. Assessments also often help identify technical assistance needs in individual countries. But frequent revisions of the standards, the need to apply the principle of proportionality, and the reliance on expert judgment in the assessment of compliance mean that it is not easy to compare the principle-by-principle ratings over time or across countries, which creates a challenge for evenhandedness (see accompanying background document).

25. Integrating standards assessments into the financial stability analysis of the FSAP is not straightforward. The methodology for formal assessments does not distinguish between individual principles based on their relevance for financial stability. Formal assessments are by design exhaustive rather than selective and risk-focused, and their recommendations are not prioritized based on the severity of the stability risks implied by the identified gaps. And the considerable detail involved in the standards, while important for a comprehensive and effective regulatory framework, is not directly relevant for financial stability. All this was noted in the 2011 Review of Standards and Codes Initiative, as well as the 2012 Financial Surveillance Strategy, which called for upgrading financial surveillance products to foster an integrated approach.

26. In addition, financial sector standards have become increasingly complex and assessments much more resource-intensive. In the aftermath of the crisis, revisions to the core principles and methodologies have extended the length of the assessment process and the final product (DARs). Assessments now require the presence of additional assessors for up to three weeks in the field and a more extensive review. This creates concerns about the Fund’s ability to continue delivering high-quality assessments in the current resource environment, and raises questions about the cost-benefit balance of these assessments for the FSAP.

27. A previous attempt to streamline standards assessments opened the door to more flexible, targeted assessments, but has not yielded tangible results. The “Revised Approach to Regulation and Supervision Standards Assessments” in 2009 allowed for “targeted” (partial) assessments of a subset of principles in a standard, in order to focus the assessment effort and make the output more user-friendly and effective in the context of an FSAP. However, the criteria for selecting the subgroup of principles to be assessed were again compliance-based, unrelated to any macrofinancial considerations. And the conditions under which targeted assessments could be undertaken were set so narrowly that since 2009, only two such assessments have taken place.10

28. The authorities’ views on standards assessments in FSAPs were positive, although a majority felt that they should be streamlined and tailored to country circumstances. Most survey respondents agreed that standards assessments provide a robust evaluation of the regulatory and supervisory framework and were well aligned with other parts of the FSAP. However, satisfaction was relatively lower with how the assessments accounted for country-specific features, and a majority of respondents saw need for streamlining standards assessments to focus on principles of most relevance to the country (see accompanying background document).

Financial safety nets

29. The 2009 review elevated the importance and defined the scope of financial safety nets. This third pillar of stability assessments under the FSAP involves an overview of the country’s liquidity management framework (instruments, collateral policies); deposit protection/insurance and lender-of-last-resort arrangements; crisis preparedness and bank resolution frameworks; and possible spillovers from the financial sector to the sovereign balance sheet. In what follows, the term “financial safety net” is used to refer to all these elements. The 2009 review noted that crisis management and safety nets were covered to some extent in most FSSAs, but were not always given sufficient prominence.” Going forward, the 2009 review set an expectation that financial safety nets would receive sufficient analysis and discussion in all FSSAs, putting them at the same level with the two other elements of the stability assessment.

30. The 2009 reforms brought about an expansion in the coverage of safety nets and related policy recommendations (Table 2). In addition, reflecting the increased coverage, the share of safety nets recommendations in key FSAP recommendations has also increased, and the variation across FSAPs has declined since the 2009 review. Within the safety nets, issues related to crisis management and bank resolution understandably received the most attention, reflecting the crisis experience; followed by discussion on deposit protection (deposit insurance), liquidity management, and broader safety net issues. Crisis preparedness and early intervention received relatively less attention.

Table 2.

Discussion of Safety Nets in FSAPs, 2005–14

article image
Note: Results based on a sample of 50 FSSAs, of which 25 published in 2010–2014 and 25 published in 2005–2009. The sample covers all geographic regions and levels of development (advanced economies, emerging markets, low income countries). The numbers reflect the frequency of textual references to the safety nets relative to textual references to regulation/supervision and stress testing, respectively. A separate analysis has examined the Technical Notes on safety nets in FSAPs, with similar results.

31. Satisfaction with the progress in covering safety nets has also been reflected in the survey of country authorities. About 75 percent of responses indicated that the coverage of safety nets in FSAPs was adequate. These responses are indicative of the progress made in relation to the goals set at the time of the 2009 review.

B. Accountability and Flexibility

32. Institutional accountability was improved markedly with the definition of the stability and development components of the FSAP. The roles of the two institutions in the FSAP are now clear and aligned with their respective mandates. This clarity has helped vis-à-vis country authorities, contributed to focusing FSAP team efforts, avoided instances of duplication, and encouraged a more focused use of resources. Fund and Bank FSAP teams have continued to cooperate effectively in joint FSAPs, particularly in cases where the links between the stability and developmental dimensions of the analysis are tight (e.g., role of state owned–banks, capital market deepening, etc.).

33. The option of modular FSAP assessments was one of the most notable achievements of the 2009 review. In total, five stability and ten development modules have taken place since 2010 (Table 3), with one more stability module (South Africa) ongoing and three development modules planned for 2015. Most of these modules took place in countries in which the authorities explicitly requested a more focused, targeted assessment. But in some cases, a modular approach was chosen to accommodate scheduling conflicts between the Bank and the Fund or other institutional constraints; in the absence of the modular option, these countries would not have undergone assessments. In that sense, modules have worked to everyone’s benefit, enabling the Fund and World Bank to tailor assessments to country needs, and providing increased flexibility to both country authorities and the Fund and Bank. At the same time, it should be noted that survey respondents did not perceive modules as being less burdensome for the authorities compared to “full” FSAPs.

Table 3.

FSAP Modules, FY2010–FY2014

article image

34. The 2009 review called for strengthened mechanisms for quality control and inter-institutional coordination to support the use of the modular approach. A stronger role was envisaged for the Bank-Fund Financial Sector Liaison Committee (FSLC), which was charged with deciding the modalities of FSAPs for emerging countries and LICs (joint missions or separate modules) taking into account the authorities’ preferences, country circumstances, and the need to ensure that both stability and developmental aspects are appropriately covered and the synergies between the work of the two institutions are maximized. Since 2010, the FSLC has provided an effective coordination mechanism where country and institutional constraints have been taken into consideration when planning FSAPs, either joint or in modules—see Box 4 for Bank and Fund staff views on the role of FSLC.

The Role of the Financial Sector Liaison Committee (FSLC)

An important vehicle for coordinating IMF-World Bank work in financial sector issues is the Financial Sector Liaison Committee (FSLC). FSLC is co-chaired by senior Fund and Bank staff. On the Fund side, it includes staff from MCM and SPR, and on the Bank side, as of July 1, 2014, staff of Finance & Markets Global Practice, Macroeconomics & Fiscal Management Global Practice, and the Development Research Group.

In addition to managing the joint aspects of the FSAP, the 2009 review called on FSLC to ensure proper governance of the modular FSAP assessments, as well as to continue acting as a contact point between the Bank and the Fund on other aspects of financial sector work. In practice, FSLC’s principal focus has been the coordination the joint aspects of the FSAP (scheduling, procedures, etc.), but it has also been active in other areas. For example, it has been a forum for discussing issues raised by international standard-setters regarding standards and codes and approaches to quality assurance for DARs/ROSCs.

Smooth Fund-Bank coordination of FSAP-related activities will remain the key task for FSLC going forward. Frictions sometimes arise in connection with differences in Bank and Fund institutional priorities (e.g., the importance to the Fund of timing missions with the Article IV consultation, and the priority that the Bank places on using the FSAP as a platform for supporting follow-up implementation through finance instruments and/or technical assistance). Moreover, there have been growing concerns on both sides of the implications of the Fund’s focus on systemic jurisdiction on non-systemic countries, especially LICs. FSLC will continue to be the place where these issues will be managed at the staff level as they emerge.

FSLC could also continue playing a role in ensuring information-sharing and coordination of other financial sector activities of the two institutions. The existing framework has been successful in addressing issues as they arise, but there remains a need to maximize institutional synergies:

  • FSAP modules. These have succeeded in providing greater clarity on the responsibilities of the two institutions and greater flexibility. However, modules should not be a substitute for “full” updates when these are more appropriate to cover country needs. FSLC should continue to play an active role in reviewing terms of reference before missions are fielded—to ensure consistent policy positions—and to facilitate cross-support in cases where the expertise of the other institution might be helpful.

  • Knowledge sharing. Given the clearer delineation of individual institutional responsibilities, it is important to maintain a strong platform for knowledge management and sharing.

  • Technical assistance. FSLC’s role in this area has waned more recently, as coordination already occurs in the context of the FIRST Initiative—a multi-donor trust fund that supports Bank and Fund TA—and, more recently in the context of the Bank-Fund Debt Management Facility. Nonetheless, there is scope for reviving FSLC’s role in this area, especially in light of the reorganization of the Bank’s work.

  • Policy development: Bank-Fund coordination on financial policy development already takes place effectively on a case-by-case basis. The FSLC continues to take the lead in coordinating work in the area of standards and codes, but when other areas arise where coordination is needed or helpful, FSLC could take a greater role in acting as a clearing house for such efforts.

Traction and Broader Impact

35. The ultimate goal of the 2009 reforms was to increase the usefulness and impact of FSAPs for the recipient countries and for global stability. This has two dimensions: the “traction” of FSAPs with country authorities, i.e., the degree to which the exercise is seen as useful by the authorities and the effect the FSAP has in shaping the domestic policy agenda; and the broader impact the FSAP has on domestic and international audiences and the markets. Although these factors are hard to measure, this section provides a tentative assessment based on different indicators.

36. Survey respondents report a high degree of satisfaction with the FSAP (Figure 3). A large majority of respondents (both country authorities and Executive Directors) thought that FSAPs had been useful, particularly in raising awareness of financial stability issues, encouraging discussion among government agencies and between government agencies and financial sector stakeholders, and improving the broader policy dialogue with the Fund. In every single one of these areas, the share of positive responses has increased compared to the 2009 survey. This picture is consistent with the findings in Box 1 of the TSR that suggests that shareholders consider Fund surveillance to add most value in the financial sector area, where FSAPs make a contribution. FSAPs appear to be less effective in promoting discussion with legislatures or the broader public on financial sector reforms (possibly reflecting differences in the extent of publicity given to FSAPs across countries—see below), although there has been substantial progress since 2009 on the former.

Figure 3.
Figure 3.

Survey Results: FSAP Traction

(Approval Rates: Strongly Agree + Agree)

Citation: Policy Papers 2014, 066; 10.5089/9781498342841.007.A001

Source: Survey results.

37. FSAP policy recommendations have a high degree of implementation and are seen as clear, candid, and feasible. This is supported by staff reporting on implementation of FSAP recommendations in subsequent FSAPs, as well as by the authorities’ own responses to the survey. In addition, some interesting patterns are worth noting.

  • The focus of FSAP recommendations has evolved. Figure 4 presents the evolution of FSAP recommendations pre- and post-2009 review grouped in five broad areas: the three core components of the financial stability assessments, financial infrastructure and—where relevant—financial development. It shows clearly that the bulk of the recommendations have always been in the area of financial oversight; but since the 2009 review, the shares of FSAP recommendations related to the analysis of vulnerabilities and resilience and—especially—to financial safety nets have increased significantly, in line with the re-focusing of the program in light of the lessons from the global financial crisis.

  • The implementation rate is high. Figure 5 presents the record of implementation of FSAP recommendations as reported in FSSAs during 2010-14 (the recommendations themselves were of course made in earlier FSAPs). At almost 80 percent, the share of fully or partially implemented recommendations is very high. There is some variation in the implementation rate across topics (implementation of recommendations that address systemic concerns have a very high probability of being implemented, but it should be noted that the absolute number of these recommendations was very limited).

  • Survey respondents expressed a very high degree of satisfaction with the clarity and candor of FSAP recommendations, though somewhat less with their prioritization and sequencing (Figure 6). Over 90 percent of survey respondents agreed that FSAP recommendations were clear and over 75 percent agreed that they were candid. Satisfaction drops to 60 percent in the areas of prioritization and sequencing. About 60 percent of respondents agreed that FSAP recommendations were feasible with a reasonable effort from the authorities. Indeed, respondents indicated that about one-third of FSAP recommendations were fully implemented, with almost all of the remaining two-thirds partially implemented.11 The main reason given by country authority respondents for non- or incomplete implementation of FSAP recommendations was disagreement over their relevance and, to a lesser extent, disagreement about their degree of priority or insufficient political support. Limited technical capacity for implementation was seen as a less significant constraint, and has declined in importance since the 2009 survey.

Figure 4.
Figure 4.

Evolution of FSAP Recommendations

Citation: Policy Papers 2014, 066; 10.5089/9781498342841.007.A001

Source: Staff calculations.Note: The composition of the pre-FY2010 recommendations is based on the recommendations evaluated in the tables on the implementation status of previous recommendations in the FY2010-2014 FSAPS
Figure 5.
Figure 5.

Implementation of FSAP Recommendations

Citation: Policy Papers 2014, 066; 10.5089/9781498342841.007.A001

Source: FSSAs and staff estimates.
Figure 6.
Figure 6.

Survey Results: FSAP Recommendations

(Approval Rates: Strongly Agree + Agree)

Citation: Policy Papers 2014, 066; 10.5089/9781498342841.007.A001

Source: Survey results.

38. Publication rates of FSSA continue to be high, but there are differences among country groups. Following the 2009 review, the Fund’s publication policy for the FSSA was aligned with that of the Article IV staff report (publication of both is presumed), and the FSB encourages its members to publish their FSSAs. As shown in Figure 7, since FY2009 all FSSAs in advanced countries have been published. Publication rates are lower for emerging market countries and LICs. On average, as well as in each country group, publication rates rose steadily in recent years, reaching 90 percent in FY2013, only to dip in FY2014, reflecting a shift in country composition toward emerging markets and LICs that year.

Figure 7.
Figure 7.

Trends in Publication of FSSAs, FY2009–2014

Citation: Policy Papers 2014, 066; 10.5089/9781498342841.007.A001

Source: FSAP Tracking System and Staff calculations.Note: FY2009 the LICs include also the regional BCEAO/WAEMU.

39. FSAP findings, particularly in countries with systemically important financial sectors, are attracting increasing public attention. Figure 8 shows the evolution of downloads from the Fund’s website of FSSAs published in a given year—a curve is shown for each groups of FSSAs, with the publication year representing the first point in the curve. The Figure reveals two important patterns. First, there is an upward trend in public interest, though with year-to-year variation. Second, the shelf life of FSSAs is relatively short, as downloads peak during the year of publication and fall markedly in the following years.

Figure 8.
Figure 8.

Trends in FSSA Downloads

Citation: Policy Papers 2014, 066; 10.5089/9781498342841.007.A001

Source: Staff estimates.Notes: In 2009 there were no FSSAs published for any of the S25 countries. Data for 2014 are annualized.

40. News about FSAPs can also have a market impact (Box 5). Using a sample of 39 FSAP-related “events” (end-of-mission press releases or publication of FSSA documents), an event study found that such “news” can have a statistically significant impact on banks’ stock prices in the relevant country. The impact can be positive or negative, suggesting that the releases conveyed useful information to market participants prompting them to reassess their valuation of banks. While purely indicative, this finding suggests that FSAPs can have a real-world impact beyond their contribution to the domestic policy debate.

The Market Impact of News about FSAPs1

A standard event study of 19 FSAPs suggests that FSAPs can provide “useful” information (e.g., information not already priced in by markets) to stock market participants. The study was used to assess the stock market reaction of banks following 38 FSAP-related outreach “events,” defined as press releases or press conferences following the FSAP mission or joint Article IV-FSAP missions or publication of FSSAs. The study covered the period 2009-13.

  • In a first-stage, a capital asset pricing model (CAPM) model was estimated using excess bank stock returns (returns minus a risk-free rate, (Rit)) and, country stock market indices (Rmt). The sample includes daily stock prices for 144 European and 67 US banks and other financial institutions, as well as country stock market indices. The estimated regression was:
    Rit=αi+βiRmt+εit.(1)
  • In a second-stage, abnormal returns (ε^it) were computed for each financial institution by subtracting fitted values from the first stage from actual values as in:
    εit=RitαiβiRmt.(2)
  • Finally, vectors of cumulative abnormal returns (CAR) were built by adding up the abnormal returns of the event date itself and the two preceding and two succeeding observations, and the statistical significance of the CARs was examined using the test statistic (tKP) developed by Kolari and Pynnönen (2010). 21 events (for 12 FSAPs) were found to contain useful information, either positive or negative:

Statistically Significant Market Reactions to FSAP Events

(Cumulative abnormal returns (CARs) and t statistics)

article image
1 Prepared by Amadou Sy.

The FSAP and Surveillance

41. The Fund’s macrofinancial surveillance with individual jurisdictions is the principal responsibility of the Article IV consultation, and covers a range of issues that extend beyond the usual scope of the FSAP. As the 2014 TSR makes clear, Article IV surveillance should cover the basic macrofinancial relationships in and around the baseline, the key drivers behind current trends, and the ways in which they interact. It should assess the level of systemic risk and engage in policy dialogue on the appropriate policy settings, including macroprudential policies. It should also examine longer-term issues, such as the efficiency of financial intermediation, the role of the financial system in monetary policy transmission, the interrelationship between evolving market practice and credit cycles, and the role of state-owned banks, where relevant. And it should cover these issues on a continuous basis, and do a better job at integrating them with the macroeconomic analysis. The FSAP, in contrast, has a narrower focus on financial stability issues and tail risks. And even in this area, given their low frequency, FSAPs cannot supplant the Article IV’s responsibility to provide a continuous assessment of vulnerabilities and risks, as well as to be the main vehicle for an ongoing dialogue with country authorities on risk-mitigating macroeconomic and financial policies.

42. Nevertheless, FSAPs can provide important input to surveillance. Their comparative advantages are to provide an in-depth and relatively sophisticated assessment of stability risks and systemic resilience; an evaluation of the regulatory framework, effectiveness of supervision, financial safety nets, and infrastructure benchmarked against international experience and best practice; and an analysis of institutional arrangements for micro- and macroprudential oversight and, more generally, financial sector policies. In emerging market and developing countries, the World Bank developmental assessments can also provide important additional insights. The breadth and depth of expertise FSAPs bring to bear in the analysis of these issues go beyond those of the typical Article IV team, and can be a valuable complement to Article IV surveillance, both in the year of the assessment and subsequently.

43. The FSAP and Article IV consultation processes are closely integrated. FSAP assessments and Article IV missions are coordinated and teams always overlap. FSSAs are joint MCM/Area Department documents and provide background information to the Article IV consultation.12 This helps ensure consistency of policy advice. And the 2014 TSR documents the progress that has been made in ensuring that Article IV teams follow up on the implementation of FSAP recommendations.

44. The FSAP can contribute to the agenda laid out in the 2014 TSR to mainstream macrofinancial issues in surveillance. In addition to maintaining the close integration between the FSAP and the Article IV processes, an enhanced systemic risk focus of the various FSAP components can help not only streamline the FSAP itself but also maximize its input to surveillance. Experience has shown that when FSAP findings and recommendations are prioritized based on their macrofinancial relevance, their integration into and follow up by Article IV missions is more effective. Beyond the FSAP itself, disseminating standardized tools for risk analysis to Area Department teams would help transfer of knowledge and expertise—although, as with FSAPs, the scope of the analysis will always depend on the availability and quality of data. Operational changes to incorporate these lessons are the topic of the second part of this paper.

45. In addition, there is scope for aligning the periodic reviews of the FSAP with the broader review of the Fund’s surveillance. The proposal in the TSR to move the review of surveillance on a quinquennial basis, if approved, provides an opportunity to link closely the two reviews, which would in turn facilitate the assessment of the integration between the two products.

46. While the FSAP is principally a bilateral instrument, there is increasing scope for incorporating regional perspectives. Regional assessments have been undertaken in three currency unions (e.g., the West African and the Central African CFA franc zones and the East Caribbean Currency Union), reflecting the fact that in these cases there are regional supervisory and regulatory authorities with whom policy discussions can be undertaken, and because the IMF undertakes regular Article IV consultations with the relevant macroeconomic authorities. More recently, an FSAP assessment was undertaken in the European Union, in response to the move by the EU toward a banking union, and as a complement to the Article IV consultation that is regularly conducted with the Euro area authorities. While the FSAP will remain primarily a bilateral instrument, just as with the Fund’s new emphasis on clustering Article IV consultations, there may be opportunities to time the conduct of national FSAP assessments for highly interconnected jurisdictions in ways that maximize their analytical depth and impact.

47. In addition, FSAPs can provide input into multilateral surveillance. FSAP assessments in systemic jurisdictions provide important insights into multilateral surveillance products, like the WEO, GFSR, and Spillover Reports. More broadly, the systematic discussion of financial sector policies in bilateral FSAPs, including notably the analysis of policy spillovers, where relevant, can help promote greater consistency in approaches to regulatory reform and the use of macroprudential policies.

48. The 2010 decision to make financial stability assessments under the FSAP a mandatory part of Article IV consultations for jurisdictions with systemically important financial sectors has had a profound impact on the role of FSAPs in surveillance. This landmark decision has provided for a risk-based deployment of global FSAP resources and strengthened the contribution of surveillance to global financial stability. The decision was embraced by the membership, and most affected jurisdictions requested assessments fairly close to the five-year mark since the last FSAP. As a result, between 2010 and 2014, 24 of the 25 mandatory financial stability assessments were conducted, plus a special assessment for the EU in 2013, thus covering the most important financial sectors in the world. In all these cases, cooperation with the authorities was excellent, and there was close and effective cooperation with the Article IV team.13 FSAP findings in systemic countries provided critical input into the Fund’s surveillance, including policy recommendations to address vulnerabilities that had contributed to the most severe bout of global instability in decades.

49. At the same time, this shift of focus toward systemic jurisdictions has had a major impact on the availability of FSAPs in non-systemic countries. Post-crisis, the FSAP program was hit by a “perfect storm” of demands: the increased complexity of post-crisis FSAPs (for example, the need for greater coverage of macroprudential and financial safety net issues); the spike in demand for formal standards assessments; and the larger number of FSAPs in large, complex, and systemic jurisdictions have driven up the average cost of individual assessments (Figures 9 and 10). The latter factor, in particular, has been paramount: FSAPs for advanced countries are, on average, nearly twice as expensive as those for emerging countries or LICs, and the cost of FSAPs for countries with globally systemically important financial sectors is more than double that of similar countries with smaller and less complex financial systems. These factors more than offset the modest rise of the overall resource envelope for the program. As a result, the average number of FSAPs per year dropped from about 18–20 prior to 2009 to 14–15 in the last few years, and the impact of this reduction has affected primarily LICs and other non-systemic countries.

Figure 9.
Figure 9.

Cost of average FSAP

Citation: Policy Papers 2014, 066; 10.5089/9781498342841.007.A001

Source: Staff estimates.
Figure 10.
Figure 10.

Main Drivers of the Cost of the FSAP Program in FY2011–2014

Citation: Policy Papers 2014, 066; 10.5089/9781498342841.007.A001

Source: Staff estimates.

50. The impact was differentiated across regions, reflecting the application of the prioritization criteria for voluntary FSAP requests. FSAP requests from non-systemic jurisdictions are prioritized according to criteria established by the Board: (i) systemic or regional importance; (ii) external sector weakness or financial vulnerabilities; (iii) major reform programs that might benefit from a comprehensive financial sector assessment; and (iv) features of the exchange rate and monetary policy regime that make the financial sector more vulnerable. Within the space framed by these criteria, staff is also supposed to take into account the balance across regions and different levels of financial sector development and the time elapsed since the last FSAP. Given this prioritization framework, during the last five years FSAP coverage in Europe, the Asia and Pacific, and Western Hemisphere regions increased substantially, mainly at the expense of coverage in Sub-Saharan Africa and the Middle East (Figure 11). The picture improves significantly if development modules are taken into account, but the issue of insufficient coverage of financial stability issues in LICs remains. The external consultants’ study on evenhandedness has analyzed the pattern of distribution of FSAPs in non-systemic countries and, while noting the regional imbalance since 2010, has concluded that it broadly reflects the application of the existing prioritization criteria (see background document).

Figure 11.
Figure 11.

Regional Distribution of FSAPs since FY2005

(Dark blue = FSAP ; Light blue = No FSAP)

Citation: Policy Papers 2014, 066; 10.5089/9781498342841.007.A001

Source: FSAP Tracking System

Lessons Learned

51. The reforms introduced at the 2009 FSAP review and subsequent changes to the program have been successful. They have strengthened the FSAP in several dimensions, as illustrated by both staff’s own assessment and the overall very positive responses to the survey of FSAP users, especially compared to the similar survey undertaken in 2009. The program has also been open to the advances in financial theory and empirics during recent years, which has allowed the gradual mainstreaming of new tools in FSAPs and has promoted institutional learning-by-doing.

52. At the same time, the experience highlights several lessons learned. The main challenge for the FSAP is to consolidate the gains of recent years, further strengthen the analytics and the macrofinancial focus of the program, adapt to the evolving global financial landscape, and continue to absorb emerging policy lessons. In the context of financial surveillance, it is also important to be aware of the FSAP’s limitations and keep it focused on its comparative advantages relative to the broader coverage expected of Article IV consultations—as outlined in the TSR.

53. Relatively straightforward steps can be taken to allow the FSAP to incorporate these lessons. Efforts should concentrate on (i) focusing all components of the financial stability assessment under the FSAP more clearly around the concept of systemic risk; (ii) continuing to upgrade the analytical underpinnings of the analysis of vulnerabilities and resilience while, at the same time, documenting more diligently the limitations of this analysis (e.g., those rooted in data quality or availability) and providing greater standardization and guidance in the selection of tools and macroeconomic scenarios; and (iii) improving the quality and clarity of the FSSA so as to ensure continuing traction with country authorities and provide better input to Article IV surveillance. These incremental improvements can be accommodated within the existing framework for the program. This second part of the paper discusses these issues in more detail.

A. Stronger Focus on Systemic Risk

54. Greater focus on systemic risk in FSAPs would benefit surveillance. Given their low frequency, FSAPs cannot provide the continuous coverage of evolving macrofinancial risks, which is the responsibility of the Article IV consultation. Nevertheless, FSAPs can make valuable complementary contributions to risk assessment in surveillance by bringing to bear more sophisticated tools than those available to the typical Article IV team; by analyzing emerging tail risks that at the time of the FSAP may still be “over the horizon” but may become relevant in the following years, before the next FSAP takes place; and by engaging authorities in a more in-depth dialogue on a broader range of financial risk mitigation policies and institutional arrangements.

55. To maximize this contribution, systemic risk should become the main organizing principle of all aspects of the financial stability assessment under the FSAP. The content of the financial stability assessment as defined by the 2009 review provides a flexible and adequate framework for focusing the work of the FSAP. As the review of recent FSSAs shows, however, this focus does not permeate all components of the financial stability assessment equally. The assessment of systemic vulnerabilities and resilience—the first component of the financial stability assessment—is squarely focused on risk and grounded in the RAM. But the evaluation of financial sector oversight and of financial safety nets—the other two components—is less risk-focused, despite the increasing coverage of MaPP frameworks evident in recent years. And FSAP recommendations, especially the large share of micro-prudential and institutional issues, are often driven by identified gaps in compliance rather than their impact on systemic risk.

56. Enhancing the systemic risk perspective of the evaluation of financial sector oversight requires an innovative approach. The evaluation of financial sector oversight and supervisory effectiveness in FSAPs relies heavily on formal compliance assessments with established international standards (DARs/ROSCs). The number of such assessments has indeed surged in recent years. But as discussed in the previous section, these assessments tend to be exhaustive rather than risk-based, as well as increasingly complex and costly. What is needed is a way to (i) guide the assessment effort based on the most relevant risks faced by the financial sector in a particular country, and (ii) integrate the findings of the evaluation of financial sector oversight—whether in a formal or informal assessment—in the overall risk assessment of the FSAP.

57. Staff has developed a “macrofinancial approach to supervisory standards assessments” for this purpose. This pioneering approach is described in the accompanying background paper, and combines quantitative and qualitative criteria to identify a macrofinancially-relevant subset of individual principles in each of the three supervisory standards (for banking, insurance, and securities). If this approach is adopted, these subsets of principles, complemented by additional principles if justified by country-specific financial stability considerations, would be the focus of the assessment work in FSAPs, with full assessments still an option in certain circumstances. Needless to say, the assessment would continue to focus not only the regulatory framework but also actual supervisory intensity and effectiveness, as at present. Based on its findings, the FSAP team would determine whether the quality of regulation and supervision mitigates or aggravates the risks facing the system, and incorporate this into the RAM. Box 6 summarizes the key elements of this approach, which is explained fully in the background paper.

A Macrofinancial Approach to Standards Assessments1

The accompanying Background Paper presents a new approach to supervisory standards assessments to help streamline them and integrate their findings more easily into the FSAP’s overall risk assessment. To map financial sector risks to the most relevant BCP, ICP and IOSCO principles, a methodology combining econometric analysis and expert judgment is applied.

The approach relies on a four-stage process.

  • In the first stage, key macrofinancial risks relevant for the banking, insurance, and securities sectors are identified.

  • In the second stage, the high correlation of “grades” across principles is addressed by different techniques, including principal component and explanatory factor analysis, that reduce the dimensionality (number of explanatory variables) by identifying a small number of factors that are relatively independent from each other and explain most of the variation of “grades” across assessments, while preserving as much information in the original data as possible.

  • In the third stage, both parametric and non-parametric econometric approaches are used to identify correlations between each principle—or each explanatory factor—and each measure of risk.

    • □ Under the parametric approach, a multi-regression search process is used to identify the principles that are most relevant for financial stability based on the sign, significance, and robustness of the interaction terms between macrofinancial variables and principles’ “grades”, as well as on their ability to minimize forecasting errors.

    • □ Under the non-parametric approach, a binary tree analysis identifies non-linear relationships between macrofinancial variables and principles’ “grades”. This analysis selects the principles for which compliance (high “grades”) seems to be most strongly correlated with low measures of risk. This selection indicates the principles that are most relevant for financial stability under the prevailing macrofinancial conditions.

  • Finally, expert judgment identifies macrofinancially relevant principles by applying a system-wide, macro perspective to regulatory standards that takes into account the sources and transmission channels of risk.

The final set of macrofinancially-relevant principles is selected by combining the econometric and expert judgment approaches. The final set includes principles that were both found to be macrofinancially-relevant in the econometric (parametric and non-parametric) analysis and considered particularly important for financial stability by expert judgment. Also included are principles that were considered important for financial stability by the experts, but either were not included in the econometric analysis—due to lack of data—or needed to be assessed in conjunction with selected principles to provide an effective assessment.

The identification of these macrofinancially relevant principles would help guide and focus the assessors’ work on the FSAP. In addition, the results of the assessment of these principles would be integrated in the RAM, which would indicate whether the overall quality of supervision and regulation in the relevant areas aggravates or mitigates the probability of realization and/or the impact of the specific risk.

1 Prepared by Laura Valderrama.

58. This approach puts forward a risk-based, macro-relevant set of criteria for targeting standards assessments in FSAPs but does not rule out full assessments. It improves upon the earlier “targeted ROSC” model by replacing the compliance-based criteria for principle identification with macro-relevant ones. It is meant to be used in FSAPs, not to replace full assessments completely. Full assessments would continue to be conducted upon the authorities’ request, for example in the context of capacity building.

59. This approach should be considered in the context of the next review of the Standards & Codes initiative. Although the proposed approach applies to FSAPs, not to standards assessments undertaken in other contexts (e.g., technical assistance), such an innovation cannot be implemented without a broader discussion—also with the standard-setters and the World Bank—of its implications for the Standards & Codes initiative, including its potential application to other standards. It is therefore presented here to elicit initial reactions, but would be considered in the context of the next Standards & Codes review in 2016.

60. In addition, and in line with the TSR, MaPP issues merit greater emphasis in FSAPs, as well as Article IV consultations. To achieve a more systematic treatment of these issues, it will be necessary to disseminate best practices, especially from recent country experiences, and implement the forthcoming Staff Guidance Note on Macroprudential Policy. In addition, FSAPs should ensure that the various dimensions of MaPP analysis are adequately integrated and the associated findings and policy recommendations are clearly articulated in the relevant documents.

61. A stronger systemic risk perspective could also be brought to bear on the analysis of financial safety nets in FSAPs. The application of international standards to safety nets is still in its nascent stage,14 and therefore an approach similar to that proposed for the supervisory standards is not yet possible. Nonetheless, a systemic risk focus should be incorporated in the evaluation of financial safety nets by exercising greater selectivity in coverage: the emphasis given to the various components of the safety nets should be commensurate to their relevance for systemic risk. In addition, there is scope for FSAP teams to integrate better the analyses of safety nets, financial stability, and the oversight policy framework guided in part by the RAM.

B. Upgraded Tools and Improved Analytical Frameworks

62. Despite the considerable progress made in this area since the 2009 review, the evolving global financial landscape is continuously raising the bar for the analysis of vulnerabilities. The discussion below outlines improvements in the areas of stress testing, interconnectedness, and spillover analysis. Beyond the steps that can be taken in each of these areas, however, a continuing challenge for FSAP teams would be to ensure adequate integration of these various strands of analysis into a coherent narrative.

63. Stress testing tools should continue to evolve to expand sectoral coverage and incorporate the latest advances in methodologies and models. Areas of priority include expanding stress tests to non-bank sectors, modeling of second-round effects, and capturing non-linearities and risk co-dependence in tail risk scenarios. The analysis of vulnerabilities in systems dominated by financial conglomerates is a particularly difficult challenge: first, because these structures often limit data availability (e.g., on intra-group transactions), and second, because current stress testing tools are not easily applicable to non-bank financial intermediaries in groups (including shadow banking entities). This analysis will require the development and deployment of new approaches, on a case-by-case basis.

64. The authorities’ cooperation, including through confidential data sharing on a voluntary basis, is a crucial component of success, and in many cases more and better data are needed. Access to institution-specific data and information that may not be publicly available improves the depth and relevance of the vulnerability assessment in the FSAP. Although there are constraints—these data go beyond the requirements of regular surveillance, and in some cases there may be limitations in national legislation—providing these data on a voluntary basis to the FSAP would, other things being equal, improve the macrofinancial analysis and result in a better product. Further progress in this area could thereby strengthen the Fund’s macrofinancial surveillance. Experience has shown that the Fund’s existing confidentiality protocol provides sufficient safeguards for the confidential treatment of this information by the FSAP team.

65. A key challenge in the area of stress testing will be to manage the tension between the need for comparability and evenhandedness on one hand and tailoring the tools and scenarios to country circumstances on the other. Clearly, one-size-fits-all rules for scenario selection and calibration are not appropriate for all countries at all times, notwithstanding their benefits in terms of ease of comparisons. But the unique experience gained by Fund staff in this area over many years of FSAPs, distilled in the recent paper on “best practice” principles for stress tests, provides a solid basis for specific guidance on scenario selection and, more broadly, stress test design. On this basis, MCM plans to develop in the near future (i) a set of guidance notes of general applicability on a range of aspects of stress test design in FSAPs,15 and (ii) in cooperation with TGS, a set of simple, portable stress testing tools, with user’s manuals. The latter would define a minimum set of requirements for stress tests in FSAPs, and could also be used in the context of Article IV surveillance or capacity-building activities. At the same time, existing requirements for detailed disclosure in the FSSA of all assumptions used in FSAP stress tests will be maintained. The combination of guidance covering most aspects of stress test design and full disclosure will improve comparability and transparency while still leaving room to FSAP teams, in cooperation with country counterparts, to tailor stress tests to the circumstances of the country in question. Moreover, given the turnover of Fund staff involved in FSAPs, these guidelines would also help knowledge management and improve continuity within FSAP stress tests.

66. There is also scope for strengthening FSAP work on interconnectedness between financial institutions and across sectors. A more systematic use of network analysis can be achieved in FSAPs so as to make interconnectedness a more regular feature of systemic risk analysis. Since data availability tends to dictate the coverage of the analysis in practice, additional efforts could be devoted to the addressing data gaps in this area and further disseminating the (relatively complex) analytical tools. Implementation of the G-20 data gaps initiative recommendations would also help close this gap. While network analysis in finance is still in its infancy, current applications in FSAPs could be better integrated with the solvency analysis and used more frequently to help identify SIFIs (Box 7).

Interconnectedness Analysis in FSAPs: Key Challenges1

An enduring lesson from financial crisis episodes, including the most recent global crisis, is that interconnectedness among financial institutions is an important transmission channel for shocks, contributing to risk spillovers and contagion. During normal periods, interconnectedness serves a useful purpose, by helping financial institutions to diversify counterparty exposure, facilitating a more efficient risk allocation, and reducing intermediation costs. However, during periods of stress, interconnectedness not only amplifies shocks but can trigger “domino” effects, with the failure of one financial institution triggering additional failures. Hence, the analysis of systemic risk should account for the positive externalities and the risk posed by “too-connected-to-fail” institutions.

No consensus, either in academia or in the policy-making community, has yet been reached on the financial architecture (or topology) that optimizes the tradeoff between the risks and the rewards associated with interconnectedness. Achieving consensus is further complicated by the nature of financial networks, which is constantly evolving and changing, reflecting both the rapid pace of financial markets and financial innovation, and the response of business models and practices to the regulatory and policy environment. Additional difficulties in monitoring exposures between financial institutions arise from the contingent payoff of financial contracts linking them owing to the contracts’ embedded options; and whether hedging strategies would work in case market liquidity dries up.

Analyzing interconnectedness, therefore, requires a deep understanding and detailed information on balance and off-balance sheet exposures, the trading book and hedging strategies in banks; and how these elements interact with credit risk, market risk, and liquidity risk. Typically, this information is not available to FSAP teams and in many instances, not even to national authorities. Notwithstanding these data shortcomings, MCM has developed tools to analyze interconnectedness arising from direct balance sheet exposures (network analysis), or using price information from traded securities when exposure data is not available (contingent claim analysis, CoRisk).

Data limitations have influenced the scope of the analysis of interconnectedness risk in FSAPs. The analysis maps the topology of the domestic financial system, when data are available, and identifies linkages between the domestic banking system and the global banking system using BIS aggregate banking data. Based on the results, FSAPs recommend strengthened surveillance of SIFIs and on cross-border exposures that could affect the domestic banking system. Within the context of the latter, FSAPs conduct a detailed analysis of cross-border exposures arising from branches and subsidiaries of international banks, including concentration limits and the international dimensions affecting the effective deployment of bank’s safety nets and resolution mechanisms.

Going forward, the analysis of interconnectedness risk should aim to integrate it within the solvency and liquidity stress testing framework. Methodologies set out by the Bank of Canada and the Bank of England offer a good starting point for advancing this work agenda further. Advances in the methodological work will also require better access to granular bank data.

1 Prepared by Jorge Chan-Lau.

67. Coverage of cross-border spillovers is essential but analytically challenging and hampered by data gaps. Progress in extending and deepening the analysis in this area will be gradual, until some of these challenges are overcome and a critical mass of cross-country experience is built among staff teams.

  • Adequate coverage of spillovers in FSAPs depends critically on the cooperation of national authorities in providing data on cross-border bilateral exposures. The lack of these data is a key obstacle preventing a more systematic treatment of spillovers in FSAPs, and the full cooperation of national authorities to address this gap is necessary. In addition, FSAP teams should seek to engage national supervisors and the BIS to access the data hub created at the BIS on bilateral credit exposures on an institution-by-institution basis for G-SIFIs.16 Market price-based models (such as the CCA) can help overcome the problems posed by the lack of granular exposure data, but reliable market data are not available for all institutions in all countries, and volatility of estimated correlations during stress periods limits their usefulness.

  • Analysis of outward spillovers is much more challenging than that of inward spillovers, but critical for systemic countries. Consistent with the 2012 Integrated Surveillance Decision encouraging member countries to take into account the effects of their policies on the operation of the international monetary system, an attempt should be made to ensure that analytical tools are used to assess outward spillovers—especially from fluctuations in cross-border funding linked to intra-group operations of global banks—at least in each of the S29 countries.

  • Different types of cross-border exposures call for different approaches. Stress tests are typically conducted on a consolidated basis, which assumes that resources available at one office location can immediately be used elsewhere at the banking group level. This assumption may not hold, however, when concerns exist about transactions between subsidiaries and parent banks. In such cases, it is imperative to monitor intra-group transactions, raise liquidity ratios on a solo basis, or reduce the dependence on parent credit lines.

  • Identifying the underlying causes of contagion detected by the spillover analysis is not always easy. In order for the analysis of spillovers to provide a solid basis for policy recommendations, it is essential to identify the economic and financial factors that explain the estimated magnitude of cross-border spillovers. For example, cross-border contagion may reflect banks’ use of similar business models, concentration of investor bases, correlated trading strategies, changes in bank competition, or structural changes linked to global banks’ funding and liquidity strategies. Even if adequate data are available, distinguishing between these factors may not be straightforward and may require reliance on judgment. Still, it is of critical importance in shaping the relevant policy recommendations.

68. Against this backdrop, FSAPs should be upfront in acknowledging the limitations of the risk assessment. Some of these limitations are inherent in the current state of understanding of macrofinancial linkages. But in many cases, these limitations are the direct result of inadequate data availability or concerns about data quality. Being clear about these limitations in the FSSA is essential both for transparency and for limiting the reputational risk for the Fund if (foreseen or unforeseen) financial sector risks materialize following the FSAP.

C. Improving Clarity and Communication

69. Stronger systemic risk focus and improved analytics in FSAP will pay off only if they are communicated clearly and effectively. Communication of FSAP findings and recommendations has always been challenging, partly because many of the issues covered are highly technical, sometimes market-sensitive, and often involve a host of legal and institutional details. This complicates the task of prioritizing the main findings and recommendations and communicating them clearly to national authorities and other stakeholders. Nonetheless, this is a critical task for the FSSA—the key FSAP document, as well as an important input to surveillance.

70. Changing the structure of the FSSA is key to better clarity and communication.

  • Using systemic risk as the organizing principle for prioritizing FSAPs findings and recommendations. Systemic risk should be used as an organizing principle in each of the three pillars of the stability assessment, as discussed earlier, with the diagnosis of vulnerabilities emphasizing threats to systemic stability, and the assessments of the oversight framework and the safety net largely focusing on ways to address those threats. While FSSAs already do this to some extent, they also often include a number of other issues that may be important in their own right but are not clearly linked to systemic risk assessment. Also, the summary of FSAP recommendations in the FSSA should be explicitly prioritized based on their relevance for mitigating systemic risk.

  • Ensuring an adequate presentation of the key macrofinancial linkages in the baseline outlook. At present, FSSAs provide a brief characterization of the current macrofinancial outlook and focus more on the characterization of the main (tail) risks faced by the economy and summarized in the RAM. Also, as discussed earlier, the RAM is typically linked to the quantitative assessment of risks and resilience (stress tests), but less so to the assessment of oversight and safety nets. In the future, FSSAs should articulate better the main macrofinancial linkages of relevance in the baseline, developed in close cooperation with the Article IV team; and these, together with the RAM, should guide the assessments and recommendations in each of the three pillars.

  • Identifying data gaps. Clearer and more candid identification of FSAP teams’ access to information and data, where these are material for the risk assessment, will help clarify the limitations of the analysis, lower reputational risk, and improve the ability to make cross-country comparisons. The standard disclaimer on risks not covered by the assessment should be expanded accordingly.

To illustrate the impact of these proposals on the FSSA, a simple comparison between its current and proposed structures is provided in Figure 12. The new structure reflects the general recommendations made earlier on the need to better focus and prioritize the work of FSAPs, but also provides a blueprint that can be used flexibly to account for country-specific features.

Figure 12.
Figure 12.

The New FSSA: Macrofinancial and Systemic Risk Focus

Citation: Policy Papers 2014, 066; 10.5089/9781498342841.007.A001

D. Addressing the Needs of LICs

71. The combination of the overall resource constraint and shifting the emphasis of FSAPs toward systemic jurisdictions has had an substantial impact on access to FSAPs for LICs. Development modules have gone some way in addressing LICs’ developmental and capacity needs, but still leave gaps in the financial stability area. Development modules often focus on sector-specific issues—e.g., the development of the insurance sector, pensions systems, financial infrastructure, access to finance, etc.—and do not assess the extent of system-wide financial vulnerabilities, the robustness of the supervisory system, and the strength of financial safety nets and crisis management frameworks.

72. In light of the pressures on the Fund’s ability to provide FSAP assessments in non-systemic jurisdictions, staff has taken several steps to fill this gap.

  • Stepped up technical assistance on financial stability issues: MCM’s recent “Technical Assistance Strategy Update, 2014–2017” notes that a much greater emphasis has been (and will continue to be) placed on TA in the area of systemic issues and crisis management, albeit from a relatively low base compared with the more traditional focus on bank supervision and central banking. These new areas of focus include TA on stress testing, systemic risk analysis, and macroprudential policies and institutional frameworks. In addition, greater volumes of TA are being provided on crisis management and contingency planning frameworks, financial safety net arrangements, and bank asset quality reviews.

  • Closer integration of financial TA with surveillance: As described in the background work for the 2014 TSR, the Fund has been engaged in a series of “pilot exercises” in which there has been an explicit pairing of financial TA missions with Article IV missions.17 This experience has been successful in improving the ability of area department teams to identify the manner in which underdeveloped financial systems may impede the effectiveness of macro-policies, but also to encourage a greater macro-relevance of Fund TA. These themes have been embodied in the 2014 TSR and MCM’s TA strategy, and are expected to continue to bear fruit in coming years.

  • More holistic approaches to technical assistance: In recent years, MCM has sought to increase diagnostic TA assessments focusing on strategies for financial system stability. These assessments have identified capacity challenges in, among others, regulatory, supervisory, and crisis management frameworks, cooperation within and among institutions, stress testing, and data coverage. At the same time, the assessments have discussed measures to tackle these challenges in the context of specific projects that could benefit from TA. The recently-introduced Capacity Assessment program also provides scope to assess the institutional capacity in the areas of supervision and central banking.

73. At the same time, TA cannot fully substitute for an FSAP assessment. While all TA has a diagnostic element, and “holistic” diagnostic assessments even more so, they are essentially guided by the demands of country authorities, are focused mainly on capacity needs, and do not provide as rigorous a scrutiny of financial vulnerabilities as FSAPs. While this may be appropriate in many cases—and indeed better suited than an FSAP to country needs under certain circumstances—it should not be seen as a fully-fledged alternative to an FSAP.

74. Looking forward, the challenge will be to continue to leverage these existing instruments as well as the work of other institutions. A key partner in this effort will continue to be the World Bank, and—as noted above—there is scope to further strengthen the role of the FSLC as a vehicle for coordinating the provision of TA. Moreover, MCM has been and will continue to work closely with donors to help bring to their attention where TA needs are high and solicit their support where appropriate.

E. Resource Issues

75. Incorporating the lessons learned from the recent experience and implementing the proposals outlined above should be possible to accommodate within existing budgets. Some of the steps outlined in the previous sections, such as using systemic risk as a basis for focusing FSAPs, prioritizing recommendations based on their macrofinancial relevance, etc. can mostly be achieved by re-directing existing resources. Others, such as expanding the analytical toolkit, creating portable quantitative tools, and establishing guidelines for stress testing in FSAPs would have an additional cost, but this is expected to be manageable since work on some of these initiatives is already underway. And introducing a macrofinancial approach to standards assessments—if endorsed in the next review of the Standards and Codes initiative—could result in some savings over the medium term.

76. Containing the cost of individual assessments should remain a priority. As the preceding discussion showed, the average cost of an FSAP assessment has been increasing. This trend largely reflects exogenous factors: the change in the country composition toward more systemic, complex cases; the surge in demand for formal compliance assessments with standards; and the expansion of the thematic coverage of the FSAP in certain areas whose importance was underscored by the crisis, notably macroprudential frameworks. The decisions taken as part of the 2009 review to strengthen internal review and quality control also played a role. The more recent easing of demand for formal compliance assessments should help arrest the trend toward ever-higher costs, and many of the steps outlined above (notably streamlining all components of the assessment around the concept of systemic risk and—if the proposal is approved—moving toward macrofinancially-relevant standards assessments) would help. But continued effort (and the cooperation of country authorities) will be needed to contain the scope and cost of the assessments.

77. However, the low frequency of FSAPs in non-systemic jurisdictions is chiefly the result of the resource constraint. Difficult tradeoffs have surfaced as a result of focusing global FSAP resources to systemic jurisdictions. Although the paper has laid out options to mitigate the impact on LICs, these are unlikely to offset the combination of constrained resources and the focus of the program on jurisdictions with systemically important financial sectors since 2010, an issue already flagged in the 2012 Financial Surveillance Strategy.18

Issues for Discussion

  • Do Directors agree with the main findings of this review of the FSAP? In particular, do Directors agree that the 2009 FSAP review was broadly successful in improving the analytical focus of FSAPs, providing greater flexibility to country authorities and the Fund, and establishing institutional accountability?

  • Do Directors agree that the key priorities emerging from the lessons learned are to strengthen the systemic risk focus of the financial stability assessment under the FSAP, continue improving the analytics while being explicit about the limitations of the risk assessment, and restructuring the FSSA to bring out these elements more clearly? Do Directors agree that more systematic provision of non publicly-available data on a voluntary basis is an important component of the effectiveness of the financial stability assessment under the FSAP and, more broadly, macrofinancial surveillance?

  • Do Directors believe that a macrofinancial approach to supervisory standards assessments is a promising avenue for enhancing the effectiveness and focus of compliance assessments in FSAPs and facilitating their integration in the overall risk assessment? Do Directors agree to explore further such an approach in the context of the next Standards & Codes Review?

  • Do Directors endorse the use of more targeted Technical Assistance as one of the ways to enhance the coverage of financial sector issues in LICs, given the shift of the focus of the FSAP program toward jurisdictions with systemically important financial sectors?

  • Do Directors agree to consider aligning and coordinating future reviews of the FSAP with the periodic reviews of surveillance?

1

Pursuant to that decision, 25 jurisdictions were determined to have systemically important financial sectors, based on 2008 data. A revision of the methodology in 2013, using more recent data, expanded the list to 29 jurisdictions.

2

Professors Masahiko Takeda and Hiroshi Ugai, Asian Public Policy Program, School of International and Public Policy, Hitotsubashi University, Tokyo.

3

The RAM presents in a tabular form the FSAP’s assessment of the key risks facing the financial sector, the probability of realization of each risk in the short to medium term, and the expected economic impact.

4

Examples include the next generation balance sheet solvency and liquidity stress test toolkits (Schmieder et al. 2011 and 2012); the network spillover analysis toolkit (Espinoza & Sole 2010); the integrated framework for solvency and liquidity stress testing (Barnhill & Schumacher 2011); a primer for stress testing pension funds (Impavido 2011); and the multi-sector multi-country extension of the Contingent Claims Analysis (CCA) through the Global Vector Autoregression CCA (CCA-GVAR), developed in cooperation with staff of the European Central Bank (Gray et al. 2013).

5

The Expert Forum is organized by MCM staff in cooperation with different central banks, is addressed to stress test practitioners in central banks and supervisory authorities, and focuses on latest developments in stress testing tools and techniques. In recent years, such fora were organized in cooperation with the Swedish Riksbank (2011), the Hong Kong Monetary Authority (2012), and the Bank of England (forthcoming, 2014).

6

This issue is also highlighted in the external consultants’ study on evenhandedness in FSAPs, included in the accompanying background document.

7

Most of these rely on network analysis based on aggregate exposure data produced by the BIS, with a few countries using also supervisory bilateral data. Market contagion (which operates through prices) was examined in several advanced economies, relying on variety of tools such as CoVaR (2), the Contingent Claims Analysis (CCA)-GVAR approach, and Joint Distress indicators.

8

The publication of both DARs and FSAP Technical Notes is voluntary for the country.

9

This discussion excludes AML/CFT assessments, which in principle need to accompany every FSAP, but could take place on a stand-alone basis and could be conducted by the Financial Action Task Force (FATF) or FATF-style regional bodies. In practice, full compliance assessments with the AML/CFT standard were rarely conducted at the same time as the FSAP missions. In its review of the Fund’s AML/CFT policy in March 2014, the Executive Board confirmed that FSAPs should include a timely and accurate discussion of AML/CFT issues based on comprehensive assessments, targeted updates or, where this is not possible, other sources of information.

10

Targeted BCP DAR for Russia and targeted IOSCO DAR for Canada.

11

This implementation rate is somewhat higher than the one documented in FSAPs. This may be explained by the fact that the two samples are different: survey responses report implementation by country, while the staff reports implementation by individual recommendations.

12

Staff has the option to submit to the Board an FSSA for consideration on a stand-alone basis. This option has been exercised in a few instances, mainly to avoid a long time gap between the FSAP discussions with the authorities and Board consideration (in case, for example, of an unexpected delay of the Article IV). Staff intends to continue exercising this option sparingly.

13

The experience with the first round of mandatory financial stability assessments under the FSAP was reviewed in the paper on “Mandatory Financial Stability Assessments Under the FSAP – Update” in November 2013.

14

Two new standards have recently been established in the area of safety nets: a standard on deposit insurance by the International Association of Deposit Insurers (IADI), and the FSB’s Key Attributes for Effective Resolution Regimes for Financial Institutions. But these standards have not yet become a regular part of the FSAP, although they are used as benchmarks for the work of the FSAP on financial safety nets.

15

These will include scenario calibration, defining the perimeter of stress tests, bottom-up stress test design, credit risk, liquidity risk, sovereign risk, and capturing cross-border exposures.

16

This data hub was launched in March 2013 by the FSB in the context of the G-20 Data Gaps Initiative.

17

See Chapter III of “2014 Triennial Surveillance Review—Background Studies,” July 2014.

18

Even raising the average annual number of FSAPs to the level of the previous decade (about 20 FSAPs per year, with all the additional FSAPs taking place in non-systemic countries) would not address the fundamental underlying tension. As an illustration, staff has estimated that this step would cost an additional US$2.75 million per annum and reduce the average gap between FSAPs for non-systemic jurisdictions from the current 14–15 years to 9-10 years.

Review of the Financial Sector Assessment Program—Further Adaptation to the Post-Crisis Era
Author: International Monetary Fund