2008 Triennial Surveillance Review - Background Information and Statistical Appendix

Country surveillance constitutes an essential part of the IMF's mandate to oversee the international monetary system and to monitor the economic and financial policies of its 185 member countries. The IMF's Executive Board conducts regularly scheduled reviews of country surveillance (the Triennial Surveillance Review) to consider ways to improve its effectiveness. The 2008 review is the first such review since the Executive Board approved, in June 2007, a new Decision on Bilateral Surveillance. This Decision affirms that the focus of bilateral surveillance is on those policies of members that can significantly influence present or prospective external stability. The review focused on the implementation of country surveillance in the recent past, as presented in the following set of papers: • The overview paper presents the main findings and priority areas for further work. The review finds that stakeholders hold the quality of IMF surveillance in high regard, but that improvements should focus on risk assessment, integration of macroeconomic and financial sector surveillance, multilateral perspectives (cross-border spillovers and cross-country analysis), and exchange rate assessments. The priority areas identified in the review served as key background for the preparation of the IMF’s Statement of Surveillance Priorities (SSP). • The Thematic Findings (Supplement 1) provides supporting analysis on the implementation of bilateral surveillance in the recent past and, particularly, on the appropriateness of its focus and its analytical value added in particular areas, including the overall “health check”, exchange rates, financial sector issues, cross-country analysis and cross-border spillover analysis (including a case study of surveillance in the run up to the subprime crisis), the degree of candor and evenhandedness in surveillance, and the effectiveness of its communication. • The Background Information paper (Supplement 2) provides further information, including a description of review methodologies, and results including interview findings, surveys of various audiences, and supporting data on the quality of consultation documents. • The External Consultant’s Report provides an independent view of IMF surveillance in Europe.

Abstract

Country surveillance constitutes an essential part of the IMF's mandate to oversee the international monetary system and to monitor the economic and financial policies of its 185 member countries. The IMF's Executive Board conducts regularly scheduled reviews of country surveillance (the Triennial Surveillance Review) to consider ways to improve its effectiveness. The 2008 review is the first such review since the Executive Board approved, in June 2007, a new Decision on Bilateral Surveillance. This Decision affirms that the focus of bilateral surveillance is on those policies of members that can significantly influence present or prospective external stability. The review focused on the implementation of country surveillance in the recent past, as presented in the following set of papers: • The overview paper presents the main findings and priority areas for further work. The review finds that stakeholders hold the quality of IMF surveillance in high regard, but that improvements should focus on risk assessment, integration of macroeconomic and financial sector surveillance, multilateral perspectives (cross-border spillovers and cross-country analysis), and exchange rate assessments. The priority areas identified in the review served as key background for the preparation of the IMF’s Statement of Surveillance Priorities (SSP). • The Thematic Findings (Supplement 1) provides supporting analysis on the implementation of bilateral surveillance in the recent past and, particularly, on the appropriateness of its focus and its analytical value added in particular areas, including the overall “health check”, exchange rates, financial sector issues, cross-country analysis and cross-border spillover analysis (including a case study of surveillance in the run up to the subprime crisis), the degree of candor and evenhandedness in surveillance, and the effectiveness of its communication. • The Background Information paper (Supplement 2) provides further information, including a description of review methodologies, and results including interview findings, surveys of various audiences, and supporting data on the quality of consultation documents. • The External Consultant’s Report provides an independent view of IMF surveillance in Europe.

I. Methodological Framework for Assessing the Effectiveness of Surveillance1

1. An important contribution of the 2008 TSR is the establishment of a strengthened methodological framework for assessing the effectiveness of Fund surveillance (EFS). The 2008 TSR is the first opportunity to establish, implement, and fine-tune the methodological framework for EFS assessment discussed by the Board in February 2007. This background paper presents the framework and reports on its implementation in the 2008 TSR.

The EFS assessment framework

2. The framework is based on a working definition of EFS and a set of seven key diagnostic questions that reflect prior work on the priority quality attributes of surveillance (Appendix). The EFS working definition is that surveillance is effective when it provides, to each member country and to the international community, the information and policy advice best-suited to help preserve global financial stability and the external stability of each member country. Whether the information is used and advice followed is critical for ultimate effectiveness, but largely beyond the Fund’s control. The diagnostic questions, which derive from staffs earlier work, endorsed by the Board in February 2007, emphasize substance, quality, and outcomes over process, quantities, and outputs. They reflect the understanding that priority quality dimensions of effective surveillance are relevance, appropriateness, practicality, candor, and evenhandedness. The questions (which are supported and clarified by more detailed sub-questions) are:

(a) Is surveillance focused on the issues that are most critical to its mandate?

(b) Is the analytical content of surveillance of sufficiently high quality to add value reflecting the Fund’s unique expertise?

(c) Is surveillance candid?

(d) Is surveillance consistent and evenhanded?

(e) Is surveillance effectively communicated to its key audiences?

(f) Is surveillance having an impact?

(g) Is surveillance cost-effective?

3. The framework also entails an agreed set of sources of information and techniques to answer the diagnostic questions, allowing for triangulation. The three categories of information deemed most relevant for EFS assessment are output quality, outcomes, and impact. In principle, TSRs would collect information on all three, but this one focuses on the quality of outputs and on value-added, which is a form of impact. In the future, information on outcomes and impact will also be collected, drawing, inter alia, on surveillance agendas. The agreed techniques and their use in the 2008 TSR are covered in detail in the next section. A key feature of the approach to information and techniques is that it supports triangulation. That is, when possible, findings about particular aspects of EFS are drawn from different sources of information or different audiences, using a variety of methods, and compared for consistency. For example, different internal and external audiences were asked similar questions about the comprehensibility of staff reports.

4. The framework provides the basis for assessing the quality of surveillance over time and across countries and for assessing progress against time-bound surveillance priorities. Findings under the main diagnostic questions establish performance benchmarks for future TSRs. Alongside such permanent components, the framework also contains specific provisions for assessing performance against the more time-bound priorities outlined in the three-year Statements of Surveillance Priorities (SSP). Collected information is analyzed not only in aggregate, for the membership as a whole, but also separately for key country groupings (e.g., advanced/emerging/developing; area departments’ coverage; floaters and peggers etc.). This permits the identification of systematic differences in treatment, which can then be examined against the background of underlying economic realities to look for suggestions of bias or lack of evenhandedness.

Assessment techniques and their application in the 2008 TSR

5. The EFS assessment framework mainly features the following techniques. For all of the techniques covered, the reader is referred to the Statistical Appendix for additional and more detailed information on how they were implemented in the 2008 TSR, as well as the results of their application.

Standardized qualitative assessment of a representative sample of staff reports

6. A representative sample of Article IV reports were internally reviewed in order to collect standardized qualitative information across the main diagnostic questions.

The review was conducted by PDR economists in a division not involved in the review process of individual country surveillance reports. The universe from which the sample was selected was defined as the 88 Article IV staff reports discussed by the Board in the period July 1, 2007 to February 28, 2008. A sample of 50 reports/countries was reviewed, rather than the universe, to allow for more qualitative and in-depth review of staff reports, compared to earlier surveillance reviews. Detailed questionnaires corresponding to the main diagnostic questions were developed by the team to collect standardized qualitative information on the reports.

7. Several precautions were taken to maximize consistency across reviewers. Pairs of economists from the review team “test-drove” the questionnaire for two countries per pair. That is, pairs of economists reviewed a given set of two countries against the full questionnaire and then met to compare their responses and exchange information about problems encountered and the solutions adopted. This measure helped develop common understandings across the reviewers on how to interpret the questions and implement the questionnaire uniformly for staff reports reflecting a range of country and operational circumstances; this in turn helped ensure consistency of assessment across reviewers. Furthermore, when the internal review was completed, the team member responsible for a given part of the questionnaire checked the results across the 50-country sample for accuracy and consistency. Countries were divided up among reviewers in a way that ensured that each reviewer reviewed a group of countries that was diverse in terms of area departments, income level, and Fund program status.

8. PDR research assistants (RAs) reviewed the reports to provide the perspective of “an average college-educated reader.” RAs who were relatively new to the Fund reviewed the executive summaries and staff appraisal of staff reports and answered several overarching questions on the readability of reports and the clarity of key conclusions provided by the report in specific areas. These questions and the RAs’ responses provided checks for questions in thematic parts of the questionnaire on candor and evenhandedness, focus, and exchange rates.

Case studies

9. Case studies are useful for drilling down, i.e., for undertaking thorough and comprehensive analysis of developments where country, temporal, or other specific circumstances need to be taken into account, as well as for topics that require gathering information beyond that available in staff reports; additional sources considered in case studies could include additional internal documents (e.g., briefing papers) and external ones (e.g., media coverage of Article IV consultations, relevant/concurrent analyses produced by other international institutions, credit rating agencies, consultancies, or research institutes), and interviews. Five case studies were conducted in the 2008 TSR (see Box 1).

Topics for Case Studies in the 2008 TSR

(i) The quality, consistency and candor in exchange rate analysis under the 2007 Decision;

(ii) Fund surveillance of countries concerned in the run up to the subprime crisis and its aftermath;

(iii) Consultations with intensive involvement of the Monetary and Capital Markets Department (MCM) following the Rajan-Lipschitz-Caruana Task Force;

(iv) Focus of staff reports’ analytical background work, including the use of crosscountry analysis, based on a review of Selected Issues Papers produced for the entire membership;

(v) The treatment of real-financial linkages and regional cross-country spillovers in a region chosen as globally systemic and highly integrated, comprising a diverse mix of countries, and ideally subject to analysis by many “competitors” (to allow a better analysis of value-added).

Stakeholder surveys

10. Surveys are particularly helpful to refine diagnostics, understand the drivers of certain phenomena, and cast light on desirable remedies to problems identified separately. Key groups are: country authorities (to gauge their views of surveillance of their country); Executive Board members (with emphasis on their views as proxy for “the international community” on surveillance of countries other than the one(s) they represent); Article IV mission chiefs (as “producers”); and the public, including financial market participants, opinion-makers (media, think tanks), and civil society organizations (CSO). Both anonymous surveys and structured interviews by staff or, where more appropriate, an independent consultant, are expected to be used in the EFS assessment framework over time. For the 2008 TSR, stakeholder consultations were carried out through surveys and, in some cases, interviews with all of the above groups as detailed in the Section III.

11. Problems encountered. Response rates for the surveys of think tanks and CSOs were so low (320 think tanks contacted yielded a 3 percent response rate; and 1000 CSOs contacted a 0.3 percent response rate) that feedback from these sectors was not incorporated, and in the future other means of obtaining feedback from these audiences will need to be explored. Ex post, scope for improving the parallelism of questions on similar topics in surveys of different audiences was recognized (for example, in the question on the overall quality of analysis, the breakdown of different areas of analysis was not identical across surveys). A more general problem is that, ex post, the sample of respondents to a survey or participants in interviews may not be fully representative of the original population targeted.

Given that participation is voluntary, little can be done about this, beyond making the results available by relevant subsets within a given targeted stakeholder group, when possible. In the case of the survey of Executive Directors, the breakdown of respondents across country categories in not known, as to preserve their anonymity, it was felt necessary to refrain from asking respondents questions related to the country(ies) they represent. Thus while the overall response rate (58 percent) was satisfactory, it is an open question whether the responses really capture the diversity of views of the full Board.

Studies by independent consultants

12. Using independent consultants to investigate particular questions, themes, and/or audiences provides a different governance structure to assessments conducted by staff. Such consultants may conduct a survey, review documents, or use other sources of information, sometimes in combination with their own background or expertise. They provide views and analysis that have the potential advantages of (i) eliciting more candid feedback from both internal and external persons who are interviewed or surveyed and (ii) providing independent, external views on surveillance. For the 2008 TSR, an independent consultant was hired to conduct one of the case studies ((v) in Box 1), and another to conduct structured interviews with country authorities. Both of their reports are provided as part of the TSR documentation.

Indicators

13. Output indicators—numerical indices summarizing information on outputs and output quality—can be useful, but should be drawn on sparingly and with caution, given their weaknesses. For the 2008 TSR, a proximity indicator that measures the sharpness of the focus of SIPs, working papers, and surveillance agendas on issues most likely to be relevant to external stability was developed as part of the analytical underpinnings of surveillance case study. Also, an indicator on the extent, quality, and impact of cross country analysis in SIPs was developed as part of the thematic study of cross country analysis and spillovers.

Reporting in Article IV staff reports of past surveillance advice uptake

14. The requirement for staff reports to report on follow up to past surveillance advice earns mixed reviews as part of the EFS assessment framework. Since the 2002 review of surveillance, staff reports have been required to include a brief assessment of the authorities’ response to the key policy challenges identified in previous consultations—a requirement instituted specifically with an eye on providing information on the effectiveness of past surveillance. The staff paper for the February 2007 Board seminar noted pitfalls of this measure, notably that the results are not detailed enough to allow conclusions to be drawn on the effectiveness of surveillance on a country-specific basis, let alone to be aggregated to form a membership-wide assessment. Also, it may be inappropriate and distortionary to assess the quality and impact of staffs past advice in the same forum where the staff’s current assessment and policy advice is presented. The measure also suffers from the pitfalls of self-assessment, since the assessment of uptake of advice is done by the same department and staff that developed the advice. The 2004 surveillance review found that the reporting in many cases was “cursory, partial, and pro-forma.” The present analysis finds that only 60 percent of staff reports include this “required” information, and in only about one-half of these cases is the information reasonably substantial, i.e., one-half page in length or more.

15. Further experience has clarified that this reporting serves two distinct functions. For some consultations, it can provide useful background information to Executive Directors; but it was instituted as a universal requirement in Article IV reports not for this purpose but as a means of EFS assessment. The first function is clearly valuable, while as the preceding paragraph indicates, the second is not well served. Staff therefore proposes that it be understood that the sole purpose of the reporting is the first function. Therefore staff could use judgment in deciding whether the information is useful background but would not be required to present the information as a form of EFS assessment. These sections of staff reports do not say anything about whether the authorities’ actions were influenced by surveillance, and there is some risk of selective reporting when key areas of advice are not identified ex ante. In future TSRs, staff will be able to draw on the three-year surveillance agendas to assess actual outcomes against those intended ex ante.

Assessing performance against the SSP

16. As part of the TSR, the performance against priorities established in the SSP will be assessed. It is expected that this assessment will draw on the assessment tools outlined in the previous section. The SSP provides that at the time of each TSR, the Managing Director will report on progress in attaining the SSP priorities, management’s and staffs contributions, and the factors that impeded progress. The specific questions set out in the box accompanying the SSP (see Annex in the overview paper) will provide useful guidance to develop specific performance benchmarks, and it is expected that assessment of the SSP priorities will draw on the various tools outlined in the section on assessment techniques above.

Basic Framework for Assessing the Effectiveness of Surveillance

Object of the assessment. Surveillance is effective when it provides, to each member country and to the international community, the information and policy advice best-suited to help preserve global financial stability and the external stability of each member country. Whether the information is used and the advice followed is critical for ultimate effectiveness, but largely beyond the Fund’s control. What the Fund is responsible for in making surveillance effective is the focus of its efforts, the richness of its analysis and advice, and the quality of its communication. It is these that enable the Fund to maximize, within its mandate, its relevance to its members. The actual impact of surveillance will also depend on the Fund’s legitimacy—a dimension largely related to governance, but for which evenhandedness in surveillance is also important.

Key questions. This broad conceptual framework can be translated into a list of overarching questions that should constitute the backbone of future TSRs. Relative emphasis on the different aspects could vary over time as circumstances require. The key questions—which conceptually apply to all forms of surveillance: bilateral, regional or global—are listed below (focus areas for the 2008 TSR are italicized).

(a) Is surveillance focused on the issues that are most critical to its mandate?

  • ❖ Is surveillance (and underlying analytical work) focused on global and single country domestic and external stability, and their interdependence?

  • ❖ Is analysis focused on areas where risks are greatest?

  • ❖ Is surveillance timely in its analysis of issues?

(b) Is the analytical content of surveillance of sufficiently high quality to add value reflecting the Fund’s unique expertise?

Analysis of risks and real-financial linkages

  • ❖ Does surveillance provide adequate health checks, including sharp analysis of exchange rate issues ?

  • ❖ Does surveillance adequately integrate the analysis of economic and financial risks?

  • ❖ Is surveillance good at diagnosing risks and anticipating adverse developments?

Multilateral perspective

  • ❖ Does surveillance help members understand the international context and implications of each member’s policies (both inward and, where relevant, outward spillovers)?

  • ❖ Does it make enough use of the Fund’s cross-country knowledge?

Quality of Policy Advice

  • ❖ Are surveillance recommendations appropriate and well-tailored to the economic situation?

  • ❖ Do surveillance recommendations have enough operational acumen? Or are they sometimes impractical given country circumstances, or too generic to be useful?

(c) Is surveillance candid? 2

  • In assessing domestic and external stability, does surveillance too often wear rose-colored spectacles (e.g., assume problems away)? Does it tend to focus too much on central scenarios at the expense of risks?

  • Does surveillance tend to couch the description of problems (e.g., vulnerabilities, exchange rate misalignment) or recommendations in unclear or overly hedged language?

(d) Is surveillance consistent and evenhanded?

  • Is the quality of surveillance (as defined in a, b and c above) consistent across countries and country groups?

  • ❖ Are surveillance policy recommendations consistent across countries (i.e., similar for countries in similar conditions)?

  • Is the degree of candor in assessing risks and recommending policy adjustments consistent across countries?

(e) Is surveillance effectively communicated to its key audiences?

  • ❖ Are the key vehicles available (mission concluding statements, staff reports, summings up, published bundles) effective and efficient in conveying the messages of surveillance to their key audiences—country authorities, the Board, the broader international community, and the public? In particular, are staff reports and bundles (in their current format) well-suited to communication with the Board, the international community, and the public?

  • ❖ Are other forms of communications and outreach used sufficiently and effectively?

  • ❖ Is surveillance communicated in a sufficiently timely fashion to its various audiences?

  • Separately from the vehicle, are surveillance messages presented in a convincing, user-friendly manner (e.g., well argued, prioritized)?

(f) Is surveillance having an impact?

  • ❖ Is surveillance significantly informing policy decision making? the public debate? markets?

  • ❖ Are the policy recommendations of surveillance followed?

(g) Is surveillance cost-effective?

  • ❖ Do the modalities of surveillance allow it to deliver the best value at the lowest cost?

II. Report on InterviewsWith Country Authorities3

1. Interviews were conducted with senior officials from 19 countries.4 According to the WEO classification, 7 countries belong to the group of advanced economies and 12 to the group of emerging market and developing countries.5 The interviews were based on a brief questionnaire distributed to interviewees before the meetings and concentrated primarily on the key questions related to implementation of surveillance to be examined in the 2008 TSR, namely: appropriateness of focus, given the mandate of surveillance and the circumstances of the member; analytical value-added; and the effectiveness of current modalities of bilateral surveillance, especially the way in which its messages are communicated to the authorities and other audiences in the country.

2. Almost all of those interviewed said IMF surveillance added significant value, through several channels: (i) as an integrated macroeconomic assessment from a global perspective; (ii) as a test against the authorities’ own judgments; (iii) as a transparent source of standardized information and, for some countries, (iv) as a source of specific policy advice. On the latter, most interviewees from advanced economies said surveillance generally did not say anything they had not already heard about policies, but it provided a useful comprehensive synthesis. This was especially appreciated by interviewees from smaller advanced economies. Many interviewees from emerging markets noted that the uniqueness of IMF surveillance as an external assessment was less than before, which highlighted the importance of adopting surveillance outputs so as to maintain IMF relevance. The messages from interviewees across different groups of countries were broadly similar, except where noted below.

Appropriateness of focus

3. Many regarded the 2007 Decision as a welcome legal framing of what IMF surveillance had already been doing in recent years. Therefore, in judging its implementation, they did not expect to see radical changes in approach. In any event, it was still too early to judge the full extent of any changes on those countries that had not yet completed a full cycle under the new framework.

4. Almost all those interviewed supported the new focus, but stressed that it had to be tailored to specific country circumstances. In particular, the new framework should not become another “check list.” For example, several of those interviewed had the impression that some surveillance reports were overemphasizing the discussion of regional spillovers or deviations of exchange rates from “equilibrium” even when the evidentiary basis was limited and other issues were more important.

5. Many of those interviewed thought that the coverage of exchange rate issues still lacked uniformity. For example, there was wide disparity in the coverage of CGER analysis across surveillance reports with different advanced and emerging market economies. Many emphasized the need for new operational guidelines on surveillance to ensure greater uniformity. Some thought the treatment of exchange rate issues in the bilateral surveillance reports for member countries of currency unions as integrated as those of the EU was especially problematic, since they did not think the current account balance of individual member countries of the union contained useful information about the exchange rate.

6. Interviewees thought the FSAP approach had added major value and had helped to deepen surveillance of financial sector issues, but the follow-up could be improved further. The integration with surveillance can be patchy and needs to move further from a follow-up of static assessments against standards to a more dynamic analysis of key risks and risk management. A number of interviewees said the insufficient technical expertise on financial sector issues of regular Article IV missions had reduced the value of some follow-up discussions on the financial sector.

7. There was broad support for deeper surveillance coverage of real-financial sector linkages. Many interviewees recognized that this involved significant analytical challenges, but that the Fund should take the lead in analyzing such linkages from a global perspective. For example, more attention should be given to approaches that treated endogenously the consequences of various shocks for credit expansion and growth. One interviewee noted that the FSAP Handbook already set out possible analytical approaches and another suggested that greater efforts could be made to incorporate World Bank analysis of the real sector into surveillance exercises.

8. Many interviewees stressed the need for realism about the Fund’s role in predicting crises. Financial markets are highly dynamic and the IMF neither could nor should be expected to detect all potential crises, especially if domestic supervisors failed to do so. The Fund role should be to highlight the most significant vulnerabilities as it sees them, with the focus on global inter-linkages. In this respect, some interviewees thought that the Fund had been slow to analyze the potential ramifications of the U.S. mortgage-financing crisis.

9. While recent surveillance reports were moving in the right direction, many interviewees would like to see even more emphasis on distilling relevant experience from other member countries. Several interviewees mentioned specific examples where greater emphasis on analyzing how other countries had addressed certain policy issues would have been useful (e.g., dealing with the macro challenges of terms of trade appreciation in advanced economy natural resource exporters; detailed policy responses to capital inflows in emerging markets). They noted, for example, that more could be done in bilateral surveillance to draw out country-specific policy lessons both from multilateral surveillance analysis and from bilateral surveillance of other countries facing similar policy issues. (See the next section for specific examples.)

10. A number of those interviewed thought that bilateral surveillance still gave too much attention to general structural reform issues, where the IMF often had little new to add. Several mentioned pro forma and rather superficial discussions of trade policy as an example. Fund surveillance should be even more selective and go deeper with topics chosen according to country circumstances and linked more dynamically to future macro challenges. Some interviewees noted that the Executive Board had to contribute to this selectivity, by refraining from new mandates outside the core competencies and not criticizing staff when greater selectivity means some issues are not covered.

11. While there was broad support for a greater emphasis on regional linkages where these were substantial, the relevance of a more “regional” approach to country surveillance depended on each country’s circumstances. Interviewees from countries where regional trade and financial linkages were not large said greater regional surveillance risked distracting from more important activities. Some from relatively smaller countries, including among the advanced economies, said a shift to regional surveillance reports could risk crowding out some issues of importance to them. In the words of one interviewee, earlier attempts to “streamline” surveillance, with distinctions between systemic and non-systemic countries, had not been well thought through, since the integrated assessment of IMF surveillance was often of greater value added to the latter countries. In countries where regional linkages were substantial, interviewees saw greater regional surveillance as a useful supplement to, rather than a replacement of, bilateral surveillance. In such cases, the main value added would come from an exploration of economic and financial spillovers within the region rather than from comprehensive regional surveys.

12. In countries with IMF-supported programs, a broader surveillance dialogue is still crowded out by the continued focus on detailed program implementation during Article IV missions. Interviewees from developing countries who commented on this issue said a great surveillance focus on the forward-looking macro challenges of enhancing growth was needed, rather than detailed reporting on program implementation. This would require greater selectivity, with some interviewees welcoming what they saw as a recent reduction in surveillance attention given to detailed social expenditure and poverty issues.

Analytical value-added

13. Many of those interviewed thought the exchange rate analysis (specifically, the CGER exercise) was not yet sufficiently robust to justify the weight put upon it in many recent surveillance reports. In particular, a good conceptual definition of ‘external stability’ was still lacking to make the framework operational. While most supported the Fund’s continued efforts to develop its analytical tools in this area, the language used in reporting results needed to be very cautious. Greater uniformity of treatment across countries was also needed. Several interviewees also said the Fund was often too quick to translate an analysis of price competitiveness into exchange rate policy advice, whereas many other policies also influenced cost structures. In contrast, some others stressed that, while the underlying analysis was inevitably imperfect, the Fund as an institution was still not sufficiently forceful in expressing its conclusions when there was evidence of major misalignments.

14. Many interviewees identified the Fund’s fiscal analysis as a particular source of value-added. Longer-term fiscal sustainability exercises that took account of ‘hidden’ fiscal costs in the broader public sector had often been helpful in adding to the transparency of the public policy debate, even in a number of cases where the underlying facts were already well-know to the authorities.

15. Treatment of financial sector issues in surveillance has greatly improved in recent years, in large part because of the FSAP, but further improvements in the analytical toolkits used in country work were needed if the Fund were not to fall behind the curve. Some interviewees said that the follow-up of the FSAP process in subsequent missions had not had the same ‘level’ of dialogue as on more general macro policies; in some cases, there as too much of a ‘checklist’ approach to following up on minor details rather than considering risks and risk management in a holistic manner. One interviewee suggested that, while it was unrealistic to expect the Fund to predict specific crises, it could do more to explore in advance possible spillover routes of financial shocks and the influence of different institutional settings. Another noted that, while the FSAP Handbook already had a useful chapter on approaches to treating endogenously credit expansion and economic growth, most FSAPs relied on simpler stress tests; consequently, the ‘next generation’ of FSAPs and surveillance needed to upgrade the analytical tools applied.

16. Many said they would like to see more evidence-based policy advice, adapting global analysis and cross-country experience to country-specific circumstances. One positive example cited of such analysis was the recent study of the impact of high oil prices, which had used the Fund’s comparative advantage to combine a global assessment with regional and country specifics. One issue mentioned where a number of interviewees from emerging market and developing countries would like to see more concrete, operational advice was on the interlinked issues of capital market reform, financial sector liberalization, and foreign exchange markets.

17. The timing of IMF analytical inputs influenced their effectiveness. For example, a number of those interviewed said the Fund’s analysis of the fallout from the sub-prime crisis, while comprehensive, had been less timely, and therefore less helpful to them in providing a quick assessment of the potential consequences for their own country than reports by others, including investment banks. A number of those interviewed said they relied more on reports produced by a few major central banks for information on developments in key advanced and emerging market economies, because of their greater frequency. Moreover, the timing of IMF analysis needed to determined as part of a broader strategy to maximize the input to the domestic policy debate, not supply-driven by the interests and capabilities of the Article IV team. Several interviewees quoted examples where IMF analysis of an issue had been of high quality, but had come too late to influence the domestic policy debate. However, many said that pre-mission consultations on what were likely to be the most critical policy issues for surveillance had improved the relevance of the dialogue and of thematic background reports in Selected Issues papers.

18. While acknowledging recent improvements, many of those interviewed said bilateral surveillance needed to take greater account of country-specific institutional elements. In addition to political economy considerations (e.g., advice to implement across the-board reductions in subsidies was often not realistic), a number quoted examples from the financial sector where they thought that greater familiarity with the institutional setting would have improved the quality of the policy advice.

19. A number of interviewees from emerging market and developing countries said the Fund approach to analyzing macroeconomic issues in their countries—built around the financial programming framework—was behind the times and needed updating.

More sophisticated and ambitious approaches, especially to analyzing real-financial sector linkages, were potentially available (see earlier discussion).

Modalities of bilateral surveillance

20. Interviewees commented on two aspects of surveillance modalities: (i) the way in which IMF surveillance messages were communicated (i.e., the frequency, speed, and product design of bilateral surveillance outputs and the quality of communications), and (ii) factors affecting the efficiency of surveillance.

21. No strong demand for a radical change in the types of surveillance output emerged from the interviews, but many thought the time lags needed to be shortened.

Many said that the key value added—in terms of influence on policymakers—came at the time of the concluding statement of the Article IV missions and any associated briefings of high officials. Production of staff reports and the subsequent Board discussion several months later generally had limited additional impact. Many thought the time between the end of the staff visit to the country and the subsequent Board discussion and release of the staff report was too long. Some also suggested that less time and resources could be devoted to preparing the formal staff reports, since the mission’s concluding statement already contained the key messages. One interviewee suggested that resources could be shifted to producing shorter, but more frequent and hence more timely, policy-oriented notes on key groups of countries.

22. In contrast, many interviewees argued strongly that the way in which surveillance messages were communicated in the various surveillance outputs needed substantial further improvement. Many said that, while the drafting of Article IV reports had improved in recent years, more work was needed to produce shorter, sharper, and more focused reports. Several gave specific examples where they had revamped the language of staff reports in order to extract salient messages to brief senior policy officials or parliamentarians in a form that would have more influence on the domestic policy debate. One interviewee said the informal approach of many of his colleagues was to “only read the Boxes,” since they tended to contain most that was original in a staff report.

23. The language of the Board Summing Up was behind the times, in terms of modern approaches to policy communications, according to a number of those interviewed. Several said that the language used was too coded and cryptic and therefore less accessible outside a narrow circle --”as if the Fund is talking to itself,” in the words of one interviewee. One Finance Minister said the Summing Up was an important document for his country, since he used it to convey the IMF messages to his Cabinet colleagues; however, he had to ask his staff to “translate” it into plain English before it could serve this purpose. Several of those interviewed suggested the Fund could learn more from recent advances in the way some central banks conveyed their policy messages (e.g., in the context of inflation-targeting frameworks). The conventions used to convey the number of Directors expressing a particular view (‘some,’ ‘a number of,’ ‘a few,’ etc.) were judged especially opaque by many interviewees, who said such language detracted from the clarity of the surveillance message.

24. A variety of views were expressed on the Fund’s transparency initiatives. Most of those interviewed favored publication of country surveillance reports, but a number argued that the emphasis on making public the Fund’s advice, except on those issues that were highly market sensitive, had gone too far and risked undermining the Fund’s role as a confidential advisor in their countries. This group thought the appropriate degree of transparency should be decided according to country-specific circumstances. In any event, many said that more scope should be given for the authorities to present their views when they disagreed with the Article IV mission’s concluding statement, since the opportunity to express such views at the time of the Board discussion could, in their view, come too late.

25. While many stressed that surveillance was a core activity of the Fund and needed adequate resources, several suggestions for improving the efficiency of surveillance operations were made during the interviews, including (i) greater use of modern communications prior to the Article IV mission to reduce the large amount of time devoted to technical questions; (ii) more use of videoconferencing with relevant HQ experts so that scarce IMF expertise (e.g., on the financial sector) is used efficiently during surveillance; (iii) devoting more in-house research resources to developing analytical toolkit modules for country teams (e.g., for analyzing real-financial sector linkages); (iv) developing more systematically a “knowledge data bank” on different country experiences with a select group of policy issues that were likely to arise frequently in coming years (possibilities mentioned included responses to capital inflows, operational intricacies of exchange markets, and comprehensive fiscal balance sheets); and (v) further efforts to reduce staff turnover on Article IV missions, although many recognized recent progress in this area.

Country Interviews Drawn Upon for this Report

article image

III. Supporting Information for the Subprime Case Study

1. This section provides more detailed, but not exhaustive, factual supporting information on staff analysis produced in the period leading up to August 2007, organized by country case. References to paragraph numbers in the main text are to Supplement 1, Chapter VII.

A. United States6

2. Many of the macroeconomic and institutional developments that led to the buildup of vulnerabilities in the run-up to the “subprime crisis,” as well as specific vulnerabilities, risks and risk transmission channels were identified ex ante (¶95-97). Analysis of the financial sector trends has been presented in different issues of the GFSR and picked up/developed in Staff Reports(SRs)/Selected Issues Papers (SIPs):

  • Credit rating agencies and pricing of structured products. GFSR, Apr 2006: “In particular, structured credit products are likely to suffer more severe, multiple notch downgrades paths...” (p. 61); “... a more differentiated rating scale may be useful for structured products” (p. 61); SR07*¡6: “In hindsight, the losses incurred on bundled subprime mortgage securities indicate that rating agencies and investors underestimated the weakening of lending standards and its impact in 2005-06”; SR07*¡25: “...credit rating agencies are playing an increasingly important role in how complex financial products are structured, which could involve potential conflicts of interest in that rating agencies have an interest in facilitating continuing investor appetite for such products to generate fees. “.

  • Securitization model. SIP06, Ch1,\3:”With securitization, banks and other mortgage originators have been able to shift significant amounts of credit and market risks to MBS holders...”; SR07\6, “The originate-to-distribute model, however, could be exacerbating incentive problems in financial markets. The intermediaries at various stages of the process— originators, securitizers, and pool managers—are remunerated primarily through fees and often bear only limited long-term balance sheet exposure to the underlying assets. This can reduce their incentives to maintain loan quality.”

  • Pro-cyclicality of risk management models. GFSR, Sept 2003: Developments such as value-at-risk models and the ratings-based approach in Basel II greatly improve risk management. They also, however, carry the risk of pro-cyclicality and amplifying volatility by requiring asset sales as volatility increases’” SIP04, ChVI,\16: “Supervisory assessments would need to include efforts to understand the systemic implications of risk management systems and instruments, and to identify critical factors for the liquidity of markets in which hedging instruments are traded.”

3. Staff flagged several weaknesses in the US financial system and regulation:

  • Systemic risks and moral hazard concerns related to the activities of the government sponsored enterprises (GSEs) Fannie Mae and Freddie Mac. SR04, Box 6: “Besides their size, the GSEs are important to financial stability for several reasons: About 30 to 50 percent of their liabilities are short-term; their role as one of the largest counterparties in the interest-rate and swaption market...” “The potential for future problems is exacerbated by the continued perception of an implicit government guarantee,” SR05,\44:”The mission supported the Administration’s plans for establishing an independent regulator, involving limits on the size of GSE portfolios and allowing the regulator to set capital requirements, design stress tests, and place a financially-weak GSE into receivership,” SR06, *¡55: “Action is still needed to carry forward the Administration’s proposals to strengthen the supervision of the housing GSEs and limit the size of their balance sheets so as to contain systemic risk in financial markets.”

  • Lax lending standards in the subprime market. SR06,*¡43: “Supervisory agencies indicated that they were close to finalizing a guidance note (subsequently issued in May) requiring banks to strengthen risk management with regard to riskier mortgage products, a move welcomed by staff.”

  • Fragmentation of the regulatory framework. SR07,\51”With several federal and many state regulators overseeing this evolving system, the new emphasis on improving regulatory effectiveness is welcome.”

  • Consumer protection. SR07*¡25 “Fed officials explained that, in light of the subprime shakeout, they are examining whether regulations implementing the Truth in Lending Act could be modified to address concerns of predatory lending, including by nonbanks that are not covered by guidance from federal regulators. Staff supported these efforts, but suggested that, given the importance of state-registered nonbanks in originating mortgages, federal legislation might also be needed to improve the consistency of enforcement. “

4. Staff also warned about possible event risks, including:

  • a sharp rise in delinquencies on mortgage loans in subprime and near prime (“Alt-A”) segments. SIP06,*¡9 As the housing market is beginning to cool, however, concerns are growing that payment resets on ARMs and nontraditional mortgages could shock many marginal households.

  • a drying up of liquidity in securitized asset markets. GFSR, Apr06: “ ...the diversity of participants within the different tranches of a CDO (or its capital structure) is often limited and secondary market liquidity is therefore often also limited” (p.56) “In the structured credit markets, we believe the risk of liquidity disturbances is material” (p.81).

  • a volatility spike. GFSR, Sept 03: “Excessive leverage often turns volatility into instability” (p. 5); GFSR, Apr07: “A volatility shock... could precipitate sharp portfolio adjustment and a disorderly unwinding of positions. The consequences of such a shock would be amplified by the rise in leveraged investment positions, the increased use of complex derivative instruments that remain untested in more volatile market conditions, rising portfolio exposure to illiquid instruments, and the prevalence of crowded trades” (p.29) “; SR07,*¡25: “...As discussed in the April 2007 editions of the Global Financial Stability Report and the Bank of England’s Financial Stability Report, a sudden rise in risk aversion could uncover unanticipated vulnerabilities, illiquidity in newer markets, and major losses in asset value.”

5. As the subprime crisis began to unfold, staff warned about similar vulnerabilities in other markets, e.g., leveraged loans: SR07,\7:”Staff see parallels to subprime mortgage developments in other market segments, including leveraged loans. In particular, the boom in leveraged buyouts is being funded through CLOs which have been in high demand from investors, and there is evidence that covenants in the underlying loans have eased. A turn in the credit cycle, especially if volatility and risk aversion rise, could expose financial vulnerabilities and unanticipated risk concentrations, with adverse effects on activity.”

6. Staffs analysis of the size and sources of cross-border risk transmission showed the importance of outward spillovers from the US through financial channels. See SIPs 07, Ch I: “Summary of Foreign Entanglements: Measuring the Size and Source of Spillovers Across Industrial Countries,” and Ch II “Summary of the Ties that Bind: Measuring International Bond Spillovers Using Inflation-indexed Bond Yields’”

What could have been done better?

7. Many issues were analyzed in depth, but the bottom line reached was overly optimistic (1199-102):

  • The likelihood and magnitude of a housing market correction. The US housing market started to falter in 2005-06. The risk of a correction was discussed in the SR04, SR05 and SR06. Staff subscribed to the official view of “no nationwide bubble”, on the basis of staffs own empirical analysis. SR04, \17:”They (officials) agreed with staff that some regional markets appeared overheated but observed that house prices in general were recovering from relatively sluggish increases during the 1990s and were broadly in line with disposable income,” SR05, \15: “Officials also noted signs of “froth” in the housing sector...However, the situation at the national level was less of a concern, and the most likely scenario was a flattening ofprices rather than outright declines... Staff agreed that house price stagnation was the most likely scenario.” The house price overvaluation was explicitly acknowledged after the correction had already started7 (SR06,^7:”Staff observed that, while conditions varied across regions, U.S. house prices seemed overvalued and a correction appeared to have started.” (Chapter 1 of the Selected Issues paper suggests “house _ prices are 1520percent above equilibrium. “)

  • The overall fragility of the US financial system, including the potential impact of a house price correction. During 05-06, the lack of concern about a nationwide housing bubble combined with the fact that banks’ loan portfolios seem to have been well diversified across regions supported staffs view that financial system was resilient to the potential real estate shock. SR05,*¡43:”Risks to banks from a correction in housing markets were judged relatively low. Despite froth in some regional markets and increasing use of riskier mortgage products, such as interest-only and adjustable rate loans, trends on a national level were less of a concern and loan portfolios were well diversified geographically,” SR06,\26:”Financial sector risks related to household borrowing appeared relatively manageable....,” SIP06,ChV,\18: “Financial soundness of Large Complex Banking Groups (LCBGs), as well as investment banks and insurers is found to have improved in 2003-05. Distance-to-Default (DD) measures are at multi-year highs, while weakening comovements of LCBG risk profiles point to diversification gains at a system level.” Even in 07, spillovers from subprime to the broader financial system were still viewed as being contained and systemic risk was still judged to be low. SR07,*¡4:”... Rising subprime delinquencies led to a jump in spreads on higher-risk mortgage-backed securities, but there has yet been little contagion outside of the near prime (“Alt-A”) segment of the mortgage market, reflecting the wide dispersion of risk and concentration of difficulties in specialist subprime originators... “

Known Unknowns, and Unthinkables

8. The full impact of the crisis would have been difficult to gauge, as it would have required data that was non-existent, or known unknowns: fl|103)

  • Incomplete information about potentially relevant risk exposures.8 e.g., staffs sensitivity analysis to the real estate shock on the LCBGs in Box 2, SIP06, ChV, focused only on retained loan book, not including MBS exposures; also, measures of leverage for LCBGs would have been more informative if they were computed on a consolidated basis (including both on and off-balance sheet positions), especially given that SIVs turned out to be not as remote as they were generally believed to be, as illustrated by the Bear Stearns’ case.

  • Difficulties in assessing certain risk exposures were acknowledged by staff. SR07,*¡50: “.. new instruments have made it more difficult to assess risks at a time when benign market conditions have encouraged risk taking and lower lending standards. Tightening financial conditions could expose unanticipated risk concentrations and links across markets. “ GFSR, Sept 03: “ Supervisors must continually improve the sophistication of their leverage measurement—both on and off-balance sheet—to keep up with market innovations” (p. 5).

  • Some of the information gaps were explicitly acknowledged as well: SIP06, ChV, “Informational lacunae are especially evident in the reporting of hedging and credit risk transfer activity;” Footnote 33: “The regulatory data do not, for instance, separate derivatives dealing positions from proprietary hedges, or clarify whether interest and exchange rate contracts represent net long or short positions, or identify holdings of collateralized debt and mortgage obligations by tranche;” \15:”...accounting data shed little light on growing risk transfer activity, while market prices cannot be assumed to perfectly reflect underlying risks,” GFSR, April 2006: “While structured credit products provide a wealth of market information, there remains a paucity of data available for public authorities to more quantitatively assess the degree of risk reduction among banks and to monitor where credit risk has gone”.. “such calculations require more detailed transaction data not currently collected by public authorities (including tranche-specific distribution data).”

9. The freezing up of the interbank money market and a sharp increase in liquidity funding risk for otherwise-solvent, systemically important core institutions were “unthinkable” and unforeseen by anybody fl|104). Any discussion of the counterparty risks tended to focus on exposures of regulated financial institutions to hedge funds. SR07,*¡23: “...staff were concerned about whether adequate due diligence—including of exposures of the core to hedge funds—was possible with limited hedge fund disclosure, whether risk management systems were sufficiently robust to a shock with consequences across a range of markets and if, given likely shifts in the investor base for hedge funds, consumer protection issues would become more pertinent.

Was the right policy advice given?

10. Staff mostly expressed support for policy measures already taken, or under consideration (¶107).

  • Staffs broad support of the Fed’s monetary policy stance was consistent with the benign view on the US housing market. SR04, \68:”The Federal Open Market Committee (FOMC) has appropriately begun preparing market for the gradual withdrawal of stimulus,” SR05 \52: The Federal Reserve’s gradual and flexible approach to monetary tightening has been effective.... Looking forward, however, monetary conditions still appear accommodative and—especially against the background of low unemployment, the recent rise in unit labor costs, and house price inflation—a more aggressive pace of interest rate hikes cannot be ruled out.”)

  • The financial sector policy recommendations appear to have been mostly in line with the authorities’ plans. SR07,\28:”Staff reiterated their support for proposals to improve regulatory effectiveness;”“SR07, *¡51: “...We strongly support plans to study the scope for rationalizing the regulatory structure, which can build on options earlier discussed by the GAO.” In some areas, staffs recommendations were more specific and/or went a bit further than the authorities’ plans at the time (SIP07, Ch IV, 1J48 did contain a specific recommendation for the US authorities to consider moving investment banks under the Fed’s supervision); staff repeatedly recommended publishing a Financial Stability Report.

were views communicated with enough emphasis?

11. The degree of concern related to flagged risks was not always clear, and headline messages tended to be reassuring (1J108).

  • Headline messages presented in SR05, SR06, SR07 continued to be reassuring. Even when financial market disruptions in the spring 2007 brought downside risks into a sharper focus, staff continued to put more weight on the “soft landing” scenario SR07,\4:”Underlying the baseline forecast of a soft landing is the continuation of supportive financial conditions, even after the emergence of problems with subprime mortgages.” SR07,\23:”Core commercial and investment banks are in a sound financial position, and systemic risks appear low. “

  • Against the benign baseline outlook, warnings were presented either as more medium-term “regulatory challenges” (SR07,*¡25) or in the form of general cautionary statements. SR07,*¡25: “Staff suggested that, while recent financial developments have helpfully spread risk, the impact of an extremely adverse market outcome may have risen... “

B. United Kingdom 9

what was called

12. All of the underlying conditions and trends (the state of the world) were discussed, and many aspects of the institutional frameworks, though some were—wrongly—not assessed as critical at the time (¶95-98). Most vulnerabilities had been recognized and risks individually investigated (¶97).

  • Rising house prices were emphasized from as early as 2002. A Selected Issues paper on House Prices (CR/03/47, p. 10, ¶15) warned “in 2002 that increases in real house prices appear significantly out of line with ... determinants’” The 2004 Report (CR/05/80, p.3) noted “houseprices are widely seen as overvalued,” referencing the September 2004 WEO research. By 2005, as house price increases decelerated, the Staff Report (CR/06/86, ¶12) noted “staff suggested that house prices are likely still overvalued” but “the degree ... is tempered by ... the stabilization of house prices over the past year.” The 2006 Report later noted (CR/07/91, ¶9), “House price growth increased [again] ... and house prices are likely overvalued;” (¶14) “risks are not insignificant” and “increases... risk of an abrupt downward adjustment.”

  • Increased risk taking and search for yield was flagged in the 2005 Selected Issues FSAP Follow-Up (CR/06/87, p.103)—while “The outlook for the U.K. banking system is favorable..’” (¶1), it offered the caveats, “medium-term risks exist, shaped by apparent expectations that benign credit conditions will continue indefinitely... risk may currently be underpriced, as investors leverage-up in their search for returns...” (¶2). The Staff Report echoed this message, including “increasingleverage and... search for yield represent downside risks” (CR/06/86, ¶6). In 2006, the house price pick up was linked to the “decline in 2-year bond yields” (CR/07/91, ¶9).

13. Institutional arrangements in the financial sector were picked up in the 2003 FSSA.

  • The eventual shortcomings of the “Tripartite Arrangement” were not foreseen, however these shortcomings at least partly reflected a failure in implementation— beyond what surveillance can be expected to see. The FSSA had noted (emphasis added): “The UK’s financial stability policy framework is well designed... While the framework has yet be tested in a genuine crisis, the MoU provides a strong framework for coordination and information sharing..., both in crisis periods and more normal times” (CR/03/46, ¶80). Later, missions asked about simulation exercises designed to identify implementation weaknesses (see below).

  • The FSSA also noted that the success of authorities’ deposit insurance scheme was “highly dependent on ... safety nets [being]... well understood by potential claimants... “ (¶104). In addition, in implicit recognition of possible problems with insurance payments under this scheme, this paragraph went on to note that at a planned review of the deposit insurance arrangements, “one of the aspects that should be considered is whether a more explicit contingency credit line from the Government would be desirable.”

  • The fact that “The UK has no special statutory regime to address the insolvency of financial institutions” was also recognized in the FSSA (¶42), however, it was not flagged as a major weakness since it was noted that “The authorities... can take a number of enforcement actions before an institution reaches the stage of statutory insolvency proceedings” (¶43). But the FSSA went on to encourage the authorities “ to keep the issue under review ... with a view to considering the scope and desirability of possible reforms to the system to broaden the ability of the FSA and/or other governmental agencies to restructure and liquidate financial institutions outside of the corporate insolvency system” (¶103).

14. Most emerging vulnerabilities and risks were picked up:

  • Exposure from the credit risk transfer instruments were (Selected Issues, CR/06/87, p.94) “a key concern ... may actually pose problems for financial sector stability in the event of a major negative shock in credit markets” and recommended “greater disclosure by financial institutions, of their holdings in CRT instruments’” But the overall message, consistent with the GFSR, was that CRT “has facilitated the dispersion of risk and enhanced the stability and efficiency of the financial system” (p. 79).

  • Risks from UK-based subprime lending in the context of search for yield was noted in (Selected Issues, CR/06/87, p.94, ¶13) “Sub-prime mortgage lending in the United Kingdom is reportedly increasing... more mainstream lenders are reportedly entering the market, albeit at the less-risky end of the sub-prime range initially.” In addition, concerns were raised about “signs of a loosening in corporate lending standards, as banks compete ... to provide funding in a low-yield environment.” (Staff Report, CR/06/86, ¶25).

  • The liquidity/wholesale funding risk, though without explicit reference to Northern Rock, was (p. 108, ¶27) also noted in the 2005 Selected Issues (CR/06/87): “...greater use of wholesale funding by banks has been observed. This strategy has somewhat increased the liquidity risk for some banks, as wholesale funding may be difficult and costly to roll over during times of company-specific or market-wide stress:’ This message was repeated in the 2006 Staff Report (CR/07/91, ¶30).

  • Crisis management arrangements were discussed during missions, where three components of crisis management (domestic, EU, international) were noted, with annual simulations: “taking the problem up to the highest echelons of decision making.” The 2005 Staff Report (CR/06/86, ¶24) likewise noted authorities “are continuously improving their ability to respond to shocks, including... crisis simulation exercises based on ... macroeconomic events.”

  • International contagion risks from systemic banks were considered (Selected Issues, CR/07/90), while the Staff Report noted links—especially through the inter-bank markets—were growing and “These growing linkages are both a strength and a vulnerability. They allow ... bad shocks to be more broadly dispersed... absorbed by individual institutions and the system as a whole. However, they also potentially allow the impact to be spread around the global financial system more widely and rapidly” (CR/07/91, ¶29). The report went on to note: “Given these growing crosscountry linkages, global risks are particularly important to the UK financial system, more for their potential severity than for their likelihood of being realized:” fl¶30). Finally, Box 5 of the Staff Report notes that due to increasing linkages between international banks “while the national focus of financial supervisory authorities has been appropriate until now, improving international cooperation in financial crisis prevention and management is becoming more important” (p. 23).

Was the right policy advice given?

15. Policy advice to mitigate the emerging risks was given, though sometimes little more could be done than to urge vigilance (¶105):

  • The FSSA noted with respect to the authorities’ deposit insurance scheme, that they “are taking steps to address [limited public awareness and should] consider whether a more explicit contingency credit line from the Government would be desirable” (CR/03/46, 1104). This was not picked up in later surveillance.

  • The Selected Issues (CR/06/87, p.128, ¶75) noted: “The FSA should continually enhance its surveillance of the CRT marketin cooperation with their overseas counterparts—given the rapid evolution of the market.” These also highlight known unknowns:

  • ➢ Absent solid research pointing to precisely what data should be disclosed, the recommendation to collect credit risk transfer data was not made in the Staff Report (CR/06/86, 1 27): “Officials... reluctant to increase the formal reporting burden... are not sure which data series would adequately capture key risks. For the time being, they preferred to focus on gathering market intelligence.”

  • ➢ The Staff Appraisal emphasized, “The rapid growth of credit risk transfer instruments, which are providing important diversification benefits, is also creating some risks. The authorities’ efforts, again, to publicize risks, but also to address the transactions backlog are right. And with due attention to the costs and benefits of new regulatory burdens, the authorities should continue to strengthen surveillance of the market and encourage initiatives to improve disclosure CRT exposures” (CR/06/86, ¶45).

  • “Supervisors’ judgment that specific risks—including... commercial property, a possible loosening of corporate lending standards, and the growth of sub-prime lending—are manageable seems reasonable... the authorities’ warnings that investors may be underpricingrisk ... are welcome” (CR/06/86, ¶45)

Were views communicated with enough emphasis?

16. The degree of concern related to flagged risks was not always clear, and headline messages tended to be reassuring (1108):

  • 2005 Article IV risk summary. “Supervisors are keenly aware of risks in the present global environment” (CR/06/86, 18). “Staff asked about risks posed by global imbalances. Officials responded that sudden shifts in international capital flows could disrupt a wide range of asset markets, potentially leading to costly and disorderly adjustments in banks’ balance sheets. Staff and officials agreed that the banking system is well-positioned to absorb substantial shocks”“ (125). “A risk to the outlook is the housing market, where valuations remain high by some metrics despite the stabilization of prices during the past year”“ (CR/06/86, ¶39)

  • 2005 Article IV bottom line. “Staff and officials agreed that the banking system is well-positioned to absorb substantial shocks” (CR/06/86, ¶25). “Macroeconomic performance in the United Kingdom remains remarkable.” (¶37) “Financial supervisors are skillfully meeting the challenge of overseeing a global financial center. Well-capitalized and cost-efficient, banks appear to be well-positioned to absorb losses that might arise from the most likely types of financial market disturbances.” fl¶45).

  • 2006 Article IV risk summary. “risks are not insignificant:... In light of estimates that house prices are already overvalued, this would increase the subsequent risk of an abrupt downward adjustment... External risks are low-probability but potentially high-impact, particularly as the global financial center could transmit shocks to the domestic economy. Higher global interest rates could trigger a reassessment of asset valuations, including UK house prices...” (CR/07/91, ¶14)

  • 2006 Article IV bottom line. “Macroeconomic performance in the United Kingdom remains impressive... Given this favorable ... outlook, financial sector prospects are strong.” (CR/07/91, ¶38) “Lastly, on the financial sector, which is in a position of strength, the authorities are appropriately promoting the system’s resilience.” (¶43)

C. Switzerland10

What was called

17. All of the underlying conditions and trends (the state of the world), and many aspects of the institutional frameworks were discussed (¶95-98).

  • Exposures to external developments, particularly in the US was flagged in the 2005 and 2006 Staff Reports, and substantiated more in the 2007 FSAP Update (f7) “the main downside risks for the financial sector appear to be external. Given their large trading portfolios, the two large banks are potentially exposed to market downturns and significant increases in volatility, associated for example with a disorderly unwinding of global imbalances that could put further pressure on U.S. exchange and interest rates, and induce a potentially severe drop in global equity markets and turbulence in financial markets. Risks would be compounded by a hard landing of housing markets in the U.S. and other key industrial countries via direct exposures and also indirectly through feedback to real economic activity.”

  • Capital adequacy problems (2007 FSAP Update, ¶10) “While capital adequacy ratios (CARs) of the two large Swiss banks are ample by current regulatory standards, other indicators suggest somewhat weaker positions. The banks have internationally comparable high risk-weighted CARs under Basel I, but their leverage ratios (equity to assets) are relatively low by international comparisons.”

  • Exposure of banks’ large trading portfolios liquidity risks (2007 FSAP, Box 1) “a systemic or institution-specific event that would disrupt market liquidity, particularly in markets with crowded trading, creates contagion risk and also could make it difficult for the banks to trade out their positions”.

  • Weaknesses in supervision (2007 FSAP Update, 127) “The Swiss Federal Banking Commission (SFBC) has made impressive progress (...). Nevertheless, two areas that remain a concern are (i) the SFBC’s budgetary independence; and (ii) the need to address liquidity monitoring.”

What could have been done better?

18. Stress tests underestimated the impact of various shocks on the banks (¶102). “Stress tests and scenario analysis indicate that the Swiss banking sector is resilient to most relevant macroeconomic shocks (...) Top-down stress tests indicated that the effect of the international scenario wiped out the sector’s profits but its effect on the sector’s capitalization level was negligible since the banking sector suffered only minor losses.” (2007 FSAP Update, ¶11). Two factors seem the most relevant to explaining why:

  • The stress tests turned out to be milder than the actual shock that ensued, even though the mission was trying to mimic a turbulent time “For the global scenario, its was agreed that the global credit conditions (...) should mirror the changes in credit conditions in 2001” (2007 FSAP, ¶52).

  • The FSAP Update was not able to quantify (although a qualitative assessment was provided) the additional problems that banks’ large trading portfolios would face in the context of a liquidity crisis (2007 FSAP, Box 1).

Was the right policy advice given?

19. Policy advice to mitigate the emerging risks was given (¶105-106):

  • “The new consolidated regulator (FINMA) needs to have operational independence and financial resources to be a constructive and appropriately forceful supervisor and regulator of the very large systemic financial institutions. Continuing efforts are needed to evaluate the large banks’ operating models and to ensure that liquidity and capital regimes are sufficient.”(2007 SR, ¶38).

  • More detailed advice was given in the 2007 FSAP Update regarding how to strengthen the monitoring of liquidity risks “the approach to liquidity regulation and supervision outlined in an SFBC Banking Ordinance should be updated to be aligned with the Basel Committee’s paper on managing liquidity in banking organizations. Indeed, the analysis needs to go beyond this given the systemic relevance of the large banks as global market players (*¡28), and the review of capital adequacy “the SFBC should review in depth the capital adequacy of the two large banks and pillar II capital requirements in the context of BASEL II implementation” (Executive Summary)

Were views communicated with enough emphasis?

20. The risks were clearly flagged, but the degree of concern attached to these risks was less clear (¶108): “ while stress testing indicates that the financial sector is generally resilient

to shocks, bank operations are increasingly complex, involving exotic instruments and high-leveraged counterparties (...) The main downside risk is that healthy balance sheets and profits may be building a degree of complacency, creating a vulnerability to increased volatility or shocks.” (2007SR, ¶37, second sentence)

D. Germany11

What was called

21. Many weaknesses of the institutional frameworks were highlighted (95-98).

  • The need for reform of the Landesbanken (LBs), and what to do, were first raised in the FSSA: “Restructuring the LBs is urgent, given their systemic importance ... “ (FSSA, ¶55) “The public LBs have started the process of adapting to the removal of government guarantees starting in 2005. However, it is unclear whether these steps are sufficient to address the trend decline in profitability. . a reduction in legal and other barriers (within or across pillars), would foster consolidation and innovation ... (FSSA, p.1 box) An 2004 Occasional paper (no. 233) further reinforced the message (p. 1). Since then, successive staff reports continued to pursue these issues. “Market-driven reform of LBs is essential, as ... they have imposed a large opportunity cost on the economy.” (SR 2006, ¶24, see also Box 3) The full analysis of opportunity costs to the taxpayer was presented in the 2006 SIP, including capital injections or ownership through other public bodies, lower financial returns, and also underscored LBs’ higher probability of insolvency. See also SR05, ¶34-38, and SR06, ¶24, which discuss the banking sector’s low profitability.

  • The increased risk taking by some Landesbanken was flagged in passing in the FSSA, and was not directly mentioned in staff reports-staff reports framed the issue as “a lack of a viable business model”. The FSSA flagged the risk in the context of developments in the derivatives markets (emphasis added): “Derivatives exposures of internationally active banks have increased . but appear manageable. This reflects in part a wider awareness of credit and market risks . but partly also regulatory arbitrage and a search for new high yielding products, especially _ for LBs’” (¶10). In 2005, the staff report noted “LBs have shored up their liquidity by raising long term funding prior to the withdrawal of state guarantees in July 2005, but they still need to develop viable business models. Although there may be isolated instances of strain, the likelihood of systemic difficulties is small.” (¶54) In 2006, this assessment seems not to have changed, with the focus squarely on the medium-term: “strains could reemerge during the next downturn and developments in neighboring countries risk leaving the German banking system behind.” (SR06, ¶53)

22. The Fund recommended legal changes to support market based-restructuring of the banking system (¶105) and facilitate consolidation. Private capital needed to be injected into public sector banks to “foster synergies and returns to scale, and help direct funds more readily to areas of highest investment needs” (SR05, ¶55), and “... to harness market signals and facilitate restructuring.” (SR06, ¶53). On supervision, staff supported the move to risk-based supervision, and urged greater disclosure and transparency on the methodology underlying risk-oriented supervision (SR06, ¶54), and of publishing “more timely financial soundness data, in particular on impaired loans.” (SR05, ¶56).

Were views communicated with enough emphasis?

23. The headline messages in the staff appraisal reinforced the need for banking reform (¶108). The staff appraisal gave tough overall messages: “Germany needs a decisive, forward-looking policy strategy to confront the serious challenges it faces.” (SR05, ¶45) In 2006, conditions had improved somewhat, but “without deeper structural change, however, the improved cyclical prospects do not alter Germany’s low growth potential.” (SR06, ¶48)

IV. Statistical Appendix12

A. Sample Selection for Review of Staff Reports

The following selection process resulted in the 50-country sample of staff reports:

1. The sample universe includes 88 Article IV consultation staff reports discussed by the Board between July 1, 2007 (after the 2007 Decision began to be implemented) and

February 28, 2008.

2. The membership was stratified by income group (advanced, emerging, and developing) and by region (5 regions according to area department), and percentages of the membership within each strata calculated. The sample universe was similarly stratified, and a target number of countries was specified (based on a total of 50 staff reports) to match the percentage of the membership within each strata. The actual country sample was then randomly drawn from within each strata to match the target number as closely as possible, within the constraints of the available universe (Tables 1 and 2).

  • Advanced: advanced economies according to the WEO classification.

  • Emerging markets: Includes countries listed in Emerging Market Bond Index (EMBI) Global, or the Morgan Stanley Capital International (MSCI) EM Free or countries that have accessed syndicated loans every year between 2002-2006.13 Excludes countries classified as Advanced in the WEO (Cyprus, Slovenia).

  • Developing: countries that fall into none of the above categories.14

3. The sample includes five program countries and three streamlined Article IV consultations.

Table 1.

List of 50 Countries for the Review of Staff Reports

article image
Notes1. Bolded countries are part of the MCM case study countries (see Appendix in FSS paper for details).2. (*) Denotes program countries.3. Italics denote members of currency unions or exchange arrangement with no separate legal tender.4. (S) denotes streamlined consultations.
Table 2.

Sample and Membership Strata by Region and Income Level

article image

B. Summary of Staff Report Reviews by Thematic Background

Table 3. Summary of Staff Report Review - FocusCircle

Table 4.

Summary of Staff Report Review - Health Check

article image
Table 5.

Summary of Staff Report Review - Financial Sector Surveillance

article image
1/four countries — Angola, Afghanistan, CAR and Congo — for which questions Q3 and Q4 were considered NOT applicable by reviewers are excluded.