Chapter 7 Lessons and Recommendations
- International Monetary Fund. Independent Evaluation Office
- Published Date:
- May 2006
Our overall assessment is that the FSAP represents a distinct improvement in the IMF’s ability to conduct financial sector surveillance and in understanding the important interlinkages between financial sector vulnerabilities and macroeconomic stability. While an overall judgment on the cost-benefit trade-off will always be difficult for such activities because of the problems in quantifying the benefits, the evaluation concludes that the FSAP has significantly deepened the IMF’s understanding of the financial sector in specific countries, helped articulate policy recommendations, prompted better discussions with authorities, and helped support policy and institutional changes. The evaluation also suggests that the joint IMF–World Bank nature of the exercise has been beneficial. Putting in place this major new initiative within a relatively short time-span represents a significant achievement.
The evaluation suggests some significant advantages of the present arrangements that should be preserved going forward: (1) an integrated approach to assessing financial sector vulnerabilities and development needs that could not be achieved by an ad hoc series of assessments of standards or analysis of particular issues; (2) an institutional link to surveillance that has greatly strengthened the operational relevance of the FSAP for IMF activities; and (3) an administrative mechanism to coordinate IMF and World Bank inputs that, while subject to some tensions, does appear to have improved coordination, with benefits for the quality of assessments. Thus, while a variety of channels to strengthen financial sector surveillance are clearly possible and would be relevant in particular country circumstances, the evaluation evidence does suggest that FSAPs and comprehensive Updates offer distinct advantages that would be difficult to replicate fully through other less comprehensive modalities. These advantages derive largely from the critical mass of expertise mobilized for an FSAP which enables comprehensive assessments of financial systems and interaction of country officials with a range of technical experts.
Despite these achievements, the initiative is at an important crossroads and there is a danger that some of the gains already achieved could be eroded without some significant modifications. The evaluation indicates two interlinked sets of problems. First, financial stability assessments have not yet been fully “mainstreamed” as a regular part of IMF surveillance. Second, looking beyond the stage of initial FSAPs, there are doubts that current incentives for participation and priority-setting procedures will be sufficient to ensure continuing coverage of the bulk of countries where strong financial sector surveillance is most needed. The evaluation also points to the need for changes in the way the IMF organizes its own activities in order to make the best use of scarce technical expertise as well as to a range of measures that would further improve the quality and effectiveness of FSAPs.
Therefore, the recommendations are organized around three key themes: (1) reconsidering incentives for participation, clarifying priorities, and strengthening the links with surveillance; (2) steps to maintain and strengthen further the quality of the FSAP and organizational changes within the IMF; and (3) the working of the joint IMF–World Bank approach. Consistent with the IEO’s mandate, the recommendations are couched in terms of actions to be taken by the IMF, although, given the joint nature of the initiative, a number of them could require decisions by both the IMF and World Bank Boards.
Incentives for Participation, Clarifying Priorities, and Strengthening the Links with Surveillance
Priority setting within the FSAP was bound to be a complicated exercise for several reasons. First, the initiative has multiple objectives, partly reflecting its joint IMF–World Bank nature. The evidence from the evaluation suggests that, in practice, this has not so far prevented priority being given to countries of systemic importance and/or with potential financial sector vulnerability concerns, provided such countries agree to participate. However, greater clarity is needed on how the balance between IMF-driven and World Bank–driven priorities will be resolved in the longer term, an issue we will return to later. Second, and probably of greater significance, there is clearly a tension between the voluntary nature of the exercise and the stated priority to be given to systemic importance and potential financial sector vulnerability. The evaluation evidence suggests this tension is increasing. The main problem is not that a minority of systematically important countries have not yet volunteered (although they certainly should be encouraged strongly to do so), but that a significant number of countries that should be high priority candidates for updated assessments have been reluctant to participate in a timely manner. The sharp tradeoffs between different objectives that one would expect the priority-setting processes to address have largely not occurred, because some authorities’ reluctance to participate has in practice been implicitly accepted when drawing up the ex ante priority lists.
Therefore, key design choices going forward are (1) how strongly the objective of the FSAP initiative should be to focus assessments on countries where the IMF judges they are most needed as an input to its global surveillance; (2) how this objective can best be matched with effective incentives for participation; and (3) how this objective can best be meshed with other objectives of the initiative through effective priority-setting procedures. There appear to be three broad choices. The first is to maintain the voluntary approach with the current set of incentives. This approach is likely to yield a result in which the coverage of FSAP Updates does not include in a timely manner many countries that the IMF would consider as high priority candidates from a global surveillance perspective. The second alternative would be to shift to a mandatory approach. The evaluation suggests that FSAPs appear to have been more effective where the assessments were most “owned” by the authorities, which suggests that the voluntary nature of the exercise can convey important advantages and should be preserved if possible. The third approach, which we favor, would be to retain a voluntary approach to the FSAP but to strengthen further the incentives for participation, especially in cases where, in the IMF’s judgment, financial sector assessments are necessary for conducting effective surveillance because of potential vulnerabilities and spillover effects to other countries.1 At the same time, other instruments for conducting financial sector surveillance, through the regular Article IV process, would also be strengthened, with the choice of the mix of instruments to be used taking into account each country’s circumstances.
In addition, the evaluation shows that the IMF is not yet using the FSAP results as effectively as it could in its overall surveillance activities. There also appear to be substantial differences of view within the IMF on what is the appropriate expected scope for financial sector surveillance outside of the FSAP. For example, the long delay in finalizing revised guidelines on financial sector surveillance reflects disagreements on what could reasonably be expected from such surveillance, given likely resource constraints. Moreover, the organization of financial sector surveillance outside the FSAP was also subject to different views, including on whether expertise should be centralized (i.e., in MFD and ICM) or decentralized (i.e., at the area department level).
These findings suggest the need for changes in how country choices for financial sector assessments are made and in how those assessments are mainstreamed into IMF surveillance. Our proposed approach contains the following mutually supporting elements: country-specific strategies for financial sector surveillance that choose between a range of modalities for such surveillance, including FSAPs and Updates, based on sharper criteria for priority setting (Recommendation 1); strengthened incentives to encourage comprehensive assessment exercises when they are judged necessary for effective surveillance, albeit within a still-voluntary framework for the FSAP (Recommendation 2); and strengthened links between FSAPs and Article IV surveillance (Recommendation 3). The overarching idea is that, to maintain its strong relevance to the IMF’s global surveillance objectives, financial sector assessments and their updates should cover most countries of systemic importance and/or with potential financial vulnerabilities in a timely manner. Both the incentives for participation and priority-setting criteria should be set with this objective in mind, and the IMF should take stock periodically of progress toward explicit benchmarks of achieving adequate country coverage.
Recommendation 1. The IMF Board and management should refine the criteria for setting priori-ties on IMF resource inputs into financial sector surveillance, including the FSAP. Based on these priorities, IMF staff should indicate, as part of its medium-term planning, what components are needed for strengthening financial sector surveillance in each country, drawing upon a range of possible modalities. These strategies would form the basis for more explicit accountability on results.
The current list of factors to be taken into account in setting priorities, including geographic diversity, is quite long. So far, the relatively broad nature of the criteria has not been a major problem because the main focus has been on encouraging countries to participate—especially those judged important for global surveillance—rather than on meeting hard choices between competing demands. Going forward, however, if incentives for participation are strengthened successfully, clearer guidance will be needed on how to manage the resource tradeoffs between, on the one hand, following up at relatively frequent intervals on vulnerability issues in countries of systemic importance (or where there are warning signs concerning the financial sector) and, on the other hand, a more extensive examination of financial sector development issues in lower income countries. Such guidance will need to be accompanied by a clear division of primary responsibilities between the Bank and the IMF, within the existing coordinating framework (see Recommendation 6).
In calling for staff to indicate country-specific plans to guide financial sector surveillance, we do not propose the preparation of additional documents. Rather, such strategies could be included either in area department work plans or in Article IV reports. They should address two basic questions: (1) how much priority and emphasis should be given to financial sector issues in surveillance (in some countries, the answer would be that these issues are a relatively low priority) and (2) what is the frequency, scope, and modality of assessments that would best fit each country’s circumstances and the relative priority accorded to these issues. In the process of elaborating the strategy, which should be a collaborative effort between area departments and MFD, the systemic importance and macro-relevance of potential financial sector vulnerabilities should be considered explicitly.2
While the particular scope of FSAP assessments will vary according to country circumstances, an approach that emphasizes more frequent assessments using a variety of modalities interspersed with relatively infrequent and more comprehensive assessments, akin to the initial FSAP, may often be more effective. In some cases, the more frequent assessments could build upon countries’ own self-assessment exercises.
While the evidence from the evaluation does not allow us to draw concrete conclusions about the merits of more explicit product differentiation between types of FSAPs,3 greater tailoring of the assessments to individual country circumstances should be an explicit objective of the country strategies—a process that is already under way. For instance, a broad range of ROSCs may be needed in some countries whereas in others it would be appropriate to cover at most the banking sector.
In many cases, these country-specific plans will involve stronger efforts to “mainstream” financial sector assessments into regular Article IV surveillance. In some cases, following an initial FSAP, it may be appropriate for subsequent Article IVs to focus periodically on financial sector issues. This would be a natural outcome of management’s intention of making Article IV reports more focused, dealing only with issues of critical importance: where domestic or international aspects of financial stability are of critical concern they would naturally form such a focus.
Area departments should be held accountable for delivering on the country-specific plans, as part of the ongoing efforts to strengthen the monitoring of surveillance effectiveness.4
Recommendation 2. To strengthen incentives and drawing upon these country-specific plans, IMF management should clearly signal to the Board those countries that it sees as the highest priorities for FSAPs and Updates, irrespective of whether these countries have volunteered. These lists should be the basis for periodic discussions by the Board of country-specific priorities.
Also, in cases where there are indications of potential financial sector vulnerabilities in systemically important countries that have not volunteered for an initial assessment or Update, IMF management should indicate to the Board where it proposes to call for an intensified analysis of financial sector issues as part of the regular Article IV surveillance.
Since it is not possible to predict whether this proposed strengthening of incentives will be sufficient, coverage of the FSAP should be reviewed again after several years, with the emphasis on the adequacy of surveillance potential. The key benchmark should be inclusion of FSAPs and/or updates for the bulk of countries signaled as high priorities for such coverage in the strategic plans. If the Board concludes at that time that coverage is falling significantly short of this benchmark, consideration should be given to shifting to a more mandatory approach.
Recommendation 3. Strengthen the links between the FSAP and surveillance by mainstreaming FSAPs and follow-up work into the IMF’s regular surveillance activities. This means incorporating the assessment of financial sector standing and vulnerabilities into the overall macroeconomic assessment of the country in a way that fosters a greater understanding of stability; policy recommendations that are set in a coherent framework combining macroeconomic and financial sector analysis; more meaningful discussion of financial sector issues with authorities; and enhanced peer review discussion at the Board. Steps that could be taken in this direction include the following:
As noted above, where financial stability issues are judged to be of high importance—either as a result of the findings of an FSAP or because of the potential global systemic importance of country’s financial system—they should be a major focus of an Article IV consultation. This would have obvious implications for the composition of the Article IV team.
The internal review process should be strengthened to ensure that key messages on macro-financial stability are fully reflected in Article IV surveillance reports. A short (1–2 page) section in each FSSA that summarizes—in candid language—the main macro-relevant findings from the FSAP and the potential macroeconomic consequences arising from any major identified financial sector risks would assist this process.
FSAP team leaders should be given a greater “voice” at the time of Board discussions, including an opportunity to summarize briefly what they see as the key FSAP findings with macro-economic relevance.
The Board itself should seek to give greater attention to financial sector issues in its surveillance discussions when the FSSA flags significant macro-relevant issues. If the potential implications for surveillance arising from the financial sector assessment are not sufficiently clear, the Board should encourage the staff to elaborate.
Steps should be taken to identify and disseminate cross-cutting messages that arise in a number of FSAPs.5 As part of this effort there is scope for integrating the macro-relevant findings of such assessments into multilateral surveillance Board papers and presentations, including informal presentations to the Board, similar to the World Economic and Monetary Development sessions, and for greater sharing of cross-country experiences in the context of FSAP reviews.
Improving the Quality and Impact of the FSAP and Organizational Changes Within the IMF
While the evaluation concludes that the overall average quality of the FSAP exercises is quite high, several shortcomings were identified. The most systematic shortcoming was the insufficient attention paid to cross-border financial linkages and their potential consequences. In addition, problems were encountered in many FSAPs with inadequate prioritization of recommendations, as well as insufficient indication of the degree of urgency of implementation. These problems hampered effective follow-up by both surveillance and technical assistance. Moreover, while the application of various analytical tools significantly strengthened the overall quality of the assessments, problems were encountered in a number of areas, of which the most frequent included (1) a tendency to understate the potential consequences of identified weaknesses in supervisory standards, especially with regard to de facto enforcement rather than de jure regulations; (2) presentations of the results of stress-testing exercises that tended to over-state what the exercises could say about the soundness of financial systems, given the data and methodological difficulties usually encountered. In some cases, these difficulties were compounded by a reluctance to investigate the potential consequences of politically sensitive shocks; (3) in a minority of cases, there was insufficient integration of the macroeconomic and financial sector components of the assessment; and (4) many authorities would have liked to see greater efforts by FSAP teams to understand the political economy context of their country and to structure recommendations—especially those concerning wide-ranging reforms—with this context in mind. More generally, the need for greater staff continuity in follow-up on financial sector issues (both in surveillance and technical assistance) was a refrain heard frequently.
Addressing these issues will require steps to improve FSAP quality, in most cases by applying more widely what is already “good practice” (Recommendation 4) but also a number of organizational changes within the IMF to use scarce expertise on the financial sector and related capital market issues more effectively (Recommendation 5).
Recommendation 4. Implement steps to improve further the quality of the FSAP and strengthen its impact. In most cases, these steps would involve applying more systematically what is already current policy or “good practice:”
Clearer prioritization of recommendations, along with a candid discussion of the potential consequences of not addressing key weaknesses.
Steps to improve the quality of stress-testing analysis, especially in emerging market and low-income countries. These steps should include more candid judgments on the quality of data available for the assessments, and stronger “health warnings” about the limitations to be placed on any results. While it is neither possible nor desirable to prespecify the precise types of shocks to be considered in particular country circumstances, it would be helpful to have greater transparency about the circumstances in which types of shocks that are likely to be politically sensitive will be analyzed.6
The greatest need is to include cross-border, financial sector linkages more systematically into the FSAP analysis. This will require, inter alia, greater ICM involvement, including at the TOR stage, in countries where cross-border linkages are of substantial importance.7
The Financial Sector Liaison Committee (FSLC) should ensure that FSAP team and deputy team leaders have adequate experience for the difficult challenges they face; if necessary, it would be better to reduce the number of FSAP missions rather than accepting any weakening in the quality of team leaders.
The evaluation indicates that the follow-up to the FSAP has been strongest in cases where the authorities have been most directly involved (i.e., have had some ownership of the FSAP results). Therefore, steps that enhance the involvement of the authorities in the process should be considered. We have the following menu of suggestions, but do not propose them as concrete recommendations so as to avoid prescribing specific procedural approaches that may not be well suited to all country circumstances:
Engage the authorities at an early stage on the objectives and scope of the FSAP, including the specific terms of reference.
Informal discussions of the key FSAP results with high-level officials, before reports are drafted, appear to have been highly effective in many cases and should be used for (1) a candid presentation of potential vulnerabilities; (2) a discussion of how to maximize the feasibility of various reform proposals; and (3) follow-up plans.
The precise modalities for such discussions would vary by country, and could include separate visits by core members of the FSAP team once the authorities have absorbed the key messages or discussions in the context of Article IV missions, with relevant Bank staff invited to participate.
The authorities should be invited (but not required) to provide a brief written response indicating where they agree and disagree with key recommendations and what their proposed plan of action is. Where appropriate, this response could be appended to the FSSA.
Recommendation 5. Introduce changes in the organization of IMF mission activities to utilize scarce financial sector technical expertise (especially in MFD and ICM) more effectively in the surveillance process. One message from the evaluation is that the scarcity of financial sector and capital markets expertise is a major constraint on the effective follow-up in subsequent surveillance of major issues raised by FSAPs.8 While efforts to improve area department staff training and experience on such issues is important, a model in which each area department relies primarily on such “inhouse” expertise would probably not be efficient and would risk reducing the broad “cross-country” perspective that many of those interviewed said was a particular potential value added of IMF financial sector surveillance. While these organizational issues involve many additional factors beyond the scope of this evaluation, they may require further changes in the way surveillance missions are organized, in the direction of a model in which the area department is the strategic coordinator of relevant specialist inputs provided by functional departments.9
Joint IMF–World Bank Nature of the FSAP
The evaluation suggests that the joint nature of the exercise has brought considerable advantages in practice. In particular, organizing joint teams that included both IMF and World Bank staff members (as well as outside experts) has contributed significantly to the depth of analytical expertise and credibility of the findings in many, but not all cases.
Going forward, however, greater clarity will be needed on how trade-offs between the objectives and priorities of the two institutions are to be handled within the FSAP framework. More specifically, if steps to strengthen incentives for participation, discussed in the earlier recommendations, are successful, then more concrete guidelines will be needed on how to manage trade-offs between more frequent updated assessments of countries of systemic importance and/or potential financial vulnerability and assessments of countries with less developed financial sectors. The division between stability and some development aspects of the financial sector is not clear cut, and the IMF clearly has an interest in many aspects of the latter. Nevertheless, there are trade-offs between, for example, devoting resources to assessing vulnerability in financial systems and identifying strategies to make financial services available to underserved sectors or groups. The approach we suggest (in Recommendation 6) is to keep the present institutional arrangements, including joint FSAP teams but, within this structure, to clarify further the respective roles of the two institutions. This will also involve each institution taking the lead in priority setting in those situations where it has primary responsibility.
The evaluation also indicates that there is often a weak framework for formulating detailed action plans to follow up on the FSAP recommendations, and identifying coordinated technical assistance support for these plans. While the country itself should take the lead to formulate such action plans, the IMF (and World Bank) can strengthen their support by (1) better prioritization of recommendations in the FSAP (see Recommendation 4); (2) more explicit discussion of follow-up plans at the end of the FSAP exercise; and (3) a clearer framework for coordinating follow-up technical assistance (Recommendation 7).
Recommendation 6. Maintain the current joint approach, but clarify further the distinctive contributions the IMF and World Bank can make, with the IMF taking the lead where significant domestic or global stability issues are present, and the Bank taking the lead where financial sector development issues are more paramount. Such clarity should include a clear delineation of primary responsibilities for setting priorities (and contributing resources). It should also recognize the distinct contributions the two institutions can make to follow up action, with the IMF taking the lead where the main need is for policy advice and TA linked to stability issues, and the Bank where the main need is for institution building or financial sector restructuring with associated advice, analysis, and financing. Clearly, this delineation cannot be set in stone for any country since the issues that are most important will change as circumstances change and should be set as part of the country-specific strategy.10 Moreover, we see the current coordinating framework for the joint approach, including a continued central role for the Financial Sector Liaison Committee, as a reasonably effective approach to ensuring that one institution taking the lead on certain issues and countries does not come at the expense of a reduced “buy in” of both institutions to the proposed strategy.
Recommendation 7. The IMF, in conjunction with the World Bank and other technical assistance providers, should seek to establish a clearer framework for coordinating follow-up capacity-building technical assistance activities, based on the country’s own action plans. Clearly, the authorities should take ownership of identifying and coordinating such activities to the maximum extent feasible, and many countries may have no need for external involvement in establishing a suitable framework. However, evidence from the evaluation suggests that this is an area where many countries would like to see stronger support from the IMF and World Bank. Establishment of such a framework will also require a clearer understanding between the two institutions of the appropriate dividing line between the FSAP as an assessment vehicle and capacity building/development activities; at present, the IMF approach involves a sharper demarkation between the two activities than occurs in the World Bank.
While details of the framework would vary from country to country, and should build upon existing institutional arrangements for donor coordination, the following steps could be considered:
Building on the discussions with the authorities of their proposed follow-up plans, relevant IMF and World Bank staff (i.e., MFD, the Bank’s Financial Sector group, and area/country departments) should meet and prepare a possible agenda of capacity-building and other technical assistance needs, with an indication of priorities whenever requested to do so by country authorities. (Such quadrilateral meetings have been held recently for a small number of countries.)11
This agenda (along with the FSSA and FSA, as soon as release is cleared by the authorities) should be shared with other relevant TA providers as early as possible, and the IMF should seek to coordinate its own activities with these providers, using existing country-based coordination mechanisms wherever possible.
MFD may wish to relax its approach whereby mission chiefs (and to a lesser extent other staff) who have participated in an FSAP assessment for a country are not involved in subsequent TA activities to that country. While the evaluation team was not able to assess fully the arguments in favor of such a demarkation, there are signs that it can adversely affect continuity and the transfer of knowledge in the IMF’s own TA activities.
Finally, a few words about the possible resource implications of the various recommendations. We are not in a position to provide specific estimates of the possible net costs of implementing each recommendation, in part because to attempt to do so would involve specifying the particular approach to be taken in much greater detail—choices that are best left to IMF management. However, in our judgment, many of the recommended actions would have limited resource cost implications that could be absorbed within the existing envelope. (For example, drafting FSAP/FSSA reports in a way that prioritizes recommendations and highlights the key findings of greatest macroeconomic relevance for surveillance is largely a question of the relative emphasis and content of the reports and should not require additional resources per se.) However, several of the recommendations would probably require additional resources, although exact quantification is not possible at this stage: (1) strengthening incentives for participation in the FSAP could raise average costs per FSAP if it results in a larger proportion of comprehensive Updates (and initial assessments) being undertaken for countries with relatively complex financial systems; (2) strengthened coverage of cross-border issues in FSAPs would require some additional staff time and specific expertise, although there may be scope for achieving some economies of scale through regional FSAPs; greater attention to these issues would be needed in some, but not all, FSAPs; (3) greater lead time before FSAP missions—to allow for further discussion with the authorities at the TOR stage, more advance notice of information requests, and so on—could raise moderately the total staff resources per FSAP, although better advance coordination with the authorities is likely to yield net benefits overall; and (4) more systematic approaches to post-FSAP follow-up, including a clearer framework for coordinating subsequent TA, are likely to involve additional costs, especially if additional country visits are required. Some of these costs would fall on the TA functions of MFD (and the respective area department), rather than the FSAP initiative per se. If the thrust of these recommendations is accepted, more precise quantification of resource costs would need to be prepared as part of any plan for implementation.
More generally, the message from this evaluation is that the FSAP has proved to be a reasonably effective vehicle for enhancing the IMF’s understanding of financial sectors, including for surveillance purposes. Going forward, the choices made on country coverage are likely to be one of the biggest influence on FSAP costs and will reflect strategic decisions on how central to strengthening financial sector surveillance globally the FSAP exercise is intended to be. Some of these choices would involve higher resource costs for the FSAP. Such decisions on overall resource allocation can only be made in the context of the IMF’s broader medium-term strategy.
The 2005 FSAP review took some steps in the direction of enhancing incentives to participate by calling for a more active promotion of Updates “through outreach programs and active encouragement by both Board [and] management and staff.” As a step to create greater awareness, IMF staff proposed “instituting annual reporting to the IMF Board on country participation in initial assessments and updates” similar to the IMF’s “Quarterly Report on the Assessment of Standards and Codes.”
Existing priority-setting processes have already moved some way in this direction, with periodic (typically every six months) meetings between MFD and each area department to discuss work program priorities; ICM attends these meetings.
For example, the idea of “developmental FSAPs” that would focus on institutional and market access issues was discussed at one point but no such specific categorization was ever introduced and hence cannot be evaluated.
This is consistent with the emphasis the IMF Board placed on clearer benchmarks for assessing the effectiveness of surveillance and the staff disseminating to the Board multiyear country work programs, articulated around a limited set of priorities (see the Summing Up of the Board discussion following the 2004 Biennial Surveillance Review, PIN 04/95 of August 24, 2004). Enhanced financial sector surveillance was one of the benchmarks established by the Board.
The 2004 staff paper on “Financial Sector Regulation—Issues and Gaps” was one good example.
For example, greater ex ante transparency about the circumstances in which the consequences for the banking system of sovereign debt events should be analyzed would be helpful, so as to avoid ad hoc decisions and unintended signaling about the expected probability of such events.
Increased use of regional FSAPs are another option for incorporating better financial cross-border analysis, especially in monetary unions. Some regional FSAPs have already been undertaken (e.g., the FSAP for the Eastern Caribbean) or are under discussion (e.g., a regional surveillance exercise for the Nordic-Baltic region).
In particular, the evaluation team’s interviews with both ICM and area department staff indicated a widespread view thatcurrent arrangements were not utilizing ICM’s expertise on capital market issues as effectively as possible, either in the FSAP or more generally in surveillance.
The report by the Review Group on the Organization of Financial Sector and Capital Markets Work at the IMF (McDonough group) addresses these organizational issues in greater depth.
As noted earlier, the case of Ghana represents a “good practice” example of how such changing priorities were handled.
Another approach that is in the pipeline for two countries with recently completed FSAPs is to organize a “post-FSAP TA providers forum” under the chairmanship of the FSAP country, bringing together the IMF, World Bank, and other potential donors to consider how to take forward post-FSAP TA plans. In one of these cases, FIRST may be commissioned to transform the FSAP action plan into a series of coordinated TA programs. While it is too early to evaluate such an approach, it appears to be a useful precedent in line with the thrust of this recommendation.
The following chart shows a schematic version of a results chain (or logical framework) for the FSAP that guides the current evaluation.1
The FSAP’s Logical Framework
See Duignan (2001) for a general discussion of evaluation methodologies.
As part of the evaluation, IEO and IEG undertook jointly a survey of the key stakeholders involved in the FSAP. This annex presents the methodology used for surveying the views of participants and a set of summary tables of various stakeholders’ responses. The main findings from the surveys have been incorporated in the report. To ensure the confidentiality of survey responses, an external company was hired to administer the implementation and collection of results.1 The surveys were conducted in the spring of 2005, to a large degree through an on-line modality.2
Survey questionnaires were sent to five groups of stakeholders, consisting of different users and producers of the FSAP:2
Authorities. A single survey was sent to the authorities of all countries that had completed an FSAP by the first quarter of 2005. Every effort was made to send the survey directly to the authorities in the country most directly involved with the FSAP.
IMF Article IV mission chiefs and area department division chiefs. The survey was sent to the relevant staff who worked on countries that had an FSAP.
World Bank country directors. The survey was sent to the relevant directors who worked on countries with an FSAP.
FSAP team leaders as well as deputies and coleaders. Team leaders and co/deputy leaders are typically drawn one each from the IMF and World Bank.4 FSAP updates were treated as a separate assessment from the original FSAP.
FSAP team members. The survey was sent to all team members from IMF and World Bank staff. External experts were not included.
Main Features of the Questionnaires5
The outline of each questionnaire followed broadly the outline of the evaluation questions in the IEO and IEG Issues/Approach papers. The main components of each questionnaire related to inputs, outputs, outcomes, and process issues.
There were about 30 questions for each group of stakeholders. Where applicable, the same questions were posed to different groups; a number of questions applied only to specific groups.
Survey questions were mostly of the closed-end type. Many consisted of specific statements where respondents were asked to identify their views on a five-point scale (ranging from “strongly agree” to “strongly disagree”). Some questions had multiple choices, and others sought “yes/no” answers. Where applicable, the respondents were given the opportunity to choose a “don’t know” option and to write in their response (“other, please specify”). At the end of the survey, all respondents were given the opportunity to provide comments on the FSAP.
The overall stakeholder response to the survey was quite high (53 percent of the net deliverable sample).6 Significantly different response rates were obtained across groups; those from the authorities and FSAP leaders and members were the highest at around 60 percent (see Table A2.1). Tables A2.2–A2.10 summarize the results of the survey.
|IMF Article IV mission chiefs||83||9||74||27||36|
|World Bank country directors||57||3||54||14||26|
|FSAP team leaders||79||8||71||45||63|
|FSAP team members||289||41||248||148||60|
|An independent assessment of the country’s financial sector||80||70||57||82|
|Recommended by IMF/World Bank||42||56||71||40|
|To learn more about the country’s financial sector||22||30||29||32|
|Concerns about financial vulnerabilities||24||15||50||26|
|FSAP is expected of every country||22||11||21||19|
|Signal to international capital market||18||19||14||40|
|Other peer countries have had FSAP||16||15||7||32|
|To facilitate lending by international financial institutions||9||15||21||18|
|Authorities||IMF Mission Chiefs||World Bank Country Directors||Team Leaders||Team Members|
|1. FSAP provided analytical insight into country’s financial sector that did not exist before||45||32||23||0||38||38||25||0||57||29||14||7|
|2. Analysis balanced financial sector stability issues with development priorities||48||41||11||16||44||39||17||28||35||50||15||7|
|3. Recommendations took into account unique country circumstances||40||40||20||0||46||46||8||0||57||29||14||7|
|4. Recommendations were prioritized||67||23||9||2||50||17||33||0||33||40||27||0|
|5. Recommendations were clear||94||7||0||0||67||33||0||0||53||33||13||0|
|6. Recommendations were candid||74||14||12||2||63||17||21||0||46||33||20||0||86||10||5||0||71||24||5||0|
|Authorities||IMF Mission Chiefs||World Bank Country Directors||Team Leaders||Team Members|
|1. Satisfaction with FSAP recommendations||81||16||2||0||59||29||13||0||54||27||20||0|
|2. Satisfaction with overall usefulness of FSAP||81||14||5||0||39||39||22||0||43||29||28||0||78||16||7||0|
|3. Most insightful FSAP components|
|a. Input into reform agenda||43||54||73||65|
|b. Integrated financial sector analysis||71||50||47||84|
|c. Financial standards and codes assessment||80||42||33||59|
|d. Assessment of development priorities||21||29||27||52|
|e. Anti-money laundering and CFT||18||13||7||11|
|4. Most useful analytical component|
|a. Financial infrastructure assessment||66||54||33||66|
|b. Stress testing||75||38||33||62|
|c. Financial soundness indicators||59||38||40||43|
|d. With public||14||4||13||26|
|5. Three most useful FSAP areas||Objective evaluation of financial sector (58 percent).||Objective evaluation of financial sector (62 percent).||Objective evaluation of financial sector (40 percent).||Objective evaluation of financial sector (30 percent).||Objective evaluation of financial sector (38 percent).|
|Integrated view of financial sector (16 percent).||Identification of new financial sector risks and learning new analytical techniques (13 percent each).||Integrated view of financial sector (31 percent).||Integrated view of financial sector, identification of new financial sector develop-ment needs, and prioritization of financial sector reform (13 percent each).||Integrated view of financial sector (15 percent).|
|Learning best international practices (11 percent).||Enabling reforms by contributing to public debate (8 percent).||Identification of new financial sector development needs, prioritization of financial sector reforms, learning new analytical techniques, learning best international practices, and identification of TA needs (8 percent each).||Identification of new financial sector risks (12 percent).||Identification of new financial sector risks (12 percent).|
|6. Three least useful FSAP areas||Identification of TA needs (58 percent).||Improved coordination among regulators (52 percent).||Enabling reforms by contributing to public debate (69 percent).||Enabling reforms by contributing to public debate (48 percent).||Enabling reforms by contributing to public debate (46 percent).|
|Enabling reforms by contributing to public debate (50 percent).||Enabling reforms by contributing to public debate (48 percent).||Improved coordination among regulators and prioritization of financial sector reforms (39 percent each).||Improved coordination among regulators (41 percent).||Improved coordination among regulators (40 percent).|
|Identification of new financial sector development needs (33 percent).||Learning new analytical techniques (35 percent).||Identification of new financial sector risks (31 percent).||Learning new analytical techniques (34 percent).||Learning new analytical techniques (28 percent).|
|Authorities||IMF Mission Chiefs||World Bank Country Directors||Team Leaders|
|1. Financial sector vulnerabilities have been reduced since the time FSAP was completed||39||45||16||13||6||39||56||44||45||18||36||18||44||34||23||44|
|2. FSAP generated negative public debate||10||3||86||10||5||5||91||9||7||7||86||7|
|3. FSAP strengthened strategic view on development of financial sector||52||37||10||2|
|Yes||No||Don’t Know||Yes||No||Don’t Know||Yes||No||Don’t Know||Yes||No||Don’t Know|
|4. Has FSAP contributed to a policy debate in the country?||56||33||11||35||48||17||40||60||0||61||19||19|
|5. Form of policy debate|
|a. Within government agencies||92||100||80||97|
|b. Financial sector stakeholders||75||75||60||71|
|c. With legislators||63||50||20||34|
|d. Public debate||13||25||0||8|
|Authorities||IMF Mission Chiefs||World Bank|
|Team Leaders||Team Members|
|1. Extent of implementation of main FSAP recommendations||66||26||7||2||42||26||32||26||11||33||55||56||53||37||10||27||37||35||28||70|
|2. Major reason for not implementing FSAP recommendations|
|a. Little political support||17||13||50||39|
|b. Recommendations too recent||17||44||14||29|
|c. Disagree with recommendations||28||17||14||9|
|d. Difficulty in prioritizing||14||4||36||7|
|e. Too many recommendations||8||4||36||3|
|Authorities||IMF Mission Chiefs||World Bank Directors||Team Leaders|
|Yes||No||Don’t know||Yes||No||Dont’s know||Yes||No||DOnt’s know||Yes||No||don’t know|
|1. Was FSAP output published?||66||29||5||58||25||17||13||33||53||65||22||13|
|2. Reasons for not publishing FSAP output|
|a. To keep it confidential||58||33||60||60|
|b. To protect market-sensitive information||50||17||20||7|
|c. Disagreement with recommendations||8||17||20||7|
|d. Government does not publish such documents||8||33||20||0|
|e. FSAP was a pilot||17||17||20||0|
|3. Modalities of IMF follow-up of FSAP|
|a. Article IV consultations||67||82||66|
|b. Technical assistance||24||41||31|
|c. IMF-supported program||21||18||26|
|d. FSAP update||17||14||10|
|e. Did not follow up||5||0||3|
|4. Modalities of World Bank follow-up|
|a. Nonlending activities||5||57||40|
|b. Don’t know/n.a.||42||14||43|
|d. World Bank facilitated technical assistance||18||37|
|e. FSAP update||11||14||10|
|f. Did not follow up||16||0||2|
|g. Regular contacts with authorities||50|
|IMF Mission Chiefs||World Bank|
|Team Leaders||Team Members|
|1. Focus of IMF Board discussion on main risks identified in FSA||11||58||31||37||52||24||24||48|
|2. Focus of World Bank Board discussion on main risks identified in FSSA||25||25||50||250||50||50||0||450|
|3. Satisfaction with IMF’s internal comments during FSAP process||14||28||58||2||13||29||58||15|
|4. Satisfaction with World Bank’s internal comments during FSAP process||49||29||22||18||45||31||24||37|
|Authorities||IMF Mission Chiefs||World Bank Country Directors||Team Leaders||Team Members|
|1. FSAP team’s technical skills||92||7||0||2||59||27||14||9||78||21||0||7|
|2. FSAP process required too much data||59||21||21||0|
|3. FSAP process provided significant lead time for preparations||46||42||12||5|
|4. FSAP process was too time consuming||42||42||17||5|
|5. Team leader was able to select a strong team||77||14||9||0|
|6. Joint IMF–World Bank FSAP is cost efficient||53||21||25||7|
|7. Joint IMF–World Bank team worked well together||82||9||9||14||62||23||16||16|
|8. Team’s access to information was adequate||81||16||3||0||83||12||6||0|
|9. Team members had enough time to complete the work||67||16||17||0||64||17||19||2|
|10. Team members had enough resources to complete the work||79||18||3||0||70||19||11||1|
|IMF Mission Chiefs||World Bank Directors||Team Leaders||Team Members|
|Yes||No||Don’t know||Yes||No||Don’t know||Yes||No||Don’t know||Yes||No||Don’t Know|
|1. Would it useful for the country to ask for an FSAP update?||58||33||9||31||35||35||39||46||14||65||33||2|
|Voluntary||Mandatory||Don’t Know||Voluntary||Mandatory||Don’t Know||Voluntary||Mandatory||Don’t Know||Voluntary||Mandatory||Don’t Know|
|2. Should FSAP be voluntary or mandatory?||84||16||0||52||35||13||69||31||0||59||39||2|
|2–3 Years||4–5 Years||Over 5 Years||Flexible||2–3 Years||4–5 Years||Over 5 Years||Flexible||2–3 Years||4–5 Years||Over 5 Years||Flexible||2–3 Years 5||4–5 Years||Over Years||Flexible|
|3. Preferred frequency of updates||58||33||4||4||63||38||0||0||80||0||0||20||35||47||14||4|
|Strongly Agree or Agree||Neither Agree nor Disagree||Strongly Disagree or Disagree||Don’t Know|
|4. FSAP should focus more on development than on stability issues||32||32||37||5|
The external company was Fusion Analytics LLC, Washington D.C.
Participants were also given an opportunity to send their responses directly to Fusion Analytics by facsimile. Only a handful of authorities replied by fax.
Stakeholders other than the authorities that had been involved with more than one FSAP were invited to submit a survey response for each country (up to a maximum of three).
In advanced economies the IMF has responsibility for the FSAP.
For those readers who are interested in seeing details of the specific questions and responses, a full version of each questionnaire and a summary of the responses is available on the IEO’s website (at www.imf.org/ieo).
Net deliverable sample is defined as the total target population minus those who could not be contacted for various reasons.
This annex presents the methodological approach used in the 25 case studies undertaken as part of the evaluation. The assessment comprised both a desk review component and the interview of key stakeholders, specifically:
Systematic review of relevant documents using a detailed template (including all FSAP papers, Article IV and program documents, comments from review departments, Public Information Notices, and Executive Board minutes).
Structured interviews with FSAP and Article IV mission leaders and World Bank staff;1 and interviews with country officials.
The review of documents involved the use of a detailed template to evaluate specific questions (see Table A3.1). The template specifies detailed criteria for the assessment and comprises two types of results: qualitative assessments and ratings (in a four-point scale). Where ratings were required, the template describes what would be expected to warrant a specific rating.
|I. Inputs, Effectiveness, and Efficiency Assessment|
|A. Scope of financial sector coverage across countries|
|1. Did terms of reference (TORs) discuss criteria for prioritization of issues?||Yes/No, describe.|
|2. Did FSAP discuss reasons for scope and depth of sectoral coverage?2||Yes/No, describe.|
|3. Which sectors were covered?||List all covered sectors.|
|4. Were the critical sectors as defined in question (5) adequately analyzed (or what was the degree of coverage)?||Rating:|
Each important sector should be rated separately. Adequate analysis means a judgment if the sector is analyzed according to its relative importance in the economy. 3 Rating scheme is the following:
|5. Did the internal review process (comments by IMF/World Bank departments) discuss appropriateness of scope and depth of sectoral coverage in FSAP?||Yes/No, describe.|
|6. How was the overall financial sector covered in FSAP?||Rating:|
|7. Do documents discuss the need for an FSAP reassessment or focused update?||Yes/No, describe.|
|8. According to your judgment, please select the appropriate category for the country.||Rating:|
|B. Cooperation between IMF and World Bank|
|9. Do documents specify how were sectoral responsibilities divided?||Yes/No, describe.|
|10. Keeping in view TOR, was the emphasis in documents more on financial development issues or on stability issues?||Describe.|
|C. Modalities and toolkit for identification of risks, vulnerabilities, and development needs|
|11. Were the evaluated risks (possible domestic shocks) and development needs linked clearly to country conditions?5||Yes/No, describe.|
|12. Has FSAP taken into account regional and global risks (external shocks), trends, and their possible impact on a country?||Rating: |
|13. Do the documents discuss the regional or global implications of the domestic (country) financial sector risk?||Yes/No, describe.|
|14. Has the analysis been cast broadly enough to capture less obvious vulnerabilities (e.g., transfer of risks between government and financial sector, perceptions of implicit government guarantees)?||Yes/No, describe. 6|
|15. Were the financial sector vulnerabilities analyzed in questions 15–19 integrated in the overall assessment?||Yes/No, describe.|
|16. Has the overall assessment considered explicitly the linkages between financial development and stability issues (i.e., potential complementarities and trade-offs)?||Rating:|
|17. Have other mechanisms of financial sector surveillance/reviews/assessments discovered new risks and vulnerabilities after FSAP?||Yes/No, describe.|
|18. Was the rationale for the choice of certain analytical tools and methodologies used in FSAP made explicit (were they explained)?||Yes/No, describe.|
|19. Were the available data for the FSAP sufficient (both quantity and quality)?||Yes/No, describe.|
|20. If the data were not sufficient (in quantity and/or quality), was this considered to be of major importance for the findings/assessment (significantly hampering analysis or even obstructing the assessment altogether)?||Yes/No describe.|
|21. Standards and codes (S&C). Do reports discuss extent to which S&C are appropriate for country conditions?||Yes/No, describe.|
|22. Do the documents discuss reasons for selection of assessment of S&C?||Yes/No, describe.|
|23. How did the assessments of S&Cs feed into the overall analysis?||Rating:|
|24. Has the assessment of S&C dealt with the distinction between de jure and de facto implementation of regulations?||Yes/No, describe. 7|
|II. The Outputs: Articulation of Findings and Recommendations|
|25. Were findings and diagnosis in the FSAP clear and candid for the overall financial sector and major sectors analyzed?||Rating:|
|26. Were the importance and consequences of findings explained?||Rating: |
|27. Were the issues and clarity/candidness in the FSSA aligned with those in the FSAP?||Rating: |
|28. Have the FSAP recommendations been clear?||Rating: |
|29. Have the FSAP recommendations been usable (e.g., specific as opposed to general)?||Rating: |
|30. Have the FSAP recommendations been prioritized (laying out a strategic agenda with attention to sequencing needs)?||Rating: |
|31. Have the FSAP recommendations taken into account “technical” country specific capacity constraints (ex ante constraints)?||Yes/No, describe.8|
|32. Were recommendations in FSAP followed up consistently in IMF/World Bank programming, operations (technical assistance, lending and analytical work), updates or other forms of financial sector surveillance (programs, Article IV)?||Rating: |
|33. Has the FSAP led to a better understanding of future World Bank/IMF assistance needs for a country?||Yes/No, describe.|
|34. Has the subsequent World Bank/IMF assistance matched the identified needs and recommendations by FSAP?||Yes/No, describe.|
|III. Outcomes: Policy and Institutional Change and Strengthening of Financial Sector|
|35. Have recommended actions/reforms been effectively carried out by country authorities? In answering assess if the momentum for reforms has picked up after the FSAP (or after an FSAP update for cases where this is applicable).||Rating: 9|
|36. Do documents give ex post reasons and constraints for lack of financial sector reform (and implementation of recommendations)?||Yes/No, describe.|
|37. Have FSAP influenced World Bank/IMF supported program design?||Yes/No, describe.|
|38. Have FSAP updates evaluated the implementation of initial FSAP recommendations?||Yes/No, describe.|
|39. Did FSAP/Article IV mention constraints and limitations of what can be expected from FSAP? 9||Rating:|
|IV. Integration with IMF Surveillance|
|40. How is FSAP included in Article IV (staff report, especially staff assessment, sometimes Selected Issues papers)?11||Rating:|
|41. Are FSAP Updates (only Updates, not FSAP itself) findings discussed and reported during Article IV consultations?||Yes/No describe (where applicable).|
|42. Have Article IV, programs, and TA assessed implementation of reforms?||Yes/No, describe.|
|43. Are the linkages between the macroeconomic variables and financial sector (both ways) well integrated in Article IV?||Yes/No, describe.|
|44. Has the intensity of coverage of financial sector issues in the Article IV consultations waned with time (in subsequent Article IV) after the attention received initially with the FSAP?||Rating scheme:|
|45. How was the FSSA reported and discussed at the Executive Board meeting? Does the PIN reflect FSSA main findings? 12||Rating:|
|46. Evaluate the overall FSAP/FSSA documents presentation.||Describe.|
Some broader questions, such as those regarding more general quality aspects of FSAPs, required combining several specific questions into a broader qualitative assessment that is discussed in the main report.
The full sample of case studies was combined into an overall database that permitted the analysis of specific questions and ratings by the various characteristic used in the sample selection, including vintage of FSAP, systemic importance of the country concerned, and so on.
Structured interviews with staff used a common set of questions across countries, tailored when necessary to the specific elements of the case. Interviews served two purposes: to triangulate evidence collected through other sources; and to obtain evidence specific to the stakeholder’s role in the FSAP process.
The interviews with country authorities were conducted by phone or in the context of a visit to the country.2 The evaluation team visited 14 countries of the 25 case studies.3 In these cases, the team held extensive consultations with a variety of country authorities, which typically included senior officials at the central bank, the ministry of finance, and various supervision agencies and regulatory bodies.
Since IEO’s evaluation of the FSAP was in parallel to that of the World Bank’s IEG (meaning extensive collaboration on all aspects of inputs into the evaluation process), IEG staff was invited to attend these interviews. Conversely, IEG held interviews with World Bank staff where IEO staff was invited to participate.
All country authorities were given an opportunity to provide feedback on their experience with the FSAP.
Countries visited included Bulgaria, Cameroon, Costa Rica, Germany, Ghana, India, Ireland, Japan, Jordan, Korea, Romania, Russia, Slovenia, and South Africa.
This annex supplements the discussion in Chapters 1 and 2 of the main report with additional information on arrangements for setting priorities; outcomes in terms of country and sectoral coverage; resource costs of the exercise; and some aspects of FSAP mission organization.
Objectives and Procedural Arrangements for Setting Priorities
While the criteria for setting FSAP priorities across countries and for the selection of topics covered within countries have been modified over time—in the direction of greater selectivity—the core of the approach remains unchanged.
For country participation, the IMF and World Bank Executive Boards endorsed a variety of criteria for setting priorities at the time of the initial review of the pilot stage, including a country’s (1) systemic importance; (2) external sector weakness and financial vulnerability; (3) features of its exchange rate and monetary policy regime that make its financial system more vulnerable—such as inconsistencies with other macroeconomic policies; (4) likelihood of upcoming major reform programs (as reflected for example in the Bank’s Country Assistance Strategy);1 and (5) geographical balance among countries. It was noted that country selection should seek to maximize the program’s contribution to the strengthening of national and international financial stability, and thus within any one year should give priority to systemically important countries (SICs). The SICs were defined as (1) countries whose capital markets intermediate the bulk of global financial transactions; and (2) emerging market economies whose financial systems have the potential to cause, or be subject to, undue volatility in cross border flows and financial system contagion. However, no explicit list of such countries has ever been made public.2
The criteria were revised by the Executive Boards in March 2003 to take into account the need to accommodate countries’ requests for FSAP reassessments and updates while balancing the expectation that all member countries would benefit from the program. While noting the need to give continued priority to industrial and emerging market economies of regional or international systemic importance, priority for reassessments and Updates would be given to countries where there have been major developments in the financial system, or a lengthy period had passed since the last assessment. These criteria were further refined in the 2005 FSAP review by noting that the interval between assessments should be shorter for SICs, and for countries where there have been significant macroeconomic shocks to the financial system, or major reforms have taken place. Furthermore, an earlier update may also be appropriate in cases where important financial sector issues were not covered in the initial FSAP but are deemed to warrant an in-depth analysis.
A few things are worth noting with regard to these criteria. First, even though they are multidimensional and with no specific weighting attached—hence leaving room for discretion in their implementation—they clearly signal the major importance attached by the Executive Boards to having systemically important countries’ participation, both in the initial assessments and the Updates. Second, in any given year, there is an expectation that priority should be given to these countries. Third, these countries should also be reassessed more frequently. Finally, the notion of systemic importance encompasses both regional and global dimensions.
Box A4.1.Financial Sector Liaison Committee Guidelines for Country Selection
The Financial Sector Liaison Committee (FSLC) developed guidelines covering all stages of the FSAP process, including country selection and scheduling, FSAP teams leader selection, terms of reference for FSAP missions, and preparation, discussion, and transmittal of FSAP reports.1 For country selection, the FSLC envisioned an iterative consultative process involving the regions in the World Bank and the area departments in the IMF. Consultation would be undertaken every six months to prepare rolling FSAP work plans covering the year ahead. These procedures were endorsed by both Executive Boards.
Under these procedures, IMF area departments and Bank regions would each identify candidates for inclusion in the program with no requirement for agreement at the initial stage. Countries were to be ranked in three groups, highest (1), high (2), and medium (3) priority in accordance with the criteria established by the Boards, with a view to generate “first-best” lists. Country authorities’ willingness to participate in the program was not supposed to be considered at this stage. MFD, and its counterpart in the Bank, the Financial Sector Vice-Presidency (FSE), would then compile separate “firstbest” lists to be discussed by the FSLC. Discussions would seek to reconcile differences in ratings, preparing a modified “first-best” list of countries that would be sent to departments and regions for comments. MFD/FSE would then send a final “first-best” list to their respective managements for approval, noting remaining differences. On the basis of the consolidated list approved by the two managements, and now taking into account other considerations (i.e., resources availability, timing for Article IV consultations), MFD/FSE would follow up on country participation to prepare operational lists for the coming year, based on countries responses.
In addition, country authorities may independently request participation in the FSAP, but in such cases the relevant Bank region and IMF area department need also to agree that the country is a suitable priority candidate for participation.
If a country’s participation in the FSAP is considered to be of high priority by IMF and World Bank staff, but the country is reluctant to commit to participating, the staff can seek the assistance of Bank and Fund management in encouraging the country’s participation.1 See, for example, the Attachment in “Progress Report on the Bank-Fund Financial Sector Liaison Committee” (SM/01/295, September 2001).
For topic coverage, FSAP policy documents sought to establish from the inception of the initiative “an approach to financial system assessments that is broadly consistent across countries, while allowing for difference in emphasis to reflect different country circumstances.”3 To this end, the staff of the IMF and World Bank developed a common template covering all important sector and issues, from both stability and developmental perspectives, but with the understanding that FSAP teams would tailor its application to country circumstances. Comprehensiveness was viewed as key to ensure that major vulnerabilities and financial sector needs are not overlooked. However, judgments on cost effectiveness and relevance were to be used to set the scope of work in each case.
The approach of comprehensiveness in scope but selectivity in depth of coverage was developed further in the 2003 review. This called for FSAPs to be more sharply focused and tailored to individual country circumstances while maintaining a broad overall assessment. The varying depth of analysis across sectors would be achieved along various dimensions including (1) the selection and timing of standards and codes (S&Cs) to be formally assessed—in principle to be limited to no more than three plus AML/CFT; and (2) the extent of quantitative analysis. Furthermore, in low-income countries with small financial systems, the approach called for greater focus on medium-term and structural issues. It was envisioned that topics and S&Cs that warranted an assessment but were not covered under an initial FSAP could be included in FSAP Updates.
The Financial Sector Liaison Committee, a joint committee of senior IMF and World Bank staff, was responsible for coordinating many aspects of the FSAP, including the selection and sequencing of countries (Box A4.1).
One key objective of the FSLC guidelines was to generate a “first-best” list of countries, with a view to identify a pool of countries in accordance with the criteria set by the Boards. This “first-best” list was considered a key input into the preparation of a working plan. In particular, area departments and regions were instructed not to be constrained in their selection by their understanding of country authorities’ willingness or unwillingness to participate. This was important because the signaling of how the IMF and World Bank viewed the importance and priority of a particular country’s participation was expected to influence incentives to participate. The survey results provide some support for this view.
Outcomes for Country and Topic Coverage
The evaluation undertook a detailed examination of how this process worked in practice based on a review of internal documents and interviews with staff involved.4 Tracking each stage of the process suggests the following (see Tables A4.1 and A4.2):
The share of SICs that are given the highest rating by IMF area departments has fallen over the period FY2002–05.5 The reasons are not fully clear, but in some cases area departments appear to be accepting countries’ reluctance to volunteer.
In effect, each institution has the ability to “block” countries appearing on the joint priority list that is signaled to the two managements. This has led to some cases of countries not been signaled as priorities for FSAPs because of concerns that to do so might disrupt the broader relationship with the country.6 Thus, when a country was rated 1 by one institution and 3 (or not rated at all) by the other, typically that country was not included in the working plan of the FSLC. This was the case with Turkey and Malaysia, which were rated 1 in three successive submissions by IMF area departments, but were either rated 3, or not rated by the World Bank regions. For the FY2005 submission, these countries were dropped from the ratings.
The share of SICs that are being flagged as high priorities for Updates is very small. One-third or less of the “pool” of SICs where the lag between initial FSAP and completion of Update would be at least four years are being signaled as “high priority.”
A very high proportion of nonsystemically important countries rated as the highest priority were included in that year’s program.
|Source of Submission and Outcomes|
|Ratings2||IMF area departments||FSLC-IMF||FSLC-World Bank||List to managements||Outcome4|
|Submission date: spring 2001 for FY2002|
|SICs priority ranking|
|Non-SICs rated priority “1”||18||18||18||24||15|
|Memorandum item SICs remaining in pool3||21||21||15||…||…|
|Submission date: spring 2002 for FY2003|
|SICs priority ranking|
|Non-SICs rated priority “1”||11||…||…||15||11|
|Memorandum item SICs remaining in pool3||16||16||11||…||…|
|Submission date: spring 2003 for FY2004|
|SICs priority ranking|
|Non-SICs rated priority “1”||14||4||14||14||13|
|Memorandum item SICs remaining in pool3||12||12||9||…||…|
|Submission date: spring 2004 for FY2005|
|SICs priority ranking|
|Non-SICs rated priority “1”||15||13||11||16||14|
|Memorandum item SICs remaining in pool3||9||9||7||…||…|
|Source of Submission and Outcomes|
|Ratings2||IMF area departments||FSLC-IMF||FSLC-World Bank||List to managements||Outcome4|
|Submission date: spring 2003 for FY2004|
|Non-SICs rated priority “1”||3||…||…||7||4|
|Memorandum item SICs remaining in pool3||5||5||4||…||…|
|Submission date: spring 2004 for FY2005|
|Non-SICs rated priority “1”||3||2||3||4||3|
|Memorandum item SICs remaining in pool3||9||9||8||…||…|
In sum, while all SICs that volunteer for the program are implemented in a timely manner, the number of such “volunteers” is declining markedly, especially for Updates, and the current system is no longer providing an effective signaling of priorities independent of such countries’ willingness to volunteer.
Finally, data on the number of detailed standards and codes assessed per FSAP indicate that the greater selectivity called for by the Boards has been implemented as planned (Table A4.3):
The average number of standards assessed (excluding the AML/CFT) has declined to under 3 since the 2003 review.
Fewer standards are being assessed in emerging market economies and even fewer (an average of only 2) in low-income countries.
The banking standards (Basel Core principles) are assessed in almost all cases.
|Standards and Codes|
|Basel Core principles|
|Total average number per country, excluding AML/CFT|
Resource Costs of the FSAP
To track costs of the FSAP for the two institutions, we combined data from the IMF (MFD) and World Bank on the expenses incurred on each FSAP (staff and experts’ time plus travel costs, but excluding overhead). While there are some differences in the way the cost data are compiled in the two institutions, we do not think the differences are sufficiently great to affect the overall conclusions.7
The main findings (see Table A4.4) are as follows:
The average direct cost has declined by about 6 percent between the pre- and post-2003 review periods, with a 10 percent decline for the IMF partially offset by a rise in average cost for the World Bank.7
The realized savings are driven by a sharp decline in the average direct cost of FSAPs in advanced countries, while the average direct cost of FSAPs in both emerging and PRGF-eligible countries have not changed much.9
Resource savings generated by the reduction in the number of formal assessments of standards and codes in emerging and PRGF-eligible countries have been reallocated to other activities in the FSAPs for these countries.
Post-2003, the average direct cost of FSAPs in emerging and PRGF-eligible countries is broadly the same (around $690,000), which is surprising.
The IMF still incurred the larger share of FSAP costs for assessments in PRGF-eligible countries after the 2003 review (about 60 percent).
FSAP Updates for which direct cost data are available suggest the potential for some savings of resources, although this would depend on the type of reassessment. Estimates range between $45,000 for a very narrow Update (Iceland) to $342,000 for a significantly more comprehensive Update (Kazakhstan), which is half the average cost of initial assessment for a PRGF-eligible country.
|Country Type||IMF Average2||World Bank Average3|
FSAP Mission Organization
The FSLC allocates FSAP team leadership responsibility between Bank and Fund staff. In practice, overall team leadership has been divided equally (Table A4.5). The IMF staff have led all FSAP teams to advanced countries, about 40 percent of teams to emerging market economies, and one-third of teams to low-income countries.
During interviews with the evaluation team, some senior Bank and IMF staff expressed concern over what they perceived to be the increasing appointment of team leaders with little experience. In their view, this runs the risk of lowering the quality of FSAPs as the task of mission leader requires the judicious combination of technical skills and balanced macrofinancial policy judgment necessary to form sound overall assessments.
An examination of trends in the experience of FSAP team leaders (defined in terms of their participation in previous FSAP missions) suggests that there may be some basis for this concern. After rising steadily as the FSAP exercise matured, the “average” previous experience of FSAP team leaders began to decline again in FY2005 and the share of team lead-ers with no or very limited previous FSAP experience began to rise (Figures A4.1 and A4.2).
Figure A4.1.FSAP Mission Leaders and Experience1
Sources: Travel Information Management System and IEO estimates.
1 A “unit” of experience is participation in an FSAP either as a team member or in a leadership capacity.
This criterion was not explicitly identified in the Chairman’s Summing Up, but it was fleshed out in subsequent reports and internal memoranda.
In a table listing the FSAPs that have been initiated through FY2003, the staff identified the following economies (among those in the list) as being systemically important: Argentina, Brazil, Canada, Germany, Hong Kong SAR, India, Japan, Korea, Luxembourg, Mexico, Russia, Singapore, Switzerland, and the United Kingdom. There was no complementary list identifying the systemically important countries that had not participated up to that point.
“IMF–World Bank Financial Sector Assessment Program (FSAP)” (SM/99/116, May 1999).
The documentation is somewhat fragmentary, particularly for FY2003 and FY2004, and an attempt was made to reconstruct country ratings based on e-mail communications.
One reason why the IMF list submitted to the FSLC has many more “high-priority” ratings than does the similar list from the World Bank is because the former includes advanced economies (including the G-7).
The FSLC uses the ratings given by the regions and area departments in the preparation of the working program. For example, whenever a country is rated 1 by both a region and area department, it is highly likely that it makes into the list, followed by those rate 1 by one of them and 2 by the other. For countries for which the IMF is solely responsible, for example the G-7, generally it suffices to have a rating of 1 to make it into the working program. The evidence shows that there are some exceptions to these working rules, reflecting at times the fact that country authorities have indicated their preference not to participate or to do so at a later date. There are also cases where a country is rated low by an area department or region, but it is nevertheless included in the working plan reflecting the drive to achieve geographical and developmental diversity, or because of a substantial lag since it volunteered.
IMF data on staff resource costs are calculated from the amount of staff and expert time spent on each FSAP (obtained from the Budget Reporting System) and applying standard labor cost factors (salaries plus benefits) provided by MFD (at an average of about $200,000 per person-year). World Bank data are actual dollar budgetary expenses reported under the FSAP accounts. The IMF data do not include expenses associated with AML/CFT assessments, whereas the World Bank data include such costs. For earlier cost estimates by the staff, see “FSAP—Indicative Fund Resource Costs” (SM/03/77, Supplement 4, February 2003), and “Financial Sector Assessment Program—Indicative Fund Resource Costs” (SM/05/67, Supplement 2, February 2005).
The post-2003 review period corresponds to the latter part of FY2003 and FY2004.
These statements and the data reported in Table A4.5 treat the FSAP for the Eastern Caribbean Currency Union (ECCU) as a single assessment. However, excluding this case does not alter the broad conclusions.
This annex provides additional information to supplement the discussion in Chapter 3, section on “Macroprudential Risk Analysis,” drawing on a review of stress-testing approaches in the 25-country sample that was examined in depth. It begins with a brief summary of the approaches most often encountered in FSAPs, proposes a possible approach to providing “benchmarks” of methodological approaches that different country peer groups could aim for, and concludes with some comments on key areas that require greater attention.
Current Stress-Testing Methodologies
FSAPs incorporate stress-testing approaches of varying degrees of sophistication. However, in reporting results, most FSAPs rarely discuss the limitations of the methodologies used and the consequent need for caution in interpreting results. We summarize here the most common approaches used.1
FSAPs in low-income countries and some emerging market countries have frequently used credit-risk methodologies based on a simple static exercise that assumed (relatively arbitrary) increases in levels of banks’ nonperforming loans together with assumptions on different provisioning levels. Usually the assessment is supplemented by a simple analysis of the direct effect of exchange rate risk, based on the application of different exchange rates to the net open position of the entire banking system. The results that can be extracted from these models are very limited.
Methodologies based on individual portfolios have been used in more advanced economies, which use highly disaggregated data from individual financial institutions (bottom-up approach). In order to conduct stress testing, one of the challenges of these models is to be able to translate the effect of a broad macroeconomic shock into a balance sheet of a financial institution. Usually this exercise requires a mapping of macro variables into a set of common risk factors that can be applied to stress individual balance sheets. Typically, institutions require two steps, one mapping from macro adjustment scenarios to a set of common risk factors, and another mapping from a set of common risk factors into all of the instruments in a portfolio. The results that can be extracted from these models are more precise in the risk measurement.
Methodologies based on aggregated portfolios have been used in some emerging market economies (top-bottom approach), that typically derive common parameters from all financial institutions in the data set through regression analysis. However, important differences have been found among the use of stress testing according to this methodology that seems to reflect a lack of a common view on certain issues such as the way in which the corporate sector or household sector risks should be included in the evaluation of the financial sector vulnerabilities.
A Possible Approach: Country Peer Groups
Stress-testing methodologies differ substantially among FSAPs, which can be attributed in large part to data constraints, relative sophistication of the financial system, cooperation from the authorities, time available for the analysis, and the judgment of the FSAP team.
While the need to tailor stress tests to country conditions is understandable, in our examination it has not been easy to find common elements among FSAPs’ stress tests, except for many cases in which methodologies converge toward the most simple approaches.
From a more dynamic perspective, in a number of cases there are no significant methodological improvements between the FSAP and its Update three or four years later.2
One possible approach to strengthening the methodological approach, and building greater cross-country knowledge, would be to develop good practices for conducting stress tests among various country peer groups. Such country peer groups would reflect common macroeconomic conditions, as well as the degree of sophistication of the financial sector of a country. This approach could help countries adapt their methodologies to good practices within groups with comparable capacity and data limitations. It would also help to recognize that stress testing practices can substantially differ between countries with widely varying levels of financial complexity. Standardizing a core set of methodologies, data sets, and sensitivity analysis within country peer groups could also help to develop benchmarks for cross-country comparisons, thus facilitating vulnerability analysis.
In addition, there is some room for standardization of certain shocks under certain circumstances. For example, one possibility would be that all noninvestment grade countries evaluate the potential effects of sovereign downgrade scenarios. Similar approaches could be used for shifts in exchange rate pegs. Greater standardization of such approaches across countries could help reduce their political sensitivity and help avoid an inadvertent signal that the IMF thinks such events are more probable in certain countries.
It would be useful for the FSAP to provide advice in the design of a road map for reaching the relevant country peer benchmark for stress testing, beginning with recommendations on the data that are necessary in order to run more appropriate stress testing. This could help countries to build financial infrastructure, collect data, and allocate resources to foster a better understanding of the vulnerabilities of the financial system.
Areas That Require Some Attention
Credit risk is the most important risk from the banking sector. In measuring credit risk, emerging markets should make efforts to move from static models to regression models that relate credit exposure to macroeconomic events in a systematic manner. It would be necessary to establish good practices for including corporate and household sectors’ exposures within the regression approaches. Although the scenarios or events may not have an associated probability of occurrence, the interpretation of the stress testing results should provide an opinion of the relative importance of the different vulnerabilities (credit risk, market risk, commodity risk, etc.) of the financial system.
The size of the shocks should reflect exceptional but plausible events. In the 25 country cases, we found that, even in recent vintages, there is insufficient explanation for the size of the shocks and insufficient use of macro models to simulate the effects of the certain scenarios and events on economic and financial variables as outputs.3 Although it is desirable that shocks be derived from macro models, some countries are not yet in a position to do so. In these cases, good practices should also be established for the simulation of scenarios and events, for example by considering methodologies that look at the joint empirical distribution of risk factors.
For example, Ghana, Kazakhstan, and Slovenia.
For example, one scenario might include a sudden reversal of capital flows and a rapid depreciation of the exchange rate. Macro simulations of this scenario could produce effects on GDP growth, price level, interest rates, and the exchange rate. These outputs would serve as the basis of a stress test of balance sheets for individual institutions.
This annex presents the analysis supporting the findings in Chapter 5, section on “Surveillance,” of the main report. The evidence is drawn from the detailed reviews of the 25 case studies and is structured around four evaluation questions: (1) Have FSAPs provided a good basis for their use in Article IV surveillance? (2) What has been the resulting coverage of findings in Article IV reports? (3) How was the integration of key FSAP findings into the overall macroeconomic assessment? and (4) What was the extent of financial sector surveillance after the FSAP?
Have FSAPs Provided a Good Basis for Their Use in Article IV Surveillance?
The evaluation criteria for case analysis comprised the following tests (the questions of the desk review matrix used are in parentheses):1
(1) Have FSAP results been articulated in a way that facilitates use in Article IV surveillance? More specifically:
Have findings and recommendations been clear, usable, prioritized, with a view to consequence? (Questions: 31, 33, 34, and 35)
Has there been an overall assessment of the financial sector? (questions: 8 and 20)
(2) Have domestic and external or global economic conditions fed into the financial sector assessment? (Questions: 15, 16, 17, and 18)
The results from the reviews of the 25 case studies show that:
Over 80 percent of cases reviewed showed that FSAPs articulated findings well, with a general assessment of the financial sector standing, facilitating the use in Article IV surveillance. Findings and recommendations are generally clear and usable. Moreover, nearly half of the cases had a very clearly articulated overall assessment, with a good sense of consequence and priorities.
Over 80 percent of cases reviewed showed that domestic and external conditions fed into the FSAP’s analysis, with a third of cases showing a very clear and strong linkage to the financial sector assessment.
In good practice cases, the FSAP’s articulation of the overall assessment has a comprehensive view of the structure, standing and trends in the financial sector. It sometimes presents also a body of analysis (including sectoral linkages, externalities, and potential spillovers) that facilitates the formulation of a broader assessment of macroeconomic vulnerabilities and a more comprehensive policy framework for recommendations (e.g., Costa Rica).
The instances where the FSAPs have fallen short of good practice involve an overall assessment that is not clearly conveyed or that lacks a clear indication of macro consequence and prioritization (e.g., Kazakhstan, Philippines). In these cases, there is a shift of what should be part of the FSAP’s expert analysis onto the Article IV team.
What Has Been the Coverage of Findings in Article IV Reports?
The analysis covered desk review questions 50 and 51.
Over 80 percent of Article IV reports make a good coverage of FSAP results, and about half of the cases present many of the good practice characteristics. The good practice coverage of FSAP results in Article IV has a clear reporting of the main findings and recommendations, the importance and consequence of findings, and the associated priorities. It also presents the interplay of these findings with the nonfinancial sector (e.g., Japan), including potential fiscal liabilities, impact on or of monetary and exchange rate policy, and corporate and household sector implications. But in some instances the coverage of FSAP results in Article IV reports has fallen short of good practice, with a cursory presentation of findings (e.g., Romania).
Box A6.1.What Makes a Good Integration of FSAP Findings into the Overall Macroeconomic Assessment?
This box presents the key characteristics of good integration of the financial sector assessment into the overall macroeconomic assessment. Ultimately, a good integration is one that results in a greater understanding of macro/systemic vulnerabilities, a more comprehensive stability assessment, and a broader policy recommendation framework. We present here some good practice examples on the various characteristics.
Macro/financial/structural linkages are identified. For example, in the assessment of Japan’s macro-economic situation, a framework with four interlinked pillars was considered, consisting of the financial and corporate sectors, and the monetary and fiscal frameworks.
Externalities are a key element of the assessment. In the case of Japan, the analysis highlights the feedback effects from bank reforms (via improved lending position) to supporting the economic recovery, and from economic recovery (and strengthen regulation) to supporting the resolution of balance sheet problems. Thus, complementarities of policies are emphasized, leading to an assessment of the need for simultaneous adoption.
Risks and vulnerabilities are evaluated in a systemic manner. For example, in Costa Rica, the implications of the exchange rate regime on dollarization and financial vulnerabilities are explicitly addressed. In Korea, the interplay of risks and vulnerabilities between the corporate and financial sectors is explicitly analyzed, considering potential externalities.
Broader policy framework for recommendations. In Japan, drawing on the elements described above, policy recommendations comprised broader and more rapid financial sector reforms, accelerated corporate restructuring, a more aggressive anti-deflationary stance, and a framework for medium-term fiscal consolidation. The “big-bang” policy recommendation was based on the increased effectiveness that was expected from the joint implementation, whereby, for example, monetary policy would be more effective to end deflation if banks’ balance sheets problems are resolved, which in turn build on corporate restructuring.
Enhanced discussions with the authorities based on the broader assessment and policy recommendations. In the case of Costa Rica, the authorities indicated that the appropriate integration of the financial sector assessment with macroeconomic conditions and policies, as well as the comprehensive scope of recommendations for reform (including monetary/exchange rate, fiscal, and financial pillars) helped to form a good, integrated platform for discussions of policy and reform needs.
How Were the Key Financial Sector Findings Integrated into the Overall Macroeconomic Assessment?
The analysis covered desk review questions 50 and 53.
In three-quarters of the cases reviewed, the overall macroeconomic assessment has benefited from the FSAP, with an enhanced assessment of the standing of the financial sector and the areas of potential macroeconomic implications (see Box A6.1). In these cases, the main FSAP findings were reported and merged into the assessment of Article IV surveillance. Moreover, in a third of cases the contribution has been significant, whereby FSAP findings contributed to shaping the overall macroeconomic assessment (i.e., not only the main messages were presented but the findings about the financial sector were well embedded into the overall appraisal).
The instances where practice has fallen short of a good integration into the overall macroeconomic assessment involve a mechanical presentation of FSAP results in the Article IV report (e.g., just a summary table of results) with little integration of the main messages (e.g., Tunisia, Romania). Typically this consisted of reporting on financial vulnerabilities but not considering potential linkages, externalities, or policy complementarities.
What Was the Extent of Financial Sector Surveillance After the FSAP?
The evaluation criteria for case analysis comprised the following tests:
Has Article IV/program work identified new vulnerabilities? (Question: 22)
Has the intensity of coverage of financial sector issues been maintained after the FSAP? (Question: 54)
Have Article IV, program, and TA activities assessed implementation of reforms? (Questions: 39 and 52)
The results from the reviews of the 25 case studies show that:
In only a few cases, financial surveillance under Article IV or program work was able to identified new vulnerabilities or understand better the extent of some risks that were noted in the FSAP (e.g., Costa Rica). In terms of areas, these have included corporate sector vulnerabilities to interest and exchange rates (Mexico), bank vulnerabilities to interest rate risk through their holding of government bonds (Philippines), and risks stemming from links between banks and other financial institutions (e.g., investment funds). The analyses in most instances comprised new stress-testing exercises. In these cases, staff from MFD participated and contributed to the analysis.
In over 80 percent of cases reviewed, the intensity of coverage in Article IV consultations has remained significant (excluding those with an Update and those too recent to have a subsequent Article IV cycle). Although in nearly half of the cases the intensity has waned over time (in terms of the depth and scope), there has not been a full mean reversion of the coverage of financial sector issues to that prevailing before the FSAP. But in some cases, the coverage of important issues highlighted in the FSAP was lost in time (e.g., Korea).
With respect to tracking of progress in implementation of reforms, the general practice has been to report on measures taken by country authorities. But there has been very little appraisal of how significant those measures are to address the vulnerabilities identified in the FSAP. In terms of areas of coverage, there tends to be greater follow-up on traditional area of surveillance (e.g., issues related to exchange rate and monetary policy) and on the banking sector (as opposed to other segments of the financial system, which sometimes receive little or no coverage). In many instances, the scope of coverage has not followed the priorities assigned by the FSAP.
In nearly a quarter of the cases, the tracking was done more comprehensively and in greater depth; these either were cases where an Update took place or where MFD staff participated in the consultations. The cases that had expert assistance were also able to cover a broader spectrum of areas in the tracking of implementation under Article IV work.
Good practice in tracking of implementation of recommendations takes place under the FSAP Updates reviewed among the 25 case studies, where the assessments are the most comprehensive in scope and depth (e.g., Slovenia, Bulgaria). Updates present an overall assessment of progress, whereby the standing of the system and remaining challenges are rearticulated (see also Annex 7 on review of Updates).
See Annex 3 for the template of the desk review matrix.
This annex presents the supporting analysis for the key messages on FSAP Updates that have been incorporated in the main report (see Box 4.3). The evaluation has analyzed all 11 Updates (completed by June 2005) of post-pilot FSAPs. The Updates for the countries that had the FSAP in the pilot phase were not considered since they may have been driven by factors specific to the inception of the initiative (e.g., FSSAs for pilot cases could not be published, even upon the request of country authorities). To conduct the desk reviews, a streamlined template was used based on the one applied in the detailed analysis of the 25 country studies. No additional interviews were held with staff or country authorities.1
The exercise has sought to provide evidence on two main aspects: (1) it analyzed what Updates have done so far in terms of the nature of the exercise (focused versus comprehensive reassessments), stock taking (assessment of implementation of reforms), and the extent to which they have taken a fresh look at vulnerabilities; and (2) it assessed the institutional use and integration with surveillance in Article IV consultations (see Table A7.1).
|Country (Financial Year)||Nature of the Assessment in the Update1||Extent of Stocktaking and Tracking of Implementation2||Degree to Which the Update Took a Fresh Look at Vulnerabilities||Integration with Surveillance and Coverage of Recommendations|
|Iceland (FY2003)||Focus is on stocktaking, with a particular emphasis on the reassessment of the BCP.||Stocktaking is most in depth in the case of the BCP and CPSS core principles (with recommendations centering on the former).||The FSSA provided an assessment of the external indebtedness of the Icelandic financial system (this topic was covered more in depth in an Annex to the 2003 Article IV Staff Report).||The accompanying Article IV Staff Report and PIN reflected the Update’s main findings and recommendations.|
|The Update discussed the bulk of the financial system (although the team consisted of only 2 staff).||There is an overall assessment of progress (even though several sectors received only a general coverage).||Prioritization and sequencing of recommendations are minimal.|
|A clear description of the rationale for the scope of the Update was lacking.|
|Ghana (FY2004)||The Update has primarily a developmental perspective and presents findings and recommendations along three broad themes: competition, integrity, and fairness. But it has also a strong coverage of stability issues.||The tracking of implementation carefully covers the Recommendations of the FSAP, and reports on the reasons for lack of progress.||The emphasis of the Update is not on taking a fresh look at vulnerabilities (stress testing is replicated from the FSAP).||There is a cursory mention of the Update’s findings in the Staff Report of the Article IV.|
|The FSSA presents a clear and some topics that were not candid overall appraisal of progress in implementation.||But the analysis includes The FSSA presents a clear and some topics that were not addressed in the FSAP (e.g., the effects of fiscal consolidation on bank profitability).||However, accompanying program documents draw significantly from the Update.|
|The PIN is cursory on financial sector issues and does not refer to the Update.|
|The key recommendations are clear, well grounded in the analysis, and fairly specific in some cases.|
|Slovenia (FY2004)||The Update was comprehensive with a broad overall assessment.||Tracking of recommendations is in depth (but does not provide reasons for lack of progress).||The Update incorporates a rather detailed fresh look at vulnerabilities (e.g., the housing market, corporate governance, and macroeconomic risks linked to the entrance into ERM2).||The Article IV Staff Report integrates the Update well (the main findings and recommendations are included in a section financial sector issues and in the staff appraisal).|
|There is no explicit rationale for the scope and timing of the exercise (Slovenia’s entrance into the EU and ERM2 has likely played a significant role).||The PIN reports the main Update messages.|
|Prioritization is good (with high specificity and detail of recommendations).|
|Kazakhstan (FY2004)||This exercise provides a comprehensive assessment of the financial sector, including banks, insurance companies, pension funds, and securities market.||The Update reports on implementations of reforms. It provides a fairly in-depth update of reforms in the banking sector, while for other sectors there is more mechanical reporting.||Some new challenges are analyzed, but generally the coverage was similar to that of the FSAP. Stress testing did not improve on the methodology used in the FSAP.||Discussions of the Update in the Article IV Staff Report are only cursory (it only mentions the creation of the FSA). The PIN mentions only briefly the Update.|
|The Update has an emphasis on assessing what has happened since the FSAP.||The FSAP presented a wide range of poorly prioritized recommendations, and the Update does not provide a clearer order of the previous and new recommendations.|
|The FSAP was not very explicit about the rationale behind the timing and coverage of the exercise.|
|El Salvador (FY2004)||The reassessment is rather focused, with coverage of the banking sector, cross-border operations, payments and securities settlement, and central bank functions. On development issues, it covered microfinance.||The analysis of progress in implementation of reform was in depth for the core sectors of focus. Other financial segments received a more mechanical reporting on implementation.||Some new vulnerabilities are included, but the emphasis is placed on topics that were covered previously.||Main findings and recommendations covered in Article IV reports. The PIN makes a general mention of the financial sector being generally in good health.|
|No rationale for the scope of the exercise was presented.||There is no overall assessment of progress in implementations.||Stress tests are largely similar.||Prioritization is not prominently discussed in the report.|
|The report presented an overall assessment of the financial sector standing, despite that the scope of analysis was rather focused.|
|Senegal (FY2005)||This is a focused Update. The terms of reference indicate that development issues will be the main focus.||The Update provides in depth follow-up on selected sectors: banking, microfinance, and legal issues. FSAP recommendations on other (smaller) sectors (insurance and pensions) are not followed up.||Stress tests are more elaborated than in the FSAP, but cover the same type of risks||The Article IV Staff Report has limited coverage of the Update. The PIN does not mention the Update.|
|There is limited explanation on the reasons for the lack of progress.||Prioritization of recommendations is weak.|
|Colombia (FY2005)||The exercise has a broad coverage and the emphasis on stability and development issues is more balanced than in the FSAP (which focused on stability in the aftermath of financial turmoil). The Update is to a good extent a stocktaking exercise.||The analysis of progress in implementation of reforms is generally thorough. The reasons for lack of progress are generally well explained.||Efforts to capture new vulnerabilities were somewhat limited. Stress tests are largely similar to those in the FSAP.||The most important findings and recommendations are reported in the Article IV Staff Report. The PIN also presents the main messages.|
|No explicit rationale is mentioned for the timing and scope of the Update.||No specific overall assessment of progress in implementations.||Prioritization is reasonably well done.|
|Peru (FY2005)||Overall, the Update provides comprehensive analysis of the issues raised in the FSAP, and of remaining vulnerabilities and developmental needs.||The tracking of implementations is in depth, although findings are scattered through the document.||The Update covers remaining vulnerability, but the implications of recent developments in the banking sector are not followed up in the analysis of risks.||The Update is adequately discussed in the second review of the Stand-By Arrangement. The PIN also conveys the main messages in the Update.|
|The report indicates that, since the FSAP had focused on the resolution of financial sector problems, the Update would cover remaining stability and developmental issues.||The Update provides reasons for areas with a lack of progress.||Prioritization is somewhat weak; e.g., the overall assessment does not indicate the priority actions.|
|Armenia (FY2005)||The assessment is fairly comprehensive, taking into consideration the small size of the financial system (especially outside of the banking sector).||There is detailed tracking of implementation of FSAP recommendations. Progress is reported, but the reasons for lack of advancement are not always provided.||Some new sectors were covered: pension system and housing finance.||The findings and recommendations have a prominent place in the accompanying PRGF document. The Update’s main conclusions are well embedded in the document.|
|The Update has a balanced focus on stability and development. The latter includes a detailed analysis of non- bank financial sector development||The Update captured newly emerging risks related to the rapid credit growth and the monetary consequences of remittances.||Although the Update provides some sense of urgency, the list of recommendations appears too long for effective prioritization.|
|Hungary (FY2005)||The Update’s nature comes close to a full reassessment, with the overall appraisal covering the broad financial system.||The tracking of implementations of reforms is comprehensive across the sectors and standards and codes assessed in the FSAP.||A new look at vulnerabilities identified new risks (e.g., the growth of the nonbanking sector, the rapid growth of unhedged foreign currency borrowing by households and small and medium-sized enterprises (SMEs).||The coverage of findings and recommendations in the Article IV Staff Report is limited. The PIN only reflects on the risks from household and SME borrowing.|
|The report was not very explicit about the rationale for selection and scope (the accession into the EU and ERM2 likely played a role).||But the degree of depth across sectors varies, and the reasons for lack of progress are not provided.||Although recommendations are clear, prioritization is limited to a time classification of medium and long term.|
|Uganda (FY2005)||The Update focuses on developmental aspects to improve improve financial intermediation, but also pays attention to overall financial stability.||The tracking of implementation of reforms is thorough and tends to provide the reasons when there is lack of progress.||The Update does not explicitly conduct an assessment of new vulnerabilities.||The discussion of Update findings and recommendations in the accompanying PRGF review covers the most important findings. The coverage in the Article IV report is minimal (largely limited to a Selected Issues paper). The PIN discusses the main findings and recommendations.|
|There is little discussion of behind the timing and scope of the exercise (the Update was conducted relatively soon after the FSAP).||There is a concise overall the rationale assessment of progress in implementation of FSAP recommendations.||The stress testing is somewhat more sophisticated than in the FSAP.||Prioritization of recommendations is largely limited to a classification of medium and long term (except those for the pension system that provide guidance on sequencing).|
The issue of country selection for Updates is covered in the analysis of the priority-setting process (see Annex 4).
Three of the countries with Updates were covered in the 25 case studies. For these countries, interviews with staff and authorities did cover their experience with the Updates.
This annex presents additional evidence on financial sector surveillance in countries that have not undertaken an FSAP, to supplement the discussion in Box 4.3 of the main report.
To assess the extent to which financial sector surveillance conducted exclusively under the aegis of Article IV consultations has been a good substitute for analysis and assessments under the FSAP, the evaluation reviewed the content of such surveillance in five systemically important countries that had not participated in the FSAP as of mid-2005 (China, Malaysia, Spain, Turkey, and the United States). For each country, documents for the last two Article IV consultation cycles (as well as any program documents, where applicable) were reviewed vis-à-vis four basic components. For each component, the standard of comparison was what one would normally expect to see in an FSAP assessment (see Table A8.1):
Scope of coverage. The coverage in FSAPs is, by design, expected to be comprehensive.
Detail and specificity of analysis. In FSAPs, the depth of analysis of specific sectors depends on the level of development and systemic importance—and can include stress testing, market infrastructure analysis, and review of the regulatory and supervisory framework.
Overall assessment of financial sector vulnerabilities. A key value added of FSAPs is the ability to present an overall, comprehensive assessment of the standing and vulnerabilities in the financial sector—especially those of macro and systemic importance. This is perhaps the most important test of the exercise.
Reported discussions with authorities. FSAPs are expected to lead to extensive and detailed discussions with authorities, including on differences of views on findings and on approaches to deal with vulnerabilities. The review of the five cases discussed here covers only what the staff reports say about the discussions. Actual discussions may have been more encompassing than reported.
|MFD and ICM Participation2||Scope of Coverage||Detail and Specificity of Analysis||Overall Assessment of Financial Sector Standing and Vulnerabilities||Reported Influence in Discussions with the Authorities|
|China||MFD participation in 2004 (1).||Banking sector only.||Mostly description of situation with some notion of urgency; greater specificity in detailed 2003 Selected Issues paper on banking.||View on progress with banking reforms and how to proceed with stock and flow problems.||Strong recommendation to participate in FSAP to guide next reform steps.|
|Malaysia||MFD (1 in 2003 and 1 in 2004) and ICM participation (1 in 2003).||Banking and capital markets (Selected Issues paper on capital market trends); corporate sector analysis (Selected Issues paper in 2003).||Limited coverage and detail in Staff Report; Selected Issues papers have better specificity and descriptive analyses. Little original risk assessment.||Attempt to have an overall view but with limited supporting analysis (e.g., general statement that “financial system soundness has improved…”).||Reporting of discussions mostly on areas of agreement.|
|Spain||MFD participation in 2004 (1).||Banking (and cajas).||There was increased coverage of banking sector in 2001 due to concern over exposure to Latin America. Some reporting of stress testing by authorities in 2003, but no coverage of other segments of financial system.||No integrated, overall assessment of financial vulnerabilities; separated reference to real estate risks and political influence in cajas.||Staff welcomed authorities’ request of FSA. Little reporting on discussions and exchange of views.|
|Turkey||MFD participation (1 in 2004; 2 in 2002).||Banking, corporate sector.||Review of crisis and reform program. Good detail on bank-king (regulation and supervision). Detailed analysis of draft banking law (by IMF Legal Department).||View on remaining crisis- related reforms, little on Overall financial sector Standing.||Little reporting on discussions, mostly on areas of agreement.|
|United States||MFD (3 in 2004) and ICM participation (1 in 2003 and 1 in 2004).||Banking, government sponsored enterprises, corporate governance, and defined-benefits pensions.||Limited original analysis of risks, regulations, and supervision; no coverage of large and important segments of financial sector and cross linkages. Greater depth in 2004 with detailed Selected Issues paper on large complex banking groups.||No overall assessment of financial sector vulnerabilities.||Mostly reporting on authorities’ views; little reporting on discussions.|
Moreover, FSAPs are expected to lay out clear, usable, and prioritized recommendations. The review of these cases of financial sector surveillance outside of the FSAP suggests that none was able to generate a comprehensive set of prioritized recommendations.1
Some of the cases reviewed have included recommended actions in staff appraisals. These have been at a relatively broad level of generality to deal with specific sectoral risks (e.g., China and Turkey on banking issues).
This annex presents further details on the econometric evidence on the links between FSAPs and IMF-supported programs, discussed in Chapter 5, section on “Links with IMF-Supported Programs,” of the main report.
The analysis examined the relationship between FSAPs and the total number of program conditions (prior actions, structural performance criteria, and benchmarks) on financial sector issues for all programs approved between 1995 and 2003. There were 93 programs approved over the period, of which 23 had FSAPs undertaken up to two years prior to the start of the program. This time frame was chosen to capture the notion that at least some of the FSAP’s finding become dated after a few years and hence the relevance to guide program design on financial sector issues wanes over time.
FSAPs have been undertaken both in countries with relatively sound financial systems and countries whose financial systems would require significant reforms to improve their soundness and foster their development. To help distinguish between different types of countries, we use an index of financial liberalization. This index was developed by the IMF Research Department and it attempts to capture the extent of distortions in the operating framework of financial systems by assessing various characteristics (e.g., government-mandated credit allocation regulations, banking sector entry barriers).1 The value of the index is normalized to the [0, 1] range, with 0 being a fully repressed system and 1 a fully liberalized system. Some properties of the data are presented in Table A9.1.
|Number of Program Conditions1||Financial Sector Liberalization Index2|
|All programs||Of which:|
|All programs||Of which:|
|Number of observations||93||23||93||23|
The econometric results are presented in Table A9.2. They show that both the FSAP and the combined variable FSAP*FSLI are statistically significant at the 90th percentile. In principle, having an FSAP would tend to increase the number of program conditions on financial sector issues. However, this effect would decline, and could in fact be reversed for those countries whose financial systems are less distorted. In other words, the fact that a country has undertaken an FSAP tends to result in one or more structural conditions in subsequent programs for countries with highly repressed financial systems, whereas there is no increase in structural conditionality following an FSAP in countries with more liberalized systems; indeed, for countries that already have open financial systems with few distortions, a previous FSAP is associated with fewer structural conditions. This could be interpreted as reflecting the fact that, in such circumstances, the FSAP contributes to greater understanding of the financial system and judgments that structural conditionality is not warranted.
|z||P> |z|||[95 Percent|
|Poisson regression||Wald Chi2 (3) = 8.4||Pseudo R2 = 0.042|
|Number of observations = 93||Prob > Chi2 = 0.039|
See Abiad and Mody (2003) for an explanation on a precursor to the index used in the regression reported in Table 2. The financial liberalization index covers such issues as the extent of free determination of interest rates, credit allocation by intermediaries, and entry barriers, certain features of the regulatory and supervisory frameworks, and capital account transactions regulations.
This annex presents supporting evidence collected in the course of the evaluation on the impact of the FSAP process in each of the 25 case studies (see Table A10.1). The sources of evidence comprise mainly interviews with country authorities; interviews with staff and reviews of documents served a complementary role.
|Country (Date of FSAP Completion)||Contribution to Policy Discussions||Evidence on Contribution to Policy and Institutional Changes1||Policy Process in General|
|Brazil (12/2002)||Senior officials indicated that the FSAP contributed to interagency discussions.||The Financial Stability Report, incorporating more comprehensive risk-based stress testing, launched. It also helped in enhancing the capacity of the government’s debt management office. Some changes in the pension fund and insurance sector also took place. There was progress in restructuring some of the large federal banks and some steps have also been taken to upgrade financial sector supervision to best international practices.||According to former officials interviewed, the FSAP contributed to the design and launching of the Financial Stability Report and to improve the debt management process.|
|Soon after the FSAP, there was an election and change in government which resulted in shifts in the priorities for the financial system: A number of officials interviewed said that this timing of the FSAP adversely affected its impact.|
|Bulgari (7/2002)||The FSAP contributed to discussions by providing an agenda of modernization for the financial sector.||After the FSAP, a new bank insolvency regime was implemented, the Deposit Insurance Fund was strengthened, and central bank powers were enhanced. In the securities area, there were amendments to the public offering law.||According to authorities, the FSAP helped to build momentum at the central bank to enhance the financial sector regulatory framework.|
|According to authorities and one rating agency, FSAP findings were instrument to achieve an upgrade in the sovereign rating|
|Cameroon (6/2000)||The FSAP (together with the one for Gabon) contributed to fostering regional awareness and discussions on the need to strengthen prudential regulations and tighten enforcement.||At the national level, microfinance institutions are being relicensed, and a regulatory and supervisory framework is being put in place in collaboration with the regional banking commission.||The authorities indicated that the FSAP contributed to raise their awareness of the need for a major upgrade in the legal framework, in the broader context of improving transparency and governance.|
|At the regional level, prudential banking standards have been brought closer in line to international standards, and a regional payment system continues to be developed.|
|Chile (8/2004)||According to senior officials, the FSAP contributed to discussions and coordination between the central bank and supervisory agencies.||Chile’s is a very recent FSAP. The Capital Markets Reform II legislation has addressed some of the findings on corporate issues, insurance, and cross-sectoral financial over-sight. Market risk regulations have been introduced, commercial banks’ restrictions to enter into derivative activities have been relaxed, and the legal protection to supervisors is being discussed in congress.||Senior officials indicated that they were already well aware of the issues raised in the FSAP, but that it contributed by providing a comprehensive view and a sense of priority and importance.|
|Authorities reported that the FSAP was useful at a time when the central bank was setting up the Financial Stability Report.|
|Costa Rica (3/2003)||The FSAP contributed to interagency coordination efforts, to discussions with financial sector participants, and in informing discussions with legislators.||Political gridlock has historically interfered with a speedy reform process. The government submitted comprehensive legislation promoting financial reforms after the FSAP, but the legislation failed to pass in congress. Subsequently, the government has submitted some measures through piecemeal legislation.||Senior officials indicated that the FSAP findings and recommendations, as well as discussions by the FSAP team with various authorities, contributed to the piecemeal work after the failed attempt at a comprehensive financial reform.|
|Dominican Republic (5/2002)||Limited initial contribution as authorities did not fully share the FSAP’s main findings and the sense of priority and importance. Substantial contributions during the program negotiations after the crisis.||Some FSAP recommendations were implemented in the postcrisis period, including a draft Monetary and Financial Law.||Financial crisis erupted following the FSAP, and attention was on crisis management. FSAP staff indicated that it contributed to improving the draft banking law.|
|Egypt (11/2002)||FSAP findings (e.g., on insurance) have been used by incoming officials as inputs In their discussions and formulation of sectoral reform agendas||Reform progress has gathered momentum since mid-2004, including the enactment of new banking sector legislation, recapitalization of state banks, and appointment of new management, unification and floating of exchange rate, and announcement of a broad reform plan for the financial sector.||According to senior officials, the FSAP helped to raise the authorities’ awareness of the need to assess the condition of state-owned banks and for wide-ranging financial sector reforms. In so doing, it helped rekindle the momentum for financial sector reform.|
|The authorities indicated that the FSAP findings and recommendations were used as background for the formulation of the government’s Financial Sector Reform Plan launched in mid-2004.|
|Germany (11/2003)||The FSAP contributed to a significant for public discussion of banking consolidation issues. It also raised some “taboo” issues in the insurance sector.||Stress testing started to be included regularly in the Financial Stability Report.||An official said the increased emphasis on stress testing was already under way, but the FSAP contributed to the momentum.|
|The FSAP helped in the effort to create a new law on reinsurance companies.|
|Ghana (6/2001)||The authorities indicated that the FSAP played a catalytic role in authorities’ discussions on financial sector issues. It also provided a platform for policy discussion with the international financial institutions on financial sector matters||Legislation was enacted granting independence and setting a narrower set of objectives to the central bank. But there has been little progress in the recapitalization of the central bank, the portfolio management practices of the social security administration, and the framework for contract and collateral enforcement.||The authorities indicated that the FSAP served as a wake-up call, and helped build up consensus, for a timely adoption of corrective actions to tackle impending problems in the banking system. It provided an important building block in the development of the authorities’ Financial Sector Strategic Plan.|
|The exposure of banking system to a single major borrower was reduced, while its dept was restructured.|
|India (6/2001)||The authorities indicated that the FSAP contributed to raising awareness of risks, informing authorities’ discussions, and opening a space for coordinating different agencies’ views and positions (Reserve Bank of India, Ministry of Finance, and state regulators).||There has been a gradual reform process involving a change in the paradigm of financial sector organization (from command and control to a more market structure) and regulation and supervision—a process that began before the FSAP and has continued thereafter.||According to senior officials, the FSAP served as a confidential second opinion to the authorities’ reform plans—buttressing the case for reforms. The mere existence of the initiative stimulated the authorities to pursue reform efforts further and show progress in the assessment. But none of the specific actions taken can be attributed to the FSAP; internal self-assessments also played a significant role.|
|Ireland (8/2000)||The FSAP contributed to the debate on whether to create a single supervisory agency (IFSRA) under the umbrella of the central bank or Greenfield.||The IFSRA was created under the central bank’s umbrella||Senior officials said the FSAP recommendations had a significant influence on the institutional location of the IFSRA and contributed importantly to the debate on insurance sector supervision.|
|Stress testing has been used more intensively for monitoring the household sector.|
|Supervision of insurance sector shifted to risk-based assessments from exclusive focus on consumer protection.|
|Japan (8/2003)||The authorities considered that the FSAP contributed to greater international understanding of Japan’s financial sector situation including a greater dissemination of knowledge about actual practices. A number. of senior officials also indicated that it contributed to inform the dialogue between key agencies, but others expressed skepticism about its contribution to the domestic policy dialogue.||There has been good progress with the stock problem of nonperforming loans in large systemic banks. There has also been more accurate reporting of data and better risk management frameworks, amid tighter oversight by the Financial Services Agency. The adoption of changes in the corporate governance framework has been slower.||There are different views among senior officials on whether the FSAP contributed to informing the policy dialogue. Some expressed the view that there was too little discussion during the FSAP process about recommendations and policy alternatives that could better conform to country conditions and that therefore the overall impact was limited. Some others had a more positive assessment. All agreed that the most significant contribution to the domestic debate had come from the opportunity to have discussions on the assessment with respected experts.|
|Jordan (4/2004)||The FSAP influenced the thinking of a small group of key policymakers that had limited impact on discussions among various agencies and regulators because circulation of the report was highly controlled. There were no public discussion as the FSSA was not published.||Progress areas include the regulation for prompt corrective actions, bank corporate governance, a new manual on licensing banks, the conduct of regular stress testing by the Central Bank of Jordan, assessment of country risk in evaluation of banks, and other segments (payment systems, insurance sector, and securities markets).||According to senior officials, the FSAP made a very positive contribution to the momentum for reforms, to setting reform priorities, and to solidifying a culture of financial stability.|
|Kazakhstan2 (12/2000)||The FSAP helped to build a road map of reforms for the capital market development.||There have been enhancements of banking supervisory powers. Consolidated prudential norms were introduced. A new regulatory framework for securities was implemented in 2003. Since 2005, International Accounting Standards are compulsory for financial institutions. A new insurance legislation was adopted in 2000 to better observe international practices. The payment system has been further reformed. A Financial System Authority was created.||Senior officials noted that the FSAP provided an impact to the various financial sector reforms as well as a broad road map of the necessary steps to develop the capital market.|
|Subsequent IMF-supported program contributed to the development of more specific action plans.|
|Korea (3/2003)||According to the authorities, the FSAP contributed to strengthening the reform agenda within the government, and recommendations influenced discussions among various agencies. The FSAP contributed to explaining reforms needed and to support the position of the government in the National Assembly.||There has been important progress in reform implementation (e.g., Bank of Korea Act, payment system). There are still pending issues including the unification of supervisory functions and further advancing with risk-based supervision.||Senior officials expressed the view that the FSAP, as an expert review, contributed to the credibility of the authorities’ reform program. It also helped with the prioritization of the reform agenda. The FSAP was an important contribution to the authorities’ thinking about the nature of risks and regulation (e.g., in insurance and banking).|
|According to officials, the FSAP also contributed to fending off pressures on directed credit to SMEs.|
|Mexico (8/2001)||Authorities said that the FSAP contributed to the government’s internal debate on development banks, crisis resolution, and improvement in the management of the de- posit insurance system.||Progress includes approval of the People’s Savings and Credit Law, approval of the Securities Market Law, approval of the new Investment Fund Law, creation of a Federal Housing Institution, overhaul of development banks, reforms of credit institutions law, strengthening of bank regulation and supervision by enhancing the mandate of the Comisión Nacional Banacaria y de Valores, reforms to the pension system law, amendments to bank’s capitalization requirements, approval of a law that regulates the operation of credit information institutions, approval of a law that regulates the operation of a new bankruptcy law, and approval of other reforms to promote development of institutional investors (with an environment of higher accountability and transparency).||According to senior officials, the FSAP served as a confidential advisor to the authorities, helping them to discuss their proposed reform agenda in a comprehensive manner with an expert team. They viewed the high-level discussions at the end of the FSAP mission, rather than the report itself, as having the most important impact. The timing of the FSAP mission (early on in the term of the new government, at the authorities’ request) was an important factor contributing to the positive effect of the FSAP.|
|New Zealand (4/2004)||The FSAP helped to generate momentum for financial reform, and served to get the attention of authorities of the government and Reserve Bank. On many occasions, the Governor and the Finance Minister have used the FSAP findings in public speeches.||New Zealand’s is a recent FSAP. Regular publication of the Financial Stability Report was started, including the use of stress testing.||Authorities indicated that FSAP recommendations were a useful contribution to set the agenda for reforms in the banking sector.|
|The areas of recent work by the Reserve Bank include (1) strengthening the ability to monitor risks by adopting a framework for independent reviewers of banks’ systems and controls; (2) reviewing the disclosure regime, which includes a framework for strengthening market discipline; and (3) enhancing the capacity to manage financial stresses by operationalizing the Reserve Bank’s lender of last resort role and determining options for responding to bank failures.||The Reserve Bank Governor adopted the FSAP recommendations as a road map for further improvements in the financial sector, and requested periodical updates of the state of implementation of the recommendations.|
|Philippines (9/2002)||It is unclear whether the FSAP had any significant impact on policy discussions.||There has been little progress with enacting reform legislation on prompt corrective action powers and protection of supervisors.||According to authorities, reform plans for the securities and exchange commission have drawn from the IOSCO assessment as input.|
|But stress testing is now part of supervision practices.|
|Romania (11/2003)||According to authorities, the FSAP encouraged discussion of financial stability issues within the government, but there was no significant public debate on the matter.||There is greater focus on stability issues and the overall financial system; and more attention to staff and building expertise. The central bank has been reorganized (new department and staff dedicated to financial stability issues). Stress testing is now a regular feature of supervision. The regulatory frame work for credit unions has been amended. Progress has been made in implementing the new banking law (on consolidated approach, corporate governance, bank internal controls, and definition of past due loans).||Authorities noted that the FSAP supported them in promoting attention on financial system stability.|
|Authorities indicated that FSAP contributed significantly to securities regulation.|
|Russia (5/2003)||A press conference was held after the FSAP. According to authorities, FSAP results were used in discussion within the central bank, and in discussions in parliament to pass legislation.||There was a change in regulations concerning large exposure, connected lending, consolidated supervision, and amendment to the method for computing bank capital. Bankruptcy regulations were also changed.||Senior officials indicated that, although most issues were not new to them, the FSAP contributed significantly to the momentum for financial sector reforms. FSAP findings contributed to central bank’s “Strategy of Banking System.” They also said that improvements in use of stress-testing approaches as part of supervision stemmed directly from the FSAP.|
|Supervision at the central bank started using regularly stress testing, and improving on methodology.|
|Reform of accounting practices toward International Accounting Standards has received new momentum.|
|Adoption of payments system action plan and implementation of Real Time Gross Settlement system.|
|Singapore (3/2004)||While issues raised in FSAP were not new to the authorities, the FSAP did contribute to increased discussion of them, especially within the Monetary Authority of Singapore.||According to senior officials, the adverse effects of the Central Provident Fund interest rates on the development of the life insurance market have started to be corrected. Authorities began to publish a Financial Stability Report, and stress testing is conducted regularly as a part of the exercises. Local capital requirements for banking sector were reviewed. A new risk based capital framework for the insurance industry was implemented.||According to senior officials, most FSAP recommendations have been implemented but they expressed that this should not be attributed the FSAP per se.|
|Slovenia (5/2001)||Senior officials indicated that the FSAP contributed to discussions at the central bank and among various agencies and institutions (e.g., on dynamic provisioning), and to inform the dialogue with parliament. There was little public discussion.||A Stability Unit (in charge of stress testing) was established in the central bank. There has been broad implementation of recommendations, including on connected lending, large exposures, market risk regulation, and governance criteria. Payments systems were upgraded.||According to authorities, the FSAP provided significant support to the authorities’ reform initiatives in parliament.|
|Other important contributing factors to the reform momentum include the EU accession and entering into ERM2 system.|
|South Africa (2/2000)||Authorities viewed the FSAP as part of their broader efforts to promote credibility by encouraging greater transparency of their policies and institutions.||Procedures for consolidated supervision were strengthened. Stress testing began to be be conducted on a regular basis and regular publication of a financial stability report began.||According to senior officials, the FSAP was useful in providing an integrated analysis and recommendations across the whole financial sector.|
|Reserve Bank officials expressed the view that most reforms would have happened in any case, but that the FSAP helped accelerated the pace in some areas.|
|Sri Lanka (9/2002)||According to authorities, the FSAP assisted discussions within the central bank and contributed to coordination with other agencies. It helped to increase transparency of the policy stance.||Progress has been made in several areas: prudential norms were strengthened, banking supervision was enhanced, and financial indicators were improved across the entire banking system (including for the two large state-owned commercial banks).||Senior officials indicated that the FSAP was helpful to identify and raise awareness of vulnera bilities, to prioritize measures, and to support the reform momentum.|
|Tunisia2 (6/2002)||Progress has been slow (including on the restructuring of state banks facing continued non-performing loan problems) amid preference for gradual reform efforts.|
The purpose of the exercise has been to identify policy and institutional changes that have taken place subsequent to the FSAP. Several important methodological caveats should be noted:
It is not possible to attribute any changes specifically to the FSAP given the complexity of the factors at work. Rather the aim is to examine (1) whether any changes have taken place (since if little has happened it is difficult to see how the FSAP could have had much impact); and (2) what qualitative evidence exists on how the FSAP might have contributed to the policy discussion and processes.
Even when specific policy and institutional changes are identified, it is generally not possible to say, with available evidence, whether these changes have effectively addressed the vulnerabilities and developmental needs highlighted in the FSAP. Such a conclusion would require an in-depth assessment akin to another FSAP (e.g., this review can say whether or not a new law was passed, but it cannot come to a conclusion about the effectiveness of the new law).
|Colombia||Ghana||Gabon||Kyrgyz Republic||Macedonia, FYR||Belarus||Jamaica|
|South Africa||Armenia||Luxembourg||Hong Kong SAR||ECCU||Italy||Uruguay|
|Iran, I.R. of||Yemen||Korea||Mauritius||Azerbaijan||Mauritania||Australia|
|Kazakhstan||Senegal||Costa Rica||Singapore||Austria||Albania||Bosnia and Herzegovina|
|Cameroon||Iceland||Sri Lanka||Oman||Netherlands||Trinidad and Tobago||Brunei Darussalam|
|Dominican Republic||United Kingdom||Tanzania||Saudi Arabia||Paraguay||Montenegro|
|United Arab Emirates||Slovak Republic||Romania||France||Bahrain||Turkey|
|Total: 12 (pilot countries)||Total: 20||Total: 20||Total: 15||Total: 16||Total: 14||Total: 14 (preliminary)|
|Hungary||Iran, I.R. of|
|Total: 0||Total: 2||Total: 1||Total: 1||Total: 4||Total: 6||Total: 7 (preliminary)|
|Country||Type of Contact with Country Authorities1||Published FSSA|
|Costa Rica||Country visit||Yes|
|Egypt||Interviews in Washington, D.C./IEG mission||No|
|Ghana||Country visit||Initial assessment: No;|
|Kazakhstan||Conference call||Initial assessment: No;|
|Mexico||Interviews in Washington, D.C.||Yes|
|New Zealand||Conference call||Yes|
|Slovenia||Country visit||Initial assessment: Yes;|
|South Africa||Country visit||No2|
|Sri Lanka||Conference call||No|
|Standards often covered in FSAPs|
|Banking supervision||Basel Committee’s “Core Principles for Effective Banking Supervision” (BCP).|
|Securities||International Organization of Securities Commissions’ (IOSCO) “Objectives and Principles for Securities Regulation.”|
|Insurance||International Association of Insurance Supervisors’ (IAIS) “Insurance Supervisory Principles.”|
|Payments systems||Committee on Payments and Settlements Systems’ (CPSS) “Core Principles for Systemically Important Payments System.”|
|Anti-money laundering and combating the financing of terrorism||Financial Action Task Force’s (FATF’s) “40+8 Recommendations.”|
|Standards sometimes covered|
|Monetary and financial policy Transparency1||IMF’s “Code of Good Practices on Transparency in Monetary and Financial Policies.”|
|Corporate governance2||Organization of Economic Cooperation and Development’s “Principles of Corporate Governance.”|
|Accounting2||International Accounting Standards Board’s “International Accounting Standards.”|
|Auditing2||International Federation of Accountants’ “International Standards on Auditing.”|
|Data transparency1||IMF’s Special Data Dissemination Standard and General Data Dissemination System.|
|Fiscal transparency1||IMF’s “Code of Good Practices on Fiscal Transparency.”|
Abiad, Abdul, and AshokaMody, 2003, “Financial Reform: What Shakes It? What Shapes It?”IMF Working Paper No. 03/70 (Washington: International Monetary Fund).
Bank for International Settlements, 2005, Stress Testing at Major Financial Institutions: Survey Results and Practice, Report by a Working Group Established by the Committee on the Global Financial System (Basel); available via the Internet atwww.bis.org/publ/cgfs24.htm.
Berg, Andrew, EduardoBorensztein, and CatherinePattillo, 2004, “Assessing Early Warning Systems: How Have They Worked in Practice?”IMF Working Paper No. 04/52 (Washington: International Monetary Fund).
Blaschke, Winfrid, Matthew T.Jones, GiovanniMajnoni, and SoledadMartinez Peria, 2001, “Stress Testing of Financial System: An Overview of Issues, Methodologies, and FSAP Experiences, 2001,”IMF Working Paper No. 01/88 (Washington: International Monetary Fund).
Borio, Claudio, 2003, “Towards a Macroprudential Framework for Financial Supervision and Regulation,”BIS Working Paper No. 128 (Basel: Bank for International Settlements).
Cady, John, 2004, “Does SDDS Subscription Reduce Borrowing Costs for Emerging Market Economies?”IMF Working Paper No. 04/58 (Washington: International Monetary Fund).
Christofides, Charis, ChristianMulder, and AndrewTiffin, 2003, “The Link Between Adherence to International Standards of Good Practice, Foreign Exchange Spreads, and Ratings,”IMF Working Paper No. 03/74 (Washington: International Monetary Fund).
Crockett, Andrew, 1996, “The Theory and Practice of Financial Stability,”De Economist,Vol. 144, No. 4, pp. 531–68.
Crockett, Andrew, 1997, “The Theory and Practice of Financial Stability,”GEI Newsletter,No. 6.
Das, Udaibir S., MarcQuintyn, and KinaChenard, 2004, “Does Regulatory Governance Matter for Financial System Stability? An Empirical Analysis,”IMF Working Paper No. 04/89 (Washington: International Monetary Fund).
Duignan, Paul, 2001, Introduction to Strategic Evaluation: Section on Evaluation Approaches, Purposes, Methods and Designs; available via the Internet atwww.strategicevaluation.info/se/documents/104f.html.
Duignan, Paul, and NilsBjorksten, 2005, “Strategy Design in Evaluating IMF Surveillance Activity,” IEO Background Paper No. 05/1 (Washington: Independent Evaluation Office of the IMF).
Eichengreen, Barry, 1999, “Strengtening the International Financial Architecture: Where Do We Stand?” paper prepared for the East-West Center Workshop on International Monetary and Financial Reform, Honolulu, October1–2.
Evans, Huw, 2000, “Plumbers and Architects: A Supervisory Perspective on International Financial Architecture,” FSA Occasional Paper Series No. 4 (London: Financial Services Authority).
Evans, Owen, Alfredo M.Leone, MahinderGill, and PaulHilbers, 2000, Macroprudential Indicators of Financial System Soundness, IMF Occasional Paper No. 192 (Washington: International Monetary Fund).
Fisher, Stanley, 2002, “Financial Crises and Reform of the International Financial System,”NBER Working Paper No. 9297 (Cambridge, Massachusetts: National Bureau of Economic Research).
Goodhart, Charles, 2004, “Some New Directions for Financial Stability?” Per Jacobsson Lecture (Washington: Per Jacobsson Foundation; Basel: Bank for International Settlements).
Houben, Aerdt, JanKakes, and GarrySchinasi, 2004, “Toward a Framework for Safeguarding Financial Stability,”IMF Working Paper No. 04/101 (Washington: International Monetary Fund).
International Monetary Fund, 1995, “IMF Executive Board Reviews the Fund’s Surveillance,” PIN/04/95 (Washington).
International Monetary Fund, 1999, “IMF–World Bank Financial Sector Assessment Program (FSAP),” SM/99/116, May (Washington).
International Monetary Fund, 2000a, “Financial Sector Assessment Program—A Review—Lessons from the Pilot and Issues Going Forward,” SM/00/263, November (Washington).
International Monetary Fund, 2000b, “Summing Up by the Acting Chairman: Financial Sector Assessment Program—A Review—Lessons from the Pilot and Issues Going Forward,” Buff/00/190, December (Washington).
International Monetary Fund, 2001, “Progress Report on the Bank-Fund Financial Sector Liaison Committee,” SM/01/295, September (Washington).
International Monetary Fund, 2003a, “The Acting Chair’s Summing Up: Financial Sector Assessment Program—Reviews, Lessons, and Issues Going Forward,” Buff/03/42, March (Washington: International Monetary Fund).
International Monetary Fund, 2003b, “Financial Sector Assessment Program—Reviews, Lessons, and Issues Going Forward,” SM/03/77, February (Washington: International Monetary Fund).
International Monetary Fund, 2003c, “Financial Soundness Indicators,” SM/03/175, May (Washington: International Monetary Fund).
International Monetary Fund, 2003d, “Financial Soundness Indicators—Comments on Draft Compilation Guide,” SM/03/175, Supplement 1, May (Washington: International Monetary Fund).
International Monetary Fund, 2003e, “Financial Soundness Indicators—Background Paper,” SM/03/176, May (Washington: International Monetary Fund).
International Monetary Fund, 2003f, “Financial Soundness Indicators—Background Paper,” SM/03/176, Correction 1, May (Washington: International Monetary Fund).
International Monetary Fund, 2003g, “IMF Executive Board Discusses Financial Soundness Indicators,” Public Information Notice No. 03/71 (Washington: International Monetary Fund).
International Monetary Fund, 2004a, “Biennial Review of the Implementation of the IMF’s Surveillance and of the 1977 Surveillance Decision—Modalities of Surveillance,” SM/04/212, Supplement 1, July (Washington).
International Monetary Fund, 2004b, “Financial Sector Regulation: Issues and Gaps,” SM/04/268, August (Washington).
International Monetary Fund, 2005a, “The Acting Chair’s Summing Up: The Standards and Codes Initiative—Is It Effective? And How Can It Be Implemented?” Buff/05/125, July (Washington).
International Monetary Fund, 2005b, Financial Sector Assessment Program—Review, Lessons, and Issues Going Forward, SM/05/67, February (Washington).
International Monetary Fund, 2005c, “The Standards and Codes Initiative—Is It Effective? And How Can It Be Improved?” SM/05/252, prepared by the staffs of the International Monetary Fund and the World Bank, July (Washington: International Monetary Fund).
International Monetary Fund, and World Bank, 2003, “Analytical Tools of the FSAP,” background paper prepared for March 14, 2003 IMF Executive Board meeting (Washington: International Monetary Fund); available via the Internet at www.imf.org/external/np/fsap/2003/022403a.htm.
Jones, Matthew T., PaulHilbers, and GrahamSlack, 2004, “Stress Testing Financial Systems: What to Do When the Governor Calls,”IMF Working Paper No. 04/127 (Washington: International Monetary Fund).
Schinasi, Garry J., 2004, “Defining Financial Stability,”IMF Working Paper No. 04/187 (Washington: International Monetary Fund).
Schneider, Benu, ed., 2003, The Road to International Financial Stability: Are Key Financial Standards the Answer? (Basingstoke, Hampshire, United Kingdom; New York: Palgrave Macmillan).
Sorge, Marco, 2004, “Stress-Testing Financial Systems: an Overview of Current Methodologies,”BIS Working Paper No. 165 (Basel: Bank for International Settlements); available via the Internet at www.bis.org/publ/work165.htm.
Sundararajan, V., CharlesEnoch, Armida SanJosé, PaulHilbers, RussellKrueger, MarinaMoretti, and GrahamSlack, 2002, Financial Soundness Indicators: Analytical Aspects and Country Practices, IMF Occasional Paper No. 212 (Washington: International Monetary Fund).