Chapter 2 Effectiveness of Priority Setting Under the FSAP

International Monetary Fund. Independent Evaluation Office
Published Date:
May 2006
  • ShareShare
Show Summary Details

Choices on priorities under the FSAP—which countries to assess and what issues to examine within each country—are critical to the program’s overall effectiveness. Three aspects of the FSAP make priority setting especially challenging. First, the voluntary nature of the exercise means that some countries that would otherwise be ranked as high priority may choose not to participate. This raises the important question of what incentives should be used to en-courage participation. Second, the FSAP exercise is resource-intensive. As a result, resource constraints have required some scaling back of the number of assessments—from an initial goal of completing 24 cases a year to a rate of 17–19 a year following the 2003 review. The latter rate would imply that a comprehensive assessment of the entire membership would take a little over a decade. The 2003 review also called for more selectivity by reducing the depth of analysis of certain issues and the number of standards to be assessed in detail for each country, while remaining comprehensive in coverage.1 Third, the FSAP’s multiple objectives as well as its joint IMF–World Bank nature, with inevitable differences in institutional priorities, implies balancing a complex set of priorities.

In this context, the evaluation asked the following questions: (1) Are the priority-setting criteria set by the two Executive Boards the relevant ones, in the sense of being clearly linked to the overall objectives of the FSAP initiative; and (2) how effectively have they been implemented in practice? Our overall assessment is as follows (see Annex 4 for further details):

The criteria established by the two Boards to select priority countries are the relevant ones but should be more sharply focused. There is a clear emphasis on maximizing the program’s contribution to the strengthening of national and international financial stability by giving priority to systemically important countries (SICs) and those potentially vulnerable to various pressures on the financial system.2 The Boards also indicated that such countries should be reassessed more frequently. However, the absence of a specific list of “systemically important countries,”3 and the use of additional priority criteria (e.g., on the need for geographical balance) have blurred this emphasis. In practice, some of the potential trade-offs between these criteria that would otherwise need to be faced have not been pressing because some countries have been reluctant to participate.

Although country selection has largely followed the guidelines set by the two Boards, a significant minority of countries that would appear to be “systemically important” and/or have vulnerable financial systems have not been assessed—because the authorities have not volunteered. We used a number of possible measures of systemic importance and potential vulnerability of financial systems.4 The results are quite similar whichever measures are used:

  • Some 20–25 percent of countries that are of some systemic importance had still not undertaken initial FSAPs after six years of operation of the initiative (FY2000–05). The list includes the United States, China, most of the emerging market countries of Southeast Asia, and the Républica Bolivariana de Venezuela.5 The gaps are primarily because the authorities of the countries concerned have been reluctant to participate in the exercise. IMF staff emphasized during interviews that the FSAP work plan has been managed sufficiently flexibly to accommodate requests for participation from systemic countries at fairly short notice, if necessary by postponing other cases. Our examination of priority setting in several such cases (e.g., France, the United Kingdom, and, more recently, an FSAP Update now scheduled for Mexico) supports this view.

  • About one-third of countries with some significant indicator of potential financial sector vulnerability have not yet undertaken an FSAP. For example, of the 17 emerging market economies tracked by S&P that received a low financial system rating in 2001, 5 had not undertaken an FSAP by June 2005; all of these countries also met several criteria of systemic importance (Table 2.1). Results using internal (MFD) judgments on potential vulnerabilities gave similar conclusions.

Table 2.1.S&P Global Financial System Ratings and FSAP Participation(Number of countries participating through FY2005)
Number of

Countries Rated
Financial systems1
10–20 percent11000
15–30 percent73112
25–40 percent83311
35–70 percent173950
Sources: S&P Global Financial System Stress, October 2001; and IEO staff.

A significant proportion of FSAPs for countries of systemic importance and/or with potential financial sector vulnerabilities are becoming dated, even though the Boards have called for these countries to be given priority in Updates.6Consequently, the actual participation of countries receiving FSAP Updates is not in line with the broader objectives of the initiative. The reluctance of many countries to request FSAP Updates, at least until all of the major economies have participated in the exercise, has been an important factor in this development. For example, of the 11 countries with post-pilot Updates as of June 2005, only 3 (Colombia, Peru, and Hungary) are in at least two of the four groupings of potential SICs discussed earlier. Of the 9 Updates currently planned, 4 (Mexico, Philippines, Poland, and Russia) would meet this criteria. As a result, the proportion of SICs and of countries with some indicator of potential financial sector vulnerability that have never had an FSAP or have one that is at least four years old has been growing (Figure 2.1).7

Figure 2.1.Number of Systemically Important Countries (SICs) with No or a Dated FSAP1,2

1 Definition of SICs is the same as that for Table 2.1 (i.e., meets at least two of the four criteria for systemic importance). Total number of SICs by this definition is 30.

2 A dated FSAP is an initial assessment or an update that is at least four years old.

The evaluation found no major evidence that the changes introduced in 2003 to be more selective about the number of issues and financial sector standards assessed in detail have had a negative impact on the quality of the overall vulnerability assessment. However, there are clearly limits to how far selectivity can be taken without losing the broad overview of intersectoral linkages that is one of the key advantages of the FSAP approach. The 2003 review introduced several changes to sharpen the scope of FSAP assessments, including (1) limiting the number of standards for which detailed assessments are undertaken; and (2) tailoring the depth of coverage of topics and analysis to country circumstances while continuing to be comprehensive in areas covered. There is no simple test to judge how well the tailoring of depth of coverage has worked, since it depends critically on individual country circumstances. Within the 25-country sample, there was inadequate coverage of cross-border issues in many cases (see below), but this is a problem that predates the 2003 efforts at streamlining. Beyond this, the only case in this sample where selectivity may have gone too far was the New Zealand FSAP, where senior officials indicated that it would have been useful for the FSAP to take a broader approach by examining payments system and insurance sector standards as well as those of the banking sector. To double check this conclusion, the evaluation team examined all initial FSAPs undertaken since the 2003 review.8 The results of this assessment suggest that there are some opportunity costs to the streamlining, especially in countries with complex and diversified financial systems (e.g., there was less discussion of macroeconomic linkages with the securities sector than one might have expected in some advanced economy FSSAs), but it is not possible to say that the overall vulnerability assessment has been thereby weakened.

The 2003 streamlining of aspects of the FSAP has generated moderate cost savings for the IMF (and overall), mainly in FSAPs for advanced economies. However, there has been no decline in the average costs of FSAPs in low-income countries, where the IMF continued to contribute the largest share of resources through FY2004. The evaluation undertook a detailed examination of the direct costs of each individual FSAP initial assessment. Different IMF and World Bank budget procedures complicate comparisons, so that any conclusions can only be approximate; however, the exercise suggests the following (see Annex 4 for further details):9

  • Average direct costs (IMF and World Bank staff, including experts, plus travel) for initial FSAPs have fallen by about 6 percent between pre- and post-2003 cases, with the savings mainly driven by lower costs in advanced economies. Reflecting the latter factor, average costs for the IMF alone have fallen by about 10 percent.

  • Since 2003, there has been surprisingly little difference between the average costs of FSAPs for advanced, emerging market, and low-income countries. This reflects the bigger impact of the 2003 streamlining measures on advanced economy FSAPs and the fact that some of the most complex financial systems were assessed prior to 2003.

  • Average direct costs of FSAPs undertaken for low-income countries have not declined and are as expensive as those for emerging economies. About 60 percent of these costs are still borne by the IMF.

There is inadequate discussion of the expected scope of the FSAP, including with the authorities, at the terms of reference (TOR) stage. Our reviews of the TOR for the 25-country sample found only a few cases where there was a serious initial discussion of priority setting and strategic trade-offs—although all of these cases were relatively recent, which suggests some improvement over time.10 Many country authorities said greater consultation at the TOR stage would have made the process more effective; some said they were surprised at a late stage by aspects of the FSAP’s scope for their countries. Our interviews with IMF staff suggest that the scope of FSAP Updates is a source of debate between Bank and IMF staff, with Bank staff frequently pressing (successfully) for a larger scope—to address medium-term development issues not taken up in the original FSAPs—whereas the IMF would have preferred smaller Updates focused on a follow-up on core issues from the earlier FSAP.

The assessment of the AML/CFT standard has little integration with other FSAP activities. This view was broadly shared by country officials and FSAP team members. In practice, there was little synergy with other FSAP activities because of the special legal and accounting aspects involved, which required a different type of expertise.11

The 2003 review called for a typical FSAP to limit the number of standards assessed in detail to three, excluding AML/CFT, compared with a previous average of about five standards per FSAP.

The criteria endorsed by the Board following the 12 pilot cases included a country’s (1) systemic importance; (2) external sector weakness and financial vulnerability; (3) features of its exchange rate and monetary policy regime that make its financial system more vulnerable—such as inconsistencies with other macroeconomic policies; (4) likelihood of upcoming major reform programs; and (5) geographical balance among countries. One point to note about these criteria is that they provide little guidance on how (or what type of) financial sector development issues would enter into the priority-setting process. The criteria were refined following the 2003 and 2005 reviews, to take into account the need to accommodate countries’ requests for Updates, but the emphasis remained the same. The reviews also clarified that countries undergoing financial crises were not expected to participate in the FSAP—other tools were considered more appropriate to help them deal with their problems.

The review conducted after the initial round of pilot cases defined the notion of SICs as (1) countries whose capital markets intermediate the bulk of global financial transactions; and (2) emerging market economies whose financial systems have the potential to cause, or be subject to, undue volatility in cross-border flows and financial system contagion.

For systemic importance, possible groups could include the G-20; G-7 plus those emerging market countries covered by the EMBI+ index; G-7 plus emerging market countries monitored regularly by the IMF’s International Capital Markets Department (ICM); or countries monitored by the BIS in its surveillance work. For potential financial sector vulnerability, we looked at two possisible indicators: the S&P global financial system ratings and internal (MFD) judgments on potential financial sector vulnerabilities.

This excludes two FSAPs that were suspended due to crisis (Argentina and Uruguay). Several countries of systemic importance have indicated their intention to participate in the FSAP in the near future (e.g., Turkey, Poland (for an Update); the FSAP for Spain began in mid-2005).

The 2005 FSAP review called for the interval between assessments to be shorter for systematically important countries and for countries where there have been significant macroeconomic shocks to the financial system or major reforms.

By comparison, 90 percent of IMF area department mission chiefs surveyed as part of the recent internal review of the standard and codes initiative thought that the shelf-life of standards assessments was four years or less. See Table 14 of “The Standards and Codes Initiative—Is It Effective? And How Can It Be Improved?” (SM/05/252, July 1, 2005).

The review was of the FSSA only, and covered the following countries: Albania, Austria, Ecuador, France, Kenya, Kuwait, the Netherlands, Norway, Pakistan, and Saudi Arabia.

These figures exclude overhead for both institutions as well as the cost of AML/CFT assessments for the IMF. The average total direct cost of a post-2003 FSAP was $668,000, of which $438,000 was accounted for by the IMF. Applying the same IMF overhead factors used in the internal 2005 review “Financial Sector Assessment Program—Review, Lessons, and Issues Going Forward” (SM/05/67, Supplement 2), the total estimated IMF cost per assessment was $721,000 for post-2003 cases.

Recent good practice examples include Chile (initial FSAP) and Ghana (Update).

This says nothing about the effectiveness or value added of the AML/CFT assessments, which was not part of the evaluation’s terms of reference.

    Other Resources Citing This Publication