Chapter

chapter 2 Were Interactions with Country Authorities Effective?

Author(s):
International Monetary Fund. Independent Evaluation Office
Published Date:
August 2010
Share
  • ShareShare
Show Summary Details

5. In assessing the effectiveness of interactions between the IMF and the authorities of member countries, the evaluation focused on the perceptions of country officials and individual Fund staff members working on those countries.2 Evidence on these perceptions was gathered through surveys of the whole membership, and interviews focused on 49 countries that explored a number of aspects of effectiveness, which were then considered in tandem with the evaluation’s documentary evidence. This chapter explores what the evaluation’s evidence has to say about the effectiveness of this interface, looking at it from a substantive perspective. The strategic, stylistic, and relationship management issues associated with the management of interactions are taken up in Chapter 4.

6. The evaluation took the view that general perceptions of overall effectiveness, to be meaningful, needed to be grounded in a common understanding on the part of the authorities and the IMF staff of what interactions were supposed to achieve, and in evidence that agreed roles were performed effectively and were of high quality. With this in mind, the evaluation framework developed measures for different aspects of perceptions of interactions. It also provides a systematic basis for considering the evaluation’s other evidence (from interviews, documents, and case studies) in forming its overall judgments.

7. On this basis, overall, the evidence is mixed. While one may be tempted to take solace from relatively high perceptions of overall effectiveness in some country groupings, such reaction needs to be tempered by clear evidence of lack of agreement between the authorities and staff on the scope of interactions in some cases, and of widely varying effectiveness in particular roles. Interactions were effective in a program and technical assistance context, and, in general, in contributing to a good exchange of views and in providing objective assessments. However, in other areas, including in the international dimensions of its surveillance and other work, where one would expect the IMF to excel, effectiveness and quality were not rated highly.

8. In turn, these findings translate into the evaluation’s broader implications about effectiveness across country groups—that the Fund has been most effective with the PRGF-eligible countries and the smaller emerging economies. They were the least effective with the advanced and large emerging economies, together accounting for about 90 percent of global GDP, where there also have been continuing differences between the authorities and staff on the Fund’s role and relevance in interacting with them.

9. Against this background, the chapter starts with cross-cutting issues—setting out first the big picture, drawn primarily from the survey evidence, and several key themes that the evidence highlights. It complements that discussion with a brief exploration of the particular issues arising by country group, drawing from the evaluation’s case studies of 10 advanced economies, 23 emerging economies, and 16 PRGF-eligible countries, which are discussed in greater detail in the three companion papers on the country groups.

A. Cross-Cutting Issues

10. Measurement framework and key themes. This section sets out the measurement framework developed by the evaluation for structured discussion of different aspects of effectiveness. It then explores four Fund activities (basic country assessment of surveillance and other Fund activities, international dimensions of Fund country analysis, policy dialogue, and country programs and technical assistance) using the framework’s building blocks.

Indicators of interactions

11. Box 2 presents composite indicators of interactions, derived from the authorities’ perceptions as recorded in the evaluation survey. They cover the authorities’ ratings for (i) role relevance, (ii) role effectiveness, (iii) quality, (iv) strategic alignment with staff views, and (v) overall perceived effectiveness—all as defined in the box. Figure 1, shown later in the chapter, presents the same indicators from the IMF staff’s perspective.

Figure 1.Composite Indicators of Interactions (Staff Results)

1Inverted scale. The average absolute percentage point difference across the ten purposes between how much the authorities wanted the IMF to fulfill each purpose, and how much staff aimed to do so (“a fair amount” or “very much”).

  • Taken together, the data show that PRGF-eligible countries were satisfied in important respects with the substance of their interactions with the Fund—and to a lesser extent so were the other emerging economies. This does not mean there were no problems in interactions with these countries or that there was not much to improve, but it does show the importance of relevant products and services for effectiveness. These results are reinforced by the interview and documentary evidence, and also by the staff survey.

  • But for the advanced and large emerging economies, the indicators are less favorable to the Fund. Role effectiveness was rated the lowest by the authorities of the large emerging economies, while role relevance and quality were rated the lowest by the authorities of the large advanced economies. Low marks also came from staff working on large advanced economies with respect to overall perceived effectiveness and other dimensions of effectiveness.

12. Key features of Box 2’s indicators are discussed below.

  • Role relevance. For the advanced and the large emerging economies, the low level of this indicator in part reflects these authorities’ limited interest in programs and technical assistance. But that is not the full story. The underlying data also point to limited interest in the Fund’s policy advice (including on operational aspects) and related outreach, and very limited interest by the large advanced economies in a contribution by the Fund to international policy coordination.3 In both cases, the results raise questions as to why these ratings are so low—and in particular whether it was the quality of the staff analysis that had caused the authorities’ interest to wane or whether they simply did not want IMF staff critiquing their policies in the first place—issues to which the report returns in the discussion of the advanced economies below (paragraphs 20–23).

  • Role effectiveness. This measure is higher for the large advanced economies than for the large emerging economies. As a composite of underlying scores, the difference reflects, inter alia, the much higher score the large advanced economies and the much lower score the large emerging economies give to the effectiveness of two roles—assessing and exchanging views on countries’ policies and prospects—to which they both assign a high priority.4 (These points are discussed further in paragraph 15 below and illustrated in Figure 2. They are discussed in more detail in the companion paper on the advanced economies.)

  • Quality. Noteworthy here is the relatively low score provided by the large advanced economies, as highlighted in Box 2. It reflects the frequent assignment of a rating of “average” to most aspects. Exceptions are the Fund’s work in responding quickly to the authorities’ requests for analytical work and actively engaging in a constructive dialogue for which most large advanced country respondents said the Fund did a good or excellent job. The quality scores provided by the other groups, especially the PRGF-eligible countries, are higher. The two activities with the worst scores overall were: (1) bringing quickly to the authorities’ attention the implications of changing external conditions and (ii) presenting alternative scenarios and addressing “what if?” or “what’s missing?” questions.

  • Strategic alignment (between authorities and staff). Most noteworthy is the much higher level of strategic alignment among the PRGF-eligible countries, and to a lesser extent the other emerging economies, than in the three other country groups. To a considerable extent, this difference reflects the much greater meeting of the minds between the authorities of PRGF-eligible countries and other emerging economies and staff on issues such as the contribution of the Fund to the development of policy frameworks, and to the development and maintenance of policy consensus outside government. There is no such meeting of the minds on these issues between the authorities of the advanced and large emerging economies and the staff—topics to which the paper returns in Chapter 4.

  • Overall perceived effectiveness. Compared with some of the other ratings, this indicator is relatively high for the large advanced economies and relatively low for the large emerging economies. It reflects the authorities’ ratings for the direct question on the overall effectiveness of interactions with the IMF over the last two years (2007–08), implicitly leaving respondents to apply the weights they deemed appropriate to particular roles. For the large advanced economies, it may be that the relatively high rating implicitly reflects the high rating given to the two roles that these authorities value highly (discussed in paragraph 15).5 This said, it cannot be ruled out that the authorities are giving the Fund at least some credit for other services it performs, for example vis-à-vis other countries. Conversely, in the large emerging economies, it also cannot be ruled out that consideration of other contextual factors—as discussed further in this chapter—is affecting this rating. Whichever the explanations, it is impossible to ignore the staff’s self-assessment of its effectiveness in these countries—measured both by role effectiveness and by overall perceived effectiveness, as illustrated in Figure 1 above.

Figure 2.Basic Interactions: Authorities’ Views of Role Relevance and Role Effectiveness

(Percent of respondents)

1Shows the share of authorities who responded “a fair amount” or “very much.”

2Shows the share of authorities who responded “effective” or “very effective.”

Box 2.Composite Indicators of Interactions

Role Relevance, Role Effectiveness, Quality, Strategic Alignment, and Overall Perceived Effectiveness of Interactions: Authorities’ Views of the Last Two Years, 2007–08

The above indicators measure five dimensions of interactions—role relevance, role effectiveness, quality, strategic alignment, and perceived effectiveness.

  • Role relevance. This is measured by the average of authorities’ interest (“a fair amount” or “very much”) in ten possible purposes of IMF interactions.1

  • Role effectiveness. This is measured by the authorities’ ratings for the Fund’s perceived effectiveness (“effective” or “very effective”) across the ten purposes, with the rating for each purpose weighted by its perceived relevance (the share of authorities who wanted each purpose “a fair amount” or “very much”).

  • Quality. This is measured by the average of the authorities’ perceptions of the IMF’s performance (“good” or “excellent”) in ten qualitative aspects associated with effective interactions.2

  • Strategic alignment. This refers to the degree of agreement between the authorities and the staff on the ten purposes of IMF interactions, whose relevance was rated by survey respondents. The indicator is shown with an inverted scale: the average absolute percentage point difference across the ten purposes between how much the authorities wanted the IMF to fulfill each purpose and how much staff aimed to do so (“a fair amount” or “very much”). (0 = perfect alignment; 100 = perfect misalignment.)

  • Overall perceived effectiveness. This indicator shows the share of authorities answering “effective” or “very effective” to a direct question on perceived effectiveness of IMF interactions during the last two years (2007–08).

Source: IEO calculations based on surveys.1These are: (i) providing a clear and objective assessment of their country’s policies and prospects; (ii) contributing to a good exchange of views; (iii) contributing to the development of policy frameworks; (iv) advising on operational aspects in implementing policies; (v) helping to build and maintain policy consensus within the government; (vi) helping to build and maintain policy consensus through contacts outside government; (vii) presenting analysis and assessments of the country’s economy to other countries, donors, or financial markets; (viii) contributing to international policy coordination, including spillover analysis; (ix) providing financial and/or monitoring support for the country; and (x) assisting in building capacity in their country.2These are: (i) listening carefully to the authorities’ perspectives; (ii) responding in general to the authorities’ changing needs and priorities; (iii) responding quickly to requests for analytical work; (iv) actively engaging in a constructive dialogue; (v) focusing on topics of interest to the country; (vi) providing advice and analysis suited to country circumstances; (vii) providing analysis based on the experiences of other countries; (viii) bringing quickly to the authorities’ attention the implications of changing external conditions; (ix) presenting alternative scenarios and addressing “what if” or “what’s missing” questions; and (x) taking a long-term strategic approach to the relationship.

13. The evaluation also constructed composite indicators based on the IMF staff survey, as pictured in Figure 1. These show the usual pattern of more positive ratings by the staff than the authorities. The sole departure, which is quite marked, is for the effectiveness ratings provided by the staff working on large advanced economies. The two effectiveness ratings for 2007–08 shown in Figure 1 are in the 44–45 percent range. Also, they show the reverse pattern displayed by the authorities of the large advanced economies, for whom relevance and quality were rated low but effectiveness, relatively high. In contrast, for the staff, relevance and quality are high and perceived effectiveness is low, suggesting that staff feel they are doing the right thing and doing it right, but somehow are not getting through. These issues are explored in more detail in the companion paper on the advanced economies.

Key themes

14. Four themes capture the essence of the evaluation survey’s findings with respect to the similarities and differences in views about the Fund’s effectiveness across the country groups—findings echoed in the evaluation’s interview and documentary evidence. They relate to: (i) the basic purposes of interactions, relevant to all members through surveillance and other processes; (ii) international dimensions of the Fund’s work, including importantly on surveillance; (iii) dialogue on country policy frameworks; and (iv) programs and technical assistance. Each is discussed briefly in turn in the following paragraphs, and in subsequent sections of the report.

15. Two basic purposes of interactions queried in the evaluation survey received very high scores for relevance from most authorities. These were: (i) providing a clear and objective assessment of their country’s policies and prospects; and (ii) contributing to a good exchange of views—two roles often associated with Article IV consultations, but also relevant to program discussions. As shown in Figure 2, the authorities of all country groups—except the large emerging economies—gave those two purposes very high effectiveness ratings, as did staff working on all country groups. In interviews, the authorities of the large emerging economies generally saw the surveillance process as “going through the motions,” providing little value added, and having too little depth—issues explored in more depth below and in the companion paper on the emerging economies.

16. Surveyed authorities were decidedly less enthusiastic about Fund performance on several international dimensions of the Fund’s work. As Figure 3 shows, the authorities did not rate highly the effectiveness of interactions in contributing to international policy coordination including analysis of spillovers. They rated somewhat higher the quality of the job done in alerting authorities about imminent external risks, and providing cross-country analysis—although still far below Figure 2’s ratings for the basic assessment and exchange of views. For the most part, only minorities of country respondents thought the Fund did a good or excellent job. One exception is PRGF-eligible country authorities’ views on the Fund’s bringing quickly to their attention the implications of changing external conditions. Based on the evaluation interviews, this reflects the Fund’s quick response to the food and fuel crises of 2007–08, which the authorities found helpful in the dialogue, and, later in the evaluation period, actions taken in September 2008 to make the Exogenous Shocks Facility easier to access. In terms of cross-country analysis, this was one area where the large emerging economies gave the highest scores. Elsewhere it was a more serious concern, for example with an official of one advanced economy capturing a more widespread sentiment in describing the staff use of relevant cross-country experience as “erratic” and seemingly accidental, depending on who happened to be on the mission and what that person happened to know. Preferably, the analysis would have been the result of a systematic effort to prepare for the mission by looking into the challenges the country was facing, drawing on knowledgeable staff, and the broader stock of knowledge accumulated within the institution, for material that might be helpful to the discussions.

Figure 3.International Dimensions of IMF Surveillance: Authorities’ Views of Quality/Effectiveness

(Percent of authorities who responded favorably)

1Shows the share of authorities who responded “effective” or “very effective.”

2Shows the share of authorities who responded “good” or “excellent.”

17. The survey results suggest limited interest by the authorities of large advanced and emerging economies in the Fund’s contribution to the development of policy frameworks, which they rated even lower for effectiveness. As shown in Figure 4, the authorities of PRGF-eligible countries and the other emerging economies, and to a lesser extent the other advanced economies wanted such inputs from the Fund, while large majorities of staff working on all country groups said they aimed to make a contribution in this area. Figure 4 also shows the authorities’ and staff’s ratings for effectiveness, on which there is convergence at low levels on the large advanced economies and at high levels on the PRGF-eligible countries. For the other advanced and emerging economies, there are sizable disconnects between the authorities’ and staff views on the effectiveness of the staff’s inputs in this area.

Figure 4.IMF Contribution to Development of Policy Frameworks: Authorities’ Views of Role Relevance and Role Effectiveness

(Percent of respondents)

1Shows the share of respondents who answered “a fair amount” or “very much.”

2Shows the share of respondents who answered “effective” or “very effective.”

18. The fourth key theme is the authorities’ interest in programs and technical assistance, which, like policy advice, is largely confined to the PRGF-eligible countries and the other emerging economies, although there was some interest also by large emerging economies, including more recently. As Figure 5 shows, the survey feedback generally has the authorities’ views on effectiveness very much in line with their interest in them.6 These issues are briefly discussed in Section B below, and explored in more depth in the companion country papers on the emerging economies and the PRGF-eligible countries.

Figure 5.Programs and Technical Assistance: Role Relevance and Role Effectiveness

(Percent of respondents)

1Shows the share of authorities who responded “a fair amount” or “very much.”

2Shows the share of authorities who responded “effective” or “very effective.” The left panel excludes those respondents who answered that they did not have an IMF financial or monitoring support program, and the right panel excludes those respondents who answered that they did not receive IMF technical assistance or training.

B. Interactions in Different Country Groups

19. The following three sections deepen the discussion of interactions on the cross-cutting issues by focusing on the underlying country issues and concerns, especially those that arose during the evaluation interviews. They also provide additional evidence drawn from the companion country papers and other sources.

Advanced economies

20. The weight of the evaluation evidence from all sources suggests that IMF interactions with the advanced economies were effective only over a very limited range of roles. The large advanced economies (the G-7) were generally uninterested in the Fund’s work on their own economies, beyond a quiet exchange of views. The authorities in most of these economies did not want contributions to the development of policy frameworks or staff presentations to other countries and markets of their country’s economic analysis and assessment. Interactions were somewhat more effective with the other advanced economies than they were with large advanced economies, although not enormously so, reflecting the greater range of Fund activities of interest to this larger group of countries and their greater “demand” for external inputs and sounding boards (especially in perilous times, as during the recent crisis). Neither the large nor the other advanced economies rated highly Fund inputs on key services such as the provision of cross-country lessons of experience, customized to their country’s conditions, the delivery of timely “heads-up” about upcoming threats, or the linkages between macro and financial sector analysis, although the Financial Sector Assessment Program (FSAP) process generally received high marks from authorities in the G-7 and the other advanced economies.

21. There were important differences of views between the authorities of the large advanced economies and staff working on them on the relevance of several possible Fund roles—including contributions to the development of policy frameworks and providing information on the economy to other countries and financial markets, roles in which the authorities and staff agreed that interactions were not effective.7 More broadly—and as illustrated in Figure 6—underpinning Box 2’s strategic alignment indicator for the large advanced economies lie disagreements on four of the ten queried roles—the two just mentioned plus two others, namely: (i) contributing to international policy coordination, including through analysis of spillover effects (discussed below) and (ii) helping build policy consensus outside government (discussed in Chapter 3). On the priority (or not) of the other six roles, the authorities and staff broadly agreed on two as priorities (objective assessment of policies and prospects, and good exchange of views) and on four as nonpriorities (capacity building; consensus inside government; operational aspects of implementing policies; and program support/monitoring).

Figure 6.Large Advanced Economies: Strategic Alignment Between Authorities and Staffs

(Percent of staff who thought IMF “aimed” to perform each role minus percent of authorities who “wanted” the IMF to do so)

22. Figure 7 illustrates the authorities’ and staff survey responses for the large and other advanced economies on the relevance and effectiveness of the Fund’s contributing to international policy coordination, including the analysis of spillovers. It shows the mismatch in perceptions between the G-7 authorities and staff on the role of the Fund, but their much greater agreement on its limited effectiveness. Indeed, only about half of G-7 survey respondents wanted such work done, while almost all staff working on G-7 countries said they had aimed to do it, notwithstanding what they perceived as a lack of interest on the other side. As one staff member working on a G-7 country confided, the only kind of spillover that the authorities of the country he worked on cared about were inward spillovers from the rest of the world to it; outward spillovers from it to the rest of the world were of no interest. Meanwhile for the other advanced economies the story is very different, as the right-hand panel of Figure 7 shows: there, a strong majority of authorities wanted the Fund to contribute to international policy coordination. But in the event, many fewer rated Fund performance in this role as effective.

Figure 7.International Policy Coordination, Including Spillover Analysis

(Percent of respondents)

Authorities’ and staff views on the IMF’s contribution to international policy coordination, including through analysis of the spillover effects from one country to another

23. More generally, majorities of all country groups, except for the large advanced economies, both wanted a greater Fund presence in international policy coordination and spillover analysis and—including the large advanced economies—gave the Fund low marks for effectiveness in this area (Figure 8). With the then unfolding global crisis in people’s minds, interviewed officials from advanced economies generally took the view that the IMF had been “no worse than others” in predicting (or not) the events of 2008. But they also said that the institution was not playing to what should be its comparative strengths in being able to analyze crosscutting global themes and identifying risks. Meanwhile, several interviewees complained about the lack of a U.S. FSAP, especially in current circumstances. In sum, several large advanced economies had not bought into key aspects of surveillance, whether for reasons associated with the perceived quality of the Fund’s work or for other reasons; at the same time, their preference to limit the Fund’s work on their own economies also contributed to and reinforced their and others’ judgments that the Fund did not add much value. Against this background, the recent decision by the membership to give the Fund a more explicit role in policy coordination provides an important opportunity to reset Fund activities in these areas, but also an important delivery challenge.

Figure 8.Policy Coordination and Spillover Analysis: Which Country Groups Want the IMF to Do It and Consider It Effective?

(Percent of respondents)

1Shows the share of authorities who responded “a fair amount” or “very much.”

2Shows the share of authorities who responded “effective” or “very effective.”

Emerging economies

24. Taken together, the survey, interview, and documentary evidence suggest that IMF interactions with the large emerging economies were effective over a fairly limited range of activities, and that they were effective over a broader range of activities with the other emerging economies. This said, as for all country groups, unique country factors shaped interactions with the Fund.

25. Overall, the large emerging economies rated the effectiveness of interactions lower than any other country group, including the other emerging economies. Box 2, for example, shows that for most composite indicators, there are sizable differences between the large and the other emerging economies. These are driven in turn by the underlying differences in views about the relevance and effectiveness of individual roles, for example on Fund contributions to policy frameworks, programs, and capacity building. Exceptions include cross-country analysis (see Figure 3) and staff turnover and continuity (treated in Chapter 4.), where the large emerging economies provided more favorable ratings than did the other emerging economies.

26. Until the recent crisis, part of the story of interactions in emerging economies was about transitions from programs to surveillance.

  • Interviews with the authorities showed that as countries transitioned to a surveillance-only relationship, interactions became more harmonious. Most interviewed emerging economies’ officials who experienced the transition, either within the evaluation period or earlier, viewed the less prescriptive relationship under surveillance more favorably, although there were exceptions to this view. Yet bad memories of past programs (and program discussions) tended to dominate the views of some interviewed country officials whose authorities would be reluctant to enter into any relationship with the IMF that would involve (or be perceived to involve) a loss of policy autonomy.

  • For Fund staff, the transition entailed a downgrade from a position of power and influence to a new regime of reduced interest by the authorities in dialogue with them, and in turn reduced access to key people and information. Indeed, in contrast to the authorities’ low interest in engaging with the Fund on the policy framework (as shown in Figure 4,), well over 80 percent of staff working on those countries said they aimed to do just that. And in interviews, they reflected on the missed opportunities they saw in the institution’s failing to define a strategy for the new terrain. Several resident representatives linked these problems to the difficulties they experienced in building relations of trust with the authorities.

27. Against this background of changing circumstances and context, interviewed officials from several large emerging economies saw the surveillance exercise as routine and uninteresting—adding little value, as per their low effectiveness ratings for interactions in Figure 2.

  • A majority of interviewees indicated that the IMF’s advice offered few new perspectives or was behind the curve on global financial developments affecting their economies. Some officials indicated that they were looking for new angles on their own policies, but did not get them from the Fund. They were attracted to formats where more interesting discussions took place, increasingly through regional and other broader fora involving interactions with peers.

  • Meanwhile, some mission chiefs tried to deepen the surveillance dialogue, for example, by dovetailing technical assistance with surveillance in ways that both sides considered useful. Other staff reached out to authorities in providing follow-up analysis and advice after missions. According to staff, the authorities tended to value seminars, which provided an informal setting for freer debate than the more formal Article IV dialogue, outside experts, and cross-country perspectives.

  • Several authorities expressed the view that they received less valuable input from Article IV consultations than from technical interactions with the IMF, notably those that took place in the context of the FSAP, Reports on the Observance of Standards and Codes (ROSCs), technical assistance (for inflation targeting in many cases), and training. Several attributed this to the fact that such activities were more closely targeted to countries’ specific needs. Further, authorities for the most part noted that the technical interactions were often conducted by individuals with implementation experience.

28. The evaluation evidence highlights large emerging economies’ concerns about the IMF’s evenhandedness of treatment of different countries, influenced in part by the reaction to the 2007 Surveillance Decision. Most telling, some large emerging economy survey respondents saw the Fund’s surveillance work to be conducted predominantly in the interests of major shareholders, more than in their interests. Figure 9 shows that 86 percent of large emerging survey respondents said that surveillance was in the interest of the “largest IMF shareholders,” while only 68 percent saw surveillance as in their own interests. (This is in contrast to the other country groups, whose responses indicated a belief that their Article IV consultations were primarily in their interests, or in the interests of the membership and the IMF itself.) Indeed, in a few prominent cases, such as Argentina and Venezuela, relations were so strained that regular consultations have not taken place for several years, and in the case of China, tension over the implementation of the 2007 Surveillance Decision led to a delay of two years in completing the Article IV consultation. Some also underscored their desire for advice that they characterized as objective and fair, based on evidence, and driven by facts rather than ideology. One authority felt that a double standard was being applied. Some staff also reported concerns about the emphasis on exchange rate issues and comments from authorities about insufficient IMF criticism of the policies of a major shareholder.

Figure 9.In Whose Interest Is Surveillance?

(Percent of respondents who answered that surveillance was aimed at serving the interests of each beneficiary “a fair amount” or “very much”)

29. The 62 other emerging economies themselves constitute a diverse group, comprising subgroups of “small states,”8 emerging Europe, and relatively small countries in the Middle East, and other regions.9

  • Interviewed officials from these economies offered more positive comments about the basics of interactions than those from large emerging economies, in line with the survey results illustrated in Figure 2. Some welcomed the Article IV process as an opportunity to test their own ideas and to debate alternatives. But others agreed with the large emerging economies that Article IV reports and discussions were often just updates with very little new or interesting to the authorities.

  • Officials from other emerging economies generally said that interactions were improving, with some associating the improvement with their countries’ transition from program to nonprogram status, and others with their countries’ recent change into program status. In the former, officials said they were positive about the changing relationship, largely because they now felt more in control and did not have to deal with conditionality. In the latter, officials cited the redefined conditionality as an improvement, with interactions in the program context very intensive and fruitful, compared to what they characterized as the irrelevance of earlier times.

  • The authorities of these economies gave Fund technical assistance—especially that delivered through its regional technical assistance centers—high marks, in the survey results and the interviews. Interviewed officials from several countries highlighted the staff’s help on banking supervision and inflation targeting. And the FSAP process came in for high praise, including from officials of small states, who more generally were using the IMF to help ratchet up their technical expertise.

PRGF-eligible countries

30. PRGF-eligible countries rated IMF interactions higher than other country group/subgroups on all evaluation indicators set out in Box 2. The high strategic alignment rating is particularly relevant given that it is only for the PRGF-eligible countries that the Fund has an institutional strategy, as discussed in Chapter 4 and developed in more detail in the companion paper on PRGF-eligible countries. In current circumstances, it means that for each of the ten purposes of interactions set out in the evaluation survey, what the authorities of PRGF-eligible countries said they wanted from IMF interactions was closely aligned with what the staff working on PRGF-eligible countries said they aimed to do. (See Figure 10—also in comparison with Figure 6 for the large advanced economies.) In turn, this alignment contributes to effectiveness, as broadly speaking the authorities and staff are both working towards the same goals in their interactions.

Figure 10.PRGF-Eligible Countries: Strategic Alignment Between Authorities and Staff

(Percent of staff who thought IMF “aimed” to perform each role minus percent of authorities who “wanted” the IMF to do so)

31. The evaluation evidence also suggests that most authorities and staff believe that during the evaluation period, the Fund added value in PRGF-eligible countries through financial and monitoring programs, debt relief, and donor signaling. This “suite” of products and services brought to those countries substantial financial benefits, direct and indirect, securely anchoring the Fund’s role with this group of countries. The result was an abundance of traction in Fund interactions—in contrast to the situation in the surveillance-only advanced and emerging economies discussed above. But there was also a downside risk. Summarizing the views of several officials from PRGF-eligible countries, one minister of finance suggested that this traction put staff in a position of power vis-à-vis the authorities, raising questions about how the exercise of that power affected interactions in both substance and style. Substantive issues are touched on below, with the style issues taken up in Chapter 4. Both are explored further in the companion paper on PRGF-eligible countries

32. The evaluation case-study evidence suggests that different countries saw the implications of the Fund’s power differently.

  • For the authorities of some PRGF-eligible countries—especially those subject to major program interruptions and/or delays in debt relief—the underlying power imbalance was seen to drive interactions, including the adoption of what they saw as demanding and inflexible positions by Fund staff to which the authorities had to agree, or else. For several interviewed authorities, “inflexibility” was a lightning rod for bitter complaints about Fund interactions on conditionality, many of which related to major program interruptions and/or delays in the delivery of debt relief. Some felt that the policy agenda and program undertakings had driven by Fund staff in Washington, in part through what some senior officials called the “tyranny of the PDR review process.”

  • For the authorities of other PRGF-eligible countries, the Fund’s power was seen as helpful to their interests whether in dealing with the Paris Club, other creditors, and donors, or in helping them to discipline spending ministries and other interests at home. Several interviewed authorities also pointed to the Policy Support Instrument as an important innovation that put them more squarely in the driver’s seat and gave more meaning to country “ownership” than did the PRGF.

  • The authorities’ perceptions about the Fund’s flexibility have improved in recent years. The survey evidence illustrated in Figure 11 shows that 30–40 percent of PRGF-eligible respondents said that Fund policies and procedures were inflexible and that the staff was insufficiently willing to innovate. But half also said the staff’s willingness to consider different approaches had improved over the past two years. Indeed, most of the specific complaints raised in the interviews of officials from PRGF-eligible countries dated from the 2002–04 period, as detailed in the companion paper on the PRGF-eligible countries.

Figure 11.Is the Fund Inflexible, Unwilling to Innovate, and Closed to New Approaches?

(Percent of authorities who responded “agree” or “strongly agree”)

33. Important exceptions notwithstanding, the authorities of PRGF-eligible countries generally gave the Fund high marks for technical assistance and the specific expertise on which it draws, while raising questions about its strategic thrust and the sustainability of its effects. Some expressed concerns about plans to introduce charges for technical assistance.

  • In the survey, 95 percent of respondents said they wanted the Fund to help build capacity and 80 percent said the Fund was effective in this role. These numbers are much higher than for other country groups, even than for the other emerging economies. They praised staff delivering the Fund’s technical assistance, especially those from the regional technical assistance centers.

  • But several interviewed officials, noting the large quantities of IMF technical assistance that had been provided to their countries over the years, questioned what results it had produced in terms of sustained improvements in local capacity. This raises questions about the programming of technical assistance, and how it relates to the country’s and Fund’s medium-term vision, and about its implementation.

  • The documentary evidence reviewed for the evaluation suggests that the links between Fund technical assistance programs and country priorities improved over the evaluation period, but still remained relatively weak. The Fund’s surveillance agendas, introduced in 2006 as a short-hand strategy brief, often included technical assistance in the section on the staff’s work program, but did not make clear how the programmed items fit into the overall agenda. The Regional Strategy Notes on technical assistance, meant to improve the prioritization of technical assistance in line with the strategic objectives of both recipient countries and the Fund, were launched late in the evaluation period and are still finding their way, especially with respect to links to other work of the Fund and technical assistance provided by donors.10

The evaluation also collected some information on the interactions between Executive Directors and member countries, which revealed very varied relationships across the membership. These issues were examined in greater depth in the recent IEO evaluation of the IMF’s governance.

As discussed and illustrated later in the chapter, in paragraph 23 and Figure 8, the other country groups have much greater interest in such work.

In brief, as a composite, this indicator is very much affected by the weights survey respondents attach to its underlying elements, whether these weights are explicitly articulated or implicit in the respondents’ answers.

Indeed, the indicator closely tracks alternative formulations of the composite indicator for role effectiveness that are weighted towards the authorities’ top priority roles.

Figure 5 shows an entry for the large advanced economies under monitoring/financial programs. This reflects an entry by one respondent to the survey.

See companion paper on “IMF Interactions with Advanced Economies.”

One distinguishing feature of these countries is their average population of 6 million people compared with 200 million in the large emerging economies or 70 million if China and India are excluded.

See IMF (2008b).

    Other Resources Citing This Publication