Chapter 2. Recently Completed Evaluations and Follow-Up on Past Evaluations

International Monetary Fund. Independent Evaluation Office
Published Date:
July 2010
  • ShareShare
Show Summary Details

During FY2010, the IEO issued the report on its evaluation of IMF Interactions with Member Countries, which was discussed by the Executive Board on December 14, 2009. A Management Implementation Plan (MIP) for the Board-endorsed recommendations of this evaluation is scheduled for Board discussion during the first quarter of FY2011. On June 8, 2009, the Board discussed the IEO evaluation of IMF Involvement in International Trade Policy Issues and on December 17, 2009, it approved a MIP for this evaluation.

IMF Interactions with Member Countries

This evaluation assessed the degree to which IMF interactions with member countries were effective and well managed in 2001–08, with particular attention paid to 2007–08. It contained a number of findings that are relevant to the tasks that lie ahead for the Fund in implementing the new responsibilities it has recently been given to help members deal with the global financial crisis.

Overall, the evidence was mixed. Survey results indicated relatively high perceptions of overall effectiveness in some country groupings, lack of agreement between the authorities and staff on the scope of interactions in some cases, and widely varying views of effectiveness in particular roles. Interactions were effective in a program and technical assistance context, and, in general, in contributing to a good exchange of views and in providing objective assessments. However, in other areas, including in the international dimensions of its surveillance and other work, effectiveness and quality were not rated highly.

The evaluation evidence showed that IMF interactions were least effective with advanced and large emerging economies. They were most effective with PRGF-eligible countries, and, to a lesser extent, with other emerging economies. Particularly troubling was the continuing strategic dissonance with large advanced economies, especially about the Fund’s role in international policy coordination, policy development, and outreach. The authorities did not give the Fund high marks for its effectiveness in these areas. Neither did staff, who nevertheless aimed to do more. The evidence also points to limited effectiveness with large emerging economies, many of whom saw the surveillance process as lacking value and/or evenhandedness.

The evaluation found that outreach with stakeholders beyond government contributed little to the effectiveness of IMF interactions. The Fund’s transparency policy did less than staff had hoped to increase the Fund’s traction, as some authorities blocked timely dissemination of mission findings. Dissemination initiatives designed to gain influence in domestic policy debates by repositioning the Fund as an informed analyst remained works in progress.

The evaluation found that interactions were undermanaged, although some individuals managed particular interactions very well. The Fund paid too little attention to the technical expertise and other skills that might have added value and neglected to manage pressures that staff felt to provide overly cautious country assessments—a finding of major concern, especially in respect to staff work on systemically important countries. In PRGF-eligible countries, an institutional strategy replete with attractive financing, debt relief, and strong links to donor funding made for an abundance of traction. But in some cases it also led to what authorities perceived to be arrogant and dictatorial staff behavior—though they saw evidence of progress in recent years. Staff incentives and training largely ignored interactions, and responsibilities and accountabilities for relationship management were not clear.

The report made a number of recommendations aimed at enhancing the effectiveness of IMF interactions with members.

  • To make the Fund more attractive to country authorities and promote traction: (i) improve the quality of the international dimensions of the Fund’s work; (ii) recruit specialist skills and bring more experts on country visits, especially where traction is waning; (iii) articulate menus of products and services for emerging market and advanced economies; and (iv) replace the now defunct country surveillance agendas with strategic agendas to enhance country focus and accountability.

  • To improve the effectiveness of outreach: (v) clarify the rules of the game on outreach; and (vi) decide how to handle the Fund’s negative reputational legacy in countries where it is a factor undermining interactions, and equip staff with the skills and resources to follow through.

  • To improve the management of interactions: (vii) develop professional standards for staff interactions with the authorities on country assessments; (viii) increase mission chief and staff tenure and training, and improve incentives for interactions; and (ix) clarify relationship management responsibilities and accountabilities.

Executive Board Discussion

The Executive Board discussed the IEO evaluation report on December 14, 2009. Executive Directors welcomed the evaluation and the important insights it offers into the effectiveness of interactions with member countries, which they saw as key to the Fund’s ability to achieve its goals. Directors took note of the finding that the majority of authorities in all country groupings had rated the Fund’s overall effectiveness positively. However, they expressed concerns about the indications of a lack of agreement between the Fund and the large advanced and large emerging countries respectively on the scope of interactions, and of widely varying effectiveness in areas where the Fund was supposed to excel. Directors observed that the report covered principally the pre-crisis period, and significant progress had been made on several fronts since then. Nevertheless, several of the report’s key findings remained a source of concern. Directors believed that if the Fund were to effectively respond to the new opportunities and challenges confronting it, careful consideration needed to be given to the IEO’s findings and recommendations, together with other possible areas for enhancement, acknowledging the complexity of interactions with Fund members.

Directors noted that the Fund’s effectiveness was perceived to have improved since the onset of the crisis, reflecting important reforms and the central role which the Fund had been asked to play in the international community’s response to the crisis. A number of Directors observed that the new responsibilities entrusted to the Fund were consistent with the recommendations for a greater role by the Fund in international policy coordination and analysis of spillovers.

Directors stressed that issues regarding the effectiveness and independence of Fund surveillance in large advanced and large emerging market economies merited serious consideration. They acknowledged that interactions with large countries involved special factors, including the extensive analyses of these countries undertaken outside the Fund. Nevertheless, the perceived lack of candor and value by these countries as well as concerns about evenhandedness in the case of emerging economies pointed to challenges requiring close follow up. Concerns were also expressed about the survey findings that staff working on all country groupings toned down their assessments to preserve the relationship with the authorities, and that many staff stated that there was a lack of support from management and senior staff when tensions arose between staff and country authorities. Directors stressed that the need for the Fund to provide candid messages was highlighted by the crisis, and that candor will remain essential in the period beyond the crisis. Some Directors highlighted the survey finding that increased outreach appeared to have done little to improve traction. Some Directors also highlighted the evaluation finding that the transparency policy had done less well in increasing traction than staff had hoped, given evidence that some authorities had resisted timely dissemination of mission findings. Many Directors underscored the importance of governance reforms to address the perceived problems with evenhandedness and to underpin effective interactions with members, based on mutual understanding and trust. A few Directors cautioned against the illusion that changes in the governance of the Fund would do much to improve member countries’ willingness to listen to the Fund’s advice.

Directors welcomed the positive assessment of relations with PRGF-eligible countries, although some cautioned that these members’ relatively weak capacity and reliance on external official funding could mask underlying challenges in the effectiveness of the Fund’s interactions. Directors agreed that a proactive outreach strategy, within overall resource constraints, could help further address some misperceptions about the Fund’s engagement with PRGF-eligible countries. Some Directors observed that a key measure of effectiveness would be a continued strong relationship beyond the PRGF. Directors also welcomed the positive assessment of the role of resident representatives and technical assistance provided by the Fund.

A number of Directors highlighted the importance of addressing internal issues related to institutional culture that undermine the Fund’s ability to establish itself as a trusted policy adviser to members, and noted that the recommendations could have gone further in considering how the institutional culture of the Fund could be made more responsive to the needs of its members. These Directors questioned whether, in part, the issues raised by the IEO evaluation report reflected a lack of clarity as to who “the client” in the Fund’s engagement with members was. At the same time, it was emphasized that membership in the IMF entailed obligations for countries.

Recommendations Endorsed by the Board

Directors broadly supported the thrust of the key IEO recommendations to enhance the traction of Fund surveillance and policy advice, improve the effectiveness of outreach, and strengthen the management of interactions. At the same time, they expressed a range of views on several specific proposals, and underlined that the complexity of the issues warranted further analysis and discussion. The point was made that effective interaction also called for efforts on the part of national authorities and Executive Directors.

Directors stressed the importance of further enhancing the international dimension of the Fund’s surveillance and policy work with greater attention to cross-country analysis and spillovers. They emphasized that, building on the flagship products of the World Economic Outlook, the Global Financial Stability Report, and the Regional Economic Outlooks, there was scope for improvements and better integration of multilateral surveillance products into bilateral surveillance. While efforts to improve traction should aim to strengthen existing instruments in the first instance, Directors remained open to exploring other products that could enhance the Fund’s cross-country analysis. Directors strongly supported efforts to enhance the candor and effectiveness of Fund engagement with country authorities. They acknowledged the critical role of management and the Executive Board in supporting staff analysis involving politically difficult messages. Directors supported continued deepening of guidance and training for staff in conducting country relations.

Directors generally supported the proposal to increase mission chief and staff tenure on country assignments. They welcomed the steps being taken in this regard and noted that this objective should be balanced against the desire for fresh and cross-country perspectives. Directors supported the expanded recruitment of experienced, mid-career professionals to ensure the right skills mix and expertise. Some Directors saw merit in the proposal to bring more experts on country visits, especially when they offer significant value added, but others were concerned that a greater role of experts could run counter to team continuity.

Directors generally considered that it would be useful to provide greater guidance on outreach efforts, while ensuring sufficient flexibility. They agreed that it was important for the Fund to deal forthrightly with current and past controversies, and that outreach efforts should be regularly updated to reflect ongoing reforms in Fund policies. While outreach efforts were seen to have a critical role in overcoming a negative reputational legacy in many countries, Directors generally did not support the proposal to develop specific guidance in this particular area. Several Directors stressed the need for care in engaging in policy debates outside the official dialogue, and a few asked for further discussion on this issue.

The discussion highlighted that efforts to strengthen the role and relevance of surveillance and policy advice across the entire membership needed to be sustained and further enhanced. As the Fund pressed ahead with efforts to assist members in securing a durable recovery from the crisis, promoting candor in staff’s assessments would remain critical. Further reflection was needed on internal cultural changes to enhance the Fund’s engagement with its members.

IMF Involvement in International Trade Policy Issues

On June 8, 2009, the IMF Executive Board discussed the IEO evaluation of IMF Involvement in International Trade Policy Issues. This evaluation came at a time when the global community was once again reminded of the risks to economic growth and stability arising from potential protectionist responses.

The evaluation covered the period since the establishment of the World Trade Organization (WTO)—1996 to 2007. It found that after 2000, as part of its streamlining efforts, the IMF had scaled back its involvement in traditional trade policy issues such as tariff and nontariff barriers to merchandise trade, especially in the context of conditionality. This was welcome, as average tariffs in most countries had fallen to relatively low levels; conditionality often did not achieve lasting changes in trade policy; and the pressure for unilateral liberalization, especially through conditionality, created tensions with multilateral negotiations in the WTO. At the same time, the evaluation pointed to several areas in which the Fund needed to play a larger and more considered role in close cooperation with other multilateral organizations: whether and how countries should liberalize trade in financial services (an issue underscored by the global financial crisis); the systemic implications of the proliferation of preferential trade agreements; and the global effects of trade policies (especially high agricultural tariffs and subsidies) of systemically important countries.

The overarching message of the report was that the IMF should recommit itself to trade policy issues that have potentially significant implications for macroeconomic and systemic stability. The evaluation recognized that the resources the IMF could devote to trade policy were limited and recommended several steps to help rebalance its role in the key areas identified: more active inter-institutional cooperation; creation of a small repository for in-house trade expertise; and periodic in-depth exploration of regional and global implications of trade policy developments in the World Economic and Regional Economic Outlooks.

In concluding the Board discussion, Executive Directors broadly agreed that the Fund has an important role to play on broad trade policy issues and their implications for external stability but highlighted the Fund’s resource constraints. Many Directors viewed engagement as being best served through cooperation with the WTO and the World Bank. Directors agreed that the evaluation provided useful impetus to discussions on what should be the priorities for trade work in the Fund going forward. Most Directors supported the IEO’s recommendations on the need for: (i) periodic Board review of guidance on trade policies; and (ii) guidance on the approach to trade in financial services and the approach to preferential trade agreements where there are issues of spillovers or significant macroeconomic effects. Directors agreed that trade-related conditionality should continue to be macro-critical and take into account country-specific circumstances, as in other policy areas. Most Directors also saw scope for multilateral surveillance to pay greater attention to the global effects of trade policies in systemically important countries.

Following this discussion, a Management Implementation Plan was prepared to follow up on Board-endorsed recommendations on trade policy. The Board agreed to this plan in December 2009. Among the follow-up steps are:

  • reviews of Fund work on trade every five years, beginning in 2014;

  • guidance to staff on trade in financial services and preferential trade agreements, to be developed by July 2010; and

  • revised guidance on trade policy advice/conditionality in the context of Fund-supported programs.

    Other Resources Citing This Publication