Chapter

Chapter 9 IEO Recommendations: A Review of Implementation

Editor(s):
Ruben Lamdany, and Hali Edison
Published Date:
December 2012
Share
  • ShareShare
Show Summary Details

This chapter reviews the implementation of recommendations made by the IEO in its evaluations of the policies and activities of the IMF. The chapter is intended not as a compliance exercise but rather as a contribution to the IEO’s ongoing efforts to improve its work; in reviewing the recommendations, the reactions from the Executive Board, and actions taken in these areas by the IMF, the goal is to promote understanding of the successes and challenges of implementation and monitoring thereof.

The exercise covers 7 of the IEO’s 18 evaluations, to provide a sense of the overall trend in implementation. It also examines selected recommendations in more detail, reviewing the Board response, summarizing the steps taken to pursue implementation, and describing the current status of each. The recommendations selected provide examples of a range of implementation outcomes.

Our main conclusion is that the IMF has taken some action on the majority of recommendations examined.1 For instance, policies and Board expectations have been clarified in a number of areas. At the same time, issues remain with respect to the implementation of many recommendations. For instance, actions may have been taken to implement a recommendation but also failed over time to satisfy the objective set out; or an issue may have persisted despite the targeted steps taken to address it. And in a few cases, the IMF has taken no or minimal actions to follow up on a recommendation endorsed by the Board.

The chapter is organized as follows. The first section explains the approach taken in assessing implementation and summarizes the main findings. The second section discusses selected recommendations in more detail. The final section concludes, highlighting ongoing issues.

Assessing Implementation: Approach and Summary Results

Typically, IEO evaluation reports include a range of recommendations. High-level recommendations represent overarching proposals, while subsidiary recommendations provide more detailed or technical proposals grouped under the high-level recommendations. Some IEO evaluations also include suggestions, by offering examples of how the IMF might pursue recommended changes that do not rise to the level of recommendations.

Approach

The exercise discussed in this chapter focuses on high-level IEO recommendations that were endorsed in whole or part by the Executive Board. We relied on the Summing Up of the Board discussion of each evaluation to determine whether and to what extent each recommendation was implemented. The Summing Up of a Board meeting summarizes Board views and decisions, which are mostly reached by consensus. Interpreting the Board’s position sometimes required a degree of judgment. In some cases, the Summing Up clearly states that Directors agreed with or supported an IEO recommendation; in others, the Summing Up indicates agreement with or support for part of an IEO recommendation; and in others, the Summing Up introduces nuances to an issue by addressing an IEO recommendation indirectly and/or advancing alternative approaches. In this exercise, we considered high-level recommendations that were clearly endorsed and those that were endorsed partially or in a nuanced way.

As input to our analysis, we took note of the IMF’s Management Implementation Plans (MIPs) to implement Board-endorsed recommendations. MIPs were introduced in 2007 following the first external evaluation of the IEO and have been produced for four of the seven evaluations covered here.2 We also collected information provided in past reporting, both the informal reporting provided in IEO Annual Reports from 2003 through 2007 and the formal Periodic Monitoring Reports (PMRs) by IMF staff that began in 2007.3 We also gathered and analyzed additional evidence of relevant actions taken by the IMF. We reviewed Fund documents ranging from IMF policy papers to internal IMF policy reviews (e.g., the Triennial Surveillance Reviews undertaken in 2008 and 2011) to information posted on the IMF internal website. We also interviewed IMF staff to gather information and confirm understanding of the extent of actions taken. In addition, we relied on relevant findings from subsequent IEO evaluations.

The analysis undertaken here does not represent a re-evaluation of the issues initially examined by the IEO reports. Rather, it considers the Board position on past IEO recommendations as given and reviews the steps taken to implement them. The assessment required a significant degree of subjective judgment, for instance about whether implementation could be considered adequate if significant actions were undertaken only recently or with substantial time lags. Another example of difficult judgment calls was how much movement would be required to satisfactorily implement recommendations that called for relative changes—that is, an improvement or strengthening of policies or practices—rather than citing specific goals. In some cases, because of weak monitoring systems, it was difficult to determine whether any action had been taken or whether, instead, the available evidence did not provide IEO with enough information to discern action. It is likely that some of the recommendations for which the IEO was not able to assess implementation were in fact implemented to a significant degree. At the same time, it is also possible that a more in-depth analysis may find that some recommendations for which implementation was deemed satisfactory did not adequately address the underlying concerns.

Summary Results

We tallied all recommendations across the 18 IEO evaluation reports issued over 10 years and determined that the IEO made 117 high-level recommendations aimed at addressing significant concerns related to IMF policies and operations. The IEO also made about 160 subsidiary recommendations and offered about 70 suggestions and/or examples of how some of the high-level recommendations could be implemented.4 Of the 117 high-level recommendations, about 85 percent gained support from the Executive Board, whether full endorsement or partial or nuanced support. About 8 percent of the high-level recommendations were rejected by the Board, while about 7 percent were not addressed. The Board supported fully, partially, or with nuance about 40 percent of the subsidiary recommendations made in IEO reports, and it rejected about 10 percent. Nearly half of the subsidiary recommendations in these 18 evaluations were not directly addressed in the Summings Up of Board discussions.5

Review Sample

The exercise focuses on the seven evaluation reports that IEO completed between 2005 and 2009.6 This period is sufficiently in the past to permit a meaningful reflection on implementation. The evaluations covered are:

  • Financial Sector Assessment Program (2006)

  • Multilateral Surveillance (2006)

  • The IMF and Aid to Sub-Saharan Africa (2007)

  • IMF Exchange Rate Policy Advice (2007)

  • Structural Conditionality in IMF-Supported Programs (2007)

  • Governance of the IMF (2008)

  • IMF Involvement in International Trade Policy Issues (2009).

These evaluation reports contain 41 high-level recommendations.7 Of these, 38 were endorsed by, or gained partial or nuanced support from, the Executive Board (20 recommendations were clearly endorsed by the Board and the other 18 were supported partially or in a nuanced way).

Status of Implementation

Using the approach outlined above, we assessed the level of implementation for the 38 Board-supported recommendations:

  • For about one-quarter of these Board-endorsed recommendations, implementation appeared to have proceeded consistent with the Board position, substantively as well as in terms of the implementation pace.

  • For an additional 50 percent, some action had been taken but there were issues with implementation; for instance, some actions were taken but efforts were partial, incomplete, or stalled.

  • For about 15 percent, minimal or no direct follow-up action was found.8

  • For about 10 percent, the evidence found was insufficient to discern the status.

These results are based on the seven evaluations examined and thus cannot be considered conclusive across all IEO evaluations. However, the analysis provides a perspective on: (1) the complexities of judging Board support; (2) challenges encountered by IMF Management and staff in implementing IEO’s recommendations; (3) the difficulties in monitoring implementation; and (4) the complications of assessing whether implementation was satisfactory.

Assessing Implementation: Selected Recommendations

This section looks more closely at selected recommendations from the seven evaluations. For each of the evaluations in the sample, the following discussion first introduces the recommendations made and then explores the follow-up on one or two of the high-level recommendations, indicating: the Executive Board’s position; actions taken by the IMF relevant to the Board’s position; and the status of implementation of the recommendation.9 The examples were chosen to help illustrate the range of implementation of Board-endorsed IEO recommendations, as well as different difficulties encountered in assessing implementation.

Evaluation: Financial Sector Assessment Program (2006)

IEO’s evaluation of the Financial Sector Assessment Program (FSAP) found that the program had deepened the IMF’s understanding of the financial sector and significantly strengthened the quality of its dialogue with member countries on financial issues.10 The evaluation concluded that the FSAP initiative faced important challenges with respect to setting priorities for coverage and ensuring that FSAP results were fully incorporated in IMF surveillance. The Executive Board agreed with the overall conclusion of the evaluation that the FSAP made an important contribution to the Fund’s work and considered that the IEO report provided a balanced and candid assessment of areas for improvement, particularly integrating financial stability assessments into bilateral and multilateral surveillance. An IMF Financial Sector Task Force examined a number of issues that were raised by the evaluation, including the integration of financial stability assessments into surveillance, although the Task Force was not specifically established to follow up on the evaluation.

Of the seven high-level recommendations that were made by the FSAP evaluation, six received full or partial support from the Executive Board. The following two examples illustrate different implementation outcomes: the first shows actions taken over time to fulfill the intent of a recommendation; the second is a case in which efforts were made but did not achieve the goals in a timely manner.

Recommendation: Set Priorities for FSAP Participation

The IEO recommended that IMF Management provide clear signals to the Board about which countries were the highest priorities for financial stability assessments and updates, even if these countries had not volunteered for the program, and that these lists be taken as the basis for periodic Board discussions of country-specific priorities. This recommendation was aimed at strengthening country incentives to participate in the program and grew out of the evaluation finding that the existing practice had allowed “some authorities’ reluctance to participate” to influence IMF staff preparation of priority lists.11

Board view: Most Executive Directors supported the recommendation that Management should communicate FSAP priorities to the Board and that Article IV staff reports should explicitly recommend a financial sector assessment or update in priority cases—although some Directors cautioned against using peer pressure, which would contradict the voluntary nature of the FSAP.

Action taken: Following the evaluation, Fund staff prepared annual reports on FSAP participation (in 2006, 2007, and 2008), which were provided to the Board for information after the fact, rather than for discussion. IMF monitoring of the status of this recommendation mentions these reports but emphasizes the Management/staff process for prioritizing and scheduling financial stability assessments, rather than communication with the Board. While this initial approach appears to fall short of the Board position following the evaluation, the issue of coverage was reconsidered in the 10-year review of the FSAP in 2009. Further policy discussions resulted in a Board decision that incentives and prioritization had not been sufficient to gain needed participation. As a result, the Board decided to make financial stability assessments mandatory for 25 systemically important countries at a minimum interval of every five years.

Status: We judged that the root of the recommendation was addressed with the IMF’s ongoing attention to participation in the FSAP and to the incentives for doing so.

Recommendation: Link FSAPs with Surveillance

The evaluation recommended that the IMF strengthen the links between the FSAP and surveillance by making the FSAP a more central part of the Fund’s work and by integrating the follow-up on financial stability assessments into surveillance activities—including in written reports, policy recommendations, discussions with authorities, and peer review in the Executive Board.12

Board view: The Executive Board concurred with the IEO recommendation to strengthen the links between the FSAP and surveillance, and in particular to improve follow-up on key vulnerabilities and gaps in country work. Directors called for short summaries of financial stability assessments, highlighting key findings and macroeconomic implications; increased focus in Article IV consultations on highly important financial stability issues; and greater attention to significant macro-relevant issues in Board discussions.

Action taken: The PMR that was issued in December 2007 indicated that implementation had been completed, since the internal review process had been strengthened to ensure clear communication of macro-relevant findings of financial sector assessments and incorporation of this analysis in Article IV reports.

Our analysis confirmed that the IMF devoted substantial attention to integrating financial sector analysis into surveillance through the 2007 Financial Sector Task Force Report and the 2008 Triennial Surveillance Review, and implemented changes including those cited by the PMR.

We also found that the IMF itself continued to raise concerns about insufficient connections between financial sector and surveillance work in the years following the evaluation. The IMF’s 2009 “Progress Report on Integrating Financial Sector Issues and FSAP Assessments into Surveillance” noted (p. 7) that financial sector issues had become more central to Article IV consultations but that “obstacles to boosting financial sector coverage remain[ed] and, in some cases, [have] become more acute.” Further, in discussing the Fund’s mandate and financial sector surveillance in 2010, the Board stated that “despite the progress in recent years, there remains much scope for improving . . . financial sector analysis and the integration among all dimensions of surveillance.”13

The 2011 IEO evaluation of IMF Performance in the Run-Up to the Financial and Economic Crisis: IMF Surveillance in 2004–07 found ongoing inadequacies in the links between macroeconomic and financial sector analysis and a tendency among IMF economists to downplay financial issues.14

The IMF’s own judgment in mid-2011 was that “integration of FSAPs into Article IVs remain[ed] insufficient.”15 As part of the 2011 Triennial Surveillance Review (TSR), the IMF undertook to lay out a strategy for financial surveillance, including integration of financial stability assessments; discussion of a work plan in this area was under discussion in April 2012.

Status: On balance, we recognized that some action was taken, and intensified in 2012, but overall judged that the recommendation was only partially implemented.

Evaluation: Multilateral Surveillance (2006)

The IEO evaluation of Multilateral Surveillance concluded that many individual components of multilateral surveillance were of high analytical quality but found considerable scope for improvement, in particular identifying a need for better integration of financial and macroeconomic dimensions within the Fund’s analysis and for a stronger linking of policy prescriptions to bilateral and multilateral analysis. The Executive Board, in discussing the evaluation, underscored the importance of multilateral surveillance, agreed that there was scope for improvement, and considered ways to accomplish this, based on the IEO’s recommendations.

This evaluation made four high-level recommendations, all of which received full or partial support from the Executive Board. Some of the issues that it raised were featured in the subsequent IMF work program—for instance via the Medium-Term Strategy (MTS) in 2006 and the TSR in 2008. The objectives of the following recommendation were only partially addressed by the steps taken.

Recommendation: Enhance Multilateral Surveillance Outputs

The IEO recommended that the IMF improve the content and form of multilateral surveillance through streamlining and more focus on key issues. This recommendation grew out of the evaluation’s findings that more attention was needed to exchange rate issues and spillovers and that macroeconomic and capital market analysis were insufficiently integrated in the Fund’s World Economic Outlook (WEO) and Global Financial Stability Report (GFSR).

Board view: The Executive Board agreed that the Fund’s multilateral surveillance outputs would have more impact on the global policy debate if they were better targeted to their core audience, streamlined, and more focused on key issues. The Board discussed but did not endorse a range of accompanying proposals or suggestions made by IEO to advance this goal.

Action taken: Reporting on this recommendation in the December 2007 PMR, IMF staff asserted that multilateral surveillance had increased its focus on cross-country messages and strengthened risk analysis—and that the Fund’s outreach on WEO messages had been enhanced.

We confirmed several steps taken by the IMF to address the effectiveness of multilateral surveillance—an issue that was also being considered as part of the 2006 Review of the IMF’s Medium-Term Strategy.16 Changes were made to the WEO, including addition of an executive summary and quarterly updates. The Statement of Surveillance Priorities was introduced in 2008 as a vehicle to help guide bilateral and multilateral surveillance and to help improve their focus by identifying objectives, priorities, and responsibilities; surveillance priorities were agreed by the Board in August 2007 and October 2008.17

Nonetheless, the IMF’s own 2008 TSR criticized “the proliferation of vehicles” and called for “[i]mprovements in brevity, timeliness and clarity, and a strategic delivery of few key messages. . . .”18 In the prior year, the IEO’s evaluation of IMF Exchange Rate Policy Advice found insufficient attention to the spillover effects of country policies on other countries and to the lack of integration of multilateral surveillance messages and financial sector issues into bilateral surveillance. And in 2011, the IEO Financial and Economic Crisis evaluation concluded, among other things, that risks were not highlighted and key messages were not integrated in the WEO and GFSR.19 The IMF’s 2011 TSR similarly pointed out gaps in coverage20 and noted that the large volume of surveillance products created risks of “overlap, inconsistency, excessive segmentation of messages, and inefficient work practices.”21

The International Monetary and Financial Committee (IMFC) requested in April 2011 the preparation of a consolidated multilateral surveillance report (CMSR), and the 2011 TSR concluded that the preparation of such a report “had the potential to increase traction” and to enhance dialogue within the institution about key issues and policies. Although in September 2011 the IMFC welcomed the first CMSR as a way to help focus discussion on key risks and policy issues, it is too early to judge whether this series of reports will sufficiently target the core audience, streamline, and focus on key messages to impact global policy debates—even six years after the evaluation.

Status: This recommendation was judged only partially implemented because insufficient action was taken, at least until very recently.

Evaluation: The IMF and Aid to Sub-Saharan Africa (2007)

The evaluation of The IMF and Aid to Sub-Saharan Africa found ambiguity and confusion about IMF policy and practice on aid and poverty reduction, as well as differences of views among members of the Executive Board about the IMF’s role and policies in low-income countries (LICs). The overarching message of the evaluation was that the Fund should clarify its policies, be clear and accountable in implementing them, and engage more proactively with partners such as the World Bank. The Executive Board broadly endorsed the report’s findings and recommendations, which it noted were particularly relevant to the Board discussions of the Fund’s role in LICs that were scheduled in the following months.

This evaluation made three high-level recommendations, each of which was endorsed by the Board. As detailed in the following two examples, clarification of policy came quickly but the IMF did not follow through with mechanisms to hold itself accountable by tracking the implementation of these policies.

Recommendation: Clarify Policies

The IEO recommended that the Board clarify IMF policies on the underlying performance thresholds for the spending and absorption of additional aid, the mobilization of aid, alternative scenarios, poverty and social impact analysis, and pro-poor and pro-growth budget frameworks.

Board view: Executive Directors fully endorsed this recommendation and provided several initial policy clarifications during the discussion of the evaluation itself.

Action taken: The Board made further clarifications in discussions of policy papers in July and October 2007. In approving the new IMF Extended Credit Facility and other facilities for LICs in 2009, the Board reiterated guidance related to the evaluation, for instance reiterating the policy that aid should generally be fully spent and absorbed over the medium term.22 Staff whom the IEO interviewed for the current exercise also emphasized that the reformed architecture for the Fund’s LIC facilities entails an expectation that programs rely on indicative floors for social and other priority spending, and that documents discuss how programs advance the country’s poverty reduction strategy objectives.23

Status: We found no issues with implementation.

Recommendation: Monitor and Evaluate Implementation

The evaluation called on IMF Management to establish transparent mechanisms for monitoring and evaluating the implementation of the clarified policy guidance.

Board view: The Board fully supported this recommendation, and reiterated its importance in its discussion of the proposed MIP.24

Action taken: The MIP called only for a three-year review of experience, rather than for the creation of mechanisms to track performance on an ongoing basis—which would have provided the input for a subsequent review. Currently, the internal review process serves as the only framework to monitor implementation of the clarified policy guidance, since there is still no other mechanism which is specifically aimed at monitoring. But the review process is intended to ensure adherence to and consistency with IMF policies and not to monitor ongoing performance.

Status: We judged that there had been no or only minimal direct follow-up to establish a mechanism for ongoing monitoring as emphasized by the Board.25

Evaluation: IMF Exchange Rate Policy Advice (2007)

The evaluation of IMF Exchange Rate Policy Advice concluded that the IMF was not as effective as it needs to be in fulfilling its core responsibility for exchange rate surveillance. While acknowledging some progress during the time period covered by the evaluation, the report called for a major refocusing of efforts aimed at enhancing the effectiveness of the IMF’s analysis and advice, and urged the Fund to re-energize its contribution to policy dialogue with member countries. The Board broadly endorsed the report’s conclusion on the effectiveness of the IMF’s exchange rate policy advice and also partially or fully supported each of the 11 high-level recommendations.

The following example is a case in which attention was directed to the recommendation made by the IEO and reiterated by the Board, but the issue persisted.

Recommendation: Integrate Analysis of Policy Spillovers

The IEO recommended the creation of incentives to develop and implement guidance for the integration of spillovers into bilateral and regional surveillance. This recommendation aimed to address the evaluation finding that the Fund had been paying too little attention to spillovers including the regional or systemic impact of large countries’ policies, offering spotty analysis for most countries and making only limited attempts to assess the effects of intervention activities on other members.

Board view: In discussing the evaluation, the Board underscored the importance of “better incorporating analysis of policy spillovers into bilateral and regional surveillance,” and welcomed initiatives taken in this area under the MTS. In preparing a MIP for this evaluation, IMF staff took its cue from the latter statement and proposed no new initiatives beyond continued efforts through the MTS to focus on regional trends, improve assessment of economic and financial market spillovers, and address spillovers emanating from systemic countries. However, in discussing the MIP, Directors identified this as one of the areas meriting further attention: some Directors indicated that additional action on spillovers might be warranted and encouraged careful monitoring of ongoing improvements.26

Action taken: The 2008 PMR asserted that this issue had “received renewed emphasis in the refocusing process and in the TSR” and noted that further initiatives were outlined in the Managing Director’s Strategic Directions paper.27 These initiatives included a “renewed” effort by the Surveillance Committee to bring global perspectives into Article IV consultations, especially in systemically important cases;28 however, these steps also coincided with the IMF downsizing effort.

Subsequent analysis of IMF advice in the period before and during the early part of the recent crisis judged the analysis of spillovers “wanting.”29 While the 2011 IEO Financial and Economic Crisis evaluation cited increased treatment of spillovers in the Fund’s World Economic and Market Developments presentation by 2008, it concluded overall that spillovers had received too little attention in the pre-crisis period. Moreover, the 2011 TSR found that analysis of spillovers in Article IV consultations was limited and that member country officials wanted more. The TSR emphasized that an understanding of spillovers provides an important bridge between bilateral and multilateral surveillance and also serves to enhance the legitimacy and traction of surveillance.

The IMF has sharply increased its work on spillovers very recently as a part of post-crisis enhancements. Spillover reports for five systemic economies were issued in mid-2011, and the IMF intended to prepare three to six more in 2012, including one on financial sector spillovers from major financial centers. Pending Board approval of an integrated surveillance decision, the expectation is that spillover analysis will become a regular feature in Article IV staff reports and multilateral surveillance.

Status: This recommendation was judged to be partially implemented, given the persistence of concerns in this area three years after the evaluation. Action has been taken very recently, and other actions are being considered, in particular for implementation following the possible approval of a new integrated surveillance decision.

Evaluation: Structural Conditionality in IMF-Supported Programs (2007)

The IEO evaluation of Structural Conditionality in IMF-Supported Programs found that the average number of structural conditions attached to IMF lending had failed to decline since the introduction of a streamlining initiative in 2000, although some progress had been made in shifting these conditions toward core areas of IMF expertise. It also highlighted the persistence of overly detailed conditions that were not macro-critical, and insufficient explanation of why such conditions were needed to bring about lasting reforms. The report emphasized the need to achieve greater parsimony in the number of conditions and greater focus on core areas of IMF expertise, as well as to concentrate on changing incentives within the IMF to tailor programs more effectively to realities on the ground.

The evaluation made six high-level recommendations, five of which were fully or partly endorsed by the Executive Board. The following discussion highlights an operational change that directly addresses a Board-endorsed recommendation and a case in which the IMF pursued implementation of a subsidiary component while not addressing the core recommendation endorsed by the Board.

Recommendation: Clarify Expectations About the Number and Scope of Structural Conditions

The evaluation findings suggested that the Board should reaffirm the need for the IMF to use structural conditions sparingly and only when they are critical for achieving program objectives or safeguarding IMF resources.

Board view: The Board broadly supported strengthened efforts to streamline conditionality, with parsimony as the guiding principle and a focus on measures critical to achieving program objectives. At the same time, however, a majority of Directors rejected the IEO’s specific recommendation on how to achieve this—that is, to cap the number of conditions—because they viewed this approach as overly rigid and mechanistic.30

Action taken: After the evaluation, IMF Management issued revised guidance to staff that reaffirmed the importance of parsimony in program-related conditions and clarity in the specification of conditions. Specific criteria were set for judging whether prior actions, performance criteria, and structural benchmarks were justified in terms of their criticality to program objectives. Subsequently, in 2009, the Board decided to eliminate structural performance criteria from IMF-supported programs, while allowing programs to continue to include structural benchmarks to monitor progress during the implementation of the program. A further revised guidance to staff was issued in January 2010 to reflect the Board decision that placed greater reliance on a review-based approach to monitoring structural reforms in Fund-supported programs.

Status: We found no issue with implementation.31

Recommendation: Monitoring and Evaluation Framework

The IEO recommended that the IMF develop a monitoring and evaluation framework linking the conditions attached to IMF lending in each program to country reforms and specified goals. As an interim measure, recognizing that developing such a framework could be a lengthy process, the IEO recommended improvements to the conditionality-tracking mechanism already in place (MONA).

Board view: Directors agreed that Board papers for Fund program arrangements should better explain the link between program goals, strategies, and conditions and that this framework should be monitored.

Action taken: The MIP for this evaluation indicated that the IMF would pursue improvements to MONA (the interim recommendation), but did not specify the steps that would be taken to develop a comprehensive monitoring and evaluation system (the main IEO recommendation). Subsequently, PMRs noted that enhancements to MONA were proceeding. But while the enhancements to MONA and full public electronic access to it are positive steps, the IEO found no evidence of a monitoring and evaluation framework that links conditions to reforms and program goals—and no plans to create such a framework.

The Board Evaluation Committee returned to this issue in January 2010 when discussing the Third PMR and suggested that more work could be done on the system to track goals and strategies and their links to the conditions attached to IMF programs. The Fourth PMR reported in March 2011 that work had begun on the 2011 Conditionality Review, utilizing the MONA database to assess how well program conditions had been linked to program goals, and that in the process staff would assess the need for any further changes to the MONA database.

Status: We judged that this recommendation saw no or only minimal direct follow-up, as an ongoing monitoring framework is not yet in place.

Evaluation: Governance of the IMF (2008)

This evaluation examined Fund governance arrangements, focusing on the International Monetary and Financial Committee (IMFC), the Board, and Management. Accordingly, the Board and Management agreed that a joint statement would be released in lieu of a Summing Up and that the requirement for an MIP should not apply. Some of the issues raised were under the purview of Fund Governors and were discussed in a variety of fora, including by the IMFC, and by ministers and heads of state at successive G-20 meetings and in respective communiqués. Several bodies were established to carry work forward, including an Executive Directors’ Working Group, which approved a work plan in September 2008. The Managing Director established an external committee on IMF governance reforms led by Trevor Manuel and a mechanism to receive civil society and other external views. The Managing Director also proposed a joint steering committee to consider the IEO evaluation report and other inputs.32

The evaluation made four high-level recommendations. Although the Executive Board did not take a view on them, all four were widely recognized as addressing important areas for action. The first of the following examples is a case of partial action on an issue that reached beyond the direct purview of IMF Management and Executive Board, and the other is a case in which we did not come to a conclusion about the status of implementation.

Recommendation: Enhance Ministerial Involvement

The evaluation noted that the IMFC functions as an advisory body, without a mandate to set strategic directions and provide high-level oversight of the IMF. The evaluation called for activation of the ministerial-level Council provided for in the IMF’s Articles of Agreement to fulfill these roles. Aware that the activation of this Council could take a very long time, the IEO recommended a series of interim (or alternative) reforms to involve ministers more systematically within the existing IMFC structure.33

Responses: The need to involve ministers more actively was widely acknowledged in the period following the evaluation. The Joint Statement by the Executive Board and IMF Managing Director (May 27, 2008) acknowledged that the evaluation report had “raised important questions [including] how to ensure effective ministerial and Board involvement in institutional decision-making processes.” G-20 leaders also specified the “Fund Governors’ involvement in the strategic oversight of the IMF” as a critical issue to be addressed, along with other governance matters, as part of the review of quotas that was to be completed by January 2011.34

Action taken: The Executive Board proposed some steps to enhance the IMFC’s functioning, though the Board does not have the standing to implement changes. These potential actions included: focusing the Committee’s meetings on a few systemic or controversial issues; eliminating set speeches; introducing a three-year term limit for chairmanship; introducing a troika model of past, present, and future chairs to facilitate leadership; overhauling deputies’ meetings so that they serve more as agenda-setting sessions; and circulating draft communiqués further in advance.35 In July 2010, IMF staff proposed the creation of the International Monetary and Financial Board, which would have certain decision-making powers, but many Directors “remained unconvinced of the need for a ministerial decision-making body.”36 In October 2010, Directors emphasized agreement that “engagement by ministers and governors is essential to the effective discharge of the institution’s responsibilities, including to promote multilateral cooperation and coherence of policies”—although they could not agree on whether to reform the advisory IMFC or to shift to a body with decision-making power.

Some changes in procedures and practice of the IMFC have taken place, in line with IEO’s recommendations. In 2008, a chairman was chosen from an emerging market country, with a term of three years; the subsequent chairman was also chosen from an emerging market country and given a fixed term of three years. In addition, there is reportedly more consultation with member governments on topics for discussion. Beginning in April 2009, restricted dinner and breakfast sessions were introduced prior to the plenary session. Communiqués are also circulated earlier.

Nonetheless, IMF Governors still saw the need for further action in October 2010, when the IMFC communiqué identified “enhanced ministerial engagement and strategic oversight” as an outstanding issue to be resolved. And in both April and September 2011, the IMFC “look[ed] forward to enhancing the role of the IMFC as a key forum for global economic and financial cooperation.”

Status: Given the calls from IMF Governors for further action in this area, we judged this recommendation to be partially implemented.

Recommendation: The Board Should Introduce an Accountability Framework for Management

While the IEO report recognized that work was already under way in this area, it called for clear proposals on: (1) performance criteria for the conduct of the ordinary business of the Fund and the quality and outcomes of the Fund’s activities; (2) processes to be used for assessing performance; and (3) the approach for translating of performance assessments into incentives. The IEO report noted that, to be effective, the evaluation of Management’s performance might need to be delegated to a Board committee, and that assessments might need to be confidential.

Responses: The Joint Statement of the Executive Board and Managing Director acknowledged that the evaluation report raised important questions in a number of areas, including how to strengthen the management accountability framework. It also noted that work in this area was under way.

Action taken: An Executive Directors’ Working Group on the Framework of the Managing Director’s Performance Evaluation was in place prior to the completion of the Governance evaluation report. The 2008 Report of the Executive Directors’ Working Group on IMF Corporate Governance noted that the Working Group on the MD’s Performance Evaluation was preparing performance objectives. The contract of the current Managing Director states that she will participate on an annual basis in a confidential and informal performance feedback process between herself and Executive Directors.37

Status: The evidence currently available to the IEO is not sufficient to assess the extent of improvements in the framework in place for Management accountability.

Evaluation: IMF Involvement in International Trade Policy Issues (2009)

The evaluation advised the IMF to rebalance its role—with a recommitment to trade policy issues that have potentially significant implications for macroeconomic and systemic stability. The Board broadly agreed that the Fund had “an important role to play on broad trade policy issues and their implications for external stability.” It endorsed five of the IEO’s six high-level recommendations while providing limited support for the other, citing IMF resource constraints.

The IMF has acted on a number of the recommendations, with the following example illustrating an operational change that is in train but will need future monitoring.

Recommendation: Periodically Review Guidance

The IEO recommended that the Board commit to periodically re-evaluate its guidance on objectives of, approaches to, and modalities of staff work on trade policies, in the context of global trends on trade.

Board view: The Board agreed that such a review is needed every five years.

Action taken: The first review is scheduled for 2014, five years after Board discussion of the evaluation.

Status: We found no issue with implementation of the recommendation. The best time to assess implementation would be after 2014, that is, after the first review is scheduled to take place.

Additional Observations and Conclusion

The assessments made in this chapter add to and are broadly consistent with discussions elsewhere of the follow-up to IEO evaluations. For instance, the 2006 report of the External Evaluation of the IEO noted that IEO evaluations were taken seriously by the IMF, highlighting the Fund’s introduction of ex post assessments, better debt sustainability analysis, and the creation of a nonmonetary policy support instrument as a policy benchmark for development assistance flows from other agencies, as well as significant changes that were expected in the framework for technical assistance.38 But the 2006 report also pointed to minimal follow-through on the need for “greater candor by staff in alerting the Board to risks to Fund-supported programs, even where major shareholder interest is high” (p. 24). In addition, the first two PMRs concluded that the IEO had “a significant impact” on IMF operations and cited a number of recommendations as examples.

A few additional observations and conclusions about the follow-up process are given below.

Set Clear Expectations About What Is Required to Implement Board-Endorsed Recommendations

The current process allows too much ambiguity about what IMF staff intends to do to implement each recommendation and about how proposed actions address the corresponding issues and problems. The introduction of MIPs provided a mechanism for IMF Management and staff to specify, and the Board to provide feedback on, what would be done to address IEO recommendations endorsed by the Board. Benchmarks were introduced in the first PMR in order to establish key markers for implementation of Board-endorsed IEO recommendations. However, sometimes the proposed benchmarks consisted of activities that IMF staff planned to undertake, such as preparing a policy paper or conducting a review, without explaining how these activities would achieve the policy or operational change requested by the Board. For example, three of the five benchmarks for follow-up on the evaluation of Aid to Sub-Saharan Africa called simply for preparation of a policy paper, rather than the objective of achieving Board-clarified policies on aid.

In discussing the first PMR, the Executive Board supported the initial effort by IMF staff to establish criteria for gauging progress in implementation. One Director noted that benchmarks were welcome as a way to try to make the monitoring results more tangible and meaningful, but also expressed concern that the chosen benchmarks seemed more to indicate the path forward than to provide measures for monitoring performance. Overall, the Board considered that monitoring would benefit from greater specificity and clarity of the follow-up actions required—including in their formulation in Board discussions, their recording in the Summing Up, and their articulation by Management in MIPs. In practice, however, the process has continued to rely on benchmarks that are often vague on how they relate to the intended goals.

Track Recommendations Until Their Goals Are Substantially Achieved

Under the current monitoring system, the implementation of an IEO recommendation (and of the corresponding benchmarks) is no longer tracked once IMF staff judges, and the Board agrees, that benchmarks have been met, or that implementation is on track for timely completion. This process allows IMF staff to discontinue tracking the implementation of recommendations without confirming whether the broader policy issue or objective set by the Board has been addressed.

This study identified several examples in which the implementation of IEO recommendations was deemed complete even while the IMF recognized that an issue was still of concern and continued work to address it. For instance, with respect to the recommendation to strengthen the links between the FSAP and surveillance, the first PMR noted that the internal review process had been strengthened in several ways to address this concern and did not identify this recommendation as among those outstanding. Meanwhile, IMF policy papers and discussions concurrent with and after this PMR acknowledged that integration of financial stability assessments and surveillance was an ongoing issue that required further action. Moreover, in 2009, the Third PMR concluded that there was no need to continue tracking implementation of any of the benchmarks for recommendations from previous IEO evaluations.39 This disconnect may result in part from the narrow benchmarks discussed above and the role they played in the process. In recognition of this issue, the Fourth PMR acknowledged the need for updates on broader issues.

Some IEO recommendations are integrated into the Fund’s ongoing work program and receive continued attention and Board review in this way.40 The recommendation to clarify expectations about the number and scope of structural conditions is one example. However, there is no procedure for identifying which IEO recommendations have been integrated in the Fund’s work program and do not require separate future monitoring, which do not need ongoing monitoring, and which should be monitored on an ongoing basis given the nature of the follow-up required. A number of recommendations were identified as outstanding in the first two PMRs, and a brief update on these was provided in subsequent PMRs. In addition, when considering PMRs, the Board Evaluation Committee typically identified several issues for further follow-up, which were then picked up in subsequent reports from IMF staff.

While one-off recommendations may not require ongoing monitoring, recommendations that address more entrenched or challenging issues of policy or practice—for instance, calling for enhanced links between the FSAP and surveillance—tend to be addressed only over time, and efforts to do so may not be easily condensed into a singular benchmark. In addition, when operational changes are introduced to address broader policy objectives—such as the creation of surveillance agendas to improve the effectiveness of dialogue with member country authorities—continued tracking could help assess whether progress is being made in meeting the objective, and whether the operational change is being maintained.

Identify and Address Shortfalls in Implementation

In reviewing the four PMRs prepared by IMF staff to date, we found that these reports have become more mechanical over time and have gradually ceased identifying and addressing shortfalls in implementation. The Evaluation Committee initially envisioned that the PMRs would identify recommendations not implemented as well as remedial actions to improve implementation of these outstanding recommendations where appropriate.41 The first two PMRs acknowledged that some recommendations had not been fully implemented, and the first PMR also identified remedial actions to improve implementation. But the practice of identifying and addressing recommendations not fully implemented was not continued in the Third and Fourth PMRs, both of which concluded that there were no outstanding performance benchmarks to be reviewed in the next PMR.42

Conclusion

This chapter concludes that the IMF attends to IEO evaluations and has taken actions to implement their recommendations. While recognizing that many IEO recommendations called for action on challenging issues, for which solutions may be difficult or may require a long time to address, the analysis indicates that more can and should be done to advance the implementation of these recommendations. Key steps in this direction include better specification of follow-up actions clearly linked to the intended goals, and a more transparent monitoring system that would allow the Board and Management to identify shortfalls in implementation and specify remedial actions where appropriate.

Annex. IEO High-Level Recommendations, Board Response, and IMF Follow-Up

This annex catalogues the high-level recommendations examined in the main text. For each recommendation, it also provides: excerpts from the Summing Up of the relevant Board meeting, indicating the Board response to that recommendation; the proposed follow-up actions as presented in the MIP, when applicable; and the status of follow-up as communicated in PMRs prepared by IMF staff. Much of the information in this annex is summarized in the main text; the annex reproduces verbatim text.

IEO RecommendationExecutive Board ResponseManagement Implementation Plan (with EVC/Board reactions to the MIP as indicated)Follow-Up as Presented in IMF Periodic Monitoring Report (with EVC/Board reactions to the PMR as indicated)
Financial Sector Assessment Program (2006)
Recommendation 2: To strengthen incentives and drawing upon these country-specific plans, IMF management should clearly signal to the Board those countries that it sees as the highest priorities for FSAPs and Updates, irrespective of whether these countries have volunteered. These lists should be the basis for periodic discussions by the Board of country-specific priorities.Summing Up of Board Discussion, January 27, 2006: [M]ost Directors agreed with the IEO proposal that management should indicate to the Board which countries it considers the highest priorities for FSAP assessments and updates. Annual reporting on country participation, as instituted following the 2005 internal FSAP review, could in this context provide useful information to guide the discussion of priority cases. In addition, most Directors considered that Article IV staff reports should explicitly recommend an initial FSAP or FSAP update in priority cases, although the way this was reported would have to be mindful of potential market sensitivities. Some Directors, however, cautioned against putting peer pressure on countries, as they judged that this runs counter to the voluntary nature of FSAPs. A number of Directors also pointed to the report’s finding that the burden of FSAPs on the authorities is high, and stressed that reducing this burden through better planning and focus is critical for increased participation.[This evaluation predates the MIP instrument.]PMR, December 2007: Scheduling of FSAPs is based on a prioritization process established by staff, based on criteria approved by the Fund and Bank Boards, which is carried out twice a year. Staff assessment of prioritization for individual countries is used to encourage participation of high priority countries in the program and is reflected in the staff appraisal of Article IV consultation reports. Annual reporting on country participation, as instituted following the 2005 internal FSAP review, is also being used to provide the Executive Board with overall information on program coverage (SM/07/272).
Recommendation 3: Strengthen the links between the FSAP and surveillance by mainstreaming FSAPs and follow-up work into the IMF’s regular surveillance activities. This means incorporating the assessment of financial sector standing and vulnerabilities into the overall macroeconomic assessment of the country in a way that fosters a greater understanding of stability; policy recommendations that are set in a coherent framework combining macroeconomic and financial sector analysis; more meaningful discussion of financial sector issues with authorities; and enhanced peer review discussion at the Board.Summing Up of Board Discussion, January 27, 2006:

Directors concurred with the IEO recommendation to strengthen links between FSAPs and surveillance. Specifically, they underscored the need to follow up on key vulnerabilities and gaps relevant for stability and macroeconomic developments and respective recommendations raised in FSAPs/FSAP Updates in country work, and to integrate such issues into Article IV surveillance reports. Directors agreed that each Financial Sector Stability Assessment should contain a short (1-2 page) section that summarizes in candid language the main macro-relevant findings of FSAPs and potential macroeconomic implications arising from key financial sector risks. Directors stressed that in cases where financial stability issues, including any potential global repercussions, are judged to be of high importance, they should be a major focus of Article IV consultations.
[This evaluation predates the MIP instrument.]PMR, December 2007: The internal review process of FSAP-related documents and Article IV reports has been strengthened to ensure that (i) the Executive Summary summarizes the main macrorelevant findings using candid language, (ii) the FSSAs clearly highlight and summarize macro-relevant findings, (iii) these findings are adequately reflected and incorporated in the analysis of Article IV reports.
Multilateral Surveillance (2006)
Recommendation 3: Improve the content and form of multilateral surveillance outputs through streamlining and more focus on key issues.Summing Up, March 24,2006: Directors observed that, to heighten the impact of multilateral surveillance outputs on the global policy debate, they could be better targeted to their core audience, streamlined, and focused on key issues. While most Directors considered that a major streamlining and focusing of the WEO are not necessary and would detract from the quality of the underlying analysis, Directors offered a number of useful suggestions for further consideration. On issues of content, some Directors supported the suggestion to integrate better financial and capital market issues in the WEO’s Chapter I. They called for more analytical treatment and discussion of exchange rate issues, with some Directors cautioning the staff to be mindful of market sensitivities in the public communication of such analyses. Several Directors also considered that greater use could be made of scenario analysis, with sharper messages for policymakers. . . .[This evaluation predates the MIP instrument.]PMR, December 2007: The World Economic Outlook (WEO) is placing even greater focus on key cross-country messages and further strengthening risk analysis. Work is ongoing to broaden post-WEO outreach on the key policy messages and to target issues of particular concern in specific regions. A new procedure has been implemented in 2007 by which quarterly WEO updates are published—twice a year—between the publications of the Spring and Fall WEO. These update the staff’s view of the global outlook on the basis of recent developments and provide greater continuity for the Fund’s surveillance and outreach. The GFSR now includes a “global risk map” to help make the staff’s overall judgment about global financial stability easier to convey. More public outreach is being coordinated with the External Relations Department to bring GFSR messages to a greater global audience, perhaps outside financial centers.
The IMF and Aid to Sub-Saharan Africa (2007)
Recommendation 1:

The Executive Board should reaffirm and/or clarify IMF policies on:
  • the underlying performance thresholds for the spending and absorption of additional aid,

  • the mobilization of aid,

  • alternative scenarios,

  • PSIA, and

  • pro-poor and pro-growth budget frameworks.

March 5,2007: Directors generally agreed with the report’s assessment that considerable scope remains for further improvements . . . and that any improvements in the Fund’s engagement in low-income countries should, in line with the MTS, continue to be focused on its core mandate. . . . In this context, most Directors confirmed that distributional policies generally lie outside the Fund’s core mandate and that poverty and social impact analysis (PSIA) should be conducted by other agencies in the context of the PRSP process, although others saw a continuing role for the Fund. Directors noted that PSIAs have not systematically supported PRGF program design, and emphasized the importance of improving Fund collaboration with development partners, in particular the World Bank, to take these issues into account when helping countries formulate their macroeconomic policies.

Directors confirmed the importance of accommodating higher aid flows through higher spending and net imports, provided that this would not jeopardize macroeconomic stability. They considered that this approach should continue to be implemented on a case-by-case basis and in the context of a multi-year strategy—with the general objective being to bring all low-income members to a situation in which aid can be fully absorbed and effectively spent. Directors concurred on the need for improved transparency and clear communications by the Fund on its stance regarding the use of aid, and on the trade-offs involved.

With regard to other aid-related issues noted in the report, Directors also offered a range of views. On the role of the Fund in developing alternative aid scenarios, many Directors indicated that, in the context of the PRSP, [IMF] staff should be available to prepare scenarios that illustrate the macroeconomic challenges of scaling up aid, including, in the view of some Directors, those based on estimates by others of additional resources needed for the MDGs when available. Most Directors emphasized, however, that the Fund’s role should be limited to assessing the consistency of additional aid flows with macroeconomic stability and the absorption capacity of the country, with more normative advice on the preparation of less likely aid scenarios falling outside the Fund’s mandate. On budget frameworks, Directors generally considered that the World Bank and other MDBs should be the lead agencies in providing advice related to expenditure composition issues.

Directors supported the report’s recommendation on the need for further clarification of Fund policy on several aid-related issues including the mobilization of aid, alternative scenarios, PSIA, and pro-poor budget frameworks. Directors asked the staff to come back with specific and costed proposals on how to clarify relevant policies and implement the report’s recommendations.
June 5, 2007: Fund policies relating to the handling of aid inflows: Board-endorsed recommendations in this area will be taken forward primarily in the context of two related sets of papers on program design and fiscal policy, both of which benefited from the discussions on the findings of the IEO Report:
  • The Role of the Fund in Managing Aid Inflows and Impact on the Design of Fund-supported Programs . . . will seek to clarify Fund policy on issues related to the spending and absorption of additional aid, the mobilization of aid, alternative scenarios and pro-poor and pro-growth budget frameworks.

  • Fiscal Policy Response to Scaled-up Aid (and accompanying background papers) . . . will provide the framework for the Fund’s approach to the fiscal challenges of increasing aid flows.





[Table 1:]
  • Follow-up to Malan Report

  • The Role of the Fund in the PRSP and Donor Coordination Processes

  • The IMF’s Communication Strategy

PMR, October 2008: The Board also discussed Fiscal Policy Response to Scaled-Up Aid and Role of the Fund in Aid Inflows and Impact on the Design of Fund-supported Programs, which examine Fund policies to handle aid inflows, including bringing all LIC members to the point where all aid can be carefully spent and absorbed without disrupting macroeconomic stability (SM/07/199 and SM/07/210, respectively). Directors noted that the papers provide important guidance for Fund engagement in LICs and stressed the need to integrate the proposals in the paper with other ongoing work in the Fund on LICs to have a comprehensive operational framework on LICs (BUFF/07/102). At the operational level, AFR is strengthening its analysis and policy advice in this area. In particular, the department has been providing macroeconomic assessments of scaled-up aid scenarios in a number of countries as a contribution to a UN-led MDG Africa Initiative, and a number of staff reports and selected issues paper provide assessments of the implications of scaled-up aid. [Footnote] Examples of such assessments are in the staff reports of Burkina Faso (EBS/07/153) and Rwanda (EBS/07/04) as well as the SIPs on Madagascar (SM/7/209) and Mozambique (SM/07/167) assess aid-scaling up scenarios.



[Table]

Policy has been clarified by the Board in this respect at the July 2007 discussion of program design and aid (SM/07/199, SM/07/210, BUFF/07/102) and at the October 2007 discussion of the IMF’s role in the PRS process and collaboration with donors (SM/07/334, BUFF/07/139 rev).

A chapeau paper on the “The Role of the Fund in LICs” was discussed by the Board in July 2008 (SM/08/170, BUFF/08/127).

As follow-up to the Malan Report, a Joint Management Action Plan (JMAP) to improve Bank-Fund collaboration—presented to the Board in October 2007 (SM/07/335)—is being implemented [and text adds] aiming at translating identified good-practice approaches into standard practices.
Recommendation 2: Management should establish transparent mechanisms for monitoring and evaluating the implementation of the clarified policy guidance.March 5, 2007: Directors welcomed the report’s recommendation to establish transparent mechanisms for monitoring and evaluating the implementation of the clarified policy guidance.June 5, 2007: Board endorsed recommendations in this area will be taken forward in the context of . . . the next review of the PRGF, currently scheduled for 2010. This review could be brought forward if necessary, though consideration would need to be given to the resource cost of such an acceleration.



Board discussion of MIP (June, 29, 2007): Directors underscored the importance of monitoring and evaluating the implementation of clarified policy guidance, and some noted that, whether or not the next review of the PRGF could be advanced to a date before 2010, a monitoring and evaluation framework (benchmarks) will be needed.
PMR, October 2008: A chapeau paper on the “The Role of the Fund in LICs” was discussed by the Board in July 2008 (SM/08/170, BUFF/08/127).



Third PMR, October 2009: In July 2009, the Executive Board approved wide-ranging modifications to upgrade concessional financing facilities for low-income countries (LICs), and the Extended Credit Facility (ECF) will succeed the PRGF. Directors agreed to review experience with the new facilities and financing framework within three years of the effective date of the decision.
IMF Exchange Rate Policy Advice (2007)
Recommendation 8: Incentives should be given to develop and implement guidance for the integration of spillovers into bilateral and regional surveillance.May 9,2007: Directors underscored the importance of better incorporating the analysis of policy spillovers into regional and bilateral surveillance and welcomed the initiatives recently taken in this area under the aegis of the Medium-Term Strategy.August 16,2007 [subsequent to June 15, 2007 Surveillance Decision]:



[Table] No new initiatives. Continued implementation of existing initiatives under the MTS.



[Text] The current initiatives under the aegis of the MTS will continue. The initiatives include a focus on overall regional trends through regional outlooks, better assessment of external economic and financial market spillovers affecting individual countries by drawing on the analysis in multilateral and regional surveillance, and spillovers emanating from systemic countries. The analysis of spillovers has benefited from increased use of the Fund’s Global Economic Model and Global Fiscal Model. These initiatives were welcomed by Directors, and no new initiatives are planned.

Board discussion of MIP (September 12, 2007): Some Directors also suggested that additional action might be warranted to better integrate spillovers into bilateral and regional surveillance. . . . The need for deeper analysis of the link between capital flows and exchange rates was also emphasized.
PMR, October 2008: This theme has received renewed emphasis in the refocusing process and in the TSR. The Managing Director’s Strategic Directions paper outlines some initiatives to integrate a multilateral perspective in bilateral surveillance.
Recommendation 6: To improve assessments of the exchange rate level, the IMF should be at the forefront of developing the needed analytical framework, while more successfully translating existing methodologies into advice that is relevant to discussion of individual country cases.Directors welcomed the finding that analysis of exchange rate levels had improved, although in several cases there remained scope for improvement in the quality of the analysis. Many noted positively that staff’s work in this area had been strengthened in recent years, with more sophisticated analytical tools being applied. . . . At the same time, Directors recognized that methodological uncertainties remain daunting in this area, and should not be discounted in Fund assessments of exchange rate levels. Directors generally agreed with the IEO that the Fund should stay at the forefront of developing the analytical framework in this area, including with respect to developing countries. Several Directors advocated caution in the Fund’s public communications on its findings on equilibrium exchange rates and misalignments, including those based on CGER assessments. In this context, a few Directors cautioned against over-reliance on model-based estimates of equilibrium exchange rates.[Table] II.3.

a. Expand and improve CGER work (including refine methodologies and expand it to key low income countries and producers of exhaustible resources).

b. Knowledge dissemination (see above).

c. Additional research in area departments, including cross-country work on oil producers in MCD.

d. Focus on issue in internal review process—increase the emphasis on assessments of exchange rate levels. This additional emphasis is already coming about as a result of the 2007 Surveillance Decision.

[Text]

Area departments are planning additional work on exchange rate assessments in the context of bilateral surveillance, using inputs from the CGER methodology, other quantitative methodologies tailored to individual countries, and qualitative analyses. Analytical work on the issue of exchange rate assessment for oil producers has been initiated in MCD.
PMR, October 2008: Priority in the CGER work program has been given to assessing the performance of past CGER predictions and improving the current methodology. Work is also underway on expanding CGER to LICs and producers of exhaustible resources. Preliminary versions of these methodologies are expected for the Spring of 2009.

Work on knowledge dissemination is proceeding and most of it is expected to be completed in the summer of 2008. This work has been designed to cover two areas:

—Information repository; housed in a dedicated web space and to contain relevant information on exchange rate analysis techniques (including templates), historical data sets, and good sample cases in Fund’s work (already available from PDR’s website).

—Training materials for CGER methodologies, exchange rate regime classification, and analysis of foreign exchange operations.

Work in area departments (through dedicated working groups) and PDR (real exchange rate benchmarks for oil exporting countries) is ongoing. Table A.2 has a list of recent Selected Issues papers on exchange rate-related topics.

Review process strengthened in the context of the implementation of the 2007 Decision. Guidance on expectations for exchange rate assessments in Article IV consultations was included in the guidance on operational aspects of the 2007 Decision.

Third PMR, December 2009: A Working Paper outlining a methodology to assess current account balances in exporters of non-renewable resources was issued as IMF WP 09/33. A second WP, which presents three CGER-type methodologies for exporters of nonrenewable resources, is forthcoming in the fall of 2009. The extension and adaptation of CGER methodologies to low income countries is also at an advanced stage. Preliminary background notes should be available for circulation in the fall of 2009, with an Occasional Paper ready by the end of the fiscal year.
Structural Conditionality in IMF-Supported Programs (2007)
Recommendation 1:

The Board should clarify what it expects in terms of numbers and focus of structural conditions. The findings of the evaluation suggest that the Board would be well advised to reaffirm the need to use structural conditionality sparingly and only when critical for achieving program objectives or safeguarding IMF resources.
December 12, 2007:

Most Directors expressed concern that the number of structural conditions did not decline significantly, and that some structural conditionality may have covered areas not critical to program goals. Many Directors recognized that it is important to consider not only the number but also the quality and coherence of structural conditions included in programs. Some Directors highlighted that criticality needs to be tailored to country-specific circumstances.

Directors broadly supported strengthened efforts to streamline conditionality, with parsimony as the guiding principle and a focus on measures critical to achieving program objectives.

[Rather than considering a notional cap on the number of structural conditions], the preferred way forward appears to be to strengthen efforts to achieve parsimony by focusing on criticality, and requiring rigorous justification for conditions. Better Board scrutiny of programs will also be important.
April 8, 2008:

[Table] 1.2. Under the 2002 Conditionality Guidelines (CG), program-related conditions will be established on all variables or measures that are (i) critical for achieving the goals of the program, (ii) critical for monitoring program implementation, and (iii) necessary for implementing specific provisions of the Articles of Agreement or policies adopted under them. The Staff Statement on the CG further explains that parsimony requires setting program-related conditions at the minimum necessary to achieve the above listed three categories. In effect this requires avoiding setting SC on reforms that might be desirable but are not critical for achieving the program goals.

The staff plans to:

• Propose changes to the Operational Guidance Note (OGN)—revised in January 2006—that would highlight the importance of a clear and thorough justification of the criticality of SC—covering to the extent possible the life of the program—in all initial staff reports. See item 2.1 below.

• Assess the need for additional changes to the OGN that would serve to highlight parsimony and criticality while minimizing subjectivity; this would include guidance on how to deal with donor-driven conditionality and structural conditions introduced at the request of country authorities (see also item 2.4 below).
Third PMR, December 2009:

The Operational Guidance Note (OGN) was revised in July 2008 to emphasize the need to apply rigorously the principles of parsimony and criticality when designing conditionality in Fund arrangements (SM/08/245 and Correction 1). The revised OGN is available on the Fund’s external web site. . . . An inter-departmental contact group established in February 2008 developed these OGN revisions. The OGN is being further revised to reflect the recent move towards a review-based structural conditionality framework.
Recommendation 4: The Fund should develop a monitoring and evaluation framework linking conditions in each program to reforms and specified goals. This would provide a more robust basis for monitoring the implementation and evaluation of programs, as well as facilitating learning on what works and what does not. Such a framework would allow staff to better define what data need to be collected before, during, and after a program.

[Related subsidiary recommendation: As an interim measure, the staff needs to improve the system used to track conditionality (MONA) with a view to disclosing these data and thus facilitating accountability as well as learning by authorities in member countries.]
December 12, 2007: Directors agreed that the link between program goals, strategies, and conditions should be better explained in Board papers—and that this should be monitored. In particular, several Directors proposed that initial program requests include a roadmap describing the sequencing and linkage of conditions to stated program goals; some Directors proposed that final program reviews should include a stock-taking to compare stated program goals with their achievement.April 8, 2008:

[MIP entry for Recommendation 4 reads: “Develop a monitoring and evaluation framework . . . and improve the system to track conditions (MONA) with a view to disclose this data.”]

[Table]

Establish framework within MONA to monitor the links between goals, reforms, and structural conditionality.



Issue annual updates (Board information) on the application of SC.



Make data in MONA available on the Fund external website—only for staff reports that are in the public domain.
Third PMR, December 2009:

[Table]

A system to link goals, reforms, and structural conditionality was established within the MONA database framework in July 2008.

[Text]

The first annual report on the application of structural conditionality, issued to the Board for information in July 2008 (SM/08/246) and made available on the Fund’s external web site, provides new data for the period 2005-07. It found that the overall number of conditions had remained unchanged relative to the findings of the IEO evaluation. However, these conditions had become more concentrated in the IMF’s core areas of expertise (including those shared with the World Bank). The second annual report will soon be issued to the Board for information and staff will recommend that it be made available on the Fund’s external website.

Third PMR—Assessment by the Evaluation Committee to the Executive Board, January 2010: With regard to the follow up on the MONA database, more work could be done on the system to track goals and strategies and its links to conditions.

Fourth PMR, March 2011: The third PMR reported that the MONA database had been launched on the Fund’s external web site in January 2009, and was updated in October 2009 to include a more up-to-date economic classification of structural conditions. Directors suggested that more work could be done on the system to track goals and strategies and its links to conditions. Since then, work has begun on the 2011 Conditionality Review, which is utilizing the MONA database to assess how well program conditions have been linked to program goals. As a part of conducting that review, staff will assess whether any further changes to the MONA database are needed.
Governance of the IMF (2008)
Recommendation 2: [T]he Fund needs more active and systematic ministerial-level involvement. . . . [T]he IMFC, as an advisory body[,] lacks a mandate for setting strategic directions and providing high-level oversight of the institution. . . . This could be achieved by activating the ministerial-level Council that is envisaged in the Articles of Agreement. . . . The IMFC/Council’s decision-making system should strive for consensus . . . and decisions on important issues should be subject to special majorities.IEO Evaluation Report

(Joint Statement by the Executive Board and the IMF Managing Director, May 27, 2008)

[T]he report has raised important questions [including] how to ensure effective ministerial and Board involvement in institutional decision-making processes.

Executive Board Report to the IMFC on Reform of Fund Governance (Oct. 2009) [W]hat to advise ministers on this issue has been controversial at the Board, with a few counseling activation, a number of others open to further deliberation (especially once quota reform progresses), and a somewhat larger group considering the proposal to be counter-productive.

Executive Board Report to the IMFC on Quota and Governance Reforms (Oct. 2010) Directors agreed that engagement by ministers and governors is essential to the effective discharge of the institution’s responsibilities, including to promote multilateral cooperation and coherence of policies. However, views on the best means of delivering such engagement—whether through reform of the advisory IMFC or a shift to a decision-making entity—continued to differ.
N/AN/A
Recommendation 4:

The Board should introduce an accountability framework for Management. Work is under way in this regard, and will need to provide clear proposals on performance criteria, on the processes to be used, and on how assessments are to be translated into incentives. The criteria should focus on Management’s conduct of the ordinary business of the Fund (including the chairmanship of the Board; consultations with authorities, Directors, and stakeholders; budget execution and financial management; and personnel and other administrative and managerial matters) and on the quality and outcomes of the Fund’s activities. To be effective, the evaluation of Management might need to be delegated to a Board committee that would canvass the views of all Directors, and that would inform the whole Board of its assessment once completed. The assessment may need to be confidential to avoid undermining the credibility of the MD vis-à-vis the membership at large.
Joint Statement by the Executive Board and the IMF Managing Director, May 27, 2008:

[T]he report has raised important questions [including] how to strengthen the management accountability framework, recognizing that work in that area is already underway.
July 2008 Report of the Executive Board Working Group on IMF Corporate Governance:

A Working Group on the Framework of the Managing Director’s Performance Evaluation chaired by the Dean is currently developing performance objectives that will be used to assess the MD’s performance. . . . The framework also envisages that the MD assesses the performance of the Executive Board. . . .

[Table]

WG to report to the Board by end-08/early 09.
IMF Involvement in International Trade Policy Issues (2009)
Recommendation 1: The Board should commit to periodic re-evaluation of its guidance on objectives of, approaches to, and modalities of staff work on trade policies. Regular reviews of guidance should be undertaken in the context of assessments of current global trends in trade and trade policy.June 8, 2009: Most Directors supported the IEO’s recommendation on the need for periodic Board review of guidance on trade policies [and] agreed that such discussions should be more focused than the 1994 Comprehensive Trade Paper and, in line with the cycle for other reviews, might be done at five-year intervals.November 2009:

[Table]

A1. Five-yearly reviews of Fund work on trade policy will be conducted. As the IEO Evaluation covers much the same ground as staff reviews, an early review of trade policy by staff would be duplicative. Taking the IEO Evaluation as the starting point for the cycle, the first five-yearly review would be expected in 2014.
[T]he new reference notes include guidance for staff on designing effective trade reforms and assessing progress in trade liberalization for occasions when attention to trade reform is appropriate. Staff observance of this guidance would be examined in the five-yearly reviews of trade policy (first one expected in 2014).

The author would like to thank Hali Edison, Sanjay Dhar, Alisa Abrams, and Roxana Pedraglio for their contributions to this study and helpful comments on the chapter. The author is also grateful to Arun Bhatnagar, Annette Canizares, and Mari Lantin for administrative assistance.

A variety of factors drive change at the IMF. Consistent with this fact, we see the IEO as a contributor to change but do not attribute any particular change in policy or practice solely to IEO evaluations or recommendations.

MIPs were issued for the evaluation of The IMF and Aid to Sub-Saharan Africa and subsequent evaluation reports in the sample except for the Governance evaluation, for which a different approach was agreed (see discussion below). The evaluations of Multilateral Surveillance and the FSAP preceded the adoption of the follow-up framework. For several evaluations covered by the present review, Management/staff established “key performance benchmarks” as part of the reporting on progress. However, often these benchmarks did not correspond one-to-one with recommendations endorsed by the Board; or they addressed only part of a recommendation; and/or they focused on actions to be taken, such as preparation of a policy paper, without reference to whether the paper yielded the result sought by the Board.

PMRs were also introduced as part of a follow-up framework agreed by the Executive Board following the 2006 external evaluation of the IEO. PMRs were created to report on the implementation status of recommendations “contained in the forward-looking implementation plans already in force and not deemed completed on the occasion of a prior periodic monitoring report” (EBAP 07/4). PMRs were also intended to indicate difficulties in implementing the original plan and to propose remedial or substitute actions whenever appropriate. The first PMR (2007) summarized the status of key recommendations for the 10 IEO evaluations completed by that time. Three subsequent PMRs (dated October 2008, October 2009, and March 2011) each reported on implementation of Board-endorsed recommendations covered in MIPs approved since the last PMR and of any recommendations or benchmarks identified in the previous PMR as outstanding.

Salop, Chapter 8 in this volume, includes a discussion of the distribution of high-level and subsidiary recommendations across the 18 evaluations. The chapter also discusses the variation in number of recommendations across evaluations and the relationship between high-level and subsidiary recommendations.

These figures exclude the 25 subsidiary recommendations made in the Governance evaluation.

The evaluation reports are available on the IEO website at www.ieo-imf.org/ieo/pages/Completed.aspx.

In a few cases, key recommendations were expressed as subsidiary to broader recommendations—for instance, the alignment of communications with Fund policy—and thus while essential to addressing IEO’s conclusions were not counted as high-level recommendations for the purposes of this review.

In some instances, IMF staff had linked particular actions to certain recommendations, but the IEO judged that these actions were not directly relevant.

The annex documents for each recommendation: the Board’s views as communicated in the Summing Up, the proposed actions as presented in the Management Implementation Plan, and IMF staff’s account of the status of implementation as reported in annual PMRs.

The Financial Sector Assessment Program (FSAP), established in 1999, provides for a comprehensive and in-depth analysis of a country’s financial sector. FSAP assessments are the joint responsibility of the IMF and World Bank in developing and emerging market countries and of the Fund alone in advanced economies. They have two major components: a financial stability assessment, which is the responsibility of the Fund and, in developing and emerging market countries, a financial development assessment, which is the responsibility of the World Bank.

IEO, Financial Sector Assessment Program, 2006, p. 39.

This reflected an IEO finding that while the FSAP had helped broaden the degree to which financial sector issues were monitored as part of surveillance, follow-up on financial sector issues through surveillance in the years following a financial sector assessment tended to diminish—and thus the IMF was “not yet using FSAP results as effectively as it could.” IEO, Financial Sector Assessment Program, 2006, p. 39.

“IMF Board Discusses Modernizing the Surveillance Mandate and Modalities and Financial Sector Surveillance and the Mandate of the Fund,” Public Information Notice No. 10/52, April 22, 2010.

Olivier Blanchard, “The Crisis: Basic Mechanisms and Appropriate Policies,” IMF Working Paper No. 09/80 (Washington: International Monetary Fund, 2009).

“2011 Triennial Surveillance Review—Overview Paper,” August 29, 2011, p. 17. More specifically, the staff background study for the 2011 TSR found that “FSAPs are infrequent, are not being incorporated into Article IV reports systematically, and questions have been raised on how to guarantee the quality of financial stability analysis in bilateral surveillance,” “2011 Triennial Surveillance Review—Staff Background Studies,” August 26, 2011, p. 34.

See “Implementing the Fund’s Medium-Term Strategy—Working Group Reports,” SM/06/114, March 20, 2006; and “The Managing Director’s Report on Implementing the Fund’s Medium-Term Strategy,” April 6, 2006.

Operational priorities were also discussed in the context of the 2011 Triennial Surveillance Review.

“2008 Triennial Surveillance Review—Overview Paper,” p. 4.

“Some of the risks that subsequently materialized were identified at different times in the Global Financial Stability Report, but these were presented in general terms, without an assessment of the scale of the problems, and were undermined by the accompanying sanguine overall outlook. These risks were not reflected in the World Economic Outlook or in the IMF’s public declarations.” IMF Performance in the Run-Up to the Financial and Economic Crisis, 2011, p. 7.

See Stephen Pickford, “TSR External Study—IMF Surveillance: Coverage, Consistency, and Coherence,” July 20, 2011.

Nonetheless, the “2011 Triennial Surveillance Review—Overview Paper” concluded that a diversity of stakeholders held multilateral surveillance outputs in high esteem and thus it did not recommend a “radical merger” of products (p. 25).

SM/09/55. With respect to the subsidiary recommendation that clear guidance be provided to staff, the Strategy and Policy Review Department website referred staff to a number of policy papers and Summings Up until the issuance of a Handbook of IMF Facilities for Low-Income Countries on March 21, 2012.

See “A New Architecture of Facilities for Low-Income Countries,” June 26, 2009, paragraph 37, pp. 18–19.

See Annex, pp. 6–7.

A review of lending to LICs that is scheduled for 2012 may serve the function of evaluating implementation of the clarified policy guidance.

“Executive Board Assessment,” Public Information Notice No. 07/119.

“Periodic Monitoring Report on the Status of Implementation Plans in Response to Board-Endorsed IEO Recommendations,” October 6, 2008, Table A.1, p. 19.

“Statement by the Managing Director on Strategic Directions in the Medium-Term Budget,” April 12, 2008.

“2011 Triennial Surveillance Review—Overview Paper,” p. 10.

Subsequently, the Board decided to eliminate structural performance criteria from IMF-supported programs. At the same time, programs could continue to include structural benchmarks towards which borrowing countries are expected to make progress during the implementation of the program.

Although the initial annual report on the Application of Structural Conditionality (ARSC) in 2008 found no change, the March 2010 ARSC reported a subsequent decline in the average number of structural conditions associated with a program review. No ARSC was issued in 2011, but there is evidence that the decline may not have been sustained in that year. IMF staff have indicated that this issue will be considered by the forthcoming conditionality review.

The Committee was expected to report back one year later, but this did not occur.

Authority for the establishment of the Council, a ministerial-level decision-making body, is enshrined in the Fund’s Articles of Agreement (Article XII, Section 1). Activation of the Council would require support by an 85 percent majority of the total voting power. This level of support has been elusive and no discussions on its activation are currently taking place. See Alisa Abrams, “The IMF Council of Governors,” Chapter 3 in Studies in IMF Governance: A Compendium, edited by Ruben Lamdany and Leonardo Martinez-Diaz (IEO, 2009).

“G-20 Leaders’ Statement,” The Pittsburgh Summit, September 24–25, 2009, paragraph 21.

“Executive Board Report to the IMFC on Reform of Fund Governance,” October 3, 2009; “Executive Board Report to the IMFC on Quota and Governance Reforms,” October 1, 2010.

“The IMF Board Discusses IMF Governance Reform,” Public Information Notice No. 10/108, August 2, 2010.

See “Terms of Appointment of Christine Lagarde as Managing Director of the International Monetary Fund,” Press Release No. 11/270, July 5, 2011, available at www.imf.org/external/np/sec/pr/2011/pr11270.htm.

“Report of the External Evaluation of the Independent Evaluation Office” (Lissakers Report), March 2006, available at www.ieo-imf.org/ieo/files/evaluationofieo/032906.pdf.

“All key performance benchmarks related to the MIPs covered in this report have either been met or are on track for timely completion, and no new remedial actions are proposed. There are no outstanding benchmarks to be reviewed in the next PMR.” “Third Periodic Monitoring Report on the Status of Implementation Plans in Response to Board-Endorsed IEO Recommendations,” October 7, 2009, paragraph 7.

The IMF press release for the Board discussion of the Third PMR states that “some aspects of the implementation plans are still being executed, as Board-endorsed IEO recommendations continue to be integrated into the Fund’s ongoing work program.” Public Information Notice No. 10/23, February 17, 2010.

“External Evaluation of the Independent Evaluation Office—Follow-Up,” EBAP/07/4, January 10, 2007.

The discussion in the Third PMR noted that “most” of the outstanding recommendations from previous PMRs had been fully addressed while noting that action remained to be taken on some. But its overall conclusion was that there were no outstanding benchmarks for review in the next PMR.

    Other Resources Citing This Publication