Title Page
IEO
Independent Evaluation Office
of the International Monetary Fund
EVALUATION REPORT
Recurring Issues from a Decade of Evaluation
Lessons for the IMF
Copyright
© 2014 International Monetary Fund
Cataloging-in-Publication Data
Joint Bank-Fund Library
Recurring issues from a decade of evaluation: lessons for the IMF / prepared by an IEO team led by Shinji Takagi; the IEO team included Roxana Pedraglio and Jérôme Prieur. - Washington, D.C.: International Monetary Fund, 2014.
p.; cm.
At head of title: IEO—Independent Evaluation Office of the International
Monetary Fund.
Includes bibliographical references.
ISBN: 978-1-48437-685-0
1. International Monetary Fund. Independent Evaluation Office—Evaluation.
I. Takagi, Shinji, 1953–. II. Pedraglio, Roxana. III. Prieur, Jérôme. IV. International Monetary Fund.
HG3881.5.I58 R43 2014
Please send orders to:
International Monetary Fund, Publication Services
P.O. Box 92780, Washington, DC 20090, U.S.A.
Tel.: (202) 623-7430 Fax: (202) 623-7201
E-mail: publications@imf.org
Internet: www.elibrary.imf.org
Contents
Foreword
Abbreviations
Executive Summary
1 Introduction
2 The Evaluation Framework
A. Identifying Issues for the Evaluation
B. Evaluation Questions
C. Sources of Evidence
3 Recurring Issues from IEO Evaluations
A. Executive Board Guidance and Oversight
B. Organizational Silos
C. Attention to Risks and Uncertainty
D. Country and Institutional Context
E. Evenhandedness
4 Conclusions and Issues for Board Consideration
A. Conclusions
B. Issues for Board Consideration
Boxes
1. Selected Recent Innovations in Executive Board Procedures and Practices
2. Selected Views of Executive Board Members on Executive Board Oversight
3. Selected Views of Executive Board Members on Evenhandedness
4. Typology of IEO Findings on Evenhandedness
Annexes
1. IEO Findings Related to Recurring Issues
2. Selected IMF Decisions and Initiatives Related to Recurring Issues, 2008–13
Sources Consulted
Statement by the Managing Director and the Chairman’s Summing up
Statement by the Managing Director
The Chairman’s Summing Up
Foreword
The Independent Evaluation Office (IEO) was established in 2001 to provide objective and independent evaluation on issues related to the IMF. During the first 12 years of its existence, it produced 20 evaluation reports on subjects covering various aspects of the IMF’s core areas of responsibility: surveillance, lending, and capacity building. In 2013, an external committee headed by José Antonio Ocampo, noting that IEO recommendations “deemed by the Fund to have been met… tend to be raised again in subsequent IEO reports,” proposed that the IEO prepare a review of “generic and substantive issues” that are not “encapsulated in specific recommendations” but deserve monitoring. This evaluation was prepared as a response to this proposal. It identifies major recurring issues from the IEO’s first 20 evaluations and assesses where they stand.
The IEO found a number of issues that had recurred across a wide range of contexts. Of these, this evaluation has focused on those issues that have been identified most frequently in past IEO evaluations, namely:
Executive Board guidance and oversight;
Organizational silos;
Attention to risks and uncertainty;
Country and institutional context; and
Evenhandedness.
The evaluation finds that though the IMF Board and Management have taken actions to address each of the five sets of issues, challenges remain in each. To varying degrees, these challenges all emanate from the IMF’s character as a multilateral institution with multiple objectives and a complex governance structure. Despite the difficulty, efforts to address these issues are important for enhancing the IMF’s effectiveness and credibility. The report thus concludes that more can and should be done, especially in terms of broad-based, strategic responses.
The IEO believes that a framework of reviewing and monitoring recurring issues would be useful in establishing incentives for improvement, strengthening the Board’s oversight, and providing learning opportunities for the IMF. In this respect, I am encouraged by the broad agreement, expressed by the Managing Director and the Executive Board, with the findings of this report. It is hoped that this report, along with systematic efforts to address the issues identified therein, will contribute to enhancing the IMF’s effectiveness in performing its important mission for the global economy.
Moises J. Schwartz
Director
Independent Evaluation Office
Recurring Issues from a Decade of Evaluation: Lessons for the IMF
This report was prepared by an IEO team led by Shinji Takagi. The IEO team included Roxana Pedraglio and Jérôme Prieur. The evaluation benefited from discussions with the authors of all past IEO evaluation reports and participants at internal workshops. It incorporates comments by IMF staff on an earlier version, though the final judgments are the responsibility of the IEO alone. Arun Bhatnagar and Annette Canizares provided administrative assistance. Rachel Weaving, Roxana Pedraglio, and Esha Ray provided editorial and production management assistance. The report was approved by Moises Schwartz.
Abbreviations
| AFR | African Department (IMF) |
| APD | Asia and Pacific Department (IMF) |
| DSA | debt sustainability analysis (IMF) |
| EMDCs | emerging market and developing country economies |
| EPA | ex post assessment (IMF) |
| EPE | ex post evaluation (IMF) |
| ESF | Exogenous Shocks Facility (IMF) |
| EU | European Union |
| EWE | Early Warning Exercise (IMF/FSB) |
| FAD | Fiscal Affairs Department (IMF) |
| FDMD | First Deputy Managing Director (IMF) |
| FSAP | Financial Sector Assessment Program (IMF/World Bank) |
| FSB | Financial Stability Board |
| FSF | Financial Stability Forum |
| FSG | Financial Sector Surveillance Group (IMF) |
| FSI | Financial Soundness Indicator (IMF) |
| FSS | financial sector surveillance (IMF) |
| FSSA | Financial Sector Stability Assessment (IMF) |
| FY | Fiscal or financial year |
| G20 | Group of Twenty |
| GFSR | Global Financial Stability Report (IMF) |
| G-RAM | Global Risk Assessment Matrix (IMF) |
| HRD | Human Resources Department (IMF) |
| ICD | Institute for Capacity Development (IMF) |
| ICM | International Capital Markets Department (IMF) |
| IEO | Independent Evaluation Office (IMF) |
| IMF | International Monetary Fund |
| IMFC | International Monetary and Financial Committee (IMF) |
| INS | IMF Institute (IMF) |
| JSA | Joint Staff Assessment (IMF/World Bank) |
| JSAN | Joint Staff Advisory Note (IMF/World Bank) |
| LIC | low-income country |
| LTPE | longer-term program engagement (IMF) |
| MAE | Monetary and Exchange Affairs Department (IMF) |
| MAP | Mutual Assessment Program (IMF/G20) |
| MCD | Middle East and Central Asia Department (IMF) |
| MCM | Monetary and Capital Markets Department (IMF) |
| MFD | Monetary and Financial Systems Department (IMF) |
| MIP | Management Implementation Plan (IMF, IEO-related) |
| OED | Offices of Executive Directors (IMF) |
| OTM | Office of Technical Assistance Management (IMF) |
| PMR | Periodic Monitoring Report (IMF, IEO-related) |
| PRGF | Poverty Reduction and Growth Facility (IMF) |
| PRGT | Poverty Reduction and Growth Trust (IMF) |
| PRS | Poverty Reduction Strategy (IMF/World Bank) |
| PRSP | Poverty Reduction Strategy Paper (IMF/World Bank) |
| PTA | preferential trade agreement |
| RAM | Risk Assessment Matrix (IMF) |
| RAP | Resource Allocation Plan (IMF) |
| RES | Research Department (IMF) |
| RSN | Regional Strategy Note (IMF) |
| SPR | Strategy, Policy, and Review Department (IMF) |
| SSP | Statement of Surveillance Priorities (IMF) |
| TA | technical assistance |
| TGS | Technology and General Services Department (IMF) |
| TOR | terms of reference |
| TSR | Triennial Surveillance Review (IMF) |
| UFR | use of Fund resources (IMF) |
| VE | Vulnerability Exercise (IMF) |
| WEO | World Economic Outlook (IMF) |
| WTO | World Trade Organization |
The following conventions are used in this publication:
An en dash (–) between years or months (for example, 2008–09 or January–June) indicates the years or months covered, including the beginning and ending years or months; a slash or virgule (/) between years or months (for example, 2008/09) indicates a fiscal or financial year, as does the abbreviation FY (for example, FY2009).
“Billion” means a thousand million; “trillion” means a thousand billion.
As used in this publication, the term “country” does not in all cases refer to a territorial entity that is a state as understood by international law and practice. As used here, the term also covers some territorial entities that are not states but for which statistical data are maintained on a separate and independent basis.
Some of the documents cited and referenced in this report were not available to the public at the time of publication of this report. Under the current policy on public access to the IMF’s archives, some of these documents will become available 3 or 5 years after their issuance. They may be referenced as EBS/YY/NN and SM/YY/NN, where EBS and SM indicate the series and YY indicates the year of issue. Certain other types of documents may become available 20 years after their issuance. For further information, see www.imf.org/external/np/arc/eng/archive.htm.
Executive Summary
This evaluation seeks to help the IMF enhance its effectiveness by identifying major recurring issues from the IEO’s first 20 evaluations and assessing where they stand. These issues have affected the IMF’s performance in all of its core areas of responsibility: surveillance, lending, and capacity development. Their recurrence in different contexts in multiple IEO evaluations suggests that they are intrinsic to the nature of the institution, with deep roots in its culture, policies, and governance arrangements.
The evaluation has been prepared in response to the 2013 External Evaluation of the IEO, which proposed that the IEO prepare a review of “generic and substantive issues” that are not “encapsulated in specific recommendations” but deserve monitoring. The External Evaluation made this proposal as a way to strengthen the follow-up process for Board-endorsed IEO recommendations, which in its authors’ view had become a “box-ticking” exercise that tended to dilute their substance (Ocampo, Pickford, and Rustomjee, 2013, pp. 23–24, 26). This report aims to contribute to strengthening the follow-up process by focusing on key issues that have recurred in IEO evaluations, rather than on specific recommendations and their implementation.
The present evaluation focuses on recurring issues in the following five areas:
Executive Board guidance and oversight;
Organizational silos;
Attention to risks and uncertainty;
Country and institutional context; and
Evenhandedness.
The evaluation finds that though the Board and Management have taken actions to address each of the five sets of issues, challenges remain in each, and are likely to persist. To varying degrees, these challenges all emanate from the IMF’s character as a multilateral institution with multiple objectives and a complex governance structure. Despite their difficulty, efforts to address these issues are important for enhancing the IMF’s effectiveness and credibility. More can and should be done, especially in terms of broad-based, strategic responses.
Issues for Board Consideration
The recurring issues identified by the evaluation in five areas—(i) Executive Board guidance and oversight, (ii) organizational silos, (iii) attention to risks and uncertainty, (iv) country and institutional context, and (v) evenhandedness—are to varying degrees inherent to the nature of the IMF and are thus likely to present ongoing challenges for the institution. This raises the question of how best to address them, going forward, in view of the IMF’s overall institutional priorities and resource constraints. Despite their long-term nature, the IMF should try to mitigate their adverse impact while keeping these issues at the forefront of its agenda.
This evaluation, given its nature as a stock-taking exercise and in keeping with the suggestion of the 2013 External Evaluation of the IEO, does not propose specific recommendations on how to address the five sets of issues reviewed in the report. Nonetheless, after preparing this evaluation, the IEO believes that a framework of reviewing and monitoring recurring issues would be useful in establishing incentives for progress, strengthening the Board’s oversight, and providing learning opportunities for the IMF.
In light of this conclusion, the IEO recommends that the following reports be prepared for the Board periodically:
An IEO report, similar to this one, identifying and reviewing important issues that have recurred in its evaluations. This could be done every five years.
A status report, prepared by staff, to monitor the progress the IMF has made in addressing recurring issues, focusing on the big picture rather than on the implementation of specific IEO recommendations that will continue to be monitored via the Periodic Monitoring Report (PMR). The first staff report could be prepared within two years, followed by similar reports every five years thereafter.