IEO
Independent Evaluation Office
of the International Monetary Fund
EVALUATION REPORT
Recurring Issues from a Decade of Evaluation
Lessons for the IMF
Annual Report 2014
© 2014 International Monetary Fund
Cover: IMF Multimedia Services
ISBN: 978-1-49835-089-1
Publication orders may be placed online, by fax, or through the mail:
International Monetary Fund, Publication Services
P.O. Box 92780, Washington, D.C. 20090, U.S.A.
Tel.: (202) 623-7430 Fax: (202) 623-7201
E-mail: publications@imf.org
Contents
-
Message from the Director
-
1 Overview
-
Budget and Staffing
-
Outreach and Communication Activities
-
-
2 IEO Outputs in FY2014
-
IMF Forecasts: Process, Quality, and Country Perspectives
-
Recurring Issues from a Decade of Evaluation: Lessons for the IMF
-
IMF Technical Assistance: Revisiting the 2005 IEO Evaluation
-
-
3 Enhancing the Follow-Up Process for IEO Evaluations
-
Implementation Plans and Monitoring Reports
-
Implementing the Recommendations of the Second External Evaluation of the IEO
-
Issues-Oriented Review
-
Revisiting Past IEO Evaluations
-
-
4 Looking Ahead: Ongoing Evaluations and the IEO Work Program
-
Work in Progress
-
Future Work Program
-
-
Table
-
1 Completed and Ongoing IEO Work Program
-
-
Appendices
-
1 Administrative Budget: Independent Evaluation Office
-
2 Outreach Activities
-
Message from the Director
I am pleased to present the eleventh Annual Report of the Independent Evaluation Office (IEO) describing IEO activities during financial year 2014, that is, from May 1, 2013, to April 30, 2014.
This year, the IEO completed two evaluations: an evaluation of IMF forecasts, which was discussed by the Executive Board in February, and an evaluation identifying and analyzing recurring issues in past IEO evaluations, which is scheduled for discussion early in FY2015. The IEO also issued an update of the 2005 evaluation of IMF technical assistance, and separately published two updates on prolonged use of IMF resources and on fiscal adjustment in IMF-supported programs that were initially issued in last year’s Annual Report.
The evaluation of IMF Forecasts: Process, Quality, and Country Perspectives found that forecasts prepared by IMF staff were generally of good quality and that country perceptions about the forecasts were broadly positive. There were no significant biases in general except during certain episodes. Specifically, the evaluation found a tendency for significant over-predictions of GDP growth in the World Economic Outlook during regional or global recessions, as well as during crises in individual countries. It also found that short-term forecasts of GDP growth and inflation made in the context of IMF-supported programs tended to be optimistic in high-profile cases characterized by exceptional access to IMF resources and that at the first program review forecast biases were typically reduced or reversed. The evaluation recommended that the IMF take steps to promote learning from past forecasting performance within the institution and to enhance transparency by improving public access to its forecasting process and historical forecasts. I am encouraged by the broad agreement expressed by the Executive Board and Managing Director with the conclusions and recommendations of the evaluation. A full summary of the evaluation report and the Executive Board discussion is provided in Chapter 2 of this Annual Report.
I am also encouraged by the decisions the Executive Board has taken this year to enhance the process of follow-up on IEO evaluation findings and recommendations, and monitoring thereof. These have been areas of concern for some time. As discussed in last year’s Annual Report, the January 2013 external evaluation of the IEO also pointed to a need to enhance follow-up on IEO evaluations. The external evaluation noted that the follow-up process for Board-approved IEO recommendations had become very bureaucratic, lacked strong ownership by the Board and its Evaluation Committee, and involved a conflict of interest for Management. In February 2014, the Board approved staff proposals to address this issue, including steps to facilitate a more accurate recording of the outcome of Board discussions of IEO evaluations and to strengthen monitoring of the implementation of Board-endorsed IEO recommendations. Chapter 3 provides further details on these changes.
The IEO has implemented two initiatives to facilitate follow up on its evaluations: we completed a meta-evaluation of high-level recurring issues from a decade of IEO evaluations; and we continued with a new series of updates of past evaluations.
The first initiative aims to address the external evaluation’s concern that the broader, more policy-oriented recommendations of our evaluations tend to be missed by the current follow-up process. The resulting report, Recurring Issues from a Decade of Evaluation: Lessons for the IMF, reviews generic and substantive issues that have recurred in the 20 evaluation reports prepared since the establishment of the IEO. It identified five such recurring issues related to: Executive Board guidance and oversight; organizational silos; attention to risks and uncertainty; country and institutional context; and evenhandedness. We found that actions had been taken to address these issues, but that challenges are likely to persist as these issues are inherent to the IMF’s character as a multilateral institution with multiple objectives and a complex governance structure. These are cross-cutting issues that are vital to the IMF’s effectiveness and credibility. They permeate many IMF activities and different aspects of the institution, and therefore cannot be readily addressed by specific recommendations. The findings of this evaluation, which will be discussed by the Executive Board in early FY2015, are presented in Chapter 2 of this report.
The second initiative, which was introduced via a pilot project in FY2013, involves IEO updates of prior evaluation reports. These updates review past evaluations to assess the current status and relevance of original key findings and recommendations. This was an area of interest in the external evaluation. The first two reports revisited, respectively, the 2002 evaluation of prolonged use of IMF resources and the 2003 evaluation of fiscal adjustment in IMF-supported programs, and were included in last year’s Annual Report. In April 2014, we published the third report in this series, IMF Technical Assistance: Revisiting the 2005 IEO Evaluation. A summary of this report is provided in Chapter 2. Chapter 3 discusses how this new series of reports and the Recurring Issues report link to the external evaluation’s recommendations for strengthening the follow-up process.
The IEO has a full agenda for the year ahead, including ongoing evaluations on IMF self-evaluation, the IMF’s response to the global financial crisis, and statistics, as well as the launch of a new evaluation on the IMF and the euro area crisis. Two more updates in the pipeline revisit the 2004 evaluation of the IMF’s role in Poverty Reduction Strategy Papers and the Poverty Reduction and Growth Facility and the 2007 evaluation of the IMF and aid to Sub-Saharan Africa jointly and the 2005 evaluation of the IMF’s approach to capital account liberalization. These ongoing and future projects are discussed in Chapter 4.
I look forward to a productive year for the IEO.
Moises J. Schwartz
Director Independent Evaluation Office