Chapter 4. Follow-Up on IEO Evaluations
- International Monetary Fund. Independent Evaluation Office
- Published Date:
- August 2013
An approach to follow-up on IEO evaluations was established in the IMF following the first external evaluation of the IEO in 2006. Under this framework, after the Executive Board discusses an IEO evaluation report, the IMF prepares a forward-looking Management Implementation Plan (MIP) for each evaluation to implement IEO recommendations endorsed by the Executive Board. Implementation status is then assessed in a Periodic Monitoring Report (PMR) prepared annually by IMF staff for Board consideration.
This chapter reports on the execution of this follow-up process in FY2013, including approval of three Management Implementation Plans and of the Fifth Periodic Monitoring Report. The chapter goes on to discuss continued concerns about the process itself, including issues raised by the External Evaluation Report and by Executive Directors, as well as the IEO’s own perspectives on this issue.
Forward-Looking Implementation Plans
Three MIPs were approved during the financial year, for the evaluations of: IMF Performance in the Run-Up to the Financial and Economic Crisis, Research at the IMF: Relevance and Utilization, and International Reserves: IMF Concerns and Country Perspectives. Each MIP is available on both the IEO and IMF websites. In addition, a proposed MIP for the Trusted Advisor evaluation will be considered by the Board in early FY2014.
International Reserves: IMF Concerns and Country Perspectives
IMF staff issued a proposed MIP for the International Reserves evaluation on March 4, 2013. Following a discussion by the Evaluation Committee, the Board agreed to the plan on March 29, 2013. The MIP outlines plans for further IMF staff work on reserve adequacy, including a review of recent experience with the reserve adequacy metric, and for stepped-up engagement on reserve adequacy, both in dedicated missions and as part of the regular surveillance cycle. The MIP notes that several recent surveillance and policy initiatives also contributed to addressing IEO recommendations in the International Reserves evaluation; these include the recently updated bilateral surveillance guidance, External Sector Report, spillover reports, and steps to integrate policy analysis and advice.
IMF Research: Relevance and Utilization
The IMF Executive Board discussed the evaluation of Research at the IMF: Relevance and Utilization on June 13, 2011. IMF staff circulated a proposed MIP to the Evaluation Committee on June 18, 2012. Following a discussion by the Evaluation Committee, the Board agreed to the MIP on November 27, 2012. Among other actions, the MIP calls for a strategic review of research across the institution within about 18 months.
IMF Performance in the Run-Up to the Financial and Economic Crisis: IMF Surveillance in 2004–07
The MIP for this evaluation was discussed and approved by the Executive Board on May 25, 2012. In addition to the MIP, the Managing Director also issued a statement for the Board discussion describing recent initiatives undertaken to address cultural and structural weaknesses identified in the IEO evaluation. Directors generally considered that the Managing Director’s statement and the proposed implementation plan complemented well the action plan previously issued for the Triennial Surveillance Review, and together should help enhance the effectiveness of IMF surveillance.
Directors emphasized that a comprehensive long-term approach is needed to tackle the shortcomings highlighted in the IEO report. They broadly supported the specific proposals in the implementation plan and welcomed Management’s statement on an ambitious agenda to break down silos and promote diverse views and candor, further advancing initiatives under way. Directors considered that the MIP, together with Management’s proposed agenda, provided a good start and encouraged Management and staff to continue to build on them, and where appropriate, engage the Board in the process.
Directors focused their discussion in particular on actions to: encourage candor and diverse views; speak up to power; integrate financial sector issues into macroeconomic assessments; break down silos; and deliver a clear and consistent message on the economic outlook and risks.
Directors stressed the importance of monitoring and verifying progress on all these fronts in the context of future PMRs. They noted that the changing institutional culture is a continuous, long-term process and looked forward to revisiting this issue in one year. The Managing Director also recognized that some Directors believed that the incremental steps already under way and proposed may still not fully address remaining concerns or more fundamental problems—including governance reforms. She pledged to monitor progress and adapt over time.
Monitoring the Implementation of Board-Endorsed IEO Recommendations
The Fifth PMR was issued by staff in September 2012 and discussed by the Evaluation Committee on February 7, 2013. This PMR focused on implementation of Board-endorsed recommendations of the evaluation of IMF Interactions with Member Countries and concluded that key performance benchmarks had been met or were on track for timely completion. It also determined that there were no outstanding performance benchmarks from other evaluations to be reviewed in the next PMR. In keeping with Directors’ previous requests to monitor progress more closely, the PMR also included information on implementation of Board-endorsed IEO recommendations for four previous evaluations: The IMF and Aid to Sub-Saharan Africa, IMF Exchange Rate Policy Advice, Structural Conditionality in IMF-Supported Programs, and IMF Involvement in International Trade Policy Issues.
The Evaluation Committee broadly endorsed the Fifth PMR’s assessment and recommended Board approval of its conclusions. At the same time, the Committee called for better monitoring of staff turnover in mission teams and mission chiefs, with more data needed on staff tenure and on the recruitment of mid-career professionals, and agreed that these issues should be considered in future strategic human resources discussions in the Executive Board. Regarding follow-up on past IEO recommendations, the Committee noted the need for sustained effort given the broad nature of these recommendations, which often entail a change in institutional culture. Committee members considered that monitoring by the Board would benefit from more regular PMRs.
The Board approved the Fifth PMR on February 25, 2013, based on the Committee’s recommendation. The PMR, the Evaluation Committee Assessment, and the related IMF Public Information Notice are available on both the IEO and the IMF websites.
Ongoing Concerns About the Follow-Up Process
As discussed in Chapter 3, the second external evaluation of the IEO identified major concerns about the process for follow-up on IEO evaluations.
On the one hand, the external evaluation found that IEO evaluations made an important contribution to the effectiveness and credibility of the IMF. IEO analysis published in Independent Evaluation at the IMF: The First Decade confirmed that IMF has taken steps toward implementing many of IEO recommendations endorsed by the Board. Further, IEO evaluations provide input for broader reviews and work streams in the IMF, such as the 2011 Triennial Surveillance Review and 2012 Conditionality Review.
At the same time, the external evaluation report pointed to fundamental weaknesses in the follow-up process, including a lack of strong ownership by the Board, conflicts of interest for Management, the absence of a mechanism for following up on broader policy and/or cultural lessons and recommendations, and the bureaucratic nature of monitoring to date. In addition, the report noted that the current structure of the follow-up process tends to dilute the substance of the recommendations and fails to provide for monitoring of broad policy conclusions and concerns unless specific recommendations have been explicitly endorsed by the Board. The report made a number of recommendations to address these issues, including stronger ownership by the Evaluation Committee and innovations in the monitoring and reporting process. It also proposed that the IEO prepare an assessment of the status of selected major issues identified in IEO evaluations and of implementation of their recommendations. As noted in Chapter 3, when discussing the external evaluation report, the Executive Board agreed on the need to improve the follow-up process.
The IEO believes that an effective follow-up framework requires:
A system for summing up Board discussions of IEO evaluations that accurately documents the lessons and recommendations that gain Board support;
A specification of follow-up actions that are clearly linked to the intended goals; and
A transparent monitoring system that identifies shortfalls in implementation, focusing not just on specific actions but also addressing progress on broader issues that resonated with the Board.
The IEO looks forward to continued discussion of the follow-up process, with the objective of enhancing the system and the IEO’s contributions to IMF effectiveness.