Chapter 3: Cross-Cutting Issues
- International Monetary Fund. Independent Evaluation Office
- Published Date:
- September 2011
IEO evaluations over the past few years have shared several common concerns. Key among these is the need to strengthen IMF governance, in particular to provide greater clarity on roles and responsibilities, better integration of different streams of work within the institution, better management of institutional change, and a clear framework for assessing results and accountability. IEO evaluations have also called for the Fund to encourage diversity of views and alternative perspectives, and to ensure consistency of treatment accorded to countries across the membership. More broadly, in reflecting on the impact of IEO evaluations, recent attention has focused on weaknesses in the process for following up on Board discussions and in particular on IEO recommendations that had been endorsed by the Executive Board.
Follow-up on IEO recommendations was a key issue in the first External Evaluation of the IEO in 2006. This Report highlighted the goal of enhancing the learning culture of the IMF as a key purpose for which the IEO was established.3 The Report noted that the Board chose to create an independent evaluation function rather than continue commissioning external evaluations as it had done in the 1990s because the latter were not always found to be useful or applicable to the Fund’s operation, IMF staff had little ownership of the results, and no mechanism existed for implementing the recommendations or following up.4 A standing independent evaluation mechanism, it was hoped, would be more effective in promoting change.
The External Evaluation Report concluded that the IEO had had some impact but found little evidence that findings and recommendations of specific IEO reports were systematically taken up and followed up by senior management and the Board. Accordingly, the Report called for a more systematic approach to follow up on IEO recommendations and monitor their implementation. It further recommended that the Board and the Evaluation Committee take responsibility and play a more active role in this regard.
Following the External Evaluation Report, the IMF put in place a process for follow-up that relies on IMF Management and staff to propose implementation steps (via a Management Implementation Plan, or MIP) and document their completion or remedial action still needed (Periodic Monitoring Report, or PMR). These documents are then reviewed by the Evaluation Committee and considered by the Board.
In considering PMRs, the Evaluation Committee has regularly endorsed staff’s assessment of the status of performance benchmarks and implementation plans in response to the Board-endorsed IEO recommendations for particular evaluations. However, while accepting staff’s assessment, the Committee typically has also made clear that further progress was needed on some issues. For instance, in its comments on the Third PMR in January 2010 the Committee called for more work on the system to track goals and strategies and links to specific conditions in IMF-supported programs; pointed out that the issue of excessive staff mobility had not been resolved; and indicated a desire for further attention to the issue of confidential information in Article IV staff reports.
As noted in the FY2010 Annual Report, the Evaluation Committee has also raised concerns about the follow-up process for IEO evaluations. Members of the Committee and other Executive Directors have noted on several occasions that IEO evaluations raised concerns that were broadly shared by the Board, but that were not included in MIPs because a specific recommendation had not been explicitly endorsed. Hence, Directors suggested that, after discussing each evaluation, more reflection by the Board was needed on what would be necessary to follow through on the lessons and recommendations in the evaluation. Moreover, Committee members pointed out cases in which specific actions contained in a MIP had been completed or were on track for completion, but where other reforms were needed to achieve the broader policy objective underlying the IEO’s conclusions or specific recommendation. Thus, ongoing concerns about the current process stem largely from the question of whether the weaknesses and/or areas needing change identified in IEO evaluations are being acted upon—not whether specific steps recommended by the IEO are being implemented as written. During FY2011, the Committee broached the possibility of improving the PMR process, as one step in tackling the need to improve the follow-up process.
These concerns have implications for how the IEO presents its conclusions and recommendations in a way that facilitates change in the IMF.
In designing its recommendations the IEO faces important trade-offs regarding the degree of specificity. The IEO can provide general recommendations to complement its conclusions, leaving it to IMF Management to propose specific actions to effect change. This approach has the advantage of focusing attention on big picture goals and allows the Board to endorse the direction of needed reforms while allowing Management the flexibility to propose how best to pursue these goals and to present specific actions in the MIP to achieve them. On the other hand, this approach makes it very difficult for the Board to assess the extent to which Management’s proposed actions would address the goals endorsed; further, often these actions are such that it is difficult for the Board to monitor their implementation.
Alternatively, the IEO can recommend specific actions to address goals and concerns raised by its evaluations. Detailed IEO recommendations have the advantage that they are more likely to be closely aligned with the conclusions of the evaluation and easier for the Board to monitor. But this approach may diminish Management and staff ownership of the implementation plan, and would not make full use of their greater institutional knowledge and their ability to integrate the implementation of Board-endorsed recommendations into other strands of IMF work. In an effort to balance these competing approaches, the IEO often provides general conclusions and recommendations, accompanied by specific recommendations intended to give examples of how to achieve these aims.5
In an effort to enhance its own work, the IEO has launched a review of how recommendations in evaluations prepared over its first ten years have been implemented. In undertaking this work, the IEO intends to help enhance the impact of its work on the IMF’s institutional effectiveness and learning. An anticipated second external evaluation of the IEO could also be expected to further this effort.
“Report of the External Evaluation of the Independent Evaluation Office,” March 29, 2006 (available at www.imf.org). The other motivations were for the IEO to serve as a watchdog to help build external credibility, to assist the Executive Board in its oversight responsibilities over the IMF by providing independent feedback, and to serve as a tool of outreach by promoting greater understanding of the IMF both among its members and the broader public.
Ibid, p. 8.
IEO evaluations also include specific recommendations that are considered in themselves, and not only as an example of how to achieve a broader goal. In context, it is usually clear whether a recommendation is just an example, or intended to be implemented as suggested in the report.