Abstract
The thrust of the report’s recommendations to strengthen the institutional framework for self-evaluation, adapt it over time to changing circumstances, and better disseminate lessons from self-evaluation are well taken. I believe, however, that there is considerable scope to address these recommendations by refining and building on existing processes and initiatives already underway rather than by introducing substantial new architecture. I see limited value added relative to the costs in following through with the report’s two other key recommendations; namely, to conduct self-assessment for every IMF-supported program and to set out plans ex ante to self-evaluate every policy and thematic review.
I would like to thank the Independent Evaluation Office (IEO) for preparing this useful report, which assesses self-evaluation conducted by the IMF. I welcome the report’s findings that there is considerable self-evaluation at the IMF; that such self-evaluation is generally of high quality; and, that it contributes usefully to reforms in policies and operations.
The thrust of the report’s recommendations to strengthen the institutional framework for self-evaluation, adapt it over time to changing circumstances, and better disseminate lessons from self-evaluation are well taken. I believe, however, that there is considerable scope to address these recommendations by refining and building on existing processes and initiatives already underway rather than by introducing substantial new architecture. I see limited value added relative to the costs in following through with the report’s two other key recommendations; namely, to conduct self-assessment for every IMF-supported program and to set out plans ex ante to self-evaluate every policy and thematic review.
Overall, the IEO report finds that considerable self-evaluation takes place at the IMF and that many of these activities and reports are of high technical quality. These findings are reassuring. The report notes that the IMF does not have an explicit over-arching framework or policy to establish what needs to be evaluated and how, who is responsible, and how follow up is to be conducted. Despite the absence of such an over-arching framework, the report finds that the flexible approach taken to self-evaluation to date has generally served the Fund well, with self-evaluation integrated with the institution’s broader policy work, its lessons feeding into the design of operations, and self-evaluation outputs and practices evolving as the IMF’s work and priorities changed.
Against this background, I see limited value in adopting a broad policy to establish an explicit, institution-wide framework for self-evaluation in the IMF that is subject to review every few years (recommendation 1). Any such statement of principles on the “goals, scope, outputs, utilization, and follow up” would have to be general, as the report rightly notes, to allow self-evaluation products and practices to continue to evolve with the policy and operational environment. I am concerned that this same level of generality would, however, render the new policy largely inefficacious in materially improving self-evaluation. On the other hand, a policy that is excessively detailed and prescriptive could introduce undesirable rigidity as to when and how the Fund conducts self-evaluation.
Rather, efforts to improve self-evaluation should be continuous and build on the multi-faceted processes and infrastructure for self-evaluation that already exist at the IMF. The Strategy, Policy, and Review Department, the Office of Internal Audit, as well as the multi-step cross-departmental review processes for policy, surveillance documents, and program reviews are an integral part of this infrastructure. Similarly, human resource, budget, and audit functions, which are excluded from the scope of the report, play a critical role in enhancing the effectiveness, learning, and accountability of duties performed by Fund staff. More generally, the IMF’s institution-wide strategic planning framework, starting from the Global Policy Agenda, and embodied in the Accountability Framework, seeks to assess progress made on the institution’s strategic and operational priorities, identify gaps from a holistic assessment of challenges facing the institution, and review performance against objectives set.
In light of this, I see scope to build on the existing self-evaluation processes and infrastructure, which reflect the Fund’s work and operational needs, to further strengthen the self-evaluation function (recommendation 1) and better distill and disseminate lessons from self-evaluation (recommendation 4). The current effort to build a common evaluation framework for the Fund’s capacity development activities is an initiative that is already going in this direction.
The report recommends a significant expansion of the scale and scope of self-evaluation in certain specific areas that I cannot support. The recommendation to conduct self-assessments for every IMF-supported program (recommendation 2) does not adequately take into account resource-related tradeoffs. Similarly setting out a plan ex ante for how the self-assessment of every policy and thematic review should be conducted (recommendation 3) risks turning self-assessment into a routine, box-ticking exercise divorced from new and rapidly evolving challenges that the institution should adapt to instead.
The quarterly or semi-annual Executive Board reviews of all Fund-supported programs already have both forward-looking and backward-looking components, with reviews serving to adjust programs based on lessons learned. These Board reviews further provide ample opportunities for country authorities to express their views. The lessons learned in all of these circumstances will continue to be supplemented by: (a) cross-cutting in-depth reviews of programs conducted in the context of policy and thematic work, including crises program reviews, conditionality reviews, or reviews of the IMF’s engagement with specific country groups such as fragile and small states; (b) the process that will replace EPAs, whereby the staff report of a new program request will contain a succinct, peer-reviewed assessment of the previous program; and (c) the continuation of EPEs. Thus, considerable self-assessment of programs will already be taking place.
Any consideration to expand self assessment of programs further needs to remain mindful of budget realities, which featured prominently in the Board’s recent decision to discontinue EPAs and to revise the approach to policy reviews. Given that any additional processes or layers of self-assessment will likely generate limited additional gains relative to large resource costs, I cannot support recommendation 2. We will, of course, continue to review the effectiveness and scope of self-assessment procedures and adjust them as and when warranted.
I agree on the need to plan on how policies and operations covered by policy and thematic reviews will be evaluated. However, carving out and solidifying specific modalities years in advance, irrespective of evolving priorities and changing circumstances that are likely to materially impact on how the evaluation is best conducted is unlikely to be a very useful exercise. Rather, the plans should be integrated with the Fund’s overall planning framework and work program and adapted to take into account the changing needs and challenges facing the institution. Management and staff are already committed to doing this and, therefore, in this light, I do not support recommendation 3.
The Managing Director’s Position on IEO Recommendations
The Managing Director’s Position on IEO Recommendations
| Recommendation | Position | |
| 1) | Institutional framework. The IMF should adopt a broad policy or general principles to establish an explicit, institution-wide framework for self-evaluation in the IMF (including its goals, scope, outputs, utilization, and follow up). It should then conduct a periodic review of this function as a basis to adapt the policy to changing circumstances. | Qualified Support |
| 1) | Self-assessment of programs. The IMF should conduct self-assessments for every IMF-supported program. The scope and format of these assessments could vary across programs, but it is critical that country authorities be given the opportunity to express their views on the design and results of each program as well as on IMF performance. | No Support |
| 3) | Self-assessment of policies and other institution-wide themes. Each policy and thematic review should explicitly set out a plan for how the policies and operations it covers will be self-evaluated going forward. | No Support |
| 4) | Distilling and disseminating self-evaluation lessons. IMF Management should develop products and activities aimed at distilling and disseminating evaluative findings and lessons in ways that highlight their relevance for Staff work and that facilitate learning. | Support |
The Managing Director’s Position on IEO Recommendations
| Recommendation | Position | |
| 1) | Institutional framework. The IMF should adopt a broad policy or general principles to establish an explicit, institution-wide framework for self-evaluation in the IMF (including its goals, scope, outputs, utilization, and follow up). It should then conduct a periodic review of this function as a basis to adapt the policy to changing circumstances. | Qualified Support |
| 1) | Self-assessment of programs. The IMF should conduct self-assessments for every IMF-supported program. The scope and format of these assessments could vary across programs, but it is critical that country authorities be given the opportunity to express their views on the design and results of each program as well as on IMF performance. | No Support |
| 3) | Self-assessment of policies and other institution-wide themes. Each policy and thematic review should explicitly set out a plan for how the policies and operations it covers will be self-evaluated going forward. | No Support |
| 4) | Distilling and disseminating self-evaluation lessons. IMF Management should develop products and activities aimed at distilling and disseminating evaluative findings and lessons in ways that highlight their relevance for Staff work and that facilitate learning. | Support |