Abstract
This paper sets out Management’s response to the Independent Evaluation Office’s (IEO) evaluation report on Self-Evaluation at the IMF. The implementation plan proposes specific actions to address the recommendations of the IEO that were endorsed by the Board in its September 18, 2015 discussion of the IEO’s report, namely: (i) adopt a broad policy or general principles for self-evaluation in the IMF, including its goals, scope, outputs, utilization, and follow-up; (ii) give country authorities the opportunity to express their views on program design and results, and IMF performance; (iii) for each policy and thematic review, explicitly set out a plan for how the policies and operations it covers will be self-evaluated; (iv) develop products and activities aimed at distilling and disseminating evaluative findings and lessons. The implementation of some of these proposed actions is already underway. The paper also explains how implementation will be monitored.
Introduction
1. This paper lays out a forward-looking management implementation plan (MIP) for the IEO evaluation “Self-Evaluation at the IMF: An IEO Assessment”. During the Board discussion of the report, Executive Directors welcomed the evaluation and were encouraged by the report’s findings that there is considerable self-evaluation at the IMF, that it is generally of high quality, and that it contributes usefully to reforms in policies and operations. At the same time, Directors noted the finding that there are gaps and weaknesses in the Fund’s self-evaluation, and they supported strengthening current mechanisms. Directors agreed on the importance of having a clearly articulated approach to self-evaluation that builds on current processes, takes due account of resource constraints, and adapts over time to changing circumstances. They also concurred on the need to better disseminate lessons from self-evaluation. This implementation plan proposes a range of actions to address the recommendations of the evaluation.
IEO Recommendations and Board Reactions
2. The IEO evaluation contains four key recommendations which aim to enhance the contribution of self-evaluation to IMF effectiveness by strengthening learning, transparency, and accountability. These are:
Adopt a broad policy or general principles to establish an explicit, institution-wide framework for self-evaluation in the IMF, including its goals, scope, outputs, utilization, and follow-up.
Conduct self-assessments for every IMF-supported program, giving country authorities the opportunity to express their views on program design and results, and IMF performance.
For each policy and thematic review, explicitly set out a plan for how the policies and operations it covers will be self-evaluated going forward.
Develop products and activities aimed at distilling and disseminating evaluative findings and lessons in ways that highlight their relevance for staff work and that facilitate learning.
3. During the Board discussion, Executive Directors put forth their views on how the issues raised by the report could best be addressed.
Directors underscored the benefits of taking a strategic approach to self-evaluation in light of its importance in guiding the institution’s efforts and promoting a learning culture. While a number of Directors saw merit in establishing a new, explicit institution-wide framework, many Directors considered it more useful to build on existing processes, integrating self-evaluation into the IMF’s institution-wide strategic planning framework. Several of these Directors thought the management implementation plan could itself provide the necessary strategic approach.
Directors recognized the value of drawing lessons from country experiences in IMF-supported programs and underscored the importance of better integrating country authorities’ views, whether utilizing existing mechanisms or other new approaches. While some Directors preferred self-assessing every IMF-supported program, most Directors noted the likely limited value this would generate relative to costs, in light of the existing significant amount of self-assessment of programs. These Directors favored a more selective, risk-based approach.
Directors broadly agreed that policy and thematic reviews should define at the outset the objectives of the review and what would constitute policy success. However, they did not support spelling out ex ante how the self-assessment of such reviews should be conducted, noting that this would allow plans for such reviews to be integrated within the Fund’s overall planning framework and adapted to take account of the changing needs facing the institution.
Directors saw scope to develop products and activities and revamp knowledge management practices aimed at better distilling and sharing lessons, as recommended by the report.
Implementation Plan
4. This MIP proposes specific actions to address the recommendations of the IEO that were endorsed by the Board and sets out how these will be monitored. The effectiveness and scope of these measures will continue to be reviewed and adjusted as and when warranted. The proposed measures are as follows:1
Recommendation 1: Adopt a broad policy or general principles to establish an explicit, institution-wide framework for self-evaluation in the IMF, including its goals, scope, outputs, utilization, and follow-up.
Statement of principles for self-evaluation: Management’s statement of principles for self-evaluation is attached (Annex III), and will be brought to the attention of staff by also being posted on the internal website as a stand-alone document. It describes the goals, scope, outputs, utilization, and follow-up of self-evaluation at the IMF. The statement builds on existing processes, as generally favored by Executive Directors, which will avoid establishing a separate self-evaluation architecture that would add layers to the review process. This statement will serve as a practical input into the work of staff and will serve as a useful reference for reviewers by outlining general principles and pointing to best practices, without establishing a checklist of requirements. This will allow its application to evolve with the policy and operational environment and avoid introducing excessive rigidity into the self-evaluation process.
Recommendation 2: Conduct self-assessments for every IMF-supported program, giving country authorities the opportunity to express their views on program design and results, and IMF performance.
Consistent with the views of most Executive Directors, and following recent measures taken to replace Ex-Post Assessments of programs for countries with longer-term program engagement (LTPE) with a succinct, peer-reviewed assessment, it is not proposed to conduct standalone self-assessments for every IMF-supported program. However, country authorities will be given opportunities to express their views after each program, both during the Article IV mission and at the Board meeting, and cross-cutting lessons will be taken up in policy and thematic reviews.2
Authorities’ views:
- 1)At the individual country level, at the time of the Article IV consultation immediately subsequent to the end of each program, country teams will be encouraged to discuss with the authorities their views on program design and results, and IMF performance. These discussions would take place at the time of the Article IV consultation, but would not formally be part of the Article IV consultation process.
- 2)Management welcomes the expression of the authorities’ views on program design, results, and IMF performance at the Board meeting for the Article IV consultation subsequent to the end of each program. These views could either be given in the Executive Director’s Buff statement, which would have the benefit of being part of the publication bundle, or provided orally by the Executive Director at the Board meeting.
Lessons learned: Staff will reflect the lessons learned from the above discussions, where appropriate, in the relevant sections of the Article IV staff report and in work on the country going forward.
Cross-cutting lessons: Policy and thematic reviews such as crisis program reviews, conditionality reviews, and reviews of the IMF’s engagement with specific country groups such as fragile states and small states will continue to draw cross-cutting lessons from analysis of programs across countries, as well as from individual country case studies.3 This mandate of these reviews will be made explicit as noted below in Recommendation 3.
Recommendation 3: For each policy and thematic review, explicitly set out a plan for how the policies and operations it covers will be self-evaluated going forward.
Mandate of policy and thematic reviews: Current best practices with respect to self-evaluation will be applied more consistently in policy and thematic reviews. This will include defining at the concept note stage (i) the objectives the policy was expected to achieve; (ii) the scope of the review, for example, key questions the review will cover, including any self-evaluation elements; (iii) what would constitute policy success; and, as noted above, (iv) whether the work of the review will include drawing lessons on program design and results, and IMF performance. These standards have been incorporated into the statement of principles for self-evaluation described above in Recommendation 1. Actual implementation will then be monitored via the interdepartmental review process.
Recommendation 4: Develop products and activities aimed at distilling and disseminating evaluative findings and lessons in ways that highlight their relevance for staff work and that facilitate learning.
Knowledge Management (KM): Ongoing work on KM will be leveraged to facilitate self-evaluation and learning from experience. Building on the recent vision laid out by the KM Working Group in 2014, management has established a Small Group on KM (SGKM) to develop a plan to enhance KM at the Fund through both “quick wins” and more fundamental proposals. Specifically, the SGKM is tasked to produce concrete plans to make available high-quality, web-based information on critical topics for Fund staff, and to enhance internal information classification and retrieval tools. The SGKM will also provide recommendations on the respective responsibilities of area, functional, and other departments, as well as each staff member, in strengthening KM for internal and external audiences. Finally, the SGKM will design a proposed permanent KM structure. These proposals will be contained in the final report of the SGKM expected to be delivered to management in Summer 2016.
Capacity development evaluations: The Institute for Capacity Development, working closely with other departments, is currently developing a common evaluation framework, including self-evaluation, for IMF capacity development, as endorsed by the Board in its 2013 discussion of the IMF’s capacity development strategy.4 This framework will lead to more focused and comparable evaluations while allowing flexibility to adapt evaluations to reflect the wide range of capacity development activities. The framework also emphasizes the importance of ensuring evaluations are easily accessible upon completion and findings are broadly shared among Fund staff so lessons learned can feed into future delivery.
Identifying good practices examples in surveillance. With the aims of exploiting cross-country policy experiences more effectively and responding to members’ needs for expert advice, SPR has launched an exercise to identify and disseminate examples of good practices in Article IV staff reports. The exercise will span all policy areas, and the results will be disseminated to area departments to inform their thinking on how other countries have tackled similar issues. A similar approach has already been applied in the initiative to mainstream macro-financial surveillance.
What’s Important in Review (WIR): This group, comprised of SPR country reviewers, was created in 2015, and meets on a periodic basis to discuss important issues in country review. This venue facilitates learning from experience, especially incorporating best practices across a broad range of country work, and disseminating lessons learned in policy reviews.
Risk Management Unit (RMU): The RMU, established in 2014, produces reports on the Fund’s risk profile at regular intervals, in the context of which it highlights, as needed, areas for additional risk analysis and mitigation efforts.
5. Progress in implementing the proposed actions will be monitored closely. Implementation of this MIP will be coordinated by the Strategy, Policy, and Review Department (SPR), in close collaboration with the Office of Budget and Planning (on budget-related issues). Monitoring will be primarily through future Periodic Monitoring Reports, which provide updates on implementation of IEO evaluations. SPR will also ensure through the regular interdepartmental review process that country teams are discussing program performance with the authorities in the subsequent Article IV, and that policy and thematic reviews follow best practices in self-evaluating policies and operations covered by the review. The Secretary’s Department will ensure the Compendium of Executive Board Work Procedures makes explicit that Directors are encouraged to convey the views of their authorities on program performance as described above.
Resource Implications
6. The new initiatives outlined above would have small resource costs. The proposals to ensure self-assessment and the inclusion of authorities’ views, and proposals to more consistently apply self-evaluation best practices in policy reviews will likely require some reprioritization, to be broadly accommodated under the existing resource envelope. The formulation of this MIP, and the creation of the statement of principles and best practices for self-evaluation, had a one-time resource cost of approximately 0.25 FTE in FY2016. The development of products and activities aimed at distilling and disseminating evaluative findings and lessons will be implemented as part of the broader effort to enhance the management and sharing of knowledge, which is likely to have resource costs, to be defined during the course of FY2017.
Annex I. IEO Evaluation of Self-Evaluation at the IMF: Recommendations, Board Response, and Proposed Follow-Up
| IEO Recommendation | Executive Directors’ Responses | Follow-Up Plan and Timeline | Accountability |
|---|---|---|---|
| 1. Adopt a broad policy or general principles to establish an explicit, institution-wide framework for self-evaluation in the IMF, including its goals, scope, outputs, utilization, and follow-up. | While a number of Directors saw merit in establishing a new, explicit institution-wide framework, many Directors considered it more useful to build on existing processes, integrating self-evaluation into the IMFs institution-wide strategic planning framework. | A high-level statement of principles for self-evaluation is attached (Annex III) and will be posted on the internal website. The statement builds on existing processes, serves as a practical input into the work of staff, and will provide a useful reference for reviewers by outlining general principles and pointing to best practices, without establishing a checklist of requirements. This will allow its application to evolve with the policy and operational environment and avoid introducing excessive rigidity into the self-evaluation process. | Management will update the statement as and when necessary. |
| 2. Conduct self-assessments for every IMF-supported program, giving country authorities the opportunity to express their views on program design and results, and IMF performance. | While some Directors preferred self-assessing every IMF-supported program, most Directors noted the likely limited value this would generate relative to costs, in light of the existing significant amount of self-assessment of programs. These Directors favored a more selective, risk-based approach. Directors underscored the importance of better integrating country authorities’ views, whether utilizing existing mechanisms or other new approaches. | At the individual country level, at the time of the Article IV consultation subsequent to the end of each program, country teams will be encouraged to discuss with the authorities their views on program design and results, and IMF performance. The lessons learned should be reflected in the staff report where appropriate. Policy and thematic reviews will continue to draw cross-cutting lessons from analysis of programs across countries, as well as from individual country case studies. These proposals reflect existing best practice and are outlined in the statement of principles for self-evaluation (Annex III). Strengthened monitoring procedures will be established following Board endorsement of this MIP. | Country teams, with SPR to monitor implementation in the normal review process. Departments conducting policy and thematic reviews will implement and monitor. |
| 3. For each policy and thematic review, explicitly set out a plan for how the policies and operations it covers will be self-evaluated going forward. | Directors broadly agreed that policy and thematic reviews should define at the outset the objectives of the review and what would constitute policy success. However, they did not support spelling out ex ante how the self-assessment of such reviews should be conducted, noting that this would allow plans for such reviews to be integrated within the Fund’s overall planning framework and adapted to take account of the changing needs facing the institution. | Current best practices will be applied more consistently with respect to self-evaluation in policy and thematic reviews. This will include defining at the concept note stage the objectives of the review, including any self-evaluation elements, what would constitute policy success, and whether the work of the review will include drawing lessons on program design and results, and IMF performance. These best practices are outlined in the statement of principles for self-evaluation (Annex III). | Departments conducting policy and thematic reviews will implement and monitor. |
| 4. Develop products and activities aimed at distilling and disseminating evaluative findings and lessons in ways that highlight their relevance for staff work and that facilitate learning. | Directors supported this recommendation. | Management has established a Small Group on Knowledge Management (SGKM) to develop a plan to enhance KM. The SGKM is tasked to (i) produce concrete plans to make available high-quality, web-based information on critical topics for Fund staff; (ii) enhance internal information classification and retrieval tools; (iii) provide recommendations on the respective responsibilities of area, functional, and other departments, as well as each staff member, in strengthening KM; and (iv) propose a permanent KM structure. The final report of the SGKM is expected to be delivered to management in Summer 2016. The Institute for Capacity Development (ICD) is currently developing a common evaluation framework, including self-evaluation, for IMF capacity development. This framework, while allowing flexibility, will lead to more focused and comparable evaluations. Evaluations will be easily accessible upon completion and findings broadly shared among Fund staff so lessons learned can feed into future delivery. An exercise has been launched by SPR to identify and disseminate examples of good practices in Article IV staff reports. The exercise will span all policy areas, and the results will be disseminated to area departments to inform their thinking on how other countries have tackled similar issues. The What’s Important in Review group, comprised of SPR country reviewers, was created in 2015 and meets on a periodic basis to facilitate learning from experience, especially incorporating best practices across a broad range of country work and disseminating lessons learned in policy reviews. The Risk Management Unit was established in 2014 and produces reports on the Fund’s risk profile at regular intervals, in the context of which it highlights, as needed, areas for additional risk analysis and mitigation efforts. | Small Group on Knowledge Management ICD to implement. SPR to continue implementation. Risk Management Unit. |
| IEO Recommendation | Executive Directors’ Responses | Follow-Up Plan and Timeline | Accountability |
|---|---|---|---|
| 1. Adopt a broad policy or general principles to establish an explicit, institution-wide framework for self-evaluation in the IMF, including its goals, scope, outputs, utilization, and follow-up. | While a number of Directors saw merit in establishing a new, explicit institution-wide framework, many Directors considered it more useful to build on existing processes, integrating self-evaluation into the IMFs institution-wide strategic planning framework. | A high-level statement of principles for self-evaluation is attached (Annex III) and will be posted on the internal website. The statement builds on existing processes, serves as a practical input into the work of staff, and will provide a useful reference for reviewers by outlining general principles and pointing to best practices, without establishing a checklist of requirements. This will allow its application to evolve with the policy and operational environment and avoid introducing excessive rigidity into the self-evaluation process. | Management will update the statement as and when necessary. |
| 2. Conduct self-assessments for every IMF-supported program, giving country authorities the opportunity to express their views on program design and results, and IMF performance. | While some Directors preferred self-assessing every IMF-supported program, most Directors noted the likely limited value this would generate relative to costs, in light of the existing significant amount of self-assessment of programs. These Directors favored a more selective, risk-based approach. Directors underscored the importance of better integrating country authorities’ views, whether utilizing existing mechanisms or other new approaches. | At the individual country level, at the time of the Article IV consultation subsequent to the end of each program, country teams will be encouraged to discuss with the authorities their views on program design and results, and IMF performance. The lessons learned should be reflected in the staff report where appropriate. Policy and thematic reviews will continue to draw cross-cutting lessons from analysis of programs across countries, as well as from individual country case studies. These proposals reflect existing best practice and are outlined in the statement of principles for self-evaluation (Annex III). Strengthened monitoring procedures will be established following Board endorsement of this MIP. | Country teams, with SPR to monitor implementation in the normal review process. Departments conducting policy and thematic reviews will implement and monitor. |
| 3. For each policy and thematic review, explicitly set out a plan for how the policies and operations it covers will be self-evaluated going forward. | Directors broadly agreed that policy and thematic reviews should define at the outset the objectives of the review and what would constitute policy success. However, they did not support spelling out ex ante how the self-assessment of such reviews should be conducted, noting that this would allow plans for such reviews to be integrated within the Fund’s overall planning framework and adapted to take account of the changing needs facing the institution. | Current best practices will be applied more consistently with respect to self-evaluation in policy and thematic reviews. This will include defining at the concept note stage the objectives of the review, including any self-evaluation elements, what would constitute policy success, and whether the work of the review will include drawing lessons on program design and results, and IMF performance. These best practices are outlined in the statement of principles for self-evaluation (Annex III). | Departments conducting policy and thematic reviews will implement and monitor. |
| 4. Develop products and activities aimed at distilling and disseminating evaluative findings and lessons in ways that highlight their relevance for staff work and that facilitate learning. | Directors supported this recommendation. | Management has established a Small Group on Knowledge Management (SGKM) to develop a plan to enhance KM. The SGKM is tasked to (i) produce concrete plans to make available high-quality, web-based information on critical topics for Fund staff; (ii) enhance internal information classification and retrieval tools; (iii) provide recommendations on the respective responsibilities of area, functional, and other departments, as well as each staff member, in strengthening KM; and (iv) propose a permanent KM structure. The final report of the SGKM is expected to be delivered to management in Summer 2016. The Institute for Capacity Development (ICD) is currently developing a common evaluation framework, including self-evaluation, for IMF capacity development. This framework, while allowing flexibility, will lead to more focused and comparable evaluations. Evaluations will be easily accessible upon completion and findings broadly shared among Fund staff so lessons learned can feed into future delivery. An exercise has been launched by SPR to identify and disseminate examples of good practices in Article IV staff reports. The exercise will span all policy areas, and the results will be disseminated to area departments to inform their thinking on how other countries have tackled similar issues. The What’s Important in Review group, comprised of SPR country reviewers, was created in 2015 and meets on a periodic basis to facilitate learning from experience, especially incorporating best practices across a broad range of country work and disseminating lessons learned in policy reviews. The Risk Management Unit was established in 2014 and produces reports on the Fund’s risk profile at regular intervals, in the context of which it highlights, as needed, areas for additional risk analysis and mitigation efforts. | Small Group on Knowledge Management ICD to implement. SPR to continue implementation. Risk Management Unit. |
Annex II. Resource Implications of the Management Implementation Plan
| IEO Recommendation | Resource Cost through FY 2017, (FTEs) | Future Recurrent Costs (FTEs per year) |
|---|---|---|
| 1. Adopt a broad policy or general principles to establish an explicit, institution-wide framework for self-evaluation in the IMF, including its goals, scope, outputs, utilization, and follow-up. | 0.25 | 0 |
| 2. Conduct self-assessments for every IMF-supported program, giving country authorities the opportunity to express their views on program design and results, and IMF performance. | 0 | 0 |
| 3. For each policy and thematic review, explicitly set out a plan for how the policies and operations it covers will be self-evaluated going forward. | 0 | 0 |
| 4. Develop products and activities aimed at distilling and disseminating evaluative findings and lessons in ways that highlight their relevance for staff work and that facilitate learning. | TBD | TBD |
| IEO Recommendation | Resource Cost through FY 2017, (FTEs) | Future Recurrent Costs (FTEs per year) |
|---|---|---|
| 1. Adopt a broad policy or general principles to establish an explicit, institution-wide framework for self-evaluation in the IMF, including its goals, scope, outputs, utilization, and follow-up. | 0.25 | 0 |
| 2. Conduct self-assessments for every IMF-supported program, giving country authorities the opportunity to express their views on program design and results, and IMF performance. | 0 | 0 |
| 3. For each policy and thematic review, explicitly set out a plan for how the policies and operations it covers will be self-evaluated going forward. | 0 | 0 |
| 4. Develop products and activities aimed at distilling and disseminating evaluative findings and lessons in ways that highlight their relevance for staff work and that facilitate learning. | TBD | TBD |
Annex III. Principles and Best Practices in IMF Self-Evaluation
Goals and Scope of Self-Evaluation
The overarching objective of self-evaluation at the IMF is to develop a culture of continuously learning from experience. Such a culture will facilitate the institution’s efforts to assess compliance with its policies, gauge the effectiveness of policies and practices, consider how to improve them, and promote accountability. The Fund relies on a combination of internal self-evaluation and the work of the Independent Evaluation Office, and takes into account the findings of self-evaluation activities in designing and implementing operational work and the Fund’s policy agenda.
Policy and Thematic Reviews
The IMF takes a strategic approach to policy and thematic reviews. A key objective of these reviews is to step back occasionally and assess in light of experience whether existing policies need to be adjusted. However, reviews of IMF policies that are conducted too frequently may lower the IMF’s ability to focus on key strategic issues and could be resource intensive. They may also have insufficient basis on which to evaluate policy implementation and thus have little strategic benefit. To balance these considerations, most reviews are conducted on an as-needed basis, with the need for a review assessed after a predetermined amount of time.5 Table 1 lists the regular policy reviews conducted at the Fund and the minimum periodicity of their review cycle.6 The IMF may also conduct, from time to time, thematic reviews on topics or groups of countries where issues are of broader relevance to the work of the institution.7
Policy Reviews 1/
A number of policy reviews, such as on charges and maturities, Extended Fund Facilities, lending into arrears, and misreporting are not included in this table, as they are already on an as-needed basis.
Represents first assessment of whether a review is needed; assessments are repeated on an annual basis until such time as a review is conducted.
Given the major reforms to these facilities in 2014, the Executive Board requested the next review to be conducted within three years of implementation from their date of effectiveness.
There is also a presumption that a review would be triggered any time the aggregate outstanding credit and commitments under the FCL and PLL reach SDR 150 billion.
Policy Reviews 1/
| Policy Review | Dept | Frequency 2/ | |
|---|---|---|---|
| Surveillance | |||
| 1 | Comprehensive Surveillance Review (CSR) | SPR/MCM | 5 years |
| 2 | Surveillance Interim Progress Report | SPR | 5 years |
| 3 | Transparency Policy Review | SPR | 5 years |
| 4 | Review of the Role of Trade in the Work of the Fund | SPR | 5 years |
| 5 | Excessive Delays in Completion of Article IV Consultations Review | SPR | 5 years |
| 6 | Review of Standards and Codes Initiative | SPR and Functional | 5 years |
| 7 | Review of Fund’s Data Standards Initiative | STA | 5 years |
| 8 | Review of Data Provision to the Fund for Surveillance Purposes | SPR/STA | 5 years |
| Use of Fund Resources (UFR) Issues | |||
| 9 | Review of Conditionality and Design of Fund-Supported Programs | SPR | 5 years |
| 10 | Review of Fund’s Facilities for Low-Income Countries | SPR | 5 years |
| 11 | Review of Flexible Credit Line (FCL), Precautionary and Liquidity Line (PLL), Rapid Financing Instrument (RFI) 3/4/ | FIN/LEG/SPR | 5 years |
| 12 | Review of Access Policy (including Exceptional Access) | FIN/SPR | 5 years |
| 13 | Review of Fund’s Strategy on Overdue Financial Obligations | FIN/LEG/SPR | 5 years |
| 14 | Review of Poverty Reduction and Growth Trust (PRGT) Eligibility | SPR | 2 years |
| 15 | Review of PRGT Interest Rates | FIN | 2 years |
| 16 | Safeguards Assessment Review | FIN | 5 years |
| 17 | Debt Sustainability Framework for Low Income Countries | SPR | 3 years |
| Other | |||
| 18 | General Review of Quotas | FIN/SPR | 5 years |
| 19 | SDR Valuation Basket and SDR Rate of Interest Review | FIN/SPR | 5 years |
| 20 | Review of the Fund’s Capacity Development Strategy | ICD | 5 years |
| 21 | Review of Bank-Fund Collaboration | SPR | 5 years |
| 22 | Review of the Effectiveness of the Anti-Money Laundering and Combating the Financing of Terrorism (AML/CFT) Program | LEG | 5 years |
A number of policy reviews, such as on charges and maturities, Extended Fund Facilities, lending into arrears, and misreporting are not included in this table, as they are already on an as-needed basis.
Represents first assessment of whether a review is needed; assessments are repeated on an annual basis until such time as a review is conducted.
Given the major reforms to these facilities in 2014, the Executive Board requested the next review to be conducted within three years of implementation from their date of effectiveness.
There is also a presumption that a review would be triggered any time the aggregate outstanding credit and commitments under the FCL and PLL reach SDR 150 billion.
Policy Reviews 1/
| Policy Review | Dept | Frequency 2/ | |
|---|---|---|---|
| Surveillance | |||
| 1 | Comprehensive Surveillance Review (CSR) | SPR/MCM | 5 years |
| 2 | Surveillance Interim Progress Report | SPR | 5 years |
| 3 | Transparency Policy Review | SPR | 5 years |
| 4 | Review of the Role of Trade in the Work of the Fund | SPR | 5 years |
| 5 | Excessive Delays in Completion of Article IV Consultations Review | SPR | 5 years |
| 6 | Review of Standards and Codes Initiative | SPR and Functional | 5 years |
| 7 | Review of Fund’s Data Standards Initiative | STA | 5 years |
| 8 | Review of Data Provision to the Fund for Surveillance Purposes | SPR/STA | 5 years |
| Use of Fund Resources (UFR) Issues | |||
| 9 | Review of Conditionality and Design of Fund-Supported Programs | SPR | 5 years |
| 10 | Review of Fund’s Facilities for Low-Income Countries | SPR | 5 years |
| 11 | Review of Flexible Credit Line (FCL), Precautionary and Liquidity Line (PLL), Rapid Financing Instrument (RFI) 3/4/ | FIN/LEG/SPR | 5 years |
| 12 | Review of Access Policy (including Exceptional Access) | FIN/SPR | 5 years |
| 13 | Review of Fund’s Strategy on Overdue Financial Obligations | FIN/LEG/SPR | 5 years |
| 14 | Review of Poverty Reduction and Growth Trust (PRGT) Eligibility | SPR | 2 years |
| 15 | Review of PRGT Interest Rates | FIN | 2 years |
| 16 | Safeguards Assessment Review | FIN | 5 years |
| 17 | Debt Sustainability Framework for Low Income Countries | SPR | 3 years |
| Other | |||
| 18 | General Review of Quotas | FIN/SPR | 5 years |
| 19 | SDR Valuation Basket and SDR Rate of Interest Review | FIN/SPR | 5 years |
| 20 | Review of the Fund’s Capacity Development Strategy | ICD | 5 years |
| 21 | Review of Bank-Fund Collaboration | SPR | 5 years |
| 22 | Review of the Effectiveness of the Anti-Money Laundering and Combating the Financing of Terrorism (AML/CFT) Program | LEG | 5 years |
A number of policy reviews, such as on charges and maturities, Extended Fund Facilities, lending into arrears, and misreporting are not included in this table, as they are already on an as-needed basis.
Represents first assessment of whether a review is needed; assessments are repeated on an annual basis until such time as a review is conducted.
Given the major reforms to these facilities in 2014, the Executive Board requested the next review to be conducted within three years of implementation from their date of effectiveness.
There is also a presumption that a review would be triggered any time the aggregate outstanding credit and commitments under the FCL and PLL reach SDR 150 billion.
Several best practices should, where relevant, be considered when undertaking policy and thematic reviews. These include:
- -Defining at the concept note stage:
the objectives the policy was expected to achieve;
the scope of the review, for example, key questions the review will cover, including any self-evaluation elements;
what would constitute policy success;
and whether the work of the review will include drawing lessons on program design and results, and IMF performance;
- -Providing cross-cutting findings and lessons to help inform future policy development;
- -Paying explicit attention to staff practices as well as broader institutional policies and operations;
- -Use of a wide variety of evaluative techniques, including the analysis of previous staff reports, and the use of surveys and interviews;
- -Consideration of external views. These can range from utilizing commentary and analysis that originates from outside the Fund to the use of external consultants and advisory groups during the self-evaluation process.
IMF Lending Arrangements
The IMF applies a multi-layered, risk-based approach to the evaluation of its lending arrangements with individual countries.
- -Ex Post Evaluations (EPEs) are required within a year after the end of any arrangement under facilities in the General Resources Account that involve exceptional access.8 EPEs contain substantial self-evaluation elements, discussing whether justifications presented at the outset of the individual program were consistent with IMF policies and reviewing performance under the program.
- -For countries with longer-term program engagement (LTPE, defined as having in place an IMF-supported lending arrangement for at least seven of the past ten years), a peer-reviewed assessment is undertaken if a request for a new arrangement is made.9 These assessments review the appropriateness of the IMF’s overall approach and soundness of its advice in the context of previous IMF arrangements. This allows the design of any new arrangement to be informed by lessons learned from previous arrangements.
- -For all programs, country authorities will be given opportunities to express their views on program design and performance after the conclusion of the program, both during the Article IV mission and at the Board meeting.10
At the time of the Article IV consultation immediately subsequent to the end of each program, country teams will be encouraged to discuss with the authorities their views on program design and results, and IMF performance. These discussions and the reporting of the authorities’ views would typically take place at the time of the Article IV consultation, but they would not formally be part of the Article IV consultation process.
Staff will reflect the lessons learned, where appropriate, in the relevant sections of the Article IV staff report, and in work on the country going forward.
It is strongly encouraged that the country authorities also express their views on these issues at the Board meeting for the Article IV consultation subsequent to the end of each program. These views could either be given in the Executive Director’s Buff statement or provided orally by the Executive Director at the Board meeting.
Surveillance
The IMF periodically conducts the Comprehensive Review of Surveillance. However, learning from experience in strengthening surveillance should also be a continuous process. It is best practice for country teams and review departments to reflect on the effectiveness of their various surveillance activities and for departments to collect lessons learned to serve as inputs into applicable broader policy or thematic reviews such as the Comprehensive Surveillance Review.
An ongoing initiative to identify good practice examples in surveillance will contribute to the learning process. SPR has launched an exercise to identify and disseminate good practice examples across Article IV staff reports. This exercise spans all policy areas, and the results will be disseminated to area departments to inform their thinking on how other countries have tackled similar issues.
Capacity Development
The IMF is developing a common evaluation framework for capacity development (i.e. technical assistance and training) that includes scope for conducting self-evaluations by staff, in addition to external evaluations. Such self-assessments may focus not only on country cases, but also on thematic areas. This framework will draw on the OECD’s Development Assistance Committee criteria (i.e. relevance, effectiveness, efficiency, impact, and sustainability) and include processes for better distilling and disseminating lessons from evaluations, including self-evaluations.
Utilization
To help disseminate the lessons from self-evaluation, it is expected that all policy and thematic reviews will be presented both within the Fund and, where applicable, to outside audiences and stakeholders. Evaluations of all types are expected to be easily accessible upon completion and findings broadly shared among Fund staff so lessons learned can feed into future delivery. Reviewing departments are expected to place high value on the dissemination of lessons from previous self-evaluation exercises when undertaking their work. To facilitate these objectives, the Small Group on Knowledge Management is producing concrete plans to make available high-quality, web-based information on critical topics for Fund staff, and to enhance internal information classification and retrieval tools.
Annex I summarizes the evaluation’s recommendations, Directors’ responses, proposed measures, timelines, and responsibilities for implementation.
In cases where a successor program is requested before the Article IV consultation subsequent to the end of a program, the proposed procedures would be conducted at that time. The procedures proposed here would apply to all programs except where more in-depth assessments are required. For countries with LTPE, the required peer-reviewed assessment would be conducted. Similarly, programs with exceptional access are subject to an ex post evaluation.
In addition, the next installment of the Fund’s history, which is currently being commissioned, will also contribute to the Fund’s culture of learning from past experience.
IMF Executive Board Reviews the Fund’s Capacity Development Strategy, Public Information Notice (PIN) No. 13/72 (6/27/2013).
Six to twelve months prior to a scheduled review, staff assess whether or not the review is warranted, bringing its assessment to the attention of the Executive Board in the context of the semi-annual work program discussion.
See Selected Streamlining Proposals under the FY16–FY18 Medium-Term Budget—Implementation Issues.
Some examples include: Crisis Program Review, Macroeconomic Developments and Selected Issues in Small Developing States, and IMF Engagement with Countries in Post-Conflict and Fragile Situations – Stocktaking.
See Ex Post Evaluations of Exceptional Access Arrangements—Revised Guidance Note.
This procedure supersedes the previous Ex Post Assessment. A guidance note is forthcoming.
In cases where a successor program is requested before the Article IV consultation subsequent to the end of a program, the proposed procedures would be conducted at that time.