Back Matter
Author:
International Monetary Fund. Independent Evaluation Office
Search for other papers by International Monetary Fund. Independent Evaluation Office in
Current site
Google Scholar
Close

Abstract

This first Annual Report describes the activities of the IEO through the first full year of operation. The report provides a summary of the main findings and recommendations of the first three evaluation projects on the prolonged use of IMF resources, the role of the IMF in recent capital account crises, and the role of fiscal adjustment in IMF-supported programs. It also indicates the status of ongoing evaluation projects, and discusses several common themes emerging from the IEO’s evaluations.

Appendix 1 Terms of Reference of the IEO

Purpose

The Independent Evaluation Office (IEO) has been established to systematically conduct objective and independent evaluations on issues, and on the basis of criteria, of relevance to the mandate of the Fund. It is intended to serve as a means to enhance the learning culture within the Fund, strengthen the Fund’s external credibility, promote greater understanding of the work of the Fund throughout the membership, and support the Executive Board’s institutional governance and oversight responsibilities. IEO has been designed to complement the review and evaluation work within the Fund and should, therefore, improve the institution’s ability to draw lessons from its experience and more quickly integrate improvements into its future work.

Structure and Accountabilities

IEO will be independent of Fund management and staff and will operate at arm’s length from the Fund’s Executive Board. Its structure and modalities of operation must protect its operational independence—both actual and perceived.

A Director, to be appointed by the Executive Board, will head IEO. The Director’s term of appointment will be for a period of four years renewable for a second term of up to three years. The Director’s appointment may be terminated at any time with the approval of the Executive Board. At the end of the term of service, the Director will not be eligible for appointment or reappointment to the regular staff of the Fund, The Director will be responsible for the selection of IEO personnel (including external consultants) on terms and conditions to be determined by the Board with a view to ensuring that the office is staffed with independent and highly qualified personnel. The majority of full-time IEO personnel will come from outside the Fund.

Responsibilities

The Director of IEO will be responsible for the preparation of the Work Program. The content of the Work Program should focus on issues of importance to the Fund’s membership and of relevance to the mandate of the Fund. It should take into account current institutional priorities, and be prepared in light of consultations with Executive Directors and management, as well as with informed and interested parties outside the Fund. The Director will present IEO’s Work Program to the Executive Board for its review.

IEO, through its Director, will report regularly to the Executive Board, including through the preparation of an Annual Report. It is also expected that the IMFC will receive regular reports on the activities and findings of IEO.

With respect to individual evaluations, staff, management, and—when appropriate—the relevant country authorities will be given an opportunity to comment on the assessments being presented to the Executive Board.

The Director of IEO, in consultation with Executive Directors, will prepare a budget proposal for IEO for consideration and approval by the Executive Board. Its preparation will be independent of the budgetary process over which management and the Office of Budget and Planning have authority, but its implementation will be subject to the Fund’s budgeting and expenditure control procedures. IEO’s budget will be appended to that of the Executive Board within the Fund’s Administrative Budget.

If requested by the Executive Board, IEO will provide technical and administrative support for any external evaluations launched directly by the Executive Board.

Consultation, Publication, and External Relations

In carrying out its mandate, including in the preparation of its Work Program, IEO will be free to consult with whomever and whichever groups it deems necessary, both within and outside the Fund.

IEO will have sole responsibility for drafting IEO evaluations, Annual Reports, press releases, and other IEO documents or public statements.

IEO’s Work Program will be made public and there will be a strong presumption that IEO reports will be published promptly (within the constraints imposed by the need to respect the confidentiality of information provided to the Fund by its members), unless, in exceptional circumstances, the Executive Board were to decide otherwise.

Publication of evaluations will be accompanied by comments from management, staff, and others, including relevant country authorities, where appropriate, along with the conclusions reached by the Board in considering the evaluation report.

Relations with Fund Staff and Management

In conducting its work, IEO should avoid interfering with operational activities, including programs, or attempting to micro-manage the institution.

Review of Experience with IEO

Within three years of the launch of IEO operations, the Executive Board should initiate an external evaluation of IEO to assess its effectiveness and to consider possible improvements to its structure, mandate, operational modalities, or terms of reference. Without prejudging how that review would be conducted, it should be understood that the review would include the solicitation of broad-based input from outside the official community.

Appendix 2

Budget of the Independent Evaluation Office for FY2003 and FY20041

(U.S. dollars)

article image

FY2003 is May 2002 to April 2003 and FY2004 is May 2003 to April 2004.

Appendix 3 Initial Broad Menu of Potential Topics for Evaluation by IEO

The following subjects were identified in the first stage of developing the work program.

Surveillance

  1. The IMF’s role and effectiveness in crisis prevention based on an examination of recent Article IV reports, including all the recent capita! account crisis cases. Are “early warning” procedures effective, and do they influence the Fund’s advice and countries’ policies?

  2. Review of Financial System Stability Assessments (FSSAs). What lessons can be learned from the experience of the first two years? Are they identifying the key vulnerabilities and proposing remedies in a cost-effective manner?

  3. Follow-up to the 1999 external review of surveillance. Specifically, the review could focus on how the recommendations agreed to by the Executive Board have been implemented.

  4. Effectiveness of IMF surveillance of industrial countries. Is the scope of surveillance appropriate and what is its value added?

  5. The IMF’s role in multilateral surveillance, including the World Economic Outlook and the International Capital Markets Report exercises.

  6. Role and effectiveness of regional surveillance (e.g., European Community, other regional groupings).

  7. The IMF’s approach to liberalization of the capital account. Possible topics include whether the Fund’s policy advice on the pace of capital account liberalization and its sequencing with other reforms, especially vis-à-vis the financial sector, have been appropriate and consistent across countries. How has policy changed in the light of experience with capital account crises?

  8. The IMF’s work on standards and codes of good practice (in collaboration with other agencies). Is the approach effective in building institutional capacity and reducing vulnerability in member countries?

IMF-Supported Programs and Related Issues

Review of individual country programs

  1. Capital account crises (e.g., Argentina, Brazil, Ecuador, Indonesia, Korea, Thailand, Turkey). Some stakeholders have suggested that IEO should evaluate all programs where exceptionally large access to Fund resources is involved.

  2. Low-income/highly indebted cases (possible cases include Bolivia, Burkina Faso, Cameroon, Honduras, Mozambique, Nicaragua, Niger, Senegal, Tanzania, Uganda, Vietnam, Zambia).

  3. Transition countries (Bulgaria, Hungary, Poland, Russia, Ukraine, other FSU countries).

  4. Selected countries that have had repeat programs could be chosen (e.g., Bolivia, Madagascar, Philippines, Zambia). Are there problems with program design that contribute to such repeat usage?

Review of broad policy issues cutting across programs

  1. Exchange rate policies in IMF-supported programs and the Fund’s policy advice on exchange rates as part of surveillance. Has the Fund’s policy advice on exchange rate regimes and associated policies been consistent across countries? Has the design of exit strategies from exchange rate pegs been appropriate? Have the contractionary consequences of devaluations, stemming from their balance sheet consequences, been underestimated in program design? Has financial program design adapted effectively to inflation-targeting regimes?

  2. Fiscal adjustment in IMF-supported programs. Does fiscal adjustment take sufficient account of longer-term goals (e.g., for growth, poverty reduction) and is it implemented in a sustainable manner? Has the potential, immediate adverse impact of programs on particular vulnerable groups been adequately assessed and taken into account in program design?

  3. Policies toward financial sector stability and financial sector restructuring. In addition to the effectiveness of IMF policy advice in helping member countries avoid financial crises (also discussed above under surveillance), possible issues could include whether the design of financial sector restructuring packages has been appropriate, how the fiscal consequences have been handled, and how the macroeconomic consequences of corporate restructuring were taken into account in program design.

  4. Debt reduction and debt sustainability issues. Possible issues (either in the context of the heavily indebted poor country (HIPC) cases or more generally) could include whether the approach to assessing the sustainability or unsustainabilily of debt positions (external or public debt) has been appropriate and consistent across countries; whether debt sustainability analyses in surveillance reports have been adequate; what use has been made of the sustainability analysis; and what can be learned from “best practice” approaches.

  5. Has IMF support of a country’s program had a positive “catalytic effect” in terms of generating additional external financing flows within a specific lime frame? Are there objective measures of this catalytic effect? What factors influence the impact on market credibility?

  6. The nature and effectiveness of conditionality and issues involving the “ownership” of national/IMF-supported programs. Also, IMF policy toward structural conditionality has been modified recently; a review of the impact of the new policy could be undertaken at the end of the first two-year experience (i.e., in FY2004).

  7. Why do many IMF-supported programs remain uncompleted and what difference does it make? Are there particular aspects of program design (e.g., optimism of projections, extent of conditionality) that have a strong influence on the probability of completion? Do outcomes depend on the extent to which programs are completed and what lessons can be learned from incomplete programs?

  8. Conditionality with respect to trade policies. Has the IMF adopted a consistent approach to trade policies in the design of conditionality? Is the approach consistent between the surveillance stage and the program stage?

  9. Private sector involvement (PSI) in crisis resolution. This is a growing area of concern in which policy is still evolving but there are lessons to be learned from experience thus far. Does the existing experience suggest that some forms of PSI are likely to be more successful than others? Many of the questions involved are also closely related to the size of the IMF financial support, including through the Supplemental Reserve Facility (SRF) for countries undergoing crises that are centered primarily in the capital account.

  10. Experience with privatization in IMF-supported programs. Possible questions to be addressed could include: was the sequencing with regard to implementation of regulatory frameworks appropriate? How realistic was the time frame for privatization? What was the impact on prices for services and investment in the privatized sectors? And what was the social impact?

Review of experience with particular lending facilities and related issues

  1. The role of the Poverty Reduction and Growth Facility (PRGF) and the need to evolve special procedures to overcome problems in achieving staled objectives in countries with PRGF-sup-ported programs have been much discussed. Although an internal review is currently under way, there is scope for an independent review beginning perhaps in late FY2003. The review could address such issues as: Have all of the lessons from the ESAF reviews and “best practices” on particular policy issues been incorporated into PRGF adjustment strategies? Has the increased emphasis on country ownership resulted in real changes in the approach to negotiations? Has program design effectively incorporated the analysis of the social impact of major reforms? Have program design and monitoring improved the targeting of spending in key sectors relevant lo growth and poverty reduction? Have resources been effectively channeled to social sectors?

  2. An external review of the enhanced HIPC Initiative would he important once a “critical mass” of countries reach their completion points.

  3. The Contingent Credit Line (CCL) would also be a candidate for evaluation, but only after there is sufficient concrete operational experience with country cases.

  4. The strategy vis-á-vis member country arrears to the IMF

  5. The IMF’s role in countries emerging from conflict.

Technical Assistance and Training

  1. A follow-up to the 1999 internal review of technical assistance. This could include an assessment of whether there are effective and consistent internal systems for evaluating technical assistance advice and for setting priorities.

  2. Does technical assistance help improve national ownership of Fund-supported programs? Has it been effective in improving program implementation or in enhancing crisis prevention?

  3. Assessment of the effectiveness of technical assistance in areas such as:

    • Tax policy and revenue administration;

    • Expenditure policy and expenditure management;

    • Banking supervision/financial stability; and

    • Debt and external reserve management.

Internal IMF Processes and Governance

  1. Are the IMF’s internal review systems adequate? For example, are there adequate mechanisms for early internal reassessments of the adequacy of program design?

  2. Is World Bank—IMF collaboration effective? How can it be improved, given their distinct operational approaches and objectives? Is the division of labor/degree of overlap between the two institutions appropriate?

  3. Do staff papers on country programs contain the necessary information and analysis for the Board to make an informed judgment on the probability of success?

Research

Since the Research Department was being restructured when the work program was prepared, this was not viewed as a high-priority area for review. However, a follow-up on the recommendations of the 1999 External Evaluation of Research Activities could be considered for the longer term.

Appendix 4 IEO Outreach Seminars and Workshops

August 2001, London, United Kingdom

  • Consultations on IEO’s work program with nongovernmental organizations (NGOs), academics, and other stakeholders.

September 10-12, 2001, Dakar, Senegal

  • Meetings on IEO’s work program at PRSP/PRGF seminar.

September 13, 2001, Paris, France

  • Meeting on IEO’s work program with NGO representatives.

January 28, 2002, Tokyo, Japan

  • Outreach seminar on the IEO with Tokyo-based academics, NGOs, and government officials.

January 29, 2002, Tokyo, Japan

  • Workshop on the capital account crises evaluation with Tokyo-based experts.

February 9-10, 2003, Rome, Italy

  • Briefings on the IEO with various officials.

April 18, 2002, Washington, D.C., United States

  • Briefing on the IEO with NGO representatives.

July 1 and 2, 2002, Berlin, Germany

  • Joint lEO/German Foundation for International Development (DSE) workshop, including academics and various NGOs on the three initial evaluation projects.

August 5, 2002, Recife, Brazil

  • Meeting on the capital account crises evaluation with NGO representatives.

September 30, 2002, Washington, D.C., United States

  • Briefing on the IEO with NGO representatives.

October 29, 2002, Cambridge, Massachusetts, United States

  • Workshop with the National Bureau of Economic Research on all three projects.

November 7, 2002, Tokyo, Japan

  • Workshop at the ADB Institute on the prolonged use of IMF resources evaluation and the capital account crises evaluation.

November 8, 2002, Chiba, Japan

  • Outreach seminar at the Institute for Developing Economics on the prolonged use of IMF resources evaluation.

November 25, 2002, London, United Kingdom

  • Workshop with the Overseas Development Institute on both the prolonged use of IMF resources and PRSP/PRGF evaluations.

January 23, 2003, Manila, Philippines

  • Workshops on the Philippine case study in connection with the prolonged use of IMF resources evaluation with the Asian Institute of Management and government officials.

April 9, 2003, Washington, D.C, United States

  • Meeting on the update of the IEO work program with NGO representatives.

May 6, 2003, Dakar, Senegal

  • Outreach workshop on the prolonged use of IMF resources and PRSP/PRGF evaluations with government officials, civil society, and donors.

Appendix 5 Evaluation of Prolonged Use of IMF Resources: Recommendations, Executive Board Response, and Subsequent Follow-Up

article image
article image
article image
article image

This column summarizes the reaction of the Executive Board on each recommendation as reported in the summing up by the Acting Chair. Although care has been taken to ensure accuracy, readers are invited to refer to the full text of the summary of the discussion which is included in the published version of the report and can be accessed from the IEO website (www.imf.org/ieo).

Conclusions of the Task Force on Prolonged Use of Fund Resources, February 4. 2003. Available at www.imf.org/externaiynp/pdr/ufr/2003/020403.htm.

Including Board Discussion of Task Force conclusions. The column on follow-up is meant to provide factual information on additional steps taken after the Board discussion. It is not intended to be an evaluation of any follow-up by management or the Executive Board.

“Guidelines on Conditionality.” September 2002. Available at www.imf.org/External/np/pdr/eond/2002/eng/gurd/092302.htm.

Appendix 6

Evaluation of the Role of the IMF in Recent Capital Account Crises: Recommendations, Executive Board Response, and Subsequent Follow-Up

article image
article image
article image

This column summarizes the reaction of the Executive Board on each recommendation as reported in the summing up by the Acting Chair. Although care has been taken to ensure accuracy, readers are invited to refer to the full text of the summary of the discussion which is included in the published version of the report and can be accessed from the IEO website (www.imf.org/reo).

The column on follow-up is meant to provide factual information on additional steps taken after the Board discussion. It is not intended to be an evaluation of any follow-up by management or the Executive Board.

Appendix 7

Evaluation of Fiscal Adjustment in IMF-Supported Programs: Recommendations, Executive Board Response, and Subsequent Follow-Up

article image
article image
article image

This column summarizes the reaction of the Executive Board on each recommendation as reported In the summing up by the Acting Chair. Although care has been taken to ensure accuracy, readers are invited to refer to the full text of the summary of the discussion which is included in the published version of the report and can be accessed from the IEO website (www.lmf.org/ieo).

The column on follow-up is meant to provide factual Information on additional steps taken after the Board discussion. It is not intended to be an evaluation of any follow-up by management or the Executive Board.

  • Collapse
  • Expand