Work on two of the three projects identified for the FY2003–04 work program commenced in the course of FY2002–03. The status of all three projects is summarized in this chapter.
Evaluation of Poverty Reduction Strategy Papers (PRSPs) and the Poverty Reduction and Growth Facility (PRGF)
This evaluation focuses on the IMF’s role in the PRSP process and the related IMF lending instrument, the PRGF. A parallel evaluation of the World Bank’s experience with the PRSP is being undertaken by the World Bank’s OED. The IEO and the OED have reached agreement on a broad division of labor that will enable them to conduct the studies while collaborating in some aspects.12 The IEO will focus on the role of the IMF in the PRSP process and the experience with PRGF, while the OED study will focus on the World Bank’s role in the PRSP process. The common element in the two evaluations is the PRSP process. The IEO evaluation will focus on issues in the area of expertise of the Fund and will not assess important issues where the primary responsibility lies with the World Bank.
The IEO evaluation will seek to address three broad sets of questions:
-
Are the objectives of these initiatives suitably defined and is their design, with regard to aspects concerning the IMF, consistent with the achievement of sustainable progress?
-
Is the IMF delivering on its commitments embedded in the PRGF/PRSP approach and with what results?
-
What accounts for any shortcomings diagnosed (including systemic constraints) and what suggestions emerge from the evaluation about how they can be remedied as far as the IMF is concerned?
The evaluation will focus on the experience of countries with full PRSPs, as this has the greatest potential for generating lessons of importance for the IMF’s role in the future. At this stage of the process, the evaluation will not be able to cast much light on final outcomes for growth and poverty, which will only be known later. The focus of the evaluation will therefore be on inputs (the PRSP process and PRGF-supported programs’ formulation), outputs (i.e.. PRSP contents and PRGF-supported programs’ design), and intermediate effects (i.e., institutional and policy changes).
The scope of the evaluation and the set of issues that it will cover are illustrated by the logical framework diagram (see Figure 2) that maps the broad stages of what the PRSP/PRGF process is meant to achieve. The evaluation will focus on only those elements of the first three stages (i.e., process, outputs, and effects) that directly concern the IMF’s role.
Schematic Logical Framework of the Evaluation
-
Process issues to be evaluated (see the terms of reference for a more comprehensive discussion) would include the extent 10 which the process of program formulation, including the IMF’s internal procedures, have changed so as to align with the PRSP emphasis on country-driven, participatory approaches and to promote PRGF-supportcd programs that are embedded in homegrown strategies oriented toward growth and poverty reduction.
-
Outputs evaluated would include assessing whether PRSPs (and related Joint Staff Assessments) have the main components expected of them in the areas of the IMF’s responsibility, including with respect to the clarity of policy choices in macroeconomic and budgetary areas, and whether the programs supported by PRGF arrangements have been explicitly derived from the broader strategies for fostering growth and reducing poverty set out in the PRSPs.
-
Intermediate effects evaluated will involve assessing how the PRSP/PRGF process has affected policy formulation and implementation in the countries concerned. For example, are PRSPs integrated into budget and public expenditure management frameworks, and has a more participatory, country-driven process altered the types of policy trade-offs considered and the actual content of policy implementation in key areas?
-
It is too early for a systematic assessment of the impact of the PRSP/PRGF on final outcomes. However, the evaluation will collect available evidence on the evolution of various program outcomes (e.g., GDP growth and key macro indicators) compared, for example, with earlier ESAF-supported programs.
In evaluating the PRGF, the evaluation will assess performance in each of the seven key dimensions in which the PRGF was expected to improve upon earlier ESAF-supported programs:13
-
Broad participation and greater country ownership;
-
Embedding the PRGF-supported program in a broader set of measures set out in an overall strategy for growth and poverty reduction;
-
Government budgets that are more pro-poor and pro-growth;
-
Ensuring appropriate flexibility in fiscal targets;
-
More selective structural conditionality;
-
Emphasis on measures to improve public resource management/accountability; and
-
Social impact analysis of major macroeconomic adjustment and structural reforms.
Methodology and time frame
The evaluation will draw upon both detailed case studies and broader cross-country analysis. Six country case studies will be undertaken: Tanzania, Mozambique, Nicaragua, Tajikistan, Guinea, and Vietnam. The countries have been chosen to reflect diverse regional experiences and economic performance, and to offer a combination of recent and mature PRSPs and PRGF-supported programs, as well as of HIPC and non-HIPC countries. The case studies will involve both deskwork (including reviews of relevant IMF documentation, both published and unpublished, as well as reviews of external evaluative evidence) and field work. They will also seek the views of all relevant stakeholders through a combination of interviews and surveys. The studies will place particular emphasis on analyzing changes in the nature of the policy debate by following through several critical policy issues in each country and assessing how their treatment has evolved under the PRSP/PRGF initiative.14
Cross-country analyses will be undertaken on the full sample of 23 countries, which have a full PRSP as of December 2002. The purpose will be to test on a broader scale the relevance of the findings made in the case studies as well as to bring out other messages of general significance.
The field work for the country case studies has been completed, and the country reports are being drafted. The cross-country analysis has been initiated. The timetable of the evaluation contemplates submission of the final report to the IMF’s Executive Board around March 2004.
The Role of the IMF in Argentina, 1991-2002
This evaluation focuses on the role of the IMF in Argentina during the period from 1991 to early 2002. In December 2001–January 2002, Argentina experienced a devastating economic crisis, leading to a collapse in output, high levels of unemployment, and political and social turmoil. This crisis has raised serious questions for the IMF not only because it occurred while the country’s economic policies were under the close scrutiny of an ongoing IMF-supported program, but also because the IMF had also been continuously engaged in Argentina since 1991, when the “Convertibility Plan” fixed the currency at parity with the U.S. dollar. Over this period, the IMF approved four successive financing arrangements, and provided extensive technical assistance, dispatching some 50 missions between 1991 and 2002.
The evaluation will review the evolution of the IMF’s advice and internal views on key areas of Argentina’s economic policy, examine how the IMF came to certain decisions at critical junctures in its relationship with the country, and assess how reasonable the decisions were in light of information available at the time. The evaluation will also consider—albeit with the benefit of hindsight—if better outcomes could have been achieved had the IMF acted differently with a different set of decisions by the IMF. As is customary with all IEO evaluations, the primary focus will be placed on drawing lessons for the IMF.
The choice of the 1991 to early 2002 time period leaves out issues related to the role of the IMF during the sharp contraction of 2002 and the subsequent moves toward economic reconstruction and recovery, as the lEO’s terms of reference do not allow an evaluation of issues directly related to the IMF’s ongoing operations.
Issues for evaluation
There are competing explanations of the primary cause of the Argentine crisis, but there is general agreement that several factors were at play. These include: (1) an excessively lax fiscal policy; (2) excessive and unpredictable swings in the volume of global capital flows to emerging market economies; (3) the slow pace of structural reform in some critical areas; (4) institutional and political factors; and (5) the rigidity of the convertibility regime itself, which limited the scope for real exchange rate adjustment. Against this background, some additional factors helped to trigger the crisis and to exacerbate the impact of these underlying weaknesses; (6) a series of adverse external shocks; (7) the impact of slow growth and high interest rates on the prospective path of the ratio of debt to GDP; and (8) in the final stage of the crisis, a weakening of prudential defenses in the banking system. The evaluation will examine how the IMF viewed each of these vulnerabilities in its surveillance activities, what the impact of its surveillance was, and how effectively it designed its adjustment programs to address them.
The overarching questions the evaluation seeks to answer include:
-
What was the IMF’s diagnosis of conditions in Argentina and how did it evolve over time?
-
How was the IMF’s diagnosis translated into program design, conditionally, and financing decisions?
-
How effective were the IMF’s policy advice and conditionality in influencing the policies actually pursued by the authorities?
-
What were the roles of IMF staff, IMF management, and the authorities of large shareholder governments in the formulation of the IMF’s decisions?
Specific issues to be addressed in the evaluation will be divided into two broad categories, corresponding to the two related but distinct phases of IMF involvement: first, the period of 1991–2000 before the latest crisis, and second, the crisis and its immediate aftermath, 2000–2002.
Methodology and time frame
The evaluation will rely on the IMF’s internal and published documents, supplemented by interviews with IMF staff and other key decision makers in the IMF, current and former officials of the Argentine government and central bank, IMF shareholder governments, and the private sector; a review of the academic literature and public discussion; and data analysis, including a detailed review of fiscal accounts, cross-country analysis of fiscal policy, assessment of macroeconomic projections in program documents, and a simulation of debt dynamics. In order to obtain a broad perspective on issues, the views of interested parties in civil society will also be sought. The approach to be taken for the evaluation is explained in greater detail in the issues paper.
A draft issues paper was posted on the IEO website in June 2003, followed by the posting of the final issues paper in late July, both in English and Spanish. Extensive discussions were held in the early phase of the evaluation with IMF staff, members of the Board, former key Argentine policymakers, and other stakeholders, including those in Argentina. The report will be submitted to the Executive Board before the end of FY2004.
Technical Assistance by the IMF
This project, which will evaluate the IMF’s technical assistance (TA) activity, is at a very early stage. Work has commenced on preparing a draft issues paper, which will be circulated for comments (including via the website) from both inside and outside the IMF.
Some of the questions which the study will address are:
-
How effective are internal IMF processes for identifying TA priorities at the country level and for allocating resources across countries and subjects?
-
Is there enough complementarity between TA and other major IMF activities, such as surveillance and use of IMF resources, and how are these activities integrated in practice?
-
How effective has the IMF’s TA been in building institutional capacity and fostering sustainable reforms and how could these effects be improved? How is absorptive capacity taken into account and how effectively is the policy advice tailored to the circumstances of each country? How is transfer of knowledge emphasized and what measures are taken to make TA more goal oriented?
-
How effective are the Fund’s internal evaluation procedures and have recent initiatives led to a reallocation away from places in which it is not being used effectively?
The methodology and time frame of the study will be spelled out in the issues paper, which will be available in October 2003. The evaluation report is expected to be completed within FY2004.
An approach paper for the OED’s parallel evaluation is available on the OED’s website at www.worldbank.org/oed/prsp.
See “Key Features of IMF Poverty Reduction and Growth Facility (PRGF) Supported Programs,” IMF (2000). Available on the IMF website (www.imf.org/extcrnal/iip/prgf/2000/cng/key.htm).
The first four will be undertaken jointly with the OED and it is expected that a single country report will be prepared for each of these countries. In addition to the four joint country studies, the OED is also undertaking case studies of Albania, Cambodia, Ethiopia, and Mauritania.
