Chapter 2 Evaluation Projects Undertaken in FY2003–04
- International Monetary Fund. Independent Evaluation Office
- Published Date:
- March 2005
This chapter summarizes the main findings and recommendations of the two most recently completed evaluation reports along with the conclusions reached in the respective Executive Board discussions. It also briefly updates the status of the Technical Assistance evaluation, which is expected to go to the Board at the beginning of 2005.
Poverty Reduction Strategy Papers and the Poverty Reduction and Growth Facility
The IMF and the World Bank introduced Poverty Reduction Strategy Papers (PRSPs) in 1999 to encourage broader-based participation in the development of country-owned, long-term strategies for growth and poverty reduction in low-income countries. Concurrently, the IMF transformed its concessional lending facility into the Poverty Reduction and Growth Facility (PRGF). PRSPs were intended to provide the basis for IMF and World Bank assistance to their low-income members. In particular, PRGF-supported programs were expected to be derived from PRSPs.
The IEO’s evaluation focused on the role of the IMF in the Poverty Reduction Strategy (PRS) process and on the extent to which the PRGF is living up to the key features that were supposed to distinguish it from its predecessor—the Enhanced Structural Adjustment Facility. This evaluation was conducted in parallel with one by the World Bank’s Operations Evaluation Department (OED), which assessed the effectiveness of the World Bank’s support to the PRS process. Collaboration between the IEO and the OED included jointly undertaking a number of country case studies.10
The evaluation concluded that while the PRS approach has the potential to encourage the development of country-owned and credible long-term strategies for growth and poverty reduction, actual achievements thus far had fallen considerably short of potential. It attributed this outcome, in part, to shortcomings in the design of the initiative that have reduced its effectiveness, including a lack of clarity about the role that the IMF should play. More specific findings are summarized below.
Participation and ownership
Participation in the formulation of PRSPs was found to be generally more broadly based than in previous approaches, although it was typically not designed to strengthen existing domestic institutional processes for policy formulation and accountability (for example, through parliament). In terms of the outputs from this participatory process in areas of direct relevance to the IMF, there has been limited discussion of alternative policy options related to the macroeconomic framework and macro-relevant structural reforms. The report attributes this in part to the absence of mechanisms to ensure that key macroeconomic issues are aired.
Results in terms of ownership have been mixed, with the least change in macroeconomic policy areas. In these areas, there has been relatively strong ownership in a narrow circle of official stakeholders responsible for driving the process, but much less among other domestic stakeholders. There continues to be a widespread perception that the PRS approach is overly influenced by the procedural requirements of the Bretton Woods institutions.
Content of PRSPs and implementation issues
In general, strategies outlined in PRSPs were found to be an improvement over previous development strategies, in the sense of providing greater poverty focus, a longer-term perspective, and some orientation toward results. However, most PRSPs were found to fall short of providing a strategic road map for policymaking, especially in the area of macroeconomic and related structural policies. PRSPs often avoided addressing key strategic choices involving “controversial” structural reforms. Thus, in many cases, PRSPs do not yet provide a policy framework in which PRGF-supported programs can be anchored. In a few countries where the process is beginning to be embedded in domestic institutions, there are signs of feedback from initial implementation to policy design, but these remain a minority.
Capacity constraints have impeded implementation. There has been insufficient attention to developing a systematic plan of action to strengthen capacity, including in the IMF’s areas of primary competence. For example, budgetary processes are weak, and the linkages between the PRSP, medium-term expenditure frameworks, and budgets are generally poor. In particular, public expenditure management systems are generally too weak to allow the PRSP to play a central role in implementing expenditure priorities or modifying them on the basis of feedback on actual costs and outcomes.
Joint Staff Assessments
The evaluation found that, on balance, Joint Staff Assessments (JSAs) have not adequately performed the many tasks expected of them. Their main contribution has been in giving feedback to country authorities on weaknesses in PRSPs, but they are virtually unknown outside narrow official circles and consequently have had no impact on broad policy debates in countries. They also do not incorporate systematic inputs from development partners and, in practice, have played a limited role in informing lending decisions, including those of the Bretton Woods institutions. Factors limiting the usefulness of assessments include the lack of explicit benchmarks in most areas on which to base assessments and the fact that they were constrained to reach a binary (yes or no) conclusion on whether the strategies presented in PRSPs constituted a sound basis for concessional lending by the two institutions.
The role of the IMF
The effectiveness of the IMF’s contribution has varied considerably across different components of the PRS initiative and across countries, with marked differences between “good” and “average” practice. Its overall contribution has fallen well short of the very ambitious goals set in the original policy documents. In particular:
IMF staff typically did not participate actively to inform the policy debate among domestic stakeholders during the PRS formulation process; in fact, IMF staff generally interpreted the emphasis on country ownership as implying that involvement on their part should be limited.
IMF contributions to developing a better understanding of country-specific micro-macro linkages have also been fairly limited. Although the process has led to much greater awareness within the IMF of the need for ex ante poverty and social impact analysis—with some “good practice” exceptions—this has not yet translated into general use of such analysis in program design. Part of the problem is that the PRS process itself does not yet generate sufficient signals—or accountability—on what the Bretton Woods institutions themselves should be delivering in terms of capacity-strengthening priorities.
On the positive side, there are signs that the “policy space” in the macroeconomic area has widened—in the sense of greater openness on the part of the IMF to considering alternative country-driven policies—at least in countries where macroeconomic stabilization is no longer a pressing issue.
Success in embedding the PRGF in the overall strategy for growth and poverty reduction has been limited in most cases, partly reflecting shortcomings in the strategies themselves. Nevertheless, program design under the PRGF has incorporated greater fiscal flexibility to accommodate aid flows, and there is no evidence of generalized “aid pessimism” or a systematic “disinflation” bias. Expenditures designated as poverty reducing have increased markedly since 1999, although there are questions about how “pro-poor” some of this spending is. IMF structural conditionality has been streamlined, but the evaluation was not able to reach a definitive conclusion on what has happened to aggregate IMF–World Bank conditionality, which is not monitored systematically by the institutions. There were only minor improvements in various measures of program implementation under the PRGF.
Measures of the quality of policies and institutions by the IMF and the World Bank suggest that countries adopting poverty reduction strategies generally started out in a better position, but did not improve at a faster pace, than those low-income countries that did not adopt such strategies.
Short-term growth for PRSP/PRGF countries is only marginally higher than in the earlier period. However, these countries seem to have weathered the worsening of the external environment in 2000–02 better than other low-income countries.
Evidence on poverty-related outcomes, drawn from the parallel OED evaluation, is still too limited to reach definitive conclusions. The most notable improvements concern various input- and output-related measures (for example, number of teachers, school enrollment, and vaccination rates), but outcomes such as maternal and infant mortality rates have generally not improved.
The report makes six broad recommendations: three on aligning incentives with the objectives of the PRSP/PRGF approach, and three on improving the IMF’s effectiveness.
Aligning incentives and objectives
Recommendation 1. Introduce greater flexibility in the implementation of the PRS approach to fit better the needs of countries at different stages of the process and with different capacities and political and administrative systems.
Countries need to be put even more firmly in the driver’s seat by determining themselves:
(i) how the policy formulation, implementation, and monitoring processes will be conducted and built up over time, and with what rules of the game;
(ii) what the output of these processes will be in terms of documents (for example, PRSP, Progress Reports, and the like) and on what periodicity they will be prepared, relying as much as possible on domestic institutional arrangements and reporting vehicles.
IMF process requirements, such as linking reviews under the PRGF to completion of specific PRSP documents, should be minimized and oriented around domestic processes so that they do not conflict with domestic timetables and/or duplicate domestic instruments.
Recommendation 2. Shift the emphasis of the initiative from the production of documents to the development of sound domestic policy formulation and implementation processes.
Implementation of this recommendation would involve the following elements:
(i) As a way of building in a greater orientation toward results, countries should be encouraged to establish—with help from the IMF and the World Bank where needed—substantive criteria for judging progress toward key intermediate objectives.
(ii) A shift in the emphasis of the incentives structure faced by countries from procedural changes and production of documents to achieving substantive changes in domestic processes and policies. The new set of incentives would have the following elements:
Transparency. Countries should present their intentions and objectives, along with the benchmarks selected to monitor progress, in a manner open to public scrutiny.
Accountability. IMF (and World Bank) staff would be responsible for providing clear and candid assessments of the progress made by each country in implementing the PRS approach, both in relation to the goals set by the country itself and against initiative-wide benchmarks.
Support by the Bretton Woods institutions. IMF (and World Bank) staff would help countries identify key constraints in making progress toward PRS objectives and support efforts to ameliorate them.
Selectivity. Donor decisions on the volume of resources provided should be linked to the progress countries are making under the approach. To facilitate this, IMF assessments in its area of expertise need to provide as clear and candid a signal as possible. The criteria guiding the IMF’s own lending decisions under the PRS approach could also be improved in this regard.
Recommendation 3. Clarify the purpose of the JSAs and redefine the vehicle accordingly.
The report’s recommendations on JSAs include the following:
(i) JSAs should focus on the adequacy of domestic policy choices and the quality of domestic processes as well as on progress toward intermediate objectives, and less on the quality of the PRSP as a document.
(ii) In order to foster clear and candid assessments, JSAs should make explicit the criteria and benchmarks used by IMF staff to form their judgments, report the views of third parties (especially local stakeholders and donors) when available and discuss any differences of view, and eliminate the need to reach a binary (yes or no) conclusion as to the adequacy of the PRS process as a basis for concessional lending by the IMF and the World Bank. JSAs should also aim to provide a graduated assessment of the strength of the PRSP and related processes as well as of the quality of policies.
Clarifying the IMF’s role and improving its effectiveness
Recommendation 4. Clarify what the PRS approach implies for the IMF’s own operations and strengthen the implementation of the agreed role.
With respect to engagement in the PRS process, more emphasis should be given to IMF activities that facilitate the IMF’s participation in broad-based policy discussions in its areas of competence. The IMF’s role should be tailored to country-specific circumstances, including to the government’s wishes; the resulting country-specific “rules of the game” should be made public and could describe how IMF staff is expected to participate in the broader policy debate.
With respect to PRGF-related activities, the rationale for IMF policy recommendations and program design should be subjected to broader scrutiny and debate. Possible steps in this direction could include (i) facilitating wider dissemination and discussion of IMF analysis that forms the basis of its policy advice (including, where appropriate, TA reports); and (ii) encouraging greater openness to “independent/external voices” as inputs into program design, when agreeable to the authorities.
There is a need to clarify the approach to be taken by the IMF in those cases where the PRS approach has added some value but has not yet produced an operational road map or the necessary institutional framework for implementation. Even in these cases, there may be significant scope for opening up the policy space and more systematically incorporating evidence on macro-micro linkages, including through poverty and social impact analysis.
Recommendation 5. Strengthen prioritization and accountability on what the IMF itself is supposed to deliver within the broader partnership framework, built around the priorities emerging from the PRS process, and ensure resources match commitments.
The IMF should tailor its involvement more closely to country needs, taking into account contributions from other partners. This can be done by generating, as part of the PRS process, specific priority actions for the IMF to assist the country concerned to reach its national objectives. The IMF’s own budgetary decisions on the allocation of administrative resources would then be geared to these priorities.
Recommendation 6. The IMF should encourage a strengthening of the framework for establishing the external resource envelope as part of the PRS approach.
A country itself, not the IMF or the World Bank, should eventually play the central role in elaborating macro frameworks and catalyzing donor support. The pace at which this transition can be made will depend on specific capacity constraints in each country, but country leadership seems essential to “owning” the process. The IMF’s role would be to provide debt and macroeconomic sustainability assessments and judgments on the policy framework, but the IMF would not be responsible for the “normative” judgment on appropriate aid levels over the medium term.
The tension between “ambition” and “realism” in determining the external resource envelope could be handled by presenting alternative projections (consistent with assumptions of stronger policy reforms as well as additional external financing). The IMF should provide increased analytical support for such approaches when requested, but the choice to prepare alternative projections should remain with the country and should not be a uniform requirement.
Executive Board response11
Executive Directors welcomed the report as a valuable contribution to the ongoing review of how to improve the effectiveness of the IMF’s engagement with low-income countries. While most Directors considered that the PRS approach has had a positive impact on economic policy design and implementation, they stressed that substantial scope exists for better implementation of the current approach, based on the evolving experience and the directions of change identified in the IEO report. At the same time, Directors cautioned against drawing premature conclusions about the ultimate success of the PRS approach based on only five years of experience with its implementation. They encouraged the staff to draw on the IEO’s recommendations to deepen and refine its analysis on ways to enhance the effectiveness of the PRS approach. They looked forward to the recommendations of the management-led committee on low-income country work to provide new impetus and focus to IMF work on low-income countries.
The Executive Board’s responses to specific recommendations are summarized below.
Greater flexibility in implementation of PRS approach
Directors agreed that the PRS approach should be implemented pragmatically and flexibly, bearing in mind country-specific circumstances and capacity constraints. They stressed the need to ensure that IMF-supported programs are designed to assure macroeconomic stability as well as to help members accelerate the pace of progress toward the Millennium Development Goals (MDGs). They indicated that the PRGF should be tightly linked to PRSPs that provide a sound operational road map. Where PRSPs are not yet operationally viable, the IMF should not insist on immediate tight alignment. Instead, IMF staff should focus on working with the countries to strengthen the macroeconomic frameworks in their PRSPs in order to move toward eventual alignment. At the same time, Directors cautioned that increased flexibility should not imply delinking the PRGF from the PRS process, and noted that the IMF would still seek to apply the PRSP principles in its program work.
Shift emphasis to the development of sound domestic processes
Directors agreed that there should be less emphasis on document preparation, and more emphasis on improving the capability of countries to develop and implement policies supportive of growth and poverty reduction. Some Directors agreed with the recommendation that countries should set explicit criteria for judging progress toward key intermediate objectives and that IMF and World Bank staff should provide candid assessments of those benchmarks. Many other Directors cautioned against excessive IMF involvement in assessing a country’s decision-making processes, which they feared would establish an unwarranted direct linkage between such assessments and IMF lending decisions, and which could undermine the legitimacy of domestic institutions and processes. Directors noted that further discussion would be needed on how the IMF should react in cases where it believes that the pace of progress chosen is not ambitious enough.
Joint Staff Assessments
Directors called for a reformulation of the JSA approach to emphasize graduated rather than binary assessments, with the objective of providing candid feedback to countries. They looked forward to discussing specific recommendations presented by the staff in the context of the annual Fund-Bank PRSP Progress in Implementation report.
Implications of the PRS approach for IMF operations
Directors agreed that the IMF needs to more clearly set out its role in the PRS approach in each country, based on the IMF’s core mandate in macro-economic and related structural policy issues. Many Directors supported a more active role for IMF staff in the public debate on macroeconomic policy design and implementation, but others thought a more proactive role would not be appropriate since it could be seen as influencing the political decision-making process of a country.
Strengthened prioritization and IMF accountability within the broader partnership framework
Directors welcomed the IEO report’s emphasis on the need to define priorities for the work of the IMF in low-income countries. They indicated that the prioritization of budget resources must be guided by the IMF’s overall mandate. They called for a careful assessment of the resource implications of adapting the IMF’s role along the lines of the report’s recommendations, and looked forward to staff views on ways to improve the IMF’s involvement in the PRSP in the context of forthcoming Executive Board discussions of the annual Fund-Bank PRSP Progress in Implementation report and the review of the resident representative program.
Framework for establishing external resources envelope
Directors indicated that the IMF should play a supportive role with donors and low-income members to help ensure adequate provision of aid to achieve the MDGs. In this regard, the IMF needs to consider how its signals can be clear and useful to its members. In particular, IMF signals should not lead to inappropriate interruption of long-term development and poverty reduction finance.
The Role of the IMF in Argentina, 1991–2001
The Argentine crisis of 2000–02 was among the most severe of recent currency crises. In December 2001, Argentina defaulted on its sovereign debt and soon afterwards abandoned the convertibility regime, under which the peso had been pegged at parity with the U.S. dollar since 1991. The crisis had a devastating economic and social impact, causing many observers to question the role played by the IMF over the preceding decade when it was almost continuously engaged in Argentina through five successive financing arrangements.
The evaluation report examined the role of the IMF in Argentina during 1999–2001, taking advantage of the IEO’s unique access to internal documents, in order to draw lessons for the IMF in improving both its surveillance and crisis management capabilities in the future. In accordance with the IEO’s terms of reference, the report did not assess issues that would have a direct bearing on ongoing operations and hence did not discuss developments later than the first few days of 2002.
Argentina’s convertibility regime was a stabilization device to deal with the hyperinflation that existed at the beginning of the 1990s, and in achieving this goal, it was very successful. The regime was also part of a larger Convertibility Plan, which included a broader agenda of market-oriented structural reforms designed to promote efficiency and productivity in the economy. Under the Convertibility Plan, Argentina saw a marked improvement in its economic performance, particularly during the early years. Inflation, which was raging at a monthly rate of 27 percent in early 1991, declined to single digits in 1993 and remained low thereafter. Growth was solid through early 1998, except for a brief setback associated with the Mexican crisis, and averaged nearly 6 percent. Attracted by a more investment-friendly climate, Argentina experienced large capital inflows in the form of portfolio and direct investments.
These impressive gains, however, masked emerging vulnerabilities, which came to light when a series of external shocks began to hit Argentina and caused growth to slow down in the second half of 1998. Fiscal policy, though improved from the previous decades, led to a steady increase in the stock of debt, much of which was foreign currency denominated and externally held. The convertibility regime ruled out nominal depreciation when a depreciation of the real exchange rate was warranted by, among other things, the sustained appreciation of the U.S. dollar and the devaluation of the Brazilian real in early 1999. Deflation and output contraction set in, while Argentina faced increasingly tighter financing constraints amid investor concerns over fiscal solvency.
The crisis resulted from the failure of Argentine policymakers to take necessary corrective measures sufficiently early, particularly in the consistency of fiscal policy with their choice of exchange rate regime. Moreover, these policymakers had ownership in the fundamental policy choices, including in particular the commitment to the convertibility regime. However, the IMF on its part erred in the precrisis period by supporting the country’s weak policies too long, even after it had become evident in the late 1990s that the political ability to deliver the necessary fiscal discipline and structural reforms was lacking. By the time the crisis hit Argentina in late 2000, there were grave concerns about the country’s exchange rate and debt sustainability, but there was no easy solution. Given the extensive dollarization of the economy, the costs of exiting the convertibility regime were already very large. The IMF supported Argentina’s efforts to preserve the exchange rate regime with a substantial commitment of resources, which was subsequently augmented on two occasions. This support was justifiable initially, but the IMF continued to provide support through 2001 despite growing signs that the existing policy regime was unsustainable and despite repeated policy inadequacies. In retrospect, the resources used in an attempt to preserve the policy regime during 2001 could have been better used to mitigate at least some of the inevitable costs of exit.
Exchange rate policy. Although the IMF was initially skeptical of the convertibility regime’s medium-term viability, its internal views as well as public statements became much more upbeat when Argentina—with financial support from the IMF—successfully weathered the aftermath of the Mexican crisis, at which time the IMF endorsed the convertibility regime as essential to price stability and fundamentally viable. Following the devaluation of the Brazilian real in early 1999, IMF staff began to consider more seriously the viability of the peg and possible exit strategies but, by this time, the risks and costs associated with any exit were already very high. Throughout the precrisis period, little substantive discussion took place with the authorities or at the Executive Board on Argentina’s exchange rate policy.
Fiscal policy. The choice of the convertibility regime made fiscal policy especially important. Fiscal policy was rightly the focus of discussion with the authorities throughout the period, but the IMF’s analysis was handicapped by its focus on annual deficits, insufficient attention to the provincial finances, and overestimation of the sustainable level of public debt for a country with Argentina’s characteristics (such as a small export sector, a small tax revenue base, high interest rates, and small domestic capital markets). Enforcement of fiscal conditionality was also weak. While fiscal policy improved from previous decades, the initial gains were not sustained and the election-driven increase in public spending led to a sharp deterioration in fiscal discipline in 1999. As a result, the stock of public debt steadily increased, which diminished the ability of the authorities to use countercyclical fiscal policy when the recession deepened.
Structural reforms. The IMF correctly identified structural fiscal reforms, social security reform, labor market reform, and financial sector reform as essential to enhancing the medium-term viability of the convertibility regime by promoting fiscal discipline, flexibility, and investment. Most of the initiatives for reform in these areas came from the authorities, with the role of the IMF largely limited to providing TA in the fiscal areas. In fact, the remarkable feature of the successive IMF-supported programs with Argentina was the paucity of formal structural conditionality. Some gains were made in the early years, but the long-standing political obstacles to deeper reforms proved formidable. Little progress was made in later years, but the lack of strong structural conditionality fostered a continued program engagement with Argentina, when the evident lack of substantive progress in structural reform should have called for an end to the program relationship.
Crisis management, 2000–01
In the fall of 2000, Argentina effectively lost access to voluntary sources of financing. The authorities approached the IMF for a substantial augmentation of financial support under the Stand-By Arrangement (SBA) approved in March 2000, which up to that time had been treated as precautionary. In response, from January to September 2001, the IMF made three decisions to provide exceptional financial support to Argentina, raising its total commitments to US$22 billion. In December 2001, however, the fifth review of the program was not completed, which marked the effective cutoff of IMF financial support. The evaluation report assesses the key decisions made by the IMF during this period, bearing in mind the information available to the staff, management, and the Executive Board at the time.
The augmentation decision in January 2001. The decision to augment the existing arrangement was based on the diagnosis that Argentina faced primarily a liquidity crisis and that any exchange rate or debt sustainability problem was manageable with strong action on the fiscal and structural fronts. The IMF was well aware that the costs of a fundamental change in the policy framework would be very large. It wished to give the authorities the benefit of the doubt, when they were evidently committed to making strong policy corrections. Given the probabilistic nature of any such decision, the chosen strategy may well have worked if the assumptions had turned out to be correct (which they were not) and if the agreed program had been impeccably executed by the authorities (which it was not). The critical error was the failure to have an exit strategy, including a contingency plan, in place, inasmuch as the strategy was known to be highly risky.
The decisions to complete the third review in May and to further augment the arrangement in September 2001. While these decisions also had to be made in circumstances of great uncertainty, the weak implementation of the program in early 2001 and the adoption—without consultation with the IMF—of a series of controversial and market-shaking measures by the authorities after March 2001 should have provided ample ground for concluding that the initial strategy had failed. In fact, even within the IMF, there was an increasing recognition that Argentina had an unsustainable debt profile, an unsustainable exchange rate peg, or both. Yet, no alternative course of action was presented to the Board, and decisions were made to continue disbursing funds to Argentina under the existing policy framework, on the basis of largely noneconomic considerations (such as to make sure that the country, and not the IMF, took responsibility for the critical decisions needed), and in hopes of seeing a turnaround in market confidence and buying time until the external economic situation improved.
The decision not to complete the review in December 2001. After the September augmentation, economic activity and market confidence continued to collapse, making it virtually impossible to achieve the program’s targets and salvage convertibility. While aware of this predicament, the IMF did not press the authorities for a fundamental change in the policy regime and announced in early December that the pending review could not be completed under the circumstances. Within a month of this announcement, economic, social, and political dislocation occurred simultaneously, leading to the resignation of the President, default on Argentina’s sovereign debt, and the abandonment of convertibility, soon followed by government decisions that further amplified the costs of the collapse of convertibility. In those circumstances, the IMF was unable to provide much help and largely stood by as the crisis unraveled.
The decision-making process. Several weaknesses were evident in the IMF’s decision-making process. First, contingency planning by IMF staff was insufficient. Too much attention was given to determining—inconclusively—which alternative policy framework should be recommended to the authorities, while little effort was made to determine what practical steps the IMF should take if the chosen strategy failed. Second, from March 2001 onward, the relationship between the IMF and the authorities became less cooperative, with the authorities taking policy initiatives that the IMF viewed as misguided but felt compelled to endorse. The authorities remained unwilling to discuss any alternative plan. Third, insufficient attention was paid to the risks of giving the authorities the benefit of the doubt beyond the point where sustainability was clearly in question. Fourth, the Executive Board did not fully perform its oversight responsibility, exploring the potential trade-offs between alternative options. To some extent, this reflected the fact that some key decisions took place outside the Board and that some critical issues were judged by management to be too sensitive for open discussion in the full Board.
Lessons from the Argentine crisis
The Argentine crisis yields a number of lessons for the IMF, some of which have already been incorporated into revised policies and procedures. The evaluation identified 10 lessons.
Surveillance and program design
Lesson 1. While the choice of exchange rate regime belongs to country authorities, the IMF must exercise firm surveillance to ensure that the choice is consistent with other policies and constraints. Candid discussion of exchange rate policy, particularly when a fixed peg is involved, must become a routine exercise.
Lesson 2. The level of sustainable debt for emerging market economies may be lower than had been thought, depending on a country’s economic characteristics. The conduct of fiscal policy should be sensitive to the overall stock of public debt.
Lesson 3. The authorities’ decision to treat an arrangement as precautionary should not weaken standards for IMF support. Weak program design and weak implementation in the context of arrangements being treated as precautionary do not help a country address its potential vulnerabilities. When there is no balance of payments need and there are serious political obstacles to needed policy adjustment or reform, it may be better not to agree to an arrangement, thus subjecting the country to market discipline rather than to program reviews by the IMF.
Lesson 4. Emphasis on country ownership in IMF-supported programs can lead to an undesirable outcome, if ownership means misguided or excessively weak policies. The IMF should be prepared not to support strongly owned policies if it judges they are inadequate to generate a desired outcome.
Lesson 5. Favorable macroeconomic performance, even if sustained over some period of time, can mask underlying institutional weaknesses that may become insuperable obstacles to any quick restoration of confidence, if growth is disrupted by unfavorable external developments. The IMF may have only a limited role to play when institutional weaknesses are deeply rooted in the political system, and structural conditionality cannot substitute for domestic ownership of the underlying reforms.
Lesson 6. Decisions to support a given policy framework necessarily involve a probabilistic judgment, but it is important to make this judgment as rigorously as possible, and to have a fallback strategy from the outset in case some critical assumptions do not materialize.
Lesson 7. The catalytic approach to the resolution of a capital account crisis works only under quite stringent conditions. When there are well-founded concerns over debt and exchange rate sustainability, it is unreasonable to expect a voluntary reversal of capital flows.
Lesson 8. Voluntary, market-based debt restructuring is costly and unlikely to improve debt sustainability if it is undertaken under crisis conditions and without a credible, comprehensive economic strategy. Only a form of debt restructuring that leads to a reduction of the net present value of debt payments or, if the debt is believed to be sustainable, a large financing package by the official sector have a chance to reverse unfavorable debt dynamics.
Lesson 9. Delaying the action required to resolve a crisis can significantly raise its eventual cost, as delayed action can inevitably lead to further output loss, additional capital flight, and erosion of asset quality in the banking system. To minimize the costs of any crisis, the IMF must take a proactive approach to crisis resolution, including providing financial support to a policy shift, which is bound to be costly regardless of when it is made.
The decision-making process
Lesson 10. In order to minimize error and increase effectiveness, the IMF’s decision-making process must be improved in terms of risk analysis, accountability, and predictability. A more rule-based decision-making procedure, with greater ex ante specification of the circumstances in which financial support will be available, may facilitate a faster resolution of a crisis, though the outcome may not always be optimum. Recent modifications to the exceptional access policy have already moved some way in this direction.
On the basis of these lessons, the evaluation offered six sets of recommendations in order to strengthen the initiatives already being taken.
Recommendation 1. The IMF should have a contingency strategy from the outset of a crisis, including in particular “stop-loss rules”—that is, a set of criteria to determine if the initial strategy is working and to guide the decision on when a change in approach is needed.
Recommendation 2. Where the sustainability of debt or the exchange rate is in question, the IMF should indicate that its support is conditional upon a meaningful shift in the country’s policy while remaining actively engaged to foster such a shift. High priority should be given to defining the role of the IMF when a country seeking exceptional access has a solvency problem.
Recommendation 3. Medium-term exchange rate and debt sustainability should form the core focus of IMF surveillance. To fulfill these objectives (which are already current policy), the IMF needs to improve tools for assessing the equilibrium real exchange rate that are more forward looking and rely on a variety of criteria, examine debt profiles from the perspective of “debt intolerance,” and take a longer-term perspective on vulnerabilities that could surface over the medium term.
Recommendation 4. The IMF should refrain from entering or maintaining a program relationship with a member country when there is no immediate balance of payments need and there are serious political obstacles to needed policy adjustment or reform.
Recommendation 5. Exceptional access should entail a presumption of close cooperation between the authorities and the IMF, and special incentives to forge such close collaboration should be adopted, including mandatory disclosure to the Board of any critical issue or information that the authorities refuse to discuss with (or disclose to) the staff or management.
The decision-making process
Recommendation 6. In order to strengthen the role of the Executive Board, procedures should be adopted to encourage: (i) effective Board oversight of decisions under management’s purview; (ii) provision of candid and full information to the Board on all issues relevant to decision making; and (iii) open exchanges of views between management and the Board on all topics, including the most sensitive ones. These initiatives will be successful only insofar as IMF shareholders—especially the largest ones—collectively uphold the role of the Board as the prime locus of decision making in the IMF. Some possible steps to modify Board procedures to strengthen governance are discussed in the concluding section of the report.
Executive Board response12
The Executive Board discussed the report on July 26, 2004. The Board welcomed the report and broadly endorsed the thrust of its findings, lessons, and recommendations. While noting that many of the report’s recommendations were in line with policies and reforms that had recently been implemented, Executive Directors stressed the importance of making further progress in incorporating them into the IMF’s operations and decision-making process.
Crisis management. Most Directors viewed contingency planning as useful. However, many Directors noted the difficulty of assessing the various contingencies that might occur in a crisis or precrisis setting and that an element of prompt adaptation to rapidly evolving events is unavoidable. Concern was also expressed that any indication that the IMF was developing a contingency plan could undermine confidence in the program. As regards “stop-loss” rules, most felt that defining and implementing such rules would be difficult or impractical. Directors agreed with the report’s recommendation that the IMF’s financial support should be conditional upon a meaningful shift in the country’s policy when the sustain-ability of debt or the exchange rate is in question.
Surveillance. The Board agreed with the report’s recommendation that medium-term exchange rate and debt sustainability analyses should form the core focus of surveillance. Directors stressed the need for greater candor in the treatment of exchange rate policy during Article IV discussions, both in meetings with the authorities and in information presented to the Executive Board. They suggested that IMF staff should explore the scope to establish procedures for handling sensitive topics during surveillance exercises in order to strike an appropriate balance between candor and confidentiality. As to debt sustain-ability analysis, while noting the progress made thus far, Directors asked the staff to continue to sharpen its analytical tools.
Program relationship. While noting the possible risks associated with precautionary arrangements, most Directors did not agree that standards for such arrangements tended to be weaker. As regards the presumption of close cooperation between the IMF and country authorities under exceptional access, Directors stressed that all cases of the use of IMF resources should entail such presumption and encouraged management and staff to keep the Board fully informed of the state of program discussions. Many Directors agreed with the report’s recommendation that there should be a requirement of mandatory disclosure to the Board of any critical issues that the authorities refuse to discuss.
The decision-making process. Directors were concerned with the report’s assessment of the IMF’s decision-making procedures during the crisis, especially the role of the Board. A number of Directors saw a need for further discussion of approaches to strengthen the Board’s role. While noting some progress, including the procedures for exceptional access adopted since the Argentina crisis, Directors called for further efforts to enhance decision making by the Board, including improving the provision of full information on all issues relevant to decision making, and opening exchanges of views between management and the Board on all topics, including the most sensitive ones.
Technical Assistance Provided by the IMF13
The IMF has provided a significant amount of TA to member countries over the years. In terms of person years, direct TA activities account for about 12 percent of the institution’s administrative budget. The objectives of TA are twofold: (i) to support the efforts of members to strengthen their capacity—in both human and institutional resources—to formulate and implement sustainable, growth-oriented, and poverty-reducing policies; and (ii) to assist countries in designing appropriate macroeconomic and structural policy reforms, taking into account the lessons learned by other countries in addressing similar economic policy concerns.
This evaluation is in its final stage, and the report will be presented to the IMF Executive Board in early 2005. The evaluation will be addressing the following set of questions as well as providing recommendations on:
What factors determine the allocation of TA resources across countries? How are TA priorities selected at the country level and across countries, and how well are they related to countries’ broader economic policy strategies? How is the balance between short-term TA (to support the policy advice of the IMF) and long-term TA (to support institution building) being established? What can be done to make the prioritization of TA more strategic?
What can be learned about specific modalities of TA delivery and ways to interact with authorities during its implementation in order to increase the effectiveness of TA? What can be done to enhance the participation of the authorities in the design of TA activities?
What has been the impact of TA, and how is progress being tracked and monitored? What are the key factors (under the control of the IMF or the authorities) that enhance the impact of TA? How effectively does the allocation and design of TA respond to such feedback?
In order to address these questions, the evaluation uses different data sets and methodologies, including cross-country and time series allocations of field time devoted to TA, desk studies to assess the link between strategic documentation (such as Article IV reports and PRSPs) and a particular TA program, and interviews with IMF staff. The evaluation also includes six in-depth country studies designed to elicit the views of authorities and local counterparts. As part of the case studies, field visits have been made to Cambodia, Honduras, Niger, Ukraine, Yemen, and Zambia.
Four joint country case studies (Mozambique, Nicaragua, Tajikistan, and Tanzania) were undertaken. In addition, both institutions’ evaluations drew on country case studies prepared separately by the IEO (Guinea and Vietnam) and the OED (Albania, Cambodia, Ethiopia, and Mauritania). For all ten case studies, a joint IEO/OED survey of local stakeholders was undertaken. A compendium volume providing summaries of all ten case studies is being published and the six full case studies in which IEO was directly involved are available in English and relevant other languages on the IEO website at www.imf.org/External/NP/ieo/2004/prspprgf/eng/index.htm.
The full Summing Up of the Executive Board discussion is available on the IEO website at www.imf.org/External/NP/ieo/2004/prspprgf/eng/index.htm. Staff and management responses to the evaluation report are also available on the website.
The full Summing Up of the Executive Board discussion is available on the IEO website at www.imf.org/External/NP/ieo/2004/arg/eng/index.htm. Staff and management responses to the evaluation report as well as a statement to Executive Board members from the Argentine authorities are also available on the website.
A detailed issues paper/terms of reference for the evaluation is available on the IEO website at www.imf.org/external/np/ieo/2003/tas/tasip.htm.