Abstract

Administrative Budget: Independent Evaluation Office

Appendix 1

Administrative Budget: Independent Evaluation Office

(In U.S. dollars)

article image

The figures for FY2014 were estimated by the Office of Budget and Planning.

Resources carried forward from the previous year under established rules, aside from FY2014 when a higher carry-forward was approved on a one-time, exceptional basis.

Appendix 2 Outreach Activities

October 2012, Tokyo, Japan

Consultations about IEO’s future work program with country delegations during the Annual Meetings.

November 2012, London, United Kingdom

Presentation of the Governance of the IMF and IMF Performance in the Run-Up to the Financial and Economic Crisis evaluations at the European Bank for Reconstruction and Development.

December 2012, Paris, France

Presentation of the results of the Crisis evaluation to Ambassadors at the Organization for Economic Cooperation and Development.

February 2013, Washington, D.C., USA

Seminar for the IMF Executive Board on how the IEO works to support the Board’s institutional governance and oversight responsibilities.

February 2013, Beijing, China

Presentation of the evaluation of International Reserves: IMF Concerns and Country Perspectives at the Chinese Academy of Social Sciences.

February 2013, Hong Kong SAR

Presentation of the evaluation of International Reserves at the Hong Kong Monetary Authority and at Lingnan University.

March 2013, Santiago, Chile

Presentation of the evaluation on The Role of the IMF as Trusted Advisor at the UN Economic Commission for Latin America and the Caribbean and the evaluation on International Reserves at the Central Bank of Chile.

March 2013, Montevideo, Uruguay

Presentation of the evaluation on International Reserves at the Central Bank of Uruguay.

March 2013, Manila, Philippines

Presentation of the International Reserves evaluation at the Asian Development Bank (ADB).

March 2013, Tokyo, Japan

Presentation of the International Reserves evaluation at the ADB Bank Institute.

April 2013, Washington, D.C., USA

Seminar for IMF-World Bank Spring Meetings participants on the Trusted Advisor evaluation.

Annexes

Annex 1 Revisiting Past IEO Evaluations: Evaluation of Prolonged Use of IMF Resources

Summary

This note revisits the 2002 IEO Evaluation of Prolonged Use of IMF Resources and examines the extent to which its findings and conclusions remain relevant, and the extent to which its recommendations remain useful and have been implemented.1

At the time of the 2002 evaluation, prolonged use was extensive and persistent, and it was seen as having problematic aspects. Beyond being at odds with the IMF’s mandate of providing temporary finance, there was a concern that prolonged use could undermine domestic ownership of policymaking, even if sometimes it might also contribute to transfer of economic management skills. Prolonged use raised questions about program design, the IMF’s credibility, and the potential stretching of the IMF’s financial resources.2

The 2002 evaluation found that various program design and implementation problems had contributed to prolonged use, including insufficient emphasis on domestic ownership and capacity, poorly focused conditionality, political considerations that might in some cases have blurred technical judgments, and insufficient opportunities to step back and consider the longer-term strategy for IMF involvement. Another key contributing factor was the desire of international donors and creditors to have an IMF “seal of approval” of country policies.

The Executive Board broadly endorsed the evaluation recommendations aimed at greater selectivity in extending financial support and in program content. It also endorsed recommendations aimed at better integration of IMF-supported programs in a longer-term strategic framework that would take into account institutional, political, and ownership challenges. Recommendations have been implemented to varying degrees. Among them:

  • The IMF adopted an explicit definition of what it considers to be prolonged use, referred to as longer-term program engagement (LTPE), for the purpose of triggering certain due diligence procedures. A list of members with LTPE is now maintained and used as a basis for due diligence procedures.

  • Systematic Ex Post Assessments of all LTPE cases are now carried out and transmitted to the Executive Board, although the assessments have been of mixed quality and many have not addressed forward-looking questions about the IMF’s continued engagement.

  • The Policy Support Instrument has been launched to provide an alternative way to signal the IMF’s “seal of approval,” but has only been used in a few cases.

  • Program design and implementation have been strengthened in some areas. However, the need for deeper analysis of country institutional, political, and ownership issues as well as the need to consider program arrangements for prolonged users in a longer-term framework remain relevant.

Prolonged use and many of the concerns around it subsided in the years since the 2002 evaluation. This review found that the decline in prolonged use was in part due to progress in the direction of the reforms recommended by the evaluation as well as a more favorable global financial environment. In fact, prolonged use is now largely confined to a relatively small group of low-income countries with longer-term macroeconomic issues to be addressed and for which longer-term IMF engagement is more widely accepted as appropriate.

The increase in the number of new IMF-supported programs in the past few years suggests that the incidence of prolonged use could rise again in the near future.3 The IMF, therefore, should be vigilant in ensuring that prolonged engagements are aimed at addressing true longer-term macroeconomic challenges. In this context, the review found that the conclusions and recommendations of the 2002 evaluation remain relevant and would be helpful to ensure that new longer-term program engagements are better targeted and more likely to succeed.

Introduction

This review is part of a pilot project to revisit some of the IEO’s early evaluations in order to examine the extent to which their findings remain relevant and the extent to which their recommendations remain useful and have been implemented. It looks again at the IEO Evaluation of Prolonged Use of IMF Resources, the first evaluation completed by the IEO, which was discussed by the Executive Board in September 2002, and the subject of a subsequent staff task force established to suggest how best to take forward the recommendations made in the evaluation.4 This review is much more modest in scope than the original evaluation, and the conclusions should be treated as more tentative than those based on a full evaluation. It is based on an examination of IMF documents and interviews with staff related to prolonged use between 2003 and 2012, and an in-depth analysis of four country cases (Armenia, Pakistan, Rwanda, and Uruguay).5

The remainder of this note is structured as follows. The first section summarizes the key findings of the original evaluation and examines available evidence on whether the problems identified remain matters of concern. The second section assesses the record on implementation of specific evaluation recommendations, seeking to focus more on how far the underlying purposes of the recommendations have or have not been met, rather than whether they have been implemented in precise detail. Where they have not been implemented in full, it discusses the extent to which they remain relevant and useful. The final section provides a few concluding observations.

Key Evaluation Findings and Their Continued Relevance

The main findings of the 2002 evaluation can be summarized as follows:

  • The evaluation found prolonged use to be extensive and persistent, with relatively few countries graduating, a finding thought to be at odds with the IMF’s mandate to provide temporary balance of payments financing, and possibly raising questions regarding the IMF’s credibility. The evaluation accepted that prolonged use had been helpful in some cases, but there were enough cases where it appeared to reflect repeated program failure to warrant a strong due diligence approach and to seek to ensure that it was only used where appropriate and likely to be successful.

  • Various program design and implementation problems were associated with and contributed to prolonged use. In particular, the evaluation noted that some macroeconomic problems take a long time to fix and that these were not being well addressed by a succession of short-term, IMF-supported programs that were not set in a longer-term context. This led to overoptimistic programs, downplaying risks, and attempting to show success at the end of unrealistically short time periods; insufficient emphasis on domestic ownership and on domestic institutional and implementation capacity; poorly focused structural conditionality; political considerations that might in some cases have blurred technical judgments; and insufficient opportunities to step back and reconsider the longer-term strategy and rationale for IMF involvement.

  • A key factor leading to prolonged use was seen to be the desire of international donors and creditors to have an IMF “seal of approval” for countries receiving their support.6 This was thought to be the main cause for prolonged use in some cases where it may not have been needed or where it was not the best type of intervention. In some cases, this had also led to pressures for relatively weak IMF-supported programs.

  • While prolonged use was seen as bringing benefits in terms of transfer of economic management skills, it was also seen to risk undermining the development of independent domestic policymaking and ownership of the policies adopted.

The IMF has adopted an explicit definition of what constitutes prolonged use, referred to as “longer-term program engagement” (LTPE), for the purpose of triggering certain due diligence procedures. LTPE is currently defined as follows: (i) all IMF members are considered as having LTPE if they have spent at least 7 of the past 10 years under programs supported by IMF resources (whether concessional or nonconcessional); and (ii) time spent under precautionary arrangements that remain undrawn does not count toward LTPE. Engagement under a Policy Support Instrument (PSI) also does not count toward LTPE.7

There are many fewer prolonged use cases today than at the time of the original evaluation. Table 1 presents the evolution of LTPE since 2003.8 The table shows that LTPE has declined since 2006, both in terms of numbers of countries and as a proportion of all IMF programs. Also, and possibly more importantly, it is now almost entirely confined to countries—such as low-income countries (LICs) and countries in transition—with acknowledged longer-term macroeconomic policy issues to be addressed for which it has come to be accepted that longer-term IMF engagement and support may be helpful. Whereas in 2002 several prolonged users were drawing on General Resources Account (GRA) resources, now all are drawing from the Poverty Reduction and Growth Trust (PRGT) (though two are drawing a blend of GRA and PRGT resources).9

Table 1.

Incidence of Longer-Term Program Engagement, 2003–12

article image
Sources: IEO calculations, based on IMF, “Report on the Incidence of Longer-Term Program Engagement,” Table 1, Members with Longer-Term Program Engagement (2003–12); “IMF Lending Arrangements” (2003–12); and Institutional Repository.

In 2003, LTPE was defined as: (i) a country having spent 7 or more of the past 10 years in arrangements, including precautionary arrangements, financed through the GRA or a blend of GRA and concessional resources; or (ii) a country having had two or more multi-year arrangements supported by concessional resources. In May 2006, the Board approved two changes in the definition: (i) all IMF members are now considered as having LTPE if they have spent at least 7 out of the last 10 years under programs supported by IMF resources (whether concessional or nonconcessional); and (ii) time spent under precautionary arrangements that remain undrawn does not count toward LTPE. As a result, 22 countries were removed from the LTPE list at that time, including 17 PRGF-eligible multi-year borrowers and 5 non-PRGF-eligible countries with precautionary arrangements. As per the 2010 revised guidance on “Ex Post Assessments of Members with a Longer-Term Program Engagement,” the Policy Support Instrument also does not count towards LTPE (IMF, 2010a).

Some countries eligible for arrangements under the Poverty Reduction and Growth Fund/Trust (PRGF/T) may have drawn only on GRA resources. Some countries may have had blend arrangements (e.g., precautionary/nonprecautionary and/or PRGF/T-eligible/non-PRGF/T-eligible).

Not all countries classified as LTPE members as of a given date had current arrangements as of that date.

The 15 LTPE members as of June 2012 were Armenia, Benin, Burkina Faso, Burundi, The Gambia, Georgia, Grenada, Kyrgyz Republic, Malawi, Mali, Mauritania, Nicaragua, Niger, Rwanda, and Sierra Leone.

In part, the fall in the incidence of LTPE may have been due to the implementation of some of the evaluation recommendations; but causality and attribution are always difficult to establish. Among other factors contributing to the drop in LTPE after 2006 were the favorable external conditions and easier market access in the period before 2008. As discussed in more detail in the next section, the IMF has attempted to address a number of design and implementation issues seen as contributing to prolonged use, although in general with less than complete success.10 Some of these issues were addressed in the specific context of IMF policy towards LTPE, others in the context of Fund-wide policies on program design and conditionality.

This review found, however, that most of the findings and conclusions of the 2002 evaluation are still relevant. There are still many persistent prolonged users: 9 of the 15 countries with longer-term program engagement as of June 2012 were also prolonged users at the time of the 2002 evaluation. Furthermore, IMF staff interviewed for this review suggested that the incidence of prolonged use may rise again in the future. Out of more than 40 countries removed from the LTPE list between 2005 and 2012, some have been reinstated and one-quarter currently have, or are discussing, an IMF program. Moreover, a recent increase in the overall number of new programs suggests that there may be a resurgence in the number of prolonged users.11

Recommendations, Implementation, and Continued Usefulness

The IEO evaluation made a set of specific recommendations designed to address the key issues related to prolonged use. The recommendations fell into three groups:

  • Recommendations on the rationale for IMF involvement. These included: the adoption of an explicit definition of prolonged use to act as a trigger for due diligence procedures; more selectivity in extending IMF support, taking more account of implementation capacity; providing alternatives to IMF-financed programs to act as a “seal of approval;” and development of an explicit exit strategy for prolonged users. The evaluation also recommended consideration of a higher rate of charge for prolonged users, but this recommendation was not endorsed by the Board.

  • Recommendations for program design and implementation. These included: much greater emphasis on ownership and on institutional development and implementation capacity; greater selectivity in and better prioritization of conditionality; stronger collaboration with the World Bank; greater efforts to cast individual arrangements in the context of a longer-term strategic framework where IMF engagement was expected to be protracted; and more explicit treatment of program uncertainties.

  • Recommendations for IMF governance. These included: systematic ex post assessments of all prolonged use cases to be transmitted to the Executive Board; stronger surveillance in prolonged use cases, with more separation between use of IMF resources (UFR) and Article IV missions; attempts to reduce the incentives leading to excessive turnover of staff working on countries; and new procedures to ensure that political considerations would be taken into account in a transparent manner.

An IMF staff task force was convened to consider the recommendations of the IEO evaluation. The task force conclusions were discussed and a strategy was agreed by the Executive Board in March 2003. In July 2003, a guidance note was issued to staff outlining procedures to be implemented to strengthen scrutiny of programs for members engaged in longer-term use of IMF resources and to put in place broad-based efforts to strengthen IMF surveillance, improve program design, and focus conditionality (IMF, 2003a).

The following paragraphs discuss the extent to which the original evaluation recommendations have been implemented and how far the recommendations remain relevant in the light of experience over the last 10 years.12 The test is whether arrangements are in place to try to ensure that prolonged use countries are indeed those seeking to address longer-term macroeconomic issues and, if so, that the engagement is now taking a form, including financing and program design and implementation, that is more likely to lead to success.

Implementation Status of Recommendations on the Rationale for IMF Involvement

As noted above, there is now an agreed definition of prolonged use, referred to as longer-term program engagement (LTPE). Moreover, a list of LTPE member countries is maintained and published every year.13 The list serves as a trigger for due diligence procedures, namely the requirement to carry out Ex Post Assessments (see the section on “Implementation Status of Recommendations on IMF Governance” below).

Has the IMF become more selective in extending financial support? Interview evidence suggests that in some cases analysis of “ownership” factors, including country political and institutional constraints, has contributed to decisions to hold back from continued support to countries that had been prolonged users, and hence has helped reduce the incidence of prolonged use. But evidence from both document review and interviews suggests that a judgment that implementation capacity is low does not appear always to have prevented IMF engagement; in some cases it acted more as a spur for extra assistance in improving capacity. This may well have been the right approach in some cases. However, it is also for consideration whether a new form of vehicle for IMF engagement, one seen as likely to lead to a UFR program after a period of support with policy formulation and implementation capacity, would be useful in other cases.

There continues to be external pressure from development partners, official creditors, and others for IMF engagement as a “seal of approval.”14 This has come in particular where donors are providing budget support, where they may be looking to an IMF program for assurances in regard to the macroeconomic framework within which to extend their own support. In line with the IEO evaluation recommendation, the Policy Support Instrument (PSI) was developed in 2005 as an alternative signaling device (IMF, 2005).15 But it has been made available only to LICs and has been used in a limited number of cases—seven to date.16 The relatively small number of PSI programs seems partly to reflect the high bar in terms of track record set for such programs and partly the very strict timetable for reviews and meeting conditionality as compared to precautionary programs.17

This continued pressure for IMF engagement as a “seal of approval” raises a number of questions for the Board and the membership at large to consider. Are different forms of prolonged IMF engagement useful and appropriate means for signaling about a country’s policies? Is the use of IMF precautionary programs an acceptable way of providing a “seal of approval” where there is no pressing immediate balance of payments need? What other means could there be to signal IMF support without providing finance? In any case, it seems that there is growing support for the view that sometimes IMF financial engagement will be required as the most convincing “seal of approval,” thus leading to continued prolonged use of IMF resources.

Finally, only some EPAs included in-depth discussion of possible exit strategies from IMF-financed arrangements (Goldsbrough, forthcoming)—a key motivation for EPAs. At the same time, such discussions often featured in Board discussions of UFR requests. In either case, however, these strategies were not always followed through, in part because sometimes they were overtaken by unexpected events.

Implementation Status of Recommendations on Program Design and Implementation

In recent years, documentation for UFR requests, including those for prolonged use countries, have paid more attention to the need for ownership and provided some analysis of implementation capacity. It has also come to be recognized that some reforms need ownership at many levels of government and society, not just within the central government. In the country cases reviewed, analysis in program documents of such factors, including country political and institutional constraints that could bear on program design and implementation, was of varying quality and depth; was perhaps limited by the resources and skills available to staff; and in some cases was based solely on World Bank proxy indicators. There were, however, examples of good practice both in analysis of such factors and recommending steps to address any issues identified which could usefully be replicated more widely.18 Interviews revealed that in some cases such analysis was carried out by staff but not reported fully in Board documents for reasons of political sensitivity.19 It is a welcome development that such analysis was carried out and used to inform program design and the IMF’s financing decisions, even if it was not reported in UFR documents.

There has been greater selectivity in conditionality as a result of general IMF policy changes. The four case studies reviewed for this study also revealed a greater recognition that some important reforms, for example to raise tax revenues, take many years to implement and require a degree of political commitment that is not always there. This again highlights the importance of a stronger analysis of domestic political factors in prolonged use cases to ensure that conditions are not only macro-critical but also realistically achievable.

The continued need for stronger collaboration with the World Bank appears to be well recognized. Country case staff reports and Board discussions have noted that this has proved difficult in some cases, particularly given the different timeframes on which the two organizations operate and the fact that the World Bank is able to be more selective in its operations than the IMF. Interview evidence also suggests that the quality of cooperation varies greatly and depends on personalities and team composition. This remains a topic requiring attention by both institutions.

In common with the 2002 evaluation, most of the UFR documents examined for this review did not set IMF programs in a longer-term framework. Although the normal focus of IMF-supported programs is short to medium term, for LTPE countries it would be useful for new UFR requests to have a section setting the proposals in a longer-term framework. Where a recent EPA is available, staff reports may be able to draw on it (see below on timeliness and quality of EPAs).

Based on the evidence from cases examined for this review, program uncertainties seemed to be relatively well treated in UFR documents. Best practice, which should be replicated more widely, is to include an explicit discussion of contingency planning. However, interviews suggest that often there are constraints to complete transparency about risks.

Implementation Status of Recommendations on IMF Governance

A system of EPAs was instituted in 2003. It requires an EPA to be carried out for members assessed to have a longer-term program engagement and where a successor program is contemplated, if an EPA has not been prepared in the past five years.20 In the case of successor programs, the EPA is to be completed in time to be considered by the Board prior to the request for the new arrangement. Guidance to staff suggests that the EPA be submitted to the Board possibly at the same time as the last program review of an existing program or at the time of an Article IV staff report, but in any event in good time for the Board to discuss the EPA and to influence the design of any new UFR proposal. The guidance also requires that the assessment be prepared by someone other than the UFR mission chief and team.

Based on the evidence from Goldsbrough (forthcoming), it seems that EPAs were largely carried out in the manner and according to the timetable prescribed, and by a mission chief and team other than the program team. It is worth noting, however, that based on the timetable set out it is possible that the EPA used for a new UFR request can be up to five years old. It is also the case that on occasion an EPA was prepared in parallel with rather than prior to negotiation of a new IMF-supported program. In addition, Board discussion usually was rather limited and subsumed in a wider discussion of an Article IV staff report or a program review and request for waivers. Goldsbrough also finds that EPAs were of mixed quality, a view shared among staff. Moreover, EPAs are intended to be forward- as well as backward-looking; and it is a requirement that they “discuss the prospects for graduating from IMF programs, including, where appropriate, an explicit exit strategy.” However, Goldsbrough finds that over two-fifths of EPAs addressed the issue of an exit strategy from IMF-supported programs poorly and that, in over one-quarter, discussion of the IMF’s future role was absent or weak.

Little change has resulted from the recommendation that there should be stronger surveillance in prolonged use cases, with greater separation between UFR and Article IV missions and assessments. For a short period after 2003 there were a few experiments with having different mission leaders for Article IV and UFR missions, on the basis also of proposals from the 2002 Biennial Surveillance Review that surveillance should be used to bring a “fresh pair of eyes” to bear in all program countries, not only prolonged use countries. A review of subsequent surveillance reviews to date shows that this practice was not followed more widely and by the time of the 2011 Triennial Surveillance Review interest in the idea had largely dissipated. An analysis of recent Article IV staff reports for the current 15 members having longer-term program engagement finds that, except in cases where programs were off-track or had been completed, Article IV assessments were always presented in the same documents as program reviews or UFR requests, carried out by the same mission leaders as UFR missions, and discussed by the Board together. While the reports did sometimes contain additional material, they did little to bring “a fresh pair of eyes” to bear on the IMF’s longer-term engagement. This is also partly perhaps because the focus of Article IV surveillance is short to medium term rather than longer term.

Tackling incentives for excessive staff turnover has been a recurrent issue in IEO evaluations to date. Directors have reiterated the need for special attention to this issue, noting that turnover of mission chiefs and mission teams need to be better monitored and that more data is needed to assess whether there has been an improvement in the duration of country assignments (IMF, 2013). Management has recently introduced reforms whereby assignments will be expected to last three years on average. A system has been put in place to collect data on tenure, and results are intended to be shared with the Executive Board as a part of briefings on strategic human resources issues.

Finally, turning to the recommendation that political considerations should be made more transparent, as noted above, most UFR documents examined for this review appeared to analyze domestic political and capacity constraints in a fairly shorthand way. Some did a better job of discussing country political factors and constraints to macroeconomic policymaking, and this practice could usefully be replicated elsewhere, accepting that sometimes there are good reasons for not being explicit about all elements of such analysis in Board documents. As for the possibility of more transparency about the wider geopolitical concerns that often drive attitudes of Board members, it may be that this was not a realistic proposal to begin with given the IMF’s mandate. In the 2002 evaluation, it was noted that such factors had clearly influenced decisions in the case of Pakistan (one of the four case studies in that evaluation). In practice, as for cases analyzed in this review, such concerns were not discussed or acknowledged in Board documents. Moreover, in the course of Board discussions, these political factors, although probably well understood by all, were rarely acknowledged. When they were discussed, they were often not reflected in the Summing Up.

Concluding Comments

This review finds that the 2002 evaluation had an impact on how the IMF engages countries that have longer-term adjustment issues, as well as on how the IMF approaches program design and implementation in countries that require recurrent financial support. The 2002 evaluation found prolonged use of IMF resources to be extensive and persistent. This was considered to be at odds with the IMF’s mandate of providing temporary finance; it also raised questions about program design and implementation, and about whether prolonged use could undermine domestic ownership of policymaking. The 2002 evaluation provided several recommendations intended to diminish the incentives for prolonged use and to reduce its adverse consequences.

The incidence of prolonged use has declined since 2006. This decline may be partly attributed to implementation of (some of) the evaluation recommendations. Today prolonged use is largely confined to a relatively small group of PRGT-eligible countries with longer-term macroeconomic issues to be addressed and for which longer-term IMF engagement is more widely accepted as appropriate. However, for some countries, prolonged use has been persistent. Moreover, the incidence of prolonged use could rise again in the near future given the number of new programs arranged in recent years. This emphasizes the need to ensure that prolonged users are seeking to address longer-term macroeconomic issues and that the engagement is taking a form likely to lead to success. In this context, this review concludes that while some progress has been made, many of the program design and implementation issues noted in the 2002 evaluation remain of concern, and most of the related recommendations remain relevant.

Annex 2 Revisiting Past IEO Evaluations: Fiscal Adjustment in IMF-Supported Programs

Summary

This review assesses the continued relevance of the findings, conclusions, and recommendations of the 2003 IEO evaluation of Fiscal Adjustment in IMF-Supported Programs. Fiscal adjustment has generally been a core element of IMF-supported programs. The 2003 evaluation assessed the design of fiscal adjustment in a sample of IMF-supported programs during 1993–2001, looking at both their quantitative and qualitative dimensions.1

The evaluation found evidence of faster-than-anticipated adjustment of the current account and buildup of reserves in the context of IMF-supported programs, largely as a consequence of lower-than-projected private spending and economic activity. It therefore appeared that the fiscal stance targeted in these programs may have been too contractionary. Moreover, the arguments justifying the fiscal targets were often not clear. The evaluation therefore suggested providing a clearer and more explicit justification for the path of fiscal adjustment proposed in IMF-supported programs. This would enhance the quality of the analysis, promote greater understanding of the risks faced, and facilitate the mid-course corrections that were often needed in practice. A separate concern of the evaluation was that many of the fiscal measures proposed did not adequately guard against policy reversals or promote resilience to shocks. In response, the evaluation suggested that a more thorough analysis of structural reform priorities in the fiscal arena, for example in the context of Article IV surveillance, would enhance the ability of IMF-supported programs to foster more durable improvements to fiscal policy.

This review found considerable progress in analyzing and articulating the rationale for fiscal adjustment in IMF-supported programs. It also found improvement in the discussion of structural reform priorities in the fiscal area in both program and surveillance documents. Nonetheless, there is room for improvement in the way IMF program documents analyze and explain the magnitude and pace of fiscal adjustment—a dimension this review could still not rate as satisfactory. The review also found instances where structural issues important to fiscal management and sustainability were not convincingly articulated.

Given renewed interest in the design of fiscal adjustment in IMF-supported programs in the aftermath of the global financial crisis, the focus of the 2003 evaluation clearly remains relevant today. Indeed, recent discussion by the IMF and others of possible underestimation of fiscal multipliers under conditions of private deleveraging and accommodative monetary policy underscores the importance of assessing the economic and policy environment and private sector responses to them as accurately as possible in the context of IMF-supported programs—which was a key objective of the 2003 evaluation’s recommendations.

Introduction

This review is part of a pilot project to revisit some of the IEO’s early evaluations in order to examine their continued relevance, and the extent to which their recommendations remain useful and have been implemented. In this context, the review assesses the IEO evaluation of Fiscal Adjustment in IMF-Supported Programs, which was discussed by the Executive Board in August 2003.

The 2003 evaluation was a far-reaching exercise. This follow-up note is more selective in scope and depth, focusing primarily on the core issues raised by the evaluation. It is based on a review of a sample of IMF-supported programs during 2005–10, other relevant IMF documents, and interviews with IMF staff.

The first section describes the key findings and recommendations of the 2003 evaluation, which, as will become apparent, remain highly pertinent a decade later. The second section assesses the implementation record in the ensuing period. The final section provides brief concluding observations.

Key Findings and Recommendations of the 2003 Evaluation

The 2003 evaluation assessed the quantitative and qualitative dimensions of fiscal adjustment in a cross-country sample of IMF-supported programs during 1993–2001. It found that some common criticisms of IMF-supported programs at the time—that they adopted “one-size-fits-all” approaches to fiscal adjustment, were inflexible, or caused declines in social spending—were not supported by empirical evidence. At the same time, it found that the rationale for key macroeconomic assumptions underlying the proposed fiscal adjustment and subsequent mid-course revisions were not clearly articulated in program documents.

There was a tendency by the IMF to adopt fiscal targets based on overoptimistic assumptions about the pace of economic recovery and investment rates. Because output and private investment often recovered at a slower pace than assumed, the result was an overcorrection of the current account deficit. In such situations a less contractionary fiscal stance might have more appropriately smoothed the adjustment—unless polices were constrained by financing or debt sustainability concerns. But the information and analysis justifying the fiscal targets proposed were often not made available: for example, most program documents reviewed for the evaluation did not adequately discuss how fiscal targets related to balance of payments problems, how the magnitude and pace of the adjustment would address such problems, or how the adjustment should depend on private sector demand and growth prospects. These considerations led the evaluation to propose that program documents should provide a stronger rationale for the specific fiscal targets proposed in programs.

A related concern was that many of the fiscal measures proposed under the IMF-supported programs did not adequately guard against policy reversals or promote fiscal resilience to shocks. Slow progress in implementing structural and institutional reforms in turn placed limits on the extent of fiscal adjustment that could be achieved in the short run and on the durability of that adjustment. Greater emphasis on structural reforms to strengthen the public finances would therefore enhance the capacity of fiscal systems to achieve more lasting adjustments.

However, program and surveillance documents did not adequately explain how key institutional reforms needed in the fiscal area to enhance fiscal sustainability could be implemented. While the evaluation recognized that the fiscal measures proposed in the relatively short-term context of an IMF-supported program could not fully incorporate the country’s longer-term fiscal reform agenda, it suggested that more effort be devoted to discussing the nature of the structural reform agenda and how the measures supported by the program related to it—in the context of surveillance if not in the program document. The reform “road map” and its prioritization would ideally be contained and updated in the course of IMF surveillance, and be used to anchor the short-term measures supported by the program.

Finally, although the evaluation did not find evidence that IMF-supported programs squeezed aggregate social sector spending, it nonetheless suggested the IMF could become more active in assisting middle-income countries to establish mechanisms to protect the most vulnerable during periods of budgetary retrenchment.

The following are the main recommendations of the 2003 evaluation, which elicited considerable support from Executive Directors:

  • Program documentation should provide a more in-depth and coherent justification for the magnitude and pace of the fiscal adjustment and how it is linked with assumptions about the recovery of private sector activity and growth.

  • Programs should give greater emphasis to the formulation and implementation of key institutional reforms in the fiscal area, even if (as is likely) they cannot be fully implemented during the program period.

  • The surveillance process should be used more explicitly to provide a longer-term road map for fiscal reforms and to assess progress achieved.

  • The IMF should clearly delineate the operational framework in which social issues will be addressed within program design in non-PRGF countries. This should include a clear indication of the IMF’s responsibilities and activities in this area.

  • The internal review mechanism should place relatively more emphasis on the early stages of the process.

Assessing Implementation of the Evaluation Recommendations

IMF staff proposed a series of follow-up actions to address the IEO recommendations, described in IMF (2004), which was discussed by the Executive Board in March 2004. The status of implementation was updated in the first “Periodic Monitoring Report” (IMF, 2007). This section discusses the extent to which the recommendations of the 2003 evaluation have been implemented. It does not try to assess the extent to which actions by the IMF can be attributed to the IEO report, recognizing that many other factors may be involved.

Rationale for Fiscal Adjustment in IMF-Supported Programs

The first recommendation of the 2003 evaluation was for IMF program documentation to provide a more explicit articulation of the basis for the proposed fiscal stance, and how it was linked with assumptions regarding the recovery of the private sector. The purpose was to promote better analysis and a clearer understanding of how various economic agents would adjust to economic shocks and the proposed fiscal adjustment; to facilitate the review process and discussions at the Board; and to provide external audiences with a more convincing explanation of the rationale for the program.

A guidance note was issued by IMF staff in January 2004 on how program documents might best present and explain the appropriate size, pattern, and composition of fiscal adjustment, consistent with the evaluation’s recommendations. The 2004 guidance note is no longer available as a self-standing document, although some of its elements were subsumed in subsequent guidance notes.

Ten years on, has there been any change in how well the rationale for the fiscal adjustment projected in IMF-supported programs is spelled out? Do program documents now contain a more explicit articulation of the basis for the proposed fiscal stance, and how it is linked with assumptions regarding the recovery of the private sector? This review replicated the analysis in the original evaluation using program documents of 13 countries that had IMF-supported programs during 2005–10. It utilized the same questions used in the original evaluation to assess the extent to which the IMF explained the rationale, magnitude, and composition of the envisaged fiscal adjustment.2 These questions are listed in footnote 1 of Figure 1, and focus on how program documents deal with the motivation for the program and the determinants of the size and composition of the fiscal adjustment. The scope of this review is limited to completed Stand-By Arrangements (SBAs) that envisaged significant fiscal adjustment (or included some further adjustment that followed significant fiscal adjustment in the recent past).3

Figure 1.
Figure 1.

Degree to Which Program Documents Explain the Rationale, Magnitude, and Composition of the Envisaged Fiscal Adjustment: Original Versus Revisited Findings

Sources: Table 3.1 in IEO (2003) and program documents.1The five questions are: (1) Do documents clearly explain the source of the existing or potential balance of payments problem motivating the program? (2) Do documents explain the country-specific mechanism by which the fiscal adjustment will help improve the balance of payments problem? (3) Do documents discuss how the pace and magnitude of fiscal adjustment is being set in order to address the actual or potential balance of payments problems? (4) If there are other factors influencing the envisaged fiscal adjustment do documents explain clearly how they influence that adjustment? (5) Do documents explain the rationale for the composition of the fiscal adjustment?2The ratings are: H=highly satisfactory, S=satisfactory, M=marginally satisfactory, and U=unsatisfactory. See Appendix 2 in IEO (2003) for an explanation of these ratings. The figure depicts averages for each question across the 13 countries.

The overall results indicate improvement in the manner in which program documents explained the rationale and magnitude of the envisaged fiscal adjustment. Figure 1 compares the original versus revisited findings for each of the five questions examined; Figure 2 compares the distribution of ratings for each question for the two sample sets. There was a clear improvement across four of the five questions covered and a slight deterioration for the fifth question. Whereas three of five questions were rated less than satisfactory in the original sample, only one was so rated in the recent sample.

Figure 2.
Figure 2.

Degree to Which Program Requests Explain the Rationale, Magnitude, and Composition of the Envisaged Fiscal Adjustment: Distribution of Original and Revisited Findings1

Sources: Table 3.1 in IEO (2003) and program documents.1The ratings are: H=highly satisfactory, S=satisfactory, M=marginally satisfactory, and U=unsatisfactory. See Appendix 2 in IEO (2003) for an explanation of these ratings.

A number of caveats are in order. The relatively small sample sizes used in both the original evaluation and this review indicate the need for caution in interpreting the results. Moreover, one possible source of bias should be pointed out. By necessity, the recent sample included more countries that were afflicted by global or regional crises than the earlier sample. As such crises are often the source of the balance of payments shocks requiring immediate adjustments, it may be easier to explain the need for fiscal adjustment in such cases—this may have been a factor driving the relatively large improvement observed for Question 1 in Figure 1. By the same token, it may be harder to lay out the fiscal adjustment agenda as clearly when the program is being prepared under crisis conditions, which would have made it more difficult to explain the rationale for the composition of the fiscal deficit adjustment proposed in the more recent sample. This may account for the lack of improvement observed for Question 5.

The one question which, although showing improvement for the recent sample period, was still rated less than satisfactory was Question 3, namely the discussion of the rationale for the pace and magnitude of fiscal adjustment needed to address balance of payments problems. This is perhaps not surprising as it requires making judgments about how fiscal adjustment should be calibrated to the likely path of private spending, the likely evolution of the current account balance under turbulent conditions, while concurrently estimating uncertain financing constraints. Nevertheless, the rationale for focusing on this question remains as pertinent today as it was at the time of the 2003 evaluation. Indeed, recent discussion of fiscal multipliers and their likely underestimation in the aftermath of the global financial crisis in IMF (2012a), Blanchard and Leigh (2013), and numerous other publications underscores the importance of assessing the economic and policy environment and private sector reactions as accurately as possible in the context of IMF-supported programs—which was a critical recommendation of the 2003 evaluation.

Coverage of the Medium-Term Fiscal Reform Agenda

The 2003 evaluation found that sustained progress on the fiscal reforms identified in the IMF-supported programs had been limited, in part due to excessive emphasis on meeting short-term quantitative targets which were reversible in the absence of parallel institutional reforms. This in turn was due in large part to a mismatch of time frames: the short horizon of programs relative to the time needed to complete institutional reforms. These findings were the basis for the evaluation’s recommendations for programs to stress the appropriate formulation and implementation of key institutional reforms in the fiscal area (even if they could not be implemented during the program period), and for surveillance to provide a longer-term road map for such reforms. These two recommendations are considered jointly here as both are concerned with the adequacy with which the medium-term fiscal reform agenda is presented in IMF documents, whether in program documents or Article IV consultations.

Following the 2003 evaluation, area departments were encouraged to make more active use of the Fiscal Affairs Department (FAD) fiscal strategy briefs to strengthen their discussions of structural fiscal issues in program countries (IMF, 2007).4 As of 2007, fiscal strategy briefs were maintained and regularly updated for about 75 countries. However, these briefs were discontinued in the aftermath of the 2008 downsizing, primarily due to budgetary cutbacks, and replaced with more succinct summaries of findings from FAD technical assistance to a large number of countries. In any event, in interviews of IMF staff conducted for this review, opinion was divided on the value of the briefs for preparing Article IVs and program documents.

The 2003 evaluation identified 101 specific fiscal-related reform measures that were subjected to conditionality, and developed various indices to measure the extent of implementation.5 For the purposes of this review, it was decided not to replicate the use of the various indices developed in the original evaluation to measure the extent of implementation of fiscal reforms since the small samples would have diminished the significance of specific comparisons. Instead, the adequacy of discussion of the structural fiscal reform agenda was assessed more broadly using the sample of 2005–10 IMF-supported programs identified in the section “Rationale for Fiscal Adjustment in IMF-Supported Programs” above. Prior Article IV documents were also reviewed to determine how well surveillance described the country’s longer-term fiscal reform agenda, focusing in particular on those instances where the discussion in the program document was found to be below satisfactory.

Overall, the discussion of fiscal reform issues was found to have improved in the recent sample compared to the sample in the 2003 evaluation, with 8 of the 13 program documents found to contain at least satisfactory discussions. However, there remains considerable variation in the extent and depth of such discussions across program documents and Article IV reports. In addition, the review encountered cases in which Article IV reports provided sufficient discussion of vital reform issues that impacted on fiscal policy, but these were not adequately reflected in the subsequent program request. In some instances, there was a discussion of the structural reform agenda in the text of the program document, but the associated structural performance criteria and benchmarks did not reflect these concerns, and tended to be overly focused on the next few months rather than the duration of the program. Finally, there were instances where the Article IV reports themselves did not contain an adequate discussion of the needed fiscal reform agenda. Illustrative examples, both positive and negative, are provided in Box 1.

Some of these findings may just reflect the need to develop an IMF-supported program rapidly. Yet both the program document and the credibility of prior surveillance would have been strengthened by discussion of the important structural fiscal policy constraints that would need to be addressed over the medium term.

Tackling Social Spending and Social Protection

The 2003 evaluation also addressed the issue of the impact of fiscal adjustment on social sector expenditures in the context of IMF-supported programs. On the basis of cross-section analysis, it found that aggregate social sector spending was not lower than it would have been in the absence of an IMF-supported program.6 At the same time, in-depth country studies indicated that even when aggregate social sector expenditures were maintained, critical areas of expenditure most relevant to the poor may be crowded out by other components of government expenditure such as wages and salaries. The evaluation concluded that these adverse effects could be limited if the most critical social sector expenditures could be better protected during times of crisis. The evaluation therefore proposed that the IMF should clarify its guidelines for work in this area, inter alia to assist middle-income countries, facing external shocks and budget retrenchment, to improve their ability to protect vulnerable groups.

Coverage of Structural Fiscal Issues in Program Cases

The 2005 Turkey Stand-By Arrangement is an example where both the program document and the preceding Article IV consultation addressed structural fiscal issues well. The fiscal policy section in the program request contained a separate subsection on structural fiscal reforms, where a robust medium-term reform agenda was presented. With respect to tax reform, the document discussed the need to simplify the personal income tax; reduce the corporate income tax rate while broadening the base and improving compliance; retain value-added tax rates; and implement a new revenue administration law. With regard to expenditure reform, it discussed the importance of dealing with the widening social security deficit and assessed the nature of the legislation the authorities had submitted to Parliament; discussed the need for further reforms during the program period relating to civil service pay and employment, budgetary support for agriculture and state economic enterprises; and linked prospective reforms to forthcoming public expenditure reviews by the IMF and World Bank. The program document also discussed ongoing improvements to expenditure management and transparency through legislation to facilitate medium-term and strategic planning, as well as local government accountability.

By contrast, the review also encountered other country cases where issues important to fiscal management and sustainability were not convincingly articulated in the program documents and/or in preceding Article IV reports. Examples include:

  • Addressing chronically low tax revenue. A more thorough discussion of the long-standing factors inhibiting revenue collection and how to address them, including by addressing weak tax administration, would have enhanced the credibility of the fiscal targets proposed.

  • Tackling energy mispricing with chronically adverse fiscal consequences. There was inadequate discussion of how institutional arrangements for energy prices to reflect cost recovery and fluctuations in international prices can undermine fiscal objectives. Such discussion could have included descriptions (for example, in parallel Article IV reports) of more experiences of tackling similar issues.

  • Pro-cyclical fiscal policy when dependence on commodity exports is high. Mechanisms to smooth the volatility of government spending over the commodity price cycle were not adequately discussed.

  • There was uneven coverage of quasi-fiscal deficits arising from the activities of state-owned enterprises.

Although the objective of better protecting critical social expenditure in the context of IMF-supported programs was endorsed in the ensuing Board discussion, there was concern that the IMF should not become involved in the selection and design of social policy, which was considered outside its mandate and expertise. Indeed, in most of the programs reviewed for this exercise, there were no explicit objectives to protect vulnerable groups.7

Nonetheless, it is worth noting that subsequent work by IMF staff, particularly in the aftermath of the recent crisis, has promoted more active involvement in these issues. For example, included in the most recent review of conditionality, IMF (2012b) is the recommendation to: “Ensure inclusion in program design, where feasible and appropriate, of policy measures to mitigate adverse short-term impacts on the most vulnerable, such as adequate safety nets and possible temporary targeted transfers, particularly in programs with higher risks.” Further, the IMF factsheet on its role in helping protect the most vulnerable in the global crisis, IMF (2012c), indicated: “In this difficult environment, the IMF is helping governments to protect and even increase social spending, including social assistance. In particular, the IMF is promoting measures to increase spending on, and improve the targeting of, social safety net programs that can mitigate the impact of the crisis on the most vulnerable in society.” The factsheet cited examples from advanced, emerging market, and low-income countries.

Enhancing the Review Process

Finally, the 2003 evaluation recommended that the IMF’s internal review process be modified to allow an intensive brainstorming process in the early stages of program formulation so as to enable more meaningful incorporation of relevant comments into the program design.

The IMF has redesigned the review process applicable for programs (as well as Article IV missions), and brought forward the opportunity to comment on policy relevant issues in a more structured setting. The policy consultation meeting, introduced in 2008, is meant to serve as a forum for conveying substantive concerns about policy directions to be proposed by the IMF mission. At the same time, IMF staff interviewed for this review indicated that (rapidly deteriorating) country circumstances and sometimes internal considerations can still compress intended intervals between the policy consultation meeting and the timing of the mission.

Concluding Observations

The core issues and recommendations raised in the IEO’s 2003 evaluation remain as relevant today as when they were proposed. As such it is encouraging that progress has taken place in recent years in providing more coherent justification for the fiscal adjustment proposed in IMF-supported programs. Not surprisingly, the most difficult element of this task—deciding on the magnitude and pace of fiscal adjustment—requires further work. It requires assessing the private sector’s response to the shock that brought about the need for the IMF to intervene, as well as to the fiscal adjustment envisioned in the IMF-supported program. Deciding on the magnitude and pace of fiscal adjustment also requires balancing these private sector demand considerations against balance of payments constraints or debt sustainability concerns. These are challenging tasks, but explicitly considering such issues in the context of the fiscal adjustments proposed would further enhance the quality of IMF-supported programs. This conclusion, dating to 2003, would appear to be validated by recent work inside and outside the IMF, which indicates how significantly fiscal multipliers can vary depending on the state of the underlying economy, the type of fiscal adjustment undertaken, and the stance of parallel policies.

The discussion of institutional reforms in the fiscal area has improved in both program and surveillance documents, but here too there is no room for complacency. While care is needed not to overwhelm the fiscal adjustment effort associated with the program with discussion of institutional and structural reform priorities, the latter need to be clearly articulated in the context of surveillance. If these medium-term issues are covered well in surveillance, it would be more feasible to integrate quantitative adjustments with the longer-term institutional reform agenda in the context of IMF-supported programs.

References

  • Goldsbrough, David (forthcoming), “Review of Ex Post Assessments of Countries with Longer-Term Program Engagement and of Ex Post Evaluations of Exceptional Access,” IEO Background Paper (Washington: Independent Evaluation Office).

    • Search Google Scholar
    • Export Citation
  • Independent Evaluation Office of the International Monetary Fund (IEO), 2002, Evaluation of Prolonged Use of IMF Resources (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund, 2002, “Summing Up by the Acting Chair: Evaluation by the Independent Evaluation Office of Prolonged Use of IMF Resources,” BUFF/02/164, September (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund, 2003a, “Operational Guidance for Assessments of Countries with a Longer-Term Program Engagement,” SM/03/233, Rev. 1, August (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund, 2003b, “IMF Board Discusses the Fund’s Role in Signaling Assessments of Members Policies,” Public Information Notice No. 03/12, February (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund, 2005, “IMF Executive Board Discusses Policy Support and Signaling in Low-Income Countries,” Public Information Notice No. 05/144, October (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund, 2006, “IMF Executive Board Concludes Review of Ex Post Assessments,” Public Information Notice No. 06/96, August (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund, 2007, “Periodic Monitoring Report on the Status of Board-Endorsed IEO Recommendations and Management Implementation Programs,” SM/07/383 (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund, 2010a, “Ex Post Assessments of Members with a Longer-Term Program Engagement—Revised Guidance Note,” SM/10/45, March (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund, 2010b, “Guidance Note on the Fund’s Transparency Policy,” SM/10/323, December (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund, 2013, “Fifth Periodic Monitoring Report on the Status of Implementation Plans in Response to Board-Endorsed IEO Recommendations—Assessment by the Evaluation Committee to the Executive Board,” SM/12/248, Sup. 1, February (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation

References

  • Blanchard, Olivier, and Daniel Leigh, 2013, “Growth Forecast Errors and Fiscal Multipliers,” IMF Working Paper No. 13/1 (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • Clements, Benedict, Sanjeev Gupta, and Masahiro Nozaki, 2011, “What Happens to Social Spending in IMF-Supported Programs?” IMF Staff Discussion Note No. 11/15 (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • Independent Evaluation Office of the International Monetary Fund (IEO), 2003, Fiscal Adjustment in IMF-Supported Programs (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund, 2004, “Follow Up on the Recommendations of the Independent Evaluation Office Report on Fiscal Adjustment in IMF-Supported Programs,” SM/04/32, February (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund, 2007, “Periodic Monitoring Report on the Status of Board-Endorsed IEO Recommendations,” SM/07/383, December (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund, 2012a, World Economic Outlook, October (Washington: International Monetary Fund).

  • International Monetary Fund, 2012b, “2011 Review of Conditionality—Overview Paper,” SM/12/148, June (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund, 2012c, “The IMF’s Role in Helping Protect the Most Vulnerable in the Global Crisis,” Factsheet, September 30. Available at www.imf.org.

    • Search Google Scholar
    • Export Citation
1

This figure excludes the 5 percent carry-over of the FY2012 budget. Appendix 1 details the IEO budget and expenditures.

2

On a one-time, exceptional basis, the IEO was authorized to carry forward up to 10 percent of its FY2013 budget into FY2014, higher than the 5 percent carry forward in previous years.

3

These indicative budgets were provided for information, rather than for endorsement, pending any changes needed in light of the Executive Board discussion of the External Evaluation report, which took place after the budget was approved.

4

These were the first and third IEO evaluations. Issues identified in the evaluation of Capital Account Crises (IEO’s second evaluation) were re-examined in the evaluation of IMF Performance in the Run-Up to the Financial and Economic Crisis and will be further considered in the upcoming evaluations of the IMF’s response to the global financial crisis.

5

During the Executive Board discussion of the 2002 evaluation, many Directors considered that prolonged use among low-income countries relying on concessional financing did not necessarily indicate a problem that needed to be corrected.

6

One hundred and seven programs (of which 26 were precautionary) were approved between 2008 and 2012, compared with 66 (of which 16 were precautionary) between 2003 and 2007.

7

The panel was composed of José Antonio Ocampo, former United Nations Under-Secretary-General for Economic and Social Affairs and Executive Secretary of the Economic Commission for Latin America and the Caribbean; Stephen Pickford, former Managing Director (International and Finance) at H.M. Treasury, United Kingdom, and former IMF Executive Director; and Cyrus Rustomjee, Director of Economic Affairs at the Commonwealth Secretariat and former IMF Executive Director. Mr. Ocampo chaired the group.

8

The full report, along with the Summing Up of the Board discussion, is available on the IEO website, www.ieo-imf.org.

1

This paper was prepared by David Peretz. The author would like to acknowledge the advice and comments of David Goldsbrough, the main author of the 2002 Evaluation of Prolonged Use of IMF Resources, and to thank Alisa Abrams for her assistance and support.

2

These considerations notwithstanding, during the Executive Board discussion of the 2002 evaluation, some Directors considered that prolonged use was not, per se, a problem that needed to be corrected, particularly among low-income countries relying on concessional financing.

3

One hundred and seven programs (of which 26 were precautionary) were approved between 2008 and 2012, compared with 66 (of which 16 were precautionary) between 2003 and 2007.

4

This note assesses the findings and recommendations of the IEO evaluation and not those of the task force, treating the latter as providing an action plan in response to the evaluation.

5

This review, however, did not benefit from discussion with country authorities. Overall, the resources and time devoted to the preparation of this review were considerably less than those devoted to the 2002 evaluation.

6

A further example was countries seeking a Fund “seal of approval” to assist the process of accession to the European Union.

7

This review also uses the terms “prolonged use” and “prolonged users” to refer to “LTPE” and “LTPE countries,” in part to facilitate references to the 2002 evaluation.

8

As explained in the notes to Table 1, in May 2006, there was an important change in the definition of LTPE which led to the removal of 22 countries from the LTPE list. This largely accounts for the change in the LTPE figures between 2003 and 2006. As shown in Table 1, the reduction in the incidence of LTPE occurred after 2006.

9

This difference is significant, since during the Executive Board discussion of the 2002 evaluation, most Directors did not consider the relatively high incidence of prolonged use in LICs relying on concessional financing administered by the Fund as necessarily indicating a problem that needed to be corrected (IMF, 2002).

10

While the IEO evaluation identified program design and insufficient Fund attention to ownership and implementation capacity as factors that had played a major role in successive program failure leading to prolonged use, the subsequent staff task force (mentioned in the next section) saw the main cause as being weak commitment and implementation on the part of program countries. The two diagnoses are not inconsistent, but the difference in emphasis is important. Evidence collected in the course of this review reinforces the 2002 evaluation finding that a range of factors sometimes has led staff to support programs where commitment and implementation capacity were weak.

11

One hundred and seven programs (of which 26 were precautionary) were approved between 2008 and 2012, compared with 66 (of which 16 were precautionary) between 2003 and 2007.

12

This analysis recognizes that not all changes made were attributable to the evaluation recommendations.

13

The initial requirement was that LTPE reports be prepared semiannually. This was done through June 2008, at which time it was agreed by the Executive Board that the frequency be shifted to annual. Additionally, a temporary suspension of the Ex Post Assessment (EPA) requirement was requested by staff in June 2009 and agreed in the context of Executive Board consideration of the 2009 Omnibus Paper on Easing Work Requirements. The suspension was lifted end-August 2010.

14

In 2003, the Executive Board agreed to discontinue the use of Staff Monitored Programs (SMPs) to convey signals on a member’s policies to official and/or private creditors because relatively lax performance reporting standards had allowed members to use the positive signal of initiating an SMP without adequate follow-up on implementation (IMF, 2003b).

15

The PSI supports low-income countries that do not want or need Fund financial assistance but seek to consolidate their economic performance with IMF monitoring and support. This support also delivers signals to donors, creditors, and the general public on the strength of a country’s policies.

16

The seven countries that have, or have had, a PSI are: Cape Verde, Mozambique, Nigeria, Rwanda, Senegal, Tanzania, and Uganda. Except for Cape Verde, all have been classified as prolonged users at some point; Rwanda remained on the LTPE list as of June 2012.

17

PSIs become clearly off track as soon as a benchmark or policy condition is missed, whereas in a precautionary program the program review can be delayed.

18

In one of the country cases reviewed, Uruguay, there was a significant amount of political economy analysis in staff reports, especially in anticipation of a change in government. Fund staff worked with the authorities to consider best ways to engage all candidates to ensure protection of core program elements and the continuity of policies during the transition and beyond. Directors appreciated staff’s report on the institutional and political contexts.

19

According to the guidance note on the Fund’s transparency policy, staff should avoid politically sensitive language (IMF, 2010b).

20

An EPA report is to be prepared the first time a member is assessed to have a longer-term program engagement. Subsequent assessments are to take the form of an EPA Update, which is expected to be more streamlined. For countries that graduate from Fund-supported programs within the five-year period after completing an EPA, a new EPA is not required. A new EPA report, rather than an EPA Update, is required however if during the period relevant for the update a program has been cancelled or interrupted for more than six months.

1

This review was prepared by Sanjay Dhar. The author would like to acknowledge the advice and inputs of Marcelo Selowsky, the main author of the 2003 evaluation on Fiscal Adjustment, and of Andrew Martinez and Roxana Pedraglio.

2

The 2003 evaluation examined program documents of 13 countries that had undergone IMF-supported programs (mainly Stand-By/Extended Fund Facility Arrangements) during 1994–2000. Its selection of countries was to some extent restricted by parallel work on IEO evaluations addressing capital account crises and low-income countries. In both the 2003 evaluation and the present review, the term “program document” refers to the initial request for a program as presented to the IMF Executive Board.

3

All nonprecautionary SBAs during 2005–10 that had been completed by mid-2012 were considered. The SBAs selected for review were: Dominican Republic (2005), Turkey (2005), Uruguay (2005), Georgia (2008), Hungary (2008), Pakistan (2008), Ukraine (2008), Angola (2009), Dominican Republic (2009), Mongolia (2009), Romania (2009), Sri Lanka (2009), and Jamaica (2010). There was little discretion in the choice of countries selected given the above criteria and our exclusion of precautionary programs, advanced economies, and Latvia from the sample. Advanced economies were excluded to maintain comparability with the earlier sample; Latvia was excluded due to the extent of unanticipated output decline.

4

Fiscal strategy briefs originated as internal notes prepared by FAD to aid in the prioritization of technical assistance.

5

The reforms were grouped into nine categories: tax policy; tax administration; wage bill and civil service reforms; social sector spending; other spending issues; public enterprise reform, privatization, and private sector development; social security and pensions; organizational reform; and pricing policy of public utilities.

6

This is corroborated by the work of others such as Clements, Gupta, and Nowzaki (2011).

7

An exception is the 2009 Dominican Republic program, for which increased coverage of the conditional cash transfer program to families in extreme poverty was included as a structural benchmark.

  • View in gallery

    Degree to Which Program Documents Explain the Rationale, Magnitude, and Composition of the Envisaged Fiscal Adjustment: Original Versus Revisited Findings

  • View in gallery

    Degree to Which Program Requests Explain the Rationale, Magnitude, and Composition of the Envisaged Fiscal Adjustment: Distribution of Original and Revisited Findings1