Annex 2 Revisiting Past IEO Evaluations: Fiscal Adjustment in IMF-Supported Programs
- International Monetary Fund. Independent Evaluation Office
- Published Date:
- August 2013
This review assesses the continued relevance of the findings, conclusions, and recommendations of the 2003 IEO evaluation of Fiscal Adjustment in IMF-Supported Programs. Fiscal adjustment has generally been a core element of IMF-supported programs. The 2003 evaluation assessed the design of fiscal adjustment in a sample of IMF-supported programs during 1993–2001, looking at both their quantitative and qualitative dimensions.1
The evaluation found evidence of faster-than-anticipated adjustment of the current account and buildup of reserves in the context of IMF-supported programs, largely as a consequence of lower-than-projected private spending and economic activity. It therefore appeared that the fiscal stance targeted in these programs may have been too contractionary. Moreover, the arguments justifying the fiscal targets were often not clear. The evaluation therefore suggested providing a clearer and more explicit justification for the path of fiscal adjustment proposed in IMF-supported programs. This would enhance the quality of the analysis, promote greater understanding of the risks faced, and facilitate the mid-course corrections that were often needed in practice. A separate concern of the evaluation was that many of the fiscal measures proposed did not adequately guard against policy reversals or promote resilience to shocks. In response, the evaluation suggested that a more thorough analysis of structural reform priorities in the fiscal arena, for example in the context of Article IV surveillance, would enhance the ability of IMF-supported programs to foster more durable improvements to fiscal policy.
This review found considerable progress in analyzing and articulating the rationale for fiscal adjustment in IMF-supported programs. It also found improvement in the discussion of structural reform priorities in the fiscal area in both program and surveillance documents. Nonetheless, there is room for improvement in the way IMF program documents analyze and explain the magnitude and pace of fiscal adjustment—a dimension this review could still not rate as satisfactory. The review also found instances where structural issues important to fiscal management and sustainability were not convincingly articulated.
Given renewed interest in the design of fiscal adjustment in IMF-supported programs in the aftermath of the global financial crisis, the focus of the 2003 evaluation clearly remains relevant today. Indeed, recent discussion by the IMF and others of possible underestimation of fiscal multipliers under conditions of private deleveraging and accommodative monetary policy underscores the importance of assessing the economic and policy environment and private sector responses to them as accurately as possible in the context of IMF-supported programs—which was a key objective of the 2003 evaluation’s recommendations.
This review is part of a pilot project to revisit some of the IEO’s early evaluations in order to examine their continued relevance, and the extent to which their recommendations remain useful and have been implemented. In this context, the review assesses the IEO evaluation of Fiscal Adjustment in IMF-Supported Programs, which was discussed by the Executive Board in August 2003.
The 2003 evaluation was a far-reaching exercise. This follow-up note is more selective in scope and depth, focusing primarily on the core issues raised by the evaluation. It is based on a review of a sample of IMF-supported programs during 2005–10, other relevant IMF documents, and interviews with IMF staff.
The first section describes the key findings and recommendations of the 2003 evaluation, which, as will become apparent, remain highly pertinent a decade later. The second section assesses the implementation record in the ensuing period. The final section provides brief concluding observations.
Key Findings and Recommendations of the 2003 Evaluation
The 2003 evaluation assessed the quantitative and qualitative dimensions of fiscal adjustment in a cross-country sample of IMF-supported programs during 1993–2001. It found that some common criticisms of IMF-supported programs at the time—that they adopted “one-size-fits-all” approaches to fiscal adjustment, were inflexible, or caused declines in social spending—were not supported by empirical evidence. At the same time, it found that the rationale for key macroeconomic assumptions underlying the proposed fiscal adjustment and subsequent mid-course revisions were not clearly articulated in program documents.
There was a tendency by the IMF to adopt fiscal targets based on overoptimistic assumptions about the pace of economic recovery and investment rates. Because output and private investment often recovered at a slower pace than assumed, the result was an overcorrection of the current account deficit. In such situations a less contractionary fiscal stance might have more appropriately smoothed the adjustment—unless polices were constrained by financing or debt sustainability concerns. But the information and analysis justifying the fiscal targets proposed were often not made available: for example, most program documents reviewed for the evaluation did not adequately discuss how fiscal targets related to balance of payments problems, how the magnitude and pace of the adjustment would address such problems, or how the adjustment should depend on private sector demand and growth prospects. These considerations led the evaluation to propose that program documents should provide a stronger rationale for the specific fiscal targets proposed in programs.
A related concern was that many of the fiscal measures proposed under the IMF-supported programs did not adequately guard against policy reversals or promote fiscal resilience to shocks. Slow progress in implementing structural and institutional reforms in turn placed limits on the extent of fiscal adjustment that could be achieved in the short run and on the durability of that adjustment. Greater emphasis on structural reforms to strengthen the public finances would therefore enhance the capacity of fiscal systems to achieve more lasting adjustments.
However, program and surveillance documents did not adequately explain how key institutional reforms needed in the fiscal area to enhance fiscal sustainability could be implemented. While the evaluation recognized that the fiscal measures proposed in the relatively short-term context of an IMF-supported program could not fully incorporate the country’s longer-term fiscal reform agenda, it suggested that more effort be devoted to discussing the nature of the structural reform agenda and how the measures supported by the program related to it—in the context of surveillance if not in the program document. The reform “road map” and its prioritization would ideally be contained and updated in the course of IMF surveillance, and be used to anchor the short-term measures supported by the program.
Finally, although the evaluation did not find evidence that IMF-supported programs squeezed aggregate social sector spending, it nonetheless suggested the IMF could become more active in assisting middle-income countries to establish mechanisms to protect the most vulnerable during periods of budgetary retrenchment.
The following are the main recommendations of the 2003 evaluation, which elicited considerable support from Executive Directors:
- Program documentation should provide a more in-depth and coherent justification for the magnitude and pace of the fiscal adjustment and how it is linked with assumptions about the recovery of private sector activity and growth.
- Programs should give greater emphasis to the formulation and implementation of key institutional reforms in the fiscal area, even if (as is likely) they cannot be fully implemented during the program period.
- The surveillance process should be used more explicitly to provide a longer-term road map for fiscal reforms and to assess progress achieved.
- The IMF should clearly delineate the operational framework in which social issues will be addressed within program design in non-PRGF countries. This should include a clear indication of the IMF’s responsibilities and activities in this area.
- The internal review mechanism should place relatively more emphasis on the early stages of the process.
Assessing Implementation of the Evaluation Recommendations
IMF staff proposed a series of follow-up actions to address the IEO recommendations, described in IMF (2004), which was discussed by the Executive Board in March 2004. The status of implementation was updated in the first “Periodic Monitoring Report” (IMF, 2007). This section discusses the extent to which the recommendations of the 2003 evaluation have been implemented. It does not try to assess the extent to which actions by the IMF can be attributed to the IEO report, recognizing that many other factors may be involved.
Rationale for Fiscal Adjustment in IMF-Supported Programs
The first recommendation of the 2003 evaluation was for IMF program documentation to provide a more explicit articulation of the basis for the proposed fiscal stance, and how it was linked with assumptions regarding the recovery of the private sector. The purpose was to promote better analysis and a clearer understanding of how various economic agents would adjust to economic shocks and the proposed fiscal adjustment; to facilitate the review process and discussions at the Board; and to provide external audiences with a more convincing explanation of the rationale for the program.
A guidance note was issued by IMF staff in January 2004 on how program documents might best present and explain the appropriate size, pattern, and composition of fiscal adjustment, consistent with the evaluation’s recommendations. The 2004 guidance note is no longer available as a self-standing document, although some of its elements were subsumed in subsequent guidance notes.
Ten years on, has there been any change in how well the rationale for the fiscal adjustment projected in IMF-supported programs is spelled out? Do program documents now contain a more explicit articulation of the basis for the proposed fiscal stance, and how it is linked with assumptions regarding the recovery of the private sector? This review replicated the analysis in the original evaluation using program documents of 13 countries that had IMF-supported programs during 2005–10. It utilized the same questions used in the original evaluation to assess the extent to which the IMF explained the rationale, magnitude, and composition of the envisaged fiscal adjustment.2 These questions are listed in footnote 1 of Figure 1, and focus on how program documents deal with the motivation for the program and the determinants of the size and composition of the fiscal adjustment. The scope of this review is limited to completed Stand-By Arrangements (SBAs) that envisaged significant fiscal adjustment (or included some further adjustment that followed significant fiscal adjustment in the recent past).3
Figure 1.Degree to Which Program Documents Explain the Rationale, Magnitude, and Composition of the Envisaged Fiscal Adjustment: Original Versus Revisited Findings
Sources: Table 3.1 in IEO (2003) and program documents.
1The five questions are: (1) Do documents clearly explain the source of the existing or potential balance of payments problem motivating the program? (2) Do documents explain the country-specific mechanism by which the fiscal adjustment will help improve the balance of payments problem? (3) Do documents discuss how the pace and magnitude of fiscal adjustment is being set in order to address the actual or potential balance of payments problems? (4) If there are other factors influencing the envisaged fiscal adjustment do documents explain clearly how they influence that adjustment? (5) Do documents explain the rationale for the composition of the fiscal adjustment?
2The ratings are: H=highly satisfactory, S=satisfactory, M=marginally satisfactory, and U=unsatisfactory. See Appendix 2 in IEO (2003) for an explanation of these ratings. The figure depicts averages for each question across the 13 countries.
The overall results indicate improvement in the manner in which program documents explained the rationale and magnitude of the envisaged fiscal adjustment. Figure 1 compares the original versus revisited findings for each of the five questions examined; Figure 2 compares the distribution of ratings for each question for the two sample sets. There was a clear improvement across four of the five questions covered and a slight deterioration for the fifth question. Whereas three of five questions were rated less than satisfactory in the original sample, only one was so rated in the recent sample.
Figure 2.Degree to Which Program Requests Explain the Rationale, Magnitude, and Composition of the Envisaged Fiscal Adjustment: Distribution of Original and Revisited Findings1
A number of caveats are in order. The relatively small sample sizes used in both the original evaluation and this review indicate the need for caution in interpreting the results. Moreover, one possible source of bias should be pointed out. By necessity, the recent sample included more countries that were afflicted by global or regional crises than the earlier sample. As such crises are often the source of the balance of payments shocks requiring immediate adjustments, it may be easier to explain the need for fiscal adjustment in such cases—this may have been a factor driving the relatively large improvement observed for Question 1 in Figure 1. By the same token, it may be harder to lay out the fiscal adjustment agenda as clearly when the program is being prepared under crisis conditions, which would have made it more difficult to explain the rationale for the composition of the fiscal deficit adjustment proposed in the more recent sample. This may account for the lack of improvement observed for Question 5.
The one question which, although showing improvement for the recent sample period, was still rated less than satisfactory was Question 3, namely the discussion of the rationale for the pace and magnitude of fiscal adjustment needed to address balance of payments problems. This is perhaps not surprising as it requires making judgments about how fiscal adjustment should be calibrated to the likely path of private spending, the likely evolution of the current account balance under turbulent conditions, while concurrently estimating uncertain financing constraints. Nevertheless, the rationale for focusing on this question remains as pertinent today as it was at the time of the 2003 evaluation. Indeed, recent discussion of fiscal multipliers and their likely underestimation in the aftermath of the global financial crisis in IMF (2012a), Blanchard and Leigh (2013), and numerous other publications underscores the importance of assessing the economic and policy environment and private sector reactions as accurately as possible in the context of IMF-supported programs—which was a critical recommendation of the 2003 evaluation.
Coverage of the Medium-Term Fiscal Reform Agenda
The 2003 evaluation found that sustained progress on the fiscal reforms identified in the IMF-supported programs had been limited, in part due to excessive emphasis on meeting short-term quantitative targets which were reversible in the absence of parallel institutional reforms. This in turn was due in large part to a mismatch of time frames: the short horizon of programs relative to the time needed to complete institutional reforms. These findings were the basis for the evaluation’s recommendations for programs to stress the appropriate formulation and implementation of key institutional reforms in the fiscal area (even if they could not be implemented during the program period), and for surveillance to provide a longer-term road map for such reforms. These two recommendations are considered jointly here as both are concerned with the adequacy with which the medium-term fiscal reform agenda is presented in IMF documents, whether in program documents or Article IV consultations.
Following the 2003 evaluation, area departments were encouraged to make more active use of the Fiscal Affairs Department (FAD) fiscal strategy briefs to strengthen their discussions of structural fiscal issues in program countries (IMF, 2007).4 As of 2007, fiscal strategy briefs were maintained and regularly updated for about 75 countries. However, these briefs were discontinued in the aftermath of the 2008 downsizing, primarily due to budgetary cutbacks, and replaced with more succinct summaries of findings from FAD technical assistance to a large number of countries. In any event, in interviews of IMF staff conducted for this review, opinion was divided on the value of the briefs for preparing Article IVs and program documents.
The 2003 evaluation identified 101 specific fiscal-related reform measures that were subjected to conditionality, and developed various indices to measure the extent of implementation.5 For the purposes of this review, it was decided not to replicate the use of the various indices developed in the original evaluation to measure the extent of implementation of fiscal reforms since the small samples would have diminished the significance of specific comparisons. Instead, the adequacy of discussion of the structural fiscal reform agenda was assessed more broadly using the sample of 2005–10 IMF-supported programs identified in the section “Rationale for Fiscal Adjustment in IMF-Supported Programs” above. Prior Article IV documents were also reviewed to determine how well surveillance described the country’s longer-term fiscal reform agenda, focusing in particular on those instances where the discussion in the program document was found to be below satisfactory.
Overall, the discussion of fiscal reform issues was found to have improved in the recent sample compared to the sample in the 2003 evaluation, with 8 of the 13 program documents found to contain at least satisfactory discussions. However, there remains considerable variation in the extent and depth of such discussions across program documents and Article IV reports. In addition, the review encountered cases in which Article IV reports provided sufficient discussion of vital reform issues that impacted on fiscal policy, but these were not adequately reflected in the subsequent program request. In some instances, there was a discussion of the structural reform agenda in the text of the program document, but the associated structural performance criteria and benchmarks did not reflect these concerns, and tended to be overly focused on the next few months rather than the duration of the program. Finally, there were instances where the Article IV reports themselves did not contain an adequate discussion of the needed fiscal reform agenda. Illustrative examples, both positive and negative, are provided in Box 1.
Some of these findings may just reflect the need to develop an IMF-supported program rapidly. Yet both the program document and the credibility of prior surveillance would have been strengthened by discussion of the important structural fiscal policy constraints that would need to be addressed over the medium term.
Tackling Social Spending and Social Protection
The 2003 evaluation also addressed the issue of the impact of fiscal adjustment on social sector expenditures in the context of IMF-supported programs. On the basis of cross-section analysis, it found that aggregate social sector spending was not lower than it would have been in the absence of an IMF-supported program.6 At the same time, in-depth country studies indicated that even when aggregate social sector expenditures were maintained, critical areas of expenditure most relevant to the poor may be crowded out by other components of government expenditure such as wages and salaries. The evaluation concluded that these adverse effects could be limited if the most critical social sector expenditures could be better protected during times of crisis. The evaluation therefore proposed that the IMF should clarify its guidelines for work in this area, inter alia to assist middle-income countries, facing external shocks and budget retrenchment, to improve their ability to protect vulnerable groups.
Box 1.Coverage of Structural Fiscal Issues in Program Cases
The 2005 Turkey Stand-By Arrangement is an example where both the program document and the preceding Article IV consultation addressed structural fiscal issues well. The fiscal policy section in the program request contained a separate subsection on structural fiscal reforms, where a robust medium-term reform agenda was presented. With respect to tax reform, the document discussed the need to simplify the personal income tax; reduce the corporate income tax rate while broadening the base and improving compliance; retain value-added tax rates; and implement a new revenue administration law. With regard to expenditure reform, it discussed the importance of dealing with the widening social security deficit and assessed the nature of the legislation the authorities had submitted to Parliament; discussed the need for further reforms during the program period relating to civil service pay and employment, budgetary support for agriculture and state economic enterprises; and linked prospective reforms to forthcoming public expenditure reviews by the IMF and World Bank. The program document also discussed ongoing improvements to expenditure management and transparency through legislation to facilitate medium-term and strategic planning, as well as local government accountability.
By contrast, the review also encountered other country cases where issues important to fiscal management and sustainability were not convincingly articulated in the program documents and/or in preceding Article IV reports. Examples include:
- Addressing chronically low tax revenue. A more thorough discussion of the long-standing factors inhibiting revenue collection and how to address them, including by addressing weak tax administration, would have enhanced the credibility of the fiscal targets proposed.
- Tackling energy mispricing with chronically adverse fiscal consequences. There was inadequate discussion of how institutional arrangements for energy prices to reflect cost recovery and fluctuations in international prices can undermine fiscal objectives. Such discussion could have included descriptions (for example, in parallel Article IV reports) of more experiences of tackling similar issues.
- Pro-cyclical fiscal policy when dependence on commodity exports is high. Mechanisms to smooth the volatility of government spending over the commodity price cycle were not adequately discussed.
- There was uneven coverage of quasi-fiscal deficits arising from the activities of state-owned enterprises.
Although the objective of better protecting critical social expenditure in the context of IMF-supported programs was endorsed in the ensuing Board discussion, there was concern that the IMF should not become involved in the selection and design of social policy, which was considered outside its mandate and expertise. Indeed, in most of the programs reviewed for this exercise, there were no explicit objectives to protect vulnerable groups.7
Nonetheless, it is worth noting that subsequent work by IMF staff, particularly in the aftermath of the recent crisis, has promoted more active involvement in these issues. For example, included in the most recent review of conditionality, IMF (2012b) is the recommendation to: “Ensure inclusion in program design, where feasible and appropriate, of policy measures to mitigate adverse short-term impacts on the most vulnerable, such as adequate safety nets and possible temporary targeted transfers, particularly in programs with higher risks.” Further, the IMF factsheet on its role in helping protect the most vulnerable in the global crisis, IMF (2012c), indicated: “In this difficult environment, the IMF is helping governments to protect and even increase social spending, including social assistance. In particular, the IMF is promoting measures to increase spending on, and improve the targeting of, social safety net programs that can mitigate the impact of the crisis on the most vulnerable in society.” The factsheet cited examples from advanced, emerging market, and low-income countries.
Enhancing the Review Process
Finally, the 2003 evaluation recommended that the IMF’s internal review process be modified to allow an intensive brainstorming process in the early stages of program formulation so as to enable more meaningful incorporation of relevant comments into the program design.
The IMF has redesigned the review process applicable for programs (as well as Article IV missions), and brought forward the opportunity to comment on policy relevant issues in a more structured setting. The policy consultation meeting, introduced in 2008, is meant to serve as a forum for conveying substantive concerns about policy directions to be proposed by the IMF mission. At the same time, IMF staff interviewed for this review indicated that (rapidly deteriorating) country circumstances and sometimes internal considerations can still compress intended intervals between the policy consultation meeting and the timing of the mission.
The core issues and recommendations raised in the IEO’s 2003 evaluation remain as relevant today as when they were proposed. As such it is encouraging that progress has taken place in recent years in providing more coherent justification for the fiscal adjustment proposed in IMF-supported programs. Not surprisingly, the most difficult element of this task—deciding on the magnitude and pace of fiscal adjustment—requires further work. It requires assessing the private sector’s response to the shock that brought about the need for the IMF to intervene, as well as to the fiscal adjustment envisioned in the IMF-supported program. Deciding on the magnitude and pace of fiscal adjustment also requires balancing these private sector demand considerations against balance of payments constraints or debt sustainability concerns. These are challenging tasks, but explicitly considering such issues in the context of the fiscal adjustments proposed would further enhance the quality of IMF-supported programs. This conclusion, dating to 2003, would appear to be validated by recent work inside and outside the IMF, which indicates how significantly fiscal multipliers can vary depending on the state of the underlying economy, the type of fiscal adjustment undertaken, and the stance of parallel policies.
The discussion of institutional reforms in the fiscal area has improved in both program and surveillance documents, but here too there is no room for complacency. While care is needed not to overwhelm the fiscal adjustment effort associated with the program with discussion of institutional and structural reform priorities, the latter need to be clearly articulated in the context of surveillance. If these medium-term issues are covered well in surveillance, it would be more feasible to integrate quantitative adjustments with the longer-term institutional reform agenda in the context of IMF-supported programs.
Blanchard, Olivier, and DanielLeigh, 2013, “Growth Forecast Errors and Fiscal Multipliers,” IMF Working Paper No. 13/1 (Washington: International Monetary Fund).
Clements, Benedict, SanjeevGupta, and MasahiroNozaki, 2011, “What Happens to Social Spending in IMF-Supported Programs?” IMF Staff Discussion Note No. 11/15 (Washington: International Monetary Fund).
Independent Evaluation Office of the International Monetary Fund (IEO), 2003, Fiscal Adjustment in IMF-Supported Programs (Washington: International Monetary Fund).
International Monetary Fund, 2004, “Follow Up on the Recommendations of the Independent Evaluation Office Report on Fiscal Adjustment in IMF-Supported Programs,” SM/04/32, February (Washington: International Monetary Fund).
International Monetary Fund, 2007, “Periodic Monitoring Report on the Status of Board-Endorsed IEO Recommendations,” SM/07/383, December (Washington: International Monetary Fund).
International Monetary Fund, 2012a, World Economic Outlook, October (Washington: International Monetary Fund).
International Monetary Fund, 2012b, “2011 Review of Conditionality—Overview Paper,” SM/12/148, June (Washington: International Monetary Fund).
International Monetary Fund, 2012c, “The IMF’s Role in Helping Protect the Most Vulnerable in the Global Crisis,” Factsheet, September30. Available at www.imf.org.