Are fiscal targets set on a “one-size-fits-all” basis?

The evidence does not support the view that IMF-supported programs adopt a one-size-fits-all approach to fiscal adjustment. The average targeted fiscal adjustment in 133 programs was 1.7 percent of GDP (1.4 percent for the primary balance) with a great deal of interprogram variation. The evidence also does not support the perception that programs always involve austerity by targeting reductions in current account and fiscal deficits or in public expenditures. In fact, in 40 percent of programs the current account deficit was projected to widen. Primary fiscal deficits were also programmed to widen and primary expenditures to increase as a percentage of GDP in slightly over one-third of cases.

In principle, the size of the fiscal adjustment proposed in each case should depend upon country-specific circumstances. They include the scale of the adjustment needed in the current account and the associated reduction in absorption to achieve this adjustment, market perceptions of the need for fiscal adjustment in view of debt sustainability problems, and allocative considerations relating to the balance between the public and private sectors. Cross-section analysis provides some insights on possible determinants of the targeted fiscal adjustment:

• The targeted adjustment seems to respond to both the initial fiscal deficit and the initial level of public expenditures. Countries with larger initial deficits and larger levels of expenditures in relation to GDP tend to have larger programmed deficit reductions.

• There is a significant positive association between the targeted fiscal adjustment and the envisaged change in the external current account. However, on average, only a small fraction (one-fifth) of the targeted change in net external financing is reflected in a corresponding change in the targeted fiscal deficit.

• The composition of the fiscal adjustment reflects initial levels of revenues and expenditures. Increases in revenues are programmed when initial revenues are low and reductions in expenditures are envisaged when initial expenditures are relatively high, and vice versa.

• In the ESAF/PRGF arrangements, two-thirds of the fiscal adjustment on average was programmed to come from the expenditure side. In contrast, in the SBA/EFF-supported programs in nontransition economies, two-thirds of the fiscal adjustment was targeted to come from the revenue side. In the transition economies, both revenues and expenditures were targeted to decline, reflecting the declining role of the state.

• On average, programs targeted a fiscal adjustment of about 1 percentage point of GDP across all types of arrangements during the first year of the program. This figure seems quite stable across different subgroups. Except for the transition economies, this represents between one-half and two-thirds of the total fiscal adjustment over a two-year period.³

• The different role that revenues and expenditure adjustments were expected to have during the lifetime of the program is particularly marked in the case of SBA/EFF in nontransition countries. In fact, spending as a share of GDP was not envisaged to decline but rather to increase in the first year, being offset by robust revenue performance to bring about a reduction in the fiscal deficit. The expected relative contributions of revenue and spending are sharply reversed during the second year of the program, when spending reductions become more important.

Surprisingly, the rationale for the proposed fiscal adjustment is not very clear when we look at the 15 individual programs studied in this evaluation. An in-depth examination of staff reports and other Executive Board papers related to these programs reveals that these documents often do not explain adequately how the magnitude and pace of the programmed fiscal adjustment have been determined. Nor do most documents explain how the fiscal targets relate to the rest of the program, in particular to assumptions about recovery in private sector demand and short-term growth prospects.

Did programs achieve their fiscal targets?

On average, programs achieved only about one-half of the programmed improvement in overall and primary fiscal balances. However, there is, once again, significant variation around this average. About 60 percent of programs underperformed but 40 percent overperformed with respect to programmed deficit targets. The highest incidence of shortfalls was for SBA/EFF-supported programs in nontransition countries and the lowest was for SBA/EFF arrangements in transition countries.

Almost all fiscal adjustment on average takes place during the first year of the program. Except in the transition country arrangements, programs were unable to achieve further fiscal gains in the second year of the program in spite of more ambitious fiscal targets.

Cross-section analysis of the subset of programs that experienced shortfalls in fiscal performance suggests the following:

• Fiscal balances on average did not improve throughout the first two years of the arrangement—either in terms of overall or primary balances—except in the transition economies. Thus shortfalls appear to reflect weak fiscal performance rather than very ambitious fiscal targets.

• Overoptimism about fiscal adjustment is partly caused by overoptimism about growth projections. Absolute levels of revenue respond to growth with shortfalls in growth leading to corresponding shortfalls in revenue. However, absolute levels of expenditures, projected on the basis of optimistic growth forecasts, do not fall when growth falls below expectations, leading to an increase in expenditure ratios.

• There is a marked difference in the nature of fiscal shortfalls between programs that target a “large” fiscal adjustment (defined here as more than 3 percentage points of GDP over a two-year horizon, a definition that covers about 30 percent of the total sample) and others. In the latter group, excess expenditure as a share of GDP was the most frequent cause of the deficit shortfall, particularly in the nontransition countries.

• In contrast, revenue shortfalls were much more important in explaining shortfalls in performance in cases of “large” targeted fiscal adjustment. This pattern, which appears to hold both for concessional arrangements and programs supported by SBA/EFF arrangements, suggests that when substantial deficit reduction was judged necessary, programs aimed to achieve it through a combination of significant increases in revenues and cuts in expenditures.⁴ However, in practice, the revenue increases achieved were much smaller, while the targeted expenditure reductions were generally achieved—perhaps forced by financing constraints.

The extent of expenditure adjustment appears to vary according to the initial fiscal imbalance. When initial fiscal deficits are moderate, expenditure as a share of GDP was little reduced if at all, notwithstanding programmed reductions. However, when the initial deficit was large, much of the fiscal adjustment was ultimately fulfilled through spending cuts. Expenditures seemed to be reduced only if strictly necessary and only if financing possibilities were unavailable. Efforts to increase revenues in situations of substantial fiscal imbalance generally fell well short of target; this pattern has important implications for structural reforms in the fiscal area, which are discussed later.

Flexibility of fiscal targets

IMF-supported programs are sometimes criticized on the grounds that they are insufficiently flexible, forcing a rigid pattern of fiscal adjustment that is not sensitive enough to changes in circumstances. The cross-country evidence does not support this view. A high proportion of the programs studied (about two-thirds) had incorporated revisions to their initial fiscal deficit targets by the completion of the second program review.⁵ Of course, measuring the proportion of program targets that are revised is a rather narrow test of fiscal flexibility; it proves nothing about the appropriateness of any revisions. Nevertheless, it is important to note that in practice fiscal targets are revised frequently and these revisions are often associated with revisions in growth prospects. The cross-section data also suggest an interesting asymmetry in the process of revision: fiscal targets are revised downward when growth is below expectations, but they are less often revised upward when growth turns out to be higher than originally projected.

An examination of program and related documents suggests that the rationale for revisions is not clearly brought out. In particular, program documents often do not identify clearly what part of the fiscal shortfall was the result of exogenous developments (or unrealistic assumptions in the original program) and what part reflected a weaker policy effort. If, as often seems to be the case, insufficient progress in fiscal structural reforms is an important factor behind fiscal shortfalls, this needs to be frankly acknowledged in program reviews, and this is often not the case at present.

What has happened to economic recovery under programs?

A robust empirical investigation of the impact of IMF-supported programs on the pace of economic recovery is beyond the scope of this evaluation, and would involve comparing actual outcomes with the counterfactual of what would have happened to economic performance without a program or with an alternative program design. There is already a large, albeit inconclusive, literature on this topic.⁶ Our analysis of actual and projected growth in a large sample of programs suggests the following conclusions:

• Average growth rates for different groups do not reveal a general tendency for growth rates to decline in program years, compared with the trend in the preceding decade. However, these averages mask considerable cross-country variation and growth did slow down, especially in the first program year, in a significant number of cases. The experience of the group of capital account crisis cases is particularly noteworthy since the average growth rate for this subgroup was negative in the first program year.

• While IMF-supported programs did not suffer from a generalized decline in growth, they did suffer from overoptimism. Except for the sub-group of transition countries (where the growth outcome was marginally better than programmed) average growth outcomes over a two-year horizon were lower than projected.

• Optimism regarding growth recovery was particularly significant in programs that started from an adverse situation. When growth was negative during the first year of the program, growth projections for the second year were on average twice as high as in reality. Moreover, programs were generally reluctant to project a slowdown in growth and very rarely projected negative growth. For example, growth slowdowns between the first and second year of the program occurred twice as often as they were projected.⁷ Negative growth for the second year of the program was projected in only 1.3 percent of cases, but in reality it happened 10 times as frequently.

• Programs were also overoptimistic in projecting investment rates. Actual investment rates in the second year of the program were below projections in 60 percent of cases in a sample of 83 SBA/EFF arrangements. In about one-fourth of cases, investment rates were 5 percentage points of GDP or more below projections. Moreover, programs projected a decline in investment rates in one-fourth of cases while in reality investment rates declined in one-half of the arrangements.

Growth optimism, and especially the reluctance to forecast downturns in programs, has many causes, including especially the understandable desire of both the IMF and the authorities to present a relatively upbeat recovery scenario. However, this has important implications for program design because it understates potential risks and preempts a systematic discussion of the appropriate role of fiscal policy in the event of a significant economic downturn. This was clearly a major factor in the capital account crisis cases in East Asia, where—as suggested by the recent IEO study of three capital account crisis cases—adverse balance sheet effects on private demand were underestimated.⁸ It also seems to have been a factor in many other SBA/EFF-supported programs in nontransition economies.

Is there a contractionary bias in fiscal design?

The fact that both output and investment appear to be consistently lower than projected raises the issue whether there is a contractionary bias in fiscal design. Critics have argued that IMF-supported programs would ensure quicker recovery if they anticipated weak investment demand more accurately and therefore adopted a less contractionary stance of fiscal policy. A tight fiscal stance is not inappropriate when it is assumed that private investment demand is buoyant and fiscal contraction creates room for private investment to be financed. However it is not appropriate in situations where there is a sharp downward shift in the investment function, or when the level of private demand responds much more sluggishly to the program than originally projected. There is evidence that investment is consistently overestimated in IMF-supported programs and there is overcorrection of the current account deficit. In a large number of the cases the overperformance in the current account deficit is combined with an excess buildup of reserves, suggesting that the economy could respond positively to a demand stimulus. In such situations, it could be argued that a less contractionary fiscal stance might have been appropriate.

This conclusion needs to be qualified in one important respect. It focuses only on the role of fiscal adjustment via its impact on aggregate demand. However, emerging market countries relying upon international financial markets also have to consider the impact of their fiscal stance on market confidence and the resulting availability of external finance. Where debt sustainability is an issue, it may be desirable to adopt a tighter fiscal stance than justifiable on countercyclical grounds alone to ensure a quicker return to confidence.

It is difficult to determine in any particular case how to weigh these different considerations and come up with a fiscal stance that provides an appropriate balance. However, these issues need to be explicitly discussed and explained in program documentation. One of the conclusions of our evaluation is that this is not done in a systematic way. Board documents generally provide insufficient analysis and justification for the proposed fiscal adjustment path or the assumptions driving the projected recovery of private spending and how it is linked to program instruments, including the fiscal stance. Inclusion of such an analysis would help to avoid growth overoptimism. It would also provide a more coherent framework for sensitivity analysis that would alert staff early on in the process to what should be monitored as the program unfolds. We recognize that fiscal fine-tuning to take account of all these factors is extremely difficult and, in practice, a large part of the outcome must be based on judgment. However, a more explicit discussion of the key macroeconomic assumptions underlying the proposed fiscal path would promote greater understanding of the risks and uncertainties involved and also facilitate necessary mid-course corrections in the fiscal stance. Many such mid-course corrections do occur in practice, but their rationale is often unclear. A clearer statement of the original rationale would permit a more transparent basis for adjusting fiscal targets in the course of program implementation.

Internal review process

An examination of the internal review process, focusing on the comments of the Policy Development and Review Department (PDR) and Fiscal Affairs Department (FAD) on the fiscal aspects of the 15 individual programs studied in this evaluation suggests the following:

• Internal review comments do pay attention to the need to justify the specific fiscal stance, but (as noted above) these comments do not lead to an explicit analysis in the final Board documents of the factors that led to the determination of the fiscal stance. The possibility that projections of private sector activity and growth recovery were overoptimistic was generally not given much attention in the review process.

• In many cases, the scope and detail of review department comments was greater at the stage of program reviews than at the stage of initial program design. A comprehensive internal debate would have the greatest value added if it took place at an early stage of program formulation and involved an exploration of alternative policy options to achieve broad objectives. This approach would also be more conducive to encouraging domestic ownership of programs. Instead, the review process is much more reactive, with reviewers commenting increasingly as programs proceed, instead of at the design stage. This may reflect relatively sanguine initial judgments (associated with overoptimism in growth prospects, and policy implementation) that the fiscal and other targets would be achieved, followed by a closer look as revisions become necessary. We understand from staff that there are often considerable informal consultations on key design issues before the formal briefing paper stage. However, these are not substitutes for a more active examination of risks and options in the initial stages. The fact that Board documents in the programs we examined incorporated overly optimistic assumptions, and did not specify the links between the fiscal stance and the recovery of private activity and output, should be a matter of concern.

Projections of aid flows in concessional programs

Concerns have been raised that IMF-supported programs in low-income countries (that depend on concessional financing) may incorporate fiscal targets based on aid projections that “taper out” too quickly relative to what donors may be willing to provide. Some have suggested that this feature of program projections may in itself create a disincentive for donors to sustain their level of aid—even when programs remain on track.

To address this issue we have examined program projections for nearly 100 ESAF/PRGF programs approved in the period 1995–2001, complemented by an in-depth study of a sample of 20 concessional programs in sub-Saharan African countries. The results show that program projections of aid do tend to decline over the medium term, albeit at a moderate pace in most cases. However, there is no evidence that projections have systematically underestimated actual aid flows for the outer years of programs. The analysis used here cannot answer the much more complex question, which goes beyond the scope of the present evaluation, of whether more ambitious program targets for public expenditure (and deficits) could have resulted in the mobilization of additional concessional financing from donors.

Effect of IMF-supported programs on the level of social spending

There has been a long-standing debate on the impact of IMF-supported programs on public sector social spending. We address this issue through an econometric analysis of 146 countries from 1985 to 2000, looking at years with and without an IMF-supported program. In order to assess the impact of programs on expenditures in health and education, we controlled for other factors affecting social spending as well as for the endogeneity of the presence of an IMF-supported program.

The results show that the presence of an IMF-supported program does not reduce public spending in either health or education—measured as a share of total public spending, GDP, or in per capita real terms. In fact, we estimate that during program periods, and with all other factors being the same, public spending in each of the health and education sectors increased by about 0.3 to 0.4 percentage points of GDP compared to a situation without a program. This increase is sustained beyond the end of the program but it diminishes over time.

From the fact that social spending increases, it is not possible to argue that the most vulnerable groups of the population are effectively protected from the economic shocks they may experience during program years. This will depend on how that increased spending is targeted and timed. Unless governments already have in place programs and budgetary mechanisms that allow for that protection, IMF-supported programs generally have too short a time frame and the IMF lacks the necessary expertise to assist in implementing such policies. This suggests that an alternative framework may be needed to address such issues.

Social concerns in program design

Current practices of the IMF in the area of social protection in non-PRGF countries follow the 1997 Guidelines on Social Expenditures, which call for the IMF staff to track health and education spending and, by relying on work by the World Bank, encourage authorities to incorporate spending targets for these sectors in the Letters of Intent (LOIs) that spell out program objectives. The guidelines also encourage staff to monitor trends in basic social indicators (such as infant mortality and school enrollment) drawing on the World Bank. However, the guidelines are quite broad and general in scope, and discussions with staff suggest that there is considerable uncertainty about what is expected in practice, at least out-side the PRGF/PRSP countries. There also appears to be some uncertainty among the staff as to how the initiative to streamline conditionality should affect the IMF’s approach in this area.

A detailed examination of the 15 sample programs (complemented by 8 additional more contemporary programs to gauge recent progress) shows substantial variation in how social expenditure issues are treated in practice. Trends are noted in some program documents for broad categories of expenditures such as education and health. However, only one-third of the sample of 15 programs analyzes these trends and identifies priority social expenditures that need protection—although the most recent group of programs shows limited improvement in this respect. Performance criteria were rarely used to support social measures; however, 9 of these 23 programs used benchmarks or indicative targets. Only half of the more recent 8 program documents analyze changes in social spending and few programs (outside the PRSP/PRGF countries) discuss how explicit monitoring and feedback systems could be established or how these aspects would be integrated with the work program of the World Bank. Thus, the empirical basis for establishing and assessing policy actions in this area is often absent.

The internal review process by PDR and FAD quite often gave feedback in this area—providing specific suggestions to design and support priority social programs to protect vulnerable groups. Most of these comments, however, were concentrated in the review phase during program implementation, and hence were too late to influence basic program design.

An important finding from the case studies is that it is not necessarily costly to preserve critical programs or budgetary allocations to protect the most vulnerable groups from external shocks or budgetary retrenchments. This can be facilitated by some reallocations in the budget—a possibility particularly relevant for middle-income countries. However, the objective of protecting critical expenditures cannot be achieved simply by monitoring trends in broad social spending categories. Such monitoring would likely fail to capture micro-level reallocations that tend to take place in periods of fiscal stress that undermine social protection. As discussed in this report, spending categories that often are most critical to vulnerable groups come under pressure and are likely to be preempted by other expenditures during these periods (e.g., basic medical or primary school supplies being preempted by personnel expenditures).

The protection of critical spending categories and well-targeted programs in the social sector can thus play an important role in protecting the most vulnerable from adverse shocks and budgetary retrenchments at fairly low cost. Efforts should, therefore, be made to build such elements into program design whenever possible. This emphasis is consistent with the IMF Articles of Agreement, especially Article I (v), which states that one of the purposes of the IMF is to make “the general resources of the Fund temporarily available to [members]…providing them with opportunity to correct maladjustments in their balance of payments without resorting to measures destructive of national or international prosperity.” It would also help to make more concrete commitments by the IMF to “minimize the adverse effects [of macroeconomic adjustment on vulnerable groups] and, when some are inevitable to achieve the desired reforms, to mitigate these effects through compensating measures.”⁹

To be effective in this area, the IMF would need to work within an operational framework that takes account of four constraints: (1) policies in this area must be truly homegrown and fully owned; the major initiatives must therefore come from the country; (2) since the IMF does not have expertise on social sector issues, nor is this an area of its comparative advantage, inputs from other agencies, especially the World Bank, are critical; (3) there is a mismatch of time frames between the short-term nature of IMF programs and the longer-term time frame needed for building institutions and budgetary systems that can provide social support in times of crisis effectively; and (4) finally, it is necessary to ensure that the attempt to incorporate social protection into IMF programs does not contradict the recent streamlining initiative by leading to an overload of conditionality.

In the case of low-income countries, the PRSP framework is expected, in principle, to meet these requirements. However, there is at present no framework for non-PRGF eligible, predominantly middle-income, countries that would ensure identification of critical and homegrown social sector support programs that could be used as mechanisms for social protection at the time of crisis. The PRSP framework is obviously not appropriate for middle-income countries, but in the absence of any framework there will be a growing divergence between the way these critical social issues are treated between PRGF and non-PRGF countries. It is, therefore, necessary to revisit the 1997 guidelines with special reference to what IMF staff should do consistent with the new emphasis and special constraints discussed above.

Some elements of a workable approach can be readily identified. First, the mismatch of time frames suggests that necessary preparatory work in this area must be undertaken not at the time of crisis but much earlier, as part of normal surveillance. In order to ensure that initiatives are homegrown, the IMF could request governments to consider identifying critical social spending to be protected, or safety nets to be activated, in the event of crisis. The IMF could also encourage countries to approach the World Bank for assistance in this area. The IMF on its part, consistent with its mandate, could report on the authorities’ responses in this area and monitor progress.

Building on recent initiatives (such as the call for increased coordination on public expenditure management (PEM) issues), both institutions could work to develop a broad understanding with the authorities on the reforms needed and an appropriate sequencing for implementation. Where joint efforts are required, for example, in public expenditure management, a country-led work program would be jointly established. On the basis of the resulting joint effort, the IMF and the World Bank could assist the authorities in setting up mechanisms to track critical social spending through the budget and identify ultimate allocations, including to local governments where a significant amount of spending is decentralized.

Balance among policy measures emphasized by programs

Fiscal adjustments in programs have focused more on the revenue side than on reallocations and reforms on the expenditure side. On the revenue side, the accent has been on introducing or increasing value-added tax (VAT) rates, with less attention paid to income and property taxes and tax administration efforts aimed at reducing evasion. Sometimes these VAT rate increases have been resisted by broad segments of the population because they have been perceived to be inequitable relative to other revenue-raising possibilities.

The VAT needs to continue being promoted as the cornerstone of a modern tax system. However, stronger and parallel efforts should be made at improving collections, curtailing discretionary exemptions, and reducing tax evasion—particularly direct taxes (personal and corporate) and customs duties. Even in the short run, these efforts could yield important revenue increases if targeted at collecting from well-known taxpayers with arrears or those believed to be significantly underpaying (hence reducing the need for large increases in VAT rates to quickly generate revenues). When tax authorities have displayed determination in this area, the results have been impressive and have received wide support. This evaluation finds that efforts by the IMF in this area have not been forceful enough, both in the context of programs and in surveillance, particularly if they affect powerful vested interests. Often, tax administration reforms in IMF-supported programs have focused on the technology side rather than on politically more difficult actions, such as legislation to empower tax agencies to pursue tax evasion forcefully and for the system to be less prone to political interference.¹⁰ More forceful actions in this area may also increase the support of society at large for the overall reform agenda supported by programs.

Improving tax collection and reducing exemptions and evasion is an aspect of fiscal reform that should be pursued more vigorously. Estimates and comparisons of the extent of tax evasion should be made public, drawing where possible on cross-country analysis. These steps require both political will and institutional changes, in different mixes according to the specific situation, and should be unbundled.

On the expenditure side, an examination of the different programs shows that conditionality has been concentrated on short-term quantitative targets to reduce public employment or cap public sector wage increases (which generally prove to be short lived because they are easy to reverse) rather than focusing on the reorientation of public spending and medium-term civil service reform. As a result, progress in reducing the wage bill has been neither sustainable nor efficient—reversals have often occurred.

The internal review process often addresses these areas of weakness, including the need for expenditure reallocations, and (perhaps most important) the need for determined actions by the executive in the areas of reducing tax exemptions, limiting tax incentives, and taking concrete actions against tax evasion and tax arrears. But again, as in other areas, these comments come too late in the process to influence initial program design.

Progress in implementation

Our evaluation shows that progress in implementing fiscal reform initiatives in the sample of 15 programs was limited. In no given reform area was implementation satisfactory in more than 40 percent of cases. Measures to reduce the public sector wage bill, achieve civil service reform, and reform the social security system have been particularly difficult to implement.

This limited progress is often the result of an excessive emphasis on measures to meet short-term quantitative targets, rather than a focus on critical institutional changes that might extend beyond the end of the program. This is largely the result of a mis-match of time frames, such as the short horizon of programs relative to the time needed to complete these institutional reforms. Such reforms may need to be broken down into several steps: some of them can be started at the outset of the program with enough determination from the executive branch; others will require time to the extent they call for legislation and improvements in the implementation capacity of agencies. Surveillance could play a key role in providing such a road map, but, as the next section suggests, it often does not do so.

Learning and the process of surveillance

We have examined program request documents to assess the extent to which they look at the past in order to draw lessons. We also examined surveillance activity over the three-year period prior to the IMF lending arrangement in the 15 sample programs studied.

Program requests are only partly successful in evaluating past fiscal performance (with an index of success of 50 percent).¹¹ The results are worse (35 percent success) when documents are judged on whether they analyze policy failures under the prior arrangement. Overall, programs tend to focus on fiscal performance during the last year prior to the program, and rather independently of previous arrangements. Few efforts are made to analyze the factors behind policy failures.

We have also examined the link between surveillance and programs. Although there is significant variability, efforts during surveillance to forcefully flag the need to accelerate reform (in areas where implementation was lacking) have been limited, with an index of success of 40 percent. Surveillance is drawing too few lessons from past failures, often not setting future paths for more complex reforms.

Focusing on the unfinished reform agenda will require strong follow-up during surveillance, as well as continuity in successive programs. Our results suggest that surveillance does not forcefully flag policy inaction—many times it is insufficiently candid in language. Although based on a very small sample, self-standing surveillance does not seem to yield better results. This is a missed opportunity because we would expect that surveillance not associated with a program request or review would have a genuine opportunity to take a more strategic perspective on both, whether fiscal reforms over time add cumulatively to better fiscal systems, and what the remaining fiscal agenda for the future should be.

Surveillance could play a much more forceful role in providing a medium-term road map of structural reforms to be followed up over time, with or without programs. Progress and reasons for inaction should be reported candidly. That road map could then provide guidance for the specific reform priorities to be taken up in successive programs—this being particularly important in repeat users of IMF resources.¹²