The Macroeconomic Effects of Fund-Supported Adjustment Programs
Author:
Mr. Mohsin S. Khan
Search for other papers by Mr. Mohsin S. Khan in
Current site
Google Scholar Close
Publication Date:
01 Jan 1990
eISBN:
9781451947069
Language:
English
Keywords:
SP; deficit; program country; program effect; adjustment program; policy package; current account balance; Current account balance; Current account; Domestic credit; Current account deficits
Download PDF (2 MB)

The evidence is reviewed on the macroeconomic effects of Fund-supported adjustment programs, and new estimates are provided of these effects for 69 developing countries with programs during 1973–88. The empirical analysis indicates that in the short run programs have led to an improvement in the current account and the balance of payments, a lowering of inflation, and a decline in growth. In the longer run the positive effects of programs on the external balance and inflation are strengthened, and the adverse growth effects reduced. There is also evidence of a positive link between macroeconomic stability and economic growth.

Abstract

The evidence is reviewed on the macroeconomic effects of Fund-supported adjustment programs, and new estimates are provided of these effects for 69 developing countries with programs during 1973–88. The empirical analysis indicates that in the short run programs have led to an improvement in the current account and the balance of payments, a lowering of inflation, and a decline in growth. In the longer run the positive effects of programs on the external balance and inflation are strengthened, and the adverse growth effects reduced. There is also evidence of a positive link between macroeconomic stability and economic growth.

The Broad objectives of a Fund-supported adjustment program are the attainment of a viable balance of payments, satisfactory long-term growth performance, and low inflation. Generally, the need for an adjustment program, whether supported by the Fund or otherwise, arises when a country experiences an imbalance between aggregate domestic demand and aggregate supply, which is reflected in a worsening of its external payments position and a rise in domestic prices. Although supply shocks can cause divergence between aggregate demand and supply, often such imbalances can be traced to inappropriate policies that expand aggregate demand too rapidly relative to the growth of productive capacity of the economy. If this relative expansion of domestic demand is allowed to persist for an extended period, the country would experience a widening current account deficit, a loss of international competitiveness, higher inflation, increased distortions in relative prices, a declining growth rate, and a heavier foreign debt burden.

The primary role of the Fund in these circumstances is to assist the member country in designing a policy package that includes measures to restore a sustainable balance between aggregate demand and supply, and to simultaneously expand the production of tradables. Such an adjustment program takes the form of a set of policy intentions by the government that is judged by the Fund to warrant financial support. The choice of policies and the policy mix in a program result from extensive discussions between the country authorities and the Fund, and thus reflect the particular economic situation of the country and the preferences of the government.

A question that is frequently raised in connection with Fund-supported programs is whether such programs have been effective in achieving their macroeconomic objectives. Some writers have argued that, at best, Fund programs do little in the way of improving the economic picture,1 while others have gone as far as to say that programs worsen the situation by inducing stagflation.2 Providing a clear-cut answer to this question turns out to be no easy task. There is at present little agreement in the profession either about how to estimate the macroeconomic effects of programs, or about what impact past programs have actually had on the principal macroeconomic variables the Fund typically is concerned with. Although there have been a number of studies on the subject over the past decade,3 one cannot say with certainty whether programs have “worked” or not.

This paper has a twofold purpose: first, to review the evidence that is available on the effects of programs, paying special attention to the methodologies employed in the various studies that have been produced in the last ten years or so; and second, to estimate the effects of programs on the balance of payments, the current account balance, inflation, and growth in a group of 69 developing countries that implemented programs supported by the Fund during the period 1973–88. The survey of the existing literature, in combination with the new empirical evidence, essentially summarizes the current state of thinking on the topic and allows one to identify the problems and gaps that remain in evaluating the macroeconomic effects of programs.

The paper is organized as follows. Section I reviews existing empirical studies on the effects of Fund programs and analyzes alternative approaches that have been employed in estimating program effects, with a view to assessing their relative strengths and weaknesses. The empirical estimates of program effects are discussed in Section II. The concluding section summarizes the main points of the paper and identifies directions for future work on the subject of program evaluation. Appendix I provides a formal description of the alternative methods for estimating the macroeconomic effects of adjustment programs and the econometric biases associated with them, and Appendix II contains the relevant information on the sample of programs and the data used in this study.

I. Empirical Studies of the Effects of Programs

It is a popular misconception that the analytical model used to design Fund programs is simply the well-known monetary approach to the balance of payments (see, for example, Dell (1982) and Diaz-Alejandro (1984)). Consequently, it is argued that the Fund’s approach to economic stabilization, generally referred to as “financial programming,” relies solely on domestic credit restraint, possibly supplemented by devaluation, to achieve a desired improvement in the balance of payments. If this were indeed the case, then one would only have to test the effects of two policies—a reduction in the rate of domestic credit expansion and devaluation—on a single target—the balance of payments—to judge the effectiveness of Fund-supported programs.

However, while it is true that the monetary approach was operationalized in the Fund in the 1950s and 1960s (see International Monetary Fund (1977, 1987)), and still plays a central role in the design of programs, it does not represent the whole story by a wide margin. Underlying Fund-supported programs are complex packages of policy measures that have multiple targets—improving the current account balance and the overall balance of payments, achieving a satisfactory rate of growth, and reducing inflation being the primary ones. Aside from monetary and exchange rate policies, a typical program calls for, among other things, fiscal measures, such as restraint of government expenditures and increases in taxation, increases in domestic interest rates and producer prices to realistic levels, policies to raise investment and improve its efficiency, trade liberalization, wage policies, and external debt policies.4 Naturally, the theory underlying such a policy package combining aggregate demand policies, supply-enhancing policies, and relative price policies, goes well beyond the relatively straightforward analytics of the monetary approach to the balance of payments.

Since it is theoretically and empirically difficult to link all the policy measures contained in a Fund program to the ultimate targets of balance of payments, inflation, and growth, most studies have attempted to assess the effects of the overall policy package. In this type of approach the precise nature of the underlying economic relationships and the specific policies adopted are not made explicit, and attention is directed solely at determining whether programs have been “effective” in some sense in achieving the broad macroeconomic objectives for which they were formulated. Cross-country studies of stand-by and extended arrangements have been undertaken periodically within the Fund, and also by writers outside the Fund looking at various aspects of adjustment programs. Although there have also been case studies examining individual country experiences, at present the literature is dominated by cross-country analyses.

Before turning to a discussion of the alternative methodologies used in the evaluation of programs, it is important to define how the effects of programs ought to be measured. At the most general level, it has been argued that macroeconomic performance under a program should be compared to the “counterfactual”—defined as the macroeconomic performance that would have taken place in the absence of a program (see Guitián (1981)). The counterfactual is perhaps the most appealing yardstick against which to assess program performance and the standard most widely employed in economics to measure the impact of government policy interventions. What would have happened in the absence of a Fund-supported program is by no means the only standard against which to judge the outcomes of programs, but in many cases it is the most appropriate one. However, the counterfactual cannot, by definition, be observed and must be estimated or approximated. The various approaches used in evaluation studies should thus be judged in terms of how good they are in providing estimates of the counterfactual.

The literature suggests that four distinct approaches have been applied to the evaluation of Fund-supported adjustment programs: (1) the before-after approach, which compares macroeconomic performance during a program and performance prior to the program; (2) the with-without approach, which compares macroeconomic performance in countries with programs and performance in a “control group” of countries without programs; (3) the actual-versus-target approach, which compares actual macroeconomic performance under the program and the performance specified in its targets; and (4) the comparison-of simulations approach, which compares the simulated performance of Fund program-type policies and simulated performance with some other set of policies.

In the discussion that follows, the studies are grouped according to the approach that was employed by them.

Before-After Approach

In the literature on the effects of Fund-supported programs, the before-after approach has been the most popular. The first study to use this approach was by Reichmann and Stillson (1978). These authors examined a total of 79 Fund-supported programs implemented during 1963–72 and compared the behavior of the balance of payments, inflation, and growth during the two-year periods before and after the implementation of the program. Using nonparametric statistical tests, they found that there was a significant improvement in the balance of payments in only about one fourth of all programs. In a majority of cases (over 70 percent) there was no significant change in the balance of payments. Of the 29 programs involving countries with high inflation during the program period, the rate of inflation fell in 6 of 11 programs for which there was a notable deceleration in the rate of domestic credit expansion; in the 9 programs involving a devaluation, there were 5 in which inflation was higher. Finally, growth performance was examined for 70 programs, and it was concluded that, on balance, Fund-supported programs did not exert adverse effects on growth rates. In 40 percent of programs the growth rate declined after the inception of the program relative to the previous two years’ average rate of growth, but at the same time growth was higher in 47 percent of the programs.

A similar procedure was followed by Connors (1979) who examined a total of 31 programs in 23 countries that were adopted during the 1973–77 period. He compared periods of one year before and after the programs. Also using a nonparametric rank test, Connors concluded that Fund programs had no discernible effects on the ultimate targets—growth, inflation,5 and the current account deficit—or on important intermediate targets, such as the ratio of the fiscal deficit to gross domestic product (GDP).

The relationship between fiscal variables and Fund program performance was examined in more detail by Kelly (1982). The methodology was primarily of the before-after variety, and in order to take into account possible lags in adjustment, comparisons were made over both one-year and three-year periods. In a sample of 77 programs (covering 33 countries) during 1971–80, Kelly observed that when a one-year comparison was used, the fiscal deficit was reduced in 56 percent of the cases, and that the current account and fiscal deficits moved in the same direction in 62 percent of the programs examined. This last result was also supported by regression analysis that showed a positive and statistically significant relationship between changes in the ratio of the fiscal deficit to gross national product (GNP) and changes in the ratio of the current account to GNP. Furthermore, in about half the cases there was a decline in the average growth rate over a three-year period, and an increase or no change in the other half.

A study by Killick (1984) also attempted to capture the effects of lags by comparing the behavior of the balance of payments, the current account, growth, and inflation a year before the program with the behavior both one and two years after the programs. Killick employed non-parametric statistical tests to gauge the effects of 38 programs covering 24 countries during the period 1974–79. In contrast to other studies, Killick found that the balance of payments and the current account deteriorated irrespective of the time period over which the comparison was made. However, the difference between the preprogram and post-program values of these variables was not statistically significant. Inflation was reduced,6 but the effects on growth were ambiguous. In the first year after the program the rate of growth was higher, but by the second year the positive effect had eroded and the net effect was zero.7

Zulu and Nsouli (1985) also constructed before-after measures of program effects in their study of 35 programs implemented in 1980–81 for 22 African countries. They found that growth was lower or the same in the year after the program in about 60 percent of the cases. For the current account and inflation targets, the split was even—with as many programs showing an improvement in the current account and inflation performance as those showing a worsening or no change.

The most recent study using the before-after approach was that by Pastor (1987) for 18 Latin American countries during 1965–81. Using one-year comparisons and on the basis of alternative statistical tests, Pastor concluded that Fund programs led to a significant improvement in the balance of payments, but that apparently there was no effect on the current account, inflation, or the rate of growth of nominal GDP.8

While easy to employ and seemingly objective, the problem with the before-after approach is that it is based on a strict assumption of all other things being equal and will not yield an estimate of the independent effect of programs on macroeconomic outcomes whenever the non-program determinants of these outcomes are changing between the preprogram period and the program period. The fact is that these non-program determinants, ranging from external factors like industrial country growth rates, terms of trade variations, and movements in international interest rates, to domestic factors such as weather conditions, do change from year to year. As illustrated in Appendix I, this means that before-after estimates of program effects will typically be biased, because this approach incorrectly attributes all of the change in outcomes between the preprogram and program periods to program factors; and unsystematic over time, because estimated program effects for a given year will often be dominated by specific nonprogram influences of that year. If, for example, industrial country growth rises in a particular year, all programs in that year will look as if they performed very well, whereas if industrial country growth falls in some later year, all programs for that later year will look as if they performed poorly.

These shortcomings of the before-after approach make it a poor estimator of the counterf actual, because the situation prevailing before the program is not likely to be a good predictor of what would have happened in the absence of the program, given that nonprogram determinants can and do change from year to year. Of course, by making a judgmental correction for the influence of nonprogram factors, it is possible to improve upon the estimates of the counterfactual that would emerge from a mechanistic application of the before-after approach. Suffice to say, however, that such judgmental corrections are difficult to make, especially when the range of nonprogram factors is wide.

With-Without Approach

The with-without approach is designed to overcome the inability of the before-after approach to distinguish between program and nonprogram determinants of macroeconomic outcomes. The basic reasoning behind this approach is as follows. Assume countries with and without programs are subject to the same nonprogram determinants—that is, they face the same external environment. Then, so the argument goes, by comparing before-after changes in outcomes in program countries to those in the control group of nonprogram countries, the effects of nonprogram determinants will cancel out—leaving the difference in group performance to reflect only the effects of Fund-supported programs; Put in terms of the counterfactual, the idea is to use the observed performance of non-program countries as an estimate of what the performance of program countries would have been in the absence of a Fund-supported program.

The with-without group approach was first used in two studies by Donovan (1981, 1982), which analyzed a sample of programs implemented from 1970 to 1980. The control group of countries without programs was taken to consist of all non-oil developing countries, and the comparisons were carried out over one-year and three-year time horizons. In the first study (Donovan (1981)), which covered a sample of 12 programs (for 12 countries) implemented during 1970–76, the improvement in the rate of growth of exports was consistently higher for program countries than for all non-oil developing countries.9 The increase in the rate of inflation for program countries was about half that of the control group during the first year, and although it rose when three-year comparisons were undertaken, it was nevertheless well below the average increase in the rate of inflation of all non-oil developing countries. The outcome for growth was not as clear cut. In the one-year comparisons there was a sharp improvement in growth in program countries relative to the control group, but in the three-year comparisons growth in program countries fell by more than it did in nonprogram countries.

In Donovan (1982) the sample of programs was expanded to 78, covering the period 1971–80, and the same analysis as in the first study was undertaken. The balance of payments and the current account positions of program countries was found to improve relative to the control group in both the one-year and three-year comparisons. The increase in inflation in program countries was about half that of non-oil developing countries in the one-year comparisons, and fell to one third in the three-year comparisons. However, in contrast to the results of the first study, the rate of growth of real GDP fell by more than the average decline experienced by non-oil developing countries in the one-year comparisons, but by less in the three-year comparisons.

Loxley (1984) applied the same types of tests as Donovan (1982) to a group of 38 least-developed economies (defined as countries with per capita incomes of $690 or less in 1980) that had programs with the Fund during 1971–82. His results, however, were less definitive than those obtained by Donovan, in that the least-developed countries with programs did no better, on average, in terms of current account, balance of payments, and growth performance relative to other least-developed countries without programs, the program countries examined by Donovan, and all non-oil developing countries. The improvement was statistically significant only in the case of inflation, and then only in the three-year comparisons.

Gylfason (1987) also used a version of the with-without approach in his study of 32 programs implemented during 1977–79. The reference group included the developing countries that had experienced balance of payments difficulties during 1975–77, and nonparametric statistical tests were used to determine if the behavior of the macroeconomic variables for program countries over a three-year period was significantly different from that of the reference group. First, there was an improvement in the balance of payments in program countries relative to the outcomes in the control group. Second, inflation in program countries did not fall, but rather remained approximately the same as the average rate of inflation in the group. Third, the growth rate was not significantly affected by the program.

This pattern of results was basically replicated by Pastor (1987) for Latin American countries. Employing the same statistical methodology as Gylfason, Pastor also found that the balance of payments performance of program countries was significantly better than that of nonprogram countries, and that the differences in inflation and growth performance were not statistically significant.

While the with-without approach copes with some of the problems of the before-after approach, it is by no means ideal. The problem is that program countries differ systematically from nonprogram countries prior to the program period, and this difference matters for performance evaluation. In short, program countries are not randomly selected. Instead, they are adversely selected in the sense of having relatively poor economic performance prior to the program period. This is not surprising, since, after all, a basic requirement for Fund financial support is that the country have a balance of payments need. This requirement alone suggests that program countries would be expected to have weaker than average external positions when the program was implemented. In any case, nonrandom selection of program countries means that simple with-without estimates of program effects may be biased.10 Intuitively, the bias occurs because, under nonrandom selection, the with-without estimator attributes differences in outcomes exclusively to program status, when in fact the difference in starting positions itself is a cause of differences in subsequent performance between the two groups. Further more, the direction of the bias can go either way. If past economic difficulties signal less serious current difficulties, even in the absence of a program, then the with-without approach will overstate the beneficial effect of a Fund-supported program. Conversely, if past difficulties signal even more serious present difficulties, then the effect of programs will be understated.

There are ways, however, as discussed by Goldstein and Montiel (1986), to modify the with-without approach to reduce some of the biases. The basic idea is to accept the nonrandom selection of program countries, to identify the specific differences between program and non-program countries in the preprogram period, and then to control for these differences in initial positions in the comparison of subsequent economic performance.11 This procedure can be carried out with the use of what might be called the “generalized evaluation estimator.” To make this new approach work, one needs to identify the relevant reduced-form relationships that link policy instruments to policy targets, and the policy reaction functions that show how policy instruments change when the state of economy changes. The former are needed to determine the effects of alternative policies on the target variables, while the policy reaction functions are needed to discern what policies would have been chosen, given preprogram conditions, in the absence of a program. These empirical relationships are not easily implemented, and a serious margin of error in estimating the counterfactual is still likely to remain. But the main point is that the error is likely to be considerably smaller than under other approaches.

Goldstein and Montiel (1986) applied the generalized evaluation estimator to a sample of 68 programs for 58 developing countries implemented during 1974–81. These authors found that program countries systematically demonstrated weaker performance—that is, higher inflation, slower growth, and larger current account and overall balance of payments deficits—than nonprogram countries in the preprogram period. Adjusting for these preprogram differences in performances and taking into account the effects of policy instruments on targets, Goldstein and Montiel used regression analysis to estimate the program effects. Two interesting sets of results emerged from this study. First, there were no statistically significant effects of programs on the current account and balance of payments, on the rate of inflation, or on the growth of real output. Second, the estimated program effects under the generalized-evaluation estimation were quite different from those obtained with the standard with-without estimator. The latter indicated an improvement in the current account, a slight worsening of the balance of payments, a reduction in inflation, and a rise in the growth rate associated with programs. When the generalized evaluation estimator is used, the improvement in the current account ratio disappears, the deterioration in the balance of payments ratio is magnified, and the favorable outcomes for inflation and growth are reversed.12

Actual-Versus-Targets Approach

Another strand in the literature on program effects compares actual outcomes for certain key macroeconomic variables to the targets for such variables specified by the authorities and the Fund at the inception of the program. This approach has not been as frequently used as the other two approaches.13 Reichmann (1978), for example, studied 21 programs for 18 countries that were in effect during 1973–75 and compared the outcomes to targets for the balance of payments, inflation, and growth. He found that the balance of payments targets were met or exceeded in nearly two thirds of the programs. The targets for inflation were, however, exceeded in over half the programs. Greater success was achieved in the case of the rate of growth, with 62 percent of programs meeting the targets.

In a similar vein, Beveridge and Kelly (1980) surveyed the fiscal content of all 105 programs supported by the Fund that were approved during 1969–78. Their focus was on intermediate targets—domestic credit expansion, government revenues and expenditures, and deficit financing—rather than on the final macroeconomic objectives. Nevertheless, the results for the intermediate targets are informative, since achieving these is generally a necessary condition for meeting the ultimate targets for the balance of payments, inflation, and growth. Beveridge and Kelly showed that almost all programs contained government revenue and expenditure forecasts, and that both actual expenditures and revenues tended to differ from these forecasts. A shortfall in revenues occurred in about 40 percent of the cases, while expenditures were above projected values in nearly 60 percent of the programs. Consequently, the overall fiscal deficit targets were achieved in only about half the programs, as were the domestic credit ceilings. Finally, governments were more successful in meeting domestic nonbank financing limits than foreign financing targets. The former were satisfied in almost 70 percent of the programs, but foreign financing of the fiscal deficit exceeded the target in over 60 percent of the cases.

Zulu and Nsouli (1985) also analyzed actual outcomes and targets in their study of programs in African countries approved in 1980–81. They found that the current account targets were met in 38 percent of programs, and the inflation targets in about 48 percent of the cases, but growth targets were only achieved in less than 20 percent of the programs.

In the actual-versus-target approach, the success of a program is measured by the extent to which these targets were achieved. This approach, however, sheds little light on how the country’s macroeconomic performance was affected by a Fund-supported program. Failing to reach program targets is not necessarily synonymous with a program having no independent effect on outcomes. For example, if targets are too ambitious, or if unexpected nonprogram factors intrude in a negative way, actual outcomes may fall short of targets—even though the program may have produced a much better outcome than would occur in its absence or under some alternative set of policies. Similarly, under ambitious targets, or positive shocks, would lead to the meeting or exceeding of targets, even if the program produced weaker effects than would occur in its absence. Because the actual-versus-target approach is not amenable to estimation of the counterfactual, it does not allow one to weigh the costs and benefits of the adopted policy course against the alternatives.

Comparison-of-Simulations Approach

Unlike the other three approaches, the comparison-of-simulations approach does not infer program effects from actual outcomes in program countries. Instead, it relies on simulations of economic models to infer the hypothetical performance of Fund-type policies or policy packages and alternative policy packages. If the aim of the exercise is to evaluate the results of a specific Fund-supported adjustment program, then the use of actual program outcomes is indispensable. However, if the purpose is to evaluate the design and effectiveness of Fund-supported programs in general, then examining the likely effects of alternative policy packages can be quite useful and revealing.

Khan and Knight (1981), for example, constructed a small dynamic econometric model and estimated its parameters on a pooled crosssection, time-series sample of 29 developing countries, most of which had programs with the Fund. They then investigated the hypothetical effects of a stabilization program that pursued an external balance target using policies that figure prominently in Fund-supported programs—namely, domestic credit restraint and reductions in government expenditures. The simulation experiments showed that such a program produced a sharp price deflation in the first year, followed by a temporary burst of inflation as prices rose back to their equilibrium level. Output, however, contracted sharply in the first year, then rose temporarily above its full-employment level, approaching that equilibrium level gradually over a period of several years.

In a further study, Khan and Knight (1985) extended their simulation analysis to a comparison of alternative policy packages. Specifically, they compared the results for the balance of payments, inflation, and real output growth of a package of demand-management policies (that is, a once-for-all reduction in the rates of growth of nominal domestic credit and nominal government expenditures, plus a devaluation) with a combined package of demand-management and structural policies (that is, the above-mentioned demand-management policies, plus a set of structural policies that would gradually raise the rate of growth of capacity output). The demand-management package improved the balance of payments almost immediately, but at the cost of a temporarily higher rate of inflation and a short-run reduction in growth. The simulations of the combined package showed that structural policies could help to partially offset any short-term adverse effects on growth that might result from demand restraint as well as the inflationary consequences of devaluation. Furthermore, the longer-run effects of Fund-type policies on the balance of payments, inflation, and growth were more favorable than the short-run effects.

Broadly speaking, the comparison-of-simulations approach carries three advantages. First, one can draw on a wider body of adjustment experience, since the data base need not be restricted to countries with Fund-supported programs. Second, since the policy simulations are specified, one does not have to worry that incomplete implementation of policies, which is often a serious problem in Fund-supported programs, will blur the results. In contrast, the approaches that rely on actual outcomes require untangling the effects of policies from the degree of implementation. Last, and probably most significant, the comparison-of-simulations approach, by its very nature, focuses on the relationship between policy instruments and policy targets. As such, it provides better information on how programs work than do approaches that just look at the bottom line—that is, at the outcomes for policy targets.

Table 1.

Summary of Studies

article image

Comparison over one-year periods, unless otherwise noted.

Direction of change; (+) indicates positive effect, (–) indicates negative effect, 0 indicates no effect.

There are, however, practical problems with the comparison-of-simulations approach. To use this approach it is necessary to have at hand an empirical model that incorporates the relations between various policies and certain macroeconomic variables. While there have been some modest attempts at building such models for developing countries,14 there is as yet no single model available that covers the whole range of policy measures contained in a typical Fund program. Existing formal models are clearly unable to analyze all the questions relating to Fund-supported programs, and in particular, they do not capture the complex ways in which policy variables are related to the ultimate objectives of programs. Even if one had a suitable model to work with, one would still have to face up to the critique of ex ante econometric policy evaluation—that is, the so-called Lucas critique. Specifically, the parameters in econometric models may not remain invariant to changes in the policy regime, so that it would be incorrect to treat such parameters as fixed across policy simulations. Put in other words, the actual effects of hypothetical policy packages may turn out to be quite different than the simulated results—and in ways that are difficult to know in advance. An additional concern is that, due to credibility factors, the effect of a given policy may be different when it is implemented within the context of a program supported by the Fund than when it is implemented outside it. Agents may believe, for example, that policies agreed to with the Fund are more likely to be carried through and thus would be more inclined to change their behavior. Such confidence, or credibility, effects are automatically captured by the outcome-based approaches, but not necessarily by a model-based approach.

The studies reviewed above have focused on the effects of Fund-supported programs on macroeconomic targets, such as the external balance, growth, and inflation. Mention should also be made of studies that have considered the probable effects of programs on the distribution of income. Rather than provide quantitative estimates of program effects, Johnson and Salop (1980) and Sisson (1986) examine the likely effects of programs on the degree of income inequality in countries (see also Addison and Demery (1985)). These two studies reach similar conclusions—namely, that programs with the Fund have not in general worsened income distribution. Insofar as empirical evidence is concerned, Pastor (1987) is the only study that attempts to ascertain the quantitative effects of Fund-supported programs on income distribution. He finds that for 18 Latin American countries the implementation of a program resulted in a fall in the labor share in income relative to both its preprogram level, as well as to the corresponding share in non-program countries. The difficulties of trying to assess empirically the effects of programs on poverty are addressed in a recent paper by Heller and others (1988). This study undertook country case studies for nine programs that were in place in seven countries in the early 1980s. The authors concluded that in view of the limited evidence available at this stage, it would be unwise to draw firm inferences either way on how Fund programs affect living standards and poverty.

A summary of the results obtained by the various studies that have evaluated the effects of Fund-supported adjustment programs on the principal macroeconomic objectives is contained in Table 1. Overall, these studies yield three conclusions. First, there is frequently an improvement in the balance of payments and the current account, although a number of studies show no effects of programs. Second, inflation is generally not affected by programs. Finally, the effects on the growth rate are uncertain, with the studies showing an improvement or no change being balanced by those indicating a deterioration in the first year of a program.

II. Empirical Analysis of the Effects of Programs

In order to take a fresh look at the empirical effects of Fund-supported programs on the balance of payments, the current account balance, inflation, and growth, three alternative estimators were applied to a large sample of programs over the period 1973–88. These estimators, discussed at length in the previous section, were the before-after estimator, the with-without estimator, and the generalized evaluation estimator that corrects for some of the biases present in the other two estimators.15 The use of alternative estimators not only makes it possible to place greater confidence in the results, but also provides useful information on the sensitivity of program effects to different estimators. Only one other study (Goldstein and Montiel (1986)) has taken a similar agnostic approach in the choice of estimator to evaluate the effects of programs.

The data set used in the empirical analysis is the most up-to-date and comprehensive of those hitherto employed. The sample contains 69 out of a total of 76 developing countries that had upper credit tranche arrangements with the Fund during 1973–88.16 In the estimation, the term “program country” refers to those country-year observations (315) in which programs were in effect, and “nonprogram country” to the remaining (789) country-year observations. The latter form the control group—that is, countries without programs, against which program performance is compared.

The precise tests of program effectiveness are specified below, and the results obtained from these are described for the sample of programs under consideration.

Specification of Tests

The before-after estimator is simply the mean change in the target variable over some relevant period, usually one year. If Ay is the change in the target variable between the program year and the previous year, and d is a dummy variable for program status,17 a convenient way of calculating the before-after estimator is to run the following regression on a pooled time-series, cross-section sample of program and non-program countries:18

Δ y ⁢   = α 1 d . ( 1 )

The outcome variables—that is, the y’s—include the ratio of the balance of payments to GDP (BOP/Y), the ratio of the current account surplus to GDP (CA/Y), the rate of inflation (ΔP/P–i), and the rate of growth of real GDP (Δyr/yr–i). Estimating equation (1) for the change in each of these variables yields a direct estimate, as measured by the coefficient α1, of program effects based on the before-after approach.

The with-without estimator can be calculated quite easily by expanding equation (1) to include a constant term—that is:

Δ y ⁢   = α 0 ⁢     +   α 1 d . ( 2 )

The estimated value for the parameter α1 is equal to the difference in mean changes of the target variables for program and nonprogram countries.19 A statistically significant value for α1 would thus indicate that the change in the target variable for the program country was different from the corresponding change in that variable in nonprogram countries (the control group).

The generalized evaluation method involves specifying a reduced-form equation linking changes in each target variable to the program dummy, lagged values of all the target variables, lagged values of policy variables, and any external variables that would have an effect on the target variables. For this test the following equation was specified for each target variable:

Δ y ⁢   = α 0 ⁢     +   α 1 ( B O P / Y ) − 1 ⁢ + α 2 ⁢ ( C A / Y ) − 1 ( 3 ) + α 3 ( Δ P / P − 1 ) − 1 + α 4 ( Δ y r / y r − 1 ) − 1 + α 5 ( Δ D / D − 1 ) − 1 ⁢ + α 6 R E X − 1 ⁢ + α 7 ( F D / Y − 1 ⁢ )   + α 8 ⁢ ( Δ T O T / T O T − 1 )   + α 9 ⁢ ( T r e n d ⁢   + α 10 d , )

where, as before, the Δy includes changes in the ratio of the balance of payments to GDP, in the ratio of the current account to GDP, in inflation, and in growth, respectively. The policy variables are the percentage change in domestic credit (ΔD/D–1), the real effective exchange rate (REX), 20 and the ratio of the fiscal balance to GDP (FD/Y). Although these three variables are by no means the only policy measures in Fund programs, they are nonetheless the more important ones. The only external variable included is changes in the terms of trade (ΔTOT/TOT–1). Other relevant international variables, such as the growth of export markets, as well as domestic exogenous variables not included in equation (3), are assumed to be captured by a linear time trend. The effects of programs will again be determined by the sign and statistical significance of the parameter α10.

Results

Equations (1)–(3) were estimated for each of the four target variables using pooled time-series, cross-section data (1,104 observations). The first set of estimates is based on a one-year comparison—that is, performance in the program year is compared to performance in the preprogram year. The results are shown in Tables 2 and 3.

Table 2.

Program Effects: One-Year Comparisons

article image
Note: Figures in parentheses below coefficients are t-values.
Table 3.

Generalized Evaluation Estimates of Program Effects: One-Year Comparisons

article image
Note: Figures in parentheses below coefficients are t-values.

Table 2 reports the results for the before-after and with-without tests. With the before-after criterion for program evaluation, the balance of payments improves during the program year in relation to the previous nonprogram year. However, the magnitude of the effect is small and not significantly different from zero at the 5 percent level. The effect of programs on the current account balance turns out to be quite strong, and the coefficient measuring this is both positive and highly significant. Finally, although the coefficients are not statistically significant, the results indicate that both inflation and growth are lower in the program year than in the previous nonprogram year. If one were to take only the signs of the coefficients into account, one would conclude from the before-after tests that programs led to improvements in the balance of payments and the current account balance, a reduction in inflation, and a decline in the growth rate. On the basis of standard statistical criteria however, it would appear that programs lead to an improvement only in the current account position.

The with-without estimations, also shown in Table 2, display broadly the same pattern as was evident in the before-after tests. The balance of payments improvement in program countries relative to nonprogram countries is quite noticeable, and this time the effect is statistically significant at the 5 percent level. The current account improvement of program countries is also larger once allowance is made for the performance of nonprogram countries, and the coefficient is highly significant. Inflation is reduced by a greater amount in program countries, but although the standard error is reduced, the effect remains insignificant. The effect of programs on the growth rate is the only one that differs in sign from the before-after estimations. In the with-without case, programs are associated with an increase in the growth rate rather than a decrease. However, one cannot take this difference in results too far, since the estimated coefficient was not statistically significant in either the before-after or the with-without tests.

Table 3 shows the results for the generalized evaluation estimator—that is, equation (3).21 First of all, it can be noted that adjustment for preprogram differences does appear to matter. The coefficients of the own lagged values of each of the outcome variables are all statistically significant in the respective equations. Furthermore, in three out of the four equations the lagged value of at least one of the other outcome variables shows up as significant. This result demonstrates the statistical importance of preprogram conditions, and therefore suggests that the simple with-without estimator is biased for this sample of programs. The lagged percentage change in domestic credit turns out to have a significant effect only in the inflation equation (although with an incorrect negative sign), and the lagged real effective exchange rate in none of the equations.22 The lagged fiscal balance variable yields very plausible results. A decrease in the fiscal deficit improves the balance of payments and the current account, lowers inflation, and raises the growth rate. The coefficient measuring the effects of the fiscal balance is also significant in the inflation and growth equations. The change in the terms of trade has positive and significant effects on the balance of payments, the current account balance, and growth, but no statistically significant effect on inflation.

The more relevant results from the standpoint of this study are those dealing with program effects. In Table 3 the coefficient of the program dummy in the balance of payments equation is positive, though unlike the with-without estimations, it is not statistically significant. The current account improves in the program year, and although this improvement is considerably less than that observed in the before-after and control-group tests, it is nonetheless significant at the 5 percent level. Programs also appear to lead to a reduction in inflation, but this change is not statistically significant. Finally, the growth rate is significantly reduced in program countries relative to the change in nonprogram countries. This last result is broadly consistent with other empirical evidence on the short-run growth effects of Fund programs (see Khan and Knight (1985)).

The use of one-year comparisons to evaluate programs can, however, be misleading. Using too short a time horizon runs the risk not only of failing to capture the full program effects, but even of misrepresenting the direction of such effects when the short-run impacts are the opposite of long-run effects. Factors such as the downward stickiness of prices and wages, limited mobility of factors of production, and the difference between short-term and long-term price elasticities can combine to produce a one-year verdict on programs that is very different than, say, a two-year or three-year verdict.

Table 4.

Generalized Evaluation Estimates of Program Effects: Two-Year Comparisons

article image
Note: Figures in parentheses below coefficients are t-values.

To address this time-horizon issue, the estimations were redone comparing the behavior of the target variables in the preprogram year with the average behavior of these variables during the program year and the succeeding year. In other words, the target variable y is defined as the average of its values in periods t and t + 1, and the change in y, as the difference between this average value and the value in the previous year.23 This “two-year comparison” is expected to pick up some of the lagged responses of the target variables to the implementation of a program.24 For convenience, the focus from now on is solely on the generalized evaluation estimator, which, as argued earlier, is the most appropriate of the three used in the empirical tests.

The results of the two-year comparisons are shown in Table 4. Broadly speaking, the results are similar to those obtained in the one-year comparisons reported in Table 3. What is noteworthy is that the program effects are in line with the hypothesis stated earlier regarding the lengthening of the time horizon. The coefficient measuring program effects on the balance of payments almost doubles in size from that obtained in the one-year comparisons, and also becomes statistically significant. The program coefficient shows a substantial increase in the current account equation as well, and the statistical significance is maintained.

Although program effects on inflation continue to be statistically insignificant, one does observe an increase in the (absolute) size of the program coefficient. This would indicate a tendency of programs to reduce inflation by more when the evaluation period is extended. A similar pattern to that of inflation emerges in the results of the growth equation. The empirical evidence confirms that the negative effects of programs on the growth rate are reduced in the two-year comparisons relative to the one-year comparisons.

What the results in Table 4 indicate basically is that the positive program effects on the balance of payments, the current account balance, and inflation become stronger, and the negative effects on the growth rate become weaker, as the time period over which the comparisons are made is stretched out. It is certainly conceivable that further extension of the time period would yield even stronger positive effects. But a word of caution is in order here. Just as too short a period can yield misleading or erroneous results, so can too long a period, because it increases the likelihood that factors unconnected with programs will influence the outcomes. The problem is that the choice of the time period over which performance is judged is essentially arbitrary. For some variables it may be sufficient to examine performance over one year, whereas for others one may need to look over several years. It would, therefore, be useful in evaluating programs to review the existing evidence on the time lags associated with various macroeconomic and structural policies. Based on this evidence the evaluation procedure could be appropriately designed to capture the total effects of programs.

One further point to note in discussing the growth effects of programs is that there is a presumption, supported by limited empirical evidence, that the restoration of macroeconomic stability—which Fund policies primarily aim for—will itself have a beneficial effect on long-run economic growth. But linking macroeconomic stability to long-run growth is no easy task. At present, the economics profession lacks a fully satisfactory theory that directly relates macroeconomic policies to growth, other than via changes in the rate of capital formation. Therefore, macroeconomic policies are judged by the effects they have on investment (and savings), with the argument being that policies that have a favorable effect on investment are good for growth. However, growth is a multi-faceted process involving many factors other than physical investment, and it is not clear precisely how macroeconomic policies will affect variables such as the growth of the labor force, changes in the efficiency of investment, technological developments, and changes in human capital, all of which are important determinants of growth.

This relationship between long-run growth and macroeconomic stability was tested for the sample of program countries using three indicators of macroeconomic stability: (1) inflation; (2) the percentage change in the real exchange rate; and (3) the ratio of the fiscal balance to GDP.25 The results of these tests are shown in Table 5. Basically, an inverse relationship can be observed between growth and inflation, with the coefficient being significantly different from zero at the 10 percent level. A similar result is obtained for the relationship between growth and the real exchange rate—an appreciation of the latter leads to a decline in the growth rate. However, in this case the coefficient was statistically significant at the 1 percent level. Finally, an improvement in the fiscal position is associated with significantly higher growth.

Table 5.

Relationship Between Growth and Macroeconomic Stability, 1973–88

article image
Note: Figures in parentheses below coefficients are t-values; R2 is the adjusted coefficient of determination.

Taken at face value, the results in Table 5 confirm the generally held view that countries that have adopted appropriate policies—keeping inflation low, not allowing overvaluation of the currency, and maintaining fiscal responsibility—have, on average, also experienced higher growth rates. One could, therefore, argue that to the extent that programs succeed in their objective of establishing macroeconomic stability in the economy, they can be expected to have a positive impact on growth in the longer run.

The last question to be addressed is whether program effects are different between the 1970s and 1980s. Because Fund policies and programs are continually evolving and adapting to changing circumstances, one would suspect that this difference would be reflected in the effects that programs have on macroeconomic variables. To answer the question, the sample was divided into two subperiods—1973–79 and 1980–88—and the generalized evaluation equation (equation (3)) was evaluated, using two-year comparisons. The results for the coefficient of the program dummy for each subperiod are reported in Table 6.

Table 6.

Generalized Evaluation Estimates of Program Effects for Subperiods

article image
Note: As measured by the coefficient of the program dummy (α10); t-values in parentheses below coefficients.

It is readily apparent from Table 6 that there were indeed substantial differences in program effects between the two subperiods. In the 1973–79 subperiod the program effects on the balance of payments were very small and statistically insignificant. In the 1980s, however, programs had a far larger positive effect on the balance of payments, and also this effect was significantly different from zero. The beneficial effect of programs on the current account, however, was reduced in the 1980–88 period as compared to the earlier period, but the effects in both cases were statistically significant. The most striking difference to emerge is that for inflation. According to the estimates in Table 6, programs had a significantly positive effect on inflation in the 1970s. Although the sign of the program coefficient was reversed in the 1980–88 period, the coefficient was not statistically significant. In growth results, the relative sizes of the coefficients of the program dummy indicate that the 1980s programs had less adverse effects on the growth rates; however, since these estimated coefficients are not statistically significant, obviously one cannot make too much of this result.

In summary, on the basis of the tests here, and in particular the generalized evaluation estimations, the following conclusions can be drawn about Fund-supported programs implemented during the 1973–88 period.

  • There was an improvement in the balance of payments, but this improvement was statistically significant only when the period of evaluation was extended beyond the program year. This result held both for the entire period of analysis as well as for the 1980s.

  • The current account deficit was reduced. This result was the most robust of all, and stood up irrespective of changes in the time period over which performance was measured or over which the analysis was conducted. On average, the implementation of a program led to about a 1 percentage point improvement in the ratio of the current account to GDP.

  • The inflation rate was lowered, but this reduction did not show up as statistically significant in any of the tests performed.

  • The growth rate declined in the program year, but when the time horizon of performance evaluation was extended beyond the program year, the adverse growth effects were diminished. Furthermore, the achievement of macroeconomic stability has had a positive impact on long-run economic growth.

IV. Conclusions

Over the years, a number of empirical studies have examined the effects of Fund-supported adjustment programs on key macroeconomic variables, like the current account and the overall balance of payments, inflation, and the rate of economic growth. Such evaluations play an important role in the design of programs, because the lessons they yield—positive and negative—can be, and have been, incorporated into the thinking and operations of the Fund. Given the important role of the Fund in the adjustment process of developing countries, examination of the programs supported by the Fund is of obvious interest to the international community as well. The objective of this paper was to survey the studies produced both within the Fund as well as outside the institution, with a view to assessing where we stand at present with respect to our knowledge about the effectiveness of past programs and, further, to provide new estimates of the effects of programs using a comprehensive data set covering most of the programs implemented during the 1970s and 1980s.

The survey of the literature on the macroeconomic effects of Fund-supported programs would seem to point to two broad conclusions.

First, the empirical analysis that is available has been conducted using different methodologies, the relative merits of which deserve careful scrutiny. Most of the studies have attempted to gauge program effectiveness by comparing macroeconomic outcomes in program countries with the targets of these programs, with the performance prior to the implementation of the program, or with the observed performance of countries without programs. If, as argued in this paper, the proper standard for measuring program effects is to compare the macroeconomic outcomes under a program with the outcomes that would have emerged in the absence of a program, or under a different set of policies, then none of the approaches generally employed so far is fully satisfactory. More recently, there have been some limited attempts to apply the so-called counterfactual criteria through estimation of policy reaction functions for program and nonprogram countries and through simulation experiments with macroeconomic models. One can place more confidence in the results yielded by these later studies.

Second, one would be hard-pressed to extract from existing studies strong inferences about the effects of Fund-supported adjustment programs on the principal macroeconomic targets. There is some apparent consistency in the results for the current account and the overall balance of payments—with a small majority of studies indicating that programs lead to an improvement in both these variables—but the picture for inflation and growth performance is very unclear. On the basis of existing studies, one certainly cannot say whether the adoption of programs supported by the Fund led to an improvement in inflation and growth performance. In fact, it is often found that programs are associated with a rise in inflation and a fall in the growth rate. By and large, the evidence is fairly inconclusive.

More definitive evidence emerges from the tests performed in this paper. First of all, it is clear that for the sample used here, both the before-after and simple with-without estimates should be rejected in favor of the results of the generalized evaluation approach. Second, and more important, when the preferred estimator was used, there was typically an improvement in the first year of the program in the balance of payments and the current account balance, and a reduction in the rate of inflation. However, programs do involve some cost in terms of a decline in the growth rate. These results are in line with the predictions of the theoretical models that underlie the design of Fund-supported programs (International Monetary Fund (1987)). In the short run the demand-reducing elements of the policy package dominate—thereby depressing output while improving the balance of payments and inflation pictures. Supply-enhancing policies start to take effect after some lag, so that one-year comparisons do not indicate any positive growth effects of programs. Support for this hypothesis is provided by the comparisons of performance in the preprogram period with the average performance in the program year and the succeeding year. Indeed, it was found that the positive effects of programs on the balance of payments, the current account, and inflation were strengthened, and the negative effects on growth were reduced when the evaluation period was lengthened. In addition, it was shown that macroeconomic stability, as measured by different indicators, has a positive association with growth performance. Programs also appear to have been more effective in improving the external balance in the 1980s, compared to the 1970s.

These new empirical results indicate that, on average, Fund-supported programs have been more “effective” than previous analyses on the subject would suggest. However, an important drawback of the approach used here is that account was not taken of the degree of implementation of the policies agreed to between the Fund and the country. In other words, all programs were treated alike, whether or not the policy intentions contained in the program were carried out. Mixing countries with varying implementation records in the sample may well have biased the judgment about effectiveness. Had the tests been restricted to only those countries that successfully implemented the recommended policies, it is conceivable that an even more positive picture would emerge. But separating countries by degree of implementation is not easy. One rough way of making a distinction could be on the basis of the amounts drawn under the arrangement—that is, if a country was able to draw more than 50 percent or 75 percent of the approved amount, one could say that the program had been “implemented.” At the same time, care has to be taken in handling waivers and modifications of programs, which may well alter program objectives and policies. In other words, midcourse corrections or changes in a program may make the program quite different from the one originally negotiated.

Where do we go from here in terms of future evaluation work? The principal message of this paper is that comparing the macroeconomic outcomes of a program with the corresponding outcomes obtained under an alternative set of feasible policies is the most appropriate way of judging the effects or programs. However, the difficulties involved in following this approach should not be downplayed. Criteria based on the determination of the counterfactual involve a great deal of subjectivity and are difficult to employ in practice. Until we are able to develop suitable techniques to estimate the counterfactual, any study of the quantitative effects of Fund-supported programs has to be necessarily limited in scope. In this context, the new approach that was used in the tests in this paper appears promising and needs to be considered seriously in future evaluation exercises. If one is able to formulate correctly the policy reactions of the authorities in the absence of a program, then one will have moved a long way toward estimating the counterfactual.

Generally speaking, more also needs to be learned about how economic structure influences the effectiveness of various policies often included in programs. Rather than, say, examining past programs to determine whether devaluation works to improve the trade balance, one might search for the structural characteristics and circumstances that make devaluation more or less effective. These would include, for example, the commodity composition of exports and imports, the nature of the wage-price determination process, and the behavior of supporting expenditure-reducing policies, and so on. With this information at hand, one could predict the effects of a devaluation in a particular country, or groups of countries, and then ascertain whether the predictions are satisfied.

The focus on economic structure also raises the issue of the proper level of country aggregation for program evaluation work. There is much to be said for dividing efforts between case studies and larger multi-country studies. Case studies permit one to delve deeply into the specifics of program design and implementation, and to identify special circumstances surrounding the program.26 Unfortunately, case studies are an extremely time-consuming and expensive way to obtain evidence about program effects, and it may be difficult to generalize from the findings of only a few case studies. Large cross-country samples, in contrast, are easier to prepare and update, are more amenable to the application of uniform statistical techniques, and have the virtue of yielding “representative” information; yet, in the process one loses some of the country-specific aspects of programs, including the degree to which policies were implemented. Seen in this light, supplementing the results from cross-country studies with information from case studies that highlight some specific issue may be a convenient compromise.

In conclusion, as long as Fund-supported programs are to be an integral part of the adjustment strategies of developing countries, the search for an appropriate way to conduct evaluations of past Fund-supported adjustment programs must obviously continue. We do not as yet have the final word on the effects of programs, but it does appear that these effects are more positive than has previously been reported. This does not imply that the design and implementation of programs cannot be improved upon. Indeed, such improvements are continuously being sought by the Fund.

Appendix I Methodological Issues in Evaluating Fund Programs

This Appendix addresses in a more formal way some of the methodological issues associated with estimating program effects.

Suppose that the j th target variable in country i is determined according to

y i j ⁢ ⁢   = β o j ⁢ + x i ′ ⁢ β j ⁢   + w ′ ⁢ α j ⁢   + β j I M F ⁢ d i ⁢   + ε i j , ( 4 )

where yij is the target variable (for example, the balance of payments, the current account, the inflation rate, and the growth rate), xi is a vector of policy instruments (for example, domestic credit, the exchange rate, the fiscal deficit), w is a vector of foreign exogenous variables (for example, the terms of trade), di is a dummy variable, and €,ij is a random shock. The parameter vectors βoj, βj, and αj are assumed for simplicity to be uniform across countries. The dummy variable di takes on the value of unity if a Fund program is in effect during the period in question and zero otherwise, and the parameter βjIMF measures the effect of the program during this period on the variable yij.

It is important to note that this definition of βjIMF means that xi refers to the policies that would be adopted in the absence of a program. The vector is therefore directly observable only for nonprogram countries; for program countries the xi must be estimated. One way in which xi can be estimated is via the following reaction function:

Δ x i ⁢   = γ ′ ⁢     [ y i d ⁢   − ( y i )   − 1 ] ⁢   + η i , ( 5 )

where yt is a vector of target variables, yid is a vector of their desired values, and ηi is a vector of random shocks. The parameter matrix γ is conformable to Δxi, and yi, and Δ is a first-difference operator. Equation (5) says that the change in country i’ s macroeconomic policy instruments between the current and previous period will be a function of the difference between the desired value of the target variables this period and their actual value last period.

This model can be employed to examine the statistical properties of the before-after and with-without approaches to estimation of program effects. The before-after estimator, call it

article image
is

⁢   β j B A ⁢   = Δ y i j ⁢       f o r ⁢     i ∈ P , ( 6 )

where P denotes the set of program countries during the current period. The expected value of this estimator, conditional on observed values of the foreign exogenous variables, is

E ⁢ ( β j B A ⁢     | i ∈ P , Δ w ′ )   = β j I M F ¯ ⁢ ⁢ + Δ w ′ ¯ α j ⁢   + E ( Δ x i ′ ⁢   ⁢ β j ⁢ + Δ ε i j |   i ∈ P , Δ w ) , ( 7 )

which is equal to the true value

article image
only if

E ⁢ ( Δ x i ′ β i ⁢   + Δ ε i j |   i ∈ P , Δ w )   = − Δ w ′ ¯ α j . ( 8 )

That is, the before-after estimator is unbiased if one expects that the non-program determinants of yij would have behaved in such a way as to leave yij unchanged, on average, between the program and nonprogram periods.

The with-without estimator, βjWW, is given by

β j W W ⁢ = Δ y i j ⁢ − Δ y ¯ R j , ( 9 )

where

article image
is the average value of Δyij over some set R of nonprogram countries. Since we can observe Δx and Δεij for all i ∈ R, our information set, call it Ω, now consists of

Ω ⁢ = { ( Δ x i , Δ ε i j ⁢         f o r ⁢         i ∈ R ) , Δ w } .

Taking expectations of βjWW conditional on i ∈ P and Ω, yields

E ⁢   ( β i W W | i ∈ P , Ω )   = β j I M F ⁢ + E ( Δ x i ′ ⁢   β j ⁢   + Δ ε i j | i ∈ P , Ω ) ⁢   − ( Δ x ′ ¯ R β j ⁢   + Δ ε ¯ R j ) . ( 10 )

The with-without estimator will be unbiased if

E ⁢   ( Δ x i ′ ⁢   β i ⁢     + Δ ε i j | i ∈ P , Ω )   = Δ x ′ ¯ R β j ⁢   + Δ ε ¯ R j ; ( 11 )

or, in other words, if it can be expected that, in the absence of the program, the program country would have behaved just like the average member of the non-program reference group. Although the with-without estimator (unlike the before-after estimator) controls for the effects of changes in the global economic environment on target variables—because such global factors are assumed to affect program and nonprogram countries equally—it introduces a new source of bias—namely, the characteristics of nonprogram countries. If nonprogram countries differ systematically from program countries prior to the program period—in ways that matter for subsequent economic performance—then βjWW will be biased.

Table 7.

Fund Programs, 1973–88

article image
article image
article image
Note: Total number of program-years: 315.

An alternative to the before-after and with-without estimators can be derived by using equation (5) to substitute out the unobservable policy changes that would occur in the absence of a Fund program (that is, for xi) from equation (4). The generalized evaluation estimator is

Δ y i j ⁢   = β 0 i j ⁢     − ( y i j )   − 1 ⁢     ( γ ′ ⁢ β j ⁢   + 1 )   + ( x i ′ )   − 1 ⁢   β j ⁢   + w ′ ⁢ α j + β j I M F ⁢   d i ⁢ + ( ε i j ⁢ + η i ⁢   β i ) , ( 12 )

where the desired values yijd have been subsumed in the constant.

Econometric estimation of (12) produces an estimate of βjIMF, which is not subject to the criticisms leveled at the before-after and with-without estimators above. This equation takes care of the estimation of the counterfactual by controlling for the factors that are systematically related to the policies that would have been followed in the country without the program—that is, the lagged values of target variables and policy instruments.

Appendix II Data Description: Programs and Sources

The empirical analysis in Section II was conducted for a sample of 259 upper credit tranche programs (extended Fund facility and stand-by arrangements) implemented for 69 countries during the period 1973–88. The countries are listed in Table 7, with the program years indicated by an asterisk. The total number of observations in the sample (1,104) comprises 315 program-years and 789 non-program-years. The seven countries excluded from the total sample of all programs were Belize, Dominica, Equatorial Guinea, Grenada, Guinea, the Lao People’s Democratic Republic, and Solomon Islands.

All data except for the real effective exchange rate (REX) were taken from the World Economic Outlook (WEO) datafile. The REX’s are calculated as trade-weighted indices of relative consumer price indices expressed in a common currency. The definitions of the variables are:

article image
article image

REFERENCES

  • Addison, Tony, and Lionel Demery, “Macroeconomic Stabilization, Income Distribution and Poverty: A Preliminary Survey,” ODI Working Paper 15 (London: Overseas Development Institute, February 1985).

    • Search Google Scholar
    • Export Citation

  • Aghevli, Bijan B., and Jorge Marquez-Ruarte, A Case of Successful Adjustment: Korea’s Experience During 1980–84, Occasional Paper 39 (Washington: International Monetary Fund, 1985).

    • Search Google Scholar
    • Export Citation

  • Beveridge, W.A., and Margaret R. Kelly, “Fiscal Content of Financial Programs Supported by Stand-By Arrangements in the Upper Credit Tranches, 1969–78,” Staff Papers, International Monetary Fund (Washington), Vol. 27 (June 1980), pp. 205–49.

    • Search Google Scholar
    • Export Citation

  • Connors, Thomas A., “The Apparent Effects of Recent IMF Stabilization Programs,” International Finance Discussion Paper 135 (Washington: Board of Governors of the Federal Reserve System, April 1979).

    • Search Google Scholar
    • Export Citation

  • Dell, Sidney, “Stabilization: The Political Economy of Overkill,” World Development, Vol. 10 (August 1982), pp. 597–612.

  • Diaz-Alejandro, Carlos, “IMF Conditionality: What Kind?” PIDE Tidings (January-February 1984), pp. 7–9.

  • Donovan, Donal J., “Real Responses Associated with Exchange Rate Action in Selected Upper Credit Tranche Stabilization Programs,” Staff Papers, International Monetary Fund (Washington), Vol. 28 (December 1981), pp. 698–727.

    • Search Google Scholar
    • Export Citation

  • Donovan, Donal J., “Macroeconomic Performance and Adjustment Under Fund-Supported Programs: The Experience of the Seventies,” Staff Papers, International Monetary Fund (Washington), Vol. 29 (June 1982), pp. 171–203.

    • Search Google Scholar
    • Export Citation

  • Goldstein, Morris, and Peter J. Montiel, “Evaluating Fund Stabilization Programs with Multicountry Data: Some Methodological Pitfalls,” Staff Papers, International Monetary Fund (Washington), Vol. 33 (June 1986), pp. 304–44.

    • Search Google Scholar
    • Export Citation

  • Guitián, Manuel, Fund Conditionality: Evolution of Principles and Practices, IMF Pamphlet Series No. 38 (Washington: International Monetary Fund, 1981).

    • Search Google Scholar
    • Export Citation

  • Gylfason, Thorvaldur, “Credit Policy and Economic Activity in Developing Countries with IMF Stabilization Programs,” Princeton Studies in International Finance, No. 60 (Princeton, New Jersey: Princeton University, August 1987).

    • Search Google Scholar
    • Export Citation

  • Heller, Peter S., and others, The Implications of Fund-Supported Adjustment Programs for Poverty: Experiences in Selected Countries, Occasional Paper 58 (Washington: International Monetary Fund, 1988).

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund, The Monetary Approach to the Balance of Payments (Washington, 1977).

  • International Monetary Fund, Theoretical Aspects of the Design of Fund-Supported Adjustment Programs, Occasional Paper 55 (Washington: International Monetary Fund, 1987).

    • Search Google Scholar
    • Export Citation

  • Johnson, Omotunde, and Joanne Salop, “Distributional Aspects of Stabilization Programs in Developing Countries,” Staff Papers, International Monetary Fund (Washington), Vol. 27 (March 1980), pp. 1–23.

    • Search Google Scholar
    • Export Citation

  • Kelly, Margaret R., “Fiscal Adjustment and Fund-Supported Programs,” Staff Papers, International Monetary Fund (Washington), Vol. 29 (December 1982), pp. 561–602.

    • Search Google Scholar
    • Export Citation

  • Khan, Mohsin S., and Malcolm D. Knight, “Stabilization Programs in Developing Countries: A Formal Framework,” Staff Papers, International Monetary Fund (Washington), Vol. 28 (March 1981), pp. 1–53.

    • Search Google Scholar
    • Export Citation

  • Khan, Mohsin S., and Malcolm D. Knight, Fund-Supported Adjustment Programs and Economic Growth, Occasional Paper 41 (Washington: International Monetary Fund, 1985).

    • Search Google Scholar
    • Export Citation

  • Killick, Tony, ed., The Quest for Economic Stabilization: The IMF and the Third World (New York: St. Martins, 1984).

  • Kopits, George, Structural Reform, Stabilization, and Growth in Turkey, Occasional Paper 52 (Washington: International Monetary Fund, 1987).

    • Search Google Scholar
    • Export Citation

  • Loxley, John, The IMF and the Poorest Countries (Ottawa, Canada: North-South Institute, 1984).

  • Pastor, Manuel, “The Effects of IMF Programs in the Third World: Debate and Evidence from Latin America,” World Development, Vol. 15 (February 1987), pp. 249–62.

    • Search Google Scholar
    • Export Citation

  • Reichmann, Thomas M., “The Fund’s Conditional Assistance and the Problems of Adjustment: 1973–75,” Finance and Development, Vol. 15 (December 1978), pp. 38–41.

    • Search Google Scholar
    • Export Citation

  • Reichmann, Thomas M., and Richard T. Stillson, “Experience with Programs of Balance of Payments Adjustment: Stand-By Arrangements in the Higher Credit Tranches, 1963–72,” Staff Papers, International Monetary Fund (Washington), Vol. 25 (June 1978), pp. 293–309.

    • Search Google Scholar
    • Export Citation

  • Saracoglu, Rusdu, “Economic Stabilization and Structural Adjustment: The Case of Turkey,” in Growth-Oriented Adjustment Programs, ed. by Vittorio Corbo, Morris Goldstein, and Mohsin Khan (Washington: International Monetary Fund and the World Bank, 1987), pp. 119–39.

    • Search Google Scholar
    • Export Citation

  • Sisson, Charles A., “Fund-Supported Programs and Income Distribution in LDC’s,” Finance and Development, Vol. 23 (March 1986), pp. 33–36.

    • Search Google Scholar
    • Export Citation

  • Taylor, Lance, “IS/LM in the Tropics: Diagrammatics of the New Structuralist Critique,” in Economic Stabilization in Developing Countries, ed. by W.R. Cline and S. Weintraub (Washington: The Brookings Institution, 1981), pp. 465–502.

    • Search Google Scholar
    • Export Citation

  • Zulu, Justin B., and Saleh M. Nsouli, Adjustment Programs in Africa: The Recent Experience, Occasional Paper 34 (Washington: International Monetary Fund, 1985).

    • Search Google Scholar
    • Export Citation
*

Mohsin S. Khan is Assistant Director in the Research Department. He is a graduate of Columbia University and the London School of Economics and Political Science.

The author is grateful to Anupam Basu, Jagdeep Bhandari, Andrew Crockett, Sebastian Edwards, Morris Goldstein, Joshua Greene, Abbas Mirakhor, Peter Montiel, and Delano Villanueva for helpful comments, and to Erika Jorgensen and Ravina Malkani for excellent assistance.

1

See, for example, Killick (1984).

2

Taylor (1981) is the leading proponent of this view.

3

At least 13 cross-country studies of Fund programs have been published since 1978.

4

For a discussion of the policy content of Fund-supported programs, see International Monetary Fund (1987).

5

In the 12 countries in which the ceiling on domestic credit was breached, Connors (1979) found that inflation was significantly higher.

6

This effect was statistically significant.

7

In neither case, however, was the effect statistically significant.

8

It is not clear why the author chose to examine the effects on nominal rather than real GDP, as others have done.

9

Donovan (1981) did not directly examine the effects on the balance of payments and the current account in this study.

10

See Appendix I for a formal demonstration of this point.

11

By selecting the control group countries to include only those that had balance of payments problems in the preprogram period, Gylfason (1987) also attempted to adjust for this bias.

12

It should be noted, however, that the differences in the outcomes for the variables between program and nonprogram countries were not statistically significant.

13

This is most probably due to its reliance on confidential information on targets that only the country authorities and Fund staff have access to.

14

See, for example, the papers surveyed in Khan and Knight (1985).

15

Data limitations made it unfeasible to perform the actual-versus-targets tests, and the absence of a suitable macroeconomic model (with the necessary parameter estimates) precluded using the comparison-of-simulations approach.

16

See Table 7 in Appendix II for a list of the countries and programs. Seven small countries with programs were excluded because of lack of data on certain macroeconomic variables.

17

That is, d takes on a value of unity in those years when a program was in effect, and a value of zero in other years.

18

For a more precise definition of the before-after estimator, see Appendix I.

19

See equation (9) in Appendix I.

20

Strictly speaking, the real exchange rate is an intermediate objective and not a policy variable. It is assumed that in the short run a change in the nominal exchange rate will be reflected in a similar change in the real exchange rate.

21

Country dummies were introduced into the specification to allow for inter-country differences in the dependent variables. The coefficients of these dummy variables are not, however, reported, in order to economize on space.

22

This does not, of course, imply that the current values of these variables have no effect on the target variables.

23

In other words, Δy = 0.5(y+i + y) –y–1.

24

Although, in principle, one could also undertake three-year comparisons as well, the way the tests are set up would require assuming that policies were unchanged for three years (see Appendix I). This assumption is obviously not very plausible.

25

The variables are defined as averages over 1973–88 for these tests.

26

See, for example, the recent case studies of Korea by Aghevli and M&rquez-Ruarte (1985), and Turkey by Kopits (1987) and Saracoglu (1987).

Same Series

Other IMF Content

Other Publishers

  • Collapse
  • Expand
Cover IMF Staff Papers
IMF Staff papers, Volume 37 No. 2
Author:
International Monetary Fund. Research Dept.
Volume/Issue:
Volume 1990: Issue 002
Publisher:
International Monetary Fund
ISBN:
9781451947069
ISSN:
1020-7635
Pages:
263
DOI: