One of the main reasons why the evaluation of Fund-supported programs has often stimulated such widely varying views is that program effects—and perhaps even more so their measurement—have been interpreted in very different ways. At the same time some of the basic characteristics of program countries, particularly their situation prior to the program period, have often gone unnoticed.
The broad objective of Fund-supported stabilization programs has been summarized as “… the restoration and maintenance of viability to the balance of payments in an environment of price stability and sustainable rates of economic growth.”7 How can one determine whether Fund programs have fulfilled these objectives? The existing literature on the evaluation of Fund-supported stabilization programs contains no fewer than five measuring rods of the effects of programs:
(1) a factual standard; the difference between macroeconomic performance under the Fund-supported program and performance prior to the program—the before-after approach; what is versus what was;
(2) a normative measure; the difference between performance under the program and the performance specified in its targets—the actual versus target approach; what is versus what was to be;
(3) a conjectural standard (as are 4 and 5); the difference between performance under the program and performance that would have taken place in the absence of a Fund program—the actual versus in-the-absence-of approach; what is versus what would have been;
(4) the difference between performance under the program and the performance that could have taken place under an optimal set of policies—the actual versus optimal policy method; and what is versus what might have been; and
(5) the difference between hypothetical performance under Fund program-type policies and hypothetical performance under some other policies—the comparison of policies approach, i.e., what might have been under policy A versus what might have been under policy B.
The main strength of the simple “before-after” method is its objectivity. Program effects can be calculated from the changes in the relevant macroeconomic outcome variables as between the pre-program period and the program period. This standard is, however, an inadequate estimator of the independent effect of Fund programs on observed outcomes, be-cause the non-program determinants of these outcomes often change markedly between the pre-program period and the program period. Since the before-after approach attributes all changes to the program, the true effects will be understated or overstated whenever non-program determinants are changing. This problem would not be so serious if the 1970s and early 1980s were not so marked by large non-program influences, including, inter alia, two rounds of large oil price increases (in 1973–74 and 1979–80), widely varying rates of economic activity in industrial countries, and large fluctuations in real global interest rates (1973–79 versus 1980–84). The problem is just as serious when, for example, the changing non-program influences are of domestic origin (e.g., shifts in weather conditions that strongly affect agricultural output). Because of the importance of these non-program factors, the before-after approach can be useful to show what happened in program countries, but not why it happened.
As an illustration, Table 1 shows some before-after comparisons of internal and external balance for program countries over the 1973–83 period.8 Both weighted and unweighted averages of individual-country outcomes are presented. The main point to note is how variable over time the estimated program effects are. For example, using the weighted average figures, whereas changes in real growth rates in program countries were positive in 1973, 1974, 1976, 1981, and 1982, they were negative in 1975, 1977–80, and in 1983. Similarly, whereas the current account deficit to gross domestic product (GDP) ratio fell in 1974, 1975, and 1980, it rose in the eight other years. Since it is unlikely that the design or implementation of Fund programs changed significantly over this period, the figures in Table 1 suggest that changing non-program factors (of domestic and external origin) were contaminating the true independent effects of programs.
Before-After Comparisons of Macroeconomic Outcomes for Program Countries, 1973–83
(Change from pre-program year, in percent)
Before-After Comparisons of Macroeconomic Outcomes for Program Countries, 1973–83
(Change from pre-program year, in percent)
| Growth Rate of Real GNP | Inflation Rate | Ratio of Current Account Deficit to GDP | Ratio of Overall BOP to GD | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Weighted | Unweighted | Weighted | Unweighted | Weighted | Unweighted | Weighted | Unweighted | ||
| 1973 | 3.8 | 1.5 | 11 | 12 | 4.3 | 2.6 | 2.5 | 1.3 | |
| 1974 | 4.9 | 1.5 | 140 | 67 | −1.8 | −4.2 | −1.4 | −1.0 | |
| 1975 | −8.3 | −4.0 | −56 | −19 | −1.8 | 0.1 | 0.4 | 0.9 | |
| 1976 | 0.3 | 2.9 | 105 | 39 | 4.0 | 6.5 | 1.6 | 1.2 | |
| 1977 | −0.2 | 1.3 | 6 | 2 | 4.0 | 4.5 | 1.1 | 1.0 | |
| 1978 | −3.4 | 0.3 | 6 | 2 | 2.0 | 0.3 | 2.0 | 0.5 | |
| 1979 | −2.7 | −3.0 | 10 | 6 | 0.7 | 0.7 | 0.6 | −0.1 | |
| 1980 | −3.4 | −2.5 | 17 | 11 | −0.8 | −1.3 | — | −1.1 | |
| 1981 | 0.4 | 0.3 | 1 | 2 | 0.1 | −0.3 | −0.9 | −0.6 | |
| 1982 | 1.6 | 2.6 | 1 | −4 | 0.7 | 2.3 | 0.7 | 0.4 | |
| 1983 | −0.7 | 0.4 | 41 | 8 | 3.2 | 1.4 | 4.5 | 2.6 | |
Before-After Comparisons of Macroeconomic Outcomes for Program Countries, 1973–83
(Change from pre-program year, in percent)
| Growth Rate of Real GNP | Inflation Rate | Ratio of Current Account Deficit to GDP | Ratio of Overall BOP to GD | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Weighted | Unweighted | Weighted | Unweighted | Weighted | Unweighted | Weighted | Unweighted | ||
| 1973 | 3.8 | 1.5 | 11 | 12 | 4.3 | 2.6 | 2.5 | 1.3 | |
| 1974 | 4.9 | 1.5 | 140 | 67 | −1.8 | −4.2 | −1.4 | −1.0 | |
| 1975 | −8.3 | −4.0 | −56 | −19 | −1.8 | 0.1 | 0.4 | 0.9 | |
| 1976 | 0.3 | 2.9 | 105 | 39 | 4.0 | 6.5 | 1.6 | 1.2 | |
| 1977 | −0.2 | 1.3 | 6 | 2 | 4.0 | 4.5 | 1.1 | 1.0 | |
| 1978 | −3.4 | 0.3 | 6 | 2 | 2.0 | 0.3 | 2.0 | 0.5 | |
| 1979 | −2.7 | −3.0 | 10 | 6 | 0.7 | 0.7 | 0.6 | −0.1 | |
| 1980 | −3.4 | −2.5 | 17 | 11 | −0.8 | −1.3 | — | −1.1 | |
| 1981 | 0.4 | 0.3 | 1 | 2 | 0.1 | −0.3 | −0.9 | −0.6 | |
| 1982 | 1.6 | 2.6 | 1 | −4 | 0.7 | 2.3 | 0.7 | 0.4 | |
| 1983 | −0.7 | 0.4 | 41 | 8 | 3.2 | 1.4 | 4.5 | 2.6 | |
The second, “actual versus target” standard is as objective and straightforward to apply as the first, since targets for the relevant variables are quantified in Fund-supported programs. A comparison of actual with targeted results may also yield useful information on program design; a comparison across many programs, for example, may help to identify those factors—such as the early adoption of planned measures, flexibility in policy formulation, or sustained implementation of adjustment measures—that are most closely associated with the achievement of target outcomes. In addition, and unlike the before-after approach, the actual versus target method can allow for the influence of non-program factors through judicious setting of the program targets.
The other side of the coin is that the actual versus target standard can distort true program effects if targets are over- or under-ambitious, or when unexpected non-program factors intrude and cause out-comes to fall short, hit, or exceed targets. Table 2, taken from a staff review of upper credit tranche standby and extended arrangements approved in 1981, shows a representative comparison of actual and target results for Fund-supported programs in 1981. The comparison disaggregates the results by the main policy content of the programs: demand restraint; supply-oriented; and mixed strategy. The data show that while the current account, reserves, and inflation were broadly in line with program targets, real growth rates were less favorable than targeted, especially in demand-restraint and in mixed strategy programs. The review goes on to state that this result “… was partly attributable to the impact of depressed international economic conditions on the performance of exports.”
Actual Results and Targets for 1981 Program Countries
(In percent)
Excludes Dominica and Grenada.
Reserve data refer to the end of the indicated period and are calculated in weeks of imports.
Excludes Costa Rica and Uganda.
Actual Results and Targets for 1981 Program Countries
(In percent)
| Demand Restraint Programs | Supply Oriented Programs | Mixed Strategy Programs | ||
|---|---|---|---|---|
| Ratio of external current account balance to GDP1 | ||||
| Target | −13.6 | −4.3 | −12.8 | |
| Actual | −12.5 | −4.3 | −12.6 | |
| International reserves2 | ||||
| Target | 9.9 | 10.8 | 6.3 | |
| Actual | 8.6 | 10.6 | 6.2 | |
| Real growth rate | ||||
| Target | 4.2 | 5.2 | 3.7 | |
| Actual | 0.4 | 4.7 | 1.5 | |
| Rate of inflation3 | ||||
| Target | 12.3 | 14.7 | 18.2 | |
| Actual | 15.0 | 11.7 | 21.7 | |
Excludes Dominica and Grenada.
Reserve data refer to the end of the indicated period and are calculated in weeks of imports.
Excludes Costa Rica and Uganda.
Actual Results and Targets for 1981 Program Countries
(In percent)
| Demand Restraint Programs | Supply Oriented Programs | Mixed Strategy Programs | ||
|---|---|---|---|---|
| Ratio of external current account balance to GDP1 | ||||
| Target | −13.6 | −4.3 | −12.8 | |
| Actual | −12.5 | −4.3 | −12.6 | |
| International reserves2 | ||||
| Target | 9.9 | 10.8 | 6.3 | |
| Actual | 8.6 | 10.6 | 6.2 | |
| Real growth rate | ||||
| Target | 4.2 | 5.2 | 3.7 | |
| Actual | 0.4 | 4.7 | 1.5 | |
| Rate of inflation3 | ||||
| Target | 12.3 | 14.7 | 18.2 | |
| Actual | 15.0 | 11.7 | 21.7 | |
Excludes Dominica and Grenada.
Reserve data refer to the end of the indicated period and are calculated in weeks of imports.
Excludes Costa Rica and Uganda.
Indeed, exports grew on average by 11 percentage points less than anticipated. The accompanying analysis reports that in early 1981, prospects for an early world economic recovery seemed considerably more favorable than proved to be the case, as Table 3 shows. If forecasts like those in Table 3 were incorporated in program targets, any under-achievement of, say, growth targets need not imply an ineffective program.9
Forecast and Actual Non-Program Variables, 1981–82
(Annual average percentage change)
Forecast and Actual Non-Program Variables, 1981–82
(Annual average percentage change)
| Forecast | Actual | ||
|---|---|---|---|
| Industrial countries | |||
| Real GDP | 1.5 | 0.4 | |
| Import volume | 1.5 | −1.5 | |
| World trade prices | |||
| Oil | 12.1 | 2.5 | |
| Non-oil primary commodities | 2.4 | −13.5 | |
| Terms of trade for NODCs | −1.3 | −3.3 | |
| Interest rates (3-month LIBOR. in percent per annum) | 12.0 | 15.0 | |
Forecast and Actual Non-Program Variables, 1981–82
(Annual average percentage change)
| Forecast | Actual | ||
|---|---|---|---|
| Industrial countries | |||
| Real GDP | 1.5 | 0.4 | |
| Import volume | 1.5 | −1.5 | |
| World trade prices | |||
| Oil | 12.1 | 2.5 | |
| Non-oil primary commodities | 2.4 | −13.5 | |
| Terms of trade for NODCs | −1.3 | −3.3 | |
| Interest rates (3-month LIBOR. in percent per annum) | 12.0 | 15.0 | |
The “actual versus in-the-absence-of” approach has at least three important strengths. First, by comparing actual program outturns to what would have happened in the absence of a program, it recognizes that the benefits and costs of a program cannot meaningfully be evaluated in a vacuum; rather, they must be weighed against the benefits and costs of the alternatives. And while the no-program alternative is not the only one, it is in many cases the most realistic. Comparing alternatives is particularly relevant for evaluating Fund programs because, as shown below, there is strong evidence that program countries are in an unfavorable situation with respect to growth, inflation, and the balance of payments before the program period begins. A comparison of actual results with the alternative of having no program also means of course that programs can have positive (or negative) effects even when the macroeconomic indications during the program period itself are unfavorable (or favorable) since the relevant alternative policies may have produced a significantly more unfavorable (or favorable) result.
The second key strength of this third approach is that in principle it permits the separation of program from non-program influences on observed outcomes, so as to produce an estimate of the independent effects of programs. Specifically since what would happen in the absence of a program reflects all non-program influences, exogenous events like oil price disturbances or marked changes in industrial-country growth need not blur the effects of a program.
The fact that estimates of the effects of programs can be adjusted for unexpected events constitutes the third advantage of this approach. Unlike the actual versus target method, which compares actual outturns with targets set prior to or simultaneous with the program period, the actual versus in-the-absence-of method can estimate what would have happened with the benefit of hindsight; it thus bypasses the problem of forecasting accuracy.
The chief problem with the actual versus in-the-absence-of approach is its subjectivity. More specifically, it turns out to be very difficult to estimate what would have happened in the absence of a Fund program, the so-called counterfactual. First, as indicated earlier, the situation before the program will usually not provide a good basis for an estimate of the counterfactual because non-program determinants of macroeconomic results often change significantly during the program period itself. Second, and perhaps less obviously, the observed macroeconomic performance of non-program countries will generally not be a good control (even when the non-program group consists of other non-oil developing countries) because: first, program and non-program countries appear to differ systematically prior to the program period in ways that probably matter for subsequent economic performance; second, macroeconomic outcomes in non-program countries may not be completely independent of Fund programs; and third, the effects of different policy strategies in program and non-program countries may not be adequately captured by the standard macroeconomic indicators unless the observation period is quite long. Each of these points merits a brief comment.
On the systematic differences between program and non-program countries, Table 4 provides the mean growth rates, inflation rates, and external positions for both program and non-program countries in the year prior to Fund programs. The calculations are done separately for each year since 1972 and for the 1973–83 period as a whole. The message is clear: program countries are different from non-program countries in the pre-program year. They had (on average) larger balance of payments and current ac-count deficits in proportion to GDP, lower rates of real output growth, and generally higher rates of inflation than non-program countries. The same result holds if one uses unweighted averages or if the program-country group is expanded to include users of the Fund’s compensatory financing facility. These differences between program and non-program countries are statistically significant (at the 95 percent confidence level), and this not only for the program country group and time period shown in Table 4 but for other samples and time periods as well.10
Macroeconomic Outcomes in Program and Non-Program Countries in Year Prior to Program Period, 1973–83
Excludes high-inflation years of 1974 and 1975.
Macroeconomic Outcomes in Program and Non-Program Countries in Year Prior to Program Period, 1973–83
| Ratio of Current Account to GDP | Ratio of Balance of Payments to GDP | Real Growth Rate | Inflation Rate | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Program | Non-program | Program | Non-program | Program | Non-program | Program | Non-program | |||
| 1973 | −4.5 | −1.5 | 0.9 | 1.2 | 3.1 | 5.5 | 8.0 | 10.9 | ||
| 1974 | −1.5 | −1.3 | −0.1 | 1.1 | −3.5 | 10.3 | 232.9 | 13.3 | ||
| 1975 | −6.0 | −3.3 | −1.2 | 0.2 | 4.5 | 5.2 | 230.5 | 19.9 | ||
| 1976 | −6.3 | −3.7 | −0.8 | −0.4 | 2.6 | 5.1 | 79.4 | 20.7 | ||
| 1977 | −11.1 | −2.4 | −0.3 | 0.9 | 3.1 | 4.0 | 22.3 | 39.0 | ||
| 1978 | −3.6 | −2.5 | −1.1 | 1.1 | 4.3 | 5.7 | 70.1 | 19.7 | ||
| 1979 | −4.3 | −2.7 | 0.2 | 1.2 | 3.7 | 6.8 | 33.8 | 23.1 | ||
| 1980 | −6.2 | −3.0 | −0.5 | 0.9 | 4.3 | 5.2 | 26.9 | 30.2 | ||
| 1981 | −4.7 | −4.2 | −1.0 | 0.2 | 3.4 | 5.4 | 20.9 | 41.6 | ||
| 1982 | −9.0 | −4.7 | −2.3 | −0.5 | 2.3 | 2.8 | 37.6 | 37.8 | ||
| 1983 | −5.4 | −3.7 | −5.6 | −0.1 | −0.2 | 3.4 | 62.8 | 16.0 | ||
| 1973–83 | −5.7 | −3.0 | −1.1 | 0.5 | 2.5 | 5.4 | 75.0 | 24.7 | ||
| 40.21 | 26.51 | |||||||||
Excludes high-inflation years of 1974 and 1975.
Macroeconomic Outcomes in Program and Non-Program Countries in Year Prior to Program Period, 1973–83
| Ratio of Current Account to GDP | Ratio of Balance of Payments to GDP | Real Growth Rate | Inflation Rate | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Program | Non-program | Program | Non-program | Program | Non-program | Program | Non-program | |||
| 1973 | −4.5 | −1.5 | 0.9 | 1.2 | 3.1 | 5.5 | 8.0 | 10.9 | ||
| 1974 | −1.5 | −1.3 | −0.1 | 1.1 | −3.5 | 10.3 | 232.9 | 13.3 | ||
| 1975 | −6.0 | −3.3 | −1.2 | 0.2 | 4.5 | 5.2 | 230.5 | 19.9 | ||
| 1976 | −6.3 | −3.7 | −0.8 | −0.4 | 2.6 | 5.1 | 79.4 | 20.7 | ||
| 1977 | −11.1 | −2.4 | −0.3 | 0.9 | 3.1 | 4.0 | 22.3 | 39.0 | ||
| 1978 | −3.6 | −2.5 | −1.1 | 1.1 | 4.3 | 5.7 | 70.1 | 19.7 | ||
| 1979 | −4.3 | −2.7 | 0.2 | 1.2 | 3.7 | 6.8 | 33.8 | 23.1 | ||
| 1980 | −6.2 | −3.0 | −0.5 | 0.9 | 4.3 | 5.2 | 26.9 | 30.2 | ||
| 1981 | −4.7 | −4.2 | −1.0 | 0.2 | 3.4 | 5.4 | 20.9 | 41.6 | ||
| 1982 | −9.0 | −4.7 | −2.3 | −0.5 | 2.3 | 2.8 | 37.6 | 37.8 | ||
| 1983 | −5.4 | −3.7 | −5.6 | −0.1 | −0.2 | 3.4 | 62.8 | 16.0 | ||
| 1973–83 | −5.7 | −3.0 | −1.1 | 0.5 | 2.5 | 5.4 | 75.0 | 24.7 | ||
| 40.21 | 26.51 | |||||||||
Excludes high-inflation years of 1974 and 1975.
This finding should not be surprising. After all, a necessary (but not sufficient) condition for the use of Fund resources is that the country display a “balance of payments need.” As such the countries that are implementing Fund programs at any given time are likely to have had worse external balance performance prior to the program period than non-program nations.11 In any case, so long as program and non-program countries differ before the program period in ways that can matter for subsequent performance,12 the behavior of non-program countries will not be a good guide to what would have happened in program countries in the absence of a program.13
The second reason that comparisons between program and non-program countries may not yield good estimates of program effects is that there may be interdependence between program and non-program countries. To the extent that non-program countries are themselves affected by Fund programs, they will not serve as a satisfactory control group. This would be so even if program and non-program countries were identical in all relevant non-program characteristics both before and during the program period.
This interdependence could exist for two reasons. One is that policy decisions among any group of countries competing for market shares are bound to be somewhat interdependent. Thus, for example, suppose country A undertakes a Fund-supported stabilization program and in so doing chooses to devalue its exchange rate by 10 percent to improve its trade account. But now consider country B, which has an export structure similar to that of A and which competes with A in third country markets. Although country B does not have a Fund program, it may also decide to devalue so as not to lose competitive advantage to A. The Fund program has, therefore, affected both a program and a non-program country. Note that if the devaluation had the same impact in the two countries, and if B were used as the control group for A, then a comparison of, say, the trade balance between the two would suggest Fund programs had no effect. In fact, of course, the program might have had quite a sizable total effect since it influenced not only program but also non-program countries, and since both groups may have gained at the expense of a third group of countries not included in the control group.14
The second reason that interdependence might exist between program and non-program countries is that direct interaction could occur through trade. Clearly, measures that affect domestic expenditure, the real exchange rate, or trade restrictions can affect non-program trading partners directly, with the magnitude of these spillovers depending in large part on the weight of program countries in the world economy. If the result of these interactions is to move macroeconomic variables in non-program and program countries in the same direction, a comparison of outcomes in the two groups is likely to understate the true total effect of Fund programs.
The third problem that can plague the comparison of program and non-program countries is that the results of markedly different policy strategies between the two groups may only be fully apparent well after the end of the program period.15 For example, suppose that two countries face identical current account deficits. Country A, with a Fund program, implements a devaluation-cum-expenditure-reducing strategy while country B, without a program, relies on increased trade restrictions and higher international borrowing. Over one year, the change in the overall balance of payments could well be similar for the two countries and it is even possible, again over the short run, that real output and employment may contract less in the non-program country. But when country B is forced to adjust, as it ultimately must, the costs in terms of lower growth, employment, and reallocation of resources may well be larger than the cumulative costs for country A. Yet this would not be reflected in a one-year comparison between program and non-program countries. Indeed, in some analyses of program effects, country A would be regarded as a non-program country after the program year. In short, it is necessary to have a good idea of the time lags associated with the effects of alternative policies if true program effects are to be adequately captured.
Finally, there is no compelling reason why the counterfactual cannot be ascertained from a more subjective, but perhaps still valid judgment about the most likely alternative policy scenario at the time of the program. The fact that it may be difficult to find a good control group for program countries, or that the global economic situation may have changed markedly since the pre program period, does not destroy the usefulness of the actual versus in-the-absence-of method. It just means that more judgment may be needed to apply it.
The “actual versus optimal policy” standard is similar in many ways to the actual versus in-the-absence-of approach. The main difference is that it uses another counterfactual—what could have happened under some hypothetical “optimal” set of policies. Those policies could differ from the policies implemented under a Fund program either in the weight given to various program objectives (less to the balance of payments for instance, and, say, more to income distribution) or in the mix of policies deemed consistent with the same objectives (less reliance might be placed, for example, on expenditure-reducing and more on expenditure-switching policy).
The usefulness of the actual versus optimal policy approach plainly depends on the feasibility both of defining the optimal set of policies relevant to program countries and of inferring their effects on macroeconomic outcomes during the program period. Four points are relevant here. First, the optimal set of policies should be defined within the international constraints and generally adverse initial conditions faced by program countries. This means, for example, that if the optimal policy calls for slower adjustment and more financing than the existing program, the optimal policy scenario will need to identify the sources of that additional financing (be they private or official) within the overall prospective climate for foreign investment or lending offered by the program country. Second, if the optimal set of policies is to be supported by Fund lending, the optimal policies will have to be framed within the constraints faced by the Fund itself, including its obligation to protect the revolving nature of its resources and its inability to dictate social or political objectives to sovereign governments; the former constraint places lower limits on the speed of adjustment that can be supported with Fund resources while the latter circumscribes the Fund’s role in initiating or appraising measures aimed specifically at, say, more equitable distribution of income. Third, the optimal set of policies needs to be defined as specifically as the actual program. For example, if the optimal policy suggests that more emphasis in external adjustment be placed on increasing exports, the price incentives or other measures to achieve the new emphasis have to be specified. Fourth and finally, as with the actual versus in-the-absence-of approach, the counterfactual has to be estimated, because what would happen under the optimal set of policies is unobservable. Indeed, this is likely to be harder to estimate than what would have happened in the absence of a program, because past observations on optimal policy configurations are apt to be harder to find (either in the program country or in other countries) than are the policy configurations of situations without programs.
None of this should mean that calculating the effects of alternative policy scenarios is not useful. The point is simply that if optimal policies are to be used as standards of comparison for the effects of a Fund-supported program, they should be subject to similar requirements. Oranges can only be compared with oranges.
The “comparison of policies” approach, the fifth estimator, is different from the other four in one important respect: it does not infer program effects by comparing actual outcomes in program countries with some past outcome or some estimated counterfactual. Instead, it infers the effects of programs from analyzing the results of policies that typically make up a Fund program. For example, if a representative Fund program calls for, inter alia, a lower fiscal deficit, a lower rate of domestic credit expansion, a depreciation of the real exchange rate, and an increase in real interest rates, and if the relationship is known between these policy instruments and the relevant macroeconomic outcome variables (such as the current account, inflation, or the real growth rate), the effects of programs can be simulated by comparing a Fund-program-type policy scenario with an alternative.16 The parameters used for that exercise often come from small macro-economic models estimated on a pooled cross section time series sample that includes both program and non-program countries.17
Table 5, adapted from the staff’s review of 1980 stand by arrangements and 1978–80 extended arrangements, shows that a typical Fund-supported program encompasses a comprehensive set of measures, with emphasis usually laid on credit ceilings, restraint of public expenditure (especially of public wages and salaries), increases in tax rates and improvements in tax administration, adjustment of tariffs and administered prices, reduction in the ratio of the public sector deficit to GDP, formulation of an investment plan, control of public or publicly guaranteed debt commitments and disbursements, exchange rate reform, export promotion, and overall wage and price policies.
Policy Content of 1980 Fund-Supported Programs
(In numbers of programs)
Policy Content of 1980 Fund-Supported Programs
(In numbers of programs)
| Policy | Stand-By Arrangements | Extended Fund Facility Arrangements | Total | |||
|---|---|---|---|---|---|---|
| Monetary policies | 17 | 13 | 30 | |||
| Credit ceilings | 17 | 12 | 29 | |||
| Reserve requirements | 3 | 2 | 5 | |||
| Interest rate policies | 11 | 7 | 18 | |||
| Other | 2 | — | 2 | |||
| Public sector policies | 17 | 13 | 30 | |||
| Restraint of: Expenditure | 14 | 12 | 26 | |||
| Investment | 8 | 4 | 12 | |||
| Subsidies | 5 | 8 | 13 | |||
| Transfers | 5 | 7 | 12 | |||
| Wages and salaries | 10 | 6 | 16 | |||
| Other current expenditure | 5 | 3 | 8 | |||
| Other | 3 | 4 | 9 | |||
| Reform/improvements | 14 | 13 | 27 | |||
| Reform of tax structure | 5 | 6 | 11 | |||
| Increase in tax rates | 9 | 8 | 17 | |||
| Improvement in tax administration | 10 | 9 | 19 | |||
| Other | 3 | 1 | 4 | |||
| Nonfinancial public enterprises (NPE) | 12 | 11 | 23 | |||
| Curtailment/rationalization of expenditure | 6 | 2 | 8 | |||
| Adjustment of tariffs and administration prices | 8 | 10 | 18 | |||
| Employment | 1 | 1 | 2 | |||
| Wages | 2 | — | 2 | |||
| Other | 1 | 3 | 4 | |||
| Overall public sector | ||||||
| Reduction in deficit-GDP | 12 | 10 | 22 | |||
| Improvement in NPEs | ||||||
| Reduced bank borrowing (real) | 3 | 3 | 6 | |||
| Reduced transfer from government (real) | 6 | 6 | 12 | |||
| Formulation of investment plan | 6 | 9 | 15 | |||
| External debt policies | 15 | 12 | 27 | |||
| Control of commitments/disbursements | ||||||
| Public/publicly guaranteed | 14 | 11 | 25 | |||
| Private sector | 3 | 2 | 5 | |||
| Improvement of maturities | ||||||
| Public/publicly guaranteed | 6 | 4 | 10 | |||
| Private sector | 2 | — | 2 | |||
| Other | 2 | 1 | 3 | |||
| Exchange and trade policies | 12 | 12 | 24 | |||
| Exchange rate reform | 6 | 6 | 12 | |||
| Fixed rate | 2 | 2 | 4 | |||
| Frequently adjusted rate | 3 | 4 | 7 | |||
| Floating rate | 1 | — | 1 | |||
| Liberalization/reform of exchange system | 3 | 6 | 9 | |||
| Liberalization/reform of trade system | 4 | 5 | 9 | |||
| Import substitution measures | — | 2 | 2 | |||
| Rationalization of import protection | 2 | 1 | 3 | |||
| Export promotion or liberalization | 5 | 6 | 11 | |||
| Reduction of arrears | ||||||
| Other | 4 | 3 | 7 | |||
| Wage and price policies | 15 | 9 | 24 | |||
| General wage restraint policies | 10 | 6 | 16 | |||
| Wage guidelines in public sector | 9 | 5 | 14 | |||
| Producer price adjustments | 7 | 3 | 10 | |||
| Retail price adjustments | 7 | 5 | 12 | |||
Policy Content of 1980 Fund-Supported Programs
(In numbers of programs)
| Policy | Stand-By Arrangements | Extended Fund Facility Arrangements | Total | |||
|---|---|---|---|---|---|---|
| Monetary policies | 17 | 13 | 30 | |||
| Credit ceilings | 17 | 12 | 29 | |||
| Reserve requirements | 3 | 2 | 5 | |||
| Interest rate policies | 11 | 7 | 18 | |||
| Other | 2 | — | 2 | |||
| Public sector policies | 17 | 13 | 30 | |||
| Restraint of: Expenditure | 14 | 12 | 26 | |||
| Investment | 8 | 4 | 12 | |||
| Subsidies | 5 | 8 | 13 | |||
| Transfers | 5 | 7 | 12 | |||
| Wages and salaries | 10 | 6 | 16 | |||
| Other current expenditure | 5 | 3 | 8 | |||
| Other | 3 | 4 | 9 | |||
| Reform/improvements | 14 | 13 | 27 | |||
| Reform of tax structure | 5 | 6 | 11 | |||
| Increase in tax rates | 9 | 8 | 17 | |||
| Improvement in tax administration | 10 | 9 | 19 | |||
| Other | 3 | 1 | 4 | |||
| Nonfinancial public enterprises (NPE) | 12 | 11 | 23 | |||
| Curtailment/rationalization of expenditure | 6 | 2 | 8 | |||
| Adjustment of tariffs and administration prices | 8 | 10 | 18 | |||
| Employment | 1 | 1 | 2 | |||
| Wages | 2 | — | 2 | |||
| Other | 1 | 3 | 4 | |||
| Overall public sector | ||||||
| Reduction in deficit-GDP | 12 | 10 | 22 | |||
| Improvement in NPEs | ||||||
| Reduced bank borrowing (real) | 3 | 3 | 6 | |||
| Reduced transfer from government (real) | 6 | 6 | 12 | |||
| Formulation of investment plan | 6 | 9 | 15 | |||
| External debt policies | 15 | 12 | 27 | |||
| Control of commitments/disbursements | ||||||
| Public/publicly guaranteed | 14 | 11 | 25 | |||
| Private sector | 3 | 2 | 5 | |||
| Improvement of maturities | ||||||
| Public/publicly guaranteed | 6 | 4 | 10 | |||
| Private sector | 2 | — | 2 | |||
| Other | 2 | 1 | 3 | |||
| Exchange and trade policies | 12 | 12 | 24 | |||
| Exchange rate reform | 6 | 6 | 12 | |||
| Fixed rate | 2 | 2 | 4 | |||
| Frequently adjusted rate | 3 | 4 | 7 | |||
| Floating rate | 1 | — | 1 | |||
| Liberalization/reform of exchange system | 3 | 6 | 9 | |||
| Liberalization/reform of trade system | 4 | 5 | 9 | |||
| Import substitution measures | — | 2 | 2 | |||
| Rationalization of import protection | 2 | 1 | 3 | |||
| Export promotion or liberalization | 5 | 6 | 11 | |||
| Reduction of arrears | ||||||
| Other | 4 | 3 | 7 | |||
| Wage and price policies | 15 | 9 | 24 | |||
| General wage restraint policies | 10 | 6 | 16 | |||
| Wage guidelines in public sector | 9 | 5 | 14 | |||
| Producer price adjustments | 7 | 3 | 10 | |||
| Retail price adjustments | 7 | 5 | 12 | |||
This comparison of policies approach has three principal advantages. First, observations are not restricted to the experience with Fund-supported programs; the approach can draw on much wider evidence on how various policies might affect the objectives of programs. By inferring what programs do from what programs are rather than from what happened during program periods, one can make use of the considerable existing literature on the effects say, of exchange rate changes, or more restrictive monetary policy. Second, by its very nature, this approach focuses on the relationship between policy instruments and targets. This provides useful information on how programs work—a feature that is not shared by those approaches (such as the before-after method) that dwell only on the “bottom line” of programs. Third, because the comparison of policies approach contrasts hypothetical policy packages, its results are not blurred by incomplete implementation of policies. In the other four approaches, program effects reflect both the degree of implementation of policies and the effects of those policies that are implemented. This is not a trivial concern. Previous staff reviews of stand-by and ex-tended arrangements suggest that most policy measures are implemented as planned in only one third to two thirds of the programs.
On the negative side, the comparison of policies approach, while it may be able to contrast the effects of “good” and “bad” policies, may give incomplete or even misleading information on Fund-supported programs, for at least three reasons. One is that the theoretical models underlying such exercises are seldom capable of simulating the range of measures that make up a Fund-supported program. As Table 5 suggests, the characterization of a Fund program by just, say, credit and fiscal deficit measures would be a poor approximation. At the very least, one should add the supply incentives included in programs to generate more domestic savings, more investment, and more exports. Failure to do so could impart a deflationary bias to the simulation exercise itself. Second, even for a given policy measure, the effects may differ depending on whether that measure is introduced within the context of a Fund-supported program. For example, a new target announced for the real exchange rate may be viewed as more likely to be adhered to if it is part of a Fund-supported program than otherwise. More generally, to the extent that participation in a Fund-supported program alters the credibility of announced policies, one cannot assume that the effects of a program depend only on the magnitude of the change in policy instruments. Third, unless model simulations take due account of the adverse (preprogram) initial position facing the hypothetical program country, they may not fully reflect the effect of policy instruments on their target variables. For ex-ample, if the profitability of producing exportables is very low relative to other activities because an increasingly overvalued exchange rate has been maintained for some time, a small or even moderate exchange rate depreciation may have little effect on the production of exportables. In such a case, a simulation of the effects of an exchange rate change based on an initial equilibrium will not produce a good estimate of the effects of such an action in the more realistic situation of disequilibrium.
Before the global effects of Fund-supported programs can be assessed, therefore, it is necessary to have a clear idea of how “effects” of programs should be defined and measured. The main message of this section, simply put, is that not only the size but even the direction of program effects are likely to be quite sensitive to alternative definitions and estimating methodologies. The review of five possible interpretations of program effects shows that the measured effects of the same programs can vary substantially, depending on, inter alia: (1) whether changes in non-program factors between the pre-program and program period are accounted for; (2) whether program targets incorporate unexpected developments in the global environment; (3) whether program countries are systematically different from non-program countries prior to the program period in ways that matter for subsequent performance; (4) whether non-program countries are themselves affected by Fund-supported programs; (5) whether the medium-term and long-run as well as the initial effects of programs are considered; (6) whether, because of confidence and credibility factors, the implementation of a given policy within the context of Fund-supported programs has different effects than without it; and, perhaps most important, (7) whether the most relevant comparison for the actual effect of a Fund-supported program is what would have happened without it, or what could have happened under some hypothetical and optimal set of policies.
A good way of illustrating how the alternative definitions of program effects can color the evaluation of programs is to use each method to assess the much discussed recent import compression experienced by countries that had Fund-supported programs in 1983. On a weighted average basis, the volume of imports by program countries fell by almost 8 percent in 1983. What role should be assigned to Fund programs in this decline?
According to the before-after approach to program effects, all changes are attributed to the program, and the interpretation would therefore be that Fund programs “caused” the fall in import volumes. Since a lower demand for imports by program countries implies, ceteris paribus, lower exports for the rest of the world, this would imply, in turn, that Fund programs had a deflationary effect on global economic activity.
The actual versus in-the-absence-of approach means comparing the actual decline in import volumes with the change that would have occurred without Fund-supported programs. In this connection, a sequence of three points is relevant. First, the Fund’s lending in 1983 exceeded SDR 12 billion and helped to secure over SDR 20 billion in new bank lending to non-oil developing countries.18 Second, without the Fund’s direct and “catalytic” lending effects, the flow of financing to 1983 program countries would have been much smaller. Finally, based on past empirical work, foreign exchange receipts are the main determinant of the demand for imports in developing countries.19 All of this would point strongly to the conclusion that Fund-supported programs meant the decline in import volume was less than it would otherwise have been.20 Hence, on the basis of the same argument, the implication would be that Fund-supported programs had an expansionary effect on global economic activity. In addition, one would also want to note that, based on preliminary trade data for 1984, the same group of 1983 program countries exhibited an average increase of 10 percent in their import volumes in 1984—which is consistent with the view that the medium-run effects of programs are probably quite different from their initial impact.
Yet a third, more mixed, verdict might well emerge from the actual versus target or versus optimal policy approaches. If, for example, import volumes fell more than targeted, the verdict might be that the external adjustment achieved under 1983 programs was both unavoidable and better managed than it would have been without programs, but still that the compression of imports went further than would be optimal or desirable from the perspective of longer-term growth. Under these methods, the decline of imports could be attributed to overachievement of fiscal targets, or to greater-than-anticipated adjustment pressures linked to higher-than-expected world real interest rates, or even to the application of restrictive trade controls by program countries that ran counter to program intentions. In any case, the conclusion from this perspective could be that the effect of programs on imports was more expansionary than in their absence but not as expansionary as would be desirable or optimal given the operating environment.
The effects of programs can, therefore, mean different things to different people. This is not all bad because, as shown earlier, none of the separate definitions of program effects is free of shortcomings. Still, unless these different definitions or interpretations of program effects are explicitly recognized, the danger exists that different views on the global effects of Fund-supported programs will be due in large part to the application of different yardsticks to the same evidence.