Rodney Ramcharan

Economic adjustment programs supported by the IMF provide member countries with financing and policy advice to help them resolve both short-term balance of payments difficulties and longer term structural impediments to growth. Given the importance of these adjustment programs, a voluminous literature has evolved to assess their effectiveness. This research summary selectively surveys some of the empirical approaches used to evaluate IMF program effectiveness.

IMF-supported programs are probably the institution’s most visible activity and affect a wide range of countries. Over the last decade, with the decline in communism and the deepening of economic crises in many emerging market economies, some 90 countries have become involved in various IMF-supported economic adjustment programs.

However, despite the prevalence of these programs, their effectiveness is still being debated in a number of respects. Why does the performance of many countries that have participated in a series of IMF-supported programs remain relatively weak?¹ How do we know the extent to which these programs work? Can patterns that emerge from past experience help improve program effectiveness in the future?

A large body of empirical research conducted within and outside the IMF has sought to assess IMF program effectiveness (Haque and Khan, 1998), but this research agenda faces three major methodological hurdles. First, IMF programs tend to be remarkably varied across countries, making meaningful cross-country comparisons difficult (Mussa and Savastano, 2000). Second, IMF programs are inherently complex constructs with multiple economic objectives spread over varying time horizons. As a result, the appropriate definition of IMF program effectiveness is not entirely clear. That said, much of the empirical literature has focused on the impact of IMF programs on macro-economic aggregates such as growth and inflation. Third, neither a country’s decision to seek an IMF program nor the IMF’s decision to agree to a program is random; thus, selection bias is a challenge to accurate estimation. Hence, because of these methodological difficulties, most of the results are tentative. On the whole, the evidence suggests that IMF programs lower inflation and improve the balance of payments; their impact on growth is less clear.

Issues of selection bias arise because countries “demand” IMF programs in times of economic difficulties. Bird (1996) notes that adopting an IMF-supported program is generally linked to a low level of reserves, increased debt, and an overvalued exchange rate. Hence, comparisons between countries that do not enter into IMF-supported programs and those that do are difficult (Krueger, 1998). An appropriate measure of program effectiveness would compare outcomes under an IMF program with those that would have occurred in the absence of the program—the counterfactual. However, by definition, the counterfactual policy stance, and the resulting macroeconomic outcome, are never observed.

Consequently, researchers have focused on the creation of artificial policy counterfactuals. The generalized evaluation estimator (GEE), pioneered by Goldstein and Montiel (1986) and Khan (1990), is a popular method of constructing these counterfactuals. This approach constructs counterfactual policy stances by using nonprogram data to link annual changes in macroeconomic policies to deviations in lagged macroeconomic outcomes from their target values. The counterfactual policy stance is included among the set of regressors in the main regression. In this way, the impact of an IMF-supported arrangement can be measured, after controlling for the set of policies that would have been observed in the absence of such an arrangement.

With differing degrees of significance, the point estimates from the GEE and the other approaches that attempt to control for the counterfactual suggest that IMF programs improve the current account balance and lower inflation (Khan, 1990; Conway, 1994; Killick, Malik, and Manuel, 1995). Some of these studies also suggest a positive impact of IMF programs on economic growth (Bagci and Perraudin, 1997; Knight and Santaella, 1997), but there is no consensus on the precise timing of this growth effect. Bulir and Moon (2003) use a similar approach to study the impact of IMF programs on fiscal adjustment, finding little significant impact on the fiscal balance. Mody and Saravia (2003) examine whether IMF programs help to catalyze capital flows. The authors report evidence of a catalytic effect when a country is in an intermediate, vulnerable zone before plunging irretrievably into a crisis; at the other extreme, programs in “tranquil” periods can also send unfavorable signals to markets, hurting rather than helping countries. In addition to differing sample periods, a key source of the disagreement across these studies is their specification of the policy counterfactual as there is little theoretical basis to guide the selection of the variables used to determine policymakers’ reaction functions.

Instead of relying on counterfactuals, another strand of empirical research uses instrumental variables methods or Heckman two-step estimation procedures to address sample selection issues. Barro and Lee (2002) and Easterly (2002) find mixed results on economic growth and poverty reduction while making the heroic assumption that political or institutional variables, such as a country’s alignment with major shareholders at the IMF, are only related to economic growth through their influence on the IMF’s lending decision. Przeworski and Vreeland (2000) find that program participation lowers growth while countries remain under a program, Chen and Thomas (2003) explore differences between programs that were successfully completed and those that ended prematurely: they find that completed programs are marginally associated with a higher subsequent growth rate, whereas programs that end prematurely are associated with a lower growth rate.

The implications of the current empirical approaches for IMF program design remain limited. For instance, the finding that IMF programs improve the balance of payments says little about whether this impact is optimal; given the trade-offs between aggregate demand and the external position, could the average improvement in the external position have been achieved with a smaller reduction in aggregate demand? Such questions are at the center of program design and are a vital area of future research. Recent work by Baqir, Ramcharan, and Sahay (2003) begins to analyze the nature of trade-offs among the program objectives. Preliminary results suggest that narrowing the gap between the current account target and its actual value reduces economic growth relative to its projected value.