Evaluating the Impact of Non-Financial IMF Programs Using the Synthetic Control Method1

Contributor Notes

Author’s E-Mail Address: mnewiak@imf.org; twillems@imf.org

We use the Synthetic Control Method to study the effect of IMF advice on economic growth, inflation, and investment. The analysis exploits the existence of IMF programs that do not involve any financing (Policy Support Instruments, “PSIs”). This enables us to focus on the effects of IMF monitoring, advice, and approval (as opposed to direct financial assistance). In addition, countries with non-financial programs are typically not crisis-struck – thereby mitigating the reverse causality problem and facilitating the construction of counterfactuals. Results suggest that treated countries add about 1 percentage point in annual real GDP per capita growth, with inflation being lower by some 3 percentage points per year. While we do not find evidence for an impact on total investment and the resulting capital stock, PSI-treatment does seem to stimulate foreign direct investment.

Abstract

We use the Synthetic Control Method to study the effect of IMF advice on economic growth, inflation, and investment. The analysis exploits the existence of IMF programs that do not involve any financing (Policy Support Instruments, “PSIs”). This enables us to focus on the effects of IMF monitoring, advice, and approval (as opposed to direct financial assistance). In addition, countries with non-financial programs are typically not crisis-struck – thereby mitigating the reverse causality problem and facilitating the construction of counterfactuals. Results suggest that treated countries add about 1 percentage point in annual real GDP per capita growth, with inflation being lower by some 3 percentage points per year. While we do not find evidence for an impact on total investment and the resulting capital stock, PSI-treatment does seem to stimulate foreign direct investment.

1 Introduction

A topic that has invoked much discussion and debate, is whether economic programs supported by international financial institutions (IFIs) harm or help the countries involved.

The fact that this debate has proved to be so difficult to settle, lies in two related reasons. First of all, IFI-supported programs are normally surrounded by adverse selection and reverse causality problems: countries tend to apply for IFI-support because they are facing persistent economic problems. This then makes it difficult to establish whether any ongoing difficulties are the cause or the effect of IFI-involvement.

Secondly, constructing a realistic counterfactual is challenging since every economic crisis has its own unique elements, while it is hard to find data from countries that live through a similarly severe economic crisis without IFI-support (it is for example not easy to construct a counterfactual for Greece during its debt-crisis). In addition, the fact that IFI-programs often involve direct financial transfers, complicates the process of disentangling the impact of IFI-involvement (advice, monitoring, and approval) on the one hand, and direct financial assistance on the other.

In this paper, we address both the “reverse causality” and “counterfactual” problem by applying the Synthetic Control Method to Policy Support Instruments set up by the International Monetary Fund (IMF).

The Policy Support Instrument (PSI)1 is a relatively new non-financial IMF-instrument, established in October 2005. It is available to countries that have no current or prospective Balance of Payments (BoP) needs (the presence of which is a necessary condition for all other IMF programs). PSIs can only be granted to countries that are “PRGT-eligible”, which implies that countries must have low per capita income levels and lack durable and substantial access to international financial markets (IMF, 2015).2 So far, PSIs have been adopted by seven countries (all in sub-Saharan Africa: Cabo Verde, Mozambique, Nigeria, Rwanda, Senegal, Tanzania, and Uganda) – and these countries are the subject of this study.

The purpose of the PSI is to offer countries the possibility to obtain the Fund’s approval for their economic policies without entering a borrowing arrangement. It intends to help countries designing effective economic programs, aimed at ensuring macroeconomic stability and debt sustainability, while encouraging structural reforms that remove constraints on growth and poverty reduction. It simultaneously aims to signal to donors, multilateral development banks, and market participants that the IMF endorses the member’s policies. For these reasons, one can think of a PSI as replicating a traditional IMF-program, without financing (Taylor, 2006).

Given that the PSI is a non-financial instrument, it tends to be used by countries that do not face a severe and urgent BoP-need. This particular feature brings several advantages in assessing its effectiveness. Firstly, in cases where the PSI is not blended with traditional (funded) IMF programs, we are able to analyze whether Fund advice, monitoring, and approval have any effects by themselves (as opposed to the direct effect of providing financial assistance). Secondly, the fact that PSI-countries are not crisis-struck, mitigates the adverse selection problem that normally complicates the process of identifying the causal effect of IMF-programs – facilitating the construction of a counterfactual.

To construct counterfactuals, we use the Synthetic Control Method (SCM). This method was first employed in Abadie and Gardeazabal (2003) and extended by Abadie, Diamond, and Hainmueller (2010). Recent macroeconomic applications include Billmeier and Nannicini (2013) and Adhikari, Duval, Hu, and Loungani (2016). It involves constructing a “synthetic control” for the treated unit. The synthetic control is constructed as a weighted-average of units that do not undergo treatment over the sample period (the underlying idea being that a combination of non-treated units might make a better counterfactual than any individual one). The weights are chosen such that relevant economic characteristics in the synthetic control unit, match the treated unit as closely as possible in the pre-intervention period. By comparing the subsequent evolution of an outcome variable in the synthetic control unit with that of the treated unit, one can obtain an estimate of the treatment effect.3

In the context of this particular paper, “treatment” for a country is the adoption of a PSI. The synthetic control is subsequently constructed from a group of “untreated” developing countries that did not have any kind of IMF-program in place over the sample period.

Results obtained in this way suggest that IMF advice, monitoring, and approval have had a positive effect on economic development in the medium run: following treatment (the adoption of a PSI), countries tend to grow faster than their synthetic controls (by about 1 percentage point per year), while they are characterized by lower rates of inflation (by about 3 percentage points per year). Even though capital stocks have increased significantly in most countries during the period of the PSI, our results suggest that this development is not caused by PSI-treatment. In contrast, we do find evidence that PSIs have stimulated foreign investment. This in itself may bring important economic benefits to a country, such as knowledge transfers and better management practices, and could explain the aforementioned positive effects – particularly on output.

The itinerary of this paper is as follows. After discussing the related literature in Section 2, we document the types of IMF-engagement for the PSI-countries in Section 3. Section 4 then details the core contents of the PSIs that the IMF has launched to date. Section 5 goes on to describe the Synthetic Control Method, which we use to construct counterfactuals. Subsequently, Section 6 discusses our data and regression specifications, after which Section 7 presents our empirical findings. Section 8 discusses this paper’s results, after which Section 9 concludes.

2 Related literature

The literature on the effects of IFI-programs (and IMF-programs in particular) is vast. Comprehensive overviews are offered in Dreher (2006, 2009) and Steinwand and Stone (2008).

Early studies include Reichmann and Stillson (1978), who compare economic performance in program countries before and after intervention, and Donovan (1981, 1982) who compares developments in target variables in program countries with those in a control-group of non-program countries. As surveyed by Haque and Khan (1998), these early studies typically conclude that IMF-programs have been successful in stabilizing the economy.

Many of these early studies can however be criticized on econometric grounds as they do not control for reverse causality or take a rather superficial approach when it comes to constructing the counterfactual.

More recent studies have therefore applied more advanced regression-based techniques to the problem at hand. Dicks-Mireaux, Mecagni, and Schadler (2000) use the General Evaluation Estimator (GEE) to construct counterfactuals. In particular, they use policy reaction functions from countries without an IMF-program to approximate the counterfactual for countries that did have one. Using this method, the authors find that IMF-supported programs have had positive effects on growth and the debt-service ratio, while no significant effect is found on inflation. The authors however note that diagnostic tests question the reliability of their results.

Przeworski and Vreeland (2000) and Atoyan and Conway (2006) use a dynamic version of the Heckman selection model and find that IMF-program participation lowers growth rates while the program is in place, with growth picking up once the program is completed. Atoyan and Conway (2006) see these results confirmed by taking a “matching” approach, where they compare countries with similar propensity scores (when it comes to asking for IMF assistance) but different loan-participation decisions.

Barro and Lee (2005) state that IMF-loans are sensitive to political-economic variables. In particular, they argue that loans tend to be larger and more frequent when a country has a bigger quota, more professional IMF-staff, and when it is more connected to the United States and major European countries. They then exploit this variation to take an instrumental variables approach, the results of which suggest that a higher IMF loan-participation rate reduces economic growth, while having no significant effects on investment, inflation, government consumption, and openness. Dreher (2006) finds similar results using an alternative instrumental variables-approach, but his findings suggest that the negative effect on growth can be mitigated by compliance with conditionality.

More recently, Bas and Stone (2014) have carefully documented that IMF programs indeed suffer from adverse selection and show that taking this into account leads to a more favorable impact evaluation in terms of higher growth rates. At the same time, Bas and Stone (2014) find large heterogeneity – with governments that are most eager to participate in IMF programs typically experiencing the least beneficial effects. Taking adverse selection into account, Binder and Bluhm (2017) furthermore show that IMF programs only boost output if they are accompanied by institutional improvements.

Further studies have focused on analyzing the effect of IMF programs on inequality, poverty, and development (Garuda, 2000; Easterly, 2003; Hajro and Joyce, 2009) but without final consensus on whether IMF programs have a beneficial or adverse effect on these variables. Most of these analyses have however focused on programs launched before the 1999-introduction of the PRGT for low-income countries (which brought an explicit focus on growth and poverty reduction). For PRGT-funded programs, IMF (2012) finds that program countries performed equally well as countries that did not seek IMF-support, while Oberdabernig (2013) and Lang (2016) tend to find that such programs did not increase (or even decreased) inequality. CGD (2007) and Clements, Gupta, and Nozaki (2013) furthermore report that social spending tends to increase under PRGT-funded programs.

The plurality of reported results illustrates that the debate on the impact of IMF-programs is ongoing. With this paper, we bring a new technique (the SCM) to this question, while limiting our focus to non-disbursing PSIs (provided to countries without current or prospective BoP-needs). This helps to mitigate the reverse causality problem that has complicated earlier analyses, while it also enables us to see whether there is any effect from pure Fund advice, monitoring, and approval (without financial disbursements).

3 PSI-countries and the IMF

To date, seven countries (all located in sub-Saharan Africa) have adopted PSIs: Cabo Verde, Mozambique, Nigeria, Rwanda, Senegal, Tanzania, and Uganda. As Figure 1 shows, all countries (apart from Nigeria4) experienced a BoP-need at some point over our full sample period (1992-2015) and entered funded IMF programs in response. In none of these cases was the BoP-need accompanied by an acute economic crisis.

Figure 1 also illustrates that PSIs typically consist of multiple multi-year programs that are launched in succession. For three countries (Mozambique, Senegal, and Tanzania), the PSI co-existed, for part of the period, with a disbursing IMF program - indicating a temporary BoP-need for those countries. In all of these cases, the programs were relatively short-lived, with the BoP-need being caused by factors beyond the countries’ direct control.5

At this stage, it is important to note that the presence of earlier programs may bias our results. However, as we argue in Section 8, this bias is likely to run in the opposite direction of the results we find. Moreover, it should be kept in mind that disbursing programs differ in focus from PSIs: while traditional IMF programs primarily aim to address BoP-problems, the main objective of the PSI is to enhance growth and reduce poverty. For this reason, the focus of this paper lies with analyzing the impact of PSIs on growth and inflation outcomes (given that the inflation tax is mainly carried by the poor; Easterly and Fischer (2001)).

4 Core contents of PSI-programs

As detailed in IMF (2016: 113), “all PSI-supported programs are aimed at maintaining or consolidating a stable and sustainable macroeconomic position consistent with strong and durable poverty reduction and growth […]. Specifically, PSI-supported programs should aim to maintain or consolidate (i) strong and durable poverty reduction and growth, (ii) low or moderate inflation, (iii) sustainable fiscal and current account balances, (iv) limited debt vulnerabilities, (v) adequate international reserves, and (vi) sufficient policy and institutional capacity to implement appropriate macroeconomic policies.”

To achieve these goals, country authorities and the IMF agree on a set of targets that forms the basis for the assessment of the progress towards the program goals. Such “conditionality” will typically consist of quantitative targets for macroeconomic variables and structural measures that are within the member country’s control.

On the quantitative side, all PSIs have included a combination of quantitative assessment criteria, such as floors on international reserves of the central bank, ceilings on monetary targets, limits on domestic financing of the government, and constraints on external debt accumulation. Other targets have related to floors on social spending, ceilings on the accumulation of new domestic arrears, and floors on government revenue.

On the structural side, conditionality within PSIs has focused on fiscal policies and actions to strengthen fiscal transparency. In particular:

  • With respect to revenues, measures included steps towards the streamlining of tax exemptions and incentives (Cabo Verde), the implementation of customs tariff regimes (Nigeria), and the limitation of tax cash payments (Senegal). In Rwanda, a package of policies to strengthen revenue collection was part of both programs, including improvements to tax form submissions (2010 program) and measures to decrease VAT exemptions, a revision of property taxation, and new tax regimes for agriculture and mining (2013 program).

  • Measures on expenditure management included restrictions on the contracting of new non-concessional financing (Mozambique), the adoption of a medium-term expenditure framework (Cabo Verde), and the development of guidelines to assess the rate-of-return on projects (Senegal). In the 2006 Uganda PSI, the program also put substantial emphasis on the clearance of domestic arrears.

  • More than one fourth of measures across PSIs focused on ensuring transparency of the budget and payments. They included the introduction of specifically designed systems and direct bank transfer to settle salary payments (Mozambique, Uganda), limiting the number of accounts around the treasury single account (Mozambique, Senegal), the introduction of financial management information systems in ministries (Rwanda), measures to improve monitoring of social spending (Tanzania), and conditionality on publishing information, ex-post audits, and cost-benefit evaluations (Senegal).

Within other areas of conditionality, the focus varied depending on country circumstances:

  • In Cabo Verde, which has a relatively developed financial system, about one quarter of structural conditionality centered around financial sector reforms. Examples include taking steps to expand and formalize the task of the Financial Stability Committee and moving forward with a banking law that unifies the regulatory framework.

  • In Mozambique, programs were also aimed at strengthening the legal and regulatory framework in the mining and petroleum sectors through the adoption of new model contracts.

  • In Nigeria, about one quarter of structural measures focused on the reform of public enterprises, such as the unbundling of the national electricity company and the opening of bids on the state telecommunications strategy.

  • In Rwanda, Tanzania, and Uganda (all part of the East African Community) a significant amount of conditionality focused on Central Bank reform. In Rwanda, such measures included making monetary policy more transparent by publishing quarterly inflation reports along with economic assessments underlying Central Bank-decisions, and actions to strengthen liquidity management. In Tanzania, they encompassed steps to move to consolidated supervision of commercial banks and strengthening the interbank market. In Uganda, the conditionality focused on actions to ensure capital adequacy.

  • In Senegal, conditionality focused on strengthening transparency, such as through the adoption of an institutional framework for implementing and monitoring their accelerated growth strategy.

In the remainder of this paper, we will deploy the Synthetic Control Method to analyze the impact of all PSI-programs on certain key macroeconomic variables.

5 The Synthetic Control Method

The SCM was developed by Abadie and Gardeazabal (2003) and extended in Abadie, Diamond, and Hainmueller (2010). Here we describe the main principles underlying the method, referring to the aforementioned sources for all computational details.

Assume that we have time-series data about J +1 units (in our context: countries), indexed by i = 1, 2,…, J + 1. Without loss of generality, say that the first unit (i = 1) undergoes treatment at time T0. All J other units remain untreated over the sample period and constitute the “donor pool” which implies that they can serve as potential controls.

Let Y be the outcome variable of interest. If we define YitN as the value of Y in unit i at time t in the absence of intervention, and YitI as its equivalent when intervention does take place, our aim is to obtain an estimate of the effect of the policy in the treated unit (i = 1), i.e. to obtain an estimate of:

α1t=Y1tIY1tN,tT0.(1)

The problem however lies in the fact that the unit of interest is actually treated, as a result of which Y1tN is counterfactual and cannot be observed. This is where the SCM comes in. Its underlying idea is that a combination of units might be able to produce a better counterfactual than any individual one.

The SCM starts by specifying a factor model for the unobserved YitN:

YitN=δt+Ziθt+λtμi+εit.(2)

Here, δt is a common factor with identical impact on all units, Zi is a vector containing observed covariates (which should not be affected by the policy intervention; Abadie, Diamond, and Hainmueller (2010: 495)), θt includes the associated unknown parameters, the vector λt contains the unobserved factors, and μi are the factor loadings. Finally, εit is the error term, which is assumed to have zero-mean for all units.

The idea is to use the above structure to estimate the counterfactual for the treated unit as a linear combination of realized outcomes in the potential controls, i.e. construct the estimate for the counterfactual as:

Y^1tN=Σi=2J+1wiYit,tT0.(3)

The unit weights wi are selected such that the synthetic control unit matches certain characteristics of the treated unit as closely as possible. In particular, let us define X1 as a vector containing the average values of pre-intervention variables of the treated unit that we wish match with our synthetic control. Typically, X1 at least includes the covariates in Z, but it may also include pre-intervention values of the outcome variable Y. The vector X0 collects the same variables for units in the donor pool. The goal now becomes to pick the weights wi such that the resulting synthetic control unit matches the pre-treatment characteristics of the treated unit (X1) as closely as possible. This will be achieved if the vector of weights W* follows from solving

minWX1X0WV=(X1X0W)V(X1X0W)(4)s.t.wi*0fori=2,,J+1Σi=2J+1wi*=1,

where the constraints make sure that the method does not rely on extrapolation of unit-characteristics (units cannot get negative weights or weights larger than one). V is a symmetric and positive semi-definite matrix that weighs the importance of all explanatory variables. It is selected to minimize the mean-squared prediction error for the outcome variable in the pre-treatment period.

Once the weights have been obtained, the counterfactual can be constructed for any tT0 by using equation (3). Subsequently, one can obtain an estimate of the treatment effect at time tT0:

α^1t=Y1tIY^1tN.(5)

To assess whether the synthetic control makes a good counterfactual, we use the pre-treatment “Fit Index” employed by Adhikari and Alm (2016) and Adhikari, Duval, Hu, and Loungani (2016). This measure starts from the root mean square prediction error (RMSPE) of the outcome variable (RMSPE=1T0Σt=1T0(Y1tΣj=2J+1wj*Yjt)2), but normalizes it - thereby enabling a cross-country comparison. In particular:

FitIndex=RMSPEbenchmarkRMSPE,(6)

where the benchmarkRMSPE=1T0Σt=1T0(Y1t)2 is the RMSPE obtained from the zero-fit model.

A Fit Index of zero implies a perfect fit; a Fit Index greater than one indicates a particularly poor fit (Adhikari, Duval, Hu, and Loungani, 2016), but this never happens in the cases presented below.

6 Data and regression specifications

Motivated by the observation that PSIs aim to promote growth, contain inflation, and reduce poverty, we primarily look at the evolution of real GDP per capita (to assess growth) and the Consumer Price Index (to assess inflation, which tends to aggravate poverty (Easterly and Fischer, 2001)). To gain more insight into the possible channels of operation, we also analyze the evolution of the capital stock (total and foreign-owned). Unfortunately, data limitations prevent us from analyzing fiscal variables or direct measures of inequality (like Gini coefficients).

The SCM requires the input of an outcome variable (Y), as well as that of covariates (Z) which are believed to influence the outcome variable. In all cases, we tie our hands by relying upon earlier studies when determining what covariates to include.

We take real GDP per capita from the IMF’s World Economic Outlook database, expressed in 2010 US dollars. All used data series are at the annual frequency and span the years 1992 (when most of the required data become available) through 2015 – except for Rwanda where we start in 1995, so that we exclude the 1994-genocidal episode. Inspired by Abadie and Gardeazabal (2003) and Barro and Sala-i-Martin (1995), we include the following variables as covariates (see Table 1 for an overview): the investment rate (gross fixed capital formation as a share of GDP), an indicator of economic openness (the sum of exports and imports as a share of GDP), population density, the sectoral share of agriculture, the sectoral share of industry, the secondary school enrolment rate, and the tertiary school enrolment rate. Since institutions are an important determinant of economic performance in our developing country sample, we follow Cavallo et al. (2013) in adding the absolute value of latitude to our specification.6 All covariate-series are taken from the World Bank’s World Development Indicators, except for the investment rate which comes from the World Economic Outlook database (which has broader coverage). Following the recommendation in Kaul et al. (2016), we include only a small number of outcome-variable observations from the pre-treatment period as predictors, namely real GDP per capita in the years 1995, 2000, and 2005.7

Table 1:

Overview of covariates Z for all outcome variables.

article image
Note: the covariate list for real GDP per capita is inspired by Barro and Sala-i-Martin (1995), the list for the CPI is inspired by Ghosh, Ostry, and Tsangarides (2010), while that for the capital stock is inspired by Blonigen et al. (2007).

We study the evolution of Consumer Price Indices (CPIs) over the same 1992-2015 sample period (apart from Rwanda and Nigeria, for which CPI-data only become available in 1995). CPI-data are taken from the IMF’s World Economic Outlook database and are rebased such that all countries have a CPI of 100 in the year 2005. Inspired by Ghosh, Ostry, and Tsangarides (2010), we include a 7-way classification of the de facto exchange rate regime8 and a measure of central bank governor turnover (a proxy for central bank independence) as covariates.9 Again, we include some outcome-variable observations from the pre-treatment period as predictors, namely the CPI in the years 1995 and 2000.10

When analyzing the evolution of the capital stock (as a percentage of GDP), we look at both the total capital stock (taken from the Penn World Tables) and the foreign-owned capital stock (taken from the updated and extended version of the Lane and Milesi-Ferretti (2007) dataset). These series end in 2014, so in this case our sample period spans 19922014. We follow Sanso-Navarro (2011) – who in turn based his choice on Blonigen et al. (2007) – by including the following covariates: population size (to reflect potential market size), openness (the sum of exports and imports as a share of GDP), the secondary school enrolment rate, the tertiary school enrolment rate, an index of capital account openness (see Chinn and Ito (2006)), and the absolute value of latitude. The latter two series are included to proxy for, what Blonigen et al. (2007) call, “host-country investment costs”.11 As with output and prices, we again include outcome-variable observations from the pre-treatment period as predictors, namely the capital stock in 1995, 2000, and 2005.

Since its inception in 2005, seven countries (all located in sub-Saharan Africa) have adopted PSIs: Cabo Verde (August 2006-February 2012), Mozambique (June 2007-June 2016), Nigeria (October 2005-October 2007), Rwanda (June 2010-December 2016), Senegal (November 2007-June 2018), Tanzania (February 2007-July 2017), and Uganda (February 2006-June 2016, now extended to June 2017).12

For each and every country, the synthetic control is constructed from a donor pool that contains all countries classified as “developing” by the World Economic Outlook database that did not have any kind of IMF-program in place over the period 1992-2016. The method has complete freedom in determining how many and which countries to pick from the donor pool (all with the objective of matching the specified characteristics of the treated country as closely as possible; recall Section 5). Given that cross-country spillovers to and from sub-Saharan African countries tend to be minimal (if present at all),13 the possibility that developments in treated countries affect control countries (which would be problematic to the SCM-approach) is not a major worry. This leaves us with 39 countries that form potential controls (see Table A1 in the Appendix for a full list). Not all of these countries would form natural individual control countries, as some of them are too rich, others are too open, and so forth. But as we will show in the next section, a weighted average of a subset of these is able to closely match Table 1’s covariates and the developments in outcome variables over the pre-treatment period.

7 Results

In this section, we report our results. For all treated countries, Tables A2-A5 in the Appendix show which countries make up what proportion of the synthetic control unit, while those tables also report pre-treatment Fit Indices (calculated as in equation (6)). Table A6 shows how well the various synthetic controls match the actual treated countries along the covariates specified. The algorithm minimizes the mean-squared prediction error for the outcome variable in the pre-treatment period by giving priority to matching those covariates that are more important for minimizing the prediction error.

7.1 Output

Figure 2 shows results obtained by applying the SCM to our seven treated countries on the real GDP per capita dimension, while Table 2 calculates average growth rates over the post-treatment period for treated countries and synthetic controls. As the figure and table show, all seven PSI-countries have managed to outperform their SCM-constructed counterfactual in the post-treatment period. Table 2 indicates that treated countries have experienced per capita real GDP growth rates that are about 1 percentage point higher than their synthetic equivalents in the post-treatment period. The fact that outperformance is consistently observed across all treated countries (with results always being based upon the exact same regression specification) gives credence to the robustness of this finding.

Table 2:

Average annual real GDP per capita growth rates post-treatment.

article image
Source: World Economic Outlook database and own calculations.

Following the SCM-literature, one can also analyze robustness along another dimension, namely by conducting placebo exercises: in them, we iteratively swap the treated country with every country in the donor pool and counterfactually pretend that the control country was actually treated. Absence of a strong treatment effect in placebo cases provides evidence in favor of the difference in Figure 2 being due to the actual treatment. In this exercise, it is customary to only include placebo runs that have good pre-treatment fit. In particular, we follow Adhikari and Alm (2016) by requiring the Fit Index for the placebo to be lower than, or equal to, the Fit Index for the treated country. If less than five placebo countries meet this criterion, we select the five placebo countries that have the lowest Fit Index (i.e.: the ones that have the best pre-treatment fit).

As shown in Figure 3, output in some placebo-countries has an ambiguous or negative response – a finding that never shows up for the actual PSI-countries. Moreover, treatment effects in PSI-countries are sizable relative to placebo effects for untreated countries.14 Typically, however, there are not many countries in the control group for which we can construct counterfactuals with a fit that is at least as good as for the treated country, which limits the informativeness of this exercise (and prevents us from calculating meaningful p-values).

Our results continue to hold when we exclude those countries that receive the largest weight in the baseline synthetic control (the only exception being Cabo Verde, where the treatment effect then disappears). Our results are similarly robust to extending our specification with the rule-of-law index as a covariate. In both cases, the size of the donor pool shrinks though - thereby bringing about a substantial worsening in the pre-treatment Fit Index.

7.2 Price level

Given that containing inflation is an explicit goal of PSIs, Figure 4 and Table 3 analyze the impact of PSI-treatment on the evolution of CPIs. For many developing countries, containing inflation is an important task – especially bearing in mind that inflation tends to aggravate poverty, as the poor often lack access to financial instruments through which they can hedge themselves against inflation (Easterly and Fischer, 2001).

Table 3:

Average annual CPI-inflation rates post-treatment.

article image
Source: World Economic Outlook database and own calculations.

In all PSI-countries except for Uganda, annual inflation rates were lower than in their synthetic control – typically by about 3 percentage points (even though growth tended to be higher; recall Section 7.1).

Inflation in Uganda was mainly pushed up by an inflationary episode spanning 201112, when inflation equaled 18.7 and 14.0 percent respectively. Interestingly, Uganda in 2011 is also the only case where a PSI review was not completed – indicating that the IMF’s Executive Board felt that its PSI program targets had not been met. In particular, the Acting Chair’s Summing Up noted that “Directors regretted the recent supplementary budget which places fiscal and monetary policies on an uncertain trajectory. They called for a renewed commitment to fiscal discipline and transparency and urged the authorities to take corrective measures to bring policies back in line with medium-term macroeconomic objectives (…) Directors also stressed that going forward it will be important to strictly limit recourse to central bank financing of the deficit, which has eroded official reserves. With emerging inflation pressures stemming from the laxer fiscal stance, higher food and fuel prices, and exchange rate depreciation, Directors encouraged the authorities to remain vigilant and take action if the price dynamics worsen.”15

When it comes to the robustness of our findings, Figure 5 again shows that placebo effects tend to be small relative to the treatment effects (except for Senegal).16 As with output, excluding those countries that receive the largest weight in each synthetic control, yields results that are very similar. This suggests that PSIs have indeed helped to control inflation.

7.3 Capital stock

Finally, we analyze whether PSI-programs have had an effect on the accumulation of capital. It is often argued that IMF-programs fulfill an important signaling role and that they catalyze investment. To check this claim, we first of all analyze the evolution of the total capital stock (in percent of GDP). As Figures 6 and 7 show, most treated countries saw substantial increases in their capital stocks over the course of their PSI. However, the comparison with the respective synthetic controls suggests that the adoption of a PSI does not seem to generate a large effect on capital accumulation. Figure 6’s panels for Cabo Verde, Tanzania, and Uganda ostensibly suggest a catalyzing impact, but for Tanzania and Uganda the positive treatment effect is not robust and very sensitive to the particular covariates included in the regression. Consequently, there does not seem to be strong evidence to back-up the claim that PSI-treatment spurs overall investment.

There could however be an effect on foreign direct investment (henceforth “FDI”), as foreign investors might put a greater weight on the signal emitted by the adoption of an IMF-program. Encouraging FDI could be important since there is evidence that having a larger foreign-owned capital stock is beneficial to economic development. As for example argued in De Mello (1999), FDI is accompanied by significant knowledge transfers, such as labor training and skill acquisition and the introduction of better management practices and organizational arrangements. Consequently, we perform the same analysis on the evolution of the foreign-owned capital stock.

Here, there is more evidence for a positive treatment effect (see Figures 8 and 9): all treated countries,17 except for Tanzania, accumulated more foreign-owned capital in the years following the adoption of a PSI than their untreated synthetic controls did18 (although the effect does not seem to have lasted in Senegal, where the foreign-owned capital stock fell sharply in 2014). From Figure 9, one can see that treatment effects in Cabo Verde, Mozambique, Rwanda, and Senegal (until 2014) are large relative to placebo effects. Results are furthermore robust to different regression specifications.

This suggests that PSI-programs emit a positive signal to foreign investors – catalyzing FDI. Although there is no strong evidence that this leads to a higher overall investment rate (through crowding out of domestic investment), attracting FDI can be an important driver of growth in itself as it is thought to bring substantial knowledge transfers.

8 Discussion

Our analysis in Section 7 suggests that the adoption of a PSI stimulates economic growth, contains inflation, while encouraging FDI-inflows. This section discusses these findings.

First of all, it is important to keep in mind that six out of the seven countries that have received PSIs, also went through traditional IMF programs over our sample period (mainly prior to their adoption of a PSI; recall Figure 1). It is important to think through how this could bias our findings. There are three possibilities:

  1. traditional IMF programs had no impact on the macroeconomic situation;

  2. traditional IMF programs had a positive impact on the macroeconomic situation;

  3. traditional IMF programs had a negative impact on the macroeconomic situation.

In case 1, there would be no bias and traditional programs can be neglected for the purposes of this paper. Case 2 would bias our results downward: in that case, the country already received a boost before adopting a PSI – making it harder for the SCM to find any additional positive effects after PSI-adoption (since it then compares the actual outcome against a stronger synthetic control, as the latter is based upon pre-treatment performance). Case 3 would be more problematic: if it were true that traditional IMF programs reduce growth and promote inflation, the positive effects that we tend to find could simply stem from the termination of a traditional program (rather than from the adoption of a PSI).

Fortunately, our sample contains three countries that operated disbursing IMF programs in parallel to a PSI (Mozambique, Senegal, and Tanzania; recall Figure 1). If traditional IMF programs indeed had significant negative macroeconomic side-effects, these countries should display smaller treatment effects. But our results do not hint in this direction. In addition, the observation that our findings for Nigeria (the “purest” PSI-country, as it is the only one not to appeal to a disbursing IMF program over our entire 1992-2015 sample) are very much in line with those for other PSI-countries, gives further credence to our findings.19

What could then explain the differences between results reported in this paper and those in earlier studies (most notably Barro and Lee (2005) and Dreher (2006), who find negative or no effects from IMF-programs)?

First, it is possible that the effectiveness of IMF-programs has changed over time: most earlier studies were conducted on data that spanned the 1970-2000 period, while IMF operations have changed considerably since. In particular, IMF programs have evolved from more prescriptive structural adjustment programs, toward programs that support country-led development agendas (Bal-Gunduz et al., 2013). In this light, Appendix B provides evidence suggesting that IMF programs have indeed become more effective over time.

Second, it is important to keep in mind that all earlier studies have focused on IMF-programs that come with financial assistance. Consequently, the samples underlying those studies include countries that were hit by severe economic crises (such as the Latin American debt crisis in the 1980s and the Asian crisis of the 1990s) – thereby introducing an urgent need for the econometrician to correct for reverse causality.20 While the afore mentioned studies have employed clever instrumental variable techniques to address this challenge, it is possible that a residual effect remains present (leading to a less favorable evaluation of IMF-programs). By exploiting the existence of non-funded IMF-programs (launched in countries without an acute crisis), this paper faces a lesser challenge along this dimension.

Finally, it is also possible that traditional IMF-programs are less effective than the PSI. We for example cannot rule out the possibility that IMF-advice is less useful or apt in times of severe crisis, or that such advice is less beneficial to countries at different stages of development (remember that all PSI-countries are sub-Saharan African developing countries – typically with limited access to external financing). It is also conceivable that the presence of a financial component plays a negative role: as demonstrated in Van Wijnbergen (1986), large financial aid inflows can lead to real appreciations with Dutch Disease-driven adverse consequences on economic development. Along these lines, Rajan and Subramanian (2011) indeed find evidence that aid inflows reduce a country’s competitiveness, while it is also possible that financial aid reduces reform-incentives (Boockmann and Dreher, 2003). At the aggregate level, IMF financing typically only constitutes a small fraction of total financial flows (downplaying the aforementioned concerns), but future research is needed to shine further light on the net effect of this important issue.

9 Conclusion

This paper is the first to apply the Synthetic Control Method to the evaluation of programs operated by an international financial institution (in this particular case, the IMF). We have exploited the convenient fact that there exist non-financial IMF programs (Policy Support Instruments, or “PSIs”). They enable us to analyze whether IMF advice, monitoring, and approval bring any benefits by themselves (eliminating the direct impact from financial assistance), while mitigating the reverse causality problems that have complicated earlier analyses.

Our results suggest that IMF involvement promotes economic growth and helps containing inflation: in the years following the adoption of a PSI, treated countries tend to add one percentage point in average annual GDP per capita growth, while inflation decreases by about 3 percentage points per year. There is also evidence that PSI treatment promotes foreign investment, which is associated with better management practices and positive knowledge transfers.

It is important to emphasize the limitations of this study: our results stem solely from analyzing PSIs. As explained in the Introduction, limiting the analysis to PSIs facilitates the process of constructing counterfactuals, while enabling us to eliminate any effects from direct financial assistance. On the downside, however, our specific focus limits the validity of our results to other types of IMF-programs: they come with a financial component, while they tend to be launched in times of severe economic crisis. Additional research is necessary to establish whether IMF-advice is equally useful in such stressing circumstances, while financial aid inflows can furthermore lead to real appreciations with adverse consequences on economic development.

The present paper however suggests that, at least for the case of PSIs, adopting countries might experience substantial macroeconomic benefits.

10 Appendix A: background information

Table A1:

Countries in the donor pool

article image
Note: The above table contains all countries that are classified as “developing” in the IMF’s World Economic Outlook database, while not having any kind of IMF-program in place over the period 19922016. Due to data limitations, not all countries can be included. Only the countries in bold are considered in the exercise for real GDP per capita; only the countries in italics are considered in the CPI-exercise; only the countries with an asterisk are considered in the capital-stock exercises.
Table A2:

Synthetic controls and fit indices for real GDP per capita

article image
Table A3:

Synthetic controls and fit indices for CPI21

article image
Table A4:

Synthetic controls and fit indices for capital stock

article image
Table A5:

Synthetic controls and fit indices for foreign-owned capital stock22

article image
Table A6:

Comparison of covariates in treated countries and their synthetic controls

article image

11 Appendix B: effectiveness of IMF programs over time

Applying the SCM to PSI-treated countries suggests that this type of IMF program has had a positive effect on economic development in those countries. This contrasts with the findings of some earlier notable studies, such as Barro and Lee (2005) and Dreher (2006) – who found negative or no treatment effects for other program types.

One possible reason for this difference lies in the fact that these earlier studies largely analyzed data that were generated before the 1999 reforms to the IMF. As explained in Bal-Gunduz et al. (2013), the 1999 reforms mainly focused on the Fund’s engagement with low-income developing countries (LIDCs) and placed poverty reduction and country ownership of programs at the center, whilst relying less on conditionality.

Unfortunately, it is not possible to analyze whether these changes have affected the effectiveness of PSIs, as that program-type was only introduced in 2005. Consequently, this Appendix proceeds by analyzing the effectiveness of IMF programs more generally (using data running from 1980-2015).

By broadening our analysis to other types of IMF programs, we end up with a larger sample size (and a longer timespan). This renders the SCM less suitable, but enables Propensity Score Matching (“PSM”). That approach is popular in the microeconometric impact evaluation literature, while several earlier studies have applied it to analyze the effect of IMF programs (see Hutchison (2004), Atoyan and Conway (2005), IMF (2012), and Bal-Gunduz et al. (2013)).

PSM is a method that is related to the SCM and operates as follows.23 For all countries in the dataset, PSM starts by estimating the probability of entering a program with the IMF in any given year, typically through a probit-regression. In a second step, it takes countries which actually entered a program and “matches” them with a control group of countries that had a similar probability of entering a program, but did not. With this step, the PSM attempts to correct for adverse selection into IMF programs. The treatment effect is finally calculated by comparing average outcomes in program countries, with average outcomes in control-group countries.

Based upon theory and the aforementioned earlier PSM-IMF studies, we have included the following variables in the probit-regression determining whether a country enters an IMF program or not:

  • level of real GDP per capita;

  • growth rate of real GDP per capita;

  • current account balance (as percent of GDP);

  • overall fiscal balance (as percent of GDP);

  • growth rate of terms-of-trade;

  • percentage point change in international reserves (as percent of GDP);

  • a dummy variable indicating a banking or currency crisis.

All variables enter the regression with a lag and, as can be seen from Table B1, they tend to affect the probability of entering an IMF program with the expected sign (especially in the regression using all IMF member countries; Table B1, Column 1). When estimating solely on LIDCs, the sample size shrinks and there is some loss of significance (Table B1, Column 2).

Table B1:

Probit regression for entering an IMF program.

article image
Note: t-statistics in parentheses; robust standard errors are clustered at the country-level. Condition index = 4.2, suggesting multicollinearity is not a major concern in this regression. * denotes significance at the 10% level, ** implies significance at the 5% level, *** indicates significance at the 1% level.

Using this probit specification in the first stage of the PSM approach, we analyze what effects IMF programs have had in LIDCs (on which PSIs focus). We look at real per capita GDP growth and inflation in the 5 years following program adoption. Applying this method to our entire 1980-2015 sample, we find that IMF programs have had a negative impact on real GDP per capita growth (by about 1 percentage point per year; Table B2, Column 1) – a result that is in line with the aforementioned studies of Barro and Lee (2005) and Dreher (2006). We fail to find a significant impact on inflation, but the point estimate suggests that IMF programs increased in inflation.

Table B2:

Average treatment effect on treated in low-income developing countries

article image
Note: these estimates were obtained through propensity score matching; z-statistics in parentheses. Inflation and growth are measured as 5 year averages. * denotes significance at the 10% level, ** implies significance at the 5% level, *** indicates significance at the 1% level.

For the sake of our paper, we are however more interested in how this effectiveness has evolved over time. Consequently, we do a sub-sample analysis on the post-reform period (2000-2015). There, we find that the negative effect of IMF programs on subsequent rates of economic growth disappears (while the response of inflation continues to be insignificant, with the point estimate moving closer towards zero; Table B2, Column 2). This indeed supports the hypothesis that the effectiveness of IMF programs in LIDCs has improved over the years – possibly due to the 1999 PRGT-reforms.

We hope that future research will be able to address this finding in greater detail and provide evidence on the exact mechanism.

12 References

  • Abadie, Alberto, Alexis Diamond, and Jens Hainmueller (2010), “Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program,Journal of the American Statistical Association, 105 (490), pp. 493505.

    • Search Google Scholar
    • Export Citation
  • Abadie, Alberto and Javier Gardeazabal (2003), “The Economic Costs of Conflict: A Case Study of the Basque Country,American Economic Review, 93 (1), pp. 113132.

    • Search Google Scholar
    • Export Citation
  • Adhikari Bibek and James Alm (2016), “Evaluating the Economic Effects of Flat Tax Reforms Using Synthetic Control Methods,Southern Economic Journal, forthcoming.

    • Search Google Scholar
    • Export Citation
  • Adhikari, Bibek, Romain Duval, Bingjie Hu, and Prakash Loungani (2016), “Can Reform Waves Turn the Tide? Some Case Studies Using the Synthetic Control Method,IMF Working Paper 16/171, Washington D.C.: IMF.

    • Search Google Scholar
    • Export Citation
  • Arezki, Rabah, Frederick van der Ploeg, and Frederik Toscani (2016), “Shifting Frontiers in Global Resource Wealth: The Role of Policies and Institutions,OxCarre Research Paper No. 180.

    • Search Google Scholar
    • Export Citation
  • Atoyan, Ruben and Patrick Conway (2006), “Evaluating the Impact of IMF Programs: A Comparison of Matching and Instrumental-Variable Estimators,Review of International Organizations, 1 (2), pp. 99124.

    • Search Google Scholar
    • Export Citation
  • Bal-Gunduz, Yasemin et al. (2013), “The Economic Impact of IMF-Supported Programs in Low-Income Countries,Occasional Paper No. 277, Washington, D.C.: IMF.

    • Search Google Scholar
    • Export Citation
  • Barro, Robert J. and Xavier Sala-i-Martin (1995), Economic Growth, New York, NY: McGraw-Hill.

  • Barro, Robert J. and Jong-Wha Lee (2005), “IMF Programs: Who is Chosen and What are the Effects?,Journal of Monetary Economics, 52 (7), pp. 12451269.

    • Search Google Scholar
    • Export Citation
  • Bas, Muhammet A. and Randall W. Stone (2014), “Adverse Selection and Growth under IMF Programs,The Review of International Organizations, 9 (1), pp. 128.

    • Search Google Scholar
    • Export Citation
  • Basdevant, Olivier, Andrew Jonelis, Borislava Mircheva, and Slavi Slavov (2014), “The Mystery of the Missing Real Spillovers in Southern Africa: Some Facts and Possible Explanations,African Departmental Paper 14/03, Washington D.C.: IMF.

    • Search Google Scholar
    • Export Citation
  • Billmeier, Andreas and Tommaso Nannicini (2013), “Assessing Economic Liberalization Episodes: A Synthetic Control Approach,Review of Economics and Statistics, 95 (3), pp. 9831001.

    • Search Google Scholar
    • Export Citation
  • Binder, Michael and Marcel Bluhm (2017), “On the Conditional Effects of IMF Program Participation on Output Growth,Journal of Macroeconomics, 51, pp. 192214.

    • Search Google Scholar
    • Export Citation
  • Blonigen, Bruce A., Ronald B. Davies, Glen R. Waddell, and Helen T. Naughton (2007), “FDI in Space: Spatial Autoregressive Relationships in Foreign Direct Investment,European Economic Review, 51 (5), pp. 13031325.

    • Search Google Scholar
    • Export Citation
  • Boockmann, Bernhard and Axel Dreher (2003), “The Contribution of the IMF and the World Bank to Economic Freedom,European Journal of Political Economy, 19 (3), pp. 633649.

    • Search Google Scholar
    • Export Citation
  • Canales-Kriljenko, Jorge, Farayi Gwenhamo, and Saji Thomas (2013), “Inward and Outward Spillovers in the SACU Area,IMF Working Paper 13/31, Washington D.C.: IMF.

    • Search Google Scholar
    • Export Citation
  • Cavallo, Eduardo, Sebastian Galiani, Ilan Noy, and Juan Pantano (2013), “Catastrophic Natural Disasters and Economic Growth,Review of Economics and Statistics, 95 (5), pp. 15491561.

    • Search Google Scholar
    • Export Citation
  • Center for Global Development (2007), “Does the IMF constrain health spending in poor countries? Evidence and an agenda for action,Report of the Working Group on IMF Programs and Health Spending, Washington, DC.

    • Search Google Scholar
    • Export Citation
  • Chinn, Menzie D. and Hiro Ito (2006), “What Matters for Financial Development? Capital Controls, Institutions, and Interactions,Journal of Development Economics, 81 (1), pp. 163192.

    • Search Google Scholar
    • Export Citation
  • Clements, Benedict, Sanjeev Gupta, and Masahiro Nozaki (2013), “What Happens to Social Spending in IMF-Supported Programmes?,Applied Economics, 48 (28), pp. 40224033.

    • Search Google Scholar
    • Export Citation
  • De Mello, Luiz R. (1999), “Foreign Direct Investment-led Growth: Evidence from Time Series and Panel Data,Oxford Economic Papers, 51 (1), pp. 133151.

    • Search Google Scholar
    • Export Citation
  • Dicks-Mireuax, Louis, Mauro Mecagni, and Susan Schadler (2000), “Evaluation the Effect of IMF Lending to Low-Income Countries,Journal of Development Economics, 61 (2), pp. 495526.

    • Search Google Scholar
    • Export Citation
  • Donovan, Donal J. (1981), “Real Responses Associated with Exchange-Rate Action in Selected Upper Credit Tranche Stabilization Programs,IMF Staff Papers, 28 (4), pp. 698727.

    • Search Google Scholar
    • Export Citation
  • Donovan, Donal J. (1982), “Macroeconomic Performance and Adjustment Under Fund-Supported Programs: The Experience of the Seventies,IMF Staff Papers, 29 (2), pp. 171203.

    • Search Google Scholar
    • Export Citation
  • Doudchenko, Nikolay and Guido W. Imbens (2016), “Balancing, Regression, Difference-In-Differences and Synthetic Control Methods: A Synthesis,NBER Working Paper No. 22791.

    • Search Google Scholar
    • Export Citation
  • Dreher, Axel (2006), “IMF and Economic Growth: The Effects of Programs, Loans, and Compliance with Conditionally,World Development, 34 (5), pp. 769788.

    • Search Google Scholar
    • Export Citation
  • Dreher, Axel (2009), “IMF Conditionality: Theory and Evidence,Public Choice, 141 (1), pp. 233267.

  • Easterly, William (2003), “IMF and World Bank Structural Adjustment Programs and Poverty,” in: Michael P. Dooley and Jeffrey A. Frankel eds, Managing Currency Crises in Emerging Markets, Chicago: University of Chicago Press.

    • Search Google Scholar
    • Export Citation
  • Easterly, William and Stanley Fischer (2001), “Inflation and the Poor,Journal of Money, Credit and Banking, 33 (2), pp. 160178.

    • Search Google Scholar
    • Export Citation
  • Garuda, Gopal (2000), “The Distributional Effects of IMF Programs: A Cross-Country Analysis,World Development, 28 (6), pp. 103151.

    • Search Google Scholar
    • Export Citation
  • Ghosh, Atish R., Jonathan D. Ostry, and Charalambos Tsangarides (2010), “Exchange Rate Regimes and the Stability of the International Monetary System,IMF Occasional Paper No. 270.

    • Search Google Scholar
    • Export Citation
  • Hajro, Zlata and Joseph P. Joyce (2009), “A True Test: Do IMF Programs Hurt the Poor?,Applied Economics, 41 (3), pp. 295306.

  • Haque, Nadeem Ul and Mohsin S. Khan (1998), “Do IMF-supported Programs Work? A Survey of the Cross-Country Empirical Evidence,IMF Working Paper No. 98/169, Washington, D.C.

    • Search Google Scholar
    • Export Citation
  • Hutchison, Michael M. (2004), “Selection Bias and Output Costs in IMF Programs,EPRU Working Paper No. 04-15.

  • IMF (2006), “Suriname: 2005 Article IV Consultation—Staff Report,IMF Country Report No. 06/135, Washington, D.C.

  • IMF (2007), “Suriname: 2007 Article IV Consultation—Staff Report,IMF Country Report No. 07/180, Washington, D.C.

  • IMF (2012), “2011 Review of Conditionality: Outcomes of Fund-Supported Programs,IMF Policy Paper, Washington, D.C.

  • IMF (2015), “Eligibility to Use the Fund’s Facilities for Concessional Financing,Washington, D.C.

  • IMF (2016), “Handbook of IMF Facilities for Low-Income Countries,Washington, D.C.

  • Kaul, Ashok, Stefan Klossner, Gregor Pfeifer, and Manuel Schieler (2016), “Synthetic Control Methods: Never Use All Pre-Intervention Outcomes as Economic Predictors,mimeo, Saarland University.

    • Search Google Scholar
    • Export Citation
  • Lane, Philip R. and Gian Maria Milesi-Ferretti (2007), “The External Wealth of Nations Mark II: Revised and Extended Estimates of Foreign Assets and Liabilities, 1970-2004,Journal of International Economics, 73 (2), pp. 223250.

    • Search Google Scholar
    • Export Citation
  • Lang, Valentin F. (2016), “The Economics of the Democratic Deficit: The Effect of IMF Programs on Inequality,Discussion Paper Series No. 617, University of Heidelberg.

    • Search Google Scholar
    • Export Citation
  • Oberdabernig, Doris A. (2013), “Revisiting the Effects of IMF Programs on Poverty and Inequality,World Development, 46, pp. 113142.

    • Search Google Scholar
    • Export Citation
  • Przeworski, Adam and James Raymond Vreeland (2000), “The Effect of IMF Programs on Economic Growth,Journal of Development Economics, 62 (2), pp. 385421.

    • Search Google Scholar
    • Export Citation
  • Rajan, Raghuram G. and Arvind Subramanian (2011), “Aid, Dutch Disease, and Manufacturing Growth,Journal of Development Economics, 94 (1), pp. 106118.

    • Search Google Scholar
    • Export Citation
  • Reichmann, Thomas M. and Richard T. Stillson (1978), “Experience with Programs of Balance of Payments Adjustment: Stand-by Arrangements in the Higher Credit Tranches, 1963-72,IMF Staff Papers, 25 (2), pp. 119139.

    • Search Google Scholar
    • Export Citation
  • Rosenbaum, Paul R. and Donald B. Rubin (1983), “The Central Role of the Propensity Score in Observational Studies for Causal Effects,Biometrika, 70 (1), pp. 4155.

    • Search Google Scholar
    • Export Citation
  • Sanso-Navarro, Marcos (2011), “The Effects of American Foreign Direct Investment in the United Kingdom from Not Adopting the Euro,Journal of Common Market Studies, 49 (2), pp. 463483.

    • Search Google Scholar
    • Export Citation
  • Steinwand, Martin C. and Randall W. Stone (2008), “The International Monetary Fund: A review of the recent evidence,Review of International Organizations, 3 (2), pp. 123149.

    • Search Google Scholar
    • Export Citation
  • Taylor, John B. (2006), “The Policy Support Instrument: A Key Component of the Recent IMF Reform Movement,” in: Edwin M. Truman ed, Reforming the IMF for the 21st Century, Special Report 19, Institute for International Economics.

    • Search Google Scholar
    • Export Citation
  • Van Wijnbergen, Sweder (1986), “Macroeconomic Aspects of the Effectiveness of Aid: On the Two-Gap Model, Home Goods Disequilibrium, and Real Exchange Rate Misalignment,Journal of International Economics, 21 (1-2), pp. 123136.

    • Search Google Scholar
    • Export Citation
  • Wooldridge, Jeffrey M. (2002), Econometric Analysis of Cross Section and Panel Data, Cambridge, MA: The MIT Press.

  • World Bank (2016), Global Economic Prospects: Spillovers amid Weak Growth, Washington D.C.: World Bank.

  • Xiong, Yi (2014), “The Role of Megaprojects and Their Relationship to Jobs and Growth,” in: Doris C. Ross et al, “Mozambique Rising—Building a New Tomorrow,African Departmental Paper, International Monetary Fund, Washington, D.C.

    • Search Google Scholar
    • Export Citation

13 Figures

Figure 1:
Figure 1:

IMF-programs in PSI countries (1992-2015)

Citation: IMF Working Papers 2017, 109; 10.5089/9781475599039.001.A001

Figure 2:
Figure 2:

SCM-results: real GDP per capita (in constant 2010 USD)

Citation: IMF Working Papers 2017, 109; 10.5089/9781475599039.001.A001

Figure 3:
Figure 3:

Placebo-results: real GDP per capita (in constant 2010 USD)

Citation: IMF Working Papers 2017, 109; 10.5089/9781475599039.001.A001

Figure 4:
Figure 4:

SCM-results: CPI (2005 = 100)

Citation: IMF Working Papers 2017, 109; 10.5089/9781475599039.001.A001

Figure 5:
Figure 5:

Placebo-results: CPI (2005 = 100)

Citation: IMF Working Papers 2017, 109; 10.5089/9781475599039.001.A001

Figure 6:
Figure 6:

SCM-results: capital stock (in percent of GDP)

Citation: IMF Working Papers 2017, 109; 10.5089/9781475599039.001.A001

Figure 7:
Figure 7:

Placebo-results: capital stock (in percent of GDP)

Citation: IMF Working Papers 2017, 109; 10.5089/9781475599039.001.A001

Figure 8:
Figure 8:

SCM-results: foreign-owned capital stock (in percent of GDP)

Citation: IMF Working Papers 2017, 109; 10.5089/9781475599039.001.A001

Figure 9:
Figure 9:

Placebo-results: foreign-owned capital stock (in percent of GDP)

Citation: IMF Working Papers 2017, 109; 10.5089/9781475599039.001.A001

1

We thank Francisco Arizala, Karim Barhoumi, Bingjie Hu, Valentin Lang, Marina Mendes Tavares, Roger Nord, Axel Schimmelpfennig, Ke Wang, Reza Yousefi and seminar participants at the IMF and the CSAE conference in Oxford for useful comments.

1

This section heavily draws on information that is available at www.imf.org/external/np/exr/facts/psi.htm.

2

“PRGT” refers to “Poverty Reduction and Growth Trust”, the IMF-facility through which all concessional lending is provided. The per capita income cutoff is set at the IDA operational cutoff, which was located at a per capita Gross National Income level of US$1,215 in 2015. Evaluation of the market access criterion is done on a more case-by-case basis.

3

This method is related to the “matching methods” developed by Donald Rubin and co-authors (Doudchenko and Imbens, 2016). That approach however requires large samples, as it is difficult to find “matches” in smaller samples; the SCM, in contrast, can even be applied if there is only one treated country (in the present study we have seven), while our donor pool contains 39 countries.

4

Nigeria had two Standby Arrangements in place (one from January 1991 to April 1992 and another one from August 2000 to October 2001) without ever drawing on them.

5

In 2009, both Mozambique and Tanzania entered a 12-month arrangement under the Exogenous Shocks Facility (ESF, a program type specifically designed to address BoP-needs caused by external developments), as the global economic slowdown deteriorated the countries’ reserve positions. In the same year, Senegal also entered an ESF as they suffered from higher food and energy prices. In 2012, the IMF’s Executive Board approved an 18-month Standby Credit Facility (SCF) for Tanzania. That program was put in place for precautionary reasons, not because of an immediate BoP-need (see http://www.imf.org/external/np/sec/pr/2012/pr12252.htm).

6

Cavallo et al. (2013) also include the polity2 score, but that variable is not available for many countries in our sample.

7

Some earlier studies have included all pre-intervention outcomes as economic predicts, but as set out in Kaul et al. (2016), this renders all other covariates irrelevant – irrespective of how important they are in predicting post-treatment outcome values. This may lead to biased estimates. Using only a limited number of pre-intervention outcomes as predictors, prevents this.

8

The classification comes from the IMF’s Annual Report on Exchange Arrangements and Exchange Restrictions. It is coded such that the value “1” represents hard pegs, “2” conventional pegs, “3” basket pegs, “4” pegs within bands, “5” crawling pegs, “6” managed oats, and “7” independent floats.

9

Ghosh, Ostry, and Tsangarides (2010) also include broad money growth, real GDP growth, fiscal balance, trade openness, and terms of trade growth as explanatory variables in their regression. Apart from the latter two series, all of these are potentially affected by the policy-treatment – invalidating them as covariates in the SCM (recall Section 5). Adding trade openness and terms of trade growth to the algorithm does not affect our findings (but significantly reduces sample size, thereby worsening pre-treatment fit), while including measures of broad money, real GDP, and the fiscal balance (so that our list of covariates exactly matches that of Ghosh, Ostry, and Tsangarides (2010)) does not alter the conclusions either.

10

Since the CPI equals 100 for every country in the year 2005, there is no point in adding that observation as a predictor.

11

Blonigen et al. (2007) use a composite risk index, but that variable is not available for most countries included in the present study.

12

Given that we have annual data, we “round” dates to their nearest integer in our empirical exercise. So for Cabo Verde, treatment starts in 2007, for Nigeria in 2006, et cetera.

14

In the placebo-exercises there is one country that – although untreated – consistently “takes off” in the post-treatment period, namely Namibia (which is the outlier visible in the panels for Cabo Verde, Nigeria, Rwanda, Senegal, Tanzania, and Uganda). This illustrates that the SCM has difficulty in dealing with idiosyncratic developments. Theoretically, all action in Figure 2 could be driven by such country-specific developments, but given that a positive treatment effect emerges in all seven treated countries, this is unlikely. A common underlying cause (PSI-treatment) seems a more plausible explanation.

15

See “The Acting Chair’s Summing Up Uganda—2010 Article IV Consultation, First Review Under the Policy Support Instrument, and Requests for Waiver of Assessment Criterion and Modification of Assessment Criteria Executive Board Meeting”, February 11, 2011.

16

Again, there is one country in the control group that displays a substantial treatment effect in the placebo-exericse, namely Suriname. This is the downward outlier visible in the panels for Cabo Verde, Mozambique, Nigeria, Senegal, and Tanzania. Without entering an engagement with the IMF, they managed to control their inflation rates by implementing reforms that strengthened Central Bank independence, stabilized the exchange rate, and anchored inflation (IMF, 2006; 2007).

17

Unfortunately, the SCM is not able to construct a reasonable synthetic control for Nigeria, which we therefore disregard in the remainder of this analysis.

18

Especially the Mozambican case illustrates that improved (perceptions of) institutions in resource rich countries can trigger impressive FDI take-offs: in Mozambique, initial exploration investments (stimulated by improvements in the business climate – an objective of the PSI) led to the discovery of a major gas field in the Rovuma basin, which attracted further FDI (Xiong, 2014). Also see Arezki, Van der Ploeg, and Toscani (2016) on these dynamics.

19

In fact, Nigeria is the country for which the SCM finds the biggest treatment effect on growth (+3.0 percentage points). This suggests that, if anything, the presence of pre-PSI-era programs in other countries biases our findings with respect to growth towards zero – as hypothesized under case 2.

20

While Mozambique, Senegal, and Tanzania also relied upon disbursing programs during their PSI-spell, this was not the result of severe economic crises and the programs launched in response were relatively small (ranging from 37.5 to 109.9 percent of quota; to compare, South Korea drew over 3000 percent of quota during their crisis in the late 1990s). As a result, adverse selection continues to be a possible source of bias in these countries (against finding positive effects from treatment), but a much smaller one. The observation that the SCM does find positive results from treatment (despite the absence of an explicit bias correction) supports this.

21

In this case, there are many control countries that obtain a weight smaller than 2%. For reasons of legibility, we only report the most important countries, i.e. the ones with weights larger than 2%.

22

Nigeria has to be disregarded in this analysis since no “fitting” synthetic control can be constructed from the donor pool (see Figure 8).

23

See the seminal contribution of Rosenbaum and Rubin (1983) and Section 18.3.2 of Wooldridge (2002) for a detailed discussion of the PSM method. Doudchenko and Imbens (2016) discuss the relation between the two methods.

Evaluating the Impact of Non-Financial IMF Programs Using the Synthetic Control Method
Author: Ms. Monique Newiak and Tim Willems