Settling the Inflation Targeting Debate: Lights from a Meta-Regression Analysis

Contributor Notes

Author’s E-Mail Address: hbalima@imf.org, kilamaericgabin@yahoo.fr, rtapsoba@imf.org(corresponding author).

Inflation targeting (IT) has gained much traction over the past two decades, becoming a framework of reference for the conduct of monetary policy. However, the debate about its very merits and macroeconomic consequences remains inconclusive. This paper digs deeper into the issue through a meta-regression analysis (MRA) of the existing literature, making it the first application of a MRA to the macroeconomic effects of IT adoption. Building on 8,059 estimated coefficients from a very broad sample of 113 studies, the paper finds that the empirical literature suffers from two types of publication bias. First, authors, editors and reviewers prefer results featuring beneficial effects of IT adoption on inflation volatility, real GDP growth and fiscal performances; second, they promote results with estimated coefficients that are significantly different from zero. However, after filtering out the publication biases, we still find meaningful (genuine) effects of IT in reducing inflation and real GDP growth volatility, but no significant genuine effects on inflation volatility and the level of real GDP growth. Interestingly, the results indicate that the impact of IT varies systematically across studies, depending on the sample structure and composition, the time coverage, the estimation techniques, country-specific factors, IT implementation parameters, and publication characteristics.

Abstract

Inflation targeting (IT) has gained much traction over the past two decades, becoming a framework of reference for the conduct of monetary policy. However, the debate about its very merits and macroeconomic consequences remains inconclusive. This paper digs deeper into the issue through a meta-regression analysis (MRA) of the existing literature, making it the first application of a MRA to the macroeconomic effects of IT adoption. Building on 8,059 estimated coefficients from a very broad sample of 113 studies, the paper finds that the empirical literature suffers from two types of publication bias. First, authors, editors and reviewers prefer results featuring beneficial effects of IT adoption on inflation volatility, real GDP growth and fiscal performances; second, they promote results with estimated coefficients that are significantly different from zero. However, after filtering out the publication biases, we still find meaningful (genuine) effects of IT in reducing inflation and real GDP growth volatility, but no significant genuine effects on inflation volatility and the level of real GDP growth. Interestingly, the results indicate that the impact of IT varies systematically across studies, depending on the sample structure and composition, the time coverage, the estimation techniques, country-specific factors, IT implementation parameters, and publication characteristics.

I. Introduction

Since its first adoption by the Reserve Bank of New Zealand in 1990, inflation targeting (IT) has gained much traction over the past two decades, becoming a framework of reference for the conduct of monetary policy. About 38 central banks are currently using IT as their monetary policy framework, and about 30 countries are considering the possibility of embracing IT in the near future (Hammond, 2012; IMF, 2016).

However, the debate about its relevance and macroeconomic consequences remains inconclusive. On the one hand, some authors challenge the very merits of this new monetary policy framework. For instance, Greenspan (2007), building on the “constructive ambiguity” theory, argues that IT adoption has considerably constrained the discretion of monetary policymakers. Stiglitz points out that IT leads central banks to raise interest rates mechanically whenever changes in prices exceed the targeted level, which can substantially reduce aggregate demand and increase the price of non-traded goods and services, particularly in developing countries (Stiglitz, 2008). The recent financial crisis has further rekindled this debate on at least two main fronts. First, many countries experienced deflation episodes in the aftermath of the recent financial crisis, raising questions about the appropriateness of monetary policy frameworks, including IT (as opposed to price level targeting), for preventing the economy from being stuck at the zero-lower bound (Walsh, 2011). Second, the crisis laid bare the limits of price stability for ensuring financial stability, especially in the face of large asset price fluctuations.2

On the other hand, the proponents of IT rather underscore the credibility and flexibility-enhancing properties of IT, on account of the enhanced central bank transparency and accountability associated with this new monetary policy framework (Bernanke and others, 1999; Bordo and Siklos, 2014; Walsh, 2009). Such enhanced transparency and accountability in turn allow IT central banks to more firmly anchor inflation expectations, thus providing more room to expand the economy in the face of adverse shocks without jeopardizing the credibility of monetary policy. In a similar vein, IT central banks are expected to have more leeway for assigning greater weights to long-term considerations and pursuing other objectives, including economic activity stabilization, through less aggressive policy rate adjustments. As such, Bernanke and Mishkin (1997) argue that IT is best described as a “framework of constrained discretion, not a mechanical policy rule.”

Beyond the above-mentioned conflicting theoretical views about the merits of IT, a large part of the debate is actually taking place in the empirical literature. Mixed results are found regarding the macroeconomic performances of IT countries versus non-IT countries. For instance, Johnson (2002) analyzes the effect of IT on the level and variability of expected inflation using a sample of industrial countries. He finds that the level of expected inflation falls after the announcement of inflation targets, but neither the variability of expected inflation nor the inflation forecast error has been affected by IT adoption. Ball and Sheridan (2003) provide a quite different interpretation when examining the economic performance of IT in industrial countries. They show that once controlled for regression to the mean, there is no evidence that IT improves performances, as measured by the behavior of inflation, output, or interest rates. Lin and Ye (2007, 2009) rather point out that previous studies, including Johnson (2002) and Ball and Sheridan (2003), do not take account of the self-selection issue in their identification strategies, which can lead to misleading conclusions. They thus make use of propensity scores-matching (PSM) methods to correct for self-selection, and find that IT adoption has been associated with significant downward trends in inflation and its dynamics in developing countries, though the effect proved not statistically significant in the case of developed countries. However, Brito and Bystedt (2010) argue that Lin and Ye’s PSM does not account for time trends, countries’ unobservable characteristics, or persistence. As a result, they build on GMM estimates, controlling for common time effects, and find no evidence that IT improves economic performance in developing countries.

In light of the plethora of conflicting findings on the macroeconomic effects of IT adoption, this study takes aim at digging deeper into the driving factors behind such diverging results. It takes advantage of the meta-regression analysis (MRA), a quantitative method that is increasingly used in various fields of international economics to take stock of existing results on the same research question from different studies (Stanley, 2001; Rusnak and others, 2013; Neves and others, 2016). Recent applications of meta-analysis in economics include studies about the trade effect of monetary unions (Rose and Stanley, 2005), the correlation of business cycles between countries (Fidrmuc and Korhonen, 2006), the effect of distance on trade (Disdier and Head, 2008), the effect of minimum wage on employment (Card and Krueger, 1995), the impact of natural resources on economic growth (Havranek and others, 2016), the trade effect of the euro (Havranek, 2010), analysis of capital controls (Magud and others, 2011), the influence of monetary policy on price level (Rusnak and others, 2013), and the relationship between inflation and central bank independence (Klomp and De Haan, 2010).

The goal is not to uncover the “true” value of the parameter under investigation, but rather to expand our understanding of the factors (model specification, estimation techniques, etc.) that can lead various studies examining the same issue to reach divergent conclusions. More specifically, the MRA allows assessing the existence of a publication selection bias, that is, a very common tendency from editors, reviewers, and/or researchers, to prefer results that are consistent with the most commonly-held views in the field (Type I bias) or that are statistically significant (Type II bias). For example, the most-commonly held view is that IT adoption helps reduce inflation. Most economists studying the effect of IT adoption on inflation would thus tend to look for econometric “tools” (estimation techniques, model specifications, alternative starting dates for IT adoption, etc.) that are more amenable to results featuring large estimated coefficients with a negative sign (Type I bias) and that that are significantly different from zero (Type II). Interestingly, the MRA allows checking whether there is a genuine effect associated with a given policy beyond the public bias, that is after filtering out the publication bias.

This paper adds to the existing literature on two main grounds. First, we construct a unique and very broad MRA database, consisting of 8,059 estimated coefficients from 113 empirical studies on the macroeconomic effects of IT. To the best of our knowledge, this is the first application of a MRA to the macroeconomic effects of IT. Second, compared with previous MRA studies, we do not focus on the relationship between IT and one outcome macroeconomic variable. We rather analyze the effect of IT on several macroeconomic outcome variables, including Price and Output Stability (as seized by the inflation rate and its volatility, and growth volatility), the State of the Real Economy (as captured by the economic growth rate), Fiscal Performance and Credibility, External Development, and Monetary and Financial Development. This makes our paper one of the largest meta-regression analysis in economics.

We unveil far-reaching results. First, the literature on the macroeconomic effects of IT adoption suffers from two types of publication selection bias. On the one hand, authors, editors, and referees tend to prefer results that are consistent with the most commonly-held views about the effects of IT adoption on inflation volatility and real GDP growth. On the other hand, they promote results that are statistically significant. Second, after filtering out the publication biases, we still uncover a meaningful (genuine) effect of IT in reducing inflation rate and real GDP growth volatility, but no significant genuine effect on inflation volatility and on the level of real GDP growth. Third, using multilevel mixed-effects restricted maximum likelihood as well as multilevel mixed-effects probit regressions, we find that differences across estimated coefficients in the literature are mainly driven by the characteristics of the study sample, including its structure and composition, the time coverage, the estimation techniques, the set of control variables considered, IT implementation parameters, and publication characteristics.

The remainder of the paper is structured as follows. Section II introduces the methodological approach. Section III discusses the meta-sample construction and the definition of associated moderator variables. Section IV discusses the MRA results, while section V briefly concludes.

II. Meta-Regression Analysis: Methodological Approach

We proceed in three steps to nail down the genuine macroeconomic effects of IT. First, we build a representative sample of empirical studies related to the macroeconomic effects of IT (called meta-sample henceforth).3 Second, we collect the estimated coefficients from these selected studies. It is worth noting that we do not systematically collect one estimate per study, but as many estimates as possible, insofar as notable methodological differences exist in at least one of the following dimensions: IT group/control group, nature of data, model specification, time coverage, or the estimation technique. Third, we assess the presence of publication selection bias in the collected estimates, and explore the drivers of heterogeneity among the collected estimates.

A. Publication Selection Bias and Genuine Effect

The sample of collected estimates might be subject to publication selection bias, that is, the very common tendency from editors, referees, and/or researchers to prefer results that are consistent with the theory/most commonly-held views about the effect of IT adoption (Type I bias) or that are statistically significant (Type II). Those biases mostly stem from the confluence of authors’ self-censoring attitudes and editors’ inclination to accept papers with highly significant estimates (Stanley and others, 2008).4 Filtering out these publication biases thus allows isolating the “genuine effect” or “true effect” (if any) of IT adoption (Stanley and Doucouliagos, 2012).

B. Model Specification

The meta-regression analysis (MRA), also known as meta-regression, is an extension of a standard meta-analysis, which allows examining the extent to which statistical heterogeneity among estimates from multiple studies can be related to one or more study characteristics (Thompson and Higgins, 2002). The MRA is thus an attempt to summarize and “make sense” of these disparate findings.

As a regression on estimates from existing regressions, the meta-analysis methodology consists of combining all these existing estimates, investigating their sensitivity to changes in the underlying assumptions associated with their estimation, identifying and filtering out possible biases in their estimation, and explaining the diversity of results across these studies in terms of study features heterogeneity (Rose and Stanley, 2005). When collecting data for a meta-analysis, three cases can be considered regarding the distribution of the “true effect”: (i) the Fixed Effects case, wherein only one estimate exists per study, and all studies have the same true effect; (ii) the Random Effects case, in which only one estimate exists per study, and true effects are heterogeneous across studies; and (iii) the Panel Random Effects case, wherein studies have multiple estimates, and true effects are heterogeneous both across and within studies (Reed et al., 2015).

Since the unit of observation in our analysis is not a study, but each regression insofar as it features a notable difference with other regressions, it is important to account for the fact that estimates within one study are likely to be interdependent (Disdier and Head, 2008). To capture the between-study heterogeneity while controlling for within-study dependence, a multilevel model (in the sense that it allows for heterogeneity both at the study level and the estimate level) is most appropriate (Doucouliagos and Laroche, 2009; and Doucouliagos and Stanley, 2009). The multilevel model allows accounting for within-study dependence through the inclusion of a random individual effect for each study, hence the reference to a multilevel random effect model. More specifically, the following equation (1) is considered:

effectij=β1+β0SEij+ei+λj(1)

where effectij stands for the ith estimate from the jth study on the effect of IT on a macroeconomic variable; SEtj for the standard error of ith estimate from the jth study. ei and λj are the estimate and study level disturbance terms, respectively. β1 stands for the “true value” of the effect of IT on the macroeconomic variable. β0SEij captures the “noise” or very tendency from the authors and reviewers to prefer results that are statistically significant, and thus to make use of alternative estimation techniques and/or model specifications to get high significance levels, especially in the face of small sample. If the number of observations increases indefinitely, the standard error SEij will tend towards zero, and the estimated effect of the effect of IT will get closer to β1. In other terms, if there is no publication selection bias, the collected estimates will vary randomly around the “true value” β1, regardless of the standard error. We correct for heteroscedasticity owing to differences across studies on the sample size and model specifications used, by dividing equation (1) by SEij, which becomes (2):

tij=β0+β1(1/SEij)+ei+λj(2)

where tij represents the collected t-values. We then assess the existence of Type I publication bias by testing the null hypothesis that the intercept term in equation (2) is equal to zero (β0 = 0), which is also known as the funnel asymmetry test (FAT). Basically, if the intercept is statistically significant, it means that the collected estimates do not vary symmetrically and randomly around the “true effect” (β0SEij in equation (1) has an influence), hence the name “funnel asymmetry test (FAT)” associated with this test. Given our meta-analysis focuses on the relationship between IT and several macroeconomic variables, hence on collected estimated with likely opposite signs, for comparability purpose, we replace the left-hand side of equation (2) with the absolute t-student value, which gives equation (3):

|tij|=β0+β1(1/SEij)+ei+λj(3)

Testing the null hypothesis of β0 = 0 in equation (3) thus allows assessing the presence of Type II publication selection bias. To check the presence of a “genuine” effect after filtering out the publication bias (if any), we follow Stanley and Doucouliagos (2012) and carry out the so-called precision-effect test (PET). Concretely, we test the null hypothesis that the parameter associated with the inverse standard error (β1) in equation (2) equals to zero. In other terms, we test whether in equation (1), the intercept (β1 or genuine effect) has a statistically significant role regardless of the outcome of the publication selection bias (influence of β0SEij). Rejecting the null hypothesis would thus signal that a genuine effect remains after filtering the publication bias.

When the multilevel random effect model includes covariates (or moderators) accounting for heterogeneity between studies, the model becomes best described as “multilevel mixed-effect model”. More specifically, equation (2) and (3) become:

tij=β0+β1(1/SEij)+βkxijSEij+ei+λj(4)

or

|tij|=β0+β1(1/SEij)+βkxijSEij+ei+λj(5)

where xij stands for a set of meta-independent variables, capturing empirical study characteristics from the meta-sample that explain the differences in estimates between studies. Several multilevel random-effect methods have been proposed in in the MRA literature to estimate the between-study variance in meta-regressions.5 The most commonly used method computes the unknown variance of the random effect model through an iterative residual (restricted) maximum likelihood process (REML), with normal distributions assumed for both the within and between-study effects. We rely on the multilevel mixed-effect REML in this study, given its appealing properties, namely avoiding not only downward-biased estimates of the between-study variance, but also underestimated standard errors and anti-conservative inference. In addition, the restricted maximum likelihood (RELM) is preferred to the maximum likelihood (ML) because the latter does not account for the degrees of freedom employed when estimating the fixed-effect portion of the model, which is a key shortcoming, especially in the face of small number of studies included in the MRA (Thompson and Sharp, 1999; and Benos and Zotou, 2014). For robustness check, we also employ the cluster-robust weighted least squares (WLS) to assess the sensitivity of the results to the chosen estimator.6

For the sake of further robustness check, we also make use of multilevel mixed-effects probit regressions to identify the circumstances under which (moderators) a primary study is likely to yield a statistically significant beneficial effect associated with IT (Koetse and others, 2009; Card and others, 2010). The multilevel mixed-effects probit presents the advantage of accounting not only for the size of the estimate, but also for whether it is significant or not in the primary studies. Unlike the multilevel mixed-effects REML, which is concerned with the study characteristics that affect the significance of the collected estimate, the multilevel mixed-effects probit model is interested in the moderators that lead a study to yield both statistically significant and beneficial estimates.

III. Meta-Samples and Moderator Variables

We now turn onto the strategy used to put together the meta-dataset, along with the study characteristics (moderators) employed in the MRA.

A. Database Construction

Studies Collection

Before going any further, let us emphasize that our main goal is to build a MRA database, which, to the extent possible, encompasses studies that empirically examine the relationship between IT and some variables. We follow a four-pronged approach, in line with Stanley (2001) and Stanley et al. (2013). First, we dig into Google Scholar citations of IT-related seminal papers (Ammer and Freeman, 1995; Bernanke and Mishkin, 1997; Masson and others, 1997; Svensson, 1997; Mishkin and Posen, 1997; Bernanke and others, 1999; Kuttner and Posen, 1999), and gradually into some more recent studies (Truman, 2003; Ball and Sheridan, 2004; Rose, 2007; Lin and Ye, 2007, 2009). This first round of exploration yields 7,537 candidate studies. Second, using “Inflation targeting” and “Monetary Policy Regime” as research keywords, we widen our search field to internet and academic repositories such as “Science Direct”, “JSTOR”, “RePec Ideas”, “Google Scholar”, “Wiley”, “Springer Science”, and “NDLTD”.7 Third, for studies that are not freely available online, we reach out bilaterally to the authors. Fourth, we rely on interlibrary loan systems to access undisclosed studies.

We then further narrow down the selection criteria within the collected studies from the search process above by excluding non-empirical studies. Within the collected empirical studies, we exclude those that do not consider at least one indicator of IT as an explanatory variable. This leaves us with a meta-dataset of 113 studies on the empirical macroeconomic effects of IT adoption. Figure 1 below highlights these 113 studies, along with their publication year and formats. A noticeable pattern is that the IT-related empirical literature bourgeoned in the early 2000s before abounding from 2010 onwards. On average, 14 studies were carried out a year.8

Figure 1.
Figure 1.

Inflation Targeting-related Empirical Studies Retained in our MRA.

Citation: IMF Working Papers 2017, 213; 10.5089/9781484320754.001.A001

Estimates Collection

We collect, from each retained study, estimates of IT effects, as well as information on the IT implementation forms, the size and composition of the sample, the estimation techniques, the covariates used, the publication year and formats, and other relevant information for the MRA. From the 113 empirical studies, we collected 8,059 estimates of IT on several outcome variables.

On balance, the collected studies rely on samples made up of 15 Inflation targeting countries (ITers) against 41 non-Inflation targeting countries (non-ITers). A striking feature of the literature is the plethora of estimates per study, with 71 regressions on average. Such a pattern owes much to a growing tendency from researchers to prove, to the extent possible, the robustness of their results through a multiplication of sensitivity tests (for instance using alternative IT adoption dates, investigating the effect of IT on various outcome indicators in a single study, etc.).

Putting the Collected Estimates Together

Since our MRA focuses on the relationship between IT and several macroeconomic variables (as opposed to a MRA focused exclusively on the relationship between IT and inflation, for example), it is critical to synthesize the collected estimates into fairly comparable sets of outcome indicators for implementation purposes— overcoming the high number of outcome variables and making it easier to interpret economically the MRA results. To this end, we split our collected estimates into the following five meta-regression groups. The first group, dubbed Price and Output Stability, consists of studies that analyze the stabilizing effects of IT, as captured by its influence on price dynamics (inflation level, inflation volatility, and inflation persistence or inflation expectations anchoring9), and on the stability of the real economy (volatility of GDP growth rate, output gap, and unemployment rate variability). 3,370 estimates from 75 studies are retained in this group. For the sake of robustness check, we further split this group into three more homogeneous sub-groups: (i) a group of studies that examine the effect of IT exclusively on the level of inflation, (ii) another group of studies interested in the effect of IT exclusively on the volatility of inflation, (iii) and a group of studies dealing solely with the effect of IT on the volatility of GDP growth.

The second group includes studies that analyze the economic costs associated with IT adoption. More specifically, this group, labelled State of the Real Economy, focuses on the output costs of IT (real GDP growth rate, unemployment rate, disinflation costs of IT, sacrifice ratio), the competitiveness costs of IT (real effective exchange rate, credit to the private sector, policy rate and its volatility), and the financial stability costs. 53 studies, with 2,085 estimates meet the criteria for this group. Again, for robustness purpose, we narrow down further this group into a more homogeneous block, consisting of studies that focus exclusively on the consequences of IT on the level of real GDP growth.

The third group lumps together papers that explore the effect of IT on fiscal policy performance and fiscal policy credibility, as captured by fiscal discipline, sovereign spreads or debt ratings, and institutional quality. This group, labelled as Fiscal Performance and Credibility, is made up of 14 studies containing 1,700 estimates. The fourth building block, called External Development, regroups studies that assess the impact of IT on external volatility (exchange rate volatility), a balance of payment component (current account, financial account), and a measure of capital or financial openness. External Development comprises 16 studies, corresponding to 733 estimates. The fifth and last group, dubbed Monetary and Financial Development, includes studies concerned with the influence of IT on liquidity conditions (broad money growth, velocity), financial depth (deposit rates, bond market health, and degree of financial dollarization). Six studies, corresponding to 171 estimates make up this group.

Note, however, that a proper meta-analysis requires roughly at least 20 studies (Stanley, 2016). As a result, we discard External Development, Fiscal Performance and Credibility, and Monetary and Financial Development from our multivariate MRA, as they contain only 16, 14, and 6 studies, respectively.10 This leaves us with two groups in the multivariate analysis, Price and Output Stability and State of the Real Economy groups. Appendix 1 reports the five meta-regression groups along with their associated studies, the dependent variables in each study, and some descriptive statistics on the estimates. A key feature of the retained meta-database lies in the significant heterogeneity across the estimates, both between and within studies in each group, as exemplified by the different mean value (which can be positive or negative) of the estimate within each study. In the following, we aim to explain this heterogeneity across studies, also known as excess study-to-study variation, using the MRA.

B. Sources of Heterogeneity

We now highlight key study characteristics (called moderators) that likely drive the heterogeneity among the collected estimates. As is common in empirical studies, an omission bias is likely to “pollute” the MRA coefficients. However, a high number of covariates relative to the number of studies may also lead to misleading results (Thompson and Higgins, 2002). We thus pay attention to striking the right balance between the risk of an omission bias and the risk of a high number of covariates-driven bias while including the set of moderators. More details on the moderators can be found in Appendix 2.

Sample Characteristics

The sample composition is bound to play a key role in any study aimed at assessing the macroeconomic consequences of IT. The empirical literature indeed finds distinct results, depending on the composition of the sample: IT is broadly found to lead to beneficial effects on price stability in developing countries, but to mixed results in advanced economies or when lumping together developed and developing countries. For instance, analyzing the influence of IT in developed and developing countries, respectively, Lin and Ye (2007, 2009) find that IT adoption helps bring down both inflation and its variability in developing countries, but fail to find a statistically significant effect in developed countries. We test for the role of sample composition through two dummy variables: (i) a binary variable taking one if the study is based on a sample of developing countries, zero otherwise; (ii) a binary variable equaling one if the study relies on a mixed sample (pool of developed and developing countries), zero otherwise. Half of the regressions from our meta-dataset rely on a sample of developing countries, while 18 percent of the regressions build on pools of advanced and developing economies. We label this source of heterogeneity as Sample characteristics.

Inflation Targeting Implementation Parameters

Factors related to the implementation forms of IT or, to some extent, to the definition of the counterfactual (control group or comparison group) could also be at work in the heterogeneity found on the impact of IT across studies. We dub this source of heterogeneity as IT characteristics, and account for it through the following two dimensions. First, we distinguish two implementation forms of IT: soft or partial IT versus full-fledged IT, as captured by two starting dates of IT adoption, namely default starting date and conservative starting date, in line with Rose (2007).11 More specifically, we capture the influence of IT implementation forms through a dummy variable equaling one if a collected estimate results from conservative starting dates, zero otherwise. About 37 percent of the regressions from our meta-dataset stem from conservative IT starting dates. Second, we factor in the role of the counterfactual definition, as the latter may weigh significantly on the results. To this end, we include a dummy variable equaling one when a collected estimate is based on benchmarking IT countries against a control group that lumps together alternative monetary policy frameworks (money growth targeting, exchange rate targeting), and zero otherwise (when money growth targeting and exchange rate targeting are not lumped together in the control group). About 91 percent of regressions from our meta-dataset benchmark inflation targeters against a pool of money growth targeters and exchange rate targeters.

Estimation Technique Characteristics

The chosen estimation technique may constitute another source of heterogeneity among the collected estimates. It is commonly agreed that a key difference between estimation techniques lies in their degree of effectiveness in handling endogeneity issues, which in turn determines the extent to which study results carry a dose of bias. Lin and Ye (2007) for instance, point out that unlike simple ordinary least squares (OLS) and difference-in-difference (DD), propensity score-matching (PSM) techniques are more effective in addressing self-selection issues. While acknowledging that the PSM corrects for self-selection, Brito and Bystedt (2010) though argue that the cross-sectional nature of PSM does not allow controlling for time trends, unobservable variables, and persistence, thus pointing to a superiority of the Generalized Methods of Moments (GMM). We account for the role of estimation techniques-driven heterogeneity through the following four dummy variables: (i) a binary variable equaling one if a collected estimate stems from a GMM estimation, zero otherwise; (ii) a binary variable equaling one if a collected estimate results from an instrumental variable (IV) estimation, zero otherwise; (iii) a binary variable equaling one if a collected estimate comes from a PSM estimation, zero otherwise; and (iv) a binary variable equaling one if a collected estimate stems from a DD estimation, zero otherwise. Around 76 percent of collected estimates stem from studies that build on the estimation techniques underlying the above-defined dummies, the remainder from studies that use OLS, fixed or random effects, or time series methods. We label this source of heterogeneity as Estimator characteristics.

Control Variables-related Characteristics

Differences in the specification of covariates might also drive the heterogeneity in estimates across studies, as the chosen vector influences the extent to which the estimate is free of omission bias. We factor in the role of covariates-driven heterogeneity, dubbed as Control variables characteristics, through a dummy variable taking the value one if the collected estimate results from a regression that accounts for the most commonly used control variables, and zero otherwise. The most commonly used covariates include government debt, fiscal balance, trade openness, exchange rate regime, central bank autonomy, financial development (broad money growth, credit to the private sector), economic development, investment, output variability or output gap, population, institution, inflation and its volatility, financial openness, and financial reforms.

Sample Structure Characteristics

Heterogeneity in estimates may reflect heterogeneity in the time span considered for evaluating the IT impact. On the one hand, some critics of IT, including Dueker and Fisher (1996) and Cecchetti and Ehrmann (2000), argue that the alleged performance of IT in the literature might just reflect common trend effects (favorable economic environment of the 1990s, known as the “Great Moderation”). Other critics, including Stiglitz (2008), argue that IT is being put to test by the recent global financial crisis, which could entail its demise in light of the zero-lower-bound, whereby IT central banks would be undershooting their inflation targets. On the other hand, proponents of IT rather stress the prominent role of credible monetary policy frameworks, including notably IT adoption, in anchoring inflation (see, e.g., Bernanke, 2004). With regard to coping with the global financial crisis, Krugman (1999), Gonçalves and Carvalho (2009), and Andersen and others (2015) also point out that IT countries absorbed better the adverse effects of the shocks without jeopardizing monetary policy credibility (through temporary deviations from the target) and thus experienced lower sacrifice ratios, thanks to their more firmly anchored inflation expectations.

To gauge the time-sensitivity of the collected estimates, labelled as Sample structure characteristics, we discriminate among the collected estimates based on the period coverage of the studies from which they are originating. More specifically, we distinguish two major time spans: the Great Moderation (1990s until the recent Great Recession) and the recent Great Recession (post-2007). We introduce a dummy variable taking the value one if the collected study covers both the Great Moderation and the recent Great Recession, and zero otherwise (when the study covers only the Great Moderation). In addition, we test whether the relative number of ITers and non-ITers in a study can be a source of heterogeneity. To this end, we control for the ratio of ITers to non-ITers.

Publication Characteristics

Finally, we factor in “qualitative difference” across studies, dubbed Publication characteristics, in three dimensions. First, we incorporate a dummy variable equaling one if a study is published in a peer-reviewed journal, zero otherwise (working papers, Ph.D. dissertations, or contribution to a book). Second, we account for the RePec impact factor of the outlet at the period of the meta-data construction. Third, we account for the role of U.S. affiliation, a common feature in most MRA, through a dummy equaling one if at least one of the co-authors is based in a U.S. institution.

IV. Results

Let us now turn onto the results. First, we discuss the results of the publication selection bias and genuine effect tests for the five MRA groups of collected estimates, namely Price and Output Stability, State of the Real Economy, External Development, Fiscal Performance and Credibility, and Monetary and Financial Development. Second, we discuss key results related to the role of moderators highlighted above, focusing exclusively on the first two MRA groups.12

A. Publication Selection Bias and Genuine Effect

Publication Selection Bias

The funnel plots below (Figure 2), which exhibit somewhat skewed distributions, point to the likelihood of publication selection biases, notably for studies examining the relationship between IT and the volatility of inflation, real GDP growth, and State of the Real Economy,.13 To test more formally for the presence of publication bias, we estimate equations (2) and (3) above using a multilevel mixed-effect (ME) REML estimator. Table 1 reports the associated results for Price and Output Stability and State of the Real Economy. Columns [1] and [5] present results for Type II publication bias, using the absolute t-statistics values, while columns [2], [3], [4], and [6] depict results for Type I publication bias, considering continuous t-values for the level of inflation, volatility of inflation, volatility of real GDP growth, and the level of real GDP growth as the dependent variable (most commonly outcome variables used in studies on the effects of IT), respectively.

Figure 2.
Figure 2.

Funnel Graphs.

Citation: IMF Working Papers 2017, 213; 10.5089/9781484320754.001.A001

Notes: We plot the estimated coefficient of IT on the corresponding outcome variable on the horizontal axis, and the precision of the estimate (1/standard error) on the vertical axis.
Table 1.

Publication Selection Bias and Genuine Effect Tests

article image
Notes: The Table presents results of publication selection bias and genuine effect tests for the Price and Output Stability, and State of the Real Economy meta-groups. Columns [1] and [5] report the results for each group, using the absolute value of the t-statistic of the collected IT estimate as dependent variable. Columns [2], [3], [4], and [6] present the MRA results for more homogeneous groups (level of inflation, volatility of inflation, volatility of real GDP growth, and level of real GDP growth, respectively), using the t-statistic of the estimate of IT as dependent variable. All estimates are obtained using a multilevel mixed-effects model. Standard errors are reported in parentheses. *** p<0.01, ** p<0.05, * p<0.1.

Let us first focus on Type II publication bias results (Table 1, columns [1] and [5]). The intercept (“constant”) terms in these regressions are positive and highly significant, pointing to the existence of Type II publication bias in each of the two meta-regressions groups (Price and Output Stability as a whole, and State of the Real Economy as a whole).14 This suggests that researchers and reviewers systematically promote statistically significant results, which is in line with most MRA findings (De Long and Lang, 1992; Card and Krueger, 1995; Ashenfelter and Greenstone, 2004; Havranek and Irsova, 2011; Rusnak and others, 2013; and Neves and others, 2016).

To refine the assessment of publication bias and derive the genuine macroeconomic effects of IT (if any), we narrow down our meta-data to sets of more homogeneous groups, consisting of estimates whereby inflation and its volatility, as well as real GDP growth and its volatility, are the outcome variables. Results associated with these more homogenous meta-data are reported in columns [2], [3], [4], and [6] of Table 1, and show that the intercepts in columns [3] and [6] are significant. This suggests the presence of a Type I publication bias (here the dependent variables are the t-statistics value of the collected estimate, as opposed to the absolute value used when considering the synthesized meta-groups as a whole) in studies analyzing the effect of IT on inflation volatility or real GDP growth. In other terms, researchers, editors and reviewers tend to prefer studies that find inflation-stabilizing and growth-enhancing effects associated with IT adoption. Our results also point to a rather “little to modest” selectivity in the IT-inflation volatility or IT-growth literature, as supported by the associated funnel asymmetry test (FAT) values.15 We do not detect any Type I publication bias in the IT-inflation or IT-volatility of growth literature since the intercepts in columns [2] and [4] are statistically insignificant.

The results for Type I and Type II publication biases still hold when we consider only estimates from studies published in peer-reviewed journals, as depicted in Table 2. More importantly, the coefficient for publication bias on real GDP growth (in column [6]) is now about 1.3, suggesting the existence of “substantial” selectivity in published studies on the IT-growth literature.

Table 2.

Publication Selection Bias and Genuine Effect Tests: Published Estimates Only

article image
Notes: The table presents results of the publication selection bias and genuine effect tests for the Price and Output Stability, and State of the Real Economy meta-groups. Columns [1] and [5] report the results for each group, using the absolute value of the t-statistics of the estimate of IT as the dependent variable. Columns [2], [3], [4], and [6] present the MRA results for more homogeneous groups (level of inflation, volatility of inflation, growth volatility, and level of growth, respectively), using the t-statistic of the estimate of IT as the dependent variable. All estimates are obtained using a multilevel mixed-effects model. Standard errors are reported in parentheses. *** p<0.01, ** p<0.05, * p<0.1.

Genuine Effect

Columns [2], [3], and [5] of Table 1 allow going beyond the publication bias, and testing for the existence of genuine effects of IT adoption on inflation level and its volatility, growth level and its volatility.16 The effects of IT after filtering out the publication bias (slope coefficients reported in columns [2] and [4]) suggest a negative effect of IT on the level of inflation and on growth volatility. Put simply, after filtering out the publication bias, IT is still found to reduce inflation and stabilize growth. However, the magnitude of the genuine effects is rather small. We do not uncover, after correcting for publication bias, any effect of IT on the volatility of inflation as well as on the level of growth (columns [3] and [6]). Note, however, that this does not mean that IT has no effect on inflation volatility and on the level of growth, but that the effect depends on several factors, which we discuss in the next section. The beneficial genuine effects of IT adoption on both the level of inflation and the volatility of growth still hold when we consider only estimates from studies published in peer-reviewed journals (see columns [2] and [4] of Table 2). The magnitude of the IT effect on inflation (GDP growth volatility) is smaller (larger) than in Table 1.

To sum up, these results show that the literature on the effects of IT adoption suffers from two types of publication bias: authors, editors, and reviewers tend to prefer studies that (i) find inflation-stabilizing and growth-enhancing effects associated with IT adoption (Type I publication bias), and (ii) promote statistically significant results (Type II publication bias). We also find that, after filtering out the publication selection bias, IT adoption still proves effective in reducing the level of inflation and stabilizing growth. In the following section, we tweak further the analysis by exploring the circumstances (outlined in section 3.2. above) that could explain the heterogeneity across findings in the empirical literature.

B. Drivers of Heterogeneity

We make use of a multilevel mixed-effect restricted maximum likelihood (REML) estimator and a multilevel mixed-effect probit model to address the following questions, respectively: (i) what characteristics affect the significance of the estimates? (ii) What factors explain the likelihood of having a significant coefficient with a particular sign? Put simply, the REML focuses on the drivers of the significance levels of the collected estimates while the probit focuses on both the significance and the economic meaning of the collected estimates. Tables 3 and 4 report the associated results.

Table 3.

Drivers of Heterogeneity, using Mixed-Effect Estimator

article image
Notes: The table presents results of the multivariate meta-regression for the Price and Output Stability, and State of the Real Economy. Columns [1] and [5] report the results for each group using the absolute value of the t-statistic of the collected estimate of IT as dependent variable. Columns [2], [3], [4], and [6] present the MRA results for more homogeneous groups (level of inflation, volatility of inflation, real GDP growth volatility, and level of real GDP growth, respectively), using the t-statistic of the estimate of IT as the dependent variable. All the estimates are obtained using a multilevel mixed-effects model. Standard errors are reported in parentheses. *** p<0.01, ** p<0.05, * p<0.1.
Table 4.

Drivers of Heterogeneity, using Probit Regressions

article image
Notes: The Table presents results of the multivariate meta-regression for the Price and Output Stability, and State of the Real Economy. Columns [1] and [5] report the results for each group. Columns [2], [3], [4,] and [6] present the MRA results for more homogeneous groups (level of inflation, volatility of inflation, real GDP growth volatility, and level of real GDP growth, respectively). All the estimates are obtained using a probit regression. Standard errors are reported in parentheses. *** p<0.01, ** p<0.05, * p<0.1.

a. Multilevel Mixed-Effect Results

Let us first discuss the results from the multilevel mixed-effect estimator, in which the t-value of the collected estimate (or its absolute value17) is the dependent variable.

Mixed-Effect Results on the Price and Output Stability Meta-Group

Columns [1] to [4] of Table 3 depict the results on the Price and Output Stability meta-group.

Role of Sample Composition

The coefficient associated with the Developing (Mixed) countries dummy variable is positive (negative) and statistically significant in column [1] of Table 3. This indicates that compared with studies that build exclusively on a sample of developed countries, studies that rely on a sample of developing countries (or on a pool of developed and developing countries) conclude a more (or less) statistically significant relationship between IT and Price and Output Stability-related variables. Analogously, results reported in column [2] and [3] of Table 3 suggest that studies based exclusively on a sample of developing countries are more likely to conclude in favor of IT effectiveness in bringing down inflation and its variability. This latter finding is in line with most existing studies that point out that IT is more effective in achieving price stability in developing countries (Gonçalves and Salles, 2008; Lin and Ye, 2009; Yamada, 2013). The rationale behind such a finding is that monetary policy credibility has yet to be earned in developing countries, such that a successful implementation of IT helps anchor inflation expectations more firmly and close this credibility gap (Bernanke and others, 1999; and Mishkin, 2000).

In column [4] of Table 3, the dummy variables for the Developing countries sample and Mixed sample exhibit statistically insignificant coefficients. This finding is in line with a few studies that show a lack of systematic difference between developing IT countries and non-IT developing countries in dampening real GDP growth volatility (IMF, 2005; and Gemayel and others, 2011).18

Role of Estimation Techniques

The chosen estimation technique also matters for the effect of IT on variables from the Price and Output Stability meta-group. The MRA indeed shows that studies that employ PSM, GMM, or IV estimation techniques yield more statistically significant results (see column [1], Table 3), while studies that rely on DD estimators lead to less significant estimates. Columns [2] and [3] show that PSM, GMM, or IV-based studies tend more to conclude in favor of IT effectiveness in reducing inflation or its variability while DD-based studies tend more to report inflation-enhancing effects of IT, both in level and variability. For real GDP growth volatility, results (column [4]) point to no significant influence of estimation techniques on the estimates (GMM, DD), except PSM-based studies that are found to more often lead to positive associations between IT and growth volatility.

Role of Control Variables

The MRA results show that the specification of the covariates vector also affects the estimates. On the one hand, controlling for fiscal balance, central bank autonomy, financial development, and GDP per capita in regressions leads to higher significance of the associated estimates, while accounting for government debt or trade openness leads to weaker significance of the associated estimates of IT on variables from the Price and Output Stability meta-group (column [1], Table 3). On the other hand, the significantly negative coefficients associated with fiscal balance, exchange rate regime, and financial development in column [2] of Table 3 suggest that accounting for these variables in regressions tends to lead to larger inflation-reducing effects of IT, while the significantly positive coefficients associated with government debt, trade openness, and central bank autonomy in columns [2] and [3] rather signals that their inclusion in regressions contributes to smaller inflation or inflation volatility-reducing effects associated with IT adoption. Put differently, the effectiveness of IT in bringing down inflation is stronger in countries with higher fiscal balances, greater flexible exchange rate regime and deeper financial systems, but weaker in countries having higher central bank autonomy, plagued with debt overhang, and more open to trade.

Moreover, countries with healthier public finances (higher fiscal balance, or lower debt-to-GDP ratio), and hence freer of fiscal dominance, are less prone to experience unpleasant monetarist arithmetic-type failure of IT to curb inflation, as pressures on the monetary authority to generate seigniorage revenues to meet the present value budget constraint are less likely in such contexts (Sargent and Wallace, 1981; Leeper, 1991; Sims, 1994; and Woodford, 1994). The stronger effectiveness of IT under greater exchange rate flexibility and deeper financial system is also in line with the existing literature, which regards exchange rate flexibility and financial development as key preconditions for a successful implementation of IT, in view of the need to commit to price stability as the overriding goal of monetary policy and of the need to have a well-greased transmission mechanism of monetary policy for a proper IT functioning (Masson and others, 1997; Debelle and others, 1998; Agénor, 2000; Mishkin, 2000; Amato and Gerlach, 2002; Sims, 2004; Bernanke and Woodford, 2004; Batini and Laxton, 2007; and Freedman and Ötker-Robe, 2009).

The limited effectiveness of IT in achieving price stability in a more open economy could be explained by the fact that fixed exchange rates, which are more amenable to exchange rate targeting rather than IT, are a better option for countries that envisage fostering trade integration and are thus more inclined to open their economy (see, e.g., Frankel and Rose, 2002). However, the mitigating effect of greater central bank independence on the price stabilizing property of IT is somewhat puzzling, as central bank independence (“operational” at least) is rather viewed as a key precondition for a successful IT adoption (Mishkin, 2000; Amato and Gerlach, 2002; and Freedman and Ötker-Robe, 2010). A possible explanation might be that the proxies (usually central bank governors’ turnover rates) used in most studies do not actually capture central bank “operational independence,” that is the autonomy to set interest rates to achieve the monetary policy objective, which is actually what matters for the effectiveness of IT. Another explanation could be that a central bank might not meet the operational independence prerequisite, but embarks into a “soft” IT, in a “tie yourself to the mast” strategy aimed at gradually buttressing operational central bank independence, before subsequently switching into a full-fledged IT (Batini and Laxton, 2007; Alpanda and Honig, 2014).19

Regarding real GDP growth volatility, column [4] in Table 3 reports a significantly positive coefficient associated with public debt, and negative with both central bank autonomy and country’s level of development (per capita real GDP). These findings suggest that studies that account for the role of fiscal sustainability (government debt), central bank independence, and the level of development are more likely to conclude in favor of stronger IT effectiveness in stabilizing growth. A corollary is that the growth-stabilizing property of IT is magnified in richer countries, with sound public finances and greater central bank independence.

Role of IT Implementation Forms and Sample Structure

The MRA shows that the implementation forms of IT matter for its impact on price and output stability. More specifically, we find a statistically significant and positive coefficient associated with the Conservative starting dates dummy variable in column [1] of Table 3, which indicates that studies relying on conservative dates of IT tend to find more significant effects of IT on variables from the Price and Output Stability meta-group. Moreover, the coefficient associated with Conservative starting dates dummy is significantly negative in column [2], suggesting that Conservative starting dates-based studies more likely result in stronger IT effectiveness in reducing inflation. Put differently, full-fledged IT implementation delivers larger inflation reductions than soft IT.

Relatedly, our findings show that the structure of the sample used, as captured by the inflation Targeters-to-non-inflation Targeters ratio, matters for empirical investigation. Table 3 indeed reports a statistically significant and positive coefficient associated with the Inflation Targeters-to-non-Inflation Targeters ratio in column [1], which signals that the relative number of IT countries included in the IT comparison group tends to enhance the significance of IT effects on variables from the Price and Output Stability meta-group.

Role of Publication Characteristics.

We do not find a significant difference between published and unpublished studies except for published studies having analyzed the effect of IT on inflation volatility (negative coefficient in column [3] of Table 3). However, studies with at least one co-author with a U.S. affiliation report more significant effects of IT on the Price and Output Stability meta-group (column [1], Table 3). In addition, studies with at least one co-author from the U.S. tend to find a larger inflation or inflation variability-reducing effect of IT (columns [2]-[3], Table 3), but a smaller growth-stabilizing effect of IT (column [4], Table 3). The results also indicate that the higher the impact factor of the journal in which a study has been published, the smaller the inflation-reducing and growth-stabilizing effects associated with IT adoption (columns [2] and [4], respectively).

Mixed-Effect Results on the State of the Real Economy Meta-Group

Let us now look at the mixed-effects results on variables from the State of the Real Economy meta-group (columns [5]-[6], Table 3).

Role of Sample Composition

We do not find a systematic difference between studies that build exclusively on a developing countries sample and those that use a mix of developing and developed countries when it comes to affecting not only the significance, but also the magnitude of the growth effect of IT (columns [5] and [6], respectively, Table 3).

Role of Estimation Techniques

Results in column [5] of Table 3 indicate that using GMM or PSM methods does not make any difference to the significance of the growth effect of IT. However, DD estimator-based studies report less significant IT effects on variables from the State of the Real Economy meta-group. In addition, using PSM leads to larger growth-reducing effects of IT (column [6]).

Role of Control Variables

The specification of the covariates vector matters for the estimated effect of IT on variables from the State of the Real Economy (columns [5] to [6], Table 3). More specifically, controlling for the exchange rate regime, financial development, and investment leads to less significant effects of IT on the State of the Real Economy (column [5]), while controlling for trade openness increases the significance of the effect. When looking particularly at the growth effects of IT (column [6]), it appears that the coefficients associated with exchange rate regime, financial development, and investment are significantly positive, suggesting that IT adoption is more likely to contribute to bolstering growth in countries with flexible exchange rate regimes, deeper financial systems, and higher investment levels. Such findings are in line with the literature that identifies financial development and public investment as key drivers of economic growth (King and Levine, 1993; Domar, 1947). However, the coefficient associated with Central bank autonomy is negative and highly significant, suggesting than the IT-driven disinflation costs are higher in countries with greater central bank independence, in line with Bleich and others (2012), who find that IT adoption drives up central bank aversion to inflation.

Role of IT Characteristics and Sample Structure

We find no role of the implementation forms of IT (as captured by the Conservative starting dates dummy) on its effects (significance and size) on the State of the Real Economy (columns [5] to [6]). In addition, the mixed-effect results indicate that using a sample that covers from the Great Moderation (1990s) to the Great Recession (post-2007) leads to less significant effects on the State of the Real Economy (column [5]) compared to a sample covering only the Great Moderation, though such a difference vanishes once we focus exclusively on the growth effect of IT.

Role of Publication Characteristics

The mixed-effect results show that publication characteristics play a role on the effects of IT on the State of the Real Economy. Studies published in journals lead to more statistically significant IT effects, while those published in high-quality journals (higher impact factor) and co-written with at least one author from a U.S. institution report less statistically significant effects of IT on variables from the State of the Real Economy meta-group (column [5]). However, when singling out the effect on economic growth, we find that studies published in journals tend to show smaller growth costs of IT, while those published in high-quality journals (higher impact factor) and co-written with at least one author from a U.S. institution more likely report growth-enhancing effects of IT.

b. Multilevel Mixed-Effect Probit Results

Table 4 presents results obtained from multilevel mixed-effect probit regressions for the Price and Output Stability meta-group and the State of the Real Economy meta-group. Building on the size and the significance of the collected estimates, the main tenet of the multilevel mixed-effect probit model is to reduce the number of possible outcomes by transforming the collected estimates into a binary variable taking the value 1 for significant beneficial effect associated with IT adoption, and zero otherwise. More specifically, we use as dependent variable, a dummy variable equaling 1 if a primary study reports a statistically significant and beneficial IT effect on variables from the Price and Output Stability meta-group (with a risk error of 10%), and zero otherwise. Regarding variables from the State of the Real Economy meta-group, we rather use as dependent variable, a dummy equaling 1 if a primary study reports a significant IT-driven economic cost, and zero otherwise. The multilevel mixed-effect probit model thus explains, with a set of study characteristics (moderators), the probability that a primary study takes a given value of the binary variable.

Results on the Price and Output Stability Meta-Group

Columns [1] to [4] of Table 4 report the results for the Price and Output Stability meta-group. Consistent with the multilevel mixed-effect REML-based results above, we find that studies that build exclusively on developing country samples increase the probability of finding a significant beneficial effect of IT on variables from the Price and Output Stability meta-group (inflation, inflation volatility, and real GDP growth volatility). But unlike the multilevel mixed-effect-based results, the coefficient associated with studies that build on a pool of developed and developing countries is now positive and statistically significant, though with a smaller magnitude compared with those that build exclusively on a sample of developing countries. This finding confirms that the beneficial effect of IT on price and output stability is stronger in developing countries. As far as the estimation techniques are concerned, the results are in line with the findings from the mixed-effects REML above in that they indicate that employing PSM, GMM, or IV estimators increases the likelihood of finding a beneficial effect of IT on the Price and Output Stability, except for the coefficient associated with DD-based studies that become statistically insignificant. Furthermore, PSM-based studies are more likely to reveal inflation or inflation volatility-reducing effects of IT (columns [2] and [3]) while DD-based papers are less likely to conclude in favor of growth-stabilizing effects of IT (column [4]). These differences of results across estimation techniques underscore, once again, the pivotal role of identification strategies, thus calling for carefully choosing them, with a view to preventing misleading policy recommendations.

The specification of control variables also matters. Regarding the whole Price and Output Stability meta-group, we find that studies that control for government debt, trade openness, exchange rate regime, central bank autonomy, or GDP per capita have lower probability of finding significant beneficial effects of IT, while those that account for fiscal balance or financial development have higher probability of ending up with significant beneficial effects of IT. In other terms, IT stands as a useful tool for macroeconomic stability in fiscally healthier countries (lower debt-to-GDP ratio and higher fiscal balances) and with greater financial development, lower exchange rate flexibility, lower trade openness and lower central bank autonomy (column [1]), consistently with the mixed-effect REML-based findings above. The same applies when we look at the results on inflation, except that the coefficient associated with financial development is no longer significant and the coefficient associated with per capita GDP becomes significantly positive (column [2]). Column [3] shows that controlling for government debt or trade openness reduces the effectiveness of IT in bringing drown inflation volatility, which is consistent with the REML-based findings above. Regarding growth volatility, we find that studies that control for government debt or central bank autonomy (fiscal balance) are more likely to find a significantly positive (negative) effect of IT on growth volatility. These findings may indicate that government recourse to countercyclical fiscal policy in bad times through debt-financed spending outweighs any growth-stabilizing effect of IT, and that the disinflation costs of IT are lower in countries with greater central bank independence.

The implementation forms of IT (as captured by the Conservative starting dates dummy) matter. The coefficient associated with that dummy is significantly positive for Price and Output Stability, as well as for inflation or inflation volatility in isolation, which suggests that fully-fledged IT delivers stronger macroeconomic stability than soft IT, consistent with the mixed-effect REML-based results above. Relatedly, we find the definition of the monetary policy framework against which IT is benchmarked matters. The coefficient associated with the Benchmark policy regime dummy is significantly positive. This indicates that the probability of finding a beneficial effect of IT on price and output stability is higher when the study compares IT to a control group that lumps together any non-IT monetary policy framework (money growth targeting, exchange rate targeting, etc.).

The structure of the sample used in primary studies also matters. The results show that papers that use samples covering from the Great Moderation (1990s) to the Great Recession (post-2007) more likely report significant favorable effects of IT on inflation rate, but not on inflation volatility (as was the case with the mixed-effect REML-based results). Finally, the probit results highlight a significant role of the publication characteristics. Papers published in journals are more likely to report significant inflation or inflation volatility-reducing effects but growth volatility-enhancing effects, while papers with at least one U.S.-based co-author are more likely to report significant favorable IT effects on inflation only (while inflation volatility-reducing effects and growth volatility-enhancing were also found to be more likely with U.S.-based co-authors in the mixed-effect REML-based results).

Results on the State of the Real Economy Meta-Group

Let us discuss now the probit results for the State of the Real Economy meta-group (columns [5] to [6], Table 4). The coefficient associated with the Developing dummy is negative and statistically significant, suggesting that the probability of finding significant IT-driven disinflation costs is lower when the study builds on a sample of developing countries exclusively.20 Such findings are at odds with Brito and Bystedt (2010), who show that the price-stabilizing property of IT comes at the expense of output loss in developing countries.

The estimation techniques also affect the results. When estimates are obtained from PSM or DD, the likelihood of finding IT-driven output costs increases, while it decreases when the estimates stem from GMM-based regressions (though the estimated cost is quantitatively meaningless for the latter). Regarding control variables, most are not statistically significant. However, we find evidence showing that the probability of IT-driven output costs declines in countries with greater exchange rate flexibility and higher public investment. We find no significant evidence supportive of increasing and declining probability of IT-driven output costs in countries with greater central bank autonomy and less developed financial system, respectively. In addition, we find that IT-driven costs on the State of the Real Economy as a whole are more likely in fiscally undisciplined countries (higher public debt and lower fiscal surplus).

The implementation forms of IT also matter, as the coefficient associated with the Conservative starting dates dummy is significantly positive (column [6]), suggesting that the output cost is higher under a fully-fledged IT compared to a soft IT, though the estimated cost is quantitatively meaningless. Consistent with the mixed-effect REML-based results, we also find that the probability of finding adverse IT effects on the real economy, and on growth more particularly, declines when a study builds on a sample that is not overwhelmingly made up of non-ITers. When a study builds on a sample that spans from the Great Moderation (1990s) to the Great Recession (post-2007), the probability of finding adverse IT effects on the real economy declines, and vanishes when focusing exclusively on the level of growth.

Finally, we find evidence supportive of a role played by publication characteristics. Studies published in peer-reviewed journals are less likely to report adverse IT effects on the State of the Real Economy, consistent with the mixed-effect REML-based results. When focusing on growth, we find that studies published in peer-reviewed journals are less likely to feature adverse effects of IT, while those published in top-ranked journals (high impact) more likely report IT-driven growth costs. We find no significant evidence of a role played by U.S.-based affiliation of one of the authors.

c. Robustness Checks

We test the robustness of the results reported in Tables 3 and 4 as follows. First, we assess the role of the nature of data. Specifically, we control for three additional dummy variables equaling 1 for Panel, Cross-sectional, or Annual data, respectively, zero otherwise. Second, we drop very extreme IT estimates, with a view to checking results sensitivity to outliers. Third, we employ an alternative estimation strategy for our baseline model, namely estimating the regressions reported in Table 3 using an empirical Bayesian iterative procedure and a moment estimator, and using a logit model (instead of the probit model) for the regressions reported in Table 4. These alternative specifications do not qualitatively alter our main results. The results are not reported for space purpose, but are available upon request to the authors.

V. Conclusion

This paper provides the first-ever application of a meta-regression analysis (MRA) to the literature on the macroeconomic effects of inflation targeting (IT). It builds on a unique and very broad dataset of 8,059 estimated coefficients of IT from 113 empirical studies. Another key novelty of this paper is that compared with previous MRA studies, it builds on studies that focus on the relationship between IT and several outcome dimensions (as opposed to a MRA based on studies that focus on the relationship between IT and a single outcome variable). The examined dimensions include Price and Output Stability (as seized by the inflation rate and its volatility, and growth volatility), State of the Real Economy (as captured by the real GDP growth rate, sacrifice ratio, disinflation cost, etc.), Fiscal Performance and Credibility, External Development, and Monetary and Financial Development. We uncovered several far-reaching results.

First, we show that the literature on the macroeconomic effects of IT adoption suffers from two types of publication bias: authors, editors, and reviewers tend to (1) prefer results that are consistent with the most commonly held views on the effect of IT on inflation volatility and real GDP growth (Type I bias); and (2) promote results that are significantly different from zero (Type II bias). This is in line with most existing meta-regressions, including De Long and Lang (1992) and Card and Krueger (1995), who suggest that publication bias is an important phenomenon in most areas of economic research. Second, after filtering out the publication biases, we still find meaningful (genuine) effects of IT on both the level of inflation and the volatility of economic growth. However, we do not find a genuine effect of IT on inflation volatility or GDP growth after correcting for the publication biases, which to some extent reflects the fact that the genuine effect of IT is conditional upon several factors. Third, differences across studies regarding the impact of IT are systematically affected by sample and empirical choice characteristics, country-specific factors, IT implementation forms, time coverage of the used sample, and publication characteristics. More particularly, using a sample of developing counties, or conservative starting IT dates as opposed to default starting dates tends to improve the beneficial effect of IT. From a policy perspective, the findings on the drivers of heterogeneity of estimates across studies suggest that IT adoption is likely to bear more fruits (price and output stability) when fully implemented in developing countries. Indeed, monetary policy credibility has yet to be earned in these countries, such that a successful implementation of IT may help anchor inflation expectations more firmly and close the credibility gap. But key preconditions need to be met for IT to yield beneficial effects, including notably sound fiscal positions, deep financial system, and greater exchange rate flexibility.

Settling the Inflation Targeting Debate: Lights from a Meta-Regression Analysis
Author: Hippolyte W. Balima, Eric G. Kilama, and Rene Tapsoba