On the Probabilistic Approach to Fiscal Sustainability: Structural Breaks and Non-Normality

Nathaniel Frank; Eduardo Ley

doi:10.5089/9781589069107.024.A002

Jump to Content Jump to Main Navigation

On the Probabilistic Approach to Fiscal Sustainability: Structural Breaks and Non-Normality

Author: Nathaniel Frank¹ and Mr. Eduardo Ley

¹ 0000000404811396https://isni.org/isni/0000000404811396International Monetary Fund

Publication Date:: 04 Nov 2009

eISBN:: 9781589069107

Language:: English

Keywords:: SP; interest rate shock; debt equation; government balance; debt stabilization; IMF DSF approach; CDOs; Fiscal stance; Debt sustainability analysis; Vector autoregression; Africa

Download PDF (603.6 KB)

This paper modifies several assumptions in the probabilistic approach to fiscal sustainability proposed by Celasun, Debrun, and Ostry (2007). First, we allow for structural breaks in the vector autoregression model for the macroeconomic variables. Second, in the Monte-Carlo simulations, we draw directly from the empirical distribution of the shocks instead of drawing from a normal distribution, thus allowing for asymmetries and thick tails. Third, we circumvent the use of a fiscal reaction function by focusing attention instead on debt-stabilizing balances, to produce more “agnostic” debt projections. The paper illustrates how these methodological modifications have significant impacts on the results for specific country cases.

Abstract

The joint World Bank-International Monetary Fund (IMF) debt-sustainability framework (DSF) produces baseline projections for debt dynamics under different macroeconomic scenarios, and conducts stress tests specifying large changes in the variables involved, keeping all else constant. These shocks are applied, in turn, to interest rates, exchange rates, or growth—without changing the values of the remaining endogenous variables. Celasun, Debrun, and Ostry (2007; CDO henceforth) discuss the limitations of this approach and propose a probabilistic scenario analysis. This paper proposes further modifications to the CDO framework.

Using a stochastic framework, CDO draw constellations of values for the variables driving the debt dynamics. For this purpose, CDO estimate a vector autoregression (VAR) model for the nonfiscal components of the system. Under the assumption of joint normality, they create fan charts for the debt-to-GDP ratio. These fan charts are produced by drawing from the fitted residual multivariate normal distribution while iterating forward the law of motion of the debt-to-GDP ratio—along an estimated fiscal reaction function. Among others, Budina and van Wijnbergen (2009), Tanner and Samake (2008), and Bandiera and others (2007) apply this framework to several country cases, and discuss how it improves on the standard World Bank-IMF DSF.

This paper builds on this work, and explores some methodological modifications to the CDO debt sustainability analysis, motivated by observed empirical regularities of the variables involved. First, in the VAR specification, we allow for structural breaks in the data-generation mechanism, through the application of Markov-Switching models. Second, in the Monte-Carlo simulations, the assumption of normally distributed shocks is relaxed and bootstrapping techniques are used to draw directly from their empirical distributions. This allows for a better risk assessment as it accounts for thicker tails and asymmetries in the debt projections. Third, the estimation of the fiscal reaction function is avoided by using instead the debt-stabilizing balance each period when producing baseline projections.

This specification is not intended to predict fiscal behavior, but rather to serve as an agnostic and sensible reference when assessing debt sustainability. It addresses the question of what would be the predictive density for the debt-to-GDP ratios if (1) the government’s balance at each period were precisely the debt-stabilizing balance while (2) interest rates, GDP growth, and the exchange rate exhibited joint behavior similar to past experience—possibly differentiating between normal (tranquil) and turbulent (nontranquil) times. However, we do not take this as a prediction of fiscal behavior, but rather as a reference. Moreover, we avoid here predicting whether the times ahead for each particular country are likely to be tranquil or nontranquil, and we leave it to the analyst to choose her weights when averaging the two scenarios.

I. Debt Sustainability Framework

Using d for debt and b for the government balance (expressed as percentages of GDP), g for real GDP growth, α^f for the share of foreign-denominated government debt, and β^f as the share of the tradable sector in GDP, the debt dynamics are given by

\begin{matrix} d_{t} = \frac{1 + i_{t}}{(1 + g_{t}) (1 + π_{t})} d_{t - 1} - b_{t} & (1) \end{matrix}

\begin{matrix} i_{t} = α^{h} i_{t}^{h} + α^{f} [i_{t}^{f} + e_{t} (1 + i_{t}^{f})] & (2) \end{matrix}

\begin{matrix} π_{t} = β^{h} π_{t}^{h} + β^{f} [π_{t}^{f} + e_{t} β^{f} (1 + π_{t}^{f})], & (3) \end{matrix}

where e_t is the rate of nominal depreciation of the local currency, i_t is the nominal interest rate, and π_t is GDP deflator (Ley, 2009). Equations (1)–(3) can be combined to obtain the familiar expression driving public-debt dynamics

\begin{array}{l} d_{t} = \frac{1 + i_{t}}{(1 + g_{t}) (1 + π_{t})} d_{t - 1} - b_{t} \\ = \frac{1 + α^{h} i_{t}^{h} + α^{f} [i_{t}^{f} + e_{t} (1 + i_{t}^{f})]}{(1 + g_{t}) {1 + β^{h} π_{t}^{h} + β^{f} [π_{t}^{f} + e_{t} β^{f} (1 + π_{t}^{f})]}} d_{t - 1} - b_{t} . & (4) \end{array}

The World Bank-IMF DSF analyses the behavior of equation (4) under some scenario assumptions for $\begin{matrix} {g_{t}, e_{t}, i_{t}^{f}, i_{t}^{h}, π_{t}} \end{matrix}$ , and conducts stress tests. For the stress tests, large shocks are applied, in turn, to interest rates, exchange rates, or growth—keeping other endogenous variables fixed—and the subsequent behavior of the debt dynamics is projected. Ideally, equation (4) would be embedded in a macroeconomic model suitable for policy analysis, but in practice these are not generally available. CDO discuss the limitations of World Bank-IMF DSF approach and implement a probabilistic scenario analysis that addresses some of its shortcomings. In particular, CDO acknowledge the interdependencies among the macroeconomic variables and draw values from an estimated joint distribution.

The CDO stochastic framework requires an empirical specification for the macroeconomic variables in the right-hand side of equation (4). First, a VAR for the aforementioned input variables, $\begin{matrix} {g_{t}, e_{t}, i_{t}^{f}, i_{t}^{h}} \end{matrix}$ , is estimated. Second, a fiscal reaction function is specified and estimated, yielding the statistical relationship between the primary balances, past levels of debt, and the output gap. Finally, in Monte-Carlo simulations, shocks for the macroeconomic variables are drawn from a joint normal distribution, parameterized using the VAR-estimated covariance matrix. These shocks are in turn used in the forward iteration of a debt equation and the fiscal reaction function. The resulting fan charts thus provide a probability distribution for future debt-to-GDP ratios, which are contrasted against the DSF criteria.

In this paper, we propose methodological modifications addressing three aspects of the empirical formulation, which we think that may be open to improvement.

(1) Structural breaks. Estimation conducted for emerging market economies often includes episodes that feature financial crises—for example, Mexico in 1995, Asia in 1997, Turkey in 2001, and Argentina in 2002. During such periods, significant volatility in interest rate spreads, exchange rates and GDP growth have historically been observed. In this context, estimation of a standard unvarying VAR is problematic, as the assumption of stationarity of the data generating processes cannot be ensured. In this paper we allow for structural breaks, which are identified through a Markov-Switching VAR (SVAR). Further estimation is then conducted over periods exhibiting constancy in the data generating processes, which ensures consistent covariance estimates.
(2) Normality. The assumption of normality with regard to the underlying shock distribution constitutes a severe limitation of the Monte-Carlo analysis in CDO. The empirical behavior of economic variables, especially during period of financial distress, displays asymmetries and thick tails. Consequently, any inferred symmetry of the CDO debt projections may merely be driven by the normality assumption and thus conceals the true degree and shape of risk exposure. We replace the normality assumption by direct sampling from the joint distribution of the shocks. Identification of stable subsamples allows for the application of bootstrapping techniques. Consistent resampling of the data and repeated estimation implies that the resulting residuals converge to their true underlying counterpart. This then allows for the analysis of any non-normality such as asymmetry and for a better specification of the tail behavior of the shocks.
(3) Fiscal reaction function. CDO makes inference with regard to the dynamics between primary balances, lagged debt, and the output gap by estimating a fiscal reaction function, which, we think, is subject to severe model uncertainty. In this paper we avoid the use of a fiscal reaction function to predict the fiscal outturn and instead we specify, in each period, the fiscal balance to be the debt-stabilizing balance. As noted before, this specification is not intended to predict fiscal behavior, but rather to serve as an agnostic and sensible reference when assessing debt sustainability. We are interested in the effect of $\begin{matrix} {g_{t}, e_{t}, i_{t}^{f}, i_{t}^{h}} \end{matrix}$ on the debt-to-GDP ratios, and this way we minimize the role of bad or good (predicted) fiscal behavior—for example, in this context, mounting debt cannot possibly be driven by the (debt-stabilizing) fiscal balance. By drawing repeated shocks from the empirical error distribution, in each period we calculate the implied time-varying debt-stabilizing balances. Alternatively, one could instead do a fiscal-gap analysis specifying in each period the required balance to bring the debt-to-GDP ratio to some desired level by some particular date.

Other methodological differences introduced here include the amendment of the debt equation in order to account for tradables, which are proxied by the export-to-GDP ratio. Furthermore, we do not assume that interest rate shocks affect the entire debt stock, but rather differentiate between its maturity structure such that merely the short-term and a uniformly distributed fraction of the medium to long-term debt is rolled over each period. Finally, with regard to data, our estimation is based on the fiscal panel and the input variables provided by CDO, except for the definition of the interest rate for foreign-denominated debt. Whereas CDO employ the deflated U.S. treasury bill rates as a proxy for borrowing costs in foreign markets, we incorporate interest rate spreads in the form of the Emerging Markets Bond Index, as we believe that this is more appropriate for emerging market countries.

II. Methodological Modifications

Markov-Switching VAR

The SVAR, proposed by Hamilton (1989) and Krolzig (1997), differs from its linear counterpart by allowing the structural coefficients and the covariance matrix to be dependent on an unobserved state variable S_t which is assumed to follow a Markov chain. This flexibility is important in our context, as we may expect, a priori, the joint distribution of the shocks to be nonconstant across our sample period. We estimate the following SVAR for the macroeconomic variables, $\begin{matrix} x_{t} = (g_{t}, e_{t}, i_{t}^{f}, i_{t}^{d}) \end{matrix}$

\begin{array}{l} x_{t} = υ s_{t} + β_{s_{t}}^{1} x_{t - 1} + … + β_{s_{t}}^{q} x_{t - q} + ɛ_{t}, \\ ɛ_{t} \sim i i d (0, Ω_{s_{t}}), S_{t} \in {S_{1}, S_{2}} . & (5) \end{array}

Furthermore, estimation of this model allows for inference with regard to the probabilities of being in the respective states of the world, such that any structural breaks are identified by regime switches. The issue of predicting—going forward outside the sample period—which regime may apply falls outside the scope of this paper. Predicting currency or fiscal crises is tricky—for example, on the prediction of sovereign debt distress, see Manase, Roubini, and Schimmelpfennig (2003).

Bootstrap Draws

As discussed above, this paper replaces the assumption of normally distributed shocks imposed in CDO by instead drawing directly from their empirical joint distribution. To this end we employ nonparametric bootstrapping techniques on the stationary subperiods exhibiting constancy in terms of the data generating processes.

Within the general bootstrapping framework, the data is replicated through resampling and subsequent estimation is repeated using these resulting pseudo series, which exhibit the same statistical properties as the original sample. The parameters of interest are estimated from each bootstrap series, yielding their respective empirical probability distributions. In the context of time-series data, random resampling leads to inconsistent estimators in the case of dependency over time as well as across individual variables (Singh, 1981). In order to account for this, in this paper, replication is conducted using blocks of consecutive observations, as proposed by Künsch (1989). Equal probability of sample selection is ensured by applying the circular block bootstrap by Politis and Romano (1992).

With regard to the selection of block length, there is a trade-off between the approximation of the observed data characteristics and the randomness of the resampling mechanism. As the block length converges to the actual sample size, the statistical properties of the replicated series approach those of the original data, but at the cost of imposing ever increasing sequential restrictions, thus reducing the variability of the sampling process. Optimality is determined by an algorithm proposed by Politis and White (2004) whereby the length is based on the degree of autocorrelation exhibited by the data. In our application, we draw 1,000 pseudo series and sets of the residuals from their empirical distribution—which provides a consistent estimate of the true joint shock distribution.

III. Empirical Results

In the estimation of the SVAR,¹ we assume the existence of two data generating processes. Figure 1 presents evidence of structural breaks in the data, which suggests that estimation over the entire sample period potentially yields inconsistent parameter and covariance estimates, and incorrect debt projections.

In Argentina, a normal or tranquil period between 1993 and 2002 is identified, after which statistically significant regime switches coincide with the observed period of financial instability. As a result, in what follows, we conduct estimation separately for these two distinct states of the world.

With regard to Brazil, significant spillovers are found during the Mexican and Argentinean crises in 1995 and 2002, respectively, between which a subperiod of tranquility is defined, albeit a marginally significant break in 1997.

In South Africa, estimation is carried out across two stable periods, the former ranging from 1980 to 1987, and the latter from 1987 to 2001.

Figure 2 displays the quantiles of the residuals against the quantiles of a normal distribution—if the residuals were drawn from a normal distribution, the scatter would be a straight line. Figure 2 presents strong evidence of non-normality, especially during turbulent times. Moreover, the p-values for all joint normality tests are smaller than 0.001.

Argentina

As outlined above, estimation for Argentina is conducted separately for the two periods, which correspond to times of tranquility and financial distress, respectively. Figure 3 presents the distribution of the residuals from 1,000 bootstrap replications. As noted above, normality of the residuals is rejected (Figure 2).

Two further points of interest are to be noted. First, during the noncrisis and the crisis periods, shocks are drawn from differing distributions, whereby the latter exhibits a greater variance relative to the former (up to a fivefold factor). This finding highlights the problems associated with estimating a VAR across the entire sample period—as in this case shocks would be drawn from a mixture of these two distributions, thus implying misleading debt projections.

Second, there is evidence of asymmetric residuals. During the normal, tranquil times the interest rate distributions exhibit greater upside risk, whereas during the crisis period, the histogram for the exchange-rate movements is characterized by a long left-hand tail. Clearly, the assumption of normality adopted in the existing literature does not allow for the quantification of this asymmetric tail behavior. The imposition of symmetry directly affects the probability distribution of the debt projections and thus leads to inappropriate conclusions with regard to fiscal sustainability.

Figure 4 compares our projections with those of CDO. We represent the second and eighth deciles, whose range covers 60 percent of the probability mass. As it can be seen, the CDO projections present a more optimistic picture than even our normal-times projections. This is driven by the CDO’s assumed prudent fiscal behavior embedded in their estimated fiscal-reaction function. In our projections, in contrast, we specify the debt-stabilizing balance in each period—conditioned on the current values of the macroeconomic variables, which then get shocked going forward. This is a more agnostic approach, allowing us to get a sense of how things might look under minimal fiscal adjustment. Another significant difference with respect to CDO arises by allowing asymmetries in the shock behavior, which is especially important during turbulent times, as shown in Figure 5.

The observed asymmetries and differing variances of the shock distributions are directly translated into the debt projections shown in Figure 4 and 5. During the noncrisis period, the mean debt level remains approximately constant, but there is evidence of upside risk in the form of the skewed probability distribution of future debt levels. This is driven primarily by the positive interest rate shocks illustrated in Figure 3. During the period of financial instability both the average debt level and the variance of the forecasts increase significantly, whereby the former is due to an upward jump in interest rate spreads raising the costs of borrowing. As a conclusion of this country example, we argue that estimation over the entire sample period will, in addition to the aforementioned parameter inconsistency, incur an additional information loss. If the structural breaks are ignored, the resulting debt projections would incorrectly be comprised of a mixture of those represented in Figure 5.

Brazil and South Africa

Figure 6 displays replicated VAR residuals for Brazil. We cannot estimate a SVAR in this case because the normal and turbulent states alternate too often, resulting in subsamples that are too short. In the case of Brazil, the most striking features with regard to asymmetries are the upside risk associated with interest rate changes, in addition to the long left-hand tail in the distribution for exchange rates—indicating greater probability of a currency depreciation. As in the case of Argentina, the assumption of joint normality is rejected. These asymmetries in the data are reflected in the debt projections in Figure 7. Whereas the mean level of future debt remains approximately constant, the box plots illustrate the significant probability mass corresponding to increased debt-to-GDP ratios.

As argued previously, for South Africa there appear to be two subsamples exhibiting constancy in their data generating mechanisms. As a result, estimation is conducted for 1980–87, which is defined as “normal” period, and for 1987–2001, “turbulent” period. This country case differs in terms of interpretation when compared with Argentina, where a distinct episode of financial distress is identified. Figure 8 shows that asymmetries in the VAR residuals are not as pronounced for South Africa, but that the variances differ across the two distributions. This propagates to the debt projections, which exhibit differences in terms of the width of their respective confidence intervals. Also, it is noteworthy, that for both subperiods the debt forecasts are downward sloping, a fact that is driven by low interest and high GDP growth rates. These are unlikely to continue in the future, and we do not take these projections as realistic—we just illustrate what the mechanical application of this framework implies (Figure 9).

IV. Debt-Stabilizing Balances

The final departure from the technique proposed by CDO is that we avoid the use of fiscal reaction functions altogether. Their usage introduces estimation and model uncertainty when attempting to make inference with regard to the statistical relationship between primary balances, lagged debt, and output growth. In CDO, shocks are drawn from the joint distribution of the residuals, which in turn are utilized in the forward iteration of the fiscal reaction function and the debt equation.

Here we invert the problem by specifying the government balance which, under the current values of $\begin{matrix} x_{t} = (g_{t}, e_{t}, i_{t}^{f}, i_{t}^{d}) \end{matrix}$ is consistent debt stabilization and we obtain the implied balances within this stochastic framework. Given the complex form of the debt equation, which accounts for differing maturity structures of debt, a simple inversion is not possible. Thus numerical techniques in the form of a grid search are employed. This fiscal balance is a benchmark for minimal adjustment. This technique has the advantage—in addition to overcoming data constraints affecting the fiscal variables—that it allows for evaluation of currently implemented policies, by contrasting them to the debt-stabilizing ones.

Figure 10 provides both the actual primary balances and those required in order to achieve debt sustainability under the true shock distribution. In Argentina and Brazil governments are currently achieving balances of approximately 5 percentage points of GDP, which is consistent with our notion of fiscal sustainability. As pointed out above, the decreasing debt levels for South Africa are driven by high levels of GDP growth and low interest rates. These factors allow their respective government to incur negative primary balances, whilst not endangering their future fiscal position.

V. Conclusion

This paper proposes several methodological modifications to the probabilistic fiscal sustainability analysis framework developed by Celasun, Debrun, and Ostry (2007).

First, in the VAR specification, we allow for structural breaks in the data-generation mechanism, through the application of Markov-Switching models. This is important when characterizing the statistical behavior of growth, interest and exchange rates for countries that may have undergone crisis or even just turbulent times. Second, in the Monte-Carlo simulations, we replace the assumption of normally distributed shocks by bootstrapping techniques to draw directly from the empirical distributions. This allows for a better risk assessment as it allows for thicker tails and asymmetries in the debt projections. Third, the estimation of the fiscal reaction function is avoided by using instead the debt-stabilizing balance each period when producing baseline projections.

This specification is not intended to predict fiscal behavior, but rather to serve as an agnostic reference when assessing debt sustainability—most of the action in the behavior of the debt-to-GDP ratios will have to come from elsewhere. We look at what would be the predictive density for the debt-to-GDP ratios if the government’s balance at each period were precisely the debt-stabilizing balance conditioned on current values of interest rates, GDP growth, and the exchange rate, while then going forward, shocks to these variables exhibited joint behavior similar to past experience—differentiating between normal (tranquil) and turbulent (nontranquil) times, when needed.

References

Bandiera, Luca, Nina, Budina, Michel, Klijn, and Sweder, van Wijnbergen, 2007, “The ‘How To’ of Fiscal Sustainability: A Technical Manual for Using the Fiscal Sustainability Tool,” World Bank Policy Research Working Paper No. 4170 (Washington, World Bank).
- Search Google Scholar
- Export Citation
Budina, Nina, and Sweder, van Wijnbergen, 2009, “Quantitative Approaches to Fiscal Sustainability Analysis: A Case Study of Turkey Since the Crisis of 2001,” World Bank Economic Review, Vol. 23, No. 1, pp. 119–40.
- Search Google Scholar
- Export Citation
Celasun, Oya, Xavier, Debrun, and Jonathan D. Ostry, 2007, “Primary Surplus Behavior and Risks to Fiscal Sustainability in Emerging Market Countries: A Fan-Chart Approach,” IMF Staff Papers, Vol. 53, No. 3, pp. 401–25.
- Search Google Scholar
- Export Citation
Dimitris, N. Politis, and J. Romano, 1992, “A Circular Block-Resampling Procedure for Stationary Data,” in Exploring the Limits of the Bootstrap, ed. by Raoul LePage and Lynne Billard (New York, John Wiley & Sons).
- Search Google Scholar
- Export Citation
Dimitris, N. Politis, and Halbert, White, 2004, “Automatic Block-Length Selection for the Dependent Bootstrap,” Econometric Reviews, Vol. 23, No. 1, pp. 53–70.
- Search Google Scholar
- Export Citation
Hans-Martin, Krolzig, 1997, Markov Switching Vector Autoregressions: Modeling, Statistical Inference and Application to Business Cycle Analysis, Lecture Notes in Economics and Mathematical Systems No. 454 (Berlin, Springer-Verlag).
- Search Google Scholar
- Export Citation
Hans, R. Künsch, 1989, “The Jackknife and the Bootstrap for General Stationary Observations,” The Annals of Statistics, Vol. 17, No. 3, pp. 1217–41.
- Search Google Scholar
- Export Citation
James, D. Hamilton, 1989, “A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle,” Econometrica, Vol. 57, No. 2, pp. 357–84.
- Search Google Scholar
- Export Citation
Kesar, Singh, 1981, “On the Asymptotic Accuracy of Efron’s Bootstrap,” The Annals of Statistics, Vol. 9, No. 6, pp. 1187–95.
- Search Google Scholar
- Export Citation
Ley, Eduardo, 2009, Fiscal (and External) Sustainability (Washington, World Bank). Available via the Internet: mpra.ub.uni-muenchen.de/13693/.
- Search Google Scholar
- Export Citation
Manase, Paolo, Nouriel, Roubini, and Axel, Schimmelpfennig, 2003, “Predicting Sovereign Debt Crises,” IMF Working Paper 03/221 (Washington, International Monetary Fund).
- Search Google Scholar
- Export Citation
Tanner, Evan, and Issouf, Samake, 2008, “Probabilistic Sustainability of Public Debt: A Vector Autoregression Approach for Brazil, Mexico, and Turkey,” IMF Staff Papers, Vol. 55, No. 1, pp. 149–82.
- Search Google Scholar
- Export Citation

Nathaniel Frank is a researcher at Nuffield College and the Oxford-Man Institute of Quantitative Finance, University of Oxford, United Kingdom. Eduardo Ley is a lead economist in the Economic Policy and Debt Department of the World Bank. The authors thank Nina Budina, Oya Celasun, Xavier Debrun, Tihomir Stucka, and Sweder van Wijnbergen for several useful comments. Thanks as well to the journal’s editor, an anonymous referee, and participants at seminars at the World Bank, and the 2008 Banca d’Italia Conference on Public Finance, for many useful suggestions.

The data used here covers the following periods—Argentina, 1993:Q1–2005:Q2; Brazil, 1995:Q1–2004:Q3; South Africa, 1980:Q1–2004:Q2. The identified breaks are (a) Argentina break, 2001:Q3, and (b) South Africa break, 1986:Q1; crisis end, 2001:Q1.

Other Publishers

Save
Cite
Email this content

Share Link

Copy this link, or click below to email it to a friend
Email this content
or copy the link directly:

https://www.elibrary.imf.org/view/journals/024/2009/004/article-A002-en.xml
The link was not copied. Your current browser may not support copying via this button.

Link copied successfully