Sustainability Assessments - Review of Application and Methodological Refinements
Author: International Monetary Fund
Publication Date:
06 Oct 2003
eISBN:
9781498329453
Language:
English
Keywords:
PP; debt crisis; sustainability assessment; debt dynamics; sensitivity test; current account; real GDP; U.S. dollar; emerging market; real interest rate; sustainability analysis; sustainability framework
Download PDF (2.8 MB)

Sustainability Assessments - Review of Application and Methodological Refinements

Abstract

Sustainability Assessments - Review of Application and Methodological Refinements

I. Introduction

1. In June 2002, as part of the Fund’s efforts at crisis prevention and resolution, the Executive Board endorsed a new framework for assessing the sustainability of countries’ public and external debt. Such assessments underpin the Fund’s policy advice in both program and surveillance contexts. The new framework was intended to bring a greater degree of consistency and discipline to sustainability analyses, including by laying bare the basis on which projections are made and subjecting them systematically to sensitivity tests.1 In endorsing the framework, Directors envisaged that it would be useful in a variety of situations: for countries with moderately high indebtedness, the framework could help identify vulnerabilities far enough in advance that policy corrections can be undertaken. For countries on the brink, or in the midst of a crisis, the framework might be used to examine the plausibility of the debt-stabilizing dynamics articulated in program projections. Finally, in the aftermath of a default, the framework could be used to examine whether alternative structures and levels of restructured debt are consistent with projected outcomes.

2. The framework consists of three main elements. The first is the baseline scenario— the set of macroeconomic projections that form the basis for understandings on a Fund-supported program, or the articulation of the authorities’ intended policies as discussed with the staff in a surveillance context—with the main assumptions and parameters clearly laid out. Second, a series of model-independent sensitivity tests are applied to the baseline scenario, providing a probabilistic upper bound for the debt dynamics under various assumptions regarding policy variables, macroeconomic developments, and costs of financing. Third, while recognizing that debt sustainability assessments inevitably involve judgment, the output of the framework needs to be interpreted in terms of the vulnerability of the country to a crisis.

3. At the time of the Board discussion, it was understood that the approach represented work in progress, and that modifications and enhancements might be introduced to each of these elements in light of experience. Directors thus asked staff to review the application of the framework after one year, and also identified a number of issues warranting further examination. Regarding the baseline, their discussion centered on whether staff projections are realistic or are systematically over-optimistic. On the design and calibration of the sensitivity tests, Directors asked whether the tests adequately capture the dynamics and co-movements of the key variables in a run-up to a crisis and, relatedly, whether the assumed shocks are either too benign or too extreme. On assessing the debt dynamics, Directors requested that staff explore the feasibility of establishing threshold values or “danger zones” for the debt ratio, and of taking account of liquidity or roll-over risk within the current framework. Directors also asked staff to explore ways in which the debt sustainability analysis could be linked more closely to assessments of financial sector vulnerability, in particular, the possible contingent liabilities arising from financial sector restructuring. Finally, Directors requested to be informed about techniques for assessing debt sustainability employed by private sector financial institutions and ratings agencies.

4. This paper takes up these issues, reports on preliminary experience with use of the framework, and proposes some enhancements in light of that experience. Section II reviews the use of the public and external debt sustainability framework in staff reports, which has become routine in Article IV consultations in countries with significant market access and in requests for GRA resources. The overall impression is that, while the framework has been introduced successfully and has probably contributed to more careful analyses of sustainability, debt sustainability assessments are not yet well integrated with the rest of the staff’s analysis of economic developments or with the policy dialogue with national authorities. In part, this is because the framework is still relatively new, but one concern is that the sensitivity tests consider shocks that are too extreme (and thus too improbable) to warrant a policy response; the opposite concern, that the assumed shocks are too benign to adequately capture all the risks to debt sustainability is also, however, potentially relevant.

5. Section III, therefore, turns to the design and calibration of the sensitivity tests using both evidence from previous debt crises and Monte Carlo simulation techniques. The conclusion of this section is that the shocks underlying the sensitivity tests broadly emulate the behavior of the key variables in a run-up to a debt crisis. Moreover, the specified sensitivity tests indeed come close to a 95–99 percent confidence interval around the projected evolution of the debt ratio (so that there is usually less than 5 (or 1) percent probability that the country’s debt will exceed this level). As such, they are appropriately calibrated to provide the intended probabilistic upper bound to the debt ratio.

6. Nevertheless, the sensitivity tests in the current version of the template may be too extreme (and too mechanical) to warrant a policy response: it might make little sense to strive for significantly stronger adjustment (or a debt restructuring) in anticipation of a shock that occurs with only low (less than 5 percent) probability. For this reason, Section IV proposes that the current set of sensitivity tests be supplemented with some additional scenarios that capture less extreme and more probable events. These will be based on historical performance, available information on market expectations of economic developments, or the main risks facing the economy. In particular, in one of these scenarios, country teams would be encouraged to model the main shocks to the economy that could result in a (one-standard deviation) decline of output growth. Since this scenario would model the main risks to the economy, it would provide a natural platform for discussions with the authorities of the key vulnerabilities, including possible contingency measures if the adverse scenarios materialize.

7. Section V examines how contingent liabilities, which have often been an important source of increases in public debt, may be incorporated in the sustainability framework. Section VI turns to the interpretation of the debt levels, and reports some further results on empirical approaches to establishing critical thresholds. Section VII summarizes the proposed enhancements to the framework including publication of debt sustainability assessments. Section VIII presents issues for discussion. An appendix describes the methods employed by investment houses and ratings agencies for assessing debt sustainability.

II. Sustainability Assessments: Experience and Conceptual Issues

A. Experience with Use of Sustainability Templates

8. During its discussion in June 2002, the Board endorsed the use of the sustainability framework for countries with significant market access. Following some initial refinements of the template2, sustainability assessments were introduced progressively in staff reports for requests for use of Fund resources under the GRA and for Article IV consultations.3 These assessments are now routine for countries with significant market access, in practice, this criterion has been interpreted to exclude mostly the PRGF-eligible countries.4 Over the nine months since the Board’s endorsement of the framework, assessments of public debt sustainability have been presented in staff reports for some 45 countries, while external debt sustainability assessments have been presented for some 30 countries; almost 40 percent of all staff reports (for non-PRGF eligible countries) issued during this period have included either the public or external debt sustainability analysis or both (Table 1).

Table 1.

Preparation and Publication of Debt Sustainability Analysis Using Standard Template

article image
Source: Staff estimates.

9. About 60 percent of the standard debt sustainability analyses prepared to date have been in the context of Article IV consultations, while 20 percent have been for stand alone requests for use of Fund resources, and the rest for combined Article IV and UFR reports. Industrialized countries form about 20 percent of the sample, emerging market countries account for 70 percent, and PRGF-eligible countries for 10 percent. The debt sustainability assessments are usually reported in an annex to staff report, though at times they are included in the Selected Issues paper or another stand-alone paper.

10. The core of the debt sustainability assessment is the historical decomposition of the debt dynamics and the baseline scenario, which is projected for a five-year horizon. All of the debt sustainability analyses have used a minimum horizon of five years. In a few cases, typically, though not exclusively, in industrialized countries, longer projection horizons—up to 40 years—have been adopted to capture the implications of demographic dynamics for pension liabilities.

11. Beyond the baseline projection, the sustainability template specifies sensitivity tests to key parameters—including the interest rate, the growth rate of the economy, the GDP deflator, the exchange rate, and the primary (or non-interest current account) balance—as well as a sensitivity test that contrasts the baseline projection to the country’s historical performance. The sensitivity tests, which are specified mechanically in terms of two-standard deviation adverse shocks (one standard deviation for the combined shock) lasting for two years, are intended not as alternative scenarios but rather as probabilistic upper bounds on the evolution of the debt ratio, akin to a “95 percent confidence interval.” The template also suggests the use of relatively long horizons—ten years, if feasible—for the calculation of the relevant averages and standard deviations of the key parameters.

12. Most of the sustainability assessments have followed this methodology. The main departure concerns the horizon over which the standard deviations are calculated, which in some cases is as short as five years. In general, the reasons for doing so are indicated—for example, the country has suffered a hyperinflation or is a transition economy—and a plausible case can be made that past performance is not relevant and that basing the analysis on it would be misleading. In some instances, however, the arguments have been less clear-cut—for example, in one case it was argued that, since the country had moved to a floating exchange rate regime, the past history of high real interest rates was not relevant as there was no longer any need to defend the exchange rate. Some country teams have included additional sensitivity tests; for instance, oil price shocks for countries where the government relies on oil revenues have been examined. Teams have also supplemented the standard sensitivity tests with “scenario” analyses in which two or three scenarios—e.g. base, optimistic, and pessimistic—are depicted.

B. Assessment

13. While it is still early days to assess the performance of the sustainability framework, the more disciplined debt sustainability assessments have been pivotal in program or Article IV discussions in several prominent cases. Application of the template to Argentina’s public debt shows that restoring sustainability will almost surely require a significant reduction in the present value of outstanding obligations. Mexico provides a good example of the effective use of the framework in a surveillance context, and moreover, comments received from the Mexican authorities on an early version of the framework led to some important modifications and improvements in the framework. Lebanon was another example where the application of the framework to analyze the sustainability of public debt dynamics, in light of assumptions on adjustment and privatization receipts, figured prominently in the Article IV consultations.

14. Standardization of the sustainability framework appears to have brought somewhat greater discipline to the assessments of public and external debt sustainability, in particular by making explicit the assumptions upon which the projections are predicated. For instance, except where the country relies mainly on concessional borrowing, there are very few cases in which the projections assume that the growth rate of the economy will exceed the effective real interest rate—even though historically it may have done so (especially on domestic debt, where interest payments can be eroded by inflation).5

15. At the same time, baseline projections probably exhibit at least some bias toward over-optimism. Looking across the sample of sustainability assessments, for instance, it is striking that—except in a couple of cases where the staff is at pains to underscore the unsustainability of current policies—the projections show not only a stabilizing debt ratio by the end of the projection horizon, but nearly always a decrease in the debt ratio relative to the starting point: the median projected decrease of public debt over the five year horizon is about 12 percent of GDP6, while external debt declines by about 17 percent of GDP.7 Without making judgments about the plausibility of individual baseline projections, statistically it seems unlikely that such a broad range of countries would all experience declining debt ratios.

16. Although the track record of the new sustainability framework is too short to establish the realism of the baseline projections, some insight about typical biases in country teams’ projections can be gained by examining the forecasts for debt dynamics underlying the WEO exercise (Box 2, Appendix 1). This analysis suggests that, at a five year horizon, the external debt ratio is under-predicted by about 3.3 percent of GDP across the full sample of the Fund’s membership (industrialized, emerging market, and developing and low-income countries). For upper-middle income countries the bias is 4.2 percent of GDP, while for those with Fund-supported programs, it amounts to more than 7 percent of GDP. To put this in perspective, it is worth recalling that, on average, the baseline projection of the recent debt sustainability assessments project a decline of the external debt ratio of 7 percent of GDP. Baseline projections, of course, are not intended to be unconditional forecasts, being predicated on the agreed policies being fully implemented and on the economy and markets responding appropriately. This means that some bias is probably unavoidable, but it also underscores the importance of adequate stress testing of the baseline projections.8

How Good (or Bad) are WEO Projections of External and Public Debt?

One way to gain some insight into the typical biases in projections undertaken by country teams is to examine the properties of WEO projections of external and public debt. As debt dynamics are not projected directly, but rather depend on underlying variables such as FDI flows, U.S. dollar value of GDP deflator, real GDP growth, as well as revenue and expenditure, errors in projecting any of these variables result in errors in projecting the debt ratio. Biases for these key variables are analyzed in Appendix 1.

Overall, the error in projecting the external debt ratio is in the range of 1 to 1½ percent of GDP per year of the projection horizon. Among countries with Fund-supported programs, the largest errors are for the upper-middle income countries and for the low-income countries.

Decomposing the results shows that the error in projecting the external debt ratio for low-income countries is almost entirely due to the error in projecting the dollar value of GDP, both in turn, because growth and the real exchange rate appreciation is over-estimated projecting the nominal exchange rates (as inflation is likely to be underestimated).

Errors in World Economic Outlook Projections of External Debt (in percent of GDP) 1/

article image

Predicted and actual values transformed by z/(1+z) for z >0, and z/(1-z) for z < 0.

Projections starting in 1995.

IMF-supported program in year being projected.

A similar exercise for assessing errors in projecting public debt can be undertaken, though the available sample is very small. The results indicate that the general government balance is overpredicted by 1–2 percent of GDP, with most of the projection errors coming from higher expenditure rather than revenue shortfalls. At the one-year horizon, general government net debt is underpredicted by about 1 percent of GDP, but almost 5 percent of GDP for countries with Fund-supported programs.

Errors in World Economic Outlook Projections of General Government Net Debt 1/

article image

Predicted and actual values transformed by z/(1+z) for z >0, and z/(1-z) for z < 0.

Projections starting in 1995.

IMF-supported program in year being projected.

1/ In part, this is because nominal revenues tend to decline automatically in a downturn whereas nominal expenditures tend to remain constant. In terms of ratios to GDP, therefore, since there is an upward bias in projecting GDP growth, the error in forecasting the balance will be mostly accounted for by the error in projecting the expenditure ratio.

17. For stress testing, the standard template specifies two-year, two-standard deviation shocks to the main macroeconomic variables (as well as a one-standard deviation combined shock).9 In addition, the public debt template includes a 10 percent of GDP contingent liability shock. Which of the sensitivity tests is the most extreme naturally depends upon the country’s specific circumstances, though across the sample of analyses undertaken to date, it is usually the exchange rate (or combined) shock that results in the largest increase of the public or external debt ratio. On average, the most extreme sensitivity test raises the public debt ratio by 20 percent of GDP relative to the baseline debt ratio (which, on average, is 60 percent of GDP) and the external debt ratio by 30 percent of GDP (against an average baseline projection of 50 percent of GDP).

18. In a few instances, particularly in the context of industrialized countries, some of the standard sensitivity tests may be less relevant and plausible than alternative tests that could be devised. For instance, countries that have adopted the euro may face significantly smaller risk of exchange rate depreciation (vis-à-vis the foreign currencies in which most of their external debt is denominated) than the historical shocks would suggest. Likewise, the template would need to be modified to properly capture the impact of an exchange rate shock in cases in which hedging instruments are being used to mitigate this risk. In many industrialized countries, the major risk to debt sustainability comes from long-term demographic changes, which is not captured within the five-year projection horizon of the standard template.10

19. Overall, the introduction of the new debt sustainability framework has been relatively smooth, and there are some notable cases in which the staff have made good use of the framework, including in discussions with the authorities (Box 2); but, in many, if not most, cases the debt sustainability assessments have not yet become an integral part of the staff’s analysis in the staff report.11 The debt sustainability analysis is typically reported in an annex (or in a separate paper) and, beyond noting the results, the discussion in the main body of the staff report is in many cases quite cursory. With one or two exceptions, it is also apparent that the sustainability analysis did not form a major part of the discussions between the staff and the authorities. Even where the program baseline requires both more ambitious adjustment and a more favorable response than past performance would suggest, the report is typically mute on what has changed in the economy and why the authorities are confident that the baseline scenario is achievable despite the historical experience.

20. While the apparent failure to integrate the sustainabilty analyses into policy discussions partly reflects presentational factors (notably the practice of placing the sustainability analysis in an annex to the main report12), it may also reflect difficulties with interpreting the sensitivity tests and in drawing policy implications from them. As noted above, these tests provide a probabilistic upper bound on the evolution of the debt ratio. Unless the baseline scenario depicts unsustainable debt dynamics (in the sense of a continually increasing debt ratio), the sensitivity tests in most cases do not exhibit unsustainable debt dynamics either—they simply indicate an increase in the debt ratio.13

Presentation of DSA in Staff Reports: An Example

The Staff Report for the 2002 Article IV consultation for Mexico (SM/02/260, 8/15/02, and SM/02/289 Supplement 1, 9/9/02) is a good application of the debt sustainability framework in surveillance context for several reasons. First, the DSA makes extensive use of the framework outlined in SM/02/166. Second, the process of preparing this analysis was comprehensive—it involved extensive discussions of preliminary results with the authorities and incorporation of their comments in a subsequent revision; indeed, their comments led to improvements in the template itself. Third, results of this analysis are integrated into the main text of the Staff Report. Fourth, while concerns regarding market reaction to publication of DSA had been expressed, there does not seem any evidence of adverse effects.

Calibration of Baseline: Reflecting common practice, tests are discussed extensively in a supplement to the Selected Issues paper (SM/02/289, Supplement 1) and a summary presented in a box in the Appendix of the Staff Report (SM/02/260). The Selected Issues supplement presents the factors underlying the baseline scenario. Assumptions regarding growth, revenue, current account, inflows of foreign direct investment, etc., are clearly spelled out, along with staff expectations regarding government’s intended policies.

Sensitivity Tests and Interpretation: Standard sensitivity tests are applied to the baseline scenarios thus outlined. A preliminary version of this DSA resulted in modification of the template to shock the real (rather than nominal) interest rate in one of the sensitivity tests. These standard tests are supplemented with a scenario analysis, whereby a sudden reduction in external capital market access—as in the capital account crisis of the 1990s—is simulated. The resulting potential financing gap is then analyzed. The exercise is clear in expressing the staff’s opinion on the plausibility of different shocks and underlying assumptions regarding authorities’ behavior in response to such shocks. Staff concludes that debt dynamics are robust to most shocks.

Use of Analysis: The analysis is included in a box in an appendix to the Staff Report. Two diagrams— one for external debt and one for gross public sector debt—are presented, with brief discussion of the results. The main text uses this analysis to highlight some of the downside risks that exist despite favorable baseline conditions, explicitly mentioning variables of concern. Gross financing needs in different scenarios are also discussed. These results inform points raised in the staff appraisal as well.

uA01fig01

Mexico: Sovereign Bond Spread

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

Market Sensitivity: Concerns were raised at the time of a possible adverse market reaction to the release of this analysis. A look at the spread (against the US dollar) suggests that any adverse impact was limited although the decline in spreads for other Latin American Countries around the same time makes a precise assessment difficult.

There are then three possibilities:

  • i. One case is that in which the increase in the debt ratio is modest and provides little cause for alarm. In this case, the debt sustainability analysis should not have to be a central part of the staff’s analysis, although there may remain concerns that the shocks assumed in the standard framework are too benign.

  • ii. A second case is that in which the results of (at least some of) the sensitivity tests are so extreme that they are dismissed as being implausible. In such instances, there is a temptation to see whether tinkering with the parameter values (for instance, by excluding periods of extreme inflation or the early transition years) yields more “reasonable” results. While preferable to simply ignoring the results, choosing an appropriate period over which to calculate the relevant parameters may nevertheless require fine judgments about whether the economy has undergone a fundamental shift in performance or whether it is still subject to large shocks.14

  • iii. The third case is that in which, although the debt dynamics appear highly sensitive to the standard shocks, given the history and characteristics of the economy, there is general consensus that such outcomes cannot be ruled out. Even in this case, however, the policy implications are unclear. If the adverse shocks materialize, the country will need to undertake additional adjustment or seek a reduction in the present value of its obligations. Yet it may make little sense to undertake significantly greater adjustment, or to seek a debt restructuring, in anticipation of a shock which could raise the debt ratio by some 20–30 percent of GDP, but which may only occur with less than 5 percent probability.

21. Conversely, a separate set of concerns, expressed particularly by Executive Directors during their initial discussion of the framework, is that the sensitivity tests may not be sufficiently extreme—and the assumed shocks may be too benign—to capture adequately all of the risks to debt sustainability. These competing concerns suggest that the design and calibration of the sensitivity tests warrant further examination (section III, below).

C. Summary

22. To summarize, following Board endorsement in June 2002, debt sustainability assessments have been progressively introduced in staff reports for use of Fund resources in the GRA, and for Article IV consultations with countries with significant market access. Although the framework has brought greater discipline to the process of making projections, baseline projections probably continue to exhibit at least some bias towards over-optimism. Moreover, with some exceptions, sustainability assessments have generally not been at the center of policy discussions between staff and national authorities. This may be because the sensitivity tests are considered too extreme to be realistic or, even if realistic, too extreme to warrant a policy response. Conversely, there remain concerns that the assumed shocks are too benign. Finally, from a presentational standpoint, debt sustainability assessments would have greater impact if they were integrated in the body of the staff report instead of being relegated to an annex.

III. Calibration of Sensitivity Tests

23. A major element of the sustainability framework is the systematic testing of the debt dynamics to various assumptions regarding policy variables, macroeconomic developments, and costs of financing. An important question, therefore, is whether these standard tests are calibrated appropriately, capturing the main risks to debt sustainability without being extreme to the point of irrelevance. This section examines the realism of the sensitivity tests in two ways: (i) by comparing the assumed shocks in the sustainability template to the behavior of the main macroeconomic variables in the run-up to previous sovereign debt crises, and (ii) by computing the probability that the debt outcome will be worse than the upper bound implied by the sensitivity tests.

A. Event Study Approach

24. One way to gauge the realism of the shocks in the standard template is to examine the behavior of the underlying variables in the run-up to previous debt crises—for instance, by how many standard deviations does the GDP growth rate fall (relative to its pre-crisis mean) in the last two years before the crisis.15 Figure 1 depicts the evolution of the four key variables of external debt dynamics—real GDP growth, interest rate, GDP deflator growth (in U.S. dollar terms) and primary current account deficit—over the ten year period prior to a debt crisis (year T is the year of the crisis) for a sample of 26 debt crises (that occurred in 24 countries) with market access during the period 1970–2001. (For each variable, the median across the sample of 26 episodes is shown.)16

Figure 1.
Figure 1.

Pre-crisis Trends in Key Variables (Sample Medians) 1/

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

1/ Crisis episodes covered in the sample include Algeria (1991), Argentina (1982, 2001), Bolivia (1980), Brazil (1983), Chile (1983), Costa Rica (1981), Dominican Republic (1981), Ecuador (1982), Egypt (1984), El Salvador (1981), Guatemala (1986), Indonesia (1998), Jordan (1989), Mexico (1982), Morocco (1983), Nigeria (1982), Pakistan (1998), Panama (1983), Paraguay (1986), Philippines (1983), South Africa (1985), Trinidad and Tobago (1988), Uruguay (1983) and Venezuela (1983,1995).

25. In the last two years, prior to the crisis, real GDP growth falls by about 4 percentage points, relative to its average during the eight pre-crisis years, a little more than one standard deviation17. Interest rates rise by about 2 percentage points (one standard deviation), while the growth of the US dollar value of the GDP deflator falls by about 10 percentage points (representing one standard deviation). Together, these shocks all tend to increase the external debt ratio prior to the crisis. The non-interest current account deficit, however, moves in the opposite direction, narrowing with the fall in imports as growth declines. Formal Chow tests indicate that for growth, the interest rate, and the GDP deflator, there is indeed a break equivalent to about one standard deviation which occurs between one and three years prior to the crisis.

26. The results suggest that the two-year, two-standard deviation shocks to individual variables are ample for capturing the shocks than have typically been observed in the run-up to a crisis. However, the results also suggest that such shocks do not happen in isolation but rather occur simultaneously. The two-year one-standard deviation combined shock in the template is thus very much consistent with the historical evidence on debt crises. Moreover, applying the sustainability template retrospectively to four recent capital account crises (see Assessing Sustainability, SM/02/166, Tables 5 and 6 for external debt and Box 3 for public debt) also suggests that the confidence bounds are appropriate to capturing the increase in the debt ratio in the run-up to the crisis without being too extreme.

Retrospective Debt Sustainability Analysis in Four Capital Account Crises

The sustainability framework was applied to retrospectively to crises in Argentina (1999), Brazil (1998), Mexico (1995), and Turkey (1999), to see whether actual outcomes fall within the stress test range (see Managing Financial Crises, Occasional Paper 217). Because it is based on IMF program projections, the framework can most usefully be applied in this way to Argentina and Turkey, where programs had been formulated well ahead of crises in 2001. In Brazil, the program was formulated after the 1998 crisis had hit, while in Mexico the 1995 crisis followed shortly after the program was formulated. As the figure shows, even though program fiscal targets were in most cases complied with, public debt generally exceeded baseline program projections. The stress test ranges, however, broadly capture most of the outcomes. More specifically:

  • The 2001 debt ratio in Argentina turned out to be slightly above the upper bound of the stress test range. The deviation from the baseline program projection mainly reflected unanticipated recession and weaker-than-programmed fiscal adjustment. If the projection had been extended to 2002, the debt ratio would be shown to have moved far beyond the upper bound of the stress test range, in part owing to the unanticipated securitization of contingent liabilities, but mainly because of a larger-than-expected real exchange rate depreciation.

  • For Turkey, the outcome for 2001 was well above the upper bound of the stress test range, despite the beneficial impact of stronger fiscal adjustment and lower real interest rates than expected. The deviation from the baseline program projection was not only the result of a much larger real depreciation and sharper recession than anticipated; in addition, the securitization of contingent liabilities added unexpectedly and significantly to the debt.

  • The post-crisis debt ratio in Brazil was at the upper end of the stress test range, despite a better-than-expected post-crisis fiscal adjustment and lower-than-expected real interest rates. An unanticipated large real depreciation was the main contributor.

  • In Mexico, the outturn in 1995 was above the one-year-ahead program baseline projection, because of a larger real depreciation, higher real interest rates, and slower growth than anticipated, and because of the securitization of contingent and unfunded liabilities. Nevertheless, the outturn was within the stress test range. However, by 1997 the debt ratio had moved into the middle of the stress test range because initial fiscal adjustment was not maintained as planned.

uA01fig02

Retrospective Debt Sustainability Analysis

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

Table 2.

Examples of Contingent Liabilities

article image
Source: Currie and Velandia (2002), Polackova (1999).
Table 3.

Estimated Cost of Banking Crises Across Selected Financial System Indicators

(in percent of GDP, unless otherwise indicated)

article image

As measured by the ratio of banking system credit to the private sector as a ratio of GDP.

As measured by the percentage point increase in the ratio of banking system credit to GDP.

Sources: IFS, Figure 4, and staff calculations.
Table 4.

Country: External Debt Sustainability Framework, 1998–2008 (In percent of GDP, unless otherwise indicated)

article image

Derived as [r – g – ρ(l+g) + εα(l+r)]/(l+g+ρ+gρ) times previous period debt stock, with r = nominal effective interest rate on external debt; ρ = change in domestic GDP deflator in US dollar terms, g = real GDP growth rate, e = nominal appreciation (increase in dollar value of domestic currency), and a = share of domestic-currency denominated debt in total external debt.

The contribution from price and exchange rate changes is defined as [-ρ(l+g) + εα(l+r)]/(l+g+ρ+gρ) times previous period debt stock, ρ increases with an appreciating domestic currency (ε > 0) and rising inflation (based on GDP deflator).

Defined as current account deficit, plus amortization on medium- and long-term debt, plus short-term debt at end of previous period.

The key variables include real GDP growth; nominal interest rate; dollar deflator growth; and both non-interest current account and non-debt inflows in percent of GDP.

The implied change in other key variables under this scenario is discussed in the text.

Long-run, constant balance that stabilizes the debt ratio assuming that key variables (real GDP growth, nominal interest rate, dollar deflator growth, and both non-interest current account and non-debt inflows in percent of GDP) remain at their levels of the last projection year.

Table 5.

Country: Public Sector Debt Sustainability Framework, 1998–2008 (In percent of GDP, unless otherwise indicated)

article image

Indicate coverage of public sector, e.g., general government or nonfinancial public sector. Also whether net or gross debt is used.

Derived as [(r -π(l+g) – g + αε(l+r)]/(l+g+π+gπ)) times previous period debt ratio, withr= interest rate; π = growth rate of GDP deflator; g = real GDP growth rate; α = share of foreign-currency denominated debt; and ε = nominal exchange rate depreciation (measured by increase in local currency value of U.S. dollar).

The real interest rate contribution is derived from the denominator in footnote 2/ as r – π (1+g) and the real growth contribution as -g.

The exchange rate contribution is derived from the numerator in footnote 2/ as ag(l+r).

Defined as public sector deficit, plus amortization of medium and long-term public sector debt, plus short-term debt at end of previous period.

Derived as nominal interest expenditure divided by previous period debt stock.

The key variables include real GDP growth; real interest rate; and primary balance in percent of GDP.

The implied change in other key variables under this scenario is discussed in the text.

Real depreciation is defined as nominal depreciation (measured by percentage fall in dollar value of local currency) minus domestic inflation (based on GDP deflator).

Long-run, constant balance that stabilizes the debt ratio assuming that key variables (real GDP growth, real interest rate, and primary balance in percent of GDP) remain at their levels of the last projection year.

27. An important caveat to this approach, however, is the implied sample selection bias. Simply put, Figure 1 shows that, conditional on a debt crisis occurring (in the following year), the underlying variables move by about one standard deviation in the two years prior to the crisis. But it says nothing about the likelihood of such shocks (or a crisis) occurring. The results reported in Assessing Sustainability (SM/02/166, Box 5) imply that the likelihood of a one standard deviation shock to any of the underlying variables is approximately 15 percent (so that, assuming serial independence, the likelihood of a two-year sequence of such shocks ranges is about 2 percent). Allowing explicitly for serial correlation in the underlying variables, and for correlations of shocks across them, however, requires a stochastic simulation approach.

B. Stochastic Simulation Approach

Probability Density Function of Debt Outcomes

28. Instead of applying individual shocks to the baseline scenario and checking the sensitivity of the debt dynamics (as is done in the current sustainability framework), an alternative approach would be to compute the explicit probability density function of the possible outcomes of the debt ratio using stochastic simulation.18

29. Figure 2 shows a typical probability distribution function taken from a stochastic simulation exercise for 41 emerging market countries that are either listed in the EMBI+ index or subject to S&P sovereign ratings, using data over the period 1980–2000.19 In the particular example depicted, the baseline scenario (created for the purposes of the simulation by setting all variables to their historical averages) projects an increase in the debt ratio from 18 percent of GDP to 27 percent of GDP. In the initial period, debt outcomes are distributed tightly around the mean. As the projection horizon lengthens, uncertainty increases so that the distribution becomes more fat-tailed and extreme outcomes cannot be ruled out.

Figure 2:
Figure 2:

Stochastic Simulation of Debt Ratio

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

30. This approach has the advantage of providing an explicit “fan chart” of possible debt outcomes, weighted by the likelihood of their occurrence. But it also has drawbacks relative to the approach of the sustainability framework (which simply traces the effects on the debt dynamics of standard shocks to the main macroeconomic parameters). Beyond the computational burden, the main drawback arises from the lack of sufficient data or stable time series for many countries. Indeed, because of structural breaks such as transition from planned economy or extreme events, including hyperinflation, there is often barely enough data to compute even the requisite means and variances, let alone estimate (rather than impose) the underlying probability distribution.20 Moreover, the technique is somewhat of a “black box,” and there is a risk that the results might be given greater weight than is warranted by the quality of the underlying data.21 For these reasons, stochastic simulations are not being proposed as the mainstay of the standard sustainability template at this juncture. Nonetheless, the approach shows some promise and merits further experimentation in its application to individual countries, at least for countries where sufficient data are available and for which debt dynamics are of particular concern.22

Calibration of Sensitivity Tests in Standard Template

31. Regardless of whether the stochastic simulations form part of the sustainability analysis, the results of the simulation exercise can be used to gauge whether the sensitivity tests incorporated in the standard template are either too extreme or too benign to capture adequately the possible debt dynamics.

32. In particular, it is possible to use the probability density function (obtained via the stochastic simulations above) to calculate the probability that the actual debt ratio, d, will exceed the upper bound implied by the (most extreme) sensitivity test, ds1. If this probability—which is sometimes called the p-value, ps1=pr(d>ds1) —is very low, then the sensitivity test may be considered extreme in that there is little chance of the actual debt ratio ever reaching the upper bound. Conversely, if the p-value is too high, then the sensitivity tests in the standard template may be too benign in the sense that the actual debt ratio could easily exceed the supposed probabilistic upper bound. A natural benchmark against which to gauge the sensitivity tests is the 95 (or 99) percent confidence interval—that is, the debt ratio, d95u, whose p -value is 5% (see Figure 3(a)). The difference between the debt ratio implied by the sensitivity test and the 95 percent confidence interval, D1≡ds1−d95u>(<)0 provides a metric for how extreme (or benign) the standard sensitivity test should be considered (Figure 3(b) and 3(c)). 23

Figure 3.
Figure 3.

Statistical Bounds and P -values

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

Source: Staff estimates.

33. One caveat to the calibration exercise concerns the assumed baseline. For the calibration exercise, the baseline projection is created by setting all variables to their historical averages. To the extent that actual baseline projections are often over-optimistic (see Appendix I), the results of the stochastic simulation are likely to be biased towards finding that the shocks in the standard template are too extreme and the associated p -value too small (Figure 3(d)).

34. The simulations yields a positive value for D1 for 34 out of 41 countries in the sample, with a median value of 15 percent of GDP. In other words, the debt ratio implied by the most extreme sensitivity test is some 15 percent of GDP greater than the 95 percent confidence bound implied by the simulated probability distribution. Even allowing for some bias in the baseline scenario (for instance, the analysis in Appendix I indicates a bias of about 7 percent of GDP for the projection of upper-middle income countries’ external debt ratio), the stochastic simulations suggest that the most extreme sensitivity test in the template is indeed quite extreme.24 This can also be seen by calculating the p -value associated with the most extreme sensitivity test. Figure 4 graphs the frequency distribution (across the sample of 41 countries) of the implied p-values. In almost three-quarters of the countries, the p -value associated with the most extreme sensitivity test is one percent or lower.

Figure 4.
Figure 4.

Frequency Distributions of P -values of First, Second and Third Most Extreme Sensitivity Tests 1/

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

1/ Crisis episodes covered in the sample include Algeria (1991), Argentina (1982, 2001), Bolivia (1980), Brazil (1983), Chile (1983), Costa Rica (1981), Dominican Republic (1981), Ecuador (1982), Egypt (1984), El Salvador (1981), Guatemala (1986), Indonesia (1998), Jordan (1989), Mexico (1982), Morocco (1983), Nigeria (1982), Pakistan (1998), Panama (1983), Paraguay (1986), Philippines (1983), South Africa (1985), Trinidad and Tobago (1988), Uruguay (1983) and Venezuela (1983,1995).

35. An analogous analysis can be undertaken for the second (and third) most extreme sensitivity tests in the standard template. For the second most extreme test, in three quarters of the countries, the p -value is less than 5 percent (while countries for which the p -value is less than 1 percent account for about 40 percent of the sample). For the third most extreme test, in three-quarters of the countries, the p -value is less than 20 percent (while countries for which the p-values is less than 1 percent account for about 30 percent of the sample).

C. Summary

36. To sum up, the shocks assumed in the stress tests of the current sustainability template are realistic inasmuch as they mimic the movements of growth, interest rates, and the U.S. dollar value of the GDP deflator that have been observed in the run-up to previous debt crises. In terms of providing probabilistic upper bounds to the evolution of the debt ratio, the most extreme sensitivity test typically provides an upper bound that is likely to be exceeded with only very low probability (less than 1 percent)—although, taking account of the likely bias in the baseline projection, the true probability should be greater. The second-most extreme sensitivity test provides an upper bound that is likely to be exceeded with a probability of about 1 to 5 percent (so that the upper bound represents a 95- to 99-percent confidence interval). Overall, the analysis suggests that the sensitivity tests are calibrated appropriately, in terms of both the magnitude of the shocks and the probabilities associated with the debt outcomes that result from the assumed shocks (especially, the second-most extreme test). Finally, stochastic simulation analysis while still experimental, shows some promise for calculating explicit probability density functions of the debt dynamics.

IV. Scenario Analysis

37. As argued above, the sensitivity tests are calibrated appropriately to provide a probabilistic upper bound on the evolution of the debt ratio. Nevertheless, they may be too extreme to warrant a policy response—inasmuch as it makes little sense to undertake significantly greater adjustment, or to seek a debt restructuring which could be highly disruptive, in anticipation of a shock that occurs with less than 5 percent probability.

38. This suggests that it may be useful to supplement the current set of sensitivity tests with some additional scenarios that consider less extreme—and thus more probable—shocks to the key parameters. It bears emphasizing that the proposed scenarios would supplement, not supplant, the current set of sensitivity tests. Indeed, it is useful to make a clear distinction between them. The current sensitivity tests are intended to provide a probabilistic upper bound on the evolution of the debt ratio (so that outturns worse than these bounds should be extremely rare). They were introduced, in part, to give greater credibility to the projections undertaken at the Fund, especially in light of cases in which outturns were persistently worse than the “bad case scenario” depicted in earlier sustainability analyses. These tests are intended to underscore the potential risks to sustainability, and in a program context, demonstrate that the Board is cognizant of these risks in approving the program. Moreover, the analysis in the staff report might consider what are the risks of reaching this upper bound under existing and prospective economic conditions and policies, and what particular configurations of shock in the specific country context could result in such outcomes.

39. The proposed new scenario analysis would depict one or two additional scenarios to the baseline, each of which would have a relatively high (20–30 percent) likelihood of being realized. The discussion could then focus on whether—under the primary balance projected in the baseline—there would still be an adequate decrease in the debt ratio, as well as the possible contingency measures that the authorities intend to take if the adverse scenarios materialize. Specifically, the scenario would assume a configuration of shocks that results in an (approximately) one-year, one-standard deviation decline in the growth rate of output.25 Instead of applying this shock mechanically, however, country teams will be encouraged to model the reasons behind this shock and the response of the economy (in terms of interest rates, the exchange rate, and the primary balance) to trace out the resulting debt dynamics.26 In particular, the scenario should be internally consistent.27 For instance, the growth slowdown might be accompanied by a widening primary budget deficit but a narrowing current account deficit. To the extent that the exchange rate is assumed to depreciate (because of lower money demand as activity weakens), the domestic currency value of the existing external debt stock will increase, but this will be partially offset by the likely improvement in the current account balance. If confidence is eroded because of weakening activity, interest rates may be expected to increase, worsening the debt dynamics. Rising debt ratios themselves may elicit widening risk premia and higher interest rates.28

40. An alternative would be to allow country teams even greater latitude in devising the country-specific scenario. For instance, it could be calibrated to result in an increase in the debt ratio that is half way between the baseline and the most extreme sensitivity test, with the team modeling the combination of events that would be most likely to result in this outcome. This approach would have the benefit of directing the analysis to the weakest elements of economic policy (and the most relevant exogenous risks). By doing so, the policy debate would naturally turn to what can be done about these risks.

41. Since the scenario (in either formulation) would be country-specific, it is not possible to calculate precisely the likelihood of the associated shocks. As reported in Assessing Sustainability (SM/02/166), however, the probability that the growth rate falls by one standard deviation is approximately 13–15 percent. Stochastic simulations (along the lines described above) likewise yield a p-value of the debt outcome under a one-standard deviation shock of about 15–20 percent (or higher) for most countries.29 This suggests that such a scenario can occur with sufficiently high probability that it would be prudent to consider possible policy responses. Articulating such a consistent scenario, tailored to the country’s specific circumstances, should provide a good basis for discussions with the authorities on the main risks to the economy and to the public and external debt dynamics. The staff report would thus be expected to discuss the authorities’ intended response should the adverse scenario materialize. This would also help integrate the debt sustainability assessments with the rest of the staff’s analysis and, to the extent that the authorities participate in defining the scenario, help increase both their ownership and technical understanding.

42. Another issue concerns the policy assumptions underlying the existing configuration of baseline and stress tests. The existing “historical case” scenario, in which all of the underlying variables, (including the primary or non-interest balance) assume their historical values, by construction; ignores any factors that may have subsequently led to a deterioration of the debt dynamics, and any measures that may have been taken since the historical period. In contrast, the baseline scenario may incorporate the measures required to avoid a deterioration of the debt dynamics, including not only those already implemented but also those whose enactment or implementation is only anticipated. In such instances, neither the baseline scenario nor the historical performance scenario captures the debt dynamics that would prevail if the required measures were not adopted. It may therefore be useful to examine an additional scenario depicting explicitly the consequences of “no policy changes”—that is, excluding the revenue effects of unenacted changes to tax rates or bases and under the assumption that all currently mandated obligations are honored. A comparison of this “no policy change” scenario to the baseline projection would give an indication both of the expected gain from programmed adjustment and the ambitiousness (or realism) of the baseline.

43. Finally, the template does not take advantage of information on market expectations to assess the plausibility of the macroeconomic assumptions. For financial variables (exchange rates and interest rates), the forward or futures values can be used, while for growth and inflation, consensus forecasts may be available. Such indicators, of course, do not exist for many countries, but where available it would be useful to incorporate them as an alternative scenario, accompanied by an explanation of any differences between this scenario and the baseline.30

V. Contingent Liabilities

44. Experience in recent crises suggests that contingent liabilities are often an important source of increases in public indebtedness.31 Accordingly, one of the sensitivity tests in the sustainability template examines the effect on the public debt dynamics of a realization of contingent liabilities, specified as an exogenous increase in the debt ratio of 10 percent of GDP. This specification is not linked to any estimate of the magnitude of such contingent liabilities. Moreover, no explicit consideration is given to the factors affecting the likelihood of such liabilities being realized, included the country’s financial sector vulnerabilities and other shocks examined in the template (e.g. to growth, interest rates, or the exchange rate). This section explores possible refinements to the contingent liability sensitivity test.

45. By their nature, contingent liabilities are difficult to measure, taking a wide variety of forms (Table 2). When estimates of possible contingent liabilities are available, these may be incorporated directly into the public debt sustainability framework.32 In the absence of such information, country teams should nonetheless examine those that may potentially be important. In most cases, the financial sector constitutes the most important source of contingent liabilities on the government, which often backs deposits and other liabilities of ailing banks through explicit deposit insurance schemes, or through implicit guarantees on liabilities in an attempt to limit the loss of confidence during periods of financial turbulence (Claessens and Klingebiel (2002)). Governments also provide liquidity support, assume bad loans, and inject equity capital in the event of financial sector distress, with obvious implications for the sustainability of public debt,33 while support in the form of regulatory forbearance can lead to larger fiscal costs in the future (Karacadag and Manzer (1997)).

A. Stress-Test Approach

46. One approach to estimating potential contingent liabilities would be to stress test individual bank balance sheets (for the entire or at least most of the banking sector) against a variety of risk factors, as outlined in Honohan (1999). After assessing the extent to which the components of the balance sheet are affected by a variety of risk factors (such as changes in the exchange rate, property market values, general economic downturn, and other shocks), capital positions of various banks and the system-wide capital deficiency could be computed under alternative sizes of shocks and associated probabilities, along with an estimate of the fiscal cost of bank recapitalization.

47. Such an approach would be in line with the stress testing exercise undertaken in the context of Financial Sector Assessment Programs (FSAP). The findings of the FSAP missions could thus provide valuable input into the debt sustainability framework in assessing the likelihood of emergence of any potential financial sector problems, as well as the associated cost of fiscal contingency arising from these problems. However, given the voluntary nature of the FSAP, there will be many instances in which an FSAP has not recently been undertaken for the country or the stress-test results are otherwise unavailable for reasons of confidentiality. In these cases, it may nevertheless be possible to “piggy-back” on current efforts at enhancing the continuous monitoring of the financial sector to obtain an estimate of the possible contingent liabilities.34 For instance, in the context of Article IV missions (or through small, dedicated parallel missions), especially in countries that are identified as being particularly vulnerable, it may be possible to stress test individual bank balance sheets to obtain a rough estimate of the projected capital shortfall (and hence the potential call on the public purse) in the face of various shocks to economic activity, interest rates, or the exchange rate. Indeed, such stress-testing exercises have already been undertaken in several countries during Article IV missions.35 Since the feasibility of the exercise depends upon the availability of individual bank balance sheet data, in cases in which there are confidentiality concerns with the provision of such information, the authorities could be urged to undertake the stress-tests themselves, and inform country teams of the aggregate results and the implied contingent liability for use in the debt sustainability analysis.36

B. Cross-Country Evidence

48. The discussion above outlines one possible approach to refining the estimate of the contingent liabilities emanating from the financial sector. Given resource and other constraints, however, such an approach will not always be feasible or even desirable. In the absence of such information, cross-country evidence on past banking system crises can be used to derive at least crude estimates of the possible contingent liabilities in particular countries.

49. Across a sample of almost 60 crises, banking crises have on average added some 14 percent of GDP to government debt, while three quarters of the countries in the sample spent up to 20 percent of GDP (Figure 5).37 Costs tended to be higher in developing or emerging market countries, especially when the banking crisis was accompanied by a currency crash.38

Figure 5.
Figure 5.

Estimated Cost of Banking Crises 1/

(in percent of GDP)

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

Source: Caprio and Klingebiel (2003), Dziobek and Pazarbasioglu (1997), Honohan and Klingebiel (2000), IMF (2003), Zoli (2001), and staff calculations.1/ The dates in parentheses for each country indicate the estimated start date of the banking crises.

50. Beyond these averages, studies of banking crises suggest that initial macroeconomic and financial sector conditions are an important determinant of cross-country differences in fiscal costs of crises (SM/03/50), with weaknesses in financial institutions increasing their vulnerability to various shocks and raising the costs of resulting financial crises.39 Table 3 reports the stylized facts on a sample of about 50 banking crises (a subset of the sample for which sufficient data are available). Other factors equal, the cost of banking crises are higher in countries with larger financial systems (as measured by the ratio of bank credit to GDP), greater credit risk exposure (proxied by the ratio of private sector credit to deposits), faster credit expansion prior to the crisis (as measured by an increase in the ratio of credit to GDP), and a higher share of the state in the banking system.40 The data also indicate that the median cost of banking crises is much greater for countries with lower-than-average capital adequacy ratio.41

51. Drawing on earlier studies of banking crises, more formal cross-country regressions can be used to relate the fiscal costs of crises to various indicators of a country’s initial macroeconomic and financial conditions, though scope for extensive analysis is circumscribed by the lack of historical data on indicators of banking system soundness. A further complicating factor is that much of the variation in the cross-country evidence is accounted for by the particular resolution strategy adopted—such as the provision of unlimited deposit guarantees, open-ended liquidity support, repeated recapitalizations, and regulatory forbearance (Honohan and Klingebiel (2000))—which, of course, is difficult to establish ex ante. With these caveats, Appendix III presents some regressions which could be used by country teams to infer likely costs of a banking crisis given the country’s macroeconomic and financial characteristics. While the various specifications imply somewhat different costs, the approach would nevertheless represent a marked improvement over the blanket assumption of a 10 percent of GDP contingent liability shock, as assumed in the current version of the sustainability template.

C. Summary

52. While still work in progress, the overall conclusion is that it should be possible to bring to bear available information on financial system soundness to public debt sustainability assessments. At the same time, some caution is required to ensure that the mere computation of the possible contingent liabilities does not create moral hazard by signaling the government’s willingness to bail out the banking system. Given the nature of the problem, this should be done on a case-by-case basis, rather than being incorporated in the standard debt sustainability template.42 Where estimates of possible contingent liabilities are available, these can be incorporated into the sensitivity test directly. Where they are not available, with the active cooperation of the supervisory authorities, some ballpark estimates of the potential capital deficiency of the banking system—and hence the call on the public purse if such funds cannot be generated by the private sector—can be established by stress-testing bank balance sheets. When this is not feasible, or is unjustified given the resource costs, cross-country econometric evidence can be used to derive some ballpark estimate of the possible fiscal liability of a banking crisis.

53. Finally, despite the emphasis here on contingent liabilities arising from banking sector restructuring, it bears reiterating that there are many other sources of contingent liabilities, including debt contracted by state enterprises and parastatals but not formally guaranteed by the state. Other examples would be contingent liabilities faced by transition economies because of unsettled claims of the pre-transition period, or claims that arise from possible court rulings. In countries where such liabilities may be important, country teams would be encouraged to incorporate any available information into the debt sustainability framework.

VI. Interpreting Debt Ratiost

54. A key challenge in making the sustainability framework operational lies in interpreting debt ratios, both actual levels, and those generated in various scenarios and sensitivity tests. It is useful, at the outset, to recall that the framework is intended to establish whether the debt ratio is likely to increase (including because of unexpected shocks) to a point at which a crisis becomes imminent. It does not, however, model the crisis itself. In particular, this means that a stable, or even declining, debt ratio at the end of the forecast horizon is of little comfort.43 Rather, there are three key questions to determining whether the country may enter a “danger zone”: (i) Is the debt ratio, either along the path or at the end of the horizon, so high that the country is vulnerable to a crisis? (ii) Can the country plausibly generate and maintain the primary surpluses required over the medium term to at least stabilize the debt ratio? (iii) Are the gross financing needs required along the path so large that the country may run into a funding crisis:? While recognizing that assessments of sustainability inevitably involve some element of judgment, this section examines how debt levels may be put into perspective.

A. Debt Thresholds

55. Assessing Sustainability (SM/02/166, Appendix 1) took a first stab at establishing debt thresholds, examining the level of external debt at which debt “corrections” occurred, using both frequency distributions and binary recursive trees.44 The analysis there found an appreciable increase in the likelihood of a debt correction when the external debt level rises above 40 percent of GDP (with a somewhat higher threshold for countries that are especially open to international trade). This does not, of course, imply that countries with debt ratios above this threshold will necessarily have a crisis: the conditional probability of a crisis is about 15–20 percent, so that another way of looking at the results is that the probability of not having a crisis is 80 percent (even when the debt ratio is above 40 percent of GDP). 45 Recent research work has tried to refine this analysis, dividing countries according to their Institutional Investor ratings.46 Excluding the advanced economies, which enjoy continuous access to capital markets, and the (mostly low-income) countries with almost no access to capital markets, the results suggest using a rule-of-thumb for external debt of 35 percent of GDP, with countries with a history of high inflation or previous defaults being more vulnerable to a crisis.

56. Several recent crises, however, have involved unsustainable public debt dynamics and have been manifested in sovereign debt servicing difficulties or defaults. A useful complement to the analysis in SM/02/166, therefore, is to examine debt ratios at which sovereign debt defaults have occurred, as classified by Standard and Poors.47

57. A simple starting point for identifying such danger zones is just the frequency distribution of the debt levels at which sovereign debt crises have occurred. As shown in Figure 6, more than half of sovereign debt crises have occurred at public (or external) debt ratios of below 40 percent of GDP, and two-thirds at public (or external) debt ratios below 60 percent of GDP.

Figure 6.
Figure 6.

Public and External Debt Ratios in Sovereign Debt Crises.

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

58. Going beyond these frequency distributions, ideally, an “early warning” model of sovereign debt crises could be developed, relating the likelihood of a crisis to the level and characteristics (such as maturity and currency composition) of the debt as well as country-specific factors. Such a model could be inverted to obtain the maximum debt level consistent with a given probability of triggering a crisis. As usual, however, choosing the trigger probability (and hence the maximum debt threshold) involves a fine balance between missing crises and generating false alarms.

59. At a practical level, the scope for econometric work is severely limited by the lack of historical data on many of the factors that were probably critical in determining whether the country experienced a debt crisis—including the currency composition of the debt, its maturity structure, whether it was indexed, and whether it was held by the domestic financial system, other domestic residents or by foreign investors. Within these constraints, and with a significant effort at collating existing data from a variety of sources, a recent research project has examined the feasibility of estimating an early warning model of debt crises (Box 4). The results suggest that the external debt-to-GDP ratio, the short-term debt-to-reserves ratio (remaining maturity), the debt service burden, the current account deficit, and inflation are all positively related to the probability of experiencing a sovereign debt crisis, while real growth and trade openness reduce that probability.48 Overall, the model correctly predicts 74 percent of all debt crisis outbreaks one year in advance while sending only 6 percent false alarms. Although this work represents an important advance, as emphasized below, identifying “danger zones” is still more of an art than a science, with a large element of judgment required.

An Early Warning Model of Sovereign Debt Crises

Recent work by Manasse, Roubini, and Schimmelpfennig (2003, hereafter MRS) represents a valiant effort to collate data from various sources and to analyze the factors that affect the probability of a sovereign debt crisis. Their work includes information on 37 countries with market access for the period 1970 to 2002, with the sample period for transition countries being 1995 to 2002, a sample that includes 31 default entries.1

In MRS, a country is defined to be in default if it is classified as being in default by Standard & Poor’s or if it receives a disbursement in the first year of a large Fund arrangement (over 100 percent of quota). Explanatory variables are drawn from many different sources. They examine the evolution of some of these variables using an event study approach, conduct binary recursive tree analysis, and run logit regressions. These explanatory variables are divided into six groups—external debt variables, public debt variables, variables from the Fund’s currency crisis Early Warning System, other macroeconomic variables, fiscal variables, and political economy variables. Given degrees of freedom considerations, MRS pursue a three-stage approach to determine the final specification of logit regressions. In the first stage, they regress each variable against the sovereign default indicator; in the second, they pool the best performers (in terms of respective z-values) within each group and run multivariate logit regressions; in the third, they combine the best performers from each group to form a ‘reduced’ model.2

The MRS logit EWS correctly predicts one year in advance 74 percent of default entries in the sample and sends only 6 percent false alarms. The logit EWS does not capture well most of the Asian Crisis. The regressors have expected signs: external debt to GDP ratio, short-term debt to reserves ratio, debt service burden, current account deficit, and inflation are positively associated with the probability of a sovereign debt crisis; higher real GDP growth and openness reduce the probability. These results are robust to various sensitivity tests.

These results may be useful in designing an early warning system for sovereign debt crises, although in practice it may be hard to apply to many developing countries because the underlying data are not available. For that reason, and because the independent variables may have to be predicted in an early warning application, a more parsimonious—but possibly varying across country groups (i.e. industrial, developing, etc.)—specification may be preferable for interpreting debt ratios in the sustainability framework.

1. Given the MRS definition, a country can be ‘in default’ for an extended period time—the average for the sample being 5.5 years—as Standard and Poor’s default episodes often continue for several years.

2. The logit regression specification used by MRS can be represented as:
Pr⁡(Dt)=11+exp−[α+β1Dt−1+β2Dt−1+β3(1−Dt−1)Xt−1]

where Dt represents sovereign default at time t, which is conditioned on whether there was a default the previous year (Dt-1), certain variables that enter the regression independently (Zt-1), and others that are interacted with the default indicator (Xt-1).

60. For the purposes of the debt sustainability framework—which seeks to assess debt dynamics over the next five years—an important requirement is that the model not use as explanatory variables indicators that are themselves difficult to forecast. 49 To this end, there has been some experimentation with more parsimonious specifications (using the same data set).50 Specifically, for assessing the level of external debt, the probability of a debt crisis may be correlated to the external debt ratio, the ratio of short-term debt to reserves, and the trade openness of the country. For public debt, the probability of a crisis is correlated to the public debt ratio, the primary balance, and the revenue ratio.51 Despite the simplicity of the model, the external debt model predicts 88 percent of debt crises while the public debt model predicts all of the observed debt crises (within the more limited sample given by the availability of public debt data), where the model is deemed to be “predicting” a crisis whenever the implied probability exceeds 0.2. However, the models also call a high percentage of false alarms (about 50 percent). Raising the trigger probability from 0.2 to 0.6, lowers this percentage of false alarms to about 9 percent, but also the percentage of crises called correctly (to about 52 percent)—a somewhat lower predictive performance compared to the full-blown early warning model outlined above.

61. Figure 7 depicts the fitted probabilities of a debt crisis implied by the model.52 For instance, for a country with a trade openness ratio (sum of exports and imports) of 100 percent, a short-term debt ratio of 100 percent, and an external debt ratio of 50 percent, the probability of a crisis is 0.26; increasing the external debt ratio to 70 percent doubles the probability to 0.53, while doubling the short-term debt ratio raises the probability to 0.32. As indicated by the hyper surfaces in Figure 7, the corresponding probabilities are higher for economies that are more closed to trade.

Figure 7:
Figure 7:

Fitted Probability of Sovereign Debt Crisis As a Function of External Debt, Short-term Debt, and Trade Openness

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

B. Debt-Stabilizing Primary Surplus

62. While such “early warning” models can give an indication of rising vulnerability, there are clearly many other factors that determine whether a country’s debt is sustainable. Discussions with analysts in private financial institutions (Appendix IV), for instance, suggest that, beyond the absolute level of debt, markets are often concerned about the ability of the government to generate the requisite primary surplus to stabilize the debt ratio (given market expectations of growth and costs of financing). This depends to a large degree on political will and social cohesion—factors that are not easily captured within a formal framework. Moreover, it is not only the short-run that matters, but also whether the country can sustain the required primary surplus over the longer-run (or generate larger surpluses in the short-run that reduce the level of debt and thus the required long-run debt-stabilizing surplus).

63. As a simple enhancement of the standard framework, therefore, it is proposed that the primary surplus required to stabilize the debt ratio (generated in the baseline scenario and the sensitivity tests) be reported. This would be defined as the constant, “steady state” primary surplus which stabilizes the debt ratio at its value at the end of the projection horizon, assuming that all other parameters are constant at their end-horizon values as well.53 Conceptually, this long-run required primary surplus may be more easily interpreted than the debt ratio itself, since the additional adjustment effort can be viewed against the country’s historical performance.54

C. Financing Needs

64. As noted above, the sustainability framework is not intended to model the actual dynamics of a debt crisis. In practice, a debt crisis is usually manifested as a funding crisis— that is, a very steep rise in marginal funding costs to the point that the borrower effectively loses access to the capital markets and is unable to roll over maturing debt. Although the sensitivity tests in the standard template include an interest rate shock and a combined shock, these are supposed to capture rising average costs of funding (which, in turn, worsen debt ratios), not the dramatic increase in marginal interest rates as the crisis erupts. Modeling the latter is enormously complex, not least because it is likely to depend very much on the particular circumstances prevailing at the time of crisis, including the state of the capital markets themselves.

65. Accordingly, the revised version of the template takes the simpler approach of tracking gross financing needs associated both with the baseline scenario and the alternative scenarios and sensitivity tests.55 For external debt, financing needs are expressed in billions of U.S. dollars, since it is generally the country’s financing needs relative to available emerging market funds that are of relevance. Given projections of the likely evolution of total emerging market funds (as well as possible regional “allocations”) this information can help inform judgments about risks to the country’s debt sustainability. For public debt, the analysis of gross financing needs may be more complex, for instance if part of the market is captive (e.g. domestic pension funds or banks that have requirements on government bond holdings) and the distinction between domestic and non-resident holders of government paper is blurred. The public debt template tracks gross financing needs, though assessing the rollover risk may be very dependent upon the country-specific circumstances.

D. Summary

66. While the framework helps inform judgments about the sustainability of a country’s public and external debt, it does not remove the need for such judgments. Accordingly, sustainability analysis needs to place the debt dynamics implied by the framework in the particular context of the country. Some of the relevant factors relate to the country: its vulnerability to shocks, its macroeconomic management, and any prior history of debt defaults or crises. Other factors relate to the debt itself, including its currency composition (or indexation) and maturity structure as well as whether it is held by foreign investors, the domestic banking system, pension funds, or other domestic residents. While early warning models and analyses of gross financing needs can capture some of these factors, both analytical and data shortcomings mean that some element of judgment is unavoidable.

67. Moreover, the template is primarily a medium-term framework, intended to identify growing vulnerabilities so that corrective policies can be taken in a timely manner. It does not, however, model the dynamics of a crisis itself, in particular the possible nexus between debt, the banking system, and the exchange rate. In some previous crises, for instance, default or restructuring of public debt resulted in de-capitalization of domestic banks which were holding (or were effectively forced to hold) government paper. (Conversely, banking system restructuring costs can lead to significant increases of public indebtedness.) Likewise, rising concerns about debt sustainability may put downward pressure on the exchange rate as capital inflows dry up, in turn forcing the government to issue shorter-maturity and foreign currency denominated or indexed debt. Any collapse of the exchange rate in a crisis then adds significantly to the debt burden, while raising interest rates to defend the currency risks worsening the dynamics of the domestic currency-denominated debt.

68. This suggests that, when the debt sustainability framework suggests rising and potentially risky debt levels, country teams should try to place the debt dynamics analysis within the context of the country’s overall vulnerability. In examining the implications of higher debt levels, associated with various possible shocks, they should take into account the potential feedback effects on the banking system, on the exchange rate, and on the availability and pattern of external and domestic financing.

VII. Proposed Changes and Enhancements

A. Design, Use, and Resource Implications of the Standard Template

69. In refining the template, it is worth recalling its basic purpose, namely to instill greater discipline in debt sustainability assessments undertaken by the Fund. To this end, the template incorporates four basic features: (i) a convenient format for laying out the key assumptions underlying the baseline projection; (ii) a method for calculating the “95 percent” probabilistic bound around that baseline; (iii) a framework for articulating alternative scenarios; and (iv), a means for tracking gross financing needs.

70. But the template is a tool for better sustainability assessments, not an end in itself. The intention is to promote best practices, not to impose a strait-jacket of the “least common denominator.” Therefore, when application of the standard template does not fulfill this objective, or when better methods are available, country teams should be encouraged to explore the use of alternative frameworks—as long as these incorporate the four basic features of the standard template enumerated above. For instance, as discussed in section III, it may be possible to use stochastic simulations to calculate explicit probability density functions from which the precise 95 percent (or any other desired) confidence interval can be established. Likewise, in the major industrialized countries, the nature of the potential debt sustainability problems are often not well captured by the standard template. In other cases, because of extreme events, such as hyperinflation, in the country’s history, the practice of using the past 10 years to calculate the parameters of the shocks leads to extreme confidence intervals.56

71. Of course, in the vast majority of cases, the standard template will be the most suitable, most convenient, and least resource-costly framework for country teams to adopt. Indeed, the new version of the template automatically calculates the standard sensitivity tests, picks the two most extreme tests, and plots them, along with the baseline scenario and the associated gross financing needs. The template also computes debt-stabilizing primary balances; Box 5.

72. The convenience of the standard template should allow country teams to focus most of their efforts—and discussions with the authorities—on formulating the baseline and alternative scenarios. Since country teams typically undertake medium-term projections, the baseline projection required for the debt sustainability template entails little additional work. The addition of the new country-specific alternative scenario, however, especially if modeled properly, will require significant additional effort. While it is difficult to quantify the resource costs explicitly, the work required is presumably of the same order as that entailed in formulating the medium-term baseline projection.

73. This suggests that a two-pronged approach might be appropriate. In cases where the baseline projection—and the sensitivity tests—indicate that the debt ratio is expected to remain well within prudent levels (as discussed in section VI) and that gross financing needs are manageable, the alternative scenario would be unnecessary (or could be calculated purely mechanically).57 In all other cases, however, given central importance of debt sustainability to the Fund’s work, and that articulating the baseline and alternative scenarios should help integrate these assessments with the rest of the staff’s analysis, it may be desirable to expect country teams to formulate the alternative scenario as well, recognizing that this will inevitably involve some additional work.

Proposed Enhancement to Debt Sustainability Framework

This Box summarizes the proposed changes and enhancements to the debt sustainability framework:

  • Supplement the current debt bounds sensitivity tests with scenario analysis. These will scenarios based on (i) all variables taking their historical values for the duration of the projection horizon; (ii) the “no policy change” primary balance in the public debt template; (iii) market expectations or consensus forecasts (where available) for macroeconomic variables; (iv) a scenario formulated by the country team in which GDP growth falls by one-standard deviation.

  • Distinguish clearly between the alternative scenarios and the debt bounds sensitivity tests, emphasizing that the latter provide a probabilistic upper bound on the evolution of the debt ratio and are not intended to depict likely outcomes.

  • Enhance the treatment of the contingent liability sensitivity test using any available country-specific information or cross-country evidence on the cost of banking crises.

  • Report the debt-stabilizing primary (or non-interest current account) balance associated with the debt levels in the baseline scenario, the alternative scenarios, and the sensitivity tests.

  • Track gross financing needs for public and external debt (Figure 8).

Mock-ups of the revised external and public debt templates are given in Tables 4 and 5. Following Board discussion, a technical guidance note will be prepared for staff.

Figure 8.
Figure 8.

Country Name: Debt Ratio and Gross External Financing Need, 2002–08

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

Source: IMF staff calculations.

74. Better focus of the Fund’s debt sustainability assessments may also mean paring down on the number of such analyses undertaken by country teams. For instance, to the extent that potential debt sustainability problems in industrialized countries arise from slow-moving demographic changes, it may make little sense to present sustainability assessments every year. Less frequent, but more carefully modeled scenarios (including the complex dynamics of demographics and pension liabilities) may be preferable. Among the bulk of emerging market and developing countries with significant market access, however, the frequency and magnitude of shocks as well as policy reversals warrants debt sustainability assessments as part of the Article IV consultation and for requests for use of GRA resources.

75. Based on these considerations, we do not envisage that implementing the proposals in the paper will have significant resource implications, either for the current year or the medium term. In the event that, against expectations, experience this year indicated this work is creating significant additional demands on staff, an estimate of the extra resources required and a statement on how they would be financed within the Fund’s medium term framework would be brought before the Board for consideration in the FY 2005 budget.

76. The revision of the framework leaves open the issue of over-optimistic baseline projections. It is expected that greater cognizance of possible biases will help discipline the projections undertaken by country teams. One possibility would be to include in staff reports an analysis of the error in the previous year’s projection, especially when the outturn was very different from the projection. The “fresh perspective” in surveillance is also expected to contribute to more realistic projections. Moreover, while over-optimism in projections is clearly an important concern, it is not easily addressed by the design of the framework. Rather, the internal review process will need to scrutinize assumed growth rates and other macroeconomic parameters against both the country’s historical performance (which is conveniently encapsulated in the standard tables) and the cross-country evidence; the analysis of typical projection errors presented in Appendix I should be helpful in this regard.

B. Publication of Debt Sustainability Assessments

77. As regards publication, during its discussion of Assessing Sustainability, the Board noted a tension between the credibility of Fund’s sustainability assessments and the possibility that publication could lead to misinterpretations by the public and adverse market impacts. To date, about 60 percent of the debt sustainability analyses presented in Board documents have been published as part of the staff report or Selected Issues paper (Table 1). In only three cases did the authorities request that the debt sustainability analysis be deleted prior to publication of the staff report, and in a further two cases, the debt sustainability assessment was reported in a stand alone paper so the issue of publication did not arise.

78. The arguments in favor of publication are that the templates do not use any information that is not routinely available to market participants, other than the baseline projection—which is automatically available if the staff report is published.58 Conversely, at the time, there had been no experience with the use of the sustainability template—which was viewed as work in progress and subject to various changes and enhancements—nor was it known how the debt sustainability assessments would be received by markets. As a result, the Board recommended that, pending further experience, the debt sustainability assessments be presented separately and the publication issue revisited.

79. While there may be further enhancements to the framework, there has already been much work on refining the template and valuable experience has been gained over the past year. Moreover, to date there have been only three cases in which the authorities requested that the debt sustainability analysis be deleted prior to publication of the staff report. Nor is the staff aware of any instances in which publication of the debt sustainability assessment led to an adverse market response.

80. Accordingly, it is proposed that, henceforth, there would be a presumption that the public and external debt sustainability assessments would be included in published staff reports. This would mean that the assessments could be in the body of the staff report which, together with the enhancements to the framework itself, would help integration with the rest of the staff’s analysis of economic developments and policy advice. In cases in which the authorities are nevertheless wary of adverse market reactions, the standard deletions policy, which provides for deletion of market sensitive material, would apply.59

VIII. Issues for Discussion

81. This paper proposes a number of refinements and modifications to the debt sustainability framework: (i) supplementing the debt bounds sensitivity tests with scenario analysis (including modeled country-specific scenario, and a “no policy change” scenario); (ii) enhancing the treatment of contingent liabilities in the public debt sustainability template; (iii) reporting the debt-stabilizing primary (or non-interest current account) balances; and (iv) tracking gross financing needs for public and external debt. The paper also suggests that, while maintaining the discipline of the standard template, alternative approaches more tailored to country-specific circumstances might also be usefully considered.

82. In their interventions, Directors may wish to address the following issues:

  • Do Directors agree with a flexible approach to debt sustainability analyses, in particular for industrial countries, with country teams encouraged to experiment with alternative frameworks as long as these incorporate the general features and meet the standards of the sustainability template?

  • Directors may wish to comment on the experience with the template. Have they found the analyses useful, and how could they be made more so?

  • Do Directors support the proposed refinements? Would they be interested in further refinements?

  • Do Directors agree that banking system restructuring costs may represent an important source of contingent liabilities, and that country authorities should be urged to provide teams with sufficient data to refine estimates of such costs?

  • Do Directors agree that debt sustainability analyses should be an integral part of staff reports to which existing publication and deletions policy would apply?

Appendix I Errors in WEO Projections of Public and External Debt

83. The sustainability analysis in the framework is centered on the baseline projections for public and external debt, and a key question is whether these projections are realistic. Since the debt sustainability framework has been in operation less than one year, it is clearly impossible to assess the realism of the baseline projections. However, it may be possible to gain some insight into the typical magnitude of biases in staff projections—and the reasons for them—by examining the properties of the projections underlying the WEO exercise.60

84. Debt dynamics are not projected directly, but rather depend upon a number of underlying variables. Thus, the evolution of the external debt-to-GDP ratio will be a function of the current account, FDI flows, the U.S. dollar value of the GDP deflator, and real GDP growth. Errors and biases in projecting any of these variables would result in a corresponding, and generally compounding, error in projecting the debt ratio. Table 1 reports the average bias (defined as outcome minus projected, and expressed in percent per year or percent of GDP, as applicable) for the key parameters underlying external debt projections.

Table 1.

Errors in World Economic Outlook Projections of External Debt Ratio

article image

Predicted and actual values transformed by z/(1+z) for z >0, and z/(1-z) for z < 0.

Projections starting in 1995

IMF-supported program in year being projected