Sustainability Assessments - Review of Application and Methodological Refinements
Author:
International Monetary Fund
Search for other papers by International Monetary Fund in
Current site
Google Scholar Close
Publication Date:
06 Oct 2003
eISBN:
9781498329453
Language:
English
Keywords:
PP; debt crisis; sustainability assessment; debt dynamics; sensitivity test; current account; real GDP; U.S. dollar; emerging market; real interest rate; sustainability analysis; sustainability framework
Download PDF (2.8 MB)

Sustainability Assessments - Review of Application and Methodological Refinements

Abstract

Sustainability Assessments - Review of Application and Methodological Refinements

I. Introduction

1. In June 2002, as part of the Fund’s efforts at crisis prevention and resolution, the Executive Board endorsed a new framework for assessing the sustainability of countries’ public and external debt. Such assessments underpin the Fund’s policy advice in both program and surveillance contexts. The new framework was intended to bring a greater degree of consistency and discipline to sustainability analyses, including by laying bare the basis on which projections are made and subjecting them systematically to sensitivity tests.1 In endorsing the framework, Directors envisaged that it would be useful in a variety of situations: for countries with moderately high indebtedness, the framework could help identify vulnerabilities far enough in advance that policy corrections can be undertaken. For countries on the brink, or in the midst of a crisis, the framework might be used to examine the plausibility of the debt-stabilizing dynamics articulated in program projections. Finally, in the aftermath of a default, the framework could be used to examine whether alternative structures and levels of restructured debt are consistent with projected outcomes.

2. The framework consists of three main elements. The first is the baseline scenario— the set of macroeconomic projections that form the basis for understandings on a Fund-supported program, or the articulation of the authorities’ intended policies as discussed with the staff in a surveillance context—with the main assumptions and parameters clearly laid out. Second, a series of model-independent sensitivity tests are applied to the baseline scenario, providing a probabilistic upper bound for the debt dynamics under various assumptions regarding policy variables, macroeconomic developments, and costs of financing. Third, while recognizing that debt sustainability assessments inevitably involve judgment, the output of the framework needs to be interpreted in terms of the vulnerability of the country to a crisis.

3. At the time of the Board discussion, it was understood that the approach represented work in progress, and that modifications and enhancements might be introduced to each of these elements in light of experience. Directors thus asked staff to review the application of the framework after one year, and also identified a number of issues warranting further examination. Regarding the baseline, their discussion centered on whether staff projections are realistic or are systematically over-optimistic. On the design and calibration of the sensitivity tests, Directors asked whether the tests adequately capture the dynamics and co-movements of the key variables in a run-up to a crisis and, relatedly, whether the assumed shocks are either too benign or too extreme. On assessing the debt dynamics, Directors requested that staff explore the feasibility of establishing threshold values or “danger zones” for the debt ratio, and of taking account of liquidity or roll-over risk within the current framework. Directors also asked staff to explore ways in which the debt sustainability analysis could be linked more closely to assessments of financial sector vulnerability, in particular, the possible contingent liabilities arising from financial sector restructuring. Finally, Directors requested to be informed about techniques for assessing debt sustainability employed by private sector financial institutions and ratings agencies.

4. This paper takes up these issues, reports on preliminary experience with use of the framework, and proposes some enhancements in light of that experience. Section II reviews the use of the public and external debt sustainability framework in staff reports, which has become routine in Article IV consultations in countries with significant market access and in requests for GRA resources. The overall impression is that, while the framework has been introduced successfully and has probably contributed to more careful analyses of sustainability, debt sustainability assessments are not yet well integrated with the rest of the staff’s analysis of economic developments or with the policy dialogue with national authorities. In part, this is because the framework is still relatively new, but one concern is that the sensitivity tests consider shocks that are too extreme (and thus too improbable) to warrant a policy response; the opposite concern, that the assumed shocks are too benign to adequately capture all the risks to debt sustainability is also, however, potentially relevant.

5. Section III, therefore, turns to the design and calibration of the sensitivity tests using both evidence from previous debt crises and Monte Carlo simulation techniques. The conclusion of this section is that the shocks underlying the sensitivity tests broadly emulate the behavior of the key variables in a run-up to a debt crisis. Moreover, the specified sensitivity tests indeed come close to a 95–99 percent confidence interval around the projected evolution of the debt ratio (so that there is usually less than 5 (or 1) percent probability that the country’s debt will exceed this level). As such, they are appropriately calibrated to provide the intended probabilistic upper bound to the debt ratio.

6. Nevertheless, the sensitivity tests in the current version of the template may be too extreme (and too mechanical) to warrant a policy response: it might make little sense to strive for significantly stronger adjustment (or a debt restructuring) in anticipation of a shock that occurs with only low (less than 5 percent) probability. For this reason, Section IV proposes that the current set of sensitivity tests be supplemented with some additional scenarios that capture less extreme and more probable events. These will be based on historical performance, available information on market expectations of economic developments, or the main risks facing the economy. In particular, in one of these scenarios, country teams would be encouraged to model the main shocks to the economy that could result in a (one-standard deviation) decline of output growth. Since this scenario would model the main risks to the economy, it would provide a natural platform for discussions with the authorities of the key vulnerabilities, including possible contingency measures if the adverse scenarios materialize.

7. Section V examines how contingent liabilities, which have often been an important source of increases in public debt, may be incorporated in the sustainability framework. Section VI turns to the interpretation of the debt levels, and reports some further results on empirical approaches to establishing critical thresholds. Section VII summarizes the proposed enhancements to the framework including publication of debt sustainability assessments. Section VIII presents issues for discussion. An appendix describes the methods employed by investment houses and ratings agencies for assessing debt sustainability.

II. Sustainability Assessments: Experience and Conceptual Issues

A. Experience with Use of Sustainability Templates

8. During its discussion in June 2002, the Board endorsed the use of the sustainability framework for countries with significant market access. Following some initial refinements of the template2, sustainability assessments were introduced progressively in staff reports for requests for use of Fund resources under the GRA and for Article IV consultations.3 These assessments are now routine for countries with significant market access, in practice, this criterion has been interpreted to exclude mostly the PRGF-eligible countries.4 Over the nine months since the Board’s endorsement of the framework, assessments of public debt sustainability have been presented in staff reports for some 45 countries, while external debt sustainability assessments have been presented for some 30 countries; almost 40 percent of all staff reports (for non-PRGF eligible countries) issued during this period have included either the public or external debt sustainability analysis or both (Table 1).

Table 1.

Preparation and Publication of Debt Sustainability Analysis Using Standard Template

article image
Source: Staff estimates.

9. About 60 percent of the standard debt sustainability analyses prepared to date have been in the context of Article IV consultations, while 20 percent have been for stand alone requests for use of Fund resources, and the rest for combined Article IV and UFR reports. Industrialized countries form about 20 percent of the sample, emerging market countries account for 70 percent, and PRGF-eligible countries for 10 percent. The debt sustainability assessments are usually reported in an annex to staff report, though at times they are included in the Selected Issues paper or another stand-alone paper.

10. The core of the debt sustainability assessment is the historical decomposition of the debt dynamics and the baseline scenario, which is projected for a five-year horizon. All of the debt sustainability analyses have used a minimum horizon of five years. In a few cases, typically, though not exclusively, in industrialized countries, longer projection horizons—up to 40 years—have been adopted to capture the implications of demographic dynamics for pension liabilities.

11. Beyond the baseline projection, the sustainability template specifies sensitivity tests to key parameters—including the interest rate, the growth rate of the economy, the GDP deflator, the exchange rate, and the primary (or non-interest current account) balance—as well as a sensitivity test that contrasts the baseline projection to the country’s historical performance. The sensitivity tests, which are specified mechanically in terms of two-standard deviation adverse shocks (one standard deviation for the combined shock) lasting for two years, are intended not as alternative scenarios but rather as probabilistic upper bounds on the evolution of the debt ratio, akin to a “95 percent confidence interval.” The template also suggests the use of relatively long horizons—ten years, if feasible—for the calculation of the relevant averages and standard deviations of the key parameters.

12. Most of the sustainability assessments have followed this methodology. The main departure concerns the horizon over which the standard deviations are calculated, which in some cases is as short as five years. In general, the reasons for doing so are indicated—for example, the country has suffered a hyperinflation or is a transition economy—and a plausible case can be made that past performance is not relevant and that basing the analysis on it would be misleading. In some instances, however, the arguments have been less clear-cut—for example, in one case it was argued that, since the country had moved to a floating exchange rate regime, the past history of high real interest rates was not relevant as there was no longer any need to defend the exchange rate. Some country teams have included additional sensitivity tests; for instance, oil price shocks for countries where the government relies on oil revenues have been examined. Teams have also supplemented the standard sensitivity tests with “scenario” analyses in which two or three scenarios—e.g. base, optimistic, and pessimistic—are depicted.

B. Assessment

13. While it is still early days to assess the performance of the sustainability framework, the more disciplined debt sustainability assessments have been pivotal in program or Article IV discussions in several prominent cases. Application of the template to Argentina’s public debt shows that restoring sustainability will almost surely require a significant reduction in the present value of outstanding obligations. Mexico provides a good example of the effective use of the framework in a surveillance context, and moreover, comments received from the Mexican authorities on an early version of the framework led to some important modifications and improvements in the framework. Lebanon was another example where the application of the framework to analyze the sustainability of public debt dynamics, in light of assumptions on adjustment and privatization receipts, figured prominently in the Article IV consultations.

14. Standardization of the sustainability framework appears to have brought somewhat greater discipline to the assessments of public and external debt sustainability, in particular by making explicit the assumptions upon which the projections are predicated. For instance, except where the country relies mainly on concessional borrowing, there are very few cases in which the projections assume that the growth rate of the economy will exceed the effective real interest rate—even though historically it may have done so (especially on domestic debt, where interest payments can be eroded by inflation).5

15. At the same time, baseline projections probably exhibit at least some bias toward over-optimism. Looking across the sample of sustainability assessments, for instance, it is striking that—except in a couple of cases where the staff is at pains to underscore the unsustainability of current policies—the projections show not only a stabilizing debt ratio by the end of the projection horizon, but nearly always a decrease in the debt ratio relative to the starting point: the median projected decrease of public debt over the five year horizon is about 12 percent of GDP6, while external debt declines by about 17 percent of GDP.7 Without making judgments about the plausibility of individual baseline projections, statistically it seems unlikely that such a broad range of countries would all experience declining debt ratios.

16. Although the track record of the new sustainability framework is too short to establish the realism of the baseline projections, some insight about typical biases in country teams’ projections can be gained by examining the forecasts for debt dynamics underlying the WEO exercise (Box 2, Appendix 1). This analysis suggests that, at a five year horizon, the external debt ratio is under-predicted by about 3.3 percent of GDP across the full sample of the Fund’s membership (industrialized, emerging market, and developing and low-income countries). For upper-middle income countries the bias is 4.2 percent of GDP, while for those with Fund-supported programs, it amounts to more than 7 percent of GDP. To put this in perspective, it is worth recalling that, on average, the baseline projection of the recent debt sustainability assessments project a decline of the external debt ratio of 7 percent of GDP. Baseline projections, of course, are not intended to be unconditional forecasts, being predicated on the agreed policies being fully implemented and on the economy and markets responding appropriately. This means that some bias is probably unavoidable, but it also underscores the importance of adequate stress testing of the baseline projections.8

How Good (or Bad) are WEO Projections of External and Public Debt?

One way to gain some insight into the typical biases in projections undertaken by country teams is to examine the properties of WEO projections of external and public debt. As debt dynamics are not projected directly, but rather depend on underlying variables such as FDI flows, U.S. dollar value of GDP deflator, real GDP growth, as well as revenue and expenditure, errors in projecting any of these variables result in errors in projecting the debt ratio. Biases for these key variables are analyzed in Appendix 1.

Overall, the error in projecting the external debt ratio is in the range of 1 to 1½ percent of GDP per year of the projection horizon. Among countries with Fund-supported programs, the largest errors are for the upper-middle income countries and for the low-income countries.

Decomposing the results shows that the error in projecting the external debt ratio for low-income countries is almost entirely due to the error in projecting the dollar value of GDP, both in turn, because growth and the real exchange rate appreciation is over-estimated projecting the nominal exchange rates (as inflation is likely to be underestimated).

Errors in World Economic Outlook Projections of External Debt (in percent of GDP) 1/

article image

Predicted and actual values transformed by z/(1+z) for z >0, and z/(1-z) for z < 0.

Projections starting in 1995.

IMF-supported program in year being projected.

A similar exercise for assessing errors in projecting public debt can be undertaken, though the available sample is very small. The results indicate that the general government balance is overpredicted by 1–2 percent of GDP, with most of the projection errors coming from higher expenditure rather than revenue shortfalls. At the one-year horizon, general government net debt is underpredicted by about 1 percent of GDP, but almost 5 percent of GDP for countries with Fund-supported programs.

Errors in World Economic Outlook Projections of General Government Net Debt 1/

article image

Predicted and actual values transformed by z/(1+z) for z >0, and z/(1-z) for z < 0.

Projections starting in 1995.

IMF-supported program in year being projected.

1/ In part, this is because nominal revenues tend to decline automatically in a downturn whereas nominal expenditures tend to remain constant. In terms of ratios to GDP, therefore, since there is an upward bias in projecting GDP growth, the error in forecasting the balance will be mostly accounted for by the error in projecting the expenditure ratio.

17. For stress testing, the standard template specifies two-year, two-standard deviation shocks to the main macroeconomic variables (as well as a one-standard deviation combined shock).9 In addition, the public debt template includes a 10 percent of GDP contingent liability shock. Which of the sensitivity tests is the most extreme naturally depends upon the country’s specific circumstances, though across the sample of analyses undertaken to date, it is usually the exchange rate (or combined) shock that results in the largest increase of the public or external debt ratio. On average, the most extreme sensitivity test raises the public debt ratio by 20 percent of GDP relative to the baseline debt ratio (which, on average, is 60 percent of GDP) and the external debt ratio by 30 percent of GDP (against an average baseline projection of 50 percent of GDP).

18. In a few instances, particularly in the context of industrialized countries, some of the standard sensitivity tests may be less relevant and plausible than alternative tests that could be devised. For instance, countries that have adopted the euro may face significantly smaller risk of exchange rate depreciation (vis-à-vis the foreign currencies in which most of their external debt is denominated) than the historical shocks would suggest. Likewise, the template would need to be modified to properly capture the impact of an exchange rate shock in cases in which hedging instruments are being used to mitigate this risk. In many industrialized countries, the major risk to debt sustainability comes from long-term demographic changes, which is not captured within the five-year projection horizon of the standard template.10

19. Overall, the introduction of the new debt sustainability framework has been relatively smooth, and there are some notable cases in which the staff have made good use of the framework, including in discussions with the authorities (Box 2); but, in many, if not most, cases the debt sustainability assessments have not yet become an integral part of the staff’s analysis in the staff report.11 The debt sustainability analysis is typically reported in an annex (or in a separate paper) and, beyond noting the results, the discussion in the main body of the staff report is in many cases quite cursory. With one or two exceptions, it is also apparent that the sustainability analysis did not form a major part of the discussions between the staff and the authorities. Even where the program baseline requires both more ambitious adjustment and a more favorable response than past performance would suggest, the report is typically mute on what has changed in the economy and why the authorities are confident that the baseline scenario is achievable despite the historical experience.

20. While the apparent failure to integrate the sustainabilty analyses into policy discussions partly reflects presentational factors (notably the practice of placing the sustainability analysis in an annex to the main report12), it may also reflect difficulties with interpreting the sensitivity tests and in drawing policy implications from them. As noted above, these tests provide a probabilistic upper bound on the evolution of the debt ratio. Unless the baseline scenario depicts unsustainable debt dynamics (in the sense of a continually increasing debt ratio), the sensitivity tests in most cases do not exhibit unsustainable debt dynamics either—they simply indicate an increase in the debt ratio.13

Presentation of DSA in Staff Reports: An Example

The Staff Report for the 2002 Article IV consultation for Mexico (SM/02/260, 8/15/02, and SM/02/289 Supplement 1, 9/9/02) is a good application of the debt sustainability framework in surveillance context for several reasons. First, the DSA makes extensive use of the framework outlined in SM/02/166. Second, the process of preparing this analysis was comprehensive—it involved extensive discussions of preliminary results with the authorities and incorporation of their comments in a subsequent revision; indeed, their comments led to improvements in the template itself. Third, results of this analysis are integrated into the main text of the Staff Report. Fourth, while concerns regarding market reaction to publication of DSA had been expressed, there does not seem any evidence of adverse effects.

Calibration of Baseline: Reflecting common practice, tests are discussed extensively in a supplement to the Selected Issues paper (SM/02/289, Supplement 1) and a summary presented in a box in the Appendix of the Staff Report (SM/02/260). The Selected Issues supplement presents the factors underlying the baseline scenario. Assumptions regarding growth, revenue, current account, inflows of foreign direct investment, etc., are clearly spelled out, along with staff expectations regarding government’s intended policies.

Sensitivity Tests and Interpretation: Standard sensitivity tests are applied to the baseline scenarios thus outlined. A preliminary version of this DSA resulted in modification of the template to shock the real (rather than nominal) interest rate in one of the sensitivity tests. These standard tests are supplemented with a scenario analysis, whereby a sudden reduction in external capital market access—as in the capital account crisis of the 1990s—is simulated. The resulting potential financing gap is then analyzed. The exercise is clear in expressing the staff’s opinion on the plausibility of different shocks and underlying assumptions regarding authorities’ behavior in response to such shocks. Staff concludes that debt dynamics are robust to most shocks.

Use of Analysis: The analysis is included in a box in an appendix to the Staff Report. Two diagrams— one for external debt and one for gross public sector debt—are presented, with brief discussion of the results. The main text uses this analysis to highlight some of the downside risks that exist despite favorable baseline conditions, explicitly mentioning variables of concern. Gross financing needs in different scenarios are also discussed. These results inform points raised in the staff appraisal as well.

uA01fig01

Mexico: Sovereign Bond Spread

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

Market Sensitivity: Concerns were raised at the time of a possible adverse market reaction to the release of this analysis. A look at the spread (against the US dollar) suggests that any adverse impact was limited although the decline in spreads for other Latin American Countries around the same time makes a precise assessment difficult.

There are then three possibilities:

  • i. One case is that in which the increase in the debt ratio is modest and provides little cause for alarm. In this case, the debt sustainability analysis should not have to be a central part of the staff’s analysis, although there may remain concerns that the shocks assumed in the standard framework are too benign.

  • ii. A second case is that in which the results of (at least some of) the sensitivity tests are so extreme that they are dismissed as being implausible. In such instances, there is a temptation to see whether tinkering with the parameter values (for instance, by excluding periods of extreme inflation or the early transition years) yields more “reasonable” results. While preferable to simply ignoring the results, choosing an appropriate period over which to calculate the relevant parameters may nevertheless require fine judgments about whether the economy has undergone a fundamental shift in performance or whether it is still subject to large shocks.14

  • iii. The third case is that in which, although the debt dynamics appear highly sensitive to the standard shocks, given the history and characteristics of the economy, there is general consensus that such outcomes cannot be ruled out. Even in this case, however, the policy implications are unclear. If the adverse shocks materialize, the country will need to undertake additional adjustment or seek a reduction in the present value of its obligations. Yet it may make little sense to undertake significantly greater adjustment, or to seek a debt restructuring, in anticipation of a shock which could raise the debt ratio by some 20–30 percent of GDP, but which may only occur with less than 5 percent probability.

21. Conversely, a separate set of concerns, expressed particularly by Executive Directors during their initial discussion of the framework, is that the sensitivity tests may not be sufficiently extreme—and the assumed shocks may be too benign—to capture adequately all of the risks to debt sustainability. These competing concerns suggest that the design and calibration of the sensitivity tests warrant further examination (section III, below).

C. Summary

22. To summarize, following Board endorsement in June 2002, debt sustainability assessments have been progressively introduced in staff reports for use of Fund resources in the GRA, and for Article IV consultations with countries with significant market access. Although the framework has brought greater discipline to the process of making projections, baseline projections probably continue to exhibit at least some bias towards over-optimism. Moreover, with some exceptions, sustainability assessments have generally not been at the center of policy discussions between staff and national authorities. This may be because the sensitivity tests are considered too extreme to be realistic or, even if realistic, too extreme to warrant a policy response. Conversely, there remain concerns that the assumed shocks are too benign. Finally, from a presentational standpoint, debt sustainability assessments would have greater impact if they were integrated in the body of the staff report instead of being relegated to an annex.

III. Calibration of Sensitivity Tests

23. A major element of the sustainability framework is the systematic testing of the debt dynamics to various assumptions regarding policy variables, macroeconomic developments, and costs of financing. An important question, therefore, is whether these standard tests are calibrated appropriately, capturing the main risks to debt sustainability without being extreme to the point of irrelevance. This section examines the realism of the sensitivity tests in two ways: (i) by comparing the assumed shocks in the sustainability template to the behavior of the main macroeconomic variables in the run-up to previous sovereign debt crises, and (ii) by computing the probability that the debt outcome will be worse than the upper bound implied by the sensitivity tests.

A. Event Study Approach

24. One way to gauge the realism of the shocks in the standard template is to examine the behavior of the underlying variables in the run-up to previous debt crises—for instance, by how many standard deviations does the GDP growth rate fall (relative to its pre-crisis mean) in the last two years before the crisis.15 Figure 1 depicts the evolution of the four key variables of external debt dynamics—real GDP growth, interest rate, GDP deflator growth (in U.S. dollar terms) and primary current account deficit—over the ten year period prior to a debt crisis (year T is the year of the crisis) for a sample of 26 debt crises (that occurred in 24 countries) with market access during the period 1970–2001. (For each variable, the median across the sample of 26 episodes is shown.)16

Figure 1.
Figure 1.

Pre-crisis Trends in Key Variables (Sample Medians) 1/

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

1/ Crisis episodes covered in the sample include Algeria (1991), Argentina (1982, 2001), Bolivia (1980), Brazil (1983), Chile (1983), Costa Rica (1981), Dominican Republic (1981), Ecuador (1982), Egypt (1984), El Salvador (1981), Guatemala (1986), Indonesia (1998), Jordan (1989), Mexico (1982), Morocco (1983), Nigeria (1982), Pakistan (1998), Panama (1983), Paraguay (1986), Philippines (1983), South Africa (1985), Trinidad and Tobago (1988), Uruguay (1983) and Venezuela (1983,1995).

25. In the last two years, prior to the crisis, real GDP growth falls by about 4 percentage points, relative to its average during the eight pre-crisis years, a little more than one standard deviation17. Interest rates rise by about 2 percentage points (one standard deviation), while the growth of the US dollar value of the GDP deflator falls by about 10 percentage points (representing one standard deviation). Together, these shocks all tend to increase the external debt ratio prior to the crisis. The non-interest current account deficit, however, moves in the opposite direction, narrowing with the fall in imports as growth declines. Formal Chow tests indicate that for growth, the interest rate, and the GDP deflator, there is indeed a break equivalent to about one standard deviation which occurs between one and three years prior to the crisis.

26. The results suggest that the two-year, two-standard deviation shocks to individual variables are ample for capturing the shocks than have typically been observed in the run-up to a crisis. However, the results also suggest that such shocks do not happen in isolation but rather occur simultaneously. The two-year one-standard deviation combined shock in the template is thus very much consistent with the historical evidence on debt crises. Moreover, applying the sustainability template retrospectively to four recent capital account crises (see Assessing Sustainability, SM/02/166, Tables 5 and 6 for external debt and Box 3 for public debt) also suggests that the confidence bounds are appropriate to capturing the increase in the debt ratio in the run-up to the crisis without being too extreme.

Retrospective Debt Sustainability Analysis in Four Capital Account Crises

The sustainability framework was applied to retrospectively to crises in Argentina (1999), Brazil (1998), Mexico (1995), and Turkey (1999), to see whether actual outcomes fall within the stress test range (see Managing Financial Crises, Occasional Paper 217). Because it is based on IMF program projections, the framework can most usefully be applied in this way to Argentina and Turkey, where programs had been formulated well ahead of crises in 2001. In Brazil, the program was formulated after the 1998 crisis had hit, while in Mexico the 1995 crisis followed shortly after the program was formulated. As the figure shows, even though program fiscal targets were in most cases complied with, public debt generally exceeded baseline program projections. The stress test ranges, however, broadly capture most of the outcomes. More specifically:

  • The 2001 debt ratio in Argentina turned out to be slightly above the upper bound of the stress test range. The deviation from the baseline program projection mainly reflected unanticipated recession and weaker-than-programmed fiscal adjustment. If the projection had been extended to 2002, the debt ratio would be shown to have moved far beyond the upper bound of the stress test range, in part owing to the unanticipated securitization of contingent liabilities, but mainly because of a larger-than-expected real exchange rate depreciation.

  • For Turkey, the outcome for 2001 was well above the upper bound of the stress test range, despite the beneficial impact of stronger fiscal adjustment and lower real interest rates than expected. The deviation from the baseline program projection was not only the result of a much larger real depreciation and sharper recession than anticipated; in addition, the securitization of contingent liabilities added unexpectedly and significantly to the debt.

  • The post-crisis debt ratio in Brazil was at the upper end of the stress test range, despite a better-than-expected post-crisis fiscal adjustment and lower-than-expected real interest rates. An unanticipated large real depreciation was the main contributor.

  • In Mexico, the outturn in 1995 was above the one-year-ahead program baseline projection, because of a larger real depreciation, higher real interest rates, and slower growth than anticipated, and because of the securitization of contingent and unfunded liabilities. Nevertheless, the outturn was within the stress test range. However, by 1997 the debt ratio had moved into the middle of the stress test range because initial fiscal adjustment was not maintained as planned.

uA01fig02

Retrospective Debt Sustainability Analysis

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

Table 2.

Examples of Contingent Liabilities

article image
Source: Currie and Velandia (2002), Polackova (1999).
Table 3.

Estimated Cost of Banking Crises Across Selected Financial System Indicators

(in percent of GDP, unless otherwise indicated)

article image

As measured by the ratio of banking system credit to the private sector as a ratio of GDP.

As measured by the percentage point increase in the ratio of banking system credit to GDP.

Sources: IFS, Figure 4, and staff calculations.
Table 4.

Country: External Debt Sustainability Framework, 1998–2008 (In percent of GDP, unless otherwise indicated)

article image

Derived as [r – g – ρ(l+g) + εα(l+r)]/(l+g+ρ+gρ) times previous period debt stock, with r = nominal effective interest rate on external debt; ρ = change in domestic GDP deflator in US dollar terms, g = real GDP growth rate, e = nominal appreciation (increase in dollar value of domestic currency), and a = share of domestic-currency denominated debt in total external debt.

The contribution from price and exchange rate changes is defined as [-ρ(l+g) + εα(l+r)]/(l+g+ρ+gρ) times previous period debt stock, ρ increases with an appreciating domestic currency (ε > 0) and rising inflation (based on GDP deflator).

Defined as current account deficit, plus amortization on medium- and long-term debt, plus short-term debt at end of previous period.

The key variables include real GDP growth; nominal interest rate; dollar deflator growth; and both non-interest current account and non-debt inflows in percent of GDP.

The implied change in other key variables under this scenario is discussed in the text.

Long-run, constant balance that stabilizes the debt ratio assuming that key variables (real GDP growth, nominal interest rate, dollar deflator growth, and both non-interest current account and non-debt inflows in percent of GDP) remain at their levels of the last projection year.

Table 5.

Country: Public Sector Debt Sustainability Framework, 1998–2008 (In percent of GDP, unless otherwise indicated)

article image

Indicate coverage of public sector, e.g., general government or nonfinancial public sector. Also whether net or gross debt is used.

Derived as [(r -π(l+g) – g + αε(l+r)]/(l+g+π+gπ)) times previous period debt ratio, withr= interest rate; π = growth rate of GDP deflator; g = real GDP growth rate; α = share of foreign-currency denominated debt; and ε = nominal exchange rate depreciation (measured by increase in local currency value of U.S. dollar).

The real interest rate contribution is derived from the denominator in footnote 2/ as r – π (1+g) and the real growth contribution as -g.

The exchange rate contribution is derived from the numerator in footnote 2/ as ag(l+r).

Defined as public sector deficit, plus amortization of medium and long-term public sector debt, plus short-term debt at end of previous period.

Derived as nominal interest expenditure divided by previous period debt stock.

The key variables include real GDP growth; real interest rate; and primary balance in percent of GDP.

The implied change in other key variables under this scenario is discussed in the text.

Real depreciation is defined as nominal depreciation (measured by percentage fall in dollar value of local currency) minus domestic inflation (based on GDP deflator).

Long-run, constant balance that stabilizes the debt ratio assuming that key variables (real GDP growth, real interest rate, and primary balance in percent of GDP) remain at their levels of the last projection year.

27. An important caveat to this approach, however, is the implied sample selection bias. Simply put, Figure 1 shows that, conditional on a debt crisis occurring (in the following year), the underlying variables move by about one standard deviation in the two years prior to the crisis. But it says nothing about the likelihood of such shocks (or a crisis) occurring. The results reported in Assessing Sustainability (SM/02/166, Box 5) imply that the likelihood of a one standard deviation shock to any of the underlying variables is approximately 15 percent (so that, assuming serial independence, the likelihood of a two-year sequence of such shocks ranges is about 2 percent). Allowing explicitly for serial correlation in the underlying variables, and for correlations of shocks across them, however, requires a stochastic simulation approach.

B. Stochastic Simulation Approach

Probability Density Function of Debt Outcomes

28. Instead of applying individual shocks to the baseline scenario and checking the sensitivity of the debt dynamics (as is done in the current sustainability framework), an alternative approach would be to compute the explicit probability density function of the possible outcomes of the debt ratio using stochastic simulation.18

29. Figure 2 shows a typical probability distribution function taken from a stochastic simulation exercise for 41 emerging market countries that are either listed in the EMBI+ index or subject to S&P sovereign ratings, using data over the period 1980–2000.19 In the particular example depicted, the baseline scenario (created for the purposes of the simulation by setting all variables to their historical averages) projects an increase in the debt ratio from 18 percent of GDP to 27 percent of GDP. In the initial period, debt outcomes are distributed tightly around the mean. As the projection horizon lengthens, uncertainty increases so that the distribution becomes more fat-tailed and extreme outcomes cannot be ruled out.

Figure 2:
Figure 2:

Stochastic Simulation of Debt Ratio

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

30. This approach has the advantage of providing an explicit “fan chart” of possible debt outcomes, weighted by the likelihood of their occurrence. But it also has drawbacks relative to the approach of the sustainability framework (which simply traces the effects on the debt dynamics of standard shocks to the main macroeconomic parameters). Beyond the computational burden, the main drawback arises from the lack of sufficient data or stable time series for many countries. Indeed, because of structural breaks such as transition from planned economy or extreme events, including hyperinflation, there is often barely enough data to compute even the requisite means and variances, let alone estimate (rather than impose) the underlying probability distribution.20 Moreover, the technique is somewhat of a “black box,” and there is a risk that the results might be given greater weight than is warranted by the quality of the underlying data.21 For these reasons, stochastic simulations are not being proposed as the mainstay of the standard sustainability template at this juncture. Nonetheless, the approach shows some promise and merits further experimentation in its application to individual countries, at least for countries where sufficient data are available and for which debt dynamics are of particular concern.22

Calibration of Sensitivity Tests in Standard Template

31. Regardless of whether the stochastic simulations form part of the sustainability analysis, the results of the simulation exercise can be used to gauge whether the sensitivity tests incorporated in the standard template are either too extreme or too benign to capture adequately the possible debt dynamics.

32. In particular, it is possible to use the probability density function (obtained via the stochastic simulations above) to calculate the probability that the actual debt ratio, d, will exceed the upper bound implied by the (most extreme) sensitivity test, ds1. If this probability—which is sometimes called the p-value, ps1=pr(d>ds1) —is very low, then the sensitivity test may be considered extreme in that there is little chance of the actual debt ratio ever reaching the upper bound. Conversely, if the p-value is too high, then the sensitivity tests in the standard template may be too benign in the sense that the actual debt ratio could easily exceed the supposed probabilistic upper bound. A natural benchmark against which to gauge the sensitivity tests is the 95 (or 99) percent confidence interval—that is, the debt ratio, d95u, whose p -value is 5% (see Figure 3(a)). The difference between the debt ratio implied by the sensitivity test and the 95 percent confidence interval, D1≡ds1−d95u>(<)0 provides a metric for how extreme (or benign) the standard sensitivity test should be considered (Figure 3(b) and 3(c)). 23

Figure 3.
Figure 3.

Statistical Bounds and P -values

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

Source: Staff estimates.

33. One caveat to the calibration exercise concerns the assumed baseline. For the calibration exercise, the baseline projection is created by setting all variables to their historical averages. To the extent that actual baseline projections are often over-optimistic (see Appendix I), the results of the stochastic simulation are likely to be biased towards finding that the shocks in the standard template are too extreme and the associated p -value too small (Figure 3(d)).

34. The simulations yields a positive value for D1 for 34 out of 41 countries in the sample, with a median value of 15 percent of GDP. In other words, the debt ratio implied by the most extreme sensitivity test is some 15 percent of GDP greater than the 95 percent confidence bound implied by the simulated probability distribution. Even allowing for some bias in the baseline scenario (for instance, the analysis in Appendix I indicates a bias of about 7 percent of GDP for the projection of upper-middle income countries’ external debt ratio), the stochastic simulations suggest that the most extreme sensitivity test in the template is indeed quite extreme.24 This can also be seen by calculating the p -value associated with the most extreme sensitivity test. Figure 4 graphs the frequency distribution (across the sample of 41 countries) of the implied p-values. In almost three-quarters of the countries, the p -value associated with the most extreme sensitivity test is one percent or lower.

Figure 4.
Figure 4.

Frequency Distributions of P -values of First, Second and Third Most Extreme Sensitivity Tests 1/

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

1/ Crisis episodes covered in the sample include Algeria (1991), Argentina (1982, 2001), Bolivia (1980), Brazil (1983), Chile (1983), Costa Rica (1981), Dominican Republic (1981), Ecuador (1982), Egypt (1984), El Salvador (1981), Guatemala (1986), Indonesia (1998), Jordan (1989), Mexico (1982), Morocco (1983), Nigeria (1982), Pakistan (1998), Panama (1983), Paraguay (1986), Philippines (1983), South Africa (1985), Trinidad and Tobago (1988), Uruguay (1983) and Venezuela (1983,1995).

35. An analogous analysis can be undertaken for the second (and third) most extreme sensitivity tests in the standard template. For the second most extreme test, in three quarters of the countries, the p -value is less than 5 percent (while countries for which the p -value is less than 1 percent account for about 40 percent of the sample). For the third most extreme test, in three-quarters of the countries, the p -value is less than 20 percent (while countries for which the p-values is less than 1 percent account for about 30 percent of the sample).

C. Summary

36. To sum up, the shocks assumed in the stress tests of the current sustainability template are realistic inasmuch as they mimic the movements of growth, interest rates, and the U.S. dollar value of the GDP deflator that have been observed in the run-up to previous debt crises. In terms of providing probabilistic upper bounds to the evolution of the debt ratio, the most extreme sensitivity test typically provides an upper bound that is likely to be exceeded with only very low probability (less than 1 percent)—although, taking account of the likely bias in the baseline projection, the true probability should be greater. The second-most extreme sensitivity test provides an upper bound that is likely to be exceeded with a probability of about 1 to 5 percent (so that the upper bound represents a 95- to 99-percent confidence interval). Overall, the analysis suggests that the sensitivity tests are calibrated appropriately, in terms of both the magnitude of the shocks and the probabilities associated with the debt outcomes that result from the assumed shocks (especially, the second-most extreme test). Finally, stochastic simulation analysis while still experimental, shows some promise for calculating explicit probability density functions of the debt dynamics.

IV. Scenario Analysis

37. As argued above, the sensitivity tests are calibrated appropriately to provide a probabilistic upper bound on the evolution of the debt ratio. Nevertheless, they may be too extreme to warrant a policy response—inasmuch as it makes little sense to undertake significantly greater adjustment, or to seek a debt restructuring which could be highly disruptive, in anticipation of a shock that occurs with less than 5 percent probability.

38. This suggests that it may be useful to supplement the current set of sensitivity tests with some additional scenarios that consider less extreme—and thus more probable—shocks to the key parameters. It bears emphasizing that the proposed scenarios would supplement, not supplant, the current set of sensitivity tests. Indeed, it is useful to make a clear distinction between them. The current sensitivity tests are intended to provide a probabilistic upper bound on the evolution of the debt ratio (so that outturns worse than these bounds should be extremely rare). They were introduced, in part, to give greater credibility to the projections undertaken at the Fund, especially in light of cases in which outturns were persistently worse than the “bad case scenario” depicted in earlier sustainability analyses. These tests are intended to underscore the potential risks to sustainability, and in a program context, demonstrate that the Board is cognizant of these risks in approving the program. Moreover, the analysis in the staff report might consider what are the risks of reaching this upper bound under existing and prospective economic conditions and policies, and what particular configurations of shock in the specific country context could result in such outcomes.

39. The proposed new scenario analysis would depict one or two additional scenarios to the baseline, each of which would have a relatively high (20–30 percent) likelihood of being realized. The discussion could then focus on whether—under the primary balance projected in the baseline—there would still be an adequate decrease in the debt ratio, as well as the possible contingency measures that the authorities intend to take if the adverse scenarios materialize. Specifically, the scenario would assume a configuration of shocks that results in an (approximately) one-year, one-standard deviation decline in the growth rate of output.25 Instead of applying this shock mechanically, however, country teams will be encouraged to model the reasons behind this shock and the response of the economy (in terms of interest rates, the exchange rate, and the primary balance) to trace out the resulting debt dynamics.26 In particular, the scenario should be internally consistent.27 For instance, the growth slowdown might be accompanied by a widening primary budget deficit but a narrowing current account deficit. To the extent that the exchange rate is assumed to depreciate (because of lower money demand as activity weakens), the domestic currency value of the existing external debt stock will increase, but this will be partially offset by the likely improvement in the current account balance. If confidence is eroded because of weakening activity, interest rates may be expected to increase, worsening the debt dynamics. Rising debt ratios themselves may elicit widening risk premia and higher interest rates.28

40. An alternative would be to allow country teams even greater latitude in devising the country-specific scenario. For instance, it could be calibrated to result in an increase in the debt ratio that is half way between the baseline and the most extreme sensitivity test, with the team modeling the combination of events that would be most likely to result in this outcome. This approach would have the benefit of directing the analysis to the weakest elements of economic policy (and the most relevant exogenous risks). By doing so, the policy debate would naturally turn to what can be done about these risks.

41. Since the scenario (in either formulation) would be country-specific, it is not possible to calculate precisely the likelihood of the associated shocks. As reported in Assessing Sustainability (SM/02/166), however, the probability that the growth rate falls by one standard deviation is approximately 13–15 percent. Stochastic simulations (along the lines described above) likewise yield a p-value of the debt outcome under a one-standard deviation shock of about 15–20 percent (or higher) for most countries.29 This suggests that such a scenario can occur with sufficiently high probability that it would be prudent to consider possible policy responses. Articulating such a consistent scenario, tailored to the country’s specific circumstances, should provide a good basis for discussions with the authorities on the main risks to the economy and to the public and external debt dynamics. The staff report would thus be expected to discuss the authorities’ intended response should the adverse scenario materialize. This would also help integrate the debt sustainability assessments with the rest of the staff’s analysis and, to the extent that the authorities participate in defining the scenario, help increase both their ownership and technical understanding.

42. Another issue concerns the policy assumptions underlying the existing configuration of baseline and stress tests. The existing “historical case” scenario, in which all of the underlying variables, (including the primary or non-interest balance) assume their historical values, by construction; ignores any factors that may have subsequently led to a deterioration of the debt dynamics, and any measures that may have been taken since the historical period. In contrast, the baseline scenario may incorporate the measures required to avoid a deterioration of the debt dynamics, including not only those already implemented but also those whose enactment or implementation is only anticipated. In such instances, neither the baseline scenario nor the historical performance scenario captures the debt dynamics that would prevail if the required measures were not adopted. It may therefore be useful to examine an additional scenario depicting explicitly the consequences of “no policy changes”—that is, excluding the revenue effects of unenacted changes to tax rates or bases and under the assumption that all currently mandated obligations are honored. A comparison of this “no policy change” scenario to the baseline projection would give an indication both of the expected gain from programmed adjustment and the ambitiousness (or realism) of the baseline.

43. Finally, the template does not take advantage of information on market expectations to assess the plausibility of the macroeconomic assumptions. For financial variables (exchange rates and interest rates), the forward or futures values can be used, while for growth and inflation, consensus forecasts may be available. Such indicators, of course, do not exist for many countries, but where available it would be useful to incorporate them as an alternative scenario, accompanied by an explanation of any differences between this scenario and the baseline.30

V. Contingent Liabilities

44. Experience in recent crises suggests that contingent liabilities are often an important source of increases in public indebtedness.31 Accordingly, one of the sensitivity tests in the sustainability template examines the effect on the public debt dynamics of a realization of contingent liabilities, specified as an exogenous increase in the debt ratio of 10 percent of GDP. This specification is not linked to any estimate of the magnitude of such contingent liabilities. Moreover, no explicit consideration is given to the factors affecting the likelihood of such liabilities being realized, included the country’s financial sector vulnerabilities and other shocks examined in the template (e.g. to growth, interest rates, or the exchange rate). This section explores possible refinements to the contingent liability sensitivity test.

45. By their nature, contingent liabilities are difficult to measure, taking a wide variety of forms (Table 2). When estimates of possible contingent liabilities are available, these may be incorporated directly into the public debt sustainability framework.32 In the absence of such information, country teams should nonetheless examine those that may potentially be important. In most cases, the financial sector constitutes the most important source of contingent liabilities on the government, which often backs deposits and other liabilities of ailing banks through explicit deposit insurance schemes, or through implicit guarantees on liabilities in an attempt to limit the loss of confidence during periods of financial turbulence (Claessens and Klingebiel (2002)). Governments also provide liquidity support, assume bad loans, and inject equity capital in the event of financial sector distress, with obvious implications for the sustainability of public debt,33 while support in the form of regulatory forbearance can lead to larger fiscal costs in the future (Karacadag and Manzer (1997)).

A. Stress-Test Approach

46. One approach to estimating potential contingent liabilities would be to stress test individual bank balance sheets (for the entire or at least most of the banking sector) against a variety of risk factors, as outlined in Honohan (1999). After assessing the extent to which the components of the balance sheet are affected by a variety of risk factors (such as changes in the exchange rate, property market values, general economic downturn, and other shocks), capital positions of various banks and the system-wide capital deficiency could be computed under alternative sizes of shocks and associated probabilities, along with an estimate of the fiscal cost of bank recapitalization.

47. Such an approach would be in line with the stress testing exercise undertaken in the context of Financial Sector Assessment Programs (FSAP). The findings of the FSAP missions could thus provide valuable input into the debt sustainability framework in assessing the likelihood of emergence of any potential financial sector problems, as well as the associated cost of fiscal contingency arising from these problems. However, given the voluntary nature of the FSAP, there will be many instances in which an FSAP has not recently been undertaken for the country or the stress-test results are otherwise unavailable for reasons of confidentiality. In these cases, it may nevertheless be possible to “piggy-back” on current efforts at enhancing the continuous monitoring of the financial sector to obtain an estimate of the possible contingent liabilities.34 For instance, in the context of Article IV missions (or through small, dedicated parallel missions), especially in countries that are identified as being particularly vulnerable, it may be possible to stress test individual bank balance sheets to obtain a rough estimate of the projected capital shortfall (and hence the potential call on the public purse) in the face of various shocks to economic activity, interest rates, or the exchange rate. Indeed, such stress-testing exercises have already been undertaken in several countries during Article IV missions.35 Since the feasibility of the exercise depends upon the availability of individual bank balance sheet data, in cases in which there are confidentiality concerns with the provision of such information, the authorities could be urged to undertake the stress-tests themselves, and inform country teams of the aggregate results and the implied contingent liability for use in the debt sustainability analysis.36

B. Cross-Country Evidence

48. The discussion above outlines one possible approach to refining the estimate of the contingent liabilities emanating from the financial sector. Given resource and other constraints, however, such an approach will not always be feasible or even desirable. In the absence of such information, cross-country evidence on past banking system crises can be used to derive at least crude estimates of the possible contingent liabilities in particular countries.

49. Across a sample of almost 60 crises, banking crises have on average added some 14 percent of GDP to government debt, while three quarters of the countries in the sample spent up to 20 percent of GDP (Figure 5).37 Costs tended to be higher in developing or emerging market countries, especially when the banking crisis was accompanied by a currency crash.38

Figure 5.
Figure 5.

Estimated Cost of Banking Crises 1/

(in percent of GDP)

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

Source: Caprio and Klingebiel (2003), Dziobek and Pazarbasioglu (1997), Honohan and Klingebiel (2000), IMF (2003), Zoli (2001), and staff calculations.1/ The dates in parentheses for each country indicate the estimated start date of the banking crises.

50. Beyond these averages, studies of banking crises suggest that initial macroeconomic and financial sector conditions are an important determinant of cross-country differences in fiscal costs of crises (SM/03/50), with weaknesses in financial institutions increasing their vulnerability to various shocks and raising the costs of resulting financial crises.39 Table 3 reports the stylized facts on a sample of about 50 banking crises (a subset of the sample for which sufficient data are available). Other factors equal, the cost of banking crises are higher in countries with larger financial systems (as measured by the ratio of bank credit to GDP), greater credit risk exposure (proxied by the ratio of private sector credit to deposits), faster credit expansion prior to the crisis (as measured by an increase in the ratio of credit to GDP), and a higher share of the state in the banking system.40 The data also indicate that the median cost of banking crises is much greater for countries with lower-than-average capital adequacy ratio.41

51. Drawing on earlier studies of banking crises, more formal cross-country regressions can be used to relate the fiscal costs of crises to various indicators of a country’s initial macroeconomic and financial conditions, though scope for extensive analysis is circumscribed by the lack of historical data on indicators of banking system soundness. A further complicating factor is that much of the variation in the cross-country evidence is accounted for by the particular resolution strategy adopted—such as the provision of unlimited deposit guarantees, open-ended liquidity support, repeated recapitalizations, and regulatory forbearance (Honohan and Klingebiel (2000))—which, of course, is difficult to establish ex ante. With these caveats, Appendix III presents some regressions which could be used by country teams to infer likely costs of a banking crisis given the country’s macroeconomic and financial characteristics. While the various specifications imply somewhat different costs, the approach would nevertheless represent a marked improvement over the blanket assumption of a 10 percent of GDP contingent liability shock, as assumed in the current version of the sustainability template.

C. Summary

52. While still work in progress, the overall conclusion is that it should be possible to bring to bear available information on financial system soundness to public debt sustainability assessments. At the same time, some caution is required to ensure that the mere computation of the possible contingent liabilities does not create moral hazard by signaling the government’s willingness to bail out the banking system. Given the nature of the problem, this should be done on a case-by-case basis, rather than being incorporated in the standard debt sustainability template.42 Where estimates of possible contingent liabilities are available, these can be incorporated into the sensitivity test directly. Where they are not available, with the active cooperation of the supervisory authorities, some ballpark estimates of the potential capital deficiency of the banking system—and hence the call on the public purse if such funds cannot be generated by the private sector—can be established by stress-testing bank balance sheets. When this is not feasible, or is unjustified given the resource costs, cross-country econometric evidence can be used to derive some ballpark estimate of the possible fiscal liability of a banking crisis.

53. Finally, despite the emphasis here on contingent liabilities arising from banking sector restructuring, it bears reiterating that there are many other sources of contingent liabilities, including debt contracted by state enterprises and parastatals but not formally guaranteed by the state. Other examples would be contingent liabilities faced by transition economies because of unsettled claims of the pre-transition period, or claims that arise from possible court rulings. In countries where such liabilities may be important, country teams would be encouraged to incorporate any available information into the debt sustainability framework.

VI. Interpreting Debt Ratiost

54. A key challenge in making the sustainability framework operational lies in interpreting debt ratios, both actual levels, and those generated in various scenarios and sensitivity tests. It is useful, at the outset, to recall that the framework is intended to establish whether the debt ratio is likely to increase (including because of unexpected shocks) to a point at which a crisis becomes imminent. It does not, however, model the crisis itself. In particular, this means that a stable, or even declining, debt ratio at the end of the forecast horizon is of little comfort.43 Rather, there are three key questions to determining whether the country may enter a “danger zone”: (i) Is the debt ratio, either along the path or at the end of the horizon, so high that the country is vulnerable to a crisis? (ii) Can the country plausibly generate and maintain the primary surpluses required over the medium term to at least stabilize the debt ratio? (iii) Are the gross financing needs required along the path so large that the country may run into a funding crisis:? While recognizing that assessments of sustainability inevitably involve some element of judgment, this section examines how debt levels may be put into perspective.

A. Debt Thresholds

55. Assessing Sustainability (SM/02/166, Appendix 1) took a first stab at establishing debt thresholds, examining the level of external debt at which debt “corrections” occurred, using both frequency distributions and binary recursive trees.44 The analysis there found an appreciable increase in the likelihood of a debt correction when the external debt level rises above 40 percent of GDP (with a somewhat higher threshold for countries that are especially open to international trade). This does not, of course, imply that countries with debt ratios above this threshold will necessarily have a crisis: the conditional probability of a crisis is about 15–20 percent, so that another way of looking at the results is that the probability of not having a crisis is 80 percent (even when the debt ratio is above 40 percent of GDP). 45 Recent research work has tried to refine this analysis, dividing countries according to their Institutional Investor ratings.46 Excluding the advanced economies, which enjoy continuous access to capital markets, and the (mostly low-income) countries with almost no access to capital markets, the results suggest using a rule-of-thumb for external debt of 35 percent of GDP, with countries with a history of high inflation or previous defaults being more vulnerable to a crisis.

56. Several recent crises, however, have involved unsustainable public debt dynamics and have been manifested in sovereign debt servicing difficulties or defaults. A useful complement to the analysis in SM/02/166, therefore, is to examine debt ratios at which sovereign debt defaults have occurred, as classified by Standard and Poors.47

57. A simple starting point for identifying such danger zones is just the frequency distribution of the debt levels at which sovereign debt crises have occurred. As shown in Figure 6, more than half of sovereign debt crises have occurred at public (or external) debt ratios of below 40 percent of GDP, and two-thirds at public (or external) debt ratios below 60 percent of GDP.

Figure 6.
Figure 6.

Public and External Debt Ratios in Sovereign Debt Crises.

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

58. Going beyond these frequency distributions, ideally, an “early warning” model of sovereign debt crises could be developed, relating the likelihood of a crisis to the level and characteristics (such as maturity and currency composition) of the debt as well as country-specific factors. Such a model could be inverted to obtain the maximum debt level consistent with a given probability of triggering a crisis. As usual, however, choosing the trigger probability (and hence the maximum debt threshold) involves a fine balance between missing crises and generating false alarms.

59. At a practical level, the scope for econometric work is severely limited by the lack of historical data on many of the factors that were probably critical in determining whether the country experienced a debt crisis—including the currency composition of the debt, its maturity structure, whether it was indexed, and whether it was held by the domestic financial system, other domestic residents or by foreign investors. Within these constraints, and with a significant effort at collating existing data from a variety of sources, a recent research project has examined the feasibility of estimating an early warning model of debt crises (Box 4). The results suggest that the external debt-to-GDP ratio, the short-term debt-to-reserves ratio (remaining maturity), the debt service burden, the current account deficit, and inflation are all positively related to the probability of experiencing a sovereign debt crisis, while real growth and trade openness reduce that probability.48 Overall, the model correctly predicts 74 percent of all debt crisis outbreaks one year in advance while sending only 6 percent false alarms. Although this work represents an important advance, as emphasized below, identifying “danger zones” is still more of an art than a science, with a large element of judgment required.

An Early Warning Model of Sovereign Debt Crises

Recent work by Manasse, Roubini, and Schimmelpfennig (2003, hereafter MRS) represents a valiant effort to collate data from various sources and to analyze the factors that affect the probability of a sovereign debt crisis. Their work includes information on 37 countries with market access for the period 1970 to 2002, with the sample period for transition countries being 1995 to 2002, a sample that includes 31 default entries.1

In MRS, a country is defined to be in default if it is classified as being in default by Standard & Poor’s or if it receives a disbursement in the first year of a large Fund arrangement (over 100 percent of quota). Explanatory variables are drawn from many different sources. They examine the evolution of some of these variables using an event study approach, conduct binary recursive tree analysis, and run logit regressions. These explanatory variables are divided into six groups—external debt variables, public debt variables, variables from the Fund’s currency crisis Early Warning System, other macroeconomic variables, fiscal variables, and political economy variables. Given degrees of freedom considerations, MRS pursue a three-stage approach to determine the final specification of logit regressions. In the first stage, they regress each variable against the sovereign default indicator; in the second, they pool the best performers (in terms of respective z-values) within each group and run multivariate logit regressions; in the third, they combine the best performers from each group to form a ‘reduced’ model.2

The MRS logit EWS correctly predicts one year in advance 74 percent of default entries in the sample and sends only 6 percent false alarms. The logit EWS does not capture well most of the Asian Crisis. The regressors have expected signs: external debt to GDP ratio, short-term debt to reserves ratio, debt service burden, current account deficit, and inflation are positively associated with the probability of a sovereign debt crisis; higher real GDP growth and openness reduce the probability. These results are robust to various sensitivity tests.

These results may be useful in designing an early warning system for sovereign debt crises, although in practice it may be hard to apply to many developing countries because the underlying data are not available. For that reason, and because the independent variables may have to be predicted in an early warning application, a more parsimonious—but possibly varying across country groups (i.e. industrial, developing, etc.)—specification may be preferable for interpreting debt ratios in the sustainability framework.

1. Given the MRS definition, a country can be ‘in default’ for an extended period time—the average for the sample being 5.5 years—as Standard and Poor’s default episodes often continue for several years.

2. The logit regression specification used by MRS can be represented as:
Pr⁡(Dt)=11+exp−[α+β1Dt−1+β2Dt−1+β3(1−Dt−1)Xt−1]

where Dt represents sovereign default at time t, which is conditioned on whether there was a default the previous year (Dt-1), certain variables that enter the regression independently (Zt-1), and others that are interacted with the default indicator (Xt-1).

60. For the purposes of the debt sustainability framework—which seeks to assess debt dynamics over the next five years—an important requirement is that the model not use as explanatory variables indicators that are themselves difficult to forecast. 49 To this end, there has been some experimentation with more parsimonious specifications (using the same data set).50 Specifically, for assessing the level of external debt, the probability of a debt crisis may be correlated to the external debt ratio, the ratio of short-term debt to reserves, and the trade openness of the country. For public debt, the probability of a crisis is correlated to the public debt ratio, the primary balance, and the revenue ratio.51 Despite the simplicity of the model, the external debt model predicts 88 percent of debt crises while the public debt model predicts all of the observed debt crises (within the more limited sample given by the availability of public debt data), where the model is deemed to be “predicting” a crisis whenever the implied probability exceeds 0.2. However, the models also call a high percentage of false alarms (about 50 percent). Raising the trigger probability from 0.2 to 0.6, lowers this percentage of false alarms to about 9 percent, but also the percentage of crises called correctly (to about 52 percent)—a somewhat lower predictive performance compared to the full-blown early warning model outlined above.

61. Figure 7 depicts the fitted probabilities of a debt crisis implied by the model.52 For instance, for a country with a trade openness ratio (sum of exports and imports) of 100 percent, a short-term debt ratio of 100 percent, and an external debt ratio of 50 percent, the probability of a crisis is 0.26; increasing the external debt ratio to 70 percent doubles the probability to 0.53, while doubling the short-term debt ratio raises the probability to 0.32. As indicated by the hyper surfaces in Figure 7, the corresponding probabilities are higher for economies that are more closed to trade.

Figure 7:
Figure 7:

Fitted Probability of Sovereign Debt Crisis As a Function of External Debt, Short-term Debt, and Trade Openness

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

B. Debt-Stabilizing Primary Surplus

62. While such “early warning” models can give an indication of rising vulnerability, there are clearly many other factors that determine whether a country’s debt is sustainable. Discussions with analysts in private financial institutions (Appendix IV), for instance, suggest that, beyond the absolute level of debt, markets are often concerned about the ability of the government to generate the requisite primary surplus to stabilize the debt ratio (given market expectations of growth and costs of financing). This depends to a large degree on political will and social cohesion—factors that are not easily captured within a formal framework. Moreover, it is not only the short-run that matters, but also whether the country can sustain the required primary surplus over the longer-run (or generate larger surpluses in the short-run that reduce the level of debt and thus the required long-run debt-stabilizing surplus).

63. As a simple enhancement of the standard framework, therefore, it is proposed that the primary surplus required to stabilize the debt ratio (generated in the baseline scenario and the sensitivity tests) be reported. This would be defined as the constant, “steady state” primary surplus which stabilizes the debt ratio at its value at the end of the projection horizon, assuming that all other parameters are constant at their end-horizon values as well.53 Conceptually, this long-run required primary surplus may be more easily interpreted than the debt ratio itself, since the additional adjustment effort can be viewed against the country’s historical performance.54

C. Financing Needs

64. As noted above, the sustainability framework is not intended to model the actual dynamics of a debt crisis. In practice, a debt crisis is usually manifested as a funding crisis— that is, a very steep rise in marginal funding costs to the point that the borrower effectively loses access to the capital markets and is unable to roll over maturing debt. Although the sensitivity tests in the standard template include an interest rate shock and a combined shock, these are supposed to capture rising average costs of funding (which, in turn, worsen debt ratios), not the dramatic increase in marginal interest rates as the crisis erupts. Modeling the latter is enormously complex, not least because it is likely to depend very much on the particular circumstances prevailing at the time of crisis, including the state of the capital markets themselves.

65. Accordingly, the revised version of the template takes the simpler approach of tracking gross financing needs associated both with the baseline scenario and the alternative scenarios and sensitivity tests.55 For external debt, financing needs are expressed in billions of U.S. dollars, since it is generally the country’s financing needs relative to available emerging market funds that are of relevance. Given projections of the likely evolution of total emerging market funds (as well as possible regional “allocations”) this information can help inform judgments about risks to the country’s debt sustainability. For public debt, the analysis of gross financing needs may be more complex, for instance if part of the market is captive (e.g. domestic pension funds or banks that have requirements on government bond holdings) and the distinction between domestic and non-resident holders of government paper is blurred. The public debt template tracks gross financing needs, though assessing the rollover risk may be very dependent upon the country-specific circumstances.

D. Summary

66. While the framework helps inform judgments about the sustainability of a country’s public and external debt, it does not remove the need for such judgments. Accordingly, sustainability analysis needs to place the debt dynamics implied by the framework in the particular context of the country. Some of the relevant factors relate to the country: its vulnerability to shocks, its macroeconomic management, and any prior history of debt defaults or crises. Other factors relate to the debt itself, including its currency composition (or indexation) and maturity structure as well as whether it is held by foreign investors, the domestic banking system, pension funds, or other domestic residents. While early warning models and analyses of gross financing needs can capture some of these factors, both analytical and data shortcomings mean that some element of judgment is unavoidable.

67. Moreover, the template is primarily a medium-term framework, intended to identify growing vulnerabilities so that corrective policies can be taken in a timely manner. It does not, however, model the dynamics of a crisis itself, in particular the possible nexus between debt, the banking system, and the exchange rate. In some previous crises, for instance, default or restructuring of public debt resulted in de-capitalization of domestic banks which were holding (or were effectively forced to hold) government paper. (Conversely, banking system restructuring costs can lead to significant increases of public indebtedness.) Likewise, rising concerns about debt sustainability may put downward pressure on the exchange rate as capital inflows dry up, in turn forcing the government to issue shorter-maturity and foreign currency denominated or indexed debt. Any collapse of the exchange rate in a crisis then adds significantly to the debt burden, while raising interest rates to defend the currency risks worsening the dynamics of the domestic currency-denominated debt.

68. This suggests that, when the debt sustainability framework suggests rising and potentially risky debt levels, country teams should try to place the debt dynamics analysis within the context of the country’s overall vulnerability. In examining the implications of higher debt levels, associated with various possible shocks, they should take into account the potential feedback effects on the banking system, on the exchange rate, and on the availability and pattern of external and domestic financing.

VII. Proposed Changes and Enhancements

A. Design, Use, and Resource Implications of the Standard Template

69. In refining the template, it is worth recalling its basic purpose, namely to instill greater discipline in debt sustainability assessments undertaken by the Fund. To this end, the template incorporates four basic features: (i) a convenient format for laying out the key assumptions underlying the baseline projection; (ii) a method for calculating the “95 percent” probabilistic bound around that baseline; (iii) a framework for articulating alternative scenarios; and (iv), a means for tracking gross financing needs.

70. But the template is a tool for better sustainability assessments, not an end in itself. The intention is to promote best practices, not to impose a strait-jacket of the “least common denominator.” Therefore, when application of the standard template does not fulfill this objective, or when better methods are available, country teams should be encouraged to explore the use of alternative frameworks—as long as these incorporate the four basic features of the standard template enumerated above. For instance, as discussed in section III, it may be possible to use stochastic simulations to calculate explicit probability density functions from which the precise 95 percent (or any other desired) confidence interval can be established. Likewise, in the major industrialized countries, the nature of the potential debt sustainability problems are often not well captured by the standard template. In other cases, because of extreme events, such as hyperinflation, in the country’s history, the practice of using the past 10 years to calculate the parameters of the shocks leads to extreme confidence intervals.56

71. Of course, in the vast majority of cases, the standard template will be the most suitable, most convenient, and least resource-costly framework for country teams to adopt. Indeed, the new version of the template automatically calculates the standard sensitivity tests, picks the two most extreme tests, and plots them, along with the baseline scenario and the associated gross financing needs. The template also computes debt-stabilizing primary balances; Box 5.

72. The convenience of the standard template should allow country teams to focus most of their efforts—and discussions with the authorities—on formulating the baseline and alternative scenarios. Since country teams typically undertake medium-term projections, the baseline projection required for the debt sustainability template entails little additional work. The addition of the new country-specific alternative scenario, however, especially if modeled properly, will require significant additional effort. While it is difficult to quantify the resource costs explicitly, the work required is presumably of the same order as that entailed in formulating the medium-term baseline projection.

73. This suggests that a two-pronged approach might be appropriate. In cases where the baseline projection—and the sensitivity tests—indicate that the debt ratio is expected to remain well within prudent levels (as discussed in section VI) and that gross financing needs are manageable, the alternative scenario would be unnecessary (or could be calculated purely mechanically).57 In all other cases, however, given central importance of debt sustainability to the Fund’s work, and that articulating the baseline and alternative scenarios should help integrate these assessments with the rest of the staff’s analysis, it may be desirable to expect country teams to formulate the alternative scenario as well, recognizing that this will inevitably involve some additional work.

Proposed Enhancement to Debt Sustainability Framework

This Box summarizes the proposed changes and enhancements to the debt sustainability framework:

  • Supplement the current debt bounds sensitivity tests with scenario analysis. These will scenarios based on (i) all variables taking their historical values for the duration of the projection horizon; (ii) the “no policy change” primary balance in the public debt template; (iii) market expectations or consensus forecasts (where available) for macroeconomic variables; (iv) a scenario formulated by the country team in which GDP growth falls by one-standard deviation.

  • Distinguish clearly between the alternative scenarios and the debt bounds sensitivity tests, emphasizing that the latter provide a probabilistic upper bound on the evolution of the debt ratio and are not intended to depict likely outcomes.

  • Enhance the treatment of the contingent liability sensitivity test using any available country-specific information or cross-country evidence on the cost of banking crises.

  • Report the debt-stabilizing primary (or non-interest current account) balance associated with the debt levels in the baseline scenario, the alternative scenarios, and the sensitivity tests.

  • Track gross financing needs for public and external debt (Figure 8).

Mock-ups of the revised external and public debt templates are given in Tables 4 and 5. Following Board discussion, a technical guidance note will be prepared for staff.

Figure 8.
Figure 8.

Country Name: Debt Ratio and Gross External Financing Need, 2002–08

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

Source: IMF staff calculations.

74. Better focus of the Fund’s debt sustainability assessments may also mean paring down on the number of such analyses undertaken by country teams. For instance, to the extent that potential debt sustainability problems in industrialized countries arise from slow-moving demographic changes, it may make little sense to present sustainability assessments every year. Less frequent, but more carefully modeled scenarios (including the complex dynamics of demographics and pension liabilities) may be preferable. Among the bulk of emerging market and developing countries with significant market access, however, the frequency and magnitude of shocks as well as policy reversals warrants debt sustainability assessments as part of the Article IV consultation and for requests for use of GRA resources.

75. Based on these considerations, we do not envisage that implementing the proposals in the paper will have significant resource implications, either for the current year or the medium term. In the event that, against expectations, experience this year indicated this work is creating significant additional demands on staff, an estimate of the extra resources required and a statement on how they would be financed within the Fund’s medium term framework would be brought before the Board for consideration in the FY 2005 budget.

76. The revision of the framework leaves open the issue of over-optimistic baseline projections. It is expected that greater cognizance of possible biases will help discipline the projections undertaken by country teams. One possibility would be to include in staff reports an analysis of the error in the previous year’s projection, especially when the outturn was very different from the projection. The “fresh perspective” in surveillance is also expected to contribute to more realistic projections. Moreover, while over-optimism in projections is clearly an important concern, it is not easily addressed by the design of the framework. Rather, the internal review process will need to scrutinize assumed growth rates and other macroeconomic parameters against both the country’s historical performance (which is conveniently encapsulated in the standard tables) and the cross-country evidence; the analysis of typical projection errors presented in Appendix I should be helpful in this regard.

B. Publication of Debt Sustainability Assessments

77. As regards publication, during its discussion of Assessing Sustainability, the Board noted a tension between the credibility of Fund’s sustainability assessments and the possibility that publication could lead to misinterpretations by the public and adverse market impacts. To date, about 60 percent of the debt sustainability analyses presented in Board documents have been published as part of the staff report or Selected Issues paper (Table 1). In only three cases did the authorities request that the debt sustainability analysis be deleted prior to publication of the staff report, and in a further two cases, the debt sustainability assessment was reported in a stand alone paper so the issue of publication did not arise.

78. The arguments in favor of publication are that the templates do not use any information that is not routinely available to market participants, other than the baseline projection—which is automatically available if the staff report is published.58 Conversely, at the time, there had been no experience with the use of the sustainability template—which was viewed as work in progress and subject to various changes and enhancements—nor was it known how the debt sustainability assessments would be received by markets. As a result, the Board recommended that, pending further experience, the debt sustainability assessments be presented separately and the publication issue revisited.

79. While there may be further enhancements to the framework, there has already been much work on refining the template and valuable experience has been gained over the past year. Moreover, to date there have been only three cases in which the authorities requested that the debt sustainability analysis be deleted prior to publication of the staff report. Nor is the staff aware of any instances in which publication of the debt sustainability assessment led to an adverse market response.

80. Accordingly, it is proposed that, henceforth, there would be a presumption that the public and external debt sustainability assessments would be included in published staff reports. This would mean that the assessments could be in the body of the staff report which, together with the enhancements to the framework itself, would help integration with the rest of the staff’s analysis of economic developments and policy advice. In cases in which the authorities are nevertheless wary of adverse market reactions, the standard deletions policy, which provides for deletion of market sensitive material, would apply.59

VIII. Issues for Discussion

81. This paper proposes a number of refinements and modifications to the debt sustainability framework: (i) supplementing the debt bounds sensitivity tests with scenario analysis (including modeled country-specific scenario, and a “no policy change” scenario); (ii) enhancing the treatment of contingent liabilities in the public debt sustainability template; (iii) reporting the debt-stabilizing primary (or non-interest current account) balances; and (iv) tracking gross financing needs for public and external debt. The paper also suggests that, while maintaining the discipline of the standard template, alternative approaches more tailored to country-specific circumstances might also be usefully considered.

82. In their interventions, Directors may wish to address the following issues:

  • Do Directors agree with a flexible approach to debt sustainability analyses, in particular for industrial countries, with country teams encouraged to experiment with alternative frameworks as long as these incorporate the general features and meet the standards of the sustainability template?

  • Directors may wish to comment on the experience with the template. Have they found the analyses useful, and how could they be made more so?

  • Do Directors support the proposed refinements? Would they be interested in further refinements?

  • Do Directors agree that banking system restructuring costs may represent an important source of contingent liabilities, and that country authorities should be urged to provide teams with sufficient data to refine estimates of such costs?

  • Do Directors agree that debt sustainability analyses should be an integral part of staff reports to which existing publication and deletions policy would apply?

Appendix I Errors in WEO Projections of Public and External Debt

83. The sustainability analysis in the framework is centered on the baseline projections for public and external debt, and a key question is whether these projections are realistic. Since the debt sustainability framework has been in operation less than one year, it is clearly impossible to assess the realism of the baseline projections. However, it may be possible to gain some insight into the typical magnitude of biases in staff projections—and the reasons for them—by examining the properties of the projections underlying the WEO exercise.60

84. Debt dynamics are not projected directly, but rather depend upon a number of underlying variables. Thus, the evolution of the external debt-to-GDP ratio will be a function of the current account, FDI flows, the U.S. dollar value of the GDP deflator, and real GDP growth. Errors and biases in projecting any of these variables would result in a corresponding, and generally compounding, error in projecting the debt ratio. Table 1 reports the average bias (defined as outcome minus projected, and expressed in percent per year or percent of GDP, as applicable) for the key parameters underlying external debt projections.

Table 1.

Errors in World Economic Outlook Projections of External Debt Ratio

article image

Predicted and actual values transformed by z/(1+z) for z >0, and z/(1-z) for z < 0.

Projections starting in 1995

IMF-supported program in year being projected

85. The average error in projecting real GDP growth amounts to about ½ percentage per year, rising to almost 1½ percentage points per year for the low income countries (Table 1, panel 1). For external debt sustainability, however, it is the U.S. dollar value of GDP that matters (Table 1, panel 2). Here the average bias is significantly larger, ranging from 1½ to 2 percentage points per year for the upper-middle income countries to 5½ to 6 percent per year for the lower income countries. The third element in the projection error for external debt is the current account deficit (Table 1, panel 3), where the mean bias ranges from 1½ to 3 percent of GDP (on average, over the projection horizon). Overall, the error in projecting the external debt ratio is about 1 to 1½ percent of GDP per year, so that, at the end of a 5-year horizon, the external debt ratio is some 3 to 4 percent of GDP higher than projected (Table 1, panel 4). Among upper-middle income countries with Fund-supported programs, however, the debt ratio is 7 percent of GDP higher than projected (and 2.2 percent of GDP higher than projected for the low-income countries). Among the low-income countries, the error in projecting the debt ratio is driven almost entirely by the error in projecting the dollar value of GDP. Indeed, in a counterfactual projection in which the dollar value of nominal GDP is assumed to be known, the bias becomes negative—in nominal U.S. dollar terms, therefore, the debt stock is over-predicted in low-income countries. For the upper-middle income countries, however, the bias in projection the debt ratio remains about 7 percent of GDP at the five-year horizon.

86. A similar exercise can be undertaken for assessing projections of fiscal variables. (Table 2). At the general government level, the overall balance is over-predicted by about 1 percent of GDP, ranging up to 2 percent of GDP for countries with Fund-supported programs. Most of this projection error comes from higher expenditures rather than revenue shortfalls, suggesting that weaker-than-expected activity is not the underlying cause of errors in projecting the balance. The error in projecting the government net debt ratio amounts to 1 percent of GDP (at the one-year horizon) for the full sample, but 5 percent of GDP for countries with Fund-supported programs; these results, however, are based on a very small sample of observations and may not be representative.

Table 2.

Errors in World Economic Outlook Projections of Public Debt Ratio

article image

Predicted and actual values transformed by z/(1+z) for z >0, and z/(1-z) for z < 0.

Projections starting in 1995

IMF-supported program in year being projected

Appendix II Stochastic Simulation of External Debt Ratio

87. The purpose of this exercise is to create a simulated probability distribution that can be compared against the output of the standard templates to ascertain the latter’s statistical properties. The richness of this analysis—which allows for random shocks to the baseline and incorporates potential correlations among shocks to forcing variables of debt dynamics-provides statistical underpinnings to test claims regarding risks of unsustainable debt dynamics.

88. To account for potential correlations among forcing variables of external debt dynamics—namely, real GDP growth(e), interest rate(r), non-interest current account balance as a share of GDP(cb), and the rate of change in U.S. dollar value of the GDP deflator(ρ)— an autoregressive data generating process is assumed for these variable. Specifically, denoting the vector of these forcing variables by z = {g, r, cb, ρ}’, the data generating process is modeled as:

( A .1 ) z t = β 0 + Σ k = 1 K β k z t − k + Σ j = 1 J γ j x t − j + ϵ t

where x is a vector of exogenous variables, and ε ~ N(0, Σ) represents a vector of serially uncorrelated normal errors with E(εtεs) = 0 for all t ≠ s . This filtering equation systematically allows both contemporaneous and serial correlations among forcing variables through {βk}k=0K,{γj}j=1J and Σ .61

89. Once the filtering equation (A.1) is estimated from historical data, an N-year random sample of forcing variables, {z˜t}t=1N, can be generated by feeding into the estimated filter thes shocks, {ϵ˜t}t=1N, that are drawn from N(0, Σ) .62 Finally, the corresponding N-year random sample of debt ratios, denoted by {d˜t}t=1N, can be obtained by substituting {z˜t}t=1N into the debt dynamics equation given by:

( A .2 ) d t − d t − 1 = λ t d t − 1 − c b t , w h e r e λ t = r t − g t − ρ t ( 1 + g t ) ( 1 + ρ t ) ( 1 + g t )

This process can be replicated as many times as desired to generate the underlying probability distribution of debt ratio.

90. Once the simulated probability distribution of debt ratio has been obtained, statistical inference on the results of the standard templates can focus on the metric D1=ds1−d95uwhereds1andd95u represent, respectively, the debt ratio emerging from the worst-case scenario of the standard templates and the 95 percent critical value of the simulated probability distribution. A positive value of D1 indicates the outcome of the most extreme shock in the standard templates to be worse than the 95 percent critical value. Equivalently, one may directly examine the probability that simulated external debt ratios are greater than or equal to ds1, denoted by ps1=pr(d≥ds1). The value of ps1 less than or equal to 5 percent would lead to the same conclusion as one reached based on the sign of D1. These statistical inferences at the individual country level can be aggregated to check the robustness of the general application of standard templates across countries. Specifically, the relative share of countries with positive D1 gives an indication of whether, typically, the sensitivity tests are either too extreme or too benign.

91. The sample for this analysis consists of 41 mainly emerging market countries that are either listed in the EMBI+ index or are subject to S&P sovereign ratings, and covers the period from 1980 to 2000.63 Given the relatively short time span of the dataset in annual frequency, a parsimonious specification was estimated for (A.1) over the full sample period by restricting the order of lags for z to 1 (i.e., K=1), while x includes only the lagged value of debt ratio, dt-1 (i.e. J =1)64 Based on the estimated filtering equation, external debt ratios were simulated 10,000 times over the 5-year period beginning from 1990. Accordingly, the standard template tests were conducted over the same period with the key parameters (values of forcing variables of debt dynamics and their standard deviations) being set to their respective historical moments for the 1990s.

92. The comparison of standard templates to stochastic simulation reported here is based on the results of the fifth year in the projection horizon. This was thought to be the most appropriate comparison considering the Fund emphasis on medium-term debt dynamics. The focus on the fifth year can also be justified by the current configuration of shocks in the standard templates, which consist of two-year sequences of two standard deviation shocks. By construction, ds1 for the first two years perturbed with two standard deviation shocks is likely to lie outside the 95 percent confidence interval of the simulated probability distribution. The opposite is likely for the remainder of the projection horizon as no further shocks are assumed to occur, while the impact of initial shocks on debt dynamics tends to dissipate over time. The fifth year of the horizon tends to balance these effects and, since the projection horizon of the template is 5 years, provides a natural point for evaluating the sensitivity tests.

93. Table 1 summarizes the results of stochastic simulation and the standard template analysis in the cross-country context. Panel A reports cross-country mean and median for the bias in the baseline debt ratio of the standard templates, the metric D1 —in absolute terms (column 2), as well as normalized by the actual debt ratio (column 3) and the debt ratio from the historical baseline (column 4)—as well as the probability measure ps1. Panel B reports the test results on whether there are systematic differences in the level and the volatility of forcing variables of debt dynamics between two subgroups of countries, with Group I being defined as countries with positive D1 and Group II as those with negative D1. Results of this exercise are discussed in the main text.

Table 1.

Cross-country Summary of Simulation Results

article image

* signicant at 5%; ** significant at 1%.

in percent

H0: mean = 5% for p1

σx represents standard deviation of x.

Appendix III Cross-Country Econometric Evidence on the Cost of Banking Crises

94. Cross-country evidence on banking crises can be used to derive some ballpark estimates of contingent liabilities in countries in which other information (such as from the FSAP process or from the smaller-scale stress-testing of the banking sector in the context of Article IV missions, or otherwise) is not available.65 The extensive use of econometrics is circumscribed, however, by the limited availability of reliable historical data, not only on indicators of financial soundness but also on the realized fiscal costs of crises.66 A further complicating factor is that the actual costs often depend upon the particular resolution strategy adopted, which is difficult to establish ex ante.

95. With these caveats, this appendix presents some econometric evidence relating the fiscal costs of banking crises to a variety of financial variables, country characteristics, and variables proxying the resolution strategy. It also presents an out-of-sample application of some of the results to a selected number of countries based on the most recently available information on their macroeconomic and financial indicators under alternative resolution strategies. The results of this analysis should be interpreted with caution, however, bearing in mind the small size of the samples and the difficulty of incorporating all relevant variables in one regression, owing to the limited degrees of freedom and lack of sufficient data on many of the key financial system variables.

96. Table 1 reports four regressions, the first two covering a small sample of 16 countries for which data is available on some key indicators of financial soundness (such as capital adequacy and NPLs), and the last two covering a larger sample and using alternative indicators. The first two regressions suggest that large costs of banking crises have been associated with a lower capital adequacy ratio and higher NPLs, and more so for the developing countries (including emerging markets). While having the expected positive sign, larger financial size or higher credit-to-deposit ratio do not appear to contribute to fiscal costs, although in alternative specifications (not reported) both variables had a positive and statistically significant effect. Equations [3] and [4] suggest that existence of a currency crisis accompanying the banking crisis and use of liquidity support, repeated recapitalizations, and various forbearance strategies to resolve the crises add significantly to the fiscal costs (the latter likely by creating additional costs from delaying to close loss-making banks). Depending on the specification used, real interest rates, the credit variables, and the foreign liability position of the banking sector can also add to fiscal costs. All together, the various sets of variables can explain some 55–65 percent of the variation in fiscal costs of banking crises, though there is still substantial underestimation (and, in some cases, overestimation) of the actual costs when the latter well exceeds 20 percent of GDP (Figure 1).

Table 1.

Fiscal Costs of Banking Crises: Regression Results 1/

article image

The bold figures indicate statistically significant coefficients at 95 percent confidence level or higher.

Index of forbearance = Forbearance-A+0.5*Forbearance-B+0.25*Forbearance-C, where:

Forbearance_A =1 when banks were left open in distress for at least 3 months. Forbearance_B =1 when undercapitalized banks were permitted to function. Forbearance_C =1 when prudential regulations were relaxed or the prevailing regulatory framework was not fully enforced for at least 12-months period.
Figure 1.
Figure 1.

Underestimation (+) of the Cost of Banking Crises, Actual-Fitted

(in percent of GDP)

Citation: Policy Papers 2003, 049; 10.5089/9781498329453.007.A001

97. The results presented here suggest that it should be possible to refine somewhat the contingent liability sensitivity test in the public debt sustainability template. In particular, contingent liabilities associated with banking crises appear to be systematically related to country characteristics, such as the vulnerability of the financial system and macroeconomic conditions, although the resolution strategy—which is hard to determine ex-ante—also has a significant impact on any eventual fiscal costs. On the basis of this cross-country evidence, as well as specific information on the condition of banks that may be available to country teams,67 ballpark estimates of contingent liabilities in a particular country can be established.

98. Table 2 presents some illustrative fiscal costs for a selected number of countries, if a banking crisis were to occur in 2003. The figures are based on regression [3] of Table 1 and the most recent values of the financial variables used under alternative assumptions about the existence of an accompanying currency crisis and on whether the government resorts to a variety of resolution measures to clean up the banking system.68 The simulated fiscal costs appear much lower than the 10 percent shock currently used to capture the effect of a realization of contingent liabilities associated with banking crises, unless there is an accompanying currency crisis and/or some combination of resolution measures is used to clean up the banks. Existence of a currency crisis before or at the time of the banking crisis can increase the fiscal cost of banking crises by close to three fold. The table also shows that fiscal costs of banking crises are substantially greater (more than 10-fold) when the government resort to all resolution measures, compared to a non-interventionist approach, generating fiscal costs in the range of 26–53 percent of GDP when there is also an accompanying currency crisis. In the case of no currency crisis, costs range from 9–18 percent of GDP.69

Table 2.

Illustrative Simulations of Costs of Banking Crises in Selected Countries 1/ 2/

(in percent of GDP)

article image

The simulations use the regression equation: Ln (Cost/GDP) = -0.494 + 1.062*Currency crisis dummy – 0.012*(BankNFA/Assets) + 0.006 (Bank credit/GDP) 0.014* Realint + 1.086*(Repeated Recap dummy) + 0.934*(Liquidity support dummy) + 0.812*(Forebearance Index).

Bank NFA/Assets ratio takes the values 4.41%, 3.18%, 4.56%, and 5.57%, Credit-to-GDP ratio takes the values 139.5%, 25.1%, 28.7%, and 9.5%, and the real interest rate takes the values 2.78%, 3.97%, 7.51%, and 4.89% in countries A, B, C, and D, respectively.

Forbearance Index is computed as I = ForbearanceA + 0.5*Forbearance_B + 0.25*Forbearance_C, where: ForbearanceA =1 when banks were left open in distress for at least 3 months. ForbearanceB =1 when undercapitalized banks were permitted to function. ForbearanceC =1 when prudential regulations were relaxed or the prevailing regulatory framework was not fully enforced for at least 12-months period.

Source: Honohan and Klingebiel (2000), Table 7, and staff estimates.

99. Some adjustments to these results should be made, however, using information about the characteristics of the country in question and other countries in the sample with similar characteristics, in view of the observed degree of under- (or over-) estimation of the actual costs of crises using the regressions reported in Table 1. Once refined, the estimate can be used to calibrate the contingent liability shock in the sustainability template.

Appendix IV Methodologies For Debt Sustainability Analysis Employed by Investment Houses and Rating Agencies

100. This appendix describes some of the methodologies employed by private sector financial institutions for assessing debt sustainability. The discussion is based on a sample of five banks with significant involvement in emerging markets, as well as two ratings agencies.70

Basic Methodology

101. The most common methodology, employed by all of the banks in the sample, uses some form of the basic debt dynamics equation, supplemented by scenario analysis. For the most part, there does not seem to be any standardization even within a given institution; analysts covering different countries may employ different methods. The analysis typically focuses on the short- to medium-term, i.e. 2–3 years, with liquidity concerns being the primary focus of analysis.71 The banks tend to rely mainly quantitative analysis, while the ratings agencies appear to rely to a greater extent on judgment and qualitative analysis.72 All institutions try to incorporate judgments about political factors into their analysis, using them to ascertain the feasibility of required adjustments.

102. Coverage differs across institutions, with in-depth sustainability analyses undertaken only for a handful of key emerging market countries, including Argentina, Brazil, Uruguay, Turkey, and Venezuela.73 The chief determinant of coverage and frequency of analysis seems to be the exposure of the institution or its clients. The analysis is updated when major political or economic events warrant, roughly three to four times per year for the more actively watched countries. The ratings agencies report a wider coverage and claim to follow developments more closely.74

Calibration of Baseline

103. Calibration of the baseline depends upon the analyst’s own judgment. Especially in countries where they have their own presence or expertise, banks prefer to use their own forecast numbers; in others, they may rely on some Fund projections, among other sources. Rating agencies also supplement the data they have with consultations with local government officials and other experts. Most of the analysts report that they find Fund documents useful, but they all noted that program projections are considered optimistic and are discounted accordingly. Coverage of debt differs across institutions; for the most part, they look at tradable external debt, with less attention being paid to domestic debt.

104. In terms of the liquidity aspect of sustainability—which is typically considered the binding constraint—the maturity structure of the debt is carefully examined. Much attention is also focused on performance under the Fund-supported program, mostly because of concerns that the program (and associated financing) might be interrupted. Analysts appear to be at least familiar with the Fund’s debt sustainability assessments, but there clearly remains a credibility gap about whether Fund projections are over-optimistic.

Sensitivity Tests

105. Statistical upper-bounds tests, as employed in the Fund template, do not seem to be used in the analyses of investment houses and rating agencies.75 Banks resort mostly to scenario analyses, looking primarily at changes in the growth rate, primary surplus, and real exchange rate. Some models also allow for Dornbusch-type initial overshooting of the exchange rate. These scenarios are not consistent across countries, and are tailored to the circumstances of a given country based on the judgment of the analyst. All of the analysts emphasized that while the calibration process is not sophisticated, effort is made to make these scenarios internally consistent. The scenarios are analyzed in terms of the feasibility of the required adjustments that the respective models project, given the assumptions.

106. Rating agencies also examine certain high/low scenarios relative to their baselines to inform their final decision. These scenarios are different for each country and are derived from discussions with relevant experts. Probabilities are assigned to such scenarios in a very informal and subjective manner.

107. A major absence in the analysis is a systematic treatment of contingent liabilities. All of the analysts recognized the difficulty in estimating contingent liabilities and stated that they were dealt with in a country-specific manner (rather than having an overarching algorithm to address it).76 Scenario analyses do not normally take into account various channels through which contingent liabilities may be realized. While rating agencies also look at various factors that may affect contingent liabilities—such as growth of domestic credit, non-performing loans, and certain other measures of vulnerability to financial stress— information necessarily has to be subsumed within a single rating.

Interpretation of Debt Ratios

108. All of the analysts emphasized that no particular threshold level of debt would be considered “unsustainable.” While they look for debt stabilizing dynamics—and the associated required changes in growth rate, primary balance, and interest rate—they put greater weight on the (country-specific) feasibility of such adjustments than on the value at which the debt stabilizes. To complement this analysis, they also examine the gross financing needs, or “cash flow” along the path, as liquidity issues are paramount.77 Perceived ability of the government to generate the debt-stabilizing primary surplus over the medium-term is also examined to determine the feasibility of required adjustments. To this end, analysts look at additional “social costs of adjustment” such as interest burden-to-revenue ratio, real wages in the government sector, and non-wage primary spending.

109. The final judgment about sustainability, however, tends to be very subjective. Given the lack of a uniform framework within institutions (a fortiori across institutions), there is no uniformity in analysis either. Judgments about whether the government will be able to deliver the required primary surplus depends very much on the perspective of the particular analyst, and is subject to change with changes in personnel. Both banks and rating agencies supplement their statements with reports that expand on their analysis.

Use of Analysis

110. Most of the debt sustainability analysis undertaken by banks is used by clients or for informing the banks’ own position (including flow inventory held by traders). Representatives from the banks may visit some of their prime clients and examine different scenarios with them; legal reasons prevent most of them from leaving their templates at their clients’ disposal.78 In addition, research reports based on such analyses are published regularly to inform institutional as well as retail clients.

111. The debt sustainability analysis undertaken by the emerging market research group of a bank may also inform the work of its internal risk management group. The latter also has its own resources, and there are strict rules on the sharing of information across the “Chinese walls.”

112. There does not seem to be much interaction between the banks and rating agencies in terms of their respective debt sustainability analyses. Banks will draw on the analysis of rating agencies, particularly for smaller or less active emerging markets where they may lack expertise. For the more prominent emerging markets, however, banks claim to have information on par with that of rating agencies. Credit rating agencies, in turn, claim not to be influenced by the market, in part because market reaction (as measured by the spread) can be very volatile and not necessarily related to country characteristics.

Conclusions

113. The approach used by private financial institutions to assess debt sustainability does not differ markedly from the Fund’s debt sustainability framework but in general, private institutions have made less effort to bring comparability to their analyses for different countries. Overall, debt sustainability assessments undertaken by the private sector appear to be more tailored toward individual country analyses (especially as regards the design of scenario analysis and stress test) than the Fund’s debt sustainability framework. By the same token, however, they are applied less systematically and are less disciplined. Analysts mainly use scenario analysis, which as outlined in the main text, can be a useful complement to statistical bounds tests.

114. Some analysts have recently experimented with explicit stochastic simulations to generate likely debt paths, similar to the approach used in section III(B) of the main text. Moreover, contingent claim models which seek to capture the interactions between the corporate, banking system, and government sectors, are under development. As greater experience is gained with these more experimental methods, they may usefully complement the Fund’s debt sustainability analyses.

References

  • Barnhill, Theodore, and George Kopits, 2003, “Assessing Fiscal Sustainability Under Uncertainty,” International Monetary Fund Working Paper: WP/03/79, April.

    • Search Google Scholar
    • Export Citation

  • Caprio, Gerard, and Daniela Klingebiel, 2003, “Episodes of Systemic and Borderline Financial Crises,” World Bank, mimeo (available at http://econ.worldbank.org/view.php?type=18&id=16172).

    • Search Google Scholar
    • Export Citation

  • Claessens, Stijn, and Daniela Klingebiel, 2002, “Measuring and Managing Government Contingent Liabilities in the Banking Sector,” in Hana Polackova-Brixi and Allen Schick, eds., Government at Risk: Contingent Liabilities and Fiscal Risk, Washington, DC: World Bank, pp 311 – 334.

    • Search Google Scholar
    • Export Citation

  • Collyns, Charles, and G. Russell Kincaid, eds., 2003, “Managing Financial Crises: Recent Experience and Lessons for Latin America,” International Monetary Fund Occasional Paper 217.

    • Search Google Scholar
    • Export Citation

  • Currie, Elizabeth, and Antonio Velandia, 2002, “Risk Management of Contingent Liabilities Within a Sovereign Asset-Liability Framework,” World Bank (available at: http://www.worldbank.org/pdm/pdf/currie_velandia_cl.pdf).

    • Search Google Scholar
    • Export Citation

  • Daniel, James, 1997, “Fiscal Aspects of Bank Restructuring,” International Monetary Fund Working Paper: WP/97/52, April.

  • Dziobek, Claudia, and Ceyla Pazarbasioglu, 1997, “Lessons from Systemic Bank Restructuring,” International Monetary Fund Working Paper: WP/97/161, December.

    • Search Google Scholar
    • Export Citation

  • Ghosh, Atish, 1997, “How Good (or Bad) are EUII Projections, International Monetary Fund, mimeo.

  • Golosov, Mikhail, and John King, 2002, “Tax Revenue Forecasts in IMF-Supported Programs,” International Monetary Fund Working Paper: WP/02/236, December.

    • Search Google Scholar
    • Export Citation

  • Hoggarth, Glenn, Ricardo Reis, and Victoria Saporta, 2002, “Costs of Banking Instability: Some Empirical Evidence,” Journal of Banking & Finance 26, pp. 825 – 855.

    • Search Google Scholar
    • Export Citation

  • Honohan, Patrick, 1999, “Fiscal Contingency Planning for Banking Crises,” World Bank Policy Research Working Paper No. 2228, December.

    • Search Google Scholar
    • Export Citation

  • Honohan, Patrick, and Daniela Klingebiel, 2000, “Controlling Fiscal Costs of Banking Crises,” World Bank Policy Research Working Paper No. 2421, September.

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund, 2002, “Assessing Sustainability,” SM/02/166.

  • International Monetary Fund, 2003, “A Framework for Managing Systemic Banking Crises,” SM/03/50.

  • International Monetary Fund, 2003, “Debt Sustainability in Low Income Countries—Towards a Forward-looking Strategy.” SM/03/185.

  • Juhn, Grace and Prakash Loungani, 2002, “Further Cross-Country Evidence on the Accuracy of Private Sector’s Output Forecasts,” IMF Staff Papers 49 (1), pp. 49 – 64.

    • Search Google Scholar
    • Export Citation

  • Kaminsky, Graciela, and Carmen Reinhart, 1999, “The Twin Crises: The Causes of Banking and Balance-of-Payments Problems,” American Economic Review, 89 (3), pp. 473 – 500.

    • Search Google Scholar
    • Export Citation

  • Karacadag, Cem, and Virginia Manzer, 1997, “Financial System Stress and Sovereign Credit Risk,” Standard and Poor’s Creditweek 10, December.

    • Search Google Scholar
    • Export Citation

  • Manasse, Paolo, Nouriel Roubini, and Axel Schimmelpfennig, 2003, “An Early Warning Model of Fiscal Crises,” International Monetary Fund, mimeo.

    • Search Google Scholar
    • Export Citation

  • Musso, Alberto, and Steven Phillips, 2001, “Comparing Projections and Outcomes of IMF-Supported Programs,” International Monetary Fund Working Paper: WP/01/45, April.

    • Search Google Scholar
    • Export Citation

  • Polackova, Hana, 1999, “Contingent Government Liabilities: A Hidden Fiscal Risk,” Finance and Development, 36(1), pp. 46 – 49.

  • Polackova-Brixi, Hana, and Sudarshan Gooptu, 2002, “Dealing with Contingent Liabilities in Indonesia and Thailand,” in Hana Polackova-Brixi and Allen Schick, eds., Government at Risk: Contingent Liabilities and Fiscal Risk, Washington, DC: World Bank, pp 251–267.

    • Search Google Scholar
    • Export Citation

  • Reinhart, Carmen, Kenneth Rogoff, and Miguel Savastano, 2003, “Debt Intolerance,” International Monetary Fund, mimeo.

  • Tanner, Evan, 2003, “Fiscal Rules and Countercyclical Policy: Frank Ramsey Meets Gramm-Rudman-Hollings,” forthcoming International Monetary Fund Working Paper.

    • Search Google Scholar
    • Export Citation

  • Zoli, Edda, 2001, “Cost and Effectiveness of Banking Sector Restructuring in Transition Economies,” International Monetary Fund Working Paper: WP/01/157, October.

    • Search Google Scholar
    • Export Citation
1

This framework drew on discussions with and contributions from members of an interdepartmental working group, consisting of FAD, ICM, MFD, PDR and RES, which was established in February 2002 to examine how debt projections undertaken by the Fund could be improved. In particular, the design of the sustainability templates drew heavily on analytical work undertaken by FAD.

2

The framework discussed in SM/02/166 was modified in three important respects in light of initial experience and comments received from some country teams and national authorities. In the public debt template, the sensitivity test is applied to the real interest rate (rather than to the nominal interest rate and to the GDP deflator separately). The template also now distinguishes between domestic and foreign currency denominated public debt. In both templates, gross financing needs associated with rollover of maturing debt are now tracked explicitly.

3

For the industrialized countries, assessments of external debt sustainability are optional in cases where the country has a strong net international investment position.

4

A framework that would be more appropriate to the circumstances of low income countries (including their reliance on concessional financing as well as greater vulnerability to commodity price shocks) is being developed. Debt Sustainability in Low-Income Countries— Toward a Forward-looking Strategy (SM/03/185, May 28, 2003) lays out the main considerations (as well as explaining why the methodology underlying the HIPC debt sustainability analysis is quite different). In the meantime, a few country teams have applied the standard template (with ad hoc modifications) to low-income countries where sufficient data and stable time series are available.

5

Since the debt-stabilizing primary surplus is given by pt = (r – g)dt ,when the growth rate of the economy is assumed to exceed the interest rate, the higher the stock of debt, the larger the primary deficit that the country can run while still stabilizing the debt. Indeed, it can be shown that as long as r < g , the country would be able to satisfy its infinite horizon intertemporal budget constraint with any (finite) amount of debt, although it would still face roll-over risk. In reality, this condition cannot persist indefinitely, since it would imply that the country continually grow faster than the rest of the world economy—in other words, the country would eventually become larger than the rest of the world. Nevertheless, assuming that the growth rate of the economy exceeds the interest rate over a finite horizon provides an “easy” way of assuming debt sustainability (though, at least over short-horizons, it may be justified in some high-growth economies). For foreign-currency denominated debt, the assumption of a continuous real exchange rate appreciation is another easy way of assuming debt sustainability.

6

It bears emphasizing that public debt ratios are not always comparable since the coverage of the public sector differs across countries. Of the 42 public debt sustainability assessment prepared to date, 2 covered central government, 15 covered general government, and 25 covered the consolidated nonfinancial public sector.

7

Nor is this a feature exclusive to countries with Fund-supported programs (where program approval requires that the country’s debt be seen to be on a sustainable path). Projections for countries without Fund-supported programs show a decrease of public debt of 12 percent of GDP, although their external debt is projected to increase by about 2 percent of GDP.

8

Some proposals for improving baseline projections, including introducing a “no policy change” scenario to gauge the ambitiousness of the baseline, are discussed in Sections IV and VII below.

9

The combined shock assumes that real GDP growth, the primary (or non-interest current account) balance, the interest rate, and the growth rate of the GDP deflator move adversely by one standard deviation for a two year period. As shown in Figure 1 (below), this mimics the typical pattern observed in debt crises.

10

Demographic changes pose a risk to fiscal sustainability (due to unfunded pension liabilities) in a number of emerging market countries as well. At the five-year horizon, however, changing demographics are typically not the most pressing threat to fiscal sustainability in these countries.

11

This is not to imply that the staff ignore warning signals from debt sustainability analyses. However, given that part of the purpose of the framework is to identify potential sustainability problems several years in advance—so that crises can be averted and corrective measures can be taken in a timely manner—greater integration with the rest of the staff’s analysis may be warranted even in cases in which there is no imminent risk of a crisis or debt-servicing difficulties.

12

At the conclusion of the Board discussion on Assessing Sustainability, it was proposed that sustainability assessments could be reported in annexes to the staff report to allow for easy deletion in cases in which there are concerns about market sensitivity.

13

Recall that the debt-stabilizing primary surplus is given by pt = (r – g)dt , where p is the primary balance, r is the real interest rate, and g is the real GDP growth rate. Since the interest rate and growth rate (and primary balance) return to their baseline values (after the two-year period of the shock), the debt ratio in the sensitivity test will only be increasing at the end of the projection horizon if the increase in debt as a result of the shock makes the baseline primary balance inadequate to stabilize the debt ratio. In practice, because the primary surplus in the baseline is projected to be sufficient to result in a decrease of the debt ratio, it is nearly always sufficient to ensure at least stabilization of the debt ratio in the sensitivity tests.

14

Argentina provides a vivid illustration of this problem. By the early-1990s, it was clear that the economy had undergone a fundamental shift such that the poor growth performance during the high-inflation 1980s was no longer relevant, and that potential growth could be as much as 4–5 percent per year. Yet, by the late-1990s, it became increasingly clear the assumption of potential growth of 4–5 percent per year had been a gross over-estimate and that predicating debt sustainability assessments on such growth rates had contributed to a dangerous increase in the debt ratio.

15

Note that it is the behavior of these variables in the run-up to the crisis—rather than once the crisis has erupted—that is of relevance for the debt sustainability framework, since it is predicting the likelihood that a configuration of shocks to growth, costs of financing and primary balances will lead to an unsustainable debt level that is of interest. Section VI provides a detailed discussion on interpretation of the results from the sustainability framework.

16

The sample covers 26 episodes of debt crises in total where a debt crisis is identified as sovereign defaults on external debt, as classified by Standard and Poors (S&P). Sovereign defaults are defined as the failure to meet principal or interest payment on external obligation on due date (including exchange offers, debt equity swaps, buy back for cash). Countries with market access are those covered in the EMBI+ index.

17

These standard deviations are calculated over the eight years prior to the crisis for a given country; the median across the sample of countries is plotted in Figure 1.

18

One of the investment houses surveyed in Appendix IV reports using such an approach on an experimental basis; see also Barnhill and Kopits (2003).

19

The exercise allows for serial correlation in the underlying variables and correlation of shocks across the variables and the probability distribution function is based on 10,000 drawings of the random variables. Details of the simulation are provided in Appendix II.

20

The stochastic simulations assume that the shocks follow a Gaussian distribution. Boot strapping methods can be used to avoid making distributional assumptions, but the very small samples (20 observations) may mean that any extreme shocks that might have occurred in the sample receive disproportionate weight. Some initial examples, comparing the imposed Gaussian distribution to the empirical boot-strapping distribution, suggest that this is not a severe problem, but further experimentation is required.

21

A further difficulty is that the distribution function may not be invariant over time, particularly in the run-up to a crisis. This would imply that the true probability distribution function may be more fat-tailed (i.e. so that extreme outcomes have higher probability) than suggested by the stochastic simulations.

22

The computer programs required to compute the probability density functions will be made available to interested country teams.

23

Traditionally, 95 percent or 99 percent confidence intervals are chosen in statistical applications as well as in banking system stress tests. Depending upon the degree of risk aversion, however, other confidence intervals may be chosen.

24

A further caveat to the claim that the standard sensitivity tests are extreme is that the true probability density function may be more fat-tailed than assumed either because (as suggested by boot-strapping simulations) it is not Gaussian distributed, or because of structural breaks in the run-up to a crisis.

25

It is not proposed to add a separate “debt crisis” scenario because, as discussed in section III, the movements of the key variables in the run-up to a crisis are well mimicked by the standard sensitivity tests. But other scenarios that may be relevant in country-specific circumstances, such as movements in key commodity prices, would also be encouraged.

26

The source of the shock would reflect the main risks that the country team and the authorities view as pertinent to the country. Thus the underlying shock may be domestic or external, financial or real (e.g. a terms of trade decline, a shock to tourism or to some component of the capital account, a sharp decline in housing or equity prices, etc.).

27

Private financial institutions emphasize the use of consistent scenarios, though most do not formally model the baseline or alternative scenarios (Appendix IV).

29

In other words, the probability that the debt ratio will exceed the debt level implied by a one-year one-standard deviation shock to the growth rate is about 15–20 percent.

30

There is some (weak) evidence that consensus forecasts are more accurate than, for instance, WEO projections, though in part this may be because they are updated more frequently (see Juhn and Loungani (2002)). One important caveat is that “consensus” forecasts for some emerging market countries can sometimes reflect the proprietary considerations of relatively few analysts, particularly in thinner, less deep markets. This may be less of a concern when these proprietary interests diverge significantly across forecasters, but this is not necessarily the case. Therefore, when applied in conjunction with broader market data such as future and forwards which reflect the consensus among a broader group of investors, there is a risk of some inconsistency between the market-based expectations data and the consensus forecasts of the macro parameters.

31

In Indonesia, for example, public debt rose by more than 50 percent of GDP in the aftermath of the crisis, much of which reflected issuance of new debt to finance the recapitalization of banks and the liquidity support provided by the central bank (see Polackova-Brixi and Gooptu (2002)). Depending upon how the costs are financed, contingent liabilities can also have an impact on external indebtedness. Typically, however, the initial effect is on the stock of public debt so that the contingent liability shock is modeled only for the public debt dynamics template.

32

Australia and New Zealand include explicit contingent liabilities and contingency expenditure provisions in government financial statements, and Italy and the United States make budget appropriations for the net present value of the future fiscal costs of loan guarantees and direct loans (Polackova (1999)). To assess the risks, governments use historical experience, and more sophisticated methods, such as actuarial, econometric, loss-estimate, and option pricing models (see Currie and Velandia, 2002, for a detailed discussion of some techniques available to value contingent liabilities). In practice, however, few governments have such information available.

33

In the absence of countervailing measures to tighten the fiscal stance, a debt-based government assistance to banks would cause the public debt-to-GDP ratio to rise faster, or fall more slowly, over the medium-term than envisaged before the assistance (Daniel (1997)).

34

These efforts include enhanced financial sector surveillance for countries that have not undertaken the FSAP and more continuous monitoring of countries (through the use of financial sector indicators (FSIs) and information on financial sector developments) to prepare vulnerability analysis as part of the Fund’s interdepartmental exercise of vulnerability assessments for emerging market countries (see SM/03/77, 2/25/03).

35

These exercises are normally undertaken jointly by the Monetary and Financial Systems Department and relevant area department. To date, about a half dozen have been undertaken in the past few years, including in Dominican Republic, Chile, Malaysia, Russia, and Seychelles, and Turkey. In a number of other cases, banking sector stress tests have been conducted in the context of TA missions that provided input into area departments’ work (e.g., Indonesia, Malaysia, the Philippines, and Venezuela).

36

One issue that would need to be resolved concerns the use of information obtained during FSAPs since certain types of information obtained during an FSAP assessment and included in the FSAP report are subject to restrictions on circulation and publication.

37

The figures refer to direct fiscal costs (i.e., outlays of the government and the central bank in terms of bond issuance and disbursement from the treasury for liquidity support, blanket guarantee, recapitalization, and purchase of NPLs) and do not incorporate economic costs of banking crises—e.g., lost output due to temporary interruptions in financial intermediation, or costs borne by depositors and other creditors of failed banks (e.g., in the form of wider spreads to compensate for the NPLs left in bank balance sheets).

39

Karacadag and Manzer (1997) and Hoggarth, et al. (2002) argue that fiscal costs are likely to be greater the larger the size of the financial system and the greater the proportion of the banking system under state ownership, the latter because state banks are seen more likely than private banks to be bailed out when they fail.

40

Except for financial size, all these factors are statistically significant (at the 90 percent level, or better) according to t-tests on the differences between sample means.

41

Anomalously, the opposite holds when the average cost measure is used, probably because of a few outliers in which the capital adequacy ratios are over-estimated because of deficient loan classification and provisioning standards.

42

For instance, in countries which have recently suffered a banking crisis, the likelihood and magnitude of further contingent liabilities would be—other things equal—presumably smaller.

43

For the reasons discussed above, including the fact that the sensitivity shocks are applied in the first two years of the projection horizon, the debt ratio will nearly always be stabilizing at the end of the horizon.

44

The term “debt correction” was intended to capture both outright defaults as well as severe adjustment and was defined as a 20 percent decline in the external debt ratio within a 2-year period. The sample was based on 147 emerging market and developing countries over the period 1980–2001, with 53 cases identified as debt corrections.

45

A further important caveat to such analysis is the very different definitions across countries of what constitutes public sector debt.

46

See Debt Intolerance (Reinhart, Rogoff and Savastano, 2003).

47

The Standard and Poors definition (underlying Figure 6), covers a sample of 72 countries over the period 1970–2001, with 53 cases identified as a debt default. These include Indonesia in 1997 (but not Korea, Thailand, or Malaysia) and Argentina in 2001.

48

Although not included in this model, Reinhart, Rogoff, and Savastano (2003) find that the country’s default history is also relevant for determining the likelihood of debt crisis.

49

For instance, whether a country suffers a currency crisis may be a good indicator of whether it also experiences a debt crisis. But forecasting a currency crisis may be as difficult as forecasting a debt crisis, in which case the early warning model would be of no operational use.

50

Multivariate logit regressions were estimated to derive the probability of a crisis given selected explanatory variables. These regressions are based on 200 observations for external debt and 79 observations for public debt, covering 44 countries, in five-year averages covering the period 1973 to 2002. For transition countries, only the last five-year period, 1998 to 2002, is used. For the purposes of evaluating the fit of the models, a 20 percent cut-probability is chosen (that is, if the predicted probability exceeded 20 percent, the model is said to be predicting a crisis). Using a higher threshold value raises the proportion of false negatives, while a lower threshold value raises the proportion of false positives (see below).

51

Additional variables (along the lines of those used in Manasse et al. (2003) and other papers on debt sustainability) were also added to specifications for both external debt and public debt, but they did not either contribute significantly to the explanatory power of the regressions or affect the significance of variables listed above.

52

An analogous figure can be drawn for calculating the probability of a crisis implied by the public debt ratio, using the logit analysis described above.

53

The analysis could be taken a step further: given historical probability distributions for interest rates, growth rates, and other key variables, the probability that a given primary surplus stabilizes the debt ratio could be computed (see, for instance, Tanner (2003)).

54

The new version of the template computes these debt-stabilizing primary balances automatically.

55

Note that the most extreme sensitivity test for the gross financing needs need not coincide with the most extreme test for the debt ratio.

56

As noted in SM/02/166, in such cases, there is provision for using shorter periods to calculate the relevant averages and standard deviations. As part of the technical guidance note for the standard framework, tables of cross-country parameters will also be provided to country teams.

57

In such cases, there would also be less need to integrate the debt sustainability assessment with the rest of the staff’s analysis, and the debt sustainability tables could be placed in an appendix.

58

See SM/02/166.

59

This would apply regardless of whether the debt sustainability assessment is presented in the main body of the staff report or in another document, as long as it is included in one of the documents covered by Decision No. 12882-(02/113) of November 11, 2002.

60

Previous studies on the quality of staff projections include Ghosh (1997); Musso and Phillips (2001), and Golosov and King (2002). Compared to these studies, which were mainly confined to program projections at one-year forecast horizons, the use of the WEO projections offers a number of advantages, including much broader country coverage and a larger set of macroeconomically consistent projections over various horizons. By including both program and non-program countries, the analysis also permits a direct test of whether projections in a program context are especially biased.

61

In principle, this formulation is not entirely free from the possibility that simulated e shocks drawn from the assumed normal distribution lead to inconceivable values of forcing variables (e.g. negative interest rate or GDP growth below minus 100 percent). Nonetheless, it would be less prone to such a problem than drawing a random sample of forcing variables directly from normal distribution without filtering.

62

In simulation, a random sample of e is constructed by pre-multiplying the lid. shocks drawn from the standard normal distribution by the Choleski factor W of Z, where W is a lower-triangular matrix that satisfies Z = WW’.

63

The following countries are included in the sample: Algeria, Argentina, Barbados, Belize, Bolivia, Brazil, Chile, China, Colombia, Costa Rica, Côte d’Ivoire, Dominican Republic, Ecuador, Egypt, El Salvador, Grenada, Guatemala, India, Indonesia, Jamaica, Jordan, Malaysia, Malta, Mexico, Morocco, Nigeria, Oman, Pakistan, Panama, Papua New Guinea, Paraguay, Peru, Philippines, Senegal, South Africa, Thailand, Trinidad and Tobago, Tunisia, Turkey, Uruguay, and Venezuela.

64

For consistent comparison with the standard test results, however, the moment matrix Z was constructed by using the estimated residuals over the 1990s only.

65

An alternative—more sophisticated—approach would be to estimate the value of the put option represented by the (implicit or explicit) deposit insurance scheme, though it may be difficult in practice due to data and analytical limitations.

66

There is significant controversy in the literature on how to calculate the cost of banking crises. The estimated fiscal costs employed in this exercise are taken from a variety of sources, and thus may not be strictly comparable across countries.

67

For example, on NPLs, capital-asset ratios, foreign exchange exposure of the banking and corporate sectors, profitability, and quality of supervision framework.

68

Given their very small sample sizes, it is not advisable to rely on regressions [1] and [2] to generate out-of-sample simulations of the fiscal costs.

69

This result is broadly consistent with that of Honohan and Klingebiel (2000), who find that open ended liquidity support, exercise of forbearance, and unlimited deposit guarantees add significantly to the fiscal cost of banking crises.

70

A staff team consisting of Messrs. Ghosh and Joshi (PDR) and Mr. Chan-Lau (ICM) visited JP Morgan Chase, Merrill Lynch; Deutsche Bank; Credit Suisse First Boston, Citigroup, as well as Standard and Poor’s and Moody’s, during April 24–25, 2003.

71

Rating agencies claim to incorporate longer-term concerns into their ratings, too, but they are appropriately discounted. Especially for more volatile economies, they put more weight on the short-term as well.

72

One approach is a ten-category Sovereign Ratings Methodology Profile (RAMP), by which analysts rate sovereigns in each category—which may not be strictly comparable either across sovereigns or over time—that are then used by a rating committee to arrive at a rating. Some rating agencies also report experimenting with a contingent claims sovereign default risk model (which tries to take explicit account of the links among the banking, corporate, and government sectors) as one of the inputs to the committee’s deliberations.

73

Other countries covered by some, but not all, banks include Ecuador, Chile, Jamaica, South Africa, and selected East Asian countries. Generally, Eastern European emerging markets tend to be covered by investment bank offices in London, and East Asian markets by offices in the region.

74

In principle, the rating committee can be called whenever an analyst covering a country feels there is sufficient information to warrant a review of a sovereign’s rating.

75

Absence of data was described as one of the reasons the standard deviation shock approach is not employed.

76

The “options-pricing” approach attempts to build a formal connection between contingent liabilities and public sector debt.

77

Beyond medium-term solvency, one bank has an experimental stochastic cash-flow model in which the sovereign defaults when it hits a zero cash-flow constraint. The main difficulty lies in parameterizing the stochastic processes. Using conservative estimates (low average growth, high volatility) means that there is an appreciable probability that (eventually) almost any government will run into the cash-flow constraint.

78

One bank, however, provides a user-friendly spreadsheet to its clients so that they can experiment with various parameters and conduct their own debt sustainability analysis.

Same Series

Other IMF Content

Other Publishers

  • Collapse
  • Expand
Cover Policy Papers
Sustainability Assessments - Review of Application and Methodological Refinements
Author:
International Monetary Fund
Volume/Issue:
Volume 2003: Issue 049
Publisher:
International Monetary Fund
ISBN:
9781498329453
ISSN:
2663-3493
Pages:
72
DOI:
https://doi.org/10.5089/9781498329453.007

Mexico: Sovereign Bond Spread
Figure 1.

Pre-crisis Trends in Key Variables (Sample Medians) 1/

Retrospective Debt Sustainability Analysis
Figure 2:

Stochastic Simulation of Debt Ratio
Figure 3.

Statistical Bounds and P -values
Figure 4.

Frequency Distributions of P -values of First, Second and Third Most Extreme Sensitivity Tests 1/
Figure 5.

Estimated Cost of Banking Crises 1/

(in percent of GDP)
Figure 6.

Public and External Debt Ratios in Sovereign Debt Crises.
Figure 7:

Fitted Probability of Sovereign Debt Crisis As a Function of External Debt, Short-term Debt, and Trade Openness
Figure 8.

Country Name: Debt Ratio and Gross External Financing Need, 2002–08
Figure 1.

Underestimation (+) of the Cost of Banking Crises, Actual-Fitted

(in percent of GDP)