2018 Review of Program Design and Conditionality
Author: International Monetary Fund. Strategy, Policy, & and Review Department
Publication Date:
20 May 2019
eISBN:
9781498315715
Language:
English
Keywords:
PP; director; Fund staff; program design; Fund program performance; Fund engagement; program objective; program success; Fund-supported program; PRGT program; Debt sustainability; Debt sustainability analysis; Nonperforming loans; Fiscal consolidation; Global
Download PDF (785 KB)

2018 Review of Program Design and Conditionality

Abstract

2018 Review of Program Design and Conditionality

Introduction

1. The 2018 Review of Program Design and Conditionality (hereinafter the “2018 RoC”) is the first comprehensive stocktaking since the global financial crisis (GFC). The 2018 RoC examines the performance of Fund-supported programs between September 2011 and end-2017, assessing the extent to which programs met their objectives.1 In doing so, the RoC considers the role of program design and the implementation of the Guidelines on Conditionality (GoC) (IMF, 2002a), including member ownership, as well as other factors affecting outcomes (e.g., external shocks). The RoC examines conditionality in a broad sense, considering the underlying macroeconomic and structural policies, and the specific methods used in Fund arrangements to ensure the achievement of program goals.2 Based on this analysis, the RoC draws lessons for the design of future programs to ensure that they adapt to the evolving needs of the membership.

2. The 2018 RoC builds on recent Fund and Independent Evaluation Office (IEO) studies and complements other (ongoing) Fund policy reviews. The 2018 RoC follows the 2011 RoC (IMF, 2012a) and the 2015 Crisis Program Review (CPR) (IMF, 2015c), recent IEO reports on the IMF and Euro Area crisis countries, Fragile States, Social Protection, and Structural Conditionality, as well as important Fund workstreams, all of which have had important implications for program design.3 In addition, the 2018 RoC supports ongoing Fund policy reviews, including the Review of Facilities for Low-Income Countries (LICs), the Debt Sustainability Framework (DSF) Review for Market Access Countries (MAC), the Review of the Fund’s Debt Limits Policy (DLP), the Strategy for IMF Engagement on Social Spending, and the workstream on Building Resilience in Developing Countries Vulnerable to Large Natural Disasters.

3. The 2018 RoC period was characterized by major structural challenges for large parts of the membership. The preceding 2011 RoC juxtaposed a period of relative global economic stability (2002–2007) with that of the GFC (2007–2011), which eclipsed the regional crises of previous decades. In contrast, the 2018 RoC period was dominated by less acute but persistent structural challenges, requiring large-scale and long-lasting adjustment. Taking into account the specific challenges, this paper analyzes the following analytical country groups (Appendix I):

  • Post-GFC countries directly affected by the GFC, particularly in the euro area, which experienced a protracted period of recovery and adjustment, as is typical following financial crises.

  • Political/economic transformation countries that faced major political transitions (e.g., Arab Countries in Transition), or economic transformations from public to private sector-led growth.

  • Commodity exporters (mainly PRGT-eligible) that experienced a major and persistent terms-of-trade shock caused by the sharp decline in commodity prices from late-2014.

  • Other developing countries (the remaining PRGT-eligible members) that continued to focus on development and poverty reduction goals.

4. The difficult task of tackling these structural challenges was exacerbated by the persistently weak global environment. Global economic activity remained sluggish during this period, as headline and potential growth disappointed repeatedly. Global inflation also remained very low, worsening nominal growth (IMF, 2015a). Hence, weak external demand weighed heavily on all countries, hampering adjustment and efforts to tackle structural issues.

5. The remainder of this paper is organized as follows. Section II evaluates overall program performance and describes program trends, setting up the major themes for this review. Section III assesses the macroeconomic outcomes of programs in more detail—focusing on the optimism in growth forecasts, inflation outcomes, the less-than-optimal path of fiscal consolidation, and the evolution of debt vulnerabilities. Section IV delves deeper into the design and implementation of structural reforms. Sections V and VI evaluate the implementation of the conditionality principles of ownership and uniformity of treatment (evenhandedness). Section VII assesses the implications of the RoC recommendations. Section VIII provides conclusions for the review, setting out important trade-offs that need to be weighed when considering how to improve program performance. The final section proposes issues for discussion.

Program Success

6. A Fund-supported program should aim to resolve a member’s Balance of Payment (BoP) problems and provide adequate safeguards for Fund resources. The GoC state that Fund-supported programs should be directed primarily toward the following macroeconomic goals: (i) solving the member’s BoP problem without recourse to measures destructive of national or international prosperity; and (ii) achieving medium-term external viability, while fostering sustainable economic growth.

7. These objectives are operationalized differently under the General Resources Account (GRA) and the Poverty Reduction and Growth Trust (PRGT). Programs supported by GRA resources are designed to resolve the member’s BoP problem during the program period.4 In contrast, the purpose of programs supported by the Extended Credit Facility (ECF) (the primary instrument in the PRGT) is to help PRGT-eligible members with protracted BoP problems implement their economic programs and make significant progress toward a stable and sustainable macroeconomic position. PRGT programs also put a greater emphasis on growth and poverty reduction. These differences in objectives are also reflected in differences in the balance between financing and adjustment (Figure 1). PRGT programs appear to be more catalytic—outside official support is larger than Fund support—and often entail less adjustment, as financing is used to support growth and poverty reduction objectives.5

Figure 1.
Figure 1.

Planned External Adjustment Versus Financing

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: IMF staff reports, WEO, and IMF staff calculations.

Solving BoP Problems

8. The nature of post-program engagement can be taken as a proxy for whether a member’s BoP problems is resolved during the course of a program. The frequency of successor program engagements—defined in this RoC as a Fund-supported program that follows another Fund financial arrangement within two years—is often considered to be an important indicator of whether Fund arrangements have failed to resolve BoP needs. However, it is important to distinguish between the different types of successor Fund-supported programs:

  • In the GRA, the RoC considers successor programs of a signaling nature as including Policy Coordination Instruments (PCIs),6 and arrangements of a financial nature that are treated as fully precautionary or involve low access.7 Such programs would indicate the absence of either an actual or a large BoP need requiring Fund financing. In contrast, a drawing successor arrangement would indicate the presence of a persistent BoP need.8 While there was an increase in the number of GRA arrangements with successor programs in the 2018 RoC sample, 12 out of the 19 were followed by programs of a signaling nature to help anchor macroeconomic policies and a structural reform agenda, and the remaining 7 were followed by drawing arrangements, which tended to be Extended Fund Facility (EFF) arrangements (Figure 2), highlighting the protracted and structural nature of the BoP needs (Figure 3).9 20 GRA programs were not followed by any successor arrangement, and 13 are still ongoing.

  • In the PRGT, the protracted nature of the BoP problem means that repeated use of ECF arrangements is expected. Two-thirds of PRGT-eligible countries had at least one Fund arrangement during 2010–17, of which about one-half maintained program engagement with the Fund during at least six of the last eight years, consistent with the protracted nature of the BoP problem that ECF arrangements help resolve. Program performance remained unchanged or improved in 77 percent of successor programs (relative to the original program).

Figure 2.
Figure 2.

Usage of EFF Arrangements

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Source: MONA.
Figure 3.
Figure 3.

Key Program Trends

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: MONA and IMF staff calculations.

External Viability and Economic Growth

9. Assessing whether programs helped achieve external viability and sustainable growth requires looking at a range of macroeconomic indicators. The RoC examines the following flow and stock indicators, common across the GRA and PRGT: (i) the current account, including the growth of import and export volumes; (ii) international reserves; (iii) growth; (iv) fiscal balance; (v) public debt and market access; and (vi) the stock of non-performing loans (NPLs) (Figure 4).

10. The macroeconomic indicators reveal a mixed picture, with significant improvements in most flow indicators failing to translate into stock adjustments, as growth disappointed. The adjustment in flows (current account and fiscal balance) proceeded broadly as planned; in most cases, reserve targets were met or came close to reaching projected levels. However, growth and the anticipated public and private sector balance sheet adjustment—proxied by public debt and NPLs, respectively—fell short of expectations, as well as outcomes in the 2011 RoC period. In part, these findings are explained by the composition of adjustment, which weighed on growth and negatively impacted balance sheet repair.

  • External adjustment often proved to be better than envisaged—in some cases this was due to significant import compression rather than export growth. Import compression reflected the disappointing growth outcomes. In addition to lower-than-expected external demand, weak average export growth likely reflected difficulties in achieving external adjustment under fixed exchange rate regimes and currency unions. During the 2018 sample period, two-thirds of GRA program countries had non-floating exchange rates.

  • Fiscal adjustment targets were often met but through less growth-friendly measures than initially planned (Section III.C).

Figure 4.
Figure 4.

Program Macroeconomic Outcomes

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: WEO, IMF country reports, and IMF staff calculations.

Implementation and Completion Rates

11. Program implementation and completion data (intermediate indicators) provide further insights. The implementation rate of quantitative performance criteria (QPCs) was around 90 percent, while implementation of structural conditions (SCs) reached 80 percent, including those “implemented with delay” (Figure 5).10 This performance was broadly in line with that of the 2011 RoC sample. However, analysis of review and program completion rates reveals a significant increase in the number of programs going off track, both quickly and by mid-program, which is not captured by the “assessed” implementation rates above.11,12 This could potentially point to weaker ownership (Figure 6 and Section V).

Figure 5.
Figure 5.

Program Implementation

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: MONA and IMF staff calculations.
Figure 6.
Figure 6.

Program Completion

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: MONA and IMF staff calculations.

Bottom-Line Assessment

12. A more systematic assessment of program success requires a methodology that can be applied in a variety of circumstances across a diverse membership. As noted above, there are important differences under the GRA and PRGT in how Fund-supported programs operationalize the overall objectives as specified in the GoC. The RoC therefore develops two separate frameworks to assess program success. For each framework, programs are classified in one of three categories: “successful,” “partially successful,” and “unsuccessful.” In the GRA, the nature of post-program Fund engagement and the evolution of vulnerability indicators are considered as measures of resolving a BoP problem and achieving medium-term viability, respectively.13 In the PRGT, program success is measured using a different set of indicators tailored to the specific objectives of the LIC facilities, including the evolution of external debt vulnerabilities, social spending, capital expenditure, revenue mobilization, real GDP growth, and inflation.

13. The consolidated (unweighted) assessment of all programs indicates that three-quarters of Fund-supported programs were at least partially successful (Figure 7).14

  • GRA. One-third of GRA programs are assessed as successful, one-quarter as unsuccessful, and the remainder as partially successful. Successful arrangements typically managed to both reduce vulnerabilities and eliminate or significantly reduce the BoP need. Partially successful programs accomplished one of these two objectives. Unsuccessful arrangements typically saw a persistence or deterioration of vulnerabilities and were often followed by a drawing successor arrangement. In the sample under review, key determinants of success include program completion (proxy for ownership), forecast realism (Section III, A), and a debt operation in countries with high debt vulnerabilities (Section III, D).15

  • PRGT. About 25 percent of PRGT programs are assessed as successful, about 50 percent as partially successful, and about 25 percent as unsuccessful. Successful PRGT arrangements avoided a substantial deterioration in DSA ratings and achieved the program targets in at least three of the five core indicators. Partially successful PRGT programs avoided a substantial deterioration in DSA ratings and achieved the program targets in one or two of the core indicators. Unsuccessful programs had DSA ratings that substantially deteriorated or remained in distress and/or met none of the program targets in the five core indicators. In the sample under review, success factors include program completion (proxy for ownership), an absence of fragilities, and, to some extent, favorable commodity prices.

Figure 7.
Figure 7.

Program Success

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: VE indicators, MONA, and IMF staff calculations.1/ The number in each cell represents the percent of programs experiencing the corresponding transition, as a share of all programs within the same successor category. The transition matrices highlight the change in vulnerability ratings (low (L)/ medium (M)/ high (H)) between the VE exercises immediately before and after the program.

14. Analytical groups had varying levels of success (Figure 8). Post-GFC programs fared better than commodity exporters and other developing countries. Political/economic transformation cases had both the highest success and failure rates of all groups.

Figure 8.
Figure 8.

Program Success by Analytical Group and Completion

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Source: IMF staff calculations.

15. Program success during the period was constrained by the scale of the challenges. Achieving success during this period often required the implementation of difficult measures to address protracted issues, and typically in an uncertain environment. At the same time, evidence suggests that Fund-supported programs may have been “risky” by design, with overly optimistic baseline assumptions (Section III, A). Success is thus relative, and one could conclude that Fund-supported programs performed reasonably well given the circumstances and the degree of program ambition, and that repairing the global economy after the GFC may simply have required more time.

Macroeconomic Policy Conditionality and Program Design

A. Growth Optimism

16. Growth outcomes were disappointing, marking a break with previous experience. In the period covering the 2011 RoC sample, forecasts were broadly accurate or only mildly optimistic for both program and surveillance countries. However, in the 2018 sample period, there was a general shift toward growth optimism.16 This trend occurred in both Fund surveillance and Fund-supported programs, suggesting that common underestimated factors were important drivers, including the downward trend in global productivity growth after the GFC (IMF, 2015a) (Figure 10).17 On average, growth outturns disappointed by slightly more than one percentage point over the short run and about 1½ percentage points over the medium term—with the largest forecast errors occurring for political/economic transformation countries and commodity exporters.

17. In most cases, growth optimism was intrinsic to program design, and ultimately appears to be a key factor influencing program success (Figure 9). Two-thirds of program request documents noted that macroeconomic risks were slanted to the downside, only around 15 percent assessed risks to be broadly balanced, and very few cases pointed to a favorable balance of risks. This indicates that a large majority of programs adopted an optimistic baseline. Optimistic growth assumptions understated the financing and adjustment required to achieve program objectives, with consequences for program implementation and success. Where growth met or overperformed projections, programs tended to be more successful in achieving the objectives stated above (Section II), while programs with the largest growth disappointments were typically unsuccessful.

Figure 9.
Figure 9.

Growth and Program Success

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: MONA and IMF staff calculations.

18. Global developments played an important role in growth optimism. Regression analysis suggests that disappointing trading partner growth and lower-than-expected commodity prices explain around one-quarter of the short-term growth forecast errors.18 This is consistent with the observed shift in forecast errors for both program and surveillance countries. The impact was particularly striking for commodity exporters, where the sharp drop in commodity prices contributed about one-third of the short-term forecast error.

Figure 10.
Figure 10.

Growth Optimism

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: Penn World Tables (version 9.0), WEO, and IMF staff calculations.

19. Programs also appear to have systematically underestimated the impact of adjustment on growth. The regression analysis suggests that staff underestimated the growth impact of adjustment, both public and private, and together, these factors account for another quarter of growth projection errors.19 This could reflect that the ex-post quality of adjustment was less-growth friendly than anticipated, and/or that assumed fiscal multipliers were too low. Fiscal multipliers were only mentioned in around 15 percent of program request documents. Even as awareness of the issue increased after the GFC, few Fund-supported programs recognized the uncertainties and risks to the outlook if multipliers proved to be larger than anticipated.

20. The remaining large and unexplained component of growth forecast errors could relate to an overestimation of productivity gains and capital investment, and some idiosyncratic factors. Since regression results include a large and unexplained residual of about 50 percent, growth accounting was used to explore the errors related to each of the key factor inputs (TFP, capital, and labor), as well as qualitative analysis.20 For most groups, lower-than-envisaged Total Factor Productivity (TFP) growth was a major contributor. This could indicate that assumed payoffs from structural reforms were overly optimistic, that staff overestimated confidence effects, or that projections did not fully internalize the impact of the downward trend in global productivity on potential output. For other developing countries, lower-than-projected capital accumulation was an important factor, consistent with a less growth-friendly fiscal consolidation than anticipated (see below). Other idiosyncratic factors also played a role in projection errors, including natural disasters (Solomon Islands, 2012 ECF), political transitions (Tunisia, 2013 SBA), and conflict (Ukraine, 2014 SBA).21 Furthermore, data limitations in some developing countries hampered analysis of the drivers of growth, with knock-on effects to projections.

Recommendations

  • Increase scrutiny of the realism of program baselines. Better calibrate risks, discuss downside scenarios, and develop contingency plans.

  • Strengthen the discussion and analysis of the impact of program policies on growth, including fiscal multipliers and the pay-offs from structural reforms.

B. Monetary Conditionality

21. During the 2018 RoC period, most Fund-supported programs saw inflation decline and modestly undershoot forecasts, driven in large part by global trends (Figure 11). On average, inflation declined in both GRA- and PRGT-supported programs and undershot projections in most programs, consistent with growth optimism, and reflecting the weak global environment. Forecast errors were relatively small compared with the previous RoC period, and consistent with the errors in surveillance countries during the period.

Figure 11.
Figure 11.

Inflation Outcomes and Monetary Policy Conditionality

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: MONA, WEO, and IMF staff calculations.

22. Monetary conditionality mainly relied on quantity-based targets, with limited use of the review-based approach to conditionality.22 Ceilings on monetary aggregates—net domestic assets (NDA) and base/reserve money (BRM)—continued to dominate program conditionality, particularly in PRGT arrangements. Inflation consultation clauses (ICCs)23 were only used in inflation-targeting countries, while Monetary Policy Consultation Clauses (MPCCs)24 were mostly adopted in countries with evolving monetary regimes. In countries with IT regimes that staff did not consider to be “fully-fledged,” conditionality was tailored to include NDA targets (Moldova, 2010 ECF-EFF; Georgia, 2012 SBA-SCF; and Armenia, 2010 ECF-EFF) or MPCCs (Armenia, 2014 EFF; and Ghana, 2015 ECF). Overall, the use of MPCCs remained limited, despite more countries evolving towards more flexible operational targets and forward-looking policies (IMF, 2014b).25

23. Implementation of monetary conditionality was strong, though inflation outcomes were facilitated by global trends. Observance of targets was high across both GRA and PRGT programs, particularly for review-based conditionality. Implementation rates for BRM targets were somewhat lower, and there were large NDA misses for commodity exporters. Inflation forecast errors were similar across all forms of conditionality, likely helped by global trends. Programs employing ICCs saw the lowest forecast errors, which may reflect a stronger capacity to implement monetary policy at the onset. However, there were cases of significant inflation overshooting, where errors tended to be driven by major shocks (i.e., military conflict, supply-side shocks, large depreciations, and embezzlement schemes), rather than failure to implement conditionality (GRA: Ukraine, 2014 SBA; Maldives, 2009 SBA; and PRGT: Mozambique, 2015 SCF; Malawi, 2010 and 2012 ECFs; Tanzania, 2010 PSI).

Recommendation

  • Review experience with ICCs and MPCCs and consider reforms to modernize the review-based monetary policy conditionality framework.

C. Quality of Fiscal Adjustment26

24. Fiscal plans aimed to mitigate the contractionary impact of consolidation on growth, particularly in post-GFC and political/economic transformation programs (Figure 12).27 Revenue measures typically focused on broadening the tax base, reducing distortions, and improving revenue administration. Around a quarter of post-GFC programs planned base-broadening tax measures, with another quarter envisaging rate-raising measures combined with the elimination of exemptions to decrease distortions. Most developing country programs focused on domestic revenue mobilization, including measures to strengthen the tax and customs administrations, and raise or introduce new taxes, particularly VAT. Spending measures in most programs centered on improving efficiency, including reductions in current subsidy and wage bill spending, to free up resources for growth-enhancing public investment, while protecting priority health and education spending, particularly for developing countries.

25. However, the realized (ex post) fiscal adjustment was often of lower quality than envisaged. In other developing countries, capital spending often fell short of initial targets, with spending cuts often coinciding with shortfalls in revenue and/or grants.28 Revenue mobilization tended to underperform, often due to inconsistent institutional development and insufficient political will (IMF, 2018a). That said, where revenues did turn out higher than envisaged (e.g., Mauritania, 2010 ECF), they were used to support infrastructure investment. In post-GFC countries, implementation of current expenditure measures generally lagged, possibly due to political constraints, particularly where programs envisaged wage bill measures, energy subsidy reform, or pension reforms. Instead, the composition of adjustment shifted towards revenues and quicker and politically easier capital expenditure cuts. In some cases, however, revenues overperformed, due to positive macroeconomic developments, efforts to strengthen tax compliance, and strong reform implementation (e.g., Armenia 2010 ECF; Portugal 2011 EFF; and Serbia 2015 SBA). For commodity exporters, the collapse in oil prices resulted in a much larger-than-projected decline in revenue, leading to capital expenditure cuts.

26. Social expenditure was generally protected in Fund-supported programs. More than half of programs in the 2018 RoC sample included indicative targets (ITs) or QPCs on social and other priority spending—more than double the share in the previous RoC period.29 In the PRGT and GRA, around 90 and 20 percent of programs, respectively, included conditionality on social spending.30 Performance against ITs on social spending was broadly satisfactory, and comparable to other types of ITs, with about 70 percent met across the PRGT and GRA. In cases where ITs were not met, technical or administrative challenges combined with weak absorptive capacity were often cited (e.g., Tunisia, 2013 SBA; and Grenada, 2014 ECF). Social assistance spending and health and education spending were common elements of ITs, and these were broadly maintained as a share of GDP and expenditure (Figure 12). For PRGT countries, real expenditure growth in these categories remained positive, on average, even under fiscal consolidation programs.31 Over the longer run, protecting social spending should also help support better social outcomes.32

27. Nevertheless, there was limited focus on the quality of social spending, and on social protection and inequality, more generally. Though case studies suggest that there was broad coverage of social issues in program Memoranda of Economic and Financial Policies (MEFPs), only one-quarter of programs included a SC related to social issues (excluding pension reforms).33 In most cases, SBs focused on social sector reforms (e.g., cash transfer programs or education expenditure frameworks), with few focused on protecting capital spending or improving the quality or effectiveness of social spending. While SCs on social sector issues often aimed at mitigating the impact of energy subsidy reform on the poor and vulnerable groups (e.g., Ukraine, 2014 SBA and 2015 EFF), authorities’ survey responses suggest that there is scope for significant improvement in assessing the social and welfare impact of program policies.34 Strengthening data quality remains key to enhancing conditionality in these areas.

Figure 12.
Figure 12.

Quality of Fiscal Adjustment 1/

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: WDI, WEO, and IMF staff calculations.1/ Based on cyclically-adjusted data. Consolidation (expansion) denotes an increase (decrease) in the cyclically-adjusted primary balance.

Recommendations

  • Use more granular fiscal conditionality. Where relevant for meeting program objectives, PCs or ITs (e.g., a floor on capital spending or revenue performance, or ceiling on current expenditure), as well as prioritizing SBs on social sector issues or capital investment management, could help ensure higher-quality fiscal adjustment, including higher levels of public investment. Nevertheless, more granularity could jeopardize parsimony, reduce flexibility and potentially have adverse implications for ownership, highlighting the importance of a case-by-case approach and streamlining conditionality in other areas.

  • Increase focus on the quality of social spending and the impact of program policies on poor and vulnerable groups. This should be underpinned by early engagement with authorities on the topics, and continued development of Fund policy advice on sustainable social spending, drawing on the expertise of international development institutions, including to strengthen data quality.

D. Public Debt

28. Public debt vulnerabilities were a pervasive issue for Fund-supported programs during the period. In post-GFC advanced economies, high debt levels were a legacy of the GFC, including the materialization of contingent liabilities and lower potential GDP growth. Initial debt levels in emerging markets (EMs) and LICs were much lower, with the latter benefiting from debt relief under the Heavily Indebted Poor Countries (HIPC) Initiative. However, this period saw a large and rapid accumulation of debt in many LICs, in part driven by easy global financing, especially new non-concessional lending, and accelerated by the massive commodity price shock of 2014 in commodity-producing countries.

Evolution of Public Debt Vulnerabilities

29. In one-third of Fund-supported programs, debt sustainability improved, particularly in those where initial vulnerabilities were high. For GRA programs, the risk assessment implied by staff’s statistical model,35 which does not incorporate judgment, shows a significant improvement in sustainability. The share of programs where debt was assessed as “unsustainable” by this model, declined from about 25 percent prior to approval of Fund arrangements to around 10 percent at end-program (Figure 13).36 In the PRGT (LIC DSA), most programs involving countries at “high risk” or in “debt distress” (prior to approval of arrangements) saw a reduction in the risk rating. While this success was somewhat offset by a deterioration in other PRGT cases, performance was relatively strong compared to the increase in the risks of debt distress in the Low-Income Developing Countries (LIDCs) group as a whole, from 2013 onwards.37 These findings are also supported by analysis of debt projection errors, which shows that around a quarter of GRA and PRGT programs overperformed initial projections, mainly due to fiscal deficit overperformance.38

30. In roughly one-half of the arrangements, the assessment of debt sustainability did not change, though in some cases this masked a gradual erosion of fiscal buffers. More than 50 percent of GRA and PRGT programs experienced no change in the statistical risk assessment (GRA) or bottom-line debt sustainability assessments (PRGT). In most of these cases, debt was already sustainable (GRA) or at moderate or low risk (PRGT). Similarly, most programs also only saw a modest overshooting of debt projections. Programs in the interquartile range saw debt targets missed, on average, by around 5 percent of GDP. Fiscal deficit slippages, exchange rate depreciation and higher real interest rates were the main contributing factors. While the overshooting was relatively modest, it still implied a reduction in fiscal space relative to initial program baselines.

31. Debt sustainability deteriorated in the upper quartile of programs, and most of these went off track.39 Debt projection errors in one-sixth of the sample were extremely high, averaging 18 percent of GDP in GRA cases and 32 percent of GDP in PRGT cases. In the majority of these cases, programs went off track, with only around 25 percent reaching completion. A range of exogenous and domestic factors were at play:

  • GRA. The leading causes of deterioration in debt sustainability were fiscal slippages (Serbia, 2011 SBA, Tunisia, 2016 EFF), lower growth (Greece, 2010 SBA; Portugal, 2011 EFF), bank recapitalization costs (Serbia, 2011 SBA), and assumption of SOE liabilities (Portugal, 2011 EFF). Other cases involving large projection errors were driven by: policy slippages (North Macedonia, 2011 PCL); conflict and security issues, with resulting fiscal loosening and disappointing growth outturns (Tunisia, 2013 SBA; Ukraine, 2014 SBA); and shocks to key trading partners (Armenia, 2014 EFF; Georgia, 2014 SBA).

  • PRGT. Risk rating downgrades were affected by conflict (Central African Republic, 2012 ECF; Yemen, 2014 ECF); the 2014 commodity price shock and natural disasters (Chad, 2014 ECF and Haiti, 2015 ECF, respectively); and the uncovering of previously undisclosed debt (Mozambique, 2013 PSI). Some of these countries saw very large projection errors. Policy slippages, contingent liabilities and off-budget guarantees, and governance/public financial management (PFM) weaknesses were all recurring themes in countries experiencing very large debt projection errors (Table 1).40

Figure 13.
Figure 13.

Evolution of Public Debt Vulnerabilities

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: IMF country reports, WEO, and IMF staff calculations.
Table 1.

High Public Debt Overshooting: Other Developing Countries

article image

Percentage points of GDP; cumulative from T to T+3.

C = completed all reviews, IP = in progress, L = largely implemented, OT= off track, QOT= quickly off track.

Note: Program completion status as of end-September 2018.Sources: IMF country reports, WEO and IMF staff calculations.

Public Debt Operations

32. A clear judgment on debt sustainability is crucial for program design. The Fund may only lend if debt is assessed to be sustainable in the medium term under the GRA and PRGT. If debt is not sustainable, the Fund is precluded from lending unless the member takes steps to restore debt sustainability, including through either debt restructuring or the provision of concessional financing. Under exceptional GRA access, public debt should be sustainable with high probability, or if debt is sustainable but not with high probability, the Fund may lend if financing provided by sources other than the Fund improves debt sustainability and enhances safeguards to Fund resources. Under normal access where there is uncertainty on the question of debt sustainability, the policy gives the Fund options of either relying on the catalytic approach or for the member to undertake some form of debt operation. Under exceptional PRGT access, countries should have a comparatively strong adjustment program and ability to repay the Fund. It is generally understood that countries either in debt distress or at high risk of debt distress should restructure their debt or obtain debt relief in order to restore debt sustainability by upgrading the debt distress rating to at least moderate. The recently reformed LIC DSF provides strong analytical underpinnings for assessing debt sustainability, and the ongoing review of the MAC DSA framework is exploring improvements to analytical tools to inform staff’s bottom-line judgment on debt sustainability.

33. Public debt operations tended to be associated with greater program success, but mainly in small and non-systemic program cases. The RoC looked at cases where debt vulnerabilities were high before approval of Fund arrangements (Figure 14) and compared the relative success of programs that did and did not include debt restructuring. In the GRA, this included 17 cases where debt was “unsustainable” or “sustainable but not with high probability” at program approval, based on the results of the probit model described above. In the PRGT, the analysis focused on 16 countries with LIC-DSA ratings either at “high risk” or in “debt distress” at program approval. About 40 percent of these programs involved debt operations: seven GRA-supported programs and six PRGT-supported programs. Evidence suggests that those involving debt operations saw a higher share of program success than those that did not involve debt operations.41 In spite of delays, debt operations were relatively successful in small island states (e.g., Seychelles, 2009 EFF; St. Kitts and Nevis, 2011 SBA; Grenada, 2014 ECF; Jamaica, 2013 EFF). In contrast, delays in debt operations in larger and more systemic economies (Greece, 2012 EFF; Ukraine, 2015 EFF) limited their potential benefits.

Figure 14.
Figure 14.

Public Debt Restructuring and Success in Programs with High Debt Vulnerabilities

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: VE indicators, WEO, and IMF staff calculations.

34. However, in some cases, there was a general reluctance to recognize that debt may not have been sustainable. Judgment on debt sustainability appears to have been tilted in favor of large fiscal adjustments and optimistic macro-frameworks in at least two cases with high debt levels—Greece (2010 SBA) and Ukraine (2014 SBA).42 The related-EPEs challenged the initial judgment that debt was sustainable (Greece), and sustainable with high probability (Ukraine).

35. The pros and cons of public debt operations have been well documented in the literature.43 On the one hand, debt operations provide immediate debt service relief and curtail the debt-stabilizing primary balance, which reduces the required fiscal adjustment. This, in turn, can minimize the negative impact on growth and facilitate a recovery of competitiveness, where debt ratios are sensitive to the real effective exchange rate. On the other hand, debt operations result in loss (at least temporarily) of market access and higher risk premiums once market access is regained. They can also weaken balance sheets of the domestic financial system and generate spillovers to other countries. The scale of the pros and cons also depends on the type of debt operation in question: restructuring (a reduction in the nominal value of the principal) tends to have larger potential implications than reprofiling (e.g., an extension of maturity). A growing literature suggests that the timely implementation of debt operations can be more beneficial than the alternative of (unrealistically) large fiscal adjustments.44 There is growing experience of specific design features that help achieve this objective (e.g., Jamaica, 2013 EFF).45

36. Nevertheless, the policy solution is not always straightforward, particularly in cases where public debt sustainability is uncertain. In some cases, policy prescriptions may be clear: if debt is sustainable, fiscal adjustment is usually sufficient to address debt vulnerabilities; if debt vulnerabilities are so large that the required fiscal adjustment is economically and/or politically infeasible (e.g., Seychelles, 2009 EFF), restructuring is inevitable. The solution may be less obvious where debt sustainability is uncertain. In such cases, the Fund has flexibility to call for debt reprofiling if appropriate, which could be a useful option, particularly if concessional official financing cannot be mobilized in sufficient amounts to restore debt sustainability and meet program financing assurances. However, debt operations need to be considered on a case-by-case basis, underpinned by a comprehensive cost-benefit analysis. Ultimately, restructuring or reprofiling to improve debt sustainability requires distributing the burden of adjustment among various stakeholders in a way that best preserves or restores financial stability and economic growth.

37. Debt sustainability and related program design issues will be considered in several ongoing and upcoming Fund workstreams. The MAC DSA Review is currently developing methodologies to better assess the extent of debt overhang and the realism of program assumptions, with a view to potentially helping mitigate any bias in judgment on debt sustainability and ensuring more balanced consideration of debt operations where applicable, including under normal access. Furthermore, although not a major issue during the 2018 RoC period, the presence of collateralized debt has proved to be a complicating factor, when considering the case for debt operations in several recent programs.46 In this context, staff is currently examining possible guidance on collateralized borrowing in Fund-supported programs, in the context of the forthcoming Review of the Fund’s DLP.

PRGT Case Studies

38. For the PRGT, case studies shed further light on the factors driving the large debt projection errors and highlight broader lessons for program design. The RoC looked at a diverse sample of the programs with very large errors, ranging from those that went off track quickly (The Gambia, 2012 ECF and São Tomé and Príncipe, 2012 ECF) to those that continued to completion (Malawi, 2012 ECF, Niger, 2012 ECF, and Rwanda, 2013 PSI).47 Key findings include:

  • Underlying drivers of residuals in projection errors were only partially identified.

  • Overoptimistic program baselines were a key factor behind projection errors in some cases.

  • The materialization of contingent liabilities and off-budget guarantees were recurring themes.

  • Conditionality evolved in response to shocks, but some programs appear to have repeatedly accommodated fiscal slippages and higher debt limits relative to the revised targets.

Recommendations

  • Sharpen debt sustainability tools, including more realistic macroeconomic assumptions and better assessments of debt overhang, to help mitigate any bias in judgment on debt sustainability and ensure more balanced consideration of debt operations, where applicable.

  • Consider SCs on improving governance arrangements for the contracting of debt and ensuring appropriate monitoring of obligations, including closer scrutiny of contingent liabilities (e.g., SOE liabilities and off-budget guarantees).

  • Review the Fund’s Debt Limits Policy, including examining possible guidance on collateralized debt in Fund programs.

Structural Conditionality and Program Design

A. Criticality and Parsimony

39. Structural conditionality in Fund-supported programs should be both critical and parsimonious. As set out in the GoC, SCs should be of critical importance for achieving the goals of the member’s program or for monitoring program implementation, or necessary for the implementation of specific provisions of the Articles of Agreement.48 SCs should also be applied parsimoniously (i.e., limited to the minimum necessary). Within the broader program goals set out in Section II, the objectives of individual programs should be as narrow and concrete as possible to be able to set the most critical conditions to achieve them. Assessing whether these principles have been implemented requires an analysis of the depth, focus, and volume of SCs. The 2018 RoC assesses each of these dimensions, following the same methodology as the 2011 RoC and the IEO 2008 report: depth is defined as the degree and durability of SCs, with measures separated into high-, medium-, and low-depth categories; focus is assessed by categorizing SCs into core, shared, and non-core areas of responsibility;49 and volume is defined as the number of conditions per program year. As a further insight into the criticality of SCs, the 2018 RoC also considers: (i) the consistency of SCs with program objectives; and (ii) the structural policy gaps identified in surveillance.50

40. The volume of conditions increased markedly with the shift to Fund-supported programs aimed at tackling structural problems. After a decline in the 2011 RoC period, the average number of SCs per program approval year rose by 30 percent in the 2018 sample period, coinciding with the shift in GRA programs from SBA to EFF arrangements (Figure 15). Most SCs continued to be low- or medium-depth, while SCs in core areas of responsibility continued to dominate conditionality.

Figure 15.
Figure 15.

Counting Structural Conditions

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: MONA and IMF staff calculations.

41. Although SCs were generally consistent with program objectives, the justification of measures, and discussion of prioritization and sequencing was often limited. An analysis of a random sample of 12 programs reveals that, in the vast majority of programs, SCs and program objectives were consistent (Figure 16). However, program objectives were often so broad (e.g., attain robust and more inclusive growth, or support the transition to a dynamic, emerging market economy) that it is difficult to judge whether a specific set of SCs were genuinely critical for program success. While most program documents provided some rationale for SCs, they rarely discussed the wider set of potential reforms, and how these should be prioritized or sequenced, to help achieve program objectives.51

42. Moreover, a mismatch between SCs and gaps identified in prior surveillance, suggests that critical reforms may have been excluded, particularly those in non-core areas of responsibility. Over half the gaps raised in surveillance were in shared areas, such as labor and product market reforms, and non-core areas like governance, anti-corruption,52 or statistics. This reflects the increasing importance of macro-critical structural challenges outside the core areas of Fund responsibility. In principle, there should be significant overlap between program SCs and the structural gaps identified in surveillance, but in practice almost half the gaps were not covered in conditionality. Two-thirds of the excluded gaps were in shared and non-core areas, with the majority of SCs remaining heavily focused on core areas. Although parsimony necessitates that not all gaps should be addressed, it is not obvious that SCs in core areas were always the most critical to achieve program goals.

43. SCs on labor and product market reforms remained limited. Labor market reforms (LMRs) and product market reforms (PMRs) can help foster wage and price flexibility to restore cost competitiveness and promote external adjustment through internal devaluation in countries with fixed exchange rate regimes. LMRs and PMRs can also help raise medium-term growth potential.53 However, LMRs and PMRs accounted for less than 3 percent of all SCs, with most concentrated in a few post-GFC programs (Greece, Portugal, and Kosovo). Moreover, LMRs to reduce structural unemployment or increase labor force participation, and PMRs to enhance competitiveness or diversification were rarely turned into SCs (see Figure 17). These reforms were even less common in PRGT countries and countries with more flexible exchange rates. With relatively low Fund capacity in these areas, success often depended on collaboration with other institutions.

Figure 16.
Figure 16.

Consistency of Structural Conditions with Program Objectives and Surveillance Gaps 1/

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: MONA and IMF staff calculations.1/ “Other macro-structural” includes: (i) social, gender, inequality, and poverty; (ii) labor market reforms; (iii) governance, including corruption; (iv) product market reforms and competition; and (v) other (e.g., trade or national accounts statistics).

44. Conditionality on gender inequality was an important innovation during the period to support higher and more inclusive growth. Several programs included SBs aimed at reducing gender inequality, including spending on public nurseries and other facilities to enhance the ability of women to actively seek jobs and improve women’s labor force participation (Egypt, 2016 EFF); studying options for lowering payroll taxes for women and youth (and identifying offsetting parametric changes in the pension system) to promote formal employment and stimulate aggregate demand (Jordan, 2016 EFF); and updating and adopting a new national gender policy (Niger, 2017 ECF). In all these cases, it will be important to monitor the effectiveness of these measures over time. While not included in conditionality, Mongolia’s 2017 EFF arrangement discussed how to create conditions for more inclusive job-creating growth, finance female entrepreneurs, and support better social outcomes to support higher female participation in the economy.

B. Implementation54

45. The assessed implementation of SCs remained relatively strong. The share of conditions that were met or implemented with delay rose slightly from 76 percent in the previous RoC to 82 percent in the current RoC (Figure 17, top panels), driven by the performance of GRA programs. This result is consistent across different Fund arrangements, areas of responsibility, and sectors. As expected, implementation rates were higher for lower-depth SCs than for high- and medium-depth SCs. Regression analysis suggests that capacity, as proxied by income level and regulatory quality, is a key driver of strong implementation. Another regression analysis suggests that the shift to review-based conditionality had no impact on implementation.55

46. However, SCs were modified more frequently during this period, possibly reflecting overambitious initial design and/or insufficient prioritization or sequencing. The average share of modified SCs increased relative to the previous RoC, in both GRA and PRGT programs (Figure 17, middle panels). Flexibility was somewhat higher for SCs in the financial sector, and for shared and non-core areas, such as pension and civil service, social sector, and other macrostructural reforms. In most cases, test dates for SCs were reset for delayed implementation following slippages, with only a small share of unmet SCs—mostly in core areas— turned into PAs. The higher share of delays and modifications may reflect the higher-than-expected reform implementation difficulties in the difficult post-GFC period, as well as the need for more rigorous prioritization and sequencing. For example, Serbia’s 2015 SBA saw several high-depth SCs on state-owned enterprise (SOE) restructuring implemented with delay or modified to a series of lower-depth SCs, to reflect capacity constraints or pushback from vested interests. In some programs, delays in implementation and modifications of SCs may have contributed to growth forecast errors.

47. Implementation in some key structural areas proved particularly challenging:

  • LMR and PMR. Gaining public support and national ownership for these measures was difficult, given the short-term costs and the delayed benefits, and the lack of space for macro policy support to offset these costs or frontload the benefits.56 In Greece and Portugal, some LMRs were reversed even before program-end. PMRs tended to be more granular in some countries (e.g., sectoral liberalization in Greece), straining the principles of criticality and parsimony. In contrast, PMRs aimed at general frameworks over a wide range of sectors (investment codes, competition law) or at strategic sectors (mining code) tended to be more successful and easier to design and monitor (e.g., Portugal) than reforms aimed at fixing distortions in specific markets.

  • Financial sector reforms. Conditionality varied depending on the financial stability challenges, ranging from urgent measures to stabilize and restructure the banking system, to longer-term structural improvements in regulation and supervision. Notably, NPLs rose for several countries, weighing on credit growth and bank and corporate profitability (Cyprus, 2013 EFF; Greece, 2010 SBA and 2012 EFF; Ireland, 2010 EFF; and, Portugal, 2011 EFF), often reflecting improved recognition of asset impairment. While NPLs started to decline toward the end of the program in some cases (Ireland and Portugal), in others NPLs have remained elevated (Cyprus and Greece), reflecting the difficulty and complexity of NPL-workout. In some cases, measures to deal with rising NPLs were delayed by the need to first stabilize the financial system (see Box 1).

48. Given these implementation challenges, a longer period of program engagement could be required. Even if program objectives are specific, reforms are well prioritized and sequenced, and ownership is high, implementation of SCs often takes time. In survey responses, around one-quarter of Mission Chiefs/Resident Representatives (MCs/RRs) and Executive Directors (EDs) thought that program duration was insufficient to accomplish program objectives, while country authorities (CAUTs) noted that successor programs often provided necessary policy support.57 A well-paced reform agenda can prevent overstraining implementation capacity and smooth adjustment, as some reforms weigh on growth in the short run, while paying off only later.58 However, in some cases, a longer timetable can exceed the political window of opportunity and trigger reform fatigue. Depending on circumstances, drawing programs could focus on macroeconomic stabilization, with follow-up PCIs (available in the absence of a Fund financing need) used to support medium-term structural agendas. Serbia serves as an example of this type of extended engagement with the Fund, to facilitate the design and execution of an ambitious reform agenda. In some specific cases (e.g., after a commodity price collapse in a country with limited capacity), longer Fund arrangements (e.g., five years) could also improve implementation by allowing a more realistic pace of deeper structural reforms (e.g., to diversify the economy). The risk of protracted periods of off-track programs would have to be managed carefully, with appropriate safeguards, to help ensure the revolving nature of Fund resources.59

Figure 17.
Figure 17.

Implementation and Modification of Structural Conditions

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: MONA and IMF staff calculations.

Financial Sector Reforms

Post-GFC arrangements saw the highest shares of SCs on the financial sector. One out of five SCs in post-GFC arrangements are on the financial sector, reflecting the significant impact of the GFC on private sector balance sheets, and the strong macro-financial linkages of corporate and household sectors. Financial sector conditions range from bank recapitalization, resolution and privatization, strengthening of supervision and regulation, and NPL resolution.

Despite being largely effective in stabilizing the financial sector, programs could not prevent the build-up of large NPLs that proved difficult to resolve. NPLs rose for several post-GFC countries, weighing on credit growth and on bank and corporate profitability. Cyprus, Greece, Ireland, and Portugal saw NPLs rise by an average of 10½ percentage points, despite all having financial stability as one of the program priorities. NPLs started declining only after the end of the program for Ireland and Portugal but remained elevated until now for Cyprus and Greece, holding back the resumption of credit growth. Experience from these countries reflects the challenge of reducing NPLs within the program period:

  • NPLs evolve slowly, and a typical cycle exceeds the duration of a Fund-supported program. The shock to private sector balance sheets, shortly after which Fund-supported programs begin, do not immediately pass through to NPLs, as firms and households first find ways to meet repayments, and banks and regulators practice forbearance. Tighter loan classification and provisioning standards within a program boost the measure further, as the true level is revealed. As a result, NPLs rise at the start of programs and peak only at the end.

  • Overoptimistic macro and asset recovery forecasts lead to an underestimation of the extent of the NPL problem. Expectations that growth recovers and, as a result, asset values rise during the course of the program reduce the urgency of dealing with NPLs. This also underestimates provisioning and capital needs, with recapitalizations sometimes failing to provide banks with sufficient buffers and incentives to address distressed loans.

  • Stabilization of the banking system is a prerequisite for NPL resolution, but often means that the issue is only tackled midway into the program. Programs often initially focus on ensuring banks have sufficient liquidity and capital and on resolving distressed entities. The objective was to reduce uncertainty, regain market confidence, and minimize spillovers. Where bank vulnerabilities were significant, programs included earmarked funds for cleaning up the banking sector from the outset (e.g., Cyprus, Greece, Ireland, Ukraine). While these steps eventually helped facilitate NPL resolution, they also delayed implementation of NPL resolution strategies.

  • Strategies to resolve NPLs require complex reforms that take time to design, legislate, and implement. Enforcement and insolvency frameworks need country-specific refinements to create incentives for creditors and borrowers to restructure loans. Political opposition to these reforms slows down legislation. Weaknesses in the court system are often additional impediments, therefore requiring judicial reforms that take even more time to implement. For example, in Ireland, legal and political constraints took time to resolve. Also, banks often lack the expertise and tools needed to implement loan restructuring, including the ability to properly assess affordability.

  • There are tradeoffs between the speed of NPL reduction and economic outcomes. For example, faster NPL resolution through write-offs could destroy value, while bespoke loan restructuring maximizes value but takes time. NPL sales to centralized Asset Management Companies (AMCs) can help jump-start a distressed asset market but involve large fiscal costs.

Financial sector conditionality played an increasing role in LICs. Conditionality in LIC programs usually focused on building financial stability infrastructure to ensure financial deepening and inclusion, without creating risks to financial stability. Therefore, the primary focus of SCs was often on banking supervision, anti-money laundering, or basic regulation, supported by TA to build capacity.

49. In general, Fund technical assistance (TA) was deployed consistently with program priorities and country needs. TA inputs to other developing countries and commodity producers amounted to 0.63 and 0.70 FTEs per program, respectively, compared to 0.47 full-time equivalents (FTEs) per post-GFC program. This prioritization of TA to countries with lower capacity—rather than to those with the highest number of SCs—explains the relatively weak link between TA and the number of SCs (Figure 17, bottom panels). TA was focused on core areas of Fund responsibility and broadly aligned with the topical distribution of SCs in core areas. Shared conditions on pensions and civil service and SOE reforms received very little Fund TA, and teams had to rely on other development partners, such as the World Bank.

50. Surveys point to generally effective Fund collaboration with other institutions. The GoC underscore the need for coordination with other multilateral institutions in designing and monitoring conditionality, particularly in shared and non-core areas of responsibility. Survey responses of MCs and RRs show that most teams collaborated effectively with the World Bank and other development partners, particularly when reforms covered shared and non-core areas such as the investment code and social policies in the context of subsidy reforms (Tunisia, 2013 SBA) and SOE restructuring and government rightsizing (Serbia, 2015 SBA). However, in some cases, greater collaboration was deemed to be desirable— e.g., in Greece where issues on the division of labor emerged (IMF, 2017b).

Recommendations

  • Identify, prioritize, and sequence reforms based on criticality, drawing on structural gaps from surveillance and TA to ensure an integrated approach.

  • Continue to build expertise in critical shared areas of responsibility (e.g., labor and product market reforms), and enhance collaboration with other institutions that have expertise in non-core areas.

  • Consider NPL resolution and related conditionality at the outset, where appropriate, recognizing the tradeoff between the speed of NPL resolution and economic outcomes, the complexity of the process, and the time needed to successfully complete reforms.

  • Apply greater realism in implementation timetables and estimated reforms payoffs, while considering longer Fund engagement to support structural reform agendas, including greater use of successor PCIs and, in some cases, longer duration EFF arrangements and longer initial duration ECF arrangements (i.e., five years), with appropriate safeguards to preserve the revolving nature of Fund resources.60

Ownership

51. Country ownership is a critical ingredient for Fund program design and performance but remains difficult to measure. The definition of national ownership highlights that the responsibility to formulate and carry out program policies lies with country officials, with the understanding that implementation is in the country’s best interest (IMF, 2002b). As such, the GoC assign primary responsibility for program design to CAUTs, and responsibility for establishing conditionality to the Fund. This approach is intended to foster greater flexibility in program design, encourage greater ownership and, as such, strengthen program implementation. While straightforward to define, ownership cannot be measured by a single indicator or metric, and RoC analysis therefore examines ownership along several dimensions, drawing on surveys, program completion rates, and lessons from case studies.

Survey Evidence

52. Survey results indicate that perceptions of ownership remain broadly positive. Around half of MCs and RRs rated ownership as “very high” or “high,” while a third rated ownership as “moderate.” In addition, MCs, RRs, EDs, and CAUTs, overwhelmingly agreed that program quantitative performance criteria targeted the appropriate macroeconomic variables and that structural reforms were consistent with national reform priorities, pointing to strong ownership. Relatedly, a majority of respondents agreed that program objectives were consistent with domestic economic and social priorities, though results were less positive than in 2011, perhaps reflecting the challenges arising from the increase in structural conditionality. Three-quarters of all respondents believed that program design was sufficiently flexible to accommodate external shocks.

53. Fund outreach with Civil Society Organizations (CSOs) appears to have strengthened, but there is still room for improvement. Survey results point to a more favorable view of Fund outreach, perhaps reflecting Fund efforts to deepen engagement with CSOs (Figure 18). Improved outreach was a notable feature of program design that helped support ownership in successful programs (e.g., Jamaica, 2013 EFF; Rwanda, 2013 PSI and 2016 SCF). Despite this progress, a significant share of MCs/RRs, CAUTs, and EDs disagreed that CSOs were actively involved in program design and implementation discussions. Furthermore, around a quarter of MCs and RRs thought the authorities had not communicated the benefits of the Fund-supported program to civil society.

Figure 18.
Figure 18.

Ownership

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: 2011 and 2018 RoC surveys, MONA and IMF staff calculations.1/ In the 2011 RoC survey “Neutral” and “Not applicable” were not provided as response options.

Completion Rates

54. While conditionality implementation rates remained relatively strong, completion rates deteriorated, possibly indicating a moderate weakening in ownership. The share of completed programs declined, and the proportion of programs that went off track mid-program roughly doubled during the period (Figure 6). Completion rates varied slightly across the four analytical groups, and more significantly across types of programs. EFF and EFF-ECF arrangements, which were generally of longer duration than SBAs, had higher completion rates. Nearly all PSIs were completed, possibly reflecting the fixed review schedule. Perceptions of factors contributing to unsatisfactory implementation varied, with MCs/RRs pointing to weak capacity and lack of ownership, and CAUTs focusing on unexpected developments or exogenous shocks. Further regression analysis suggests that better institutional capacity was a crucial factor for program completion. In contrast, political developments and external shocks were insignificant and only weakly associated with completion rates, respectively, suggesting that program design was sufficiently flexible.61

55. Prior actions (PAs) and low completion rates tend to go hand-in-hand. There was a small increase in PAs in GRA-supported programs and no change in PRGT cases. Across analytical groups, political/economic transformation cases—facing difficult and urgent reforms, coupled with the lack of a track record—saw the highest number of PAs. The evidence suggests that while PAs may have been effective in implementing a specific measure, their abundant use did not translate into higher program completion rates, but rather the opposite (Figure 18). Regression analysis points to a negative association between PAs at arrangement approval and completion rates. While this finding is consistent with the notion that many PAs may indicate weak ownership, it also indicates that PAs are not a substitute for ownership. More guidance may be needed on the application of PAs.

Staff-Monitored Programs (SMPs)

56. More use of SMPs may help address the issue of off-track Fund-supported programs, particularly in the GRA. During the 2018 RoC sample period, there were 12 SMPs: one-half were aimed at building a track record, and two-thirds of these were successful and paved the way to a Upper Credit Tranche (UCT) successor arrangement (Afghanistan 2015, Chad 2013, Iraq 2016, Madagascar 2015).62 During this period, SMPs were not used to bring existing programs back on track, despite some notable successes in this regard during the 2011 RoC period, including in GRA countries (Djibouti 2004, Kosovo 2011). Given the observed reduction in program completion rates and increase in off-track programs, SMPs could be a useful (currently underutilized) option for helping manage extended program interruptions, which are often associated with weak ownership and program performance. Authorities could request an SMP to ensure monitoring of macroeconomic policies, while they build political support for critical reforms, whose delay has interrupted the program. SMP-related documents would also provide information to the Executive Board and the public on discussions in the context of off-track programs. There is however a need to de-stigmatize SMPs.

Case Studies: Lessons

57. Some important lessons stand out from case studies that cover a varying degree of ownership.63 Key findings include:

  • National reform plans. Programs that benefit from well-designed national reform plans tend to have higher completion rates. In high-ownership cases among both GRA- and PRGT-supported programs, design (and associated discussions in staff reports) generally reflected national reform plans (e.g., Jamaica, 2013 EFF; Rwanda, 2015 PSI and 2016 SCF). Further, national economic program oversight committees (e.g., Jamaica, 2013 EFF) can be effective in supporting program implementation. Conversely, where programs quickly went off track, national plans received little discussion.

  • Communication. Outreach to the public supports ownership and reform implementation. While staff reports and program documentation often do not discuss plans for communicating program strategies or measures to the public,64 some good examples stand out (see above). These highlight the importance of outreach to civil society, including non-governmental organizations. However, as communication is often treated as a separate issue, there could be scope for better integrating such strategies into program discussions.

  • Track record and implementation capacity. As noted above, continued attention to policy implementation track records and reform implementation capacity is critical to support ownership. Where applicable, staff reports gave credit to the authorities for strong track records. For countries without clear track records (e.g., in some new programs), staff reports generally considered forward-looking aspects, including the country’s administrative capacity and TA needs. In fact, most of the case studies related to ownership had concurrent TA, largely for implementation of fiscal and public financial management reforms.

  • Political economy. While no consistent pattern is found between the political cycle and ownership, additional attention to political economy risks would be prudent. Most staff reports, especially at arrangement approval, discussed the extent of the political base for reforms, the election cycle, and risks of political or social instability. However, discussions were usually brief and did not always explicitly state the political constraints that may complicate the implementation of specific reforms.

Recommendations

  • Encourage well-integrated national reform plans as an anchor for Fund arrangements.

  • Improve two-way communication with the broader public to support buy-in.

  • Encourage voluntary use of SMPs, particularly in the GRA, to ensure monitoring of macroeconomic policies, while the authorities build political support for the critical reforms, whose delay has interrupted the program.

  • Strengthen analysis of institutional and political capacity to deliver program objectives on a realistic timetable. Provide additional guidance on the use of PAs and analyzing institutional and political capacity.

Tailoring and Uniformity of Treatment (Evenhandedness)

58. The GoC require that Fund lending decisions be both tailored and evenhanded. Use of Fund Resources (UFR) decisions must reflect both member country circumstances and uniformity of treatment (¶¶4 and 5, IMF, 2002a). Balancing tailoring with evenhandedness does not require that member countries be treated identically—but, rather, that countries in similar circumstances be treated similarly. This inevitably requires judgment because country circumstances (e.g., BoP problems, track record, and implementation capacity) vary significantly, and, as the Executive Board noted in the 2011 RoC, this is particularly challenging in the UFR context.

59. While general perceptions of tailoring and evenhandedness in UFR tend to be positive, some stakeholders continue to raise concerns. The 2018 RoC surveys of MCs, RRs, EDs, and CAUTs broadly support a view that UFR decisions are mostly evenhanded, including with respect to the tailoring of conditionality and access decisions. Nevertheless, a significant minority of respondents indicated concerns in these areas (Figure 19), which resonates with issues raised previously and reported by the IEO and the G24.65 There are perceptions about the lack of evenhandedness of access both within the GRA and between the GRA and PRGT. As noted in the forthcoming 2018–19 Review of LIC Facilities, these could have been reinforced by the observed and projected erosion of PRGT access limits relative to GDP and to gross financing needs.

Figure 19.
Figure 19.

Perceptions of Tailoring and Uniformity of Treatment

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: 2018 RoC surveys.

60. More fundamentally, the lack of up-to-date cross-country information on conditionality and access is viewed as a constraint in monitoring and comparing programs. Following the “2007 IEO Evaluation of Structural Conditionality in IMF-Supported Programs”, the IMF made the MONA database public. However, in its recent update to this study, the IEO (Structural Conditionality in IMF-Supported Programs: Evaluation Update) pointed to “significant shortcomings in the usability, accuracy and replicability of the [MONA] database, which limits its value as a monitoring or tracking tool.” These issues were echoed by the Executive Board and external stakeholders in the run-up to the 2018 RoC. The ongoing MONA revamp, scheduled for completion in 2019, will be critical for addressing these underlying concerns.

Tailoring Conditionality

61. Evidence suggests that conditionality was generally tailored to country needs and program objectives during the period. Conditionality and program design typically included some tailoring to reflect members’ circumstances, and the provisions of the applicable Fund facility or instrument. For example, quantitative conditionality for LICs included external debt limits to maintain debt sustainability while ensuring adequate external financing,66 and monetary policy consultation clauses were increasingly incorporated into programs of EMs and LICs with evolving monetary policy regimes. Analysis also indicates tailoring of structural conditions across groups, reflecting salient economic characteristics: fragile states had relatively more conditions relating to PFM and revenue administration, while political/economic transformation programs put more emphasis on SOE reform and the financial sector (Figure 20).

62. Nevertheless, there appears to be scope for further tailoring in Fund-supported programs for fragile and small states:67

  • Fragile states. The “2012 Staff Guidance Note on the Fund’s Engagement with Countries in Fragile Situations” (IMF, 2012b) calls for a strict, prioritized and gradual structural agenda in the UFR context, reflecting capacity constraints. However, the overall number and depth of SCs in fragile state programs during this period were broadly in line with the average for the overall sample (Figure 21). While implementation rates were similar across groups, a higher proportion (about half) of fragile state programs did not complete all reviews and went off track, and fragile states also had lower success rates than other countries. This suggests that programs may have been hampered by an excessively expansive agenda, which failed to adequately reflect low capacity and specific sources of fragility.

  • Small states. The revised 2017 Staff Guidance Note on the Fund’s Engagement with Small Developing States calls for a focus on (i) growth-friendly fiscal consolidation, particularly in heavily-indebted small states; (ii) reforms to deepen the financial sector; and (iii) reforms to build resilience to frequent and severe shocks from natural disasters (IMF, 2017). Program conditionality in the few small state programs in the 2018 RoC period was justifiably focused on PFM, revenue administration, the financial sector, and SOE reform. However, in some cases, conditionality did not include resilience building to natural disasters, despite this being a program objective or a key program risk (Solomon Islands, 2011 SCF and 2012 ECF; Grenada, 2014 ECF). Further tailoring to support resilience building efforts is needed, informed by IMF-WB joint Climate Change Policy Assessments (CCPA).68 Such tailoring would help build buffers, enhance disaster preparedness, strengthen institutions, and coordinate capacity building. A Fund-supported program with resilience building as its core objective could also help catalyze climate change financing, for which access procedures of other donors and institutions can often be complex.

Figure 20.
Figure 20.

Tailoring and Uniformity of Treatment

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: MONA and IMF staff calculations.
Figure 21.
Figure 21.

Number and Depth of Structural Conditions

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: MONA and IMF staff calculations.

Access

63. Access decisions should be determined by a range of country-specific factors, as well as underlying GRA and PRGT policies. In principle, determination of levels of access to Fund resources in individual arrangements should reflect factors that are common across GRA and PRGT arrangements: (i) the country’s BoP need; (ii) program strength and capacity to repay; and (iii) the amount of the member’s outstanding use of Fund credit and its record in using Fund resources in the past. At the same time, there are differences in the lending frameworks of the GRA and PRGT. For example, PRGT resources are limited69 and have access norms70 and limits.71 That said, PRGT-eligible countries are not restricted to reliance on PRGT resources and have the right to access GRA resources on the same terms and conditions as other members, including based on staff’s assessment of a member’s capacity to repay.

64. During this period, there were significant differences in access at arrangement approval, both within the GRA, and between GRA and PRGT programs. Analysis points to much larger variation in access within the GRA sample, compared to the PRGT, mostly driven by EA GRA cases (Figure 22). Moreover, there was a large disparity between average GRA and PRGT access levels, with access at arrangement approval in GRA program cases 3 percent of GDP higher on average than PRGT cases during the period.

Figure 22.
Figure 22.

Access

Citation: Policy Papers 2019, 012; 10.5089/9781498315715.007.A001

Sources: Fund Financing Arrangements database and IMF staff calculations.

65. Regression analysis explains a large share—almost 70 percent—of the variation in access decisions.72 Pooled regressions of both GRA and PRGT programs suggest that the differences in access level are largely driven by Fund policies (i.e., EA policy and PRGT access norms). Separate regressions were estimated for the PRGT and GRA because of the differences in applicable policies. In the GRA, the analysis finds that gross financing needs, capital account crisis, and the normal access limit were important explanatory variables. However, the EA dummy remains an important driver of access, most likely capturing the sizeable effects of large BoP crises, above and beyond those of a typical capital account crisis. In the PRGT, there were strong links with PRGT access norms and the size of adjustment, confirming that the strength of policies matters in access decisions.

66. Policies regarding the access of PRGT-eligible countries to financial support from the Fund were clarified by the Executive Board in November 2016.73 The principle that access to Fund resources should be determined on the basis of the standard criteria, including balance of payments need, program strength, and capacity to repay the Fund, informed by DSA, was reaffirmed. Directors also underscored that “access norms, as used in PRGT facilities, are neither a ceiling nor a floor on the level of access provided in PRGT-supported arrangements. Norms should help inform the assessment of access levels and should not be misconstrued as access limits or entitlements.” In line with this clarification, access levels in programs with PRGT-eligible countries in the period since November 2016 have, on average, deviated by a much wider margin from the relevant norm than in the prior period, with several countries facing large BoP needs in the wake of the commodity price shocks.

Recommendations

  • Revamp the MONA database and introduce periodic, standardized reports to the Board to ensure transparency and facilitate the monitoring and comparison of programs.

  • Improve tailoring of SCs for fragile and small states: for fragile states, analyze sources of fragility more systematically and streamline objectives and related SCs, by focusing on short-term realistic measures, taking into account capacity constraints; and for small states, focus SCs on resilience building to natural disasters, where appropriate.

  • Consider increasing PRGT access norms and limits, promote blending of GRA and PRGT resources, and increase the flexibility of SCF arrangements while maintaining PRGT self-sustainability. These issues will be discussed in the forthcoming 2018–19 Review of LIC Facilities.

Implementation and Risks

67. The RoC recommendations require implementation on multiple fronts (Table 2). Staff does not see a need at this stage to update the GoC. Instead, recommendations will be implemented by: (i) updating the Operational Guidance Note on Conditionality; (ii) delivering ongoing and planned workstreams (e.g., MAC DSA and DLP reviews, and monetary policy conditionality review); and (iii) if there is sufficient Executive Board interest, producing a possible follow-up paper to consider longer duration EFFs to support structural reform agendas.

68. The budgetary impact is not expected to be significant, in the context of efforts already planned to strengthen Fund lending and policies. The cost of updating the Operational Guidance Note on Conditionality will be manageable, and subsequent implementation will likely require a change in culture and approach, rather than significant additional investment or practices/processes. Most other proposals (e.g., sharper MAC DSA tools, continued building of expertise in critical shared areas of responsibility, and monetary policy conditionality review) are part of ongoing or planned workstreams that are already included in the medium-term budget. The proposals for Communication Department (COM) TA and work to explore longer Fund engagement would entail additional costs, which would need to be considered alongside other priorities within the existing budget process.

69. If successfully implemented, the recommendations should reduce risks to lending operations, and the Fund more generally. Many of the proposals should increase the likelihood of program success and reduce the risks to the use of Fund resources. The recommendation to improve forecast realism, MAC DSA tools, debt transparency, and tailoring to fragile and small states, as well as to develop contingency planning, could also have positive spillovers to Fund surveillance. Better prioritization of reforms based on surveillance and TA gaps would support further integration of core functions. Improved program design, performance, and communication would also help protect the Fund’s reputation. While longer programs could increase short-term risks to the revolving use and adequacy of Fund resources, these could be mitigated with appropriate safeguards. Over time, successful longer programs should also reduce risks by lowering the frequency of successor arrangements.

Conclusion

70. With strong encouragement from the membership, the Fund stepped up efforts to provide financial support to member countries during the difficult post-GFC period. The 2018 RoC period was dominated by persistent structural challenges that required large-scale and long-lasting adjustment. At the same time, the protracted recovery from the GFC also weighed heavily on external demand, hampering macroeconomic adjustment and creating headwinds to difficult structural reforms. This risky and uncertain environment provided the backdrop for most Fund-supported programs, as the institution faced difficult dilemmas and tradeoffs on how to achieve program success in such an environment. Ultimately, the Fund took significant risks to support the membership, as acknowledged in the staff reports accompanying most program requests. In this context, the limited success of some Fund-supported program should not come as a surprise.

71. Program design and conditionality involve significant tradeoffs. Based on the lessons from the 2018 RoC, certain tradeoffs need to be re-assessed:

  • Realism versus ambition. More scrutiny of program baselines is required, but forecasting is an imperfect science, and some degree of error should be expected. Program design should tackle uncertainty through analysis of downside scenarios and contingency planning. However, there could be an inevitable trade-off as less ambition could potentially undermine reform momentum.

  • Granularity versus flexibility. More granular conditionality, for example on revenue and spending measures, could help deliver higher quality of adjustment. However, this would also imply a significant reduction in program flexibility, and the ability to respond to shocks. This could also test ownership in an often-difficult political environment.

  • Gradualism versus speed. Implementation of complex reform agendas in the face of protracted structural challenges requires time, ownership, capacity building, and substantial financing. Therefore, a more gradual approach, which would justify longer EFF arrangements, may work better. Alternatively, disbursing programs could focus on macroeconomic stabilization with PCIs used increasingly to support medium-term structural agendas. However, a planning horizon longer than four years may not always work with political cycles and may increase the risks of reform fatigue.

  • Parsimony versus more conditionality. Given the deep-seated structural challenges that the membership faces, the Fund needs to continue building expertise on structural issues. The design and tailoring of structural conditionality will need to be improved to maximize the impact of reforms. Fewer but deeper reforms may yield better results. Achieving parsimony may require the Fund to attach more conditionality to critical shared and non-core areas of responsibility at the expense of non-critical traditional core areas.

  • Debt operation versus adjustment. Sharper DSA tools would help mitigate any bias in judgment on debt sustainability and ensure more balanced consideration of the trade-off on a case-by-case basis.

72. Ultimately, these tradeoffs need to be carefully weighed to mitigate risks to the Fund. Paying more attention to realism, granularity, gradualism, parsimony and in some cases debt operation options, when weighing program design tradeoffs, would improve the chances of program success.

Issues for Discussion

73. Directors may want to consider the following issues:

  • Do Directors agree with the overall assessment of program success?

  • Do Directors agree with the RoC recommendations?

  • How do Directors view the trade-offs and risks inherent in program design?

  • Do Directors agree that the Guidelines on Conditionality remain broadly appropriate, but that staff guidance should be updated in line with recommendations?

  • Do Directors see the need for a follow-up Board paper on a possible longer duration of EFF arrangements?

Table 2.

2018 Review of Conditionality Roadmap

article image

Appendix I. 2018 RoC Analytical Country Groups

article image

Appendix II. Definitions

A. Structural Conditions: Classification

1. SCs span all sectors of the economy and are classified by category, area of responsibility, and implementation success. The SCs are categorized into areas of responsibility; that is, core, shared, and non-core areas of Fund responsibility (Table 1). This largely follows the classification used by the IEO (2007) and the 2011 RoC, with the only change being the shift in classification of financial sector SCs from shared to core for both AMs and EMs/LIC, reflecting the increasing importance of financial sector issues to the Fund during the period.

Table 1.

Classification of Structural Conditions by Subject

article image
Note: Font color indicates area of Fund responsibility: core, shared, non-core.

B. Structural Conditions: Depth

2. SCs are classified into measures of high, medium, or low depth. Following the methodology of the 2011 RoC, as well as the IEO (2008) classification, structural reforms were grouped based on their implied change and durability, if implemented fully.

  • High depth. Reforms that lead to permanent institutional changes, such as by involving legislative changes (parliamentary approval), or conditions with long-lasting impact (e.g., civil service reforms, SOE reforms, privatization).

  • Medium depth. Reforms that lead to a significant change but are one-off in nature (e.g., budget approval or one-time change in tariff rates as compared to a permanent change such as institutionalizing an automatic tariff adjustment mechanism).

  • Low depth. Reforms that in themselves do not bring about change but are steps towards a change.

C. Ownership: Definition

3. National ownership is defined as follows: “A commitment to a program of policies, by country officials who have the responsibility to formulate and carry out those policies, based on an understanding that the program is achievable and is in the country’s best interest” (IMF, 2002b).

D. Ownership: Program Completion Rates

4. Program completion rates are sorted into six groups:

  • Completed. All reviews completed during the program period, including if they were completed with delays, after rephasing, or during a program extension.

  • Largely implemented. All but one review has been completed after accounting for a potential change in the number of reviews following rephasing of the review schedule.

  • Off track mid-program. At least two reviews were completed and at least two reviews were not completed at the end of the program (i.e., the program did not come back on track).

  • Quickly off track. At most one review was completed and at least two reviews were not completed at the end of the program.

  • Replaced. Captures programs (that are not “completed” or “largely implemented”) where a successor program was approved, while another program was still ongoing. This would be the case when a program request includes a request for cancellation of an existing arrangement.

  • In progress. Captures programs that were still active as of October 1, 2018.

References