2018 Review of Program Design and Conditionality—Supplementary Information

International Monetary Fund. Strategy, Policy, &amp; Review Department

doi:10.5089/9781498315715.007.A002

Jump to Content Jump to Main Navigation

2018 Review of Program Design and Conditionality—Supplementary Information

Author: International Monetary Fund. Strategy, Policy, & and Review Department

Publication Date:: 20 May 2019

eISBN:: 9781498315715

Language:: English

Keywords:: PP; director; Fund staff; program design; Fund program performance; Fund engagement; IMF staff calculation; program success; IMF program detail; program completion; Balance of payments need; Debt restructuring; Global

Download PDF (2.7 MB)

2018 Review of Program Design and Conditionality

Abstract

2018 Review of Program Design and Conditionality

2018 ROC Surveys

Staff surveyed mission chiefs, resident representatives, Executive Directors, and country authorities in countries with programs during the RoC period. While the surveys were tailored to the four groups, all recipients were asked to reflect on ownership, tailoring and uniformity of treatment, as well as other program design issues.¹

A. Ownership

1. Despite a moderation since 2011, perceptions of the authorities’ program ownership remain broadly positive. About half of mission chiefs (MCs) and resident representatives (RRs) rated program ownership as “high” or “very high” and about one third as “moderate” (Figure 1, top panels). In addition, MCs/RRs and country authorities (CAUTs) and Executive Directors (EDs) overwhelmingly agreed that program quantitative performance criteria (QPCs) targeted the appropriate macroeconomic variables. The vast majority of MCs/RRs and CAUTs also agreed that structural reforms were consistent with national reform priorities, and that program objectives were consistent with domestic economic and social priorities, though EDs were more cautious (Figure 1, bottom left panel). Furthermore, roughly three-quarters of all respondents agreed or strongly agreed that program design was sufficiently flexible to accommodate external shocks (Figure 1, bottom right panel).

2. While program design is often perceived as flexible, there may be a need to pay additional attention to capacity constraints. A slight majority of respondents found program design sufficiently flexible to accommodate changing government priorities (Figure 2, left panel). Yet, approximately 15 percent disagreed or strongly disagreed with this statement and 20 to 25 percent were neutral. Furthermore, 20 percent of MCs/RRs and CAUTs and more than a quarter of EDs disagreed or strongly disagreed that the program implementation timeline was consistent with the authorities’ existing technical capacity to implement reforms (Figure 2, right panel). Around 20 percent of MCs/RRs also disagreed/strongly disagreed that program implementation was consistent with program commitments. Most MCs/RRs attributed unsatisfactory program implementation to weak capacity and lack of ownership, while CAUTs cited unexpected developments or exogenous shocks (Figure 3).

3. There is potential scope for longer duration of some programs to help achieve objectives. Roughly a quarter of MCs/RRs and EDs disagreed that the duration of their program was sufficient to accomplish its overall objectives (Figure 4, left panel). More than half of MCs/RRs and CAUTs reported follow-up programs for their assigned country during the review period. The reason most frequently cited by MCs/RRs was that “the previous program was not fully implemented”, while CAUTs highlighted “the need for the policy support” (Figure 4, right panel).

B. Tailoring and Uniformity of Treatment

4. Access decisions are viewed broadly as evenhanded. A significant majority of MCs/RRs and somewhat lower percentages of EDs and CAUTs agreed that Fund-supported programs struck the right balance between policy adjustment and programmed financing (Figure 5). A sizable majority of all respondents agreed that financing reflected fairly countries’ balance of payments needs, program strength, and repayment capacity, though agreement was somewhat lower for EDs and CAUTs.

5. Conditionality is regarded as somewhat less evenhanded. A significant minority of respondents disagreed or strongly disagreed that “Fund-supported programs have similar conditionality across countries with similar characteristics and qualifications,” pointing to concerns about uniformity of treatment among some stakeholders (Figure 6).

6. Perceptions of tailoring in program design remain positive but may have become somewhat less pronounced. In 2011, over 90 percent of respondents to the MC, RR, and CAUT surveys agreed or strongly agreed that program design took country circumstances into account (Figure 7, left panels). This share declined to about three-quarters in 2018 (Figure 7, right panels). A similar trend was observed regarding the tailoring of program objectives with domestic economic and social priorities (Figure 8). This may reflect challenges related to tailoring more structural programs in the 2018 sample, or the addition of “neutral” and “not applicable” responses in the 2018 surveys.

C. Collaboration

7. Teams have generally coordinated or collaborated with development partners. This is particularly the case in Poverty Reduction and Growth Trust (PRGT) programs (Figure 9, top panels). A large majority of respondents from all surveys felt that Fund coordination with bilateral and multilateral donors had been effective, and that IMF policy advice was consistent with that of other international institutions (Figure 9, bottom left panel). Further, Fund-supported programs leveraged outside expertise to support the design of social sector conditionality. Close to 80 percent of MCs and RRs agreed that coordination with the World Bank or other development partners helped facilitate an understanding of the social impact of consolidation measures under the program (Figure 9, bottom right panel).

8. The surveys point to a need to better assess the social and welfare impacts of program policies. A quarter of EDs and close to 20 percent of CAUTs disagreed or strongly disagreed that program design was consistent with protecting vulnerable groups, while 20 percent of EDs felt similarly about distributional effects (Figure 10, left panel). A comparable percentage of CAUTs disagreed or strongly disagreed that programs took into account the social costs of reform implementation. These concerns were less pronounced among MCs/RRs (Figure 10, right panel).

D. Outreach

9. While the surveys point to a significant improvement in outreach relative to 2011, scope remains to expand outreach to in-country civil society organizations (CSOs). Roughly a quarter of MCs/RRs, 18 percent of CAUTs, and 16 percent of EDs disagreed or strongly disagreed that CSOs were actively involved in the design of Fund-supported programs (a third of EDs responded “don’t know”) (Figure 11, left panel). Further, around a quarter of MCs/RRs disagreed or strongly disagreed that the authorities had publicly explained the benefits of the Fund-supported program to civil society, with another 15 percent neutral (Figure 11, right panel). See also Box 1 for outreach to CSOs in the context of the 2018 RoC.

Box 1.

2018 Review of Program Design and Conditionality: Outreach

The 2018 RoC benefited from outreach. The Fund invited online comments from external stakeholders, including CSOs. This was followed by a conference call with Fund staff to discuss their views.

Comments covered a range of issues:

Shift in types of arrangements. Some stakeholders noted the shift in recent years from shorter-term SBAs to longer-term EFF arrangements. They encouraged the Fund to explore the drivers of this phenomenon.
Ownership. Some participants emphasized the need for a clearer definition of ownership and a more focused assessment thereof in Fund-supported programs.
Program design. Many stakeholders emphasized that program design should be more attentive to the potential negative impacts of conditionality on public investment, inequality, and labor rights.
Gender issues. While participants welcomed the Fund’s efforts to incorporate gender issues in program design, they highlighted the need for a more systematic approach to gender issues in program conditionality.

Previous Policy Reviews

This section provides an overview of recent studies and reforms at the Fund and the Independent Evaluation Office that are relevant for program design and/or are follow-up actions to the 2011 RoC recommendations.

10. While providing an overall positive assessment, the 2011 RoC included a comprehensive set of recommendations. Overall, the 2011 RoC (IMF, 2012a) found that in most cases, programs succeeded in meeting their objectives, program design adapted flexibly to challenges, and the Conditionality Guidelines were followed appropriately. Recommendations of the 2011 RoC were addressed through a number of follow-up actions and workstreams (Table 1) and program design and conditionality have subsequently evolved in line with their implementation.

Table 1.

Follow-Up Actions on the 2011 RoC Recommendations

Source: IMF staff.

Table 1.

Follow-Up Actions on the 2011 RoC Recommendations

2011 RoC recommendation	Concluded follow-up action	Ongoing follow-up action	Overall assessment
Keeping focused— consolidating progress in streamlining conditionality	Fund Engagement with Countries in Fragile Situations, April 2012 Conditionality in Evolving Monetary Policy Regimes, March 2014, and Evolving Monetary Policy Frameworks in Low-Income and other Developing Countries, October 2015 Small States’ Resilience to Natural Disasters and Climate Change – Role for the IMF, November 2016, and Fund Engagement with Small Developing States, January 2018	Management Implementation Plan on Fragile States Building Resilience in Countries Vulnerable to Large Natural Disasters (forthcoming)	Ongoing progress
Improving risk diagnostics—tailoring robustness tests and strengthening DSAs	Public Debt Sustainability Analysis in Market-Access Countries, May 2013 Reform of the Public Debt Limits Policy in Fund-supported Programs, November 2014 Reform of the Fund’s Lending Framework, April 2015 (including Exceptional Access framework) Bank-Fund Debt Sustainability Framework for Low Income Countries, February 2018	Review of the MAC DSA Framework (ongoing) Review of the Debt Limits Policy (ongoing)	Ongoing progress
More consistent consideration of macro-social aspects, including dialogue with authorities and analysis of long-term benefits and short-term costs	Jobs and Growth Issues in Surveillance and Program Work, September 2013 IMF Engagement on Social Safeguards in Low-Income Countries, July 2018	IMF Engagement on Social Spending: A Strategic Framework (forthcoming) Pilots on Inequality and Gender (mainstreamed)	Substantial progress
Enhanced ownership and transparency, through discussion of alternative policy options, greater clarity in program documents, and new avenues to collect external views	Fund Engagement with Countries in Fragile Situations, April 2012 Small States’ Resilience to Natural Disasters and Climate Change – Role for the IMF, November 2016, and Fund Engagement with Small Developing States, January 2018 Review of the Fund’s External Communication Strategy with annual updates to the Executive Board	Building Resilience in Countries Vulnerable to Large Natural Disasters (forthcoming)	Ongoing progress
Leveraging surveillance, particularly through contingency planning and analysis	Modernizing the Legal Framework for Surveillance – An Integrated Surveillance Decision, June 2012 Approaches to Macrofinancial Surveillance, March 2017 (including preceding work ongoing since 2014) Macroprudential Policy, November 2014 Structural Reforms and Macroeconomic Performance, October 2015 (macrostructural pilot initiative) Governance – A Proposed Framework for Enhanced Fund Engagement, April 2018	Macro-structural pilot initiative (mainstreamed)	Substantial progress
Stronger partnerships with other institutions, including currency unions and RFAs	Collaboration between Regional Financing Arrangements and the IMF, July 2017 Program Design in Currency Unions, March 2018	Outreach to and engagement with RFAs (ongoing)	Substantial progress

Source: IMF staff.

Table 1.

Follow-Up Actions on the 2011 RoC Recommendations

2011 RoC recommendation	Concluded follow-up action	Ongoing follow-up action	Overall assessment
Keeping focused— consolidating progress in streamlining conditionality	Fund Engagement with Countries in Fragile Situations, April 2012 Conditionality in Evolving Monetary Policy Regimes, March 2014, and Evolving Monetary Policy Frameworks in Low-Income and other Developing Countries, October 2015 Small States’ Resilience to Natural Disasters and Climate Change – Role for the IMF, November 2016, and Fund Engagement with Small Developing States, January 2018	Management Implementation Plan on Fragile States Building Resilience in Countries Vulnerable to Large Natural Disasters (forthcoming)	Ongoing progress
Improving risk diagnostics—tailoring robustness tests and strengthening DSAs	Public Debt Sustainability Analysis in Market-Access Countries, May 2013 Reform of the Public Debt Limits Policy in Fund-supported Programs, November 2014 Reform of the Fund’s Lending Framework, April 2015 (including Exceptional Access framework) Bank-Fund Debt Sustainability Framework for Low Income Countries, February 2018	Review of the MAC DSA Framework (ongoing) Review of the Debt Limits Policy (ongoing)	Ongoing progress
More consistent consideration of macro-social aspects, including dialogue with authorities and analysis of long-term benefits and short-term costs	Jobs and Growth Issues in Surveillance and Program Work, September 2013 IMF Engagement on Social Safeguards in Low-Income Countries, July 2018	IMF Engagement on Social Spending: A Strategic Framework (forthcoming) Pilots on Inequality and Gender (mainstreamed)	Substantial progress
Enhanced ownership and transparency, through discussion of alternative policy options, greater clarity in program documents, and new avenues to collect external views	Fund Engagement with Countries in Fragile Situations, April 2012 Small States’ Resilience to Natural Disasters and Climate Change – Role for the IMF, November 2016, and Fund Engagement with Small Developing States, January 2018 Review of the Fund’s External Communication Strategy with annual updates to the Executive Board	Building Resilience in Countries Vulnerable to Large Natural Disasters (forthcoming)	Ongoing progress
Leveraging surveillance, particularly through contingency planning and analysis	Modernizing the Legal Framework for Surveillance – An Integrated Surveillance Decision, June 2012 Approaches to Macrofinancial Surveillance, March 2017 (including preceding work ongoing since 2014) Macroprudential Policy, November 2014 Structural Reforms and Macroeconomic Performance, October 2015 (macrostructural pilot initiative) Governance – A Proposed Framework for Enhanced Fund Engagement, April 2018	Macro-structural pilot initiative (mainstreamed)	Substantial progress
Stronger partnerships with other institutions, including currency unions and RFAs	Collaboration between Regional Financing Arrangements and the IMF, July 2017 Program Design in Currency Unions, March 2018	Outreach to and engagement with RFAs (ongoing)	Substantial progress

Source: IMF staff.

11. In addition, the 2015 Crisis Program Review concluded that GRA programs had helped chart a path through the Global Financial Crisis (GFC) and avoid a cataclysmic meltdown of the global economic system. It found that: (i) external adjustment, driven less by the exchange rate and more by demanding internal devaluation, had required ambitious structural reforms beyond the duration of a program; (ii) fiscal deficits had fallen in line with targets, but with greater-than-envisaged impact on output, in part due to underestimated fiscal multipliers; (iii) structural conditionality may need to be more extensive to support internal devaluation, but should recognize capacity limitations, the risk of reform fatigue, and that payoffs can be modest; (iv) the impact of private sector balance sheets on growth and fiscal adjustment was more severe than anticipated, with priorities including legal frameworks and out-of-court settlement, prudential measures to incentivize debt write-offs and restructuring; and (v) guidelines for cooperation with regional financial arrangements (RFAs) and currency unions should clarify the role of the Fund (e.g., macroeconomic analysis and debt sustainability analysis (DSAs)).

12. Recent Independent Evaluation Office (IEO) reports also included recommendations relevant to the 2018 RoC:

The IMF and the Crises in Greece, Ireland and Portugal (2016) found that programs incorporated overly optimistic growth projections, and that more realistic projections would have made clear the likely impact of fiscal consolidation on growth and debt dynamics.
The IMF and Social Protection (2017) found that IMF-supported programs almost always took account of social protection concerns, albeit with mixed success in implementation, recognizing the need to mitigate potential adverse impact of program measures on the most vulnerable. However, the report concluded that the Fund needs more realistic and effective approaches to conditionality to deliver these objectives.
The IMF and Fragile States (2018) concluded that despite the inherent challenges of limited capacity, weak governance, and an unstable political and security environment, IMF involvement had been quite effective, but efforts to adapt policies and practices to the needs of fragile states had been insufficient.
Structural conditionality in IMF-supported Programs—Evaluation Update (2018) found that structural conditionality had generally been streamlined, with conditions more focused in areas of IMF expertise.

Assessing Program Success

This section provides the analytical underpinnings for staff’s assessment of program success.

A. Methodology

13. A systematic assessment of “program success” requires a methodology that can be applied in a variety of circumstances across a very diverse membership. As defined in the Guidelines on Conditionality (IMF, 2002), Fund-supported programs are directed primarily to: (i) solving the member’s balance of payment (BoP) problems without recourse to measures destructive of national or international prosperity; and (ii) achieving medium-term external viability while fostering sustainable economic growth. To reflect the Fund’s diverse membership, there are important differences in how Fund-supported programs operationalize these overall objectives under the general resources account (GRA) and the PRGT. The RoC, hence, develops two separate frameworks reflecting these differences, as elaborated below. For each framework, programs are classified as either “successful,” “partially successful,” or “unsuccessful.”

Methodology for GRA-Supported Programs

14. Under a GRA-supported program, the definition of program success should incorporate evidence of no BoP need² and of medium-term external viability after program completion. The nature of post-program Fund engagement is used as a proxy for resolving a BoP problem, and the evolution of vulnerability indicators is considered when assessing medium-term external viability.

15. For the purposes of the RoC, post-program engagement is defined as the two-year period following a Fund arrangement, and separated into the following three categories:

Drawing successor programs. Defined as an arrangement of a financial nature (Stand-By Arrangement (SBA) and Extended Fund Facility (EFF)),³ excluding those with low access, with the cut-off set at a quarter of the (annual) exceptional access (EA) threshold.⁴
Successor programs of a signaling nature. For the purpose of this analysis, successor program engagements of a signaling nature include Policy Coordination Instruments (PCIs), and arrangements of a financial nature that are treated as fully precautionary or involve low access as defined above. These types of successor program engagements mainly aim to send signals of strong macroeconomic policies.
No successor program. Defined as a Fund-supported program that was not followed by another Fund arrangement or a PCI within two years of its completion or expiration.

16. With respect to the evolution of vulnerability indicators, the RoC draws on the Vulnerability Exercise (VE). The VE is a multisectoral approach to detect risks that could make a country vulnerable to BoP pressures (Ahuja, Syed, and Wiseman, 2017). It encompasses an expansive set of indicators, as well as staff’s judgment. As part of the exercise, IMF staff evaluates underlying vulnerabilities in the fiscal, external, and domestic financial sectors, as well as financial and asset pricing risks, where appropriate. A Fund-supported program that reduces the VE final overall rating is regarded as successfully addressing macroeconomic imbalances.

17. Information from both the nature of post-program Fund engagement and the vulnerability indicators is then combined to determine program success. The change of the vulnerability indicators between program inception and program completion is represented by two separate transition matrices depending on the nature of post-program engagement (Figure 12). Programs are categorized based on the color coding in the transition matrix: “successful” (green), “partially successful” (orange), or “unsuccessful” (red). The transition matrix for programs with a BoP need post-program is more red (less green) than the transition matrix for signaling/no successor arrangement.

BoP need post-program (Figure 12, left panel): Programs that reduced vulnerabilities to low or lowered them from high are considered partially successful. All remaining programs are considered unsuccessful, as the BoP need was unresolved, or vulnerabilities did not improve.
No BoP need post-program (Figure 12, right panel): Programs that ended with low vulnerabilities or reduced vulnerabilities from high to medium are considered successful. Programs that maintained vulnerabilities at high and medium levels are considered partially successful, as the BoP need was resolved. Programs are considered unsuccessful only if vulnerabilities increased during the program period.

Methodology for PRGT-Supported Programs and Policy Support Instruments (PSIs)

18. The Fund’s concessional facilities and instruments are aimed at providing flexible and tailored support to low-income countries (LICs). The RoC analyzes three types of LIC instruments:

ECF arrangements. The purpose of an ECF arrangement is to assist PRGT-eligible member countries with a protracted BoP problem in implementing economic programs aimed at making significant progress toward a stable and sustainable macroeconomic position consistent with strong and durable poverty reduction and growth. Given the protracted nature of the BoP problem, repeated use of ECF arrangements is not necessarily a sign of insufficient progress.
SCF arrangements. The purpose of an SCF arrangement is to assist eligible member countries with short-term BoP needs in implementing economic programs aimed at achieving, maintaining, or restoring a stable and sustainable macroeconomic position consistent with strong and durable poverty reduction and growth.
PSI. The PSI is a tool that enables PRGT-eligible members with no BoP need to secure Fund advice and policy support without a borrowing arrangement. This support from the Fund also delivers clear signals to donors, creditors, and the general public about the strength of the member’s economic policies to deliver durable poverty reduction and growth.

19. The RoC employs a two-step approach to measure program success in PRGT cases. In stage one, program performance is assessed against the evolution of external debt vulnerabilities. In stage two, program performance is assessed against performance on relevant macroeconomic indicators (Figure 13).

Stage 1. Evaluating sustainability of policy frameworks. Similar to the GRA methodology, a transition matrix is employed to identify unsuccessful cases (Figure 14). The ratings are based on the LIC Debt Sustainability Framework (LIC-DSF).⁵ A program is assessed to be “unsuccessful” (red) if there are substantial risks to external public and publicly guaranteed debt sustainability, with the rating either: (i) remaining in debt distress (“DD”); or (ii) increasing to “in DD” or high (“H”). In these cases, the sustainability of fiscal policy to address poverty and development needs is in serious doubt, therefore justifying the “unsuccessful” label. Programs that are outside of the red area proceed to the second stage.
Stage 2. Distributing programs into three categories. The second stage uses five indicators to assess program success.
- i. The five indicators can be divided into three groups: (i) proxies for anti-poverty spending policy, comprising social expenditure (health and education expenditures) and government capital expenditure; (ii) non-grant fiscal revenue (capturing progress on domestic revenue mobilization); and (iii) macroeconomic stability, proxied by inflation and real GDP growth.
- ii. Indicators are considered met when the average projection error (actual minus projected) during T+1 to T+3 has a favorable sign. T is defined as the program approval year.⁶
- iii. A program is considered “successful” when three or more indicators were met, “partially successful” when one or two indicators were met, and “unsuccessful” when no indicator was met.⁷

B. Data and Degree of Success

20. Program success is assessed for 78 programs (Figures 15 and 16). All cases assessed here are part of the 2018 RoC sample. Ongoing programs (as of end-September 2018) were excluded. The sample is further reduced by data constraints.

GRA. This exercise covers 28 completed/expired GRA programs out of 52 GRA programs in the 2018 RoC sample. 11 cases did not have VE ratings and 13 cases were still ongoing.
PRGT. The exercise covers 50 completed/expired PRGT programs out of 81 PRGT programs in the 2018 RoC sample. 23 PRGT programs were still ongoing.

21. Analytical groups had varying degrees of success, and success appears to have been possible under a large set of circumstances (Figure 17). In particular, post-GFC programs had higher success rates than commodity exporters and other developing countries (Figure 17, top left panel). Political/economic transformation countries had both the highest success and failure rates of all groups. Although programs approved in any given year faced different environments and shocks, at least 20 percent of programs were successful in any year of program approval (Figure 17, top right panel). This could be interpreted as a positive message that program success could be achieved under various circumstances, underpinned by flexible program design that could adapt to those circumstances. Exceptional access (EA) programs had broadly similar success rates (Figure 17, 2^nd panel, left).

C. Success Factors

22. The small sample size constrains the analysis of success factors. A multinomial logit regression approach can link program outcomes with several explanatory variables (Table 2). These include initial conditions, shocks throughout the program period, country characteristics, and program design elements. Given the small sample of 78 observations, we first identify the explanatory variables that are individually statistically significant predictors and then conduct regressions with combinations of such variables. Due to the small sample size, combined regressions do not show statistically significant coefficients and are not reported.

Table 2.

Explanatory Variables for Program Success in GRA and PRGT Countries

Source: IMF staff calculations.Notes: Stars denote significance: *** p<0.01, ** p<0.05, * p<0.1. † 0.1 <p<0.2.Multinomial logit regressions are conducted with partially successful indicated as the base case. In the logit specification, the dependent variables are log(prob(successful)/prob(partially successful)) and log(prob(unsuccessful)/prob(partially successful)).

^{^1/}Completed programs are coded as 1, off-track or quickly off-track as zero. Lapsed or replaced programs (total of 6) did not enter the regression.

^{^2/}Average GDP forecast error over T+1 to T+3 where T is the program approval year. Forecast errors for trading partner growth and commodity prices as faced by country i for year t as reported in the j’s WEO forecast vintage. These are then averaged over the program period.

Table 2.

Explanatory Variables for Program Success in GRA and PRGT Countries

Dependent variable: log of relative probability of being successful or unsuccessful	Successful	Unsuccessful	Successful	Unsuccessful	Successful	Unsuccessful	Successful	Unsuccessful	Successful	Unsuccessful	Successful	Unsuccessful
*GRA sample:*
Completion status (1=completed, 0=off-track) ^1/	2.4**	-1.0
Growth forecast error (percent) ^2/			0.4	-1.5*
*PRGT sample:*
Completion status (1=completed, 0=off-track) ^1/					16.5	-0.8^†
Forecast error of commodity prices (percent) ^2/							0	-0.1 ^†
Fragile state dummy (Fragile=1)									-1.0^†	-0.5
*Combined GRA and PRGT samples:*
Having an IMF program in the past 5 years											-0.4	-0.7^†
Observations	26		28		46		50		50		78
Pseudo-R2	0.15		0.25		0.14		0.02		0.02		0.01

^{^1/}Completed programs are coded as 1, off-track or quickly off-track as zero. Lapsed or replaced programs (total of 6) did not enter the regression.

Table 2.

Explanatory Variables for Program Success in GRA and PRGT Countries

Dependent variable: log of relative probability of being successful or unsuccessful	Successful	Unsuccessful	Successful	Unsuccessful	Successful	Unsuccessful	Successful	Unsuccessful	Successful	Unsuccessful	Successful	Unsuccessful
*GRA sample:*
Completion status (1=completed, 0=off-track) ^1/	2.4**	-1.0
Growth forecast error (percent) ^2/			0.4	-1.5*
*PRGT sample:*
Completion status (1=completed, 0=off-track) ^1/					16.5	-0.8^†
Forecast error of commodity prices (percent) ^2/							0	-0.1 ^†
Fragile state dummy (Fragile=1)									-1.0^†	-0.5
*Combined GRA and PRGT samples:*
Having an IMF program in the past 5 years											-0.4	-0.7^†
Observations	26		28		46		50		50		78
Pseudo-R2	0.15		0.25		0.14		0.02		0.02		0.01

^{^1/}Completed programs are coded as 1, off-track or quickly off-track as zero. Lapsed or replaced programs (total of 6) did not enter the regression.

23. Several explanatory variables are statistically significant in pairwise regressions with economically meaningful coefficients. For GRA programs, program completion and growth forecast errors (measured as GDP growth forecast errors over the program period compared to initial expectations) are significant predictors of success (Table 2). For PRGT programs, program completion is also a significant predictor of success, with fragility and negative commodity shocks reducing the probability of success. To gauge the economic significance of these results, the predictive probabilities of each outcome bucket are calculated (Figure 18). For GRA programs, program completion increases the chance of success by 49 percentage points, and negative GDP forecast error of one standard deviation increases the probability of an unsuccessful program by 32 percentage points. For PRGT programs, program completion increases the probability of success by about 40 percentage points, while a one standard deviation negative commodity shock increases the probability of an unsuccessful outcome by 7 percentage points.

24. There are important caveats to this analysis. A large set of variables were tested that did not provide statistically significant predictive powers, including the country’s type of exchange rate regime, the public debt-to-GDP ratio in the year of program approval, and forecast errors in the growth of trading partners in PRGT cases. The lack of statistical significance within the sample, however, does not imply such variables are not important for individual countries. Moreover, it is inevitable that programs also face non-economic shocks, such as wars, epidemics, and political turmoil that are not accounted for in this exercise. Given the small sample, disentangling causality from association is a challenge. Causality in some cases may run both ways, for example, growth optimism could undermine program success, but challenging programs also entail traits that make accurate forecasts more difficult. Either way, it remains crucial to strive for better macroeconomic forecasts to underpin program design.

Recent Fund Experience with Debt Restructuring and Reprofiling

This section discusses Fund experience with debt restructuring and reprofiling during the 2018 RoC period.⁸

25. When it is clear that debt is unsustainable, debt restructuring is required for the Fund to provide financial support, but in practice such operations have often been delayed. Where debt is unsustainable, the extent of feasible economic adjustment combined with new borrowing is not sufficient to address the member’s underlying BoP problem, also given new borrowing may actually exacerbate the member’s solvency issues. In such cases, steps should be taken to restore debt sustainability and enable the Fund to provide financial support. However, there is often a significant delay between staff’s initial assessment that a member’s debt is unsustainable and the actual implementation of debt restructuring. Such examples include even ultimately successful restructuring cases: Seychelles, where staff noted that debt was unsustainable in the 2003 Article IV staff report but restructuring only began in 2009–10 after default in 2008, and St. Kitts and Nevis, where Article IV staff reports showed debt on an explosive path from 2006, but restructuring was only announced in 2011.

26. In some cases, delayed restructuring has resulted in the claims of private creditors being replaced by those of the official sector. Most prominently, during Greece’s 2010 SBA, staff assessed that debt was sustainable but not with high probability. Nonetheless, the Fund approved the SBA involving EA because the Executive Board modified the second criterion under the EA policy for all members going forward to enable the Fund to lend if there was a risk of significant systemic spillover effects. Contagion was a major concern for euro area members, given banking exposures to the crisis-hit euro area countries and the absence of a firewall. The decision not to restructure debt at the outset of the crisis allowed some €40 billion (around 20 percent of 2011 GDP) in maturing bonds to be fully repaid in the first year of the SBA. The restructuring was announced in July 2011, but drawn-out negotiations meant that some further €10 billion (around 5 percent of 2012 GDP) continued to be repaid in full until the restructuring was completed in 2012. The systemic exemption under the EA policy was subsequently used for Ireland, Portugal and the 2012 EFF arrangement for Greece, before being eliminated in 2015. The delayed debt reprofiling operation under the Ukraine 2014 SBA also resulted in larger near-term financing needs of about US$7.5 billion (around 9 percent of 2015 GDP) in 2014–15.

27. In such cases, the reluctance to restructure debt may have been driven by concerns about the economic, financial, and political fallout, as well as spillover effects in the region. Such concerns can be particularly acute if the domestic financial sector holds a significant amount of public debt. Authorities may also be concerned about the impact of restructuring on market re-access and spillover effects on the private sector. Furthermore, official creditors may sometimes contribute to delays, out of concern that a restructuring would reduce incentives for adjustment. Private creditors naturally wish to avoid debt restructuring and press for a bailout by the official sector. The fear of contagion is an additional factor that may create delay. This is most acute where the economy of the debtor is closely integrated with other economies, such as in the case of Greece.

28. In cases where debt sustainability is uncertain, reprofiling can be effective in reducing debt vulnerabilities. While reprofiling involves costs, due to the triggering of a credit event and rating downgrade, investors can react positively if they think the reprofiling resolves the underlying problems that led to the loss of market access. Resources that would otherwise have been paid out to creditors will be retained, relieving financing pressures and enabling a less constraining fiscal adjustment path under a Fund-supported program. More gradual adjustment can be particularly beneficial in a high-multiplier crisis, postponing part of the adjustment to a point in time when multipliers will be lower. A stylized calibrated Fund model of output, fiscal policy, and debt accumulation suggests that with more gradual adjustment, higher GDP growth is preserved, and the output gap is smaller in the first years of the crisis, while potential GDP is permanently higher than in a non-reprofiling scenario. Moreover, reprofiling reduces future haircuts, benefiting longer-term creditors and increasing the likelihood of a rapid return to the market, as the debt stock will be less burdened by senior claims from official creditors.

29. In these circumstances, staff analysis suggests that reprofiling is often less costly than nominal value debt restructuring. The impact of reprofiling has typically been less severe than restructuring, resulting in: (i) lower sovereign spreads (at the time of announcement and debt exchange); (ii) less severe sovereign credit rating downgrades and faster recoveries within 12 months; and (iii) faster restoration of market access (i.e., a new global bond issuance or normalization of spreads) (IMF, 2014a). Negotiations also tend to be shorter in reprofiling cases with higher participation rates and fewer litigations than restructuring cases. Reprofiling can also have significant benefits for the Fund by reducing the amount of Fund financing required and strengthening the member’s position to regain financial stability and external viability, and its capacity to repay. To the extent that reprofiling helps mitigate moral hazard, it can also reduce the incidence of future crises, benefitting the international monetary system.

30. The impact of reprofiling on the domestic financial system has also tended to be relatively limited (IMF, 2014b). Debt operations can have a significant impact on the financial system through a number of channels. Sovereign stress can affect domestic banks through direct exposure to the sovereign and indirectly given the sovereign’s role as backstop to the financial system. If banks suffer mark-to-market losses on their holdings of government bonds, they could become undercapitalized. Concerns about the health of such banks could lead to deposit runs, spilling over to otherwise healthy banks. Exchange rate depreciation associated with worsening market sentiment could also exacerbate bank FX funding costs and expose unhedged FX borrowers. While these risks can never be fully avoided, recent reprofiling cases suggests that they can be successfully mitigated through carefully designed operations and Fund programs. In the two debt operations in Jamaica (2010 and 2013), some domestically-held debt was excluded from reprofiling, regulatory incentives were provided for banks, and capital and liquidity support mechanisms were established.

Staff-Monitored Programs

This section finds that SMPs were used parsimoniously to either build a track record or clear arrears. SMPs were largely successful in building a track record towards an Upper Credit Trance (UCT)-quality program.

31. Staff-monitored programs (SMPs) are informal arrangements that can fulfil multiple objectives. SMPs are informal agreements between national authorities and Fund staff (without endorsement by the Executive Board) to monitor the implementation of the authorities’ economic program with a view to establishing a track record of policy implementation. As such, SMPs are used to: (i) establish a track record to meet conditions for a full-fledged Fund-supported financing arrangement, sometimes in conjunction with disaster-related financing arrangements (e.g., Rapid Credit Facility (RCF) and Rapid Financing Instrument (RFI)); (ii) support the authorities’ efforts in clearing arrears; and (iii) help put an existing off-track arrangement back on track. Overall, the use of SMPs has declined in recent years, with 12 SMPs during the 2018 RoC period compared to 28 during the 2011 RoC period.

32. SMPs helped address weak implementation capacity. SMPs were predominantly used by countries with limited institutional capacity, domestic fragility or instability, or weak economic policy implementation. Countries entering an SMP were characterized by lower Country Policy and Institutional Assessment (CPIA) scores (Figure 19).⁹ On average, members entering SMPs had significantly weaker CPIA scores than LICs in UCT-quality programs without a prior SMP, or non-LICs in UCT-quality programs.

33. SMPs with satisfactory performance were often followed by a UCT-quality program. During the 2018 RoC, six members used an SMP to build a track record; of these, two were used in conjunction with Fund emergency assistance. The four successful SMPs all led to a UCT-quality program within two years of the SMP approval (Table 3). There was mixed success with SMPs that were used for clearing arrears.

Table 3.

SMPs During the 2018 RoC Period

Sources:MONA and IMF staff calculations.

^{^1/}Program ongoing.

^{^2/}Standard deviation from the average IDA country in the year of program approval.

Table 3.

SMPs During the 2018 RoC Period

Country	SMP approval	Main purpose	CPIA index	CPIA score deviation^2/	Successor UCT-quality program?	Successor program approval
Afghanistan	Apr-15	Track record	2.7	-1.1	ECF^1/	Jul-16
Chad	Apr-13	Track record	2.6	-1.3	ECF	Aug-14
Comoros	Nov-16	Track record	2.9	-0.7
Gambia, The	Apr-17	Track record/RCF	3.0	-0.4
Iraq	Jan-16	Track record/RFI	..	..	SBA	Jul-16
Madagascar	Sep-15	Track record	3.1	-0.1	ECF^1/	Jul-16
Myanmar	Jan-13	Arrears clearance	3.0	-0.6
Somalia	May-16	Arrears clearance	1.8	-2.8
Somalia	May-17	Arrears clearance	1.8	-2.7
Sudan	Jan-14	Arrears clearance	2.4	-1.8
Zimbabwe	Apr-13	Arrears clearance	2.7	-1.3
Zimbabwe	Oct-14	Arrears clearance	2.3	-2.0

Sources:MONA and IMF staff calculations.

^{^1/}Program ongoing.

^{^2/}Standard deviation from the average IDA country in the year of program approval.

Table 3.

SMPs During the 2018 RoC Period

Country	SMP approval	Main purpose	CPIA index	CPIA score deviation^2/	Successor UCT-quality program?	Successor program approval
Afghanistan	Apr-15	Track record	2.7	-1.1	ECF^1/	Jul-16
Chad	Apr-13	Track record	2.6	-1.3	ECF	Aug-14
Comoros	Nov-16	Track record	2.9	-0.7
Gambia, The	Apr-17	Track record/RCF	3.0	-0.4
Iraq	Jan-16	Track record/RFI	..	..	SBA	Jul-16
Madagascar	Sep-15	Track record	3.1	-0.1	ECF^1/	Jul-16
Myanmar	Jan-13	Arrears clearance	3.0	-0.6
Somalia	May-16	Arrears clearance	1.8	-2.8
Somalia	May-17	Arrears clearance	1.8	-2.7
Sudan	Jan-14	Arrears clearance	2.4	-1.8
Zimbabwe	Apr-13	Arrears clearance	2.7	-1.3
Zimbabwe	Oct-14	Arrears clearance	2.3	-2.0

Sources:MONA and IMF staff calculations.

^{^1/}Program ongoing.

^{^2/}Standard deviation from the average IDA country in the year of program approval.

34. SMPs were rarely used in situations where Fund-supported programs went off track. While a quarter of GRA and PRGT programs went off track during the 2011 and 2018 RoC periods— partly reflecting capacity constraints and other factors impacting ownership—few countries with off-track programs used an SMP. Only two SMPs were initiated in the 2011 RoC period to bring a program back on track, and no SMP was initiated for this purpose during the 2018 RoC period. Kosovo’s 2011 SMP succeeded in bridging to a successor SBA within one year, while Congo’s performance under the 2007 SMP was not sufficiently satisfactory to bring its 2004 PRGF-supported program back on track.

Tailoring and Uniformity of Treatment: Fragile States and Small States

Some stakeholders have questioned the adequacy of tailoring to fragile and small states, given heightened vulnerabilities and weak capacity. In this context, this section discusses the experience with tailoring of conditionality in these countries.

A. Fragile States

35. Program performance among fragile states was weaker than in the full sample. During the 2018 RoC sample period, 26 fragile states (3 GRA and 23 PRGT-eligible countries) engaged in 49 Fund-supported programs (41 ECFs, 1 EFF, and 7 SBAs).¹⁰ On average, the number of quantitative conditions and SBs were broadly comparable to the rest of the 2018 RoC sample (Figure 20, top panels), and completed reviews indicate only slightly weaker implementation (Figure 20, bottom panels). However, about half of these programs did not complete all reviews and went off track. In some cases, countries entered an SMP to build a policy track record prior to requesting a program, including Afghanistan, Chad, and Madagascar (see Section V).

36. Structural conditionality focused on multiple areas of reform needs, and on average the number of SBs remained high. Program measures focused on stabilization and capacity-building to help build public support and maintain social stability. Accordingly, SBs tended to target public financial management (PFM) and revenue administration, and social measures (see Figure 21 in the main paper) but also other structural reforms. On average, the number of SBs in fragile states was broadly the same as for the full sample throughout most of the 2018 RoC period and even increased in 2017 (Figure 20, top right panel), despite significant capacity constraints in fragile states.

37. Supported by the Fund’s capacity-building framework in fragile states, TA provision to program countries was scaled up and prioritized (Figure 21).¹¹ TA was largely provided in the fiscal area, building capacity in both revenue (e.g., simple taxes requiring limited capacity) and expenditure areas.

38. Good practices in conditionality design and TA support were identified in two case studies. Kosovo (2015 SBA) and Mali (2013 ECF) had fewer structural conditions (SCs) compared to the average for fragile states, with well-sequenced sets of SCs attuned to capacity, and critical program-related TA support.

Kosovo (2015 SBA). Program implementation in this post-conflict state faced several challenges, including weak governance and corruption. Considering these institutional weaknesses, Kosovo’s 2015 SBA focused on key reforms to reduce its budget deficit and restore credibility of the fiscal rule, rebuild fiscal buffers, and improve the composition of fiscal spending. In addition, the program sought to implement risk-based supervision. Conditionality consisted of SBs that broke down reforms into intermediate steps. For instance, the implementation of a general procurement law was broken down into four steps over three reviews, supported by a close collaboration with donor agencies that provided extensive TA (Figure 22, left panel). Implementing risk-based supervision was supported by two SBs on on-site exams and subsequent roll-out of reports over the course of two reviews, supported by a TA mission (Figure 22, right panel).
Mali (2013 ECF). Growing insurgency and a rise in terrorism in 2011, combined with severe drought to put Mali among the 10 poorest nations, adding to its fragility. This period witnessed a sharp decline in economic growth, government revenues, volatile terms of trade, and banking system problems. Program objectives and TA focused on customs and tax administration, mining and petroleum fiscal regimes, PFM frameworks, and debt management. On fiscal expenditure, conditionality and TA focused on fuel subsidy reforms which were monitored through a set of SBs including on communication strategy (Figure 24, left panel). On the financial sector, TA supported the program objectives of strengthening financial stability (Figure 24, right panel). The SB on developing a strategy for reducing non-collateralized NPLs was, however, first modified and later not met.

B. Small States

39. Small states are often repeat users of Fund-supported arrangements, reflecting deep-seated structural challenges. During the 2018 ROC period, four small states engaged in six Fund-supported programs: Solomon Islands (2011 precautionary SCF, followed by a 2012 successor ECF), Sao Tomé and Príncipe (two consecutive ECFs in 2012 and 2015), Grenada (2014 ECF), and Seychelles (2014 EFF).¹²

40. In line with Fund guidance on engagement with small states, program design has increasingly reflected the specific challenges and capacity constraints faced by such countries. The revised 2017 operational guidance note points to the need to focus program design in small states on: (i) growth-friendly fiscal consolidation, particularly in heavily-indebted small states; (ii) reforms to deepen the financial sector; and (iii) reforms to build resilience to frequent and severe disasters (IMF 2017a). Fund-supported arrangements should provide a structured framework for design, implementation, and monitoring of resilience-building policies and help coordinate capacity-development activities. Likely reflecting this guidance, the number of QPCs and SBs declined in 2017 relative to the average for the 2018 RoC sample (Figure 24). Implementation of conditionality was somewhat weaker than in the full sample, in particular for PFM/RA and State-Owned Enterprise reforms (Figure 26).

41. Structural conditionality was generally tailored to capacity building needs and reform objectives of small states. Of the six programs under review, SCs were focused on strengthening policies and institutions to address fiscal and debt sustainability and deepening the financial sector through regulations and reforms (see Figure 21 in the main paper). The latter aspects were introduced in Sao Tomé and Príncipe 2012 ECF and Seychelles 2014 EFF to resolve the loss of correspondent banking relationships by addressing AML/CFT concerns and bringing them in line with Financial Action Task Force standards, supported by TA (Figure 25). Other programs recognized innovative debt instruments, which small states vulnerable to natural disasters contracted—such as Grenada’s 2015 ‘hurricane clause’ bond, where a one-off debt service deferral is triggered by a predefined event, in this case, a hurricane of a given intensity.¹³ Some programs (e.g., Sao Tomé and Príncipe 2015 ECF and Grenada 2014 ECF) benefited from tailored TA aimed at strengthening PFM to ensure transparent and efficient use of public resources.

42. Yet further tailoring for small states is needed to support ex-ante resilience building to natural disasters. In examining Fund-supported programs with small states, program conditionality rightly focused on capacity development and building fiscal and external buffers—through revenue mobilization, PFM, and structural reforms, including central bank and financial sector reforms. However, ex-ante resilience building to natural disasters did not feature explicitly in cases where it has been identified as a program objective or a key risk to economic outlook (e.g., Solomon Islands (2011 precautionary SCF, 2012 ECF)). To this end, further tailoring in program design to support small states’ resilience building efforts is needed. This could be informed by the IMF-World Bank joint Climate Change Policy Assessment (CCPA), which provides country-specific assessments of climate mitigation and adaptation policies.¹⁴ In addition to creating fiscal space and reserve accumulation, tailored program conditionality would help enhance disaster preparedness, strengthen institutions, coordinate the delivery of capacity building, and provide financing to address BoP needs.¹⁵

Technical Notes

This section provides methodological background information on analytical work referenced in the main paper.

A. Program Success: Financing Versus Adjustment

Methodology

43. The BoP need in a Fund-supported program is decomposed into its financing and adjustment components that help address this need. The methodology follows IMF (2015). For any program approved at year t, the BoP need and adjustment/financing to cover it during t to t + h is captured by an equation, where h = 0,l, 2,3:

B o P N e e d (t + h) = C A A D J (t + h) + I M F F I N (t + h) + I F I F I N (t + h) + O T H E R F A (t + h)

44. Variables are defined as follows:

BoP Need (t + h): financing gap in the absence of an IMF program, estimated as described below.
CA AD] (t + h): targeted current account adjustment excluding grants and official transfers between t - 1 and t + h. While grants and official transfers may be sizeable in PRGT programs, these are excluded in order to separate the underlying external adjustment from the hypothetical catalytic effect of the IMF programs in bringing in additional financing.
IMF FIN (t + h): IMF financing.
IFI FIN (t + h): IFI financing, including official current and capital transfers.
OTHER FA (t + h): residual that balances the equation above.

45. The financing gap in the absence of an IMF program (BoP Need (t + h)) is generally not reported in IMF staff reports and must therefore be estimated. The BoP need as presented in staff reports already incorporates the envisaged financing package and is therefore different from the BoP need that would have prevailed if policies at t – 1 had been continued. To estimate the BoP need at time t (BoP Need (t)), information on gross external financing needs is used as follows: of policy reforms to internalize ex ante resilience investments in the macro-fiscal framework, buffers, and the DSA, and to tailor policy advice informed by Climate Change Policy Assessments (CCPA). Other ongoing work on Fiscal Policies for Paris Climate Strategies: From Principle to Practice considers Fund policy advice to help members address climate commitments through integrating carbon charges into fuel taxes; allocating carbon pricing revenues; and integrating climate risks and financing into macro-fiscal frameworks.

\begin{matrix} B o P N e e d (t) = F i n a n c i n g N e e d (t) - F i n a n c i n g s o u r c e s (t) \\ ≃ (C A e x c l G r a n t s (t - 1) + S T D e b t (t)) \times \frac{G D P (t)}{G D P (t - 1)} - F i n a n c i n g s o u r c e s (t) \end{matrix}

Here, Financing sources (t) refer to the sum of IMF and IFI financing, and ST Debt (t) refers to short-term external debt at remaining maturity, falling due in year t. The financing gap in year t is thus an estimate of the BoP need in year t if policies in t - 7 were to continue.

Data

46. The main data sources are IMF staff reports at program approval. They provide the initial expectation about the path of the current account, gross financing needs, expected IMF disbursements, and IFI and bilateral financing over the program period. Other variables are estimated as described above.

Results

47. External adjustment and financing patterns differ notably between PRGT and GRA programs (Figure 27). Average external adjustment in GRA programs is sizeable while PRGT programs display smaller external adjustment (excluding grants) and a larger contribution from IFIs and bilateral support. This reflects different objectives, with PRGT programs geared towards catalyzing financing, while GRA programs are expected to strengthen a member’s BoP by the time repurchases become due.

B. Macro Optimism: Drivers of Growth Forecast Errors

Methodology

48. We use a regression-based approach to examine the contribution of external and domestic factors to growth forecast errors in Fund-supported programs. We build on the methodology developed by Blanchard and Leigh (2013) and Ismail, Perelli, and Yang (forthcoming) and apply it to the 2018 RoC sample. The baseline regression model can be described in an equation:

G r o w t h E r r_{i t}^{j} = C o n s \tan t + β_{e} E x t e r n a l E r r_{i t}^{j} + β_{d} D o m e s t i c P o l i c y_{i t}^{j} + f e_{i}^{j} + ε_{i t}^{j}

49. Variables are defined as follows:

$G r o w t h E r r_{i t}^{j}$ : Forecast error for GDP growth for country i for year t as reported in the j‘s WEO forecast vintage.
$E x t e r n a l E r r_{i t}^{j}$ : Forecast errors for trading partner growth and oil and commodity prices as faced by country i for year t as reported in the j‘s WEO forecast vintage.
$D o m e s t i c P o l i c y_{i t}^{j}$ : Planned fiscal and current account adjustments for country i for year t as reported in the j‘s WEO forecast vintage. Here, the current account adjustment captures non-fiscal policy actions, such as monetary policy, which would impact private-sector demand and, thereby, the current account. If the parameters (e.g., fiscal multipliers) in country teams’ forecast models are correctly specified, we should not expect to see a relationship between forecasts of domestic adjustments and growth forecast errors.¹⁶ However, a mis-estimation of the impact of domestic adjustments on growth would lead to GDP growth forecast errors.
Non linearity: Planned fiscal and current account adjustments were also tested for non-linearity by including an additional variable for high-adjustment cases.
$f e_{i}^{j}$ : Fixed effect for country i at the j‘s WEO forecast vintage. This allows us to control for time-invariant country-specific factors for a particular vintage of WEO forecasts.

Data

50. The underlying data are based on WEO vintages from April 2003 to October 2017. The various vintages are used to construct a dataset of IMF’s historical biannual forecasts for GDP growth, fiscal adjustment, current account adjustment, and commodity prices for 198 countries over the period 2003–17. The October 2018 WEO database is used to construct actual outturns for the same set of variables. Baseline regressions are then run using all WEO forecasts made during the 2018 RoC sample period. Corresponding regressions for surveillance cases are based on forecasts from 2012 to 2017.

Results

51. About half of the growth forecast errors in the overall sample can be explained by external and domestic policy factors (Figure 28). In the full RoC sample (Table 4, column 1), external factors (i.e., trading partner growth and commodity price forecast errors) contributed one quarter to short-term growth optimism, highlighting the significant influence of global forecast errors. Optimism regarding the impact of domestic adjustment on growth translated to another one quarter. Furthermore, the effects are nonlinear, as large planned fiscal and current account adjustments are often associated with a larger degree of forecast optimism than average-sized adjustments. Table 4 also presents the regression results for 2018 RoC subsamples (columns 2–7). Growth forecasts are more sensitive to oil price forecast errors and domestic fiscal adjustment in other developing countries (columns 3 and 4). External factors as well as planned fiscal adjustment are more closely related to growth forecasts in PRGT countries than in GRA countries (columns 5 and 6). In countries with a managed exchange rate, forecast errors are significantly associated with domestic policies such as fiscal adjustment, while in countries with a floating exchange rate, forecast errors are strongly related to external factors such as trading partner growth (column 6 and 7). Regression results are broadly similar using the 2011 RoC sample and surveillance countries during the 2018 RoC period.

Table 4.

Regression Results: Growth Optimism

Sources: WEO and IMF staff calculations.Notes: Standard errors in parentheses. Stars denote significance: *** p<0.01, ** p<0.05, * p<0.1.

Table 4.

Regression Results: Growth Optimism

Dependent variable:	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)
Growth forecast error (projected – actual)	2018 RoC	Other developing countries	Remaining country groups	PRGT	GRA	Floating exchange rate	Managed exchange rate	2011 RoC	Surveillance
Trading partner growth forecast error	0.475*** (0.102)	0.514*** (0.141)	0.455*** (0.144)	0.494*** (0.133)	0.405*** (0.141)	1.094*** -0.256	0.219** -0.107	0.857*** (0.0627)	0.445*** (0.0440)
Oil price forecast error	2.357*** (0.648)	3.475*** (0.934)	0.470 (0.856)	3.343*** (0.860)	-0.349 (0.850)	1.501 -1.505	1.605** -0.747	2.260*** (0.472)	0.590** (0.300)
Oil exporter x oil price forecast error	5.689*** (1.680)	18.05*** (4.785)	3.832** (1.608)	10.26*** (3.216)	3.547* (1.553)		8.102*** -1.617	-0.0409 (1.272)	0.852** (0.340)
Commodity price forecast error	-3.761* (2.104)	-5.061* (3.036)	-0.938 (2.763)	-4.244 (2.846)	0.206 (2.677)	-5.502 -4.817	-1.184 -2.29	-1.703 (1.160)	-1.548 (1.032)
Commodity exporter x commodity price forecast error	12.27*** (2.268)	11.94*** (3.053)	12.57*** (3.432)	10.02*** (2.685)	-2.415 (91.69)	41.89*** -4.841	3.943 -2.481	-0.694 (1.457)	3.285** (1.542)
Forecast of fiscal adjustment	0.151*** (0.0494)	0.131** (0.0633)	0.0945 (0.0804)	0.127** (0.0612)	0.0857 (0.0841)	-0.233 -0.156	0.226*** -0.0494	-0.0108 (0.0214)	0.0154 (0.0259)
Forecast of fiscal adjustment under high adj	0.164** (0.0695)	0.370*** (0.0934)	-0.140 (0.102)	0.331*** (0.0911)	0.0858 (0.0971)	0.384* -0.2	0.103 -0.0713	0.00996 (0.0377)	0.154*** (0.0496)
Forecast of CA adjustment	0.0255 (0.0326)	-0.0106 (0.0447)	0.144*** (0.0466)	0.0176 (0.0375)	0.137* (0.081 4)	0.0996 -0.0972	0.00327 -0.0332	0.0494** (0.0173)	0.0476** (0.0203)
Forecast of CA adjustment under high adj	0.218*** (0.0466)	0.301*** (0.0604)	-0.155** (0.0786)	0.244*** (0.0543)	0.173* (0.105)	0.184 -0.114	0.257*** -0.0519	0.787*** (0.0288)	0.0974*** (0.0309)
Global financial crisis dummy	-1.350 (0.928)	-0.899 (1.274)	-0.595 (1.355)	-0.441 (1.241)	-0.831 (1.345)	-11.58** -4.66	-1.174 -0.941	-1.016*** (0.244)
Constant	0.446*** (0.0971)	0.519*** (0.146)	0.608*** (0.125)	0.531*** (0.134)	0.469*** (0.125)	0.501*** -0.193	0.729*** -0.119	0.0773 (0.0938)	0.584*** (0.0377)
Observations	2,715	1,420	1,295	1,821	894	906	1,806	4,273	8,326
R-squared	0.122	0.204	0.055	0.166	0.035	0.182	0.149	0.382	0.038

Sources: WEO and IMF staff calculations.Notes: Standard errors in parentheses. Stars denote significance: *** p<0.01, ** p<0.05, * p<0.1.

Table 4.

Regression Results: Growth Optimism

Dependent variable:	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)
Growth forecast error (projected – actual)	2018 RoC	Other developing countries	Remaining country groups	PRGT	GRA	Floating exchange rate	Managed exchange rate	2011 RoC	Surveillance
Trading partner growth forecast error	0.475*** (0.102)	0.514*** (0.141)	0.455*** (0.144)	0.494*** (0.133)	0.405*** (0.141)	1.094*** -0.256	0.219** -0.107	0.857*** (0.0627)	0.445*** (0.0440)
Oil price forecast error	2.357*** (0.648)	3.475*** (0.934)	0.470 (0.856)	3.343*** (0.860)	-0.349 (0.850)	1.501 -1.505	1.605** -0.747	2.260*** (0.472)	0.590** (0.300)
Oil exporter x oil price forecast error	5.689*** (1.680)	18.05*** (4.785)	3.832** (1.608)	10.26*** (3.216)	3.547* (1.553)		8.102*** -1.617	-0.0409 (1.272)	0.852** (0.340)
Commodity price forecast error	-3.761* (2.104)	-5.061* (3.036)	-0.938 (2.763)	-4.244 (2.846)	0.206 (2.677)	-5.502 -4.817	-1.184 -2.29	-1.703 (1.160)	-1.548 (1.032)
Commodity exporter x commodity price forecast error	12.27*** (2.268)	11.94*** (3.053)	12.57*** (3.432)	10.02*** (2.685)	-2.415 (91.69)	41.89*** -4.841	3.943 -2.481	-0.694 (1.457)	3.285** (1.542)
Forecast of fiscal adjustment	0.151*** (0.0494)	0.131** (0.0633)	0.0945 (0.0804)	0.127** (0.0612)	0.0857 (0.0841)	-0.233 -0.156	0.226*** -0.0494	-0.0108 (0.0214)	0.0154 (0.0259)
Forecast of fiscal adjustment under high adj	0.164** (0.0695)	0.370*** (0.0934)	-0.140 (0.102)	0.331*** (0.0911)	0.0858 (0.0971)	0.384* -0.2	0.103 -0.0713	0.00996 (0.0377)	0.154*** (0.0496)
Forecast of CA adjustment	0.0255 (0.0326)	-0.0106 (0.0447)	0.144*** (0.0466)	0.0176 (0.0375)	0.137* (0.081 4)	0.0996 -0.0972	0.00327 -0.0332	0.0494** (0.0173)	0.0476** (0.0203)
Forecast of CA adjustment under high adj	0.218*** (0.0466)	0.301*** (0.0604)	-0.155** (0.0786)	0.244*** (0.0543)	0.173* (0.105)	0.184 -0.114	0.257*** -0.0519	0.787*** (0.0288)	0.0974*** (0.0309)
Global financial crisis dummy	-1.350 (0.928)	-0.899 (1.274)	-0.595 (1.355)	-0.441 (1.241)	-0.831 (1.345)	-11.58** -4.66	-1.174 -0.941	-1.016*** (0.244)
Constant	0.446*** (0.0971)	0.519*** (0.146)	0.608*** (0.125)	0.531*** (0.134)	0.469*** (0.125)	0.501*** -0.193	0.729*** -0.119	0.0773 (0.0938)	0.584*** (0.0377)
Observations	2,715	1,420	1,295	1,821	894	906	1,806	4,273	8,326
R-squared	0.122	0.204	0.055	0.166	0.035	0.182	0.149	0.382	0.038

Sources: WEO and IMF staff calculations.Notes: Standard errors in parentheses. Stars denote significance: *** p<0.01, ** p<0.05, * p<0.1.

C. Macro Optimism: Growth Accounting

Methodology

52. A standard growth accounting methodology is applied. Let g_Y denote the growth rate of aggregate output (Y), g_TFP the growth rate of total factor productivity (TFP), g_K the growth rate of aggregate capital (K), g_L the growth rate of aggregate human capital (L), and α the capital share. Assuming a neoclassical production function and perfect competition in factor markets, we then decompose real GDP growth into its factor inputs and compare contributions based on (i) WEO forecasts for the 2018 RoC sample at the time of program requests and (ii) actual data as they materialized:

g_{Y} = g_{T F P} + α g_{K} + (1 - α) g_{L}

Data

53. Data are based on the WEO databases and the Penn World Tables. Growth forecast errors are calculated as actual minus forecasted growth as published in the WEO databases. Human capital is calculated as the product of employment (WEO) and the human capital index based on years of schooling and returns to education (Penn World Table version 9.0). Capital is computed using gross fixed capital formation (WEO) and the perpetual inventory method. Data constraints limit the sample to 28 countries. Where the capital share was missing, it was assumed at 0.5. Where data on gross fixed capital formation was missing, gross capital formation was used.

Results

54. Shortfalls in expected capital growth and disappointing TFP developments were the main contributors to growth forecast errors. In the analytical group of other developing countries, but also in PRGT-eligible countries more broadly, lower-than-expected growth can be explained by shortfalls in capital accumulation (Figure 29, top panels). In non-developing countries and GRA countries more broadly, the largest share of the growth forecast error can be attributed to lower TFP growth (Figure 29, bottom panels).

D. Public Debt: Sustainability Assessment for Market Access Countries Seeking IMF Support

Methodology

55. The probit model provides a probabilistic assessment of debt distress for countries with market access in the sample. The probit model is multivariate, statistically-based, and allows for a wide range of explanatory variables.¹⁷ The binary probit is constructed to estimate the discrete probability of debt distress. The dependent variable takes the value of one if there was a default or restructuring and zero if there was no such event. The regression controls for several explanatory variables, such as external factors (GDP per capita, foreign exchange reserves), macroeconomic factors (real GDP growth, real effective exchange rate (REER) misalignment), and indicators of liquidity pressure (primary balance) and the debt burden (public debt-to-GDP ratio).

56. The model-generated probabilities of sovereign default and restructuring are compared to probability thresholds to classify a country into one of three debt sustainability categories. The probability threshold used to identify unsustainable cases— threshold (a)—is derived using the noise-to-signal methodology (Box 2). It is an optimal threshold, above which the model predicts a default. This threshold minimizes the sum of proportions of false alarms and missed crises, subject to the constraint that this threshold should be higher than the average fitted probability of default for the sample as a whole. The thresholds for the three sustainability categories are defined as follows:

Box 2.

Noise-to-Signal Methodology to Generate Probability Threshold

The Noise-to-Signal methodology is used to compute a threshold for identifying a stress event. Using the relationship between debt default or restructuring and a number of early warning indicators (using the same sample as in the probit model), a threshold is derived, above which the indicator would signal a stress event. The signal-prediction outcomes (signals compared to a list of crises) are then compared to whether or not the sovereign actually underwent a debt default or restructuring to determine if:

the signal correctly predicted a crisis;
the signal breached the threshold, but there was no crisis (false alarm);
the signal did not breach the threshold, but a crisis ensued (missed crisis); or
there was no breach of the threshold and no crisis.

The derived optimal threshold for the stress event is defined as the level that minimizes the sum of errors (i.e., the sum of false alarms and of missed crises).

“Unsustainable.” Based on the threshold above which the noise-to-signal model predicts a stress event.
“Sustainable with high probability.” This threshold—threshold (b)—is derived from the probit model (Table 5) and is based on the 80th percentile of fitted probabilities of EA cases that did not involve debt restructuring.¹⁸ In these cases, the country was able to manage the debt situation, despite the considerable stress implied by the very high access levels.
“Sustainable but not with high probability.” This is a “gray zone” and comprises probabilities between the thresholds for (a) and (b) above.

Table 5.

Probit Model ^1/

Source: IMF staff calculations.

^{^1/}Dependent variable: “default” = 1 in a year when a country (under an IMF-supported program) experiences a default or restructuring, and zero otherwise.

^{^2/}Stars denote significance: ** p<0.05, * p<0.1.

Data

57. The sample includes countries experiencing sovereign stress, measured by defaults or debt restructurings that took place under IMF-supported programs during 1990–2013. The variable on debt default and restructuring events was constructed from several databases: fiscal crises episodes from Baldacci, Petrova, Belhocine, Dobrescu, and Mazraani (2011); private sovereign debt restructurings from Cruces and Trebesch (2013); Paris Club arrangements; Moody’s study on sovereign defaults and restructuring, covering selected restructurings of foreign and local currency bonds (Moody’s, 2013); and World Development Indicators (WDI), providing data on arrears to official and private creditors.

Results

58. The estimated parameters of the probit model are in line with ex ante expectations (Table 5). A higher public debt ratio is associated with a higher likelihood of default or restructuring and is robust across various samples. Higher real GDP growth is associated with a lower probability of default or restructuring, reflecting more favorable interest-growth differentials. Countries with higher GDP per capita tend to have a lower likelihood of default or restructuring. A higher level of international reserves is associated with a lower probability of default or restructuring at a given debt level. A higher degree of REER overvaluation is associated with an increased probability of default or restructuring. The primary balance and rate of inflation are not significantly associated with the probability of default or restructuring, although the signs on the estimated coefficients are in line with expectations. With an adjusted R-squared of 30 percent, the fit is good for this type of model.

59. Translating resulting probabilities into debt sustainability categories suggests that debt sustainability improved in about one-third of programs but deteriorated in a significant minority of cases. The model was used to assess debt sustainability in 42 GRA programs ongoing between September 2011 and end-2017. Close to half the programs in the sample saw debt remain “sustainable with high probability” throughout the course of the program (Figure 30). In close to a third of the cases, sustainability improved to “sustainable with high probability” or “sustainable but not with high probability” (green areas), in some cases due to debt restructuring.¹⁹ Nevertheless, in the sizable remainder of programs, debt either remained “unsustainable” (4.8 percent) or deteriorated to “unsustainable” (4.8 percent) and “sustainable but not with high probability” (7.1 percent) (red areas).

E. Structural Conditionality and Program Design: Impact of Moving to Review-Based Conditionality

60. Reform of the Fund’s lending framework has modernized structural conditionality. In 2009, the Executive Board approved an overhaul of the Fund’s lending framework. Changes included a shift away from Structural Performance Criteria (SPCs). Instead, monitoring of structural reform implementation became review-based. This reform was intended to reduce stigma, as countries no longer needed formal waivers if they failed to implement a structural reform by a particular date. However, concerns were raised that this reform could lead to a deterioration of structural reform implementation.

Methodology

61. A regression framework is employed to examine the impact on reform implementation of moving to review-based conditionality. We use the following panel fixed effects regression to test the determinants of the implementation record:

N M_{i, t} = α N M_{i, t - 1} + β X_{i, t} + φ_{i} + η_{i, t} f o r t = 1, ..., T a n d i = 1, ..., N

Here, NM_i,t is the ratio of “Not Met” SBs in program i at review year t (including SBs that were implemented with delay); X_i,t is a vector of control variables: total number of SBs, a PRGT program dummy, income level, regulatory quality, trade openness, average depth of SBs, and a 2018 RoC sample dummy.

Data

62. Data sources are as follows. Information regarding structural conditions, program implementation, and identifiers of program type are calculated from the IMF’s MONA database. Depth scores of SBs are assigned by staff based on the description of each SB and following the 2011 RoC methodology (IMF, 2012a). GDP per capita is from the WDI. The regulatory quality index (capturing perceptions of the government’s ability to formulate and implement sound policies and regulations that permit and promote private sector development) is from the World Governance Indicators (WGI). Trade openness is calculated based on the IMF’s WEO database.

Results

63. The regression does not indicate a significant difference in implementation of SBs between the 2011 and the 2018 RoC. Panel fixed effects regressions of the “Not Met” ratio as noted above suggest that the track record (i.e., past implementation) significantly matters for the performance against SBs (Table 6). A larger total number of SCs of each review year is associated with better implementation (i.e., lower “Not Met” ratio), which may relate to more granular steps of reforms helping to achieve better outcomes. The dummy for the 2018 RoC period is insignificant, supporting the conclusion that the shift to review-based conditionality did not affect program implementation.

Table 6.

Regression Results: Implementation of Structural Conditions

Sources: MONA, WEO, and IMF staff calculations.Notes: Standard errors in parentheses. Stars denote significance: *** p<0.01, ** p<0.05, * p<0.1.

Table 6.

Regression Results: Implementation of Structural Conditions

Dependent variable: Program completion	(1)	(2)	(3)	(4)	(5)
Lagged Not Met ratio	-0.323*** (0.0916)	-0.305*** (0.0939)	-0.335*** (0.100)	-0.327*** (0.111)	-0.148 (0.439)
Total number of SBs	-0.0111** (0.00462)	-0.0106** (0.00501)	-0.0105** (0.00501)	-0.0128** (0.00581)	-0.00380 (0.0248)
PRGT dummy	-0.120 (0.233)	-0.114 (0.230)	-0.116 (0.230)	-0.118 (0.229)	-0.216 (0.535)
GDP per capita, in log terms		0.105 (0.458)	0.248 (0.487)	0.295 (0.611)	3.563 (3.625)
Regulatory quality			-0.191 (0.220)	-0.276 (0.237)	0.0793 (1.403)
Trade openness				0.143 (0.279)	-0.162 (1.048)
Average depth of SB					0.0578 (0.362)
2018 RoC period dummy	0.0517 (0.0667)	0.0481 (0.0685)	0.0497 (0.0686)	0.0412 (0.0698)	-0.0446 (0.226)
Constant	0.484** (0.198)	-0.273 (3.296)	-1.370 (3.532)	-1.769 (4.387)	-27.12 (28.11)
Observations	329	295	295	245	50
R2	0.098	0.097	0.101	0.121	0.142
Number of countries	147	128	128	110	29

Sources: MONA, WEO, and IMF staff calculations.Notes: Standard errors in parentheses. Stars denote significance: *** p<0.01, ** p<0.05, * p<0.1.

Table 6.

Regression Results: Implementation of Structural Conditions

Dependent variable: Program completion	(1)	(2)	(3)	(4)	(5)
Lagged Not Met ratio	-0.323*** (0.0916)	-0.305*** (0.0939)	-0.335*** (0.100)	-0.327*** (0.111)	-0.148 (0.439)
Total number of SBs	-0.0111** (0.00462)	-0.0106** (0.00501)	-0.0105** (0.00501)	-0.0128** (0.00581)	-0.00380 (0.0248)
PRGT dummy	-0.120 (0.233)	-0.114 (0.230)	-0.116 (0.230)	-0.118 (0.229)	-0.216 (0.535)
GDP per capita, in log terms		0.105 (0.458)	0.248 (0.487)	0.295 (0.611)	3.563 (3.625)
Regulatory quality			-0.191 (0.220)	-0.276 (0.237)	0.0793 (1.403)
Trade openness				0.143 (0.279)	-0.162 (1.048)
Average depth of SB					0.0578 (0.362)
2018 RoC period dummy	0.0517 (0.0667)	0.0481 (0.0685)	0.0497 (0.0686)	0.0412 (0.0698)	-0.0446 (0.226)
Constant	0.484** (0.198)	-0.273 (3.296)	-1.370 (3.532)	-1.769 (4.387)	-27.12 (28.11)
Observations	329	295	295	245	50
R2	0.098	0.097	0.101	0.121	0.142
Number of countries	147	128	128	110	29

Sources: MONA, WEO, and IMF staff calculations.Notes: Standard errors in parentheses. Stars denote significance: *** p<0.01, ** p<0.05, * p<0.1.

F. Ownership: Understanding Completion Rates

Methodology

64. A probit model is employed to examine the factors associated with ownership as reflected in completion rates. For the model, a program receives a value of 1 if fully completed or largely implemented or zero if off track or quickly off track. Programs that were in progress or had been replaced as of end-September 2018 are excluded from the analysis.

Data

65. Data are taken from several different sources. IMF program details are computed based on information in the IMF’s MONA database. Macroeconomic variables are from the WEO database. Institutional and political variables are taken from the Worldwide Governance Indicators (WGI) database and the World Bank’s Database of Political Institutions. Data cover the period 2002–18.

Results

66. Better institutional capacity appears important for program completion. A higher degree of government effectiveness is associated with a significantly higher probability of program completion (Table 7). In contrast, more prior actions at program initiation are negatively associated with completion rates, highlighting that prior actions are not a substitute for ownership. Fragile states tend to have lower completion rates than other groups. Political economy considerations are not significantly related to completion rates (columns 8 and 9, and Figure 31). This potentially reflects that such factors have already been adequately taken into consideration in program design. External shocks are only weakly associated with completion rates, with only commodity price shocks showing as significant in the regression (column 13).

Table 7.

Regression Results: Determinants of Program Completion Rates

Sources: Database of Political Institutions, MONA, WEO, WGI database, and IMF staff calculations.Notes: Standard errors in parentheses. Stars denote significance: *** p<0.01, ** p<0.05, * p<0.1. Dependent variable = 1 if completed or largely implemented; 0 otherwise.

Table 7.

Regression Results: Determinants of Program Completion Rates

Dependent variable: Program completion	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)	(10)	(11)	(12)	(13)
Number of prior actions	-0.0698* (-2.07)	-0.0698* (-2.07)	-0.0744* (-2.16)	-0.0715* (-2.09)	-0.0710* (-2.06)	-0.0701* (-2.07)	-0.0687* (-2.03)	-0.135* (-1.97)	-0.0518 (-1.35)	-0.0676* (-2.00)	-0.0732* (-2.15)	-0.0736* (-2.15)	-0.0978** (0.0357)
Number of structural benchmarks	0.0304 (1.38)	0.0315 (1.43)	0.0447 (1.78)	0.0311 (1.40)	0.0298 (1.34)	0.0296 (128)	0.0311 (1.41)	0.0240 (0.66)	0.0341 (1.46)	0.0278 (1.26)	0.0298 (1.35)	0.0292 (1.32)	0.0565* (0.0241)
Initial GDP per capita, in log terms	-0.285* (-2.41)	-0.255* (-2.01)	-0.268* (-2.18)	-0.315* (-2.11)	-0.289* (-2.41)	-0.280* (-2.20)	-0.289* (-2.39)	-0.460* (-2.12)	-0.316* (-2.52)	-0.292* (-2.43)	-0.282* (-2.34)	-0.276* (-2.24)	-0.236 (0.124)
Government effectiveness	0.644** (2.80)	0.631** (2.73)	0.592* (2.40)	0.644** (2.80)	0.641** (2.78)	0.639** (2.73)	0.399 (1.53)	0.796* (2.12)	0.593* (2.49)	0.621** (2.67)	0.649** (2.81)	0.625** (2.68)	0.603* (0.238)
Initial GDP growth rate, in percent		0.0163 (0.65)
Growth forecast error			0.0402 (1.79)
GRA program dummy				0.0933 (0.32)
Number of reviews at program application					0.00748 (0.19)
Initial program length						0.0166 (0.12)
Fragile states dummy							-0.520* (-2.03)
Near-term election dummy								-0.01000 (-0.97)
Majority in government									-0.000009 (-0.02)
Initial public debt-to-GDP ratio										0.000845 (0.56)
Planned current account adjustment											0.00234 (0.15)
Planned fiscal adjustment												0.00519 (0.43)
Commodity price shock													6.078** (1.931)
Constant	2.884** (3.11)	2.581* (2.49)	2.709** (2.82)	3.062** (2.84)	2.872** (3.09)	2.808* (2.48)	2.943** (3.12)	4.554* (2.48)	3.006** (3.12)	2.879** (3.09)	2.888** (3.08)	2.824** (2.93)	2.100* (0.985)
Number of observations	202	202	188	202	202	202	202	81	179	199	201	198	199

Table 7.

Regression Results: Determinants of Program Completion Rates

Dependent variable: Program completion	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)	(10)	(11)	(12)	(13)
Number of prior actions	-0.0698* (-2.07)	-0.0698* (-2.07)	-0.0744* (-2.16)	-0.0715* (-2.09)	-0.0710* (-2.06)	-0.0701* (-2.07)	-0.0687* (-2.03)	-0.135* (-1.97)	-0.0518 (-1.35)	-0.0676* (-2.00)	-0.0732* (-2.15)	-0.0736* (-2.15)	-0.0978** (0.0357)
Number of structural benchmarks	0.0304 (1.38)	0.0315 (1.43)	0.0447 (1.78)	0.0311 (1.40)	0.0298 (1.34)	0.0296 (128)	0.0311 (1.41)	0.0240 (0.66)	0.0341 (1.46)	0.0278 (1.26)	0.0298 (1.35)	0.0292 (1.32)	0.0565* (0.0241)
Initial GDP per capita, in log terms	-0.285* (-2.41)	-0.255* (-2.01)	-0.268* (-2.18)	-0.315* (-2.11)	-0.289* (-2.41)	-0.280* (-2.20)	-0.289* (-2.39)	-0.460* (-2.12)	-0.316* (-2.52)	-0.292* (-2.43)	-0.282* (-2.34)	-0.276* (-2.24)	-0.236 (0.124)
Government effectiveness	0.644** (2.80)	0.631** (2.73)	0.592* (2.40)	0.644** (2.80)	0.641** (2.78)	0.639** (2.73)	0.399 (1.53)	0.796* (2.12)	0.593* (2.49)	0.621** (2.67)	0.649** (2.81)	0.625** (2.68)	0.603* (0.238)
Initial GDP growth rate, in percent		0.0163 (0.65)
Growth forecast error			0.0402 (1.79)
GRA program dummy				0.0933 (0.32)
Number of reviews at program application					0.00748 (0.19)
Initial program length						0.0166 (0.12)
Fragile states dummy							-0.520* (-2.03)
Near-term election dummy								-0.01000 (-0.97)
Majority in government									-0.000009 (-0.02)
Initial public debt-to-GDP ratio										0.000845 (0.56)
Planned current account adjustment											0.00234 (0.15)
Planned fiscal adjustment												0.00519 (0.43)
Commodity price shock													6.078** (1.931)
Constant	2.884** (3.11)	2.581* (2.49)	2.709** (2.82)	3.062** (2.84)	2.872** (3.09)	2.808* (2.48)	2.943** (3.12)	4.554* (2.48)	3.006** (3.12)	2.879** (3.09)	2.888** (3.08)	2.824** (2.93)	2.100* (0.985)
Number of observations	202	202	188	202	202	202	202	81	179	199	201	198	199

G. Tailoring and Uniformity of Treatment: Understanding Access Decisions

Methodology

67. A regression framework is used to explore the empirical link between access decisions and the factors underpinning such decisions. Overall, access decisions should reflect a country’s BoP need, program strength, and capacity to repay, as well as the underlying GRA and PRGT lending frameworks. To test the effects of these factors, a two-stage approach is followed:

Stage 1. Given the large set of potential determinants of access, a Bayesian Model Averaging approach is used to select the main determinants of access.²⁰
Stage 2. A standard Ordinary Least Squares regression framework is employed to examine how well the main determinants identified in stage 1 help explain access decisions. All programs in the sample are initially pooled into one sample. Separate regressions for GRA and PRGT programs are then run to examine potential inconsistencies in access decisions between the PRGT and GRA and within the PRGT and GRA. For program i, the regression can be presented as follows:
$A c c e s s_{i} = C + β_{1} B o P_{-} N e e d_{i} + β_{2} \Pr g_{-} S t r e n g t h_{i} + β_{3} C a p a c i t y_{-} Re p a y_{i} + β_{4} F u n d_{-} P o l i c y_{i} + ε_{i}$
Here, as a proxy for BoP need, the baseline regression includes an estimate of gross financing needs and dummies that capture the nature of the BoP need (i.e., whether the country experienced a capital account crisis and whether the arrangement is precautionary). Program strength is proxied by the size of the planned fiscal and current account adjustments. Governance indicators, reflecting institutional strength, are used as measures of capacity to repay. The access limit for GRA arrangements, the access norm for PRGT arrangements, and an EA dummy are included to reflect the importance of the underlying Fund policies and lending frameworks for access decisions.

Data

68. The data cover 209 Fund-supported programs approved between 2002 and 2017. Macroeconomic conditions and projected adjustments are collected from the first WEO vintage published after the Board approval date for each program. Institutional variables are from the WGI database.

Results

69. The parsimonious baseline regressions explain a large part of the variation in access decisions. The baseline regressions for the full sample, for the GRA, and for the PRGT can explain almost 70 percent of the variation in access levels (columns 3, 6, and 8 in Table 8). There is also a tight relationship between actual access levels and access levels predicted by the regressions (Figure 32, left panel).

Table 8.

Regression Results: Determinants of Access Levels at Program Approval

Sources: WEO database, WGI database, and IMF staff calculations.

^{^1/}Gross financing need (in percent of GDP) is defined as the sum of change in current account balance, change in reserves and debt amortization projected during the program period.

^{^2/}A country had a capital account crisis if the country experienced a fall in net portfolio inflows above 3 percent of GDP at time t or t-1.

^{^3/}This variable combines access limit for GRA programs and access norm for PRGT programs.

^{^4/}The GRA access limit is expressed in percent of GDP in the regressions. The access limit is 600 percent of quota if a program was approved before March 2016 and 435 percent of quota if a program was approved in or after March 2016.

^{^5/}The PRGT access norm is expressed in percent of GDP in the regressions. The access norm is 90 percent of quota per 3-year ECF arrangement or per 18-month SCF for countries with total outstanding concessional IMF credit under all facilities of less than 75 percent of quota and is 56.25 percent of quota with outstanding concessional credit of between 75 percent of quota and 150 percent of quota. For countries whose outstanding concessional credit is above 150 percent of quota, the norm does not apply, and access is guided by consideration of the cumulative normal access limit of 225 percent of quota.

^{^6/}White’s heteroscedasticity-robust standard errors in parentheses. Stars denote significance: *** p<0.01, ** p<0.05, * p<0.1.

Table 8.

Regression Results: Determinants of Access Levels at Program Approval

Dependent variable:	Full sample			GRA			PRGT
Access/GDP	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
Gross financing need ^1/	0.00856*** (0.00235)	0.00733*** (0.00226)	0.00594*** (0.00148)	0.00571*** (0.00205)	0.00377** (0.00175)	0.00446*** (0.00165)	0.0227*** (0.00660)	0.00372 (0.00398)
Capital account crisis ^2/	2.441*** (0.919)	2.357*** (0.766)	2.788*** (0.627)	2.776** (1.120)	2.866*** (0.867)	3.134*** (0.765)	-0.0369 (0.425)	0.555** (0.226)
Planned CA adjustment	0.0621*** (0.0234)	0.0413* (0.0243)	0.0321 (0.0204)	0.0942** (0.0432)	0.0420 (0.0463)	0.0479 (0.0427)	0.0253 (0.0266)	0.00648 (0.0176)
Planned fiscal adjustment	-0.00249 (0.0206)	0.00831 (0.0253)	0.0456*** (0.0143)	0.0435 (0.0517)	0.103** (0.0505)	0.0882* (0.0471)	0.0226 (0.0215)	0.0608*** (0.0182)
Political stability	-0.270 (0.300)	-0.227 (0.282)	-0.293 (0.223)	-0.321 (0.361)	-0.310 (0.314)	-0.316 (0.279)	-0.346 (0.550)	-0.356 (0.271)
Rule of law	0.211 (0.420)	-0.120 (0.378)	-0.0588 (0.333)	0.751 (0.661)	0.328 (0.534)	0.0950 (0.512)	-0.257 (0.510)	0.0192 (0.337)
Precautionary arrangement	-2.607*** (0.539)	-2.410*** (0.475)	-1.953*** (0.460)	-2.503*** (0.677)	-2.362*** (0.546)	-2.148*** (0.515)	-0.930 (0.704)	0.232 (0.490)
Successor program	-1.031*** (0.388)	-0.658* (0.372)	-0.866*** (0.308)	-0.831 (0.584)	-0.158 (0.523)	-0.473 (0.506)	-1.058** (0.456)	-0.869*** (0.239)
GRA dummy	2.272*** (0.497)	1.337*** (0.447)	-0.262 (0.457)
Exceptional access		3.282*** (0.606)	4.418*** (0.593)		3.491*** (0.636)	4.119*** (0.624)
GRA access limit/PRGT access norm ^3/			0.491*** (0.0896)
Access limit ^4/						0.318*** (0.0939)
PRGT access norm ^5/								0.720*** (0.101)
Number of observations	209	209	209	103	103	103	106	106
R-squared ^6/	0.420	0.512	0.681	0.472	0.617	0.676	0.185	0.734

Sources: WEO database, WGI database, and IMF staff calculations.

^{^1/}Gross financing need (in percent of GDP) is defined as the sum of change in current account balance, change in reserves and debt amortization projected during the program period.

^{^2/}A country had a capital account crisis if the country experienced a fall in net portfolio inflows above 3 percent of GDP at time t or t-1.

^{^3/}This variable combines access limit for GRA programs and access norm for PRGT programs.

^{^6/}White’s heteroscedasticity-robust standard errors in parentheses. Stars denote significance: *** p<0.01, ** p<0.05, * p<0.1.

Table 8.

Regression Results: Determinants of Access Levels at Program Approval

Dependent variable:	Full sample			GRA			PRGT
Access/GDP	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
Gross financing need ^1/	0.00856*** (0.00235)	0.00733*** (0.00226)	0.00594*** (0.00148)	0.00571*** (0.00205)	0.00377** (0.00175)	0.00446*** (0.00165)	0.0227*** (0.00660)	0.00372 (0.00398)
Capital account crisis ^2/	2.441*** (0.919)	2.357*** (0.766)	2.788*** (0.627)	2.776** (1.120)	2.866*** (0.867)	3.134*** (0.765)	-0.0369 (0.425)	0.555** (0.226)
Planned CA adjustment	0.0621*** (0.0234)	0.0413* (0.0243)	0.0321 (0.0204)	0.0942** (0.0432)	0.0420 (0.0463)	0.0479 (0.0427)	0.0253 (0.0266)	0.00648 (0.0176)
Planned fiscal adjustment	-0.00249 (0.0206)	0.00831 (0.0253)	0.0456*** (0.0143)	0.0435 (0.0517)	0.103** (0.0505)	0.0882* (0.0471)	0.0226 (0.0215)	0.0608*** (0.0182)
Political stability	-0.270 (0.300)	-0.227 (0.282)	-0.293 (0.223)	-0.321 (0.361)	-0.310 (0.314)	-0.316 (0.279)	-0.346 (0.550)	-0.356 (0.271)
Rule of law	0.211 (0.420)	-0.120 (0.378)	-0.0588 (0.333)	0.751 (0.661)	0.328 (0.534)	0.0950 (0.512)	-0.257 (0.510)	0.0192 (0.337)
Precautionary arrangement	-2.607*** (0.539)	-2.410*** (0.475)	-1.953*** (0.460)	-2.503*** (0.677)	-2.362*** (0.546)	-2.148*** (0.515)	-0.930 (0.704)	0.232 (0.490)
Successor program	-1.031*** (0.388)	-0.658* (0.372)	-0.866*** (0.308)	-0.831 (0.584)	-0.158 (0.523)	-0.473 (0.506)	-1.058** (0.456)	-0.869*** (0.239)
GRA dummy	2.272*** (0.497)	1.337*** (0.447)	-0.262 (0.457)
Exceptional access		3.282*** (0.606)	4.418*** (0.593)		3.491*** (0.636)	4.119*** (0.624)
GRA access limit/PRGT access norm ^3/			0.491*** (0.0896)
Access limit ^4/						0.318*** (0.0939)
PRGT access norm ^5/								0.720*** (0.101)
Number of observations	209	209	209	103	103	103	106	106
R-squared ^6/	0.420	0.512	0.681	0.472	0.617	0.676	0.185	0.734

Sources: WEO database, WGI database, and IMF staff calculations.

^{^1/}Gross financing need (in percent of GDP) is defined as the sum of change in current account balance, change in reserves and debt amortization projected during the program period.

^{^2/}A country had a capital account crisis if the country experienced a fall in net portfolio inflows above 3 percent of GDP at time t or t-1.

^{^3/}This variable combines access limit for GRA programs and access norm for PRGT programs.

^{^6/}White’s heteroscedasticity-robust standard errors in parentheses. Stars denote significance: *** p<0.01, ** p<0.05, * p<0.1.

70. The results suggest that access decisions largely reflect the Fund’s policies and lending frameworks, as well as fundamentals (Table 8).

Full sample (columns 1–3): After controlling for the BoP need, program strength, and capacity to repay, the difference in access levels between GRA and PRGT programs can be largely explained by the Fund’s lending frameworks. Adding access limits for GRA programs and access norms for PRGT programs to the regression renders the GRA dummy insignificant.
GRA sample (columns 4–6): The coefficients for economic fundamentals are similar to the full sample estimation. The EA dummy remains an important driver of access, potentially capturing sizeable effects of large BoP crises, above and beyond those of a typical capital account crisis (e.g., including systemic considerations). In the GRA sample, successor programs are not associated with lower access levels.
PRGT sample (columns 7–8): The PRGT access norm explains 50 percent of the PRGT sample variation, pointing to a strong link between the PRGT access norm and access decisions (Figure 32, right panel). Furthermore, there are strong links between access levels and the size of adjustment, confirming that the strength of policies also matters for access decisions.

References

Ahuja, Ashvin, Murtaza Syed, and Kevin Wiseman, 2017, “Assessing Country Risk—Selected Approaches—Reference Note,” Technical Notes and Manuals (Washington: International Monetary Fund).
- Search Google Scholar
- Export Citation
Baldacci, Emanuele, Iva Petrova, Nazim Belhocine, Gabriela Dobrescu, and Samah Mazraani, 2011, “Assessing Fiscal Stress,” IMF Working Paper No. 11/100 (Washington: International Monetary Fund).
- Search Google Scholar
- Export Citation
Blanchard, Olivier and Daniel Leigh, 2013, “Growth Forecast Errors and Fiscal Multipliers,” IMF Working Paper No. 13/1 (Washington: International Monetary Fund).
- Search Google Scholar
- Export Citation
Cruces, Juan, and Christoph Trebesch, 2013, Sovereign Defaults: The Price of Haircuts, American Economic Journal: Macroeconomics, Vol. 5 (3), pp. 85–117.
- Search Google Scholar
- Export Citation
Cruz, Cesi, Philip Keefer, and Carlos Scartascini, 2018, “Database of Political Institutions 2017,” DPI2017 (Washington: Inter-American Development Bank).
- Search Google Scholar
- Export Citation
International Monetary Fund, 2002, “Guidelines on Conditionality” (Washington).
- Search Google Scholar
- Export Citation
International Monetary Fund, 2012, “2011 Review of Conditionality. Background Paper 4: Technical Appendices" (Washington).
- Search Google Scholar
- Export Citation
International Monetary Fund, 2014a, “The Fund’s Lending Framework and Sovereign Debt—Annexes,” Annex V: Potential Gains from Slower Fiscal Consolidation (Washington).
- Search Google Scholar
- Export Citation
International Monetary Fund, 2014b, “The Fund’s Lending Framework and Sovereign Debt—Annexes,” Annex IV: Reprofiling and Domestic Financial Stability: Recent Experiences (Washington).
- Search Google Scholar
- Export Citation
International Monetary Fund, 2015, “Crisis Program Review” (Washington).
- Search Google Scholar
- Export Citation
International Monetary Fund, 2017a, 2017 Staff Guidance Note on the Fund’s Engagement with Small Developing States (Washington).
- Search Google Scholar
- Export Citation
International Monetary Fund, 2017b, “State-Contingent Debt Instruments for Sovereigns” (Washington).
- Search Google Scholar
- Export Citation
Ismail, Kareem, Roberto Perrelli, and Jessie Yang, “Optimistic Bias in Growth Forecasts—The Role of Planned Policy Adjustments,” IMF Working Paper, forthcoming (Washington: International Monetary Fund).
- Search Google Scholar
- Export Citation
Leamer, Edward E., 1978, “Specification Searches: Ad hoc Inference with Nonexperimental Data,” Vol. 53 (New York: John Wiley & Sons Incorporated).
- Search Google Scholar
- Export Citation
Moody’s, 2013, Sovereign Defaults Series Compendium, October.
- Search Google Scholar
- Export Citation
Raftery, Adrian E., David Madigan, and Jennifer A. Hoeting, 1997, “Bayesian Model Averaging for Linear Regression Models,” Journal of the American Statistical Association, Vol. 92 (437), pp. 179–91.
- Search Google Scholar
- Export Citation
Sala-i-Martin, Xavier, Gernot. Doppelhofer, and Ronald I. Miller, 2004, “Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach”. The American Economic Review, Vol. 94 (4), pp. 813–35.
- Search Google Scholar
- Export Citation

The 2011 and 2018 survey results are not directly comparable as the 2011 RoC (IMF, 2012a) used interviews of EDs, while the 2018 RoC used written questionnaires.

Programs supported by GRA resources must be designed to resolve the member’s BOP problem during the program period. More specifically, the policy measures that need to be taken to resolve a member’s BOP need should be undertaken during the program period. Such policies should be implemented in a manner that will lead to a strengthening of the member’s BoP before repurchases begin.

A “drawing arrangement” here refers to a case where a member actually made a drawing.

This means that lower access programs include annualized access below 50 percent of quota for programs approved before January 2016 (when the quota reform was completed) and 33.75 percent after that.

Under the LIC-DSF, IMF country teams are required to report an updated external public debt sustainability assessment every year.

Different from the GRA methodology, performance is measured relative to program design (except for the DSF rating) rather than relative to past outcomes. This approach is used to avoid penalizing PRGT countries experiencing significant historic volatility of growth, inflation, and budget-related indicators.

Given different emphasis on specific PRGT-mandated objectives in various programs, progress on at least three indicators is deemed sufficient for full program success.

The experience discussed in this section predates the adoption of revisions to the Fund’s EA policy in 2016.

The CPIA score is a proxy for policy and institutional quality based on expert judgment consisting of four dimensions: (i) economic management; (ii) structural policies; (iii) policies for social inclusion and equity; and (iv) public sector management and institutions. The index, which is provided by the World Bank, is also used by the IMF and the World Bank to identify fragile states.

LICs in fragile situations mostly used the ECF given its focus on structural reforms and longer program duration, allowing for time to implement reforms.

The decline in 2014 reflects a drop in the number of Fund-supported programs in fragile states (from 10 in 2012 to 2 in 2013 and 2014), which translated into less TA delivery.

Suriname (SBA 2016) is excluded from the analysis due to the early termination of the program before completing the first review. Seychelles (2017 PCI) is excluded, as the program is in its initial stages and reviews assessed by the Executive Board fall beyond the end-2017 cutoff date for the 2018 RoC period.

For more details on the potential use of state-contingent debt instruments for small open economies subject to large exogenous shocks, such as natural disasters and commodity prices shocks, see IMF (2017b).

Jointly with the Bank, comprehensive Climate Change Policy Assessments (on a pilot-basis) have been conducted for Belize (2018), Seychelles (2017) and St. Lucia (2018). The CCPA for Seychelles informed SBs under the ongoing PCI program, approved in December 2017.

For more details, please refer to Small States’ Resilience to Natural Disasters and Climate Change—Role for the IMF (IMF 2016). Ongoing work on Building Resilience in Countries Vulnerable to Large Natural Disasters considers the role

For this to be true, there should be no systematic forecast errors of the size of domestic adjustments. As noted in the main paper, the size of adjustments was close to planned, as also observed by Blanchard and Leigh (2013).

This probit model was developed by staff for internal purposes to provide a probabilistic assessment of debt distress for a MAC seeking IMF support. New models are being developed in the context of the ongoing MAC DSA Review.

The choice of the 80^th percentile for the high probability threshold is for the purposes of this analysis. The results are reasonably robust to alternative thresholds.

End-program for ongoing programs refers to the latest data point.

Leamer (1978) discusses the use of Bayesian methods to select econometric models. Raftery and others (1997) introduces the Bayesian Model Averaging as an alternative approach to hypothesis testing and model selection in social sciences. Sala-i-Martin et al. (2004) first applied the Bayesian Model Averaging methodology to growth regressions.

Time horizon	1991–2014
Observations with Dep. = 0	192
Observations with Dep. = 1	19
Explanatory variables	Coefficient estimate ^2/
Constant	-1.748**
Public debt to GDP (t-1)	0.016**
Primary balance to GDP (t-1)	-0.020
Real GDP growth (t-1)	-0.090**
Inflation (t-1)	0.017
Real GDP per capita (t-1)	-0.0001**
International reserves to GDP (t-1)	-4.348*
Real exchange rate overvaluation (t-1)	0.030**
McFadden R-squared	0.30

Time horizon	1991–2014
Observations with Dep. = 0	192
Observations with Dep. = 1	19
Explanatory variables	Coefficient estimate ^2/
Constant	-1.748**
Public debt to GDP (t-1)	0.016**
Primary balance to GDP (t-1)	-0.020
Real GDP growth (t-1)	-0.090**
Inflation (t-1)	0.017
Real GDP per capita (t-1)	-0.0001**
International reserves to GDP (t-1)	-4.348*
Real exchange rate overvaluation (t-1)	0.030**
McFadden R-squared	0.30

Topics

Countries

Series

About

Resources

Abstract

2018 ROC Surveys

A. Ownership

B. Tailoring and Uniformity of Treatment

C. Collaboration

D. Outreach

Previous Policy Reviews

Assessing Program Success

A. Methodology

Methodology for GRA-Supported Programs

Methodology for PRGT-Supported Programs and Policy Support Instruments (PSIs)

B. Data and Degree of Success

C. Success Factors

Recent Fund Experience with Debt Restructuring and Reprofiling

Staff-Monitored Programs

Tailoring and Uniformity of Treatment: Fragile States and Small States

A. Fragile States

B. Small States

Technical Notes

A. Program Success: Financing Versus Adjustment

Methodology

Data

Results

B. Macro Optimism: Drivers of Growth Forecast Errors

Methodology

Data

Results

C. Macro Optimism: Growth Accounting

Methodology

Data

Results

D. Public Debt: Sustainability Assessment for Market Access Countries Seeking IMF Support

Methodology

Data

Results

E. Structural Conditionality and Program Design: Impact of Moving to Review-Based Conditionality

Methodology

Data

Results

F. Ownership: Understanding Completion Rates

Methodology

Data

Results

G. Tailoring and Uniformity of Treatment: Understanding Access Decisions

Methodology

Data

Results

References

Same Series

Other IMF Content

Other Publishers

Asian Development Bank

Inter-American Development Bank

Nordic Council of Ministers

The World Bank