2021 Comprehensive Surveillance Review—Background Paper on The Surveillance Priority Confronting Risks and Uncertainties

The coverage of risks has become more systematic since the Global Financial Crisis (GFC): staff reports now regularly identify major risks and provide an assessment of their likelihood and economic impact, summarized in Risk Assessment Matrices (RAM). But still limited attention is paid to the range of possible outcomes. Also, risk identification is useful only so much as to inform policy design to preemptively respond to relevant risks and/or better prepare for them. In this regard, policy recommendations in surveillance could be richer in considering various risk management approaches. To this end, progress is needed on two dimensions: • Increasing emphasis on the range of potential outcomes to improve policy design. • Encouraging more proactive policy advice on how to manage risks. Efforts should continue to leverage internal and external resources to support risk analysis and advice in surveillance.

Abstract

The coverage of risks has become more systematic since the Global Financial Crisis (GFC): staff reports now regularly identify major risks and provide an assessment of their likelihood and economic impact, summarized in Risk Assessment Matrices (RAM). But still limited attention is paid to the range of possible outcomes. Also, risk identification is useful only so much as to inform policy design to preemptively respond to relevant risks and/or better prepare for them. In this regard, policy recommendations in surveillance could be richer in considering various risk management approaches. To this end, progress is needed on two dimensions: • Increasing emphasis on the range of potential outcomes to improve policy design. • Encouraging more proactive policy advice on how to manage risks. Efforts should continue to leverage internal and external resources to support risk analysis and advice in surveillance.

Introduction

1. Mitigating risks is one of the key objectives of surveillance. The Articles of Agreement require the Fund to oversee the international monetary system in order to ensure its effective operation, including sustained sound economic growth, and to oversee the compliance of each member with its obligations, including the member’s obligation to “seek to promote stability by fostering orderly underlying economic and financial conditions and a monetary system that does not tend to produce erratic disruptions” (IMF Articles of Agreement, Article IV, Section 1). As set out in the Integrated Surveillance Decision, the Fund, in its surveillance over the policies of individual members, will clearly and candidly, assess relevant risks and policies of the member (Bilateral and Multilateral Surveillance Executive Board Decision, para 14).

2. Risks have been shifting across income groups and over time, with a trend toward more stability interrupted by severe crises. Figure 1 illustrates the evolution of risks, proxied by the volatility of GDP growth and the frequency of crisis events (in external, financial and fiscal sectors). Low-income countries (LICs) have long suffered from relatively high instability, reflecting their exposure to terms-of-trade shocks, and natural and man-made disasters. In emerging markets (EMs), current and capital account reversals culminated in the waves of crises triggered by events in Mexico (1994), Asia (1997) and Russia (1998), contributing to high volatility in the 1980s and 1990s. There has nevertheless been a trend toward more stability in both groups over the past three decades and the compression in volatility was even more apparent for Advanced Economies (AEs). But the Global Financial Crisis (GFC) brought a realization that policies producing low volatility could lead to a buildup of vulnerabilities ultimately leading to a more severe crisis. Similarly, the recent Covid-19 crisis is a stark reminder that risks may be looming behind the façade of apparent stability, possibly coming from forces outside the control of economic policymakers.

Figure 1.
Figure 1.

Recurring Volatility and Crises

Citation: Policy Papers 2021, 029; 10.5089/9781513578613.007.A001

Sources: WEO and IMF (2020a).Notes: Volatility is constructed as percent deviation around 3-year one-sided moving average of GDP growth. Crises cover external, financial, and fiscal crises as defined in the VE; See IMF (2020a) for definitions.

3. Fund surveillance started paying more attention to risks in the aftermath of the GFC. The 2008 Triennial Surveillance Review (TSR) recommended a more systematic exploration of risks and possible outcomes through an improved toolkit to bring more rigor to quantifiable risk assessments, and greater emphasis on tail events. Subsequent TSRs (2011 and 2014) recommended deepening risk assessment and better integrating it with policy advice. These findings were reflected in the 2015 Guidance Note for Surveillance. The 2018 Interim Surveillance Review (ISR) documented considerable progress in the Fund’s ability to map risks and vulnerabilities and greater consistency between identification and discussion of risks.

4. But surveillance needs to adapt more decisively to recognize a plethora of risks on the horizon, and more importantly, be ready to respond to them through policy design. Large and costly (even if infrequent) shocks, high uncertainty surrounding their medium- and long-term implications (including spillovers and spillbacks), and—for some countries—fast-growing vulnerabilities and dwindling policy space come together to create a precarious risk environment. Structural shifts add to hazards, including frequent natural disasters attributed to climate change, cyber-risks in digitalized economies, geopolitical competition, and growing inequalities attributed to the side effects of automation and globalization. Executive directors, authorities and mission chiefs find confronting these risks and uncertainties a key priority for surveillance (Figure 2). The scale and range of potential challenges call for a deeper and more holistic approach to risk assessment. A deeper appreciation of the range of outcomes should also be accompanied with a tighter integration of risk management within policy advice.

Figure 2.
Figure 2.

CSR Survey: Confronting Risks and Uncertainties is Key Priority

(Responses in percent)

Citation: Policy Papers 2021, 029; 10.5089/9781513578613.007.A001

Source: CSR Survey.

5. This paper discusses how to adjust surveillance in response to these challenges. To set the stage, the paper starts by defining risk and the concept of risk management in the macroeconomic context. It then discusses the coverage of risk in Fund surveillance through this lens and identifies gaps. Finally, it discusses how to improve the assessment of risks and strengthen risk-oriented policy advice by better leveraging analytical tools and more fully tapping internal and external expertise.

Topology of Risks and Policy Responses

6. What is risk? Risk is typically defined as the 'effect of uncertainty on objectives', which can arise on both sides of the baseline projections. Uncertainty arises from events (which may or may not happen), potential impacts on objectives (both negative and positive), and ambiguity and lack of information.1 In the macroeconomic context:

  • Random events—or shocks—are the underlying source of risks. They could originate from autonomous changes in sentiment (Keynes’ “animal spirits”), natural phenomena (including natural disasters) or developments in the global economy, such as changes in commodity prices (which small economies take as given) or in risk aversion driving capital flows. Shocks can have different statistical properties, ranging from normal distributions (with all moments defined) to distributions with skews or heavy tails and not all moments finite (characterizing crisis-like events). Finally, some events/possibilities cannot be captured by probability models. They are often referred to as ‘unknown unknowns’ and include unforeseeable extreme events (‘black swans’).2

  • The impact of shocks on objectives is shaped by broadly defined economic fundamentals. Objectives are typically various measures of welfare (e.g., national income). Fundamentals include the structure of the economy, physical infrastructure, institutions, policy frameworks, and macro-financial fundamentals. Some may cushion the impact of negative shocks or better enable to respond to positive shocks (e.g., diversified economic structure, quality infrastructure, flexible product and labor markets, and credible and well-design policy frameworks). Others may amplify them, creating vulnerabilities (e.g., high levels of debt or fragilities in financial markets).

  • Uncertainty. In addition to shocks, uncertainty arises from limited information (e.g., about the impact of a vaccine rollout on pandemic dynamics in the COVID crisis), and imprecise understanding of the mechanisms governing the functioning of the economy. This could lead to policy mistakes and be a separate source of risks.

7. How can macroeconomic risks be managed? Typically, risk management comprises identification, assessment, mitigation and insurance against risks, and development of contingency plans for remaining risks. While traditional risk management tends to concentrate on the potential negative effects of uncertainty, risk practitioners now promote a more symmetric treatment of threats and opportunities (Hillson, 2001). In the macroeconomic context these concepts translate to:

  • Risk identification and assessment. Identification and assessment comprise data and techniques to detect relevant shocks and their statistical properties (to assess likelihoods of realizations of different outcomes) and determine their potential impact on objectives using historical patterns and models. Risk identification also includes keeping in mind unlikely but plausible events whose statistical properties cannot be determined (‘unknown unknowns’).

  • Risk mitigation. Risk mitigation involves developing policies to reduce threats to objectives from adverse shocks and to enhance opportunities from the realization of positive shocks. Examples of policies to increase resilience to adverse shocks include investment in infrastructure to limit the impact of natural disasters or incentivizing diversification to cushion the economy from changes in terms-of-trade. While such policies could simultaneously help enhance general economic opportunities (e.g., diversification may encourage a shift to higher-value-added activities), risk mitigation may also consider additional policies to maximize benefit from the realization of positive shocks (e.g., investment in bandwidth to benefit from yet unknown opportunities in the digital revolution). Risk mitigation policies are pre-emptive: they are decided and implemented before shocks occur. They make baseline more robust to risks by limiting the impact of exogenous shocks but in some instances also by lowering the likelihood of self-fulfilling negative risks materializing (e.g., keeping debt levels below certain limits can reduce the likelihood of a self-fulfilling debt crisis).

  • Insurance. Insurance transfers negative risk to a party best able to manage it. Insurance options depend on the nature of risk, with complexity and cost related to the ex-ante ability to accurately predict future values of random variables. Insurance though financial instruments is readily available against events characterized by normal distributions (e.g. rainfalls). It is more costly though against harder-to-predict macro-relevant fluctuations in commodity prices, and only partial insurance is available against even less predictable natural disasters (through catastrophe bonds).3 In the case of sudden stops, in addition to unpredictability, market insurance is further complicated by the large scale of potential needs and the problems of adverse selection and moral hazard.4 Swap lines and contingency lines from multilateral institutions such as the Flexible Credit Line (FCL) are a form of insurance that overcome some of these problems, including through surveillance in the case of the FCL. Insurance transfers negative risk, but equivalent approaches could be considered for handling positive risk: transferring it to a party best able to maximize the probability of its occurrence and to increase potential benefits should it materializes (government granting exploration rights to natural resources to a third party is an example of such an approach).

  • Self-insurance (precautionary savings). Self-insurance is holding buffers such as international reserves or special stabilization funds to help smooth domestic absorption when risks materialize. In some instances, buffers (and market-based insurance or multilateral contingency lines discussed above) can also reduce the risk of a self-fulfilling crisis similarly to the mechanism discussed above.5

  • Contingency plans. Many policies to reduce negative risks or enhance opportunities are costly in terms of forgoing resources that could be productively used elsewhere or consumed. It is therefore optimal to accept some levels of risks, both positive and adverse. Contingency plans to respond to residual risks (and, to the extent possible, risks arising from events that are entirely unpredictable—'black swans’) should be part of the risk management strategy.

Risk Analysis at the Fund

A. Framework

8. Changes to surveillance recommended in post-GFC reviews aimed to bring elements of risk management into Fund policy advice. The 2008 Triennial Surveillance Review (TSR) focused on risk identification and assessment. It recommended a more systematic exploration of risks by bringing more rigor to quantifiable risk assessments through an improved toolkit (including the selective use of fan charts) and greater emphasis on tail events, even if unquantifiable, to raise awareness and institutional preparedness (scenario analysis was recommended as a helpful tool for this purpose). Subsequent TSRs (2011 and 2014) shifted attention to risk mitigation through better integration of risk assessment with policy advice.

9. These changes were articulated in the 2015 Guidance Note for Surveillance (GNS). The GNS recommends that Article IV consultations include risk assessment tailored to country circumstances and integrated throughout the staff report. The identification of risks (around the baseline and tail), should be followed by the assessment, including, where feasible, the probability of risks materializing and their likely impact. This should, in turn, motivate policy discussions: where feasible, staff should propose actions to lower the probability of negative risks materializing (pre-emptive policies) or to mitigate their impact if they materialize (insurance) and suggest possible contingent actions (contingency planning).

10. The GNS introduced Risk Assessment Matrix (RAM) to bring rigor and consistency to the coverage of risk in staff reports. The RAM is a summary of major global, regional, or country-specific macroeconomic, financial sector and geopolitical risks, as well as risks to implementation. It should include an assessment of the likelihood and impact of risks, based on the evaluation of vulnerabilities and their interactions with external and domestic shocks. This assessment should motivate policy discussion and advice on risk treatment, summarized in the RAM but also discussed throughout the staff report. To ensure consistency, global and regional risks should be drawn from the Global Risk Assessment Matrix (G-RAM) listing key global and regional risks and assigning to each a broad probability of realization (low-medium-high).6

11. The GNS also encouraged applying a broad set of quantitative tools for risk assessment. For risks with a material impact on the outlook, recommended tools include quantified scenarios (from multi-country structural and empirical models) to shed light on the transmission channels and impact (scenarios based on global risks should be based on the G-RAM). Staff is also expected to leverage multilateral surveillance products—for instance downside scenarios in the WEO and GFSR—together with other quantitative tools, such as sectoral crisis prediction models underpinning the Vulnerability Exercise (VE).7

12. Specialized tools were recommended for sectoral assessments. The assessment of fiscal sustainability should be informed by a public debt DSA, and staff is also encouraged to assess risks to the public sector using balance sheet analysis where feasible. Financial systemic risk analysis can draw on various tools developed for the FSAP, for instance bank stress tests, network analysis to capture dependencies among institutions, and liquidity assessment tools.

B. Implementation and Gaps

13. The risk coverage in staff reports has become more structured and comprehensive following the issuance of the GNS but there are still gaps. While several elements of the guidance note have been implemented, some of the key recommendations of the surveillance reviews remain outstanding: i) there has been limited progress toward quantification; ii) policy recommendations often lack depth in considering various risk management policies and tools, and tradeoffs among them; and iii) risk assessment is not well integrated throughout staff reports (which focus predominantly on the baseline).8

  • Risk identification in staff reports has improved with the widespread use of the G-RAM and RAMs. RAMs have been widely adopted in staff reports, reporting country-specific and—drawing on the G-RAM—global risks. The G-RAM risks—identified based on expert judgment—identify the risks stemming from broad macro developments (e.g., growth slowdown) as well as from specific events (e.g., a trade war). Identification of country-specific risks have included for example an erosion of fiscal discipline, natural disasters, and political instability.

  • Risk assessment has also strengthened. Staff report RAMs provide expert-judgment-based assessments of the likelihood of the materialization of risks and their impact. Some global risks have been quantified (typically using RES large-scale models) and a few staff reports have picked up on these to present alternative scenarios to test the implications to their countries of alternative states of the world based on internal forecasting tools. Even fewer staff reports have quantified risks to projections in the form of predictive density (fan charts). Staff reports, however, have usually included an assessment of the overall balance of risk, with most reports assessing risks as skewed to the downside, especially for EMs and LICs (Figure 3; upper panel).Distributions of historical growth forecast errors are skewed to the left (Figure 3; lower panel), suggesting that the assessment of the balance of risk is supported by the data, but also pointing to an average forecast bias in projections as reported in the 2018 Review of Program Design and Conditionality (IMF, 2018). These skews could often be tracked to crises events, potentially foreseen in RAMs. Besides the regular risk assessment, DSAs have typically included a rigorous and quantitative discussion of risks to debt sustainability but these (longer-term) risks were rarely linked to assessments in RAMs.

Figure 3.
Figure 3.

Balance of Risk Assessment and Forecast Errors in Staff Reports

Citation: Policy Papers 2021, 029; 10.5089/9781513578613.007.A001

Sources: WEO, staff automated review of Article IV reports; and IMF (2020a).Notes: Illustrative example—the results are qualitatively similar for 2011-19. External and fiscal crises indicate the parts of error distributions attributable to countries experiencing crisis as defined in the VE.
  • Risk mitigation policies are typically identified, but the discussions often lack depth. RAMs in a sample of surveyed staff reports covered risk management policies, including preemptive macroeconomic policies (maintaining or increasing exchange rate flexibility, structural reforms, and macro-prudential policies to ameliorate or contain financial risks), building buffers (fiscal and external), and contingent policies in responses to shocks. However, the policy responses, while comprehensive, were not sufficiently targeted, often proposing the same policy responses for different types of risks. They were also typically presented without discussing alternative risk mitigation policies and tools or discussing tradeoffs among them. And, the impact of policy recommendations in response to risks was rarely quantified.

  • There has been only partial integration of risk assessment and risk-related policy advice in staff reports. The probability and impact of RAM risks were typically summarized in the ‘Outlook and Risks’ sections of staff reports, but without further elaboration. And policy responses to risks were rarely integrated with broader policy advice. For instance, it was often unclear whether and how pre-emptive policies considered in RAMs (e.g., moving towards greater exchange rate flexibility, accelerating structural reforms, or building fiscal buffers) were related to policy recommendations in the baseline.

  • The coverage and integration of financial and macro-financial issues in Article IV surveillance have been strengthened, but quality remains uneven (see also background paper on macro-financial risks). Several analytical tools (e.g., stress test templates) have helped Article IV teams deepen their macro-financial analysis and link it to financial policy advice. The toolkit for assessing financial risks has now been expanded to include Growth-at-Risk (GaR; discussed below) and Capital Flows-at-Risk, and the Global Stress Test. Still, the quality remains uneven, with the coverage of macro-financial linkages often limited in depth (IEO, 2019).

Way Forward

14. The focus of surveillance should more decisively shift toward risk assessment and advice on how to manage risks. Although considerable advances have been made in this direction since the GFC, the Fund’s surveillance focus and advice still pays inadequate attention to alternative outcomes and how they can be shaped for the better by policies. The post-COVID macroeconomic environment, with many economies facing risks on both sides depending on the prospects in vaccination, requires a more decisive shift in order to better support our membership in designing policies that are robust to a range of possible outcomes. To this end, progress is needed on two dimensions:

  • Increasing emphasis on the range of potential outcomes to improve policy design. This could be done through greater use of scenarios or fan charts (or other representations of predictive densities), illustrating the impact of potential shocks—both negative and positive —under various risk management policies. Considering such a range would instill a mindset of internalizing risks in policy design and help reduce forecast bias in projections. Where data challenges constrain quantitative analysis of risks (such as in some low-income countries), risks could still be priced in policy advice through simple conceptual or even qualitative exercises (see below). Notwithstanding potential communications challenges (discussed below), teams should also consider low-probability high-impact risks and ‘black swan’ events.

  • Encouraging more proactive policy advice towards risks. This includes proferring clear advice on the relative importance and effectiveness of various risk management policies and tools, including intra-temporal tradeoffs between taking calculated bets and addressing risks robustly (e.g., self-insurance through a buildup of buffers could be a substitute for risk mitigation through economic diversification and enhancing market flexibility), inter-temporal trade-offs (e.g., by providing immediate support to economic activity at the cost of a buildup of more uncertain longer-term risks), and tradeoffs between reducing negative risks and enhancing opportunities (e.g., by investing in disaster-proof infrastructure at the cost of forgoing investment in bandwidth infrastructure and widening the digital divide). Such a meaningful analysis of alternative policy options should make the baseline policy advice more robust to risks and, where relevant, help develop contingency plans. The aim should be to deliver advice that confronts the most important risks (reflecting economic fundamentals and exposures to shocks) with policies that are calibrated to country-specific circumstances (including policy implementation capacity).

15. Such a recalibration in Fund advice will require reducing the resource costs of undertaking risk analysis and improving communication. Technical improvements discussed below would help address the resource costs of risk analysis. Concerns about triggering self-fulfilling tendencies by discussing downside risks (especially low-probability high-impact risks and hypothetical ‘unknown unknowns’) will, however, need to be carefully managed. In this regard, concerns could be alleviated by ensuring more systemic coverage of risks across all staff reports (avoiding the impression of ‘singling out’ countries where risks are reported); using common global assumptions (akin to a standardized treatment of shocks in the DSA); including an appropriately long horizon for the risk analysis (making the materialization of risks potentially avoidable with sufficient time for policy interventions to address them); and critically, by discussing the benefits of addressing risks preemptively.

A. Adjusting the Framework

16. Better leveraging old and new analytical tools will need to be a key element of the work plan. CSR survey responses indicate that country teams see the need for developing specialized tools to help guide projections and policy recommendations, as well as for stronger support from specialized departments in applying such tools. Overall, country teams prefer tools that are easily implementable—high skills requirements, data or time demands are seen as main obstacles to a shift toward risk-oriented surveillance more —and have intuitive messages that are easy to communicate to authorities.9 These needs could be met by boosting the G-RAM/RAM framework through greater use of quantitative scenarios based on standardized global assumptions, with a clear focus on developing policy advice in response to risks. Modernizing existing forecasting and policy analysis tools (as discussed further below) would reduce the costs of producing such scenarios for country teams. This could be supplemented and informed by rolling out new quantitative models suitable for policy analysis recently developed at the Fund (again developing user-friendly interfaces to facilitate their use) and utilizing more strategic foresight techniques.

17. Presenting quantitative scenarios incorporating the risks noted in individual country RAMs would help bring better balance and focus. It would help overcome the average forecast bias by considering a range of possible outcomes and help assess the efficacy of policy advice across these outcomes, and, if necessary, design contingency plans. A more regular and routine discussion of scenarios could also produce a fruitful discussion of assumptions, potential outcomes, plausibility and policy implications, both within the Fund and with the authorities. Moving further along this direction would require in concrete terms:

  • More frequent quantification of G-RAM risks. More systematic quantification of G-RAM risks (e.g., using the Fund’s multi-country DSGE models) would provide individual country teams with a set of global macroeconomic assumptions spanning the range of possible outcomes—both positive and adverse—that could be used for country-specific scenario analysis,. It could also be used to sharpen the likelihood assessment of G-RAM risks by comparing percentile-wise positions of quantified G-RAM scenarios against WEO predictive densities (for growth and possibly other variables), making it more objective and data-driven, and improving consistency of risk assessments across Fund products.10

  • Quantifying country-level impacts of global and domestic risks. Quantitative global G-RAM scenarios could be used to assess country-specific impacts of global risks. This should be complemented by the quantification of purely country-specific risks also included in the RAMs. Teams currently use eclectic modelling tools to produce a set of detailed baseline economic forecasts summarized in country spreadsheets. This approach offers internal consistency, flexibility and helps overcome the limitations of using exclusively any particular forecasting model. But producing scenarios within such a framework is resource-intensive, partly explaining the limited use of quantitative RAM scenarios in staff reports thus far. To help teams develop the capacity to cope easily and rapidly with alternative scenarios, the macro-framework methodology could be usefully improved through greater standardization and automation. A flexible “Financial Programming Environment” currently under development (see Box 1) is a step in this direction, integrating judgment, models, and data with a user-friendly Excel front-end that can produce various standardized outputs. The planned environment is scenario-oriented by design and makes it simple to prepare, compare and analyze alternative risk scenarios (including under various policy responses) in a coherent framework. It can also be used for countries with data limitations—such as fragile states and LICs—where scenarios are an effective tool for communicating risks and shaping policy advice (see Box 2).

New Financial Programming Environment

The Financial Programming Environment (FPE) is a new software solution for working with macro- frameworks. The FPE is being developed jointly by ICD and ITD with support from the iLab and is part of a broader initiative to modernize the financial programming toolkit (FP 2.0 project).

The FPE is designed to facilitate building more coherent macro-frameworks with emphasis on forward-looking analysis of economic imbalances and risk quantification. It introduces new important features including support for generating alternative (risk) scenarios and ability to link the macro-frameworks to a broad class of forecasting models (DSGE, econometric, machine learning, etc.) implemented in virtually any software (Matlab, EViews, Python, Excel, etc.). Its design is flexible to accommodate macro-frameworks and forecasting tools of various degrees of complexity reflecting specific country and country team circumstances (e.g. data quality and availability, characteristics of the economy and experience of the team with different forecasting tools). Country teams can also choose not to rely on any forecasting models and build purely judgmental forecasts as is often the case in their current macro-frameworks.

The FPE also improves user experience in working with macro-frameworks. It introduces a structured and transparent way of combining main elements of macro-frameworks (accounting macroeconomic identities, model-based and judgmental forecasts, and data). In addition, the environment eliminates vulnerabilities of the current solution which relies on interrelated Excel spreadsheets.

Figure 1 shows an example of a macro-framework in the FPE. In this example, the user has prepared two macro-framework scenarios—baseline and a downside alternative based on a more pessimistic assumption about the COVID-19 pandemic. Both scenarios use the same forecasting model to make projections of the main macro variables.

Figure 1.
Figure 1.

Example of the macro-framework in the new Financial Programming Environment

Citation: Policy Papers 2021, 029; 10.5089/9781513578613.007.A001

The Use of Alternative Scenarios

Scenarios have been an effective way to cover risks in surveillance, including in EMs and LICs. They are easy to communicate and, since in most cases they are built as modifications to the baseline macro-framework, effective in illustrating the impact of shocks under baseline and alternative policies. The following recent examples illustrate some best practices.

The 2019 South Africa Article IV Staff Report—using scenarios to guide the policy discussions. Two alternative scenarios were developed: i) a downside scenario illustrating the risk from policy delays against the background of a materialization of external risks of tighter global financial conditions (informed by the G-RAM); and ii) an upside scenario, assuming implementation of staff’s policy advice and illustrating the benefits of comprehensive reforms.

The 2019 South Sudan Article IV Staff Report—illustrating uncertainty related to the peace process. The Report recognized that the economic outlook critically depended on whether credible and lasting peace could be established after the signing of the peace agreement. In a downside scenario, driven by political instability, oil production would be substantially weaker, putting pressure on external and fiscal balances, and authorities would resort to central bank financing of the deficit, leading to severe depreciation and high inflation. Upside scenarios modelled a stronger recovery path for the oil sector given its importance for the South Sudanese economy.

The 2021 Nigeria Article IV mission—simulating the impact of a resurgence of the pandemic. Scenarios to demonstrate the impact on a resurgence of the COVID-19 pandemic on growth and public debt were generated from a newly developed DIGNAR-19 toolkit. Similarly to the Financial Programming Environment discussed in Box 1, DIGNAR-19 is an example of a specialized model-based tool using a user-friendly interface to generate alternative scenarios (focusing on pandemic-related shocks).

18. Rolling out new risk-centered models would help inform advice on risk management over time. Several new methods suitable for informing risk assessment and management are currently being developed at the Fund. They bring more focus to the range of potential outcomes by generating predictive densities for target variables and could optionally complement the scenario-based approach when practicable (applying such methods to all countries may not be feasible because of their analytical complexity and, in some cases, heavy data requirements). Annex I discusses these models in greater detail, but they can be classified into three broad groups:

  • Quantile-regression-based methods. GaR and related methods use quantile regressions to construct predictive densities of a response variable (e.g. GDP growth) conditional on macro-financial data.11 These methods are well-suited to capture asymmetries and fat tails in predictive densities—an important feature of risks as discussed above. Even though they are not informative of causal links (being reduced-form models), they can still guide policies, for instance by tracking the impact of changes in policy instruments on predictive densities of the response variable (e.g., the impact of changes in macroprudential indicators in the case of the GaR). The GaR has been regularly used to assess global macro-financial risks in the GFSR and there is a growing number of country-specific applications. A more general model has been developed for constructing predictive densities for global growth in the WEO.

  • Risk-focused structural models. Density forecasts can also be constructed from estimated or calibrated structural econometric models. These models are well suited to analyze the impact of alternative policies to help evaluate the effectiveness of different risk management tools and their tradeoffs. Large-scale DCGE models (e.g., the FSGM) can be used to construct predictive densities by drawing from estimated historical shocks. Given their rich structure, such models can be used to analyze a wide set of alternative policies, including various structural reforms. Smaller, specialized models are in turn better suited to analyze self-fulfilling tail risks such as those arising from financial crises or sudden stops (as these models generate them endogenously) and design risk mitigating policies. While applications at the Fund so far are rare, development of such models is a burgeoning field in academia, and a model incorporating endogenous macro-financial risks has been developed in the context of the work on the Integrated Policy Framework.

  • Crisis prediction models. These models evaluate likelihoods of crises originating in different sectors (e.g., external, fiscal and financial). The new crisis prediction models use state-of-the-art machine learning methodologies that underpin the VE (IMF, 2021), and related models for predicting fiscal stress (using more traditional techniques) inform safety thresholds for debt levels in the DSA (IMF, 2018 and IMF, 2021). While these methods cannot be used to generate predictive densities directly, they can still inform the assessment of tail risks (notwithstanding communication challenges in presenting high-impact low-probability risks discussed above).

19. Strategic foresight tools are an important addition to the Fund risk preparedness framework and there is room to use them more in surveillance. Strategic foresight uses a range of methods to scan the future for ‘unknown unknowns’. Techniques used at the Fund include megatrends analysis, variations of scenario planning exercises, and policy gaming (see Annex II for more details). These methods lack probabilistic assessment, but they cover an important area of risk management by considering unforeseeable but plausible high-impact events and constructing easy-to-communicate risk narrative. So far, these techniques have mostly been used to develop strategic responses to various threats at the institutional level, but there is scope to leverage them more in surveillance through:

  • Identifying G-RAM risks. Strategic foresight techniques are well suited to identifying which specific risk events should be included in the G-RAM and country RAMs.12

  • Developing contingency plans. Strategic foresight tools can also be used to inform policy design in response to high-impact risks.

  • Analyzing long-term risks. Trends extending beyond the short- to medium-term surveillance horizon may lead to a buildup of imbalances and risks to domestic and external sustainability. Strategic foresight techniques can be used to identify these trends and help formulate policies that are needed within the surveillance horizon to address the risks (e.g., climate change).

B. Implementation

20. The Fund has been tapping internal and external resources to improve its capacity to identify and assess risks. Development of new models based on traditional econometric techniques (e.g., GaR) relied on the strong internal expertise at the Fund concentrated in functional departments, leveraging contacts in academia. Modernizing the VE methodology using machine-learning techniques benefited from the pool of young talent at the Fund, but also, given the novelty of these techniques, from engagement with a broader group of outside experts, including in the private sector. Similarly, strategic foresight techniques—often unfamiliar to economists—have been tailored to the Fund’s needs with the assistance of practitioners in fields outside economics (e.g., experts in military planning for policy gaming techniques).

21. Going forward, engagement with outside scholars and practitioners needs to be strengthened. Keeping abreast of fast-moving developments on risk identification and assessment requires an even more active engagement with external experts. In addition to established channels such as participation in academic conferences, a visiting scholars program and occasional outreach missions, a more permanent engagement could rely on establishing or joining networks of experts active in these fields. The Fund is already a participant in the Government Foresight Community, which brings together strategic foresight practitioners in the public sector from around the world.13

22. External engagement should be expanded to make risk identification more holistic and interdisciplinary. Regular risk identification and assessment already rely on interactions with outside experts, including joint exercises with other multilateral organizations (e.g., EWE conducted jointly with the FSB) and frequent and regular contacts with market participants. However, with the growing recognition of potentially major economic repercussions of non-economic shocks (as the COVID-19 outbreak attests), this engagement needs to be expanded to experts outside economics and finance. Several initiatives at the Fund—many under the umbrella of strategic foresight work—have recently been tapping expertise in the fields outside economics to identify emerging trends and risks (e.g., tech firms, political scientists, environmental experts, and—well ahead of the COVID-19 outbreak—epidemiologists). Formalizing such engagements through establishing regular networks of consulting experts in different fields would greatly improve Fund preparedness to risks originating outside the field of economics.

23. Progress in risk identification and assessment should be leveraged to strengthen policy advice on risk management. The new risk assessment models can already be used to help develop policy advice as discussed above. But additional analytical efforts may be needed to develop a framework for assessing the economic impact of non-economic risks and designing policies to best address them. To this end, the Fund should build capacity by encouraging the inflow of a more diverse talent mix, including expertise relevant for the assessment of risks coming from outside of the traditional open economy macro context.

24. Internally, knowledge sharing of both risk assessment and policy design to manage risks needs to be improved. The Senior Risk Group provides a well-functioning coordination mechanism for risk work at the Fund, including updates of key risk assessment products such the VE and the G-RAM. It is supported by the Deputy Risk Group, coordinating technical work and providing a forum for presenting risk-related work at various stages of development for early feedback. To improve the dissemination of knowledge, the resources and the role of Risk Group coordinators could be expanded at the technical level by forming teams of dedicated departmental “risk experts” to assist country teams in the rollout of new tools for risk assessment and policy design. In addition to being depositories of risk-related knowledge, data and tools in departments, such experts could establish cross-departmental networks to participate in the development of new tools reflecting departmental demands (similar but less formalized model proved useful in the development and implementation phase of the new machine-learning-based VE models). Separately, consideration should be given to better leveraging the extensive work on tail risks in surveillance (while reducing potential overlaps across the EWE, Tail Risk Group, and strategic foresight workstreams).

25. Functional departments should still play a leading role in model development and training in close collaboration with area departments. Functional departments would still be expected to take a lead in the development of new tools given their expertise and potential economies of scale in the production of standardized tools. But the process would be collaborative, with area departments actively involved early on through their “risk experts” (and possibly by assigning or seconding their own staff for tool development). Internal training would remain an important element of strengthening country teams’ capacity to first absorb and then contribute to the development of new tools.

26. Capacity development should be an important element of risk-oriented surveillance. Training and technical assistance to member countries on collecting data for monitoring risks and on the application of risk assessment tools would not only help build their technical capacity but, by deepening the dialogue, also enhance traction and effectiveness of Fund surveillance.14 Several Fund member countries have already expressed interest in learning about the newly developed (machine-learning-based) risk assessment tools and sharing the technical expertise in this area more broadly with member countries would be an important public good provided by the Fund.

Annex I. Quantitative Risk Assessment Tools

Density Forecasting Methods

1. What is density forecast and how is it used at the Fund? A density forecast is an estimate of the probability distribution of the possible future values of a random variable. Density forecasts can be produced using different methods (varying in complexity and data requirements) and presented in different ways (as fan charts, a distribution at a certain point in the future, or a probability of falling below a certain threshold—Figure 1). They all convey quantified information that could help reduce forecast bias and design risk mitigation policies, but some methods are more suitable to identify drivers of risks and analyze policy responses.

Figure A1.
Figure A1.

Representations of Predictive Densities for Risk Assessment

Citation: Policy Papers 2021, 029; 10.5089/9781513578613.007.A001

Predictive Densities from Surveys and Financial Markets Data

2. Predictive densities can be constructed from surveys and financial markets data. IMF WEO publications have used both sources to derive predictive densities for selected ‘risk factors’: equity prices (derived from option market data by extracting market participants’ perceptions of the underlying asset price distribution) and inflation, oil prices, and the slope of the yield curve (all derived from the survey of professional forecasters).1 They were then used to build a predictive density for global growth by assuming that its variance and skew are proportional to those of the risk factors and incorporating expert judgment.

3. The methodology has limitations but is nonetheless informative about global risks. Data requirements for constructing ‘risk factors’—availability of surveys or financial instruments—are relatively heavy and the method does not identify factors contributing to risks2 (which motivated the development of alternative methods discussed below). Still, predictive densities for individual risk factors (e.g., commodity prices or financial conditions) convey the market assessment of risks which is relevant for surveillance directly and could be leveraged in other products.

Growth-at-Risk and Related Methods

4. Tools based on quantile regressions are more flexible and link risks and vulnerabilities. Quantile regressions estimate conditional quantiles of the response variable rather than conditional mean as in the standard regression. This allows for constructing predictive densities capturing asymmetries and fatter tails.3 There is a growing number of applications at the Fund:

  • Growth-at-Risk (GaR). GaR uses quantile regressions to generate predictive densities for GDP growth by linking it to financial conditions, macro-financial vulnerabilities, and other risk factors.4 The premise is that buoyant financial conditions contribute to a buildup of macro-financial vulnerabilities, raising the risk of a systemic financial crisis risks when conditions are tightened. The methodology produces a full predictive density for GDP growth, but the focus is on its behavior in the tail: the results are presented as the size of the potential growth decline that would occur with a five percent probability. The method has been applied for systemic economies and the global risk assessment in GFSR, but there is a growing number of country-specific applications.5

  • Generalized predictive densities. The methodology has been extended into a more general tool to generate predictive densities for country-specific and—accounting for dependencies across economies—global growth.6 It uses a larger set of conditioning variables, including financial conditions as in the GaR, but also leading indicators of economic activity, variables proxying external sector and commodity price developments, global financial markets developments, measures of uncertainty, and changes in fiscal stance. It also allows for a flexible incorporation of country-specific information.

5. These methods allow for a richer policy analysis. In addition to the assessment of the balance of risks, GaR-based models can be used for scenario analysis, for instance by shocking macro-financial conditions to assess how the shape of the future growth distribution would change. The models are reduced-form and not informative of causal links, but tracking the impact of shocks (even without considering feedbacks) can still guide policies (particularly macroprudential in the case of GaR, given its focus on financial vulnerabilities). The relative drawback of the methodology is that it requires long data series covering several business and financial cycles (including some tail events) to produce accurate parameter estimates.

Predictive Densities from Structural Models

6. Density forecasts can also be constructed from estimated or calibrated structural econometric models. The advantage of structural models is that they can be used to analyze the impact of alternative policies on predictive densities. But even within the class of structural models, variations in assumptions about the nature of shocks and the degree of non-linearity affect their suitability for risk analysis:

  • Predictive densities from large-scale DCGE models offer a comprehensive picture of risks and allow for constructing policy scenarios. The IMF G20MOD model can be used to construct predictive densities by sampling from historical structural shocks and using the draws to simulate counterfactual paths.7 Such predictive densities capture dependencies across countries and over time and are available for several macro variables. Judgment element can be introduced by over- and under-weighting some periods in the sample (e.g., based on the distance between current and historical financial conditions). The method has recently been applied in the WEO for assessing the risk of a sharp growth slowdown (IMF, 2019). Given the rich structure of the model, it can be used to generate predictive densities under alternative policy responses, for instance structural policies to strengthen resilience.

  • Models generating crisis dynamics endogenously are better suited to analyze high-impact risks. Models with crisis-generating mechanisms are typically smaller and do not allow for the analysis of a broad range of policies as in large-scale models, but they are a powerful tool when tailored to specific policy questions. Examples include models producing financial crises or sudden stops, endogenous debt defaults, or liquidity traps. Box 1 reviews applications of such models for risk-related analysis and Figure 2 illustrates how such a model can be used to construct predictive densities under different policy scenarios (the impact of macroprudential policies in a model generating endogenous financial crises). Applications of such models at the Fund are rare. In the context of the work on the Integrated Policy Framework, MCM developed a hybrid model with error terms featuring endogenous volatility depending on model variables. This model bridges the gap between bigger structural models and those allowing for the richer analysis of risks (while also replicating stylized facts from GaR models).8

Models Generating Crisis Dynamics Endogenously

Small models endogenously generating crisis dynamics can be highly relevant for policy analysis when tailored to specific questions:

  • Endogenous sovereign risk models generate endogenous debt limits (beyond which a sovereign defaults), risk premia, and probabilities of observing a fiscal crisis. Allowing for such endogeneity is highly relevant for economies prone to credit events: Mendoza and Yue (2012) find that it nearly doubles the impact of a negative TFP shock on GDP growth. The literature looked at the range of related policy-relevant issues, such as the role of debt maturity (Hatchondo and Martinez (2009)), procyclicality of fiscal policy (Cuadra et al., 2010), contagion (Arellano et al., 2018); the role of FX reserves (Bianchi et al, 2018), relationship between banking and sovereign crises (Adler and Lizarazo, 2015; Bianchi et al., 2019), and the impact on inequality (D’Erasmo and Mendoza, 2017; Jeon and Kabukcuoglu, 2018). The Fund has not yet developed models featuring endogenous debt defaults.

  • Financial crisis and sudden stop models focus on the impact of endogenous credit constraints (affecting the value of financial collaterals and borrowing capacities) on asset prices and capital flows. They are particularly suitable to analyze the impact of macroprudential policies, as illustrated by Mendoza (2010), Bianchi (2011), Bianchi and Mendoza (2011), Benigno et al. (2013), Fornaro (2015), Boz and Mendoza (2015). Fund’s large-scale models feature financial accelerator mechanisms, but an endogenous amplification of risks is limited as credit constraints are stochastic but exogenous, and international spillovers are determined by cross-country correlations between exogenous risk premia.

  • Non-linear New Keynesian monetary models analyze monetary policy in a very low inflation environment, where it is limited by the zero-lower-bound (ZLB). Fernandez-Villaverde et al. (2015) shows large differences between dynamics generated from linear and non-linear models with ZLBs. Gus et al. (2016) show that linearized models with ZLBs provide a poor description of the economy during deep recessions and distort the size of the fiscal multiplier, potentially misguiding policy advice.

Figure A2.
Figure A2.

Predictive Densities for Policy Analysis

(GDP Growth Projected from DSGE Model of Financial Crisis)

Citation: Policy Papers 2021, 029; 10.5089/9781513578613.007.A001

Source: Staff calculations and Bianchi et al. 2016

Crisis Prediction Models

7. Crisis prediction models evaluate the likelihood of extreme events originating from risks in different sectors. Fund staff developed over time several crisis prediction models underpinning the VE, now largely replaced by state-of-the-art machine learning methodologies (ML) (IMF, 2021). Related models for predicting fiscal stress (using more traditional techniques) inform safety thresholds for debt levels in the DSA (IMF, 2018 and IMF, 2020).

8. New VE models show better predictive power and ability to uncover complex interactions in the data. The recently developed new generation of VE models uses ML methodologies and expands the scope and cross-country coverage of risk assessment (IMF, 2020). The models assess the risk of external crises (sharp exchange rate movements or sudden stops to capital flows), financial crises, fiscal pressures, and sharp growth slowdowns for the entire membership.9 ML not only performs better than older techniques in out-of-sample prediction, but also captures more complex interactions in the data and non-linearities. This provides valuable insights into how domestic vulnerabilities could interact with global and political factors to trigger a crisis. To uncover results, an innovative communication toolkit has been developed to summarize the evolution and distribution of risks and their drivers, and identify countries with similar risk profiles.

9. The VE models do not generate full predictive densities but their focus on extreme risks is highly relevant for surveillance. As discussed in the introduction, realizations of extreme risks after periods of apparent stability have become a salient feature of economic systems, which is why GaR models similarly focus on the tail of growth distributions. The ML VE models can potentially uncover more complex interactions and are less constrained by data availability as they allow for pooling a broader sample of countries for estimation. The models also allow for comparative static analysis of changes in selected factors contributing to risks.

Annex II. Strategic Foresight Tools

1. Strategic foresight uses a range of methods to scan the future in order to prepare for potential risks and opportunities. These methods typically lack probabilistic assessment as they consider possibilities that cannot be captured by probability models (‘unknown unknowns’). Techniques used at the Fund include megatrends analysis, scenario planning, and policy gaming.

2. Megatrend analysis is a framework for identifying key trends determining the future. It identifies and explores key structural trends and their potential interactions based on in-house research and inputs from outside experts (as the relevant trends may lay outside the scope of traditional economics). The Fund used such an analysis to sensitize management and staff to long-term trends and related uncertainties (potential breaks in the trends) that could affect the institution and its members. The trends were also an input to the subsequent scenario building exercise.

3. Scenarios are descriptions of plausible futures combining the trends. Building scenarios starts from determining the scope and purpose of the exercise: Fund’s applications included strategic considerations (on how the institutions should adapt to potential changes in global environment), testing surveillance priorities (applying a ‘premortem’ method—identifying potential failures in the identification of surveillance priorities in scenarios), and regional surveillance (on policy responses to technological and other changes driving labor markets in Africa). Scenario building proceeds with the identification of relevant trends from a variety of sources (discussed above) and combining them into a narrative describing their possible interactions and plausible shock events. Scenarios are typically built in a workshop involving multiple participants knowledgeable on the topic. Such scenarios are not fully quantified but may include some quantifiable elements and use models to check their internal consistency.

4. Policy gaming is a tool for exploring endogenous interactions between various actors. Policy games are dynamic strategic simulations and include structured roleplay and elements of chance. There is typically a starting scenario and an imagined shock event (relevant for the defined purpose of the exercise) that prompts actions from participants. Each game results in a narrative for how a shock-prompted action-reaction sequence might unfold. To date, the Fund has used matrix policy games—a variant that allows for analyzing complex relationships among multiple players with aligned and competing interests. Policy gaming has recently been applied to the analysis of trade tensions and policy responses to the Covid-19 pandemic.

References

  • Adler G. and S. Lizarazo. 2015. “Intertwined sovereign and bank solvencies in a simple model of self-fulfilling crisis.” International Review of Economics and Finance, 2015, vol. 39, issue C, 428-448.

    • Search Google Scholar
    • Export Citation
  • Adrian, T., N. Boyarchenko, and D. Giannone, 2019, “Vulnerable Growth,” American Economic Review, VOL. 109, NO. 4, pp. 1263-89.

  • Adrian, T. and F. Duarte, N. Liang, and P. Zabczyk. 2020. “Monetary and Macroprudential Policy with Endogenous Risk”. IMF Working Paper, WP/20/236.

    • Search Google Scholar
    • Export Citation
  • Adrian T. and F. Vitek. 2020. “Managing macro financial risk.” IMF Working Paper, WP/20/151.

  • Ananthakrishnan, Prasad, S. Elekdag, P. Jeasakul, R. Lafarguette, A. Alter, A. X. Feng, and C. Wang. 2019. “Growth at Risk: Concept and Application in IMF Country Surveillance,” IMF Working Paper, WP/19/36.

    • Search Google Scholar
    • Export Citation
  • Andrle M., P. Blagrave, P. Espailllat, K. Honjo, B. Hunt, M. Kortelainen, R. Lalonde, D. Laxton, E. Mayroeidi, D. Muir, S. Mursula, and S. Snudden. 2015. “The Flexible System of Global Models -FSGMIMF Working Paper WP/15/64.

    • Search Google Scholar
    • Export Citation
  • Arellano C., Y. Bai, and S. Lizarazo. 2017. “Sovereign Risk Contagion.” NBER Working Paper #24031.

  • Benigno G., H. Chen, C. Otrok, A. Rebucci, and E. Young. 2013. “Financial crises and macro-prudential policies.” Journal of International Economics, Vol. 89, Issue 2, March 2013, pages 453-470.

    • Search Google Scholar
    • Export Citation
  • Bianchi J. 2011. “Overborrowing and systemic externalities in the business cycle.” American Economic Review, 101 (7).

  • Bianchi J. and E. Mendoza. 2011. “Overborrowing, Financial Crises and 'Macroprudential' Policy.” IMF Working Paper, WP/11/24. February 2011.

    • Search Google Scholar
    • Export Citation
  • Bianchi J., J. Hatchondo, and L. Martinez. 2018. “International Reserves and Rollover Risk.” American Economic Review, Vol. 108. No 9, pp. 2629-70

    • Search Google Scholar
    • Export Citation
  • Bianchi J., Lizarazo S., and H. Sapriza. 2019. “Bailouts Guarantees, Banking Crises and Sovereign Debt Crises.” Working paper June 2019.

    • Search Google Scholar
    • Export Citation
  • Boz E., and E. Mendoza. 2014. “Financial innovation, the discovery of risk, and the U.S credit crisis.” Journal of Monetary Economics 62, (2014) 1-22

    • Search Google Scholar
    • Export Citation
  • Britton, E., P.G. Fisher, and J.D. Whitley. 1998. ‘The Inflation Report Projections: Understanding the Fan Chart’. Bank of England Quarterly Bulletin, Vol. 38, pp. 3037.

    • Search Google Scholar
    • Export Citation
  • Calvo, G., A. Izquierdo, and R. Loo-Kung. 2013. "Optimal Holdings of International Reserves: Self-insurance against Sudden Stops," Monetaria, Centro de Estudios Monetarios Latinoamericanos, CEMLA, vol. 0(1), pp. 1-35.

    • Search Google Scholar
    • Export Citation
  • Cuadra G., J. Sanchez, and H. Sapriza. 2010. “Fiscal policy and default risk in emerging markets.” Review of Economic Dynamics, Vol 13, Issue 2, April 2010, pages 452-469

    • Search Google Scholar
    • Export Citation
  • D’Erasmo P. and E. Mendoza. 2017. “Optimal Domestic (and External) Sovereign Default.” Working paper 17-4 Federal Reserve Bank of Philadelphia.

    • Search Google Scholar
    • Export Citation
  • Eichengreen, Barry, P. Gupta, and A. Mody. 2008. “Sudden Stops and IMF-Supported Programs.” Financial Markets Volatility and Performance in Emerging Markets, pages 219-266, National Bureau of Economic Research, Inc.

    • Search Google Scholar
    • Export Citation
  • Elekdag, S. and P. Kannan. 2009. “Incorporating Market Information into the Construction of the Fan Chart.” IMF Working Paper, WP/09/178.

    • Search Google Scholar
    • Export Citation
  • Fernandez-Villaverde, J., G. Gordon, P. Guerron-Quintana, and J.F. Rubio-Ramirez. 2015. “Nonlinear adventures at the zero-lower bound.” Journal of Economic Dynamics & Control 57 p. 182-204.

    • Search Google Scholar
    • Export Citation
  • Fornaro L. 2015. “Financial crises and exchange rate policy.” Journal of International Economics, Elsevier, vol 95(2), pages 202-215

    • Search Google Scholar
    • Export Citation
  • Gust C., E. Herbst, D. Lopez-Salido, and M. Smith. 2017. “The empirical implications of the interest-rate lower bound.” American Economic Review, vol. 107, No 7, July 2017, pp 1971-2006.

    • Search Google Scholar
    • Export Citation
  • Hansen, Lars Peter and T. J. Sargent. "Four types of ignorance." Journal of Monetary Economics (2014).

  • Hatchondo J., and L. Martinez. 2009. “Long-duration bonds and sovereign defaults.” Journal of International Economics, 2009, vol. 79, issue 1, 117-125

    • Search Google Scholar
    • Export Citation
  • Hillson, D. (2001). Effective strategies for exploiting opportunities. Paper presented at Project Management Institute Annual Seminars & Symposium, Nashville, TN. Newtown Square, PA: Project Management Institute.

    • Search Google Scholar
    • Export Citation
  • Independent Evaluation Office. 2019. IMF Financial Surveillance Evaluation Report, https://ieo.imf.org/~/media/IEO/Files/evaluations/completed/01-15-2019-financial-surveillance/typeset-report/fis-report-glossy-version.ashx

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund. 2008. 2008 Triennial Surveillance Review—Overview Paper, September 2, 2008, https://www.imf.org/external/np/pp/eng/2008/090208a.pdf

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund. 2011 Triennial Surveillance Review, July 28, 2017, https://www.imf.org/external/np/spr/triennial/2011/

  • International Monetary Fund. 2012. Bilateral and Multilateral Surveillance Executive Board Decision, July 30, 2012, https://www.imf.org/en/News/Articles/2015/09/28/04/53/pn1289

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund. 2015. Guidance Note for Surveillance under Article IV Consultation, May 2015, https://www.imf.org/external/np/pp/eng/2015/031915.pdf

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund. 2017. 2014 Triennial Surveillance Review, July 28, 2017, https://www.imf.org/external/np/spr/triennial/2014/index.htm

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund. 2018. “Guidance Note on the Bank-Fund Debt Sustainability Framework for Low Income Countries.” 2018. Washington, D.C.

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund. 2018. 2018 Interim Surveillance Review, April 17, 2018, https://www.imf.org/en/Publications/Policy-Papers/Issues/2018/04/17/pp030718

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund. 2019. “Global Financial Stability Report: Vulnerabilities in a Maturing Credit Cycle.” April 2019. Washington, D.C.

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund. 2019. South Africa: 2019 Article IV Consultation Staff Report, January 30, 2019, (Washington), https://www.imf.org/en/Publications/CR/Issues/2020/01/29/South-Africa-2019-Article-IV-Consultation-Press-Release-and-Staff-Report-and-Statement-by-49003

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund. 2019. “South Sudan 2019 Article IV Consultation Staff Report.” IMF Country Report No. 19/153. Washington, D.C.

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund. 2020. Articles of Agreement, March 2020, https://www.imf.org/external/pubs/ft/aa/index.htm

  • International Monetary Fund. 2020. “Density Aggregation: A Model for Global GDP Growth.” 2020. Washington, D.C.

  • International Monetary Fund. 2021. “How to Assess Country Risk: The Vulnerability Approach using Machine Learning.” Forthcoming.

  • International Monetary Fund. 2021. Nigeria: 2020 Article IV Consultation Staff Report, February 2021, https://www.imf.org/-/media/Files/Publications/CR/2021/English/1NGAEA2021001.ashx

    • Search Google Scholar
    • Export Citation
  • International Monetary Fund. 2021. “Review of The Debt Sustainability Framework For Market Access Countries.” 2021. Washington, D.C.

    • Search Google Scholar
    • Export Citation
  • International Organization for Standardization. 2009. ISO Guide 73:2009 (en), https://www.iso.org/obp/ui/#iso:std:iso:guide:73:ed-1:v1:en

    • Search Google Scholar
    • Export Citation
  • Jeon K., and Z. Kabukcuoglu. 2018. “Income inequality and sovereign default.” Journal of Economic Dynamics and Control, volume 95, October 2018, pages 211-232

    • Search Google Scholar
    • Export Citation
  • Mendoza E. 2010. “Sudden stops, financial crises, and leverage: a Fisherian deflation of Tobin’s Q.” American Economic Review, 100 pp 1941-1966.

    • Search Google Scholar
    • Export Citation
  • Mendoza E. and V.Z. Yue. 2012. “A general equilibrium model of sovereign default and business cycles.” The Quarterly Journal of Economics, Oxford University Press, 127(2), pages 889-946.

    • Search Google Scholar
    • Export Citation
  • Prasad, A., S. Elekdag, P. Jeasakul, R. Lafarguette, A. Alter, A. X. Feng, and C. Wang. 2019. “Growth at Risk: Concept and Application in IMF Country Surveillance.” IMF Working Paper WP/19/36.

    • Search Google Scholar
    • Export Citation
  • Vegh, Carlos A.; G. Vuletin, D. Riera-Crichton, J. P. Medina, D. Friedheim, L. Morano, and L. Venturi. 2018. “From Known Unknowns to Black Swans: How to Manage Risk in Latin America and the Caribbean.” LAC Semiannual Report. October 2018. World Bank. Washington D.C.

    • Search Google Scholar
    • Export Citation
1

Based on ISO 31000 definition (ISO, 2009).

2

They are also known as “Knightian uncertainty” per Knight’s (1921) distinction between quantifiable risk and unquantifiable uncertainty. This paper uses the concept of uncertainty in the context of a knowledge of a true model of the economy (Hansen and Sargent, 2014).

4

See Eichengreen et at. 2008 for a discussion of barriers to the provision of private insurance against sudden stops.

6

In a separate process, an interdepartmental Tail Risk Group of junior economists explores less plausible tail risks and the Early Warning Exercise (EWE) presents low-probability but high-impact risks to a restricted audience of key policymakers. While there are no direct links between the latter two exercises and individual Article IV staff reports, they are indicative of the broader focus on risks placed by the Fund in recent years.

7

The VE is an interdepartmental process to assess country-specific vulnerabilities to crises (factors that make crises more likely when interacted with shocks), combining results generated from a suite of empirical models with desks’ judgement. While the results are confidential, staff is encouraged to use insights from the exercise to inform policies to reduce vulnerabilities. We will discuss potential for greater use of empirical VE models below.

8

The assessment is based on results from the automated text-scraping exercise covering all staff reports published between 2011 and 2018 and the sample of 30 surveyed staff reports including LICs (Afghanistan (2015), Mongolia (2015), Ethiopia (2016), Honduras (2016), Myanmar (2016); Sudan (2016), Cambodia (2017), Moldova (2017), Nicaragua (2017), Nigeria (2017), Somalia (2017)), EMs (Belarus (2015), India (2015), Chile (2016), Indonesia (2016), Lebanon (2016), Namibia (2016), Poland (2016), Algeria (2017), Brazil (2017), Hungary (2017), Indonesia (2017), South Africa (2017)) and AEs (UK (2015), Australia (2016), Canada (2016), Norway (2016), Canada (2017), Sweden (2017)).

9

Based on the feedback on various risk assessment tools from country teams received at the intra-departmental iLab seminar held on March 13, 2019.

10

Alternatively, a qualitative assessment of the balance of risks based on the G-RAM could be used as a prior to inform the WEO predictive densities or quantitative risk assessments based on growth-at-risk models discussed below.

12

Scenarios developed as part of the strategic foresight increased institutional awareness of potential risks from a pandemic and led to the inclusion of Covid-19 as a global—rather than China-specific—risk in the G-RAM before it was recognized as such more widely.

13

The OECD’s Government Foresight Community brings together strategic foresight practitioners in the public sector from around the world. GFC members exchange information and content on the latest foresight developments in government policy making.

14

The Fund already provides extensive technical assistance on specific aspects of risk, including through the Financial Sector Assessment Program (FSAP) and FAD assistance in analyzing and managing fiscal risks (e.g., building integrated fiscal risks analysis frameworks and incorporating probabilistic methods into fiscal projections).

2

Data requirements could be ameliorated by relying more on expert judgment (e.g., as Bank of England does for constructing inflation fan charts, see Britton et al., 1998).

3

Quantile responses are fitted to a more flexible t-skew rather than the two-piece normal distribution as in the case of methodologies based on extracting data from surveys and financial markets data.

4

Prasad et al. (2019) discuss the GaR methodology and applications in surveillance.

5

It has also been used to analyze the behavior of other variables: capital flows (capital-flow-at-risk) and housing (housing-at-risk) (IMF, 2019a).

6

See IMF (2020) for a more detailed description of the methodology.

7

The Flexible System of Global Models is a suite of large-scale semi-structural models allowing for a broad country coverage. The G20MOD module covers G20 members (Andrle et al., 2015).

8

See Adrian et al. (2020) for conceptual framework and Adrian and Vitek (2020) for application to a large-scale model.

9

Financial crisis models use vulnerability indicators compiled in the newly-built ‘Systemic Risk Tracker’, which provides an economy-wide snapshot of systemic risks through easily-generated distribution of indicators grouped by sector and type of underlying risks.

2021 Comprehensive Surveillance Review— Background Paper on The Surveillance Priority Confronting Risks and Uncertainties
Author: International Monetary Fund. Strategy, Policy, & Review Department
  • View in gallery

    Recurring Volatility and Crises

  • View in gallery

    CSR Survey: Confronting Risks and Uncertainties is Key Priority

    (Responses in percent)

  • View in gallery

    Balance of Risk Assessment and Forecast Errors in Staff Reports

  • View in gallery

    Example of the macro-framework in the new Financial Programming Environment

  • View in gallery

    Representations of Predictive Densities for Risk Assessment

  • View in gallery

    Predictive Densities for Policy Analysis

    (GDP Growth Projected from DSGE Model of Financial Crisis)