2008 Triennial Surveillance Review - Overview Paper

This triennial surveillance review (TSR) takes place in a time of change, with the Fund in the midst of a major refocusing effort. That surveillance has not been as effective as it should be has been amply documented, notably in previous surveillance reviews and several IEO evaluations. Since the 2004 review of bilateral surveillance (BSR), considerable efforts have been made to enhance its effectiveness, including an overhaul of the policy framework with the adoption of a new Surveillance Decision in 2007. And as part of the refocusing effort, more change is underway to deliver on the Managing Director’s vision of an institution making better use of its comparative advantage, to be “more alert to emerging issues, more critical in its assessments (especially in good times), and more assertive in communicating its concerns.”

Abstract

This triennial surveillance review (TSR) takes place in a time of change, with the Fund in the midst of a major refocusing effort. That surveillance has not been as effective as it should be has been amply documented, notably in previous surveillance reviews and several IEO evaluations. Since the 2004 review of bilateral surveillance (BSR), considerable efforts have been made to enhance its effectiveness, including an overhaul of the policy framework with the adoption of a new Surveillance Decision in 2007. And as part of the refocusing effort, more change is underway to deliver on the Managing Director’s vision of an institution making better use of its comparative advantage, to be “more alert to emerging issues, more critical in its assessments (especially in good times), and more assertive in communicating its concerns.”

I. Introduction

1. This triennial surveillance review (TSR) takes place in a time of change, with the Fund in the midst of a major refocusing effort. That surveillance has not been as effective as it should be has been amply documented, notably in previous surveillance reviews and several IEO evaluations.1 Since the 2004 review of bilateral surveillance (BSR), considerable efforts have been made to enhance its effectiveness, including an overhaul of the policy framework with the adoption of a new Surveillance Decision in 2007. And as part of the refocusing effort, more change is underway to deliver on the Managing Director’s vision of an institution making better use of its comparative advantage, to be “more alert to emerging issues, more critical in its assessments (especially in good times), and more assertive in communicating its concerns.”

2. The TSR’s focus reflects this changing environment.2 The review takes stock of experience under the new surveillance framework and seeks to gauge the distance to meeting key stakeholders’ expectations. It focuses on bilateral surveillance in the recent past and, particularly, on the appropriateness of its focus, its analytical value-added—with an emphasis on exchange rates, financial sector, and spillover analyses, its degree of candor and evenhandedness, and the effectiveness of its communication. The findings should inform the definition of key medium-term operational priorities, as reflected in a draft Statement of Surveillance Priorities (SSP) (Annex).

3. This TSR focuses on the substance of surveillance rather than its process, and its methodology seeks to examine depth of analysis rather than just breadth of coverage. The review was informed by diverse sources of information and techniques of analysis, with findings crosschecked against each other. These included (i) a review of a random representative sample of 50 staff reports for consultations conducted since mid-2007; (ii) wide consultations of stakeholders conducted through anonymous surveys (Executive Directors, country authorities, mission chiefs, market participants) and through interviews by independent consultants (country authorities) and staff (journalists);3 and (iii) in-depth topical case studies, including one carried out by an independent consultant.4 This TSR also lays out a full framework of questions and techniques to ensure continuity with future TSRs.5

4. The structure of the TSR reports is as follows. This overview paper first examines the value-added of bilateral surveillance, and progress made since the 2004 BSR. It then focuses on areas where stakeholders’ expectations consistently call for progress. Three background papers detail the thematic findings and recommendations, and provide the main raw data underpinning the assessment.6

II. The Strengths of Bilateral Surveillance

A. Bilateral Surveillance Adds Value in Multiple Ways and Areas

5. The overall quality of Fund surveillance is held in high regard by outside stakeholders. Almost all country officials interviewed by the independent consultants noted that Fund surveillance added significant value. Authorities value the Fund as the provider of an integrated macroeconomic assessment placed in a global perspective, and a source of policy advice against which their own judgment can be tested and enriched through dialogue. The consultation of media professionals also revealed that they regard surveillance highly for its credibility, independence, and quality of analysis. Financial market participant survey respondents rate the quality of Fund analysis above competitors’ in their field and most use surveillance products to at least some extent to inform their own work. Furthermore, the review of staff reports sections on follow up to past surveillance advice revealed a broadly positive record (Box 1).

6. Executive Directors, and financial markets, have a high appreciation of the value-added of surveillance, even more so than country authorities. This finding is consistent across areas (Figure 1). In one way, it is not surprising, as country authorities know and think a great deal about their own country’s issues. At the same time, just as surveillance brings in-depth information on individual countries to the international community (the Board, financial markets), it might be expected to bring to country officials a multilateral perspective that is not easy to obtain otherwise-a point further discussed in Section III.

Assessing Follow-up to Past Advice

The review of staff reports’ sections on follow up to past surveillance advice suggests a broadly positive record.

  • In most cases, countries’ actions were “mostly” or “partly” consistent with past Fund advice. In only about 10 percent cases was Fund advice found mostly not followed.

  • Among the cases where countries’ actions were consistent with Fund advice, a quarter each related to fiscal policy and structural reform in core areas, and 15 percent to monetary policy. Only 5 percent related to exchange rate policies.

  • When reasons for not following Fund advice were identified, in about half of the cases countries broadly agreed with the advice but wanted to proceed at a slower pace. In a quarter of cases, they also agreed, but needed technical assistance. In other cases, countries did not agree with the advice.

These findings are suggestive but insufficient to draw conclusions about the impact of surveillance, because they do not say anything about whether the authorities’ actions were influenced by surveillance (and because there is some risk of selective reporting when key areas of advice are not identified ex ante). In future TSRs, staff will be able to draw on three-year surveillance agendas to assess actual outcomes against those intended ex ante. Together with information from stakeholder surveys, this should help form a better substantiated view on the traction of surveillance advice.

Given this and the broader refocusing context, it is proposed to discontinue the requirement to report in each Article IV on follow-up to staff past advice. However, while this systematic reporting did not prove useful as a tool to assess the effectiveness of surveillance overall, the information provided may be important in some cases for understanding policy developments. It is therefore proposed to leave it to staffs judgment to provide such background information as they see fit.1

1 This staff proposal was not endorsed by the Board as Executive Directors considered that reports on follow-up to past Fund advice provide useful information to assess Fund advice over time.

7. Fund surveillance is seen as adding the greatest value in, and on, developing economies, next in emerging markets, followed by advanced economies (Figure 2).7 This is not surprising: advanced economies and, to some extent emerging markets, tend to be subject to more vivid economic debate and scrutiny, including by domestic institutions. In developing economies, the Fund is often one of the few —if not the sole—sources of independent and integrated macro-analysis and advice.

8. Fiscal policy remains a top area of value-added of surveillance.8 As noted in the case study of surveillance in Europe, this owes to what is seen as the Fund’s exceptional skill at placing fiscal policy in an integrated macroeconomic framework. Moreover, longer-term fiscal sustainability exercises are viewed as helpfully contributing to the transparency of the public policy debate, particularly when they take account of hidden fiscal costs in the broader public sector. The review of staff reports conducted for this TSR also showed that a large majority of staff reports clearly articulated short-term and medium-term fiscal policy recommendations, putting short-term measures on clear medium-term road map. The Debt Sustainability Framework has proved a very valuable tool in this regard.

Figure 1.
Figure 1.

The Value Added is Perceived Differently by the Various Stakeholders, and for Different Areas

Citation: Policy Papers 2008, 005; 10.5089/9781498334266.007.A001

Source: TSR Stakeholders’ survey of Authorities, Executive Directors, and Financial Market Participants.1/ See Statistical Appendix for detailed results.

9. The analysis of recent economic and policy developments and short-term outlook provided by Fund surveillance is also highly valued.9 The staff report review showed the generally very good quality of staffs work in these two areas, which are the ones of greatest value-added for Executive Directors. Likewise, for financial market participants, one of the main areas of value-added of surveillance reports comes from the overall health check of the economies.

Figure 2.
Figure 2.

Areas of IMF Staff Report Significantly Valuable to Stakeholders

Citation: Policy Papers 2008, 005; 10.5089/9781498334266.007.A001

Source: TSR Stakeholders’ survey of Authorities, Executive Directors, and Financial Market Participants.1/ See Statistical Appendix for detailed results.

B. Significant Progress on the 2004 BSR Priority Objectives

10. The value-added of surveillance also reflects improvement in the priority areas identified by the 2004 BSR. The 2004 BSR set as monitorable objectives for the next surveillance review “sharpening the focus of Article IV consultations, and, within this, ensuring a deeper treatment of exchange rate issues; enhancing financial sector surveillance (FSS); and deepening the coverage of regional and global spillovers in bilateral surveillance.”10 The Fund made significant efforts to meet these priorities, including through a number of initiatives launched under the 2006 Medium-Term Strategy and more recently under the refocusing exercise. To a large extent, these efforts have paid off.

11. The focus of surveillance on the Fund’ s core mandate has sharpened significantly overall.11 Whereas the 2004 BSR had identified excessive breadth of coverage as an issue, the review of staff reports showed clearly that surveillance now focuses primarily on issues that have a significant impact on domestic or external stability. Moreover, the share of Selected Issues Papers (SIPs) devoted to issues that the Surveillance Decision identifies as always relevant—exchange rate, monetary, fiscal, and financial sector policies (both their macroeconomic aspects and macro-economically relevant structural aspects)—went up by 9 percentage points between the second half of 2006 and the first half of 2008 to 86 percent. Close to 90 percent of country officials consulted perceived surveillance as rightly focused, while almost all mission chiefs were confident, with the benefit of hindsight, that recent consultations focused on the most relevant issues. However, Executive Directors were fairly critical of the focus of surveillance in advanced economies, with about half of respondents thinking that only some reports for these countries focused on the right issues and noting no improvement in recent years. In addition, as discussed below, more effort is needed to deliver focused key messages effectively.

12. The coverage and quality of analysis of exchange rate issues has also improved.12 Evidence of greater coverage is clear. Whereas the IEO found that less than two thirds of a sample of 2006 staff reports provided an assessment of the exchange rate, in this TSR’s sample, more than 90 percent of the reports did. Moreover, the share of SIPs dealing with exchange rate and competitiveness issues doubled from 2006 to 2008, to 20 percent. The sophistication of the analysis has also increased. While the IEO found that reports rarely used analytical tools beyond a real exchange rate chart, this TSR finds that more than half the reports in the sample provide a robust assessment of the exchange rate level, based on a multitude of techniques, such as basic indicators, PPP-approaches, and econometric techniques, including those developed by the Consultative Group on Exchange Rate issues (CGER). The improvement is particularly marked in the more recent period.

13. Progress on the treatment of spillovers was uneven but, together with more cross-country analysis, enhanced the multilateral perspective in bilateral surveillance.13 Over half of mission chiefs noted that they have paid more attention to analyzing inward spillovers, and more staff reports refer to the influence of at least one global factor, often backed up by a selected issues paper. However, discussions of outward spillovers for systemic countries remain few and far between. Cross-country analysis, by contrast, now features in the vast majority of staff reports.14 For Executive Directors, it is among the three highest areas of value-added from surveillance.

14. Financial sector surveillance is receiving increasing attention in Article IV consultations, leading to substantial progress in identifying financial sector vulnerabilities.15 Most mission chiefs indicated that they have paid more attention to financial sector issues in recent consultations and devoted more resources to them, including through mission participation of MCM specialists, a trend broadly validated by budgetary information. The review of staff reports confirmed the improved coverage and depth of analysis of financial sector issues, in particular regarding the identification of vulnerabilities: 80 percent of reports analyzed financial stability indicators and close to 60 percent—and 75 percent for advanced economies—provided a clear diagnostic of financial sector vulnerabilities. Significant progress was also found in integrating into Article IV reports the findings of FSAPs, which are seen as systematically adding value. Stakeholders confirmed the improvement: Two thirds of Executive Directors and three-quarters of country officials noted a better coverage of financial sector issues and most country officials stated that surveillance generally contributed to improve their understanding of financial sector issues. In addition, analysis of financial sector risks and vulnerabilities was the area where the highest share of financial market participants ranked the Fund’s work much higher on average than other institutions’.

III. Where To Focus Efforts Next

15. More is needed to reach the frontier targeted by the Fund’s refocusing exercise, with fewer resources. Despite progress made since the 2004 BSR, Fund surveillance overall does not fully meet expectations. In some areas, bridging the shortfall may simply require better aligning processes and resources and “just doing it,” because the expertise, the tools and, the data are already there. So are the good practices, which just need to be mainstreamed. In other areas, making progress will require extending the analytical frontier —another type of challenge. And in other cases, like financial sector surveillance, the goalposts are constantly moving: as the world we analyze is becoming more complex, so is its analysis. In these cases, future surveillance reviews can be expected to note both that progress was made and that more efforts are needed. Resource constraints, which have become tighter, pose an additional challenge to making further progress while preserving existing strengths. Better leverage of multilateral and cross-country work and sharper focus on key issues will help, but there will inevitably be, at times, trade-offs in what issues can be covered in greater depth.

A. The Highest Priority Areas

16. Four areas stand out as deserving priority attention in the period ahead. These are areas where the value-added is currently considered the weakest by stakeholders, despite their undisputed centrality to the Fund’s surveillance mandate and effectiveness. They are thus proposed for inclusion as operational priorities in the SSP.

Risk assessment16

17. Surveillance needs to pay better attention to risks surrounding the baseline. Only about 60 percent of staff reports presented the short-term economic outlook, including risks to this outlook, in a substantiated way. Weaknesses found in remaining reports relate to lack of clarity in describing risks to the short-term outlook, or an unconvincing discussion of them. While close to three-quarters of reports offer a well-articulated, medium-term scenario, there were still fewer thorough discussions of risks surrounding this baseline: only about half of reports whose baseline MT/LT scenario was identifiable (36 percent of total) were considered to include a satisfactory discussion of baseline risks. Instead, about a third of reports suffered from some combination of not seeing beyond the most obvious risks, missing crucial external risks, providing a long list of risks without explaining their relative importance and, at times, excessive hedging about the possible implications of the risks which blunted the message. And 8 percent of all reports did not discuss risks to the outlook at all. Furthermore, only about a third of Executive Directors thought that most staff reports took as baseline a medium-term and long-term outlook that was neither too optimistic, nor pessimistic.

18. The subprime crisis case study (Box 2) further illustrated how good economic performance can lead to relaxed vigilance about vulnerabilities and risks, including low-probability but high-cost ones. Among the reasons why the scale and impact of the crisis were not foreseen, it noted, in addition to “known unknowns” and wrong calls, a failure to “connect the dots” between different risks and vulnerabilities and to articulate the tail risks that eventually materialized. Surveillance thereby failed to anticipate —and provide policy recommendations to guard against— the scale and ultimate impact of the crisis.

19. A more systematic exploration of risks and possible outcomes is needed and could be achieved by:

  • Pressing for more systematic assessments and effective presentation of risks around the baseline and presenting them effectively. Greater consistency is needed from staff in examining the magnitude of, and policy response to, risks. As part of this effort, the selective use of “fan charts” to describe risks surrounding the baseline could prove a useful device and should be encouraged.

  • Emphasizing more tail events with potentially seriously damaging implications, even if unquantifiable. When uncertainty dominates and risks cannot be measured, it can be difficult to capture attention in policy discussions. Yet, precisely because extreme events with large costs are often ignored in the domestic debate, surveillance can help focus policy-makers’ attention on the need to guard against worst outcome scenarios. In general, staff should ask more “what if” questions on country-specific events in the bilateral surveillance dialogue and policy recommendations should take account the need to insure against low-probability but high-cost events.

  • Leveraging better in Article IV consultations the analysis of risks conducted at the multilateral or regional levels. The discussion of inward spillovers should not only focus on how the most probable scenario for the global environment or the main economic partners is likely to affect the domestic economy, but also on how risks identified at these levels could be transmitted to the domestic economy.

20. These recommendations do not mean that surveillance should explore in depth all possible universes for all countries at all times. Repeatedly raising false alarms and overloading Fund reports with concerns that do not materialize could hurt the Fund’s credibility. While “what if” questions should become more routine practice, careful prioritization will be needed to determine which risks to analyze more closely. The subprime crisis shows that history alone is not always a wise guide. Inevitably, sometimes these choices will be wrong, but accepting this ex ante is necessary to avoid continued incentives to excessively hedge.

Integration of macroeconomic and financial sector surveillance17

21. The principles laid out by the 2007 Financial Sector Task Force provide a gauge of the quality of financial sector surveillance: Staff reports should help form a view on the channels of interaction between the real and financial sectors, the role of the financial sector in potentially disturbing the real economy, and—where relevant for external stability—its role in facilitating growth. They should do so notably through a full exploitation of information drawn from financial markets.

22. The analysis of transmission channels from financial to real sectors is a key area for improvement. Only about a quarter of the reports reviewed for this TSR clearly identified key risk transmission channels between the financial sector and real economy, and assessed whether the financial sector risks may materially contribute to real domestic or external instability. This finding is consistent with Executive Directors’ fairly critical opinion in this regard, and with the view expressed by mission chiefs, who considered analysis of potential macro- or external stability implications of financial sector developments to be the most challenging aspect of financial sector surveillance. It is also consistent with the findings of the subprime crisis case study (Box 2).18

Case Study: Bilateral Surveillance in the Run-up to the Subprime Crisis

Bilateral surveillance was successful in identifying most factors, vulnerabilities, and individual risks that subsequently developed into the “subprime crisis,” but the scale and impact of the crisis were not foreseen by the Fund or most observers. The case study assessed surveillance in four countries (U.S., U.K., Switzerland, and Germany) up to August 2007, at the outbreak of the crisis. Chapter VII of Supplement 1 presents the findings.

What was seen

  • Vulnerabilities and risks were recognized and individually investigated (e.g., with respect to the housing market, securitization, increased financial system leverage, lack of transparency on ultimate risk exposures), and potential spillover channels identified (e.g., the importance of outward financial spillovers from the U.S.). A recent FSAP (in Switzerland) and MCM participation (in all cases) provided invaluable support in identifying emerging risks.

  • The assessment was facilitated by comprehensive documentation of underlying macroeconomic conditions (low global interest rates, housing boom in many advanced economies) and financial sector trends (e.g., securitization and financial innovation). Also, many institutional aspects had been correctly diagnosed, such as regulatory issues and prudential issues (government sponsored enterprises including Fannie Mae and Freddie Mac, regulatory standards in the U.S.; liquidity risks and capital adequacy thresholds in Switzerland; risk-based supervision in Germany), crisis management framework (U.K.), and structural reform of the banking system (Germany). FSAPs provided comprehensive assessments of institutional frameworks, though keeping such assessments up to date is a challenge in an environment of rapid financial innovation.

What might have been done better

  • Surveillance made wrong calls with respect to the U.S. housing bubble and the growing fragility of the financial system, and did not recognize the aggregate implications of the individually identified risks. In addition, the impact of such risks materializing was underestimated, or not assessed quantitatively. Past good performance, including the resilience and sophistication of the U.S. financial system, seem to have weighed in reaching an overall favorable assessment.

  • Some institutional weaknesses might have been uncovered in the context of FSAPs (e.g., weaknesses in regulatory frameworks in the U.S., which had no FSAP.)

  • Policy advice was often given to mitigate the risks that were flagged, but consequences of certain risks materializing were not always drawn, impeding calls for policy action.

  • Surveillance tended to obscure the degree of concern related to risks through reassuring headline messages-here, bilateral and multilateral surveillance needed to be well-integrated.

What was not seen and why

  • Lack of information on exposures and the complexity of securities themselves were “known unknowns” that added to the overall uncertainties and generated the large spillover effects.

  • Some features of the crisis were “unthinkables,” particularly the interbank money market freeze.

What can be done better in future?

Overall, justified concern that the full scale and impact of the crisis were not foreseen should not obscure the progress made in monitoring and analyzing financial systems. Nevertheless:

  • Surveillance should strive to “connect the dots” better, think the unthinkable, and highlight known unknowns more forcefully. Progress will likely be incremental, as it is linked to strengthening analysis of macro-financial linkages, and financial sector spillovers.

  • Difficult prioritization challenges lie ahead, particularly with respect to keeping institutional framework assessments up to date, and selecting the appropriate tail risks on which to focus analytical efforts, as both are very resource intensive and can only be done selectively.

  • The Fund needs to learn to be bolder in communicating its concerns. Though the balance to be struck is always delicate, it should be possible for surveillance to take clear and defensible positions, instead of fully hedged and therefore less informative assessments.

23. Despite progress, there is also room to further strengthen the diagnosis of and advice on financial sector issues, including through greater specificity. The review of staff reports found that 40 percent still do not allow readers to form a clear view on the degree of financial sector vulnerabilities. This is usually because of a lack of quantitative evidence to substantiate assessments, or because an overly hedged language blurs staff’s messages. In addition, not all reports that diagnose financial sector vulnerabilities or development issues offer recommendations to address them. Moreover, in about a quarter of reports, financial sector policy recommendations were found to be either not clearly justified or too generic. In some cases, this may reflect lack of diagnostic evidence or analysis. In other cases, mission teams may not have sufficient specialist expertise or experience in the relevant policy area to go beyond rather general statements.

24. Many examples of good practices on all these fronts can be found, but there are impediments to their mainstreaming. As is clear from the background studies, these challenges have been handled successfully in a number of country cases. But mainstream practice is lagging behind, reflecting potent impediments. The three main difficulties cited by mission chiefs as hampering financial sector surveillance are: (i) the lack of a clear analytical framework for macro-financial linkages, (ii) the lack of clarity on what is expected in terms of integrating financial sector issues into macroeconomic analysis, and (iii) limitations in data or access to information. In addition, good examples of financial sector surveillance, including priority countries identified to follow up on the Task Force recommendation, indicate that availability of special resources or expertise is key. As confirmed by country authorities, these cases show that major advances can be achieved when MCM staff or other experts participate in missions or work for a period in area departments.

25. Three key recommendations would promote further progress:

  • A clearer organizing framework for more rigorous and systematic macro-financial surveillance should be developed. Such a framework should help systematically analyze financial sector issues and their implications in Article IV consultations, motivate policy advice, and leverage analyses from FSAPs and technical assistance. It should also focus attention on the need for judgment about the relative likelihood and impact of key financial stability risks, on vulnerabilities and development issues, and on the integration of financial sector issues into the macroeconomic analysis, including through risk-based scenarios and the analysis of tail event risks. And it would help guide further research in these areas. Separately, examples of best practice should be disseminated more widely.

  • The Fund’s FSS toolkit needs to be continuously refreshed, and quantitative modeling and other methodological development work needs to continue apace. The most critical areas are financial stability assessments (including the development of advanced stress testing and risk modeling approaches), macro-financial linkages, and cross-border spillovers. Improving mechanisms for early warning of risks to global financial stability (which is already part of the staff’s work program) and analysis of financial channels of risk transmission at the regional and global levels should be high priorities.

  • Macro-financial expertise must be further built and used strategically. The Fund demand for financial expertise exceeds current supply, while the institution’s tighter budget leaves little room for maneuver to solve this problem. The response should be twofold. First, financial capacity should be developed through new training programs and HR policies facilitating training assignments, swaps and the recruitment of financial sector specialists. Second, use of existing resources should follow a risk-based approach. Expertise should be deployed in priority according to the criteria of systemic/regional importance, importance of vulnerabilities, and importance of financial development issues for present or prospective macroeconomic or external stability.

Multilateral perspective19

26. Stakeholders expect more treatment of inward, and even more so outward, cross country spillovers. Though at the technical level such factors are well-integrated into macro projections, including through the WEO processes, staff reports on average pay too little attention to placing explicitly the country in the global context, failing stakeholders’ expectations in this regard. Only a minority of Executive Directors felt that many or most staff reports met their expectation regarding the analysis of inward spillovers, and even fewer felt so for outward spillovers. A majority of country officials noted that staffs analysis of inward spillovers did not improve their understanding of the issues or bring new insights, suggesting that surveillance did not deliver on its potential to help the authorities understand better their country in the global context. Market participants considered that this was among the areas where the Fund’s contribution was the weakest, although they also felt staff analysis was on average better than found elsewhere. The review of staff reports backs these perceptions. While most reports now refer to some external factors (e.g., commodity prices), only about a third explicitly discuss how global trends impact their countries. Furthermore, the coverage of outward spillovers in reports for systemic countries remains spotty, possibly reflecting in part the greater difficulty for country teams to mobilize sufficient knowledge about countries on the receiving end of the spillovers.

27. Nonetheless, a number of good practice examples show the way forward. Many SIP chapters are devoted to cross-border spillover issues; a few cases illustrate good practice of integrating risks identified in multilateral surveillance into bilateral surveillance. For example, the reports on Chile and Uruguay quantified the potential impact on the banking system of the risks of abrupt reversal in capital inflows and slowdown in global and U.S. growths that were identified in the 2007 issues of the WEO and GFSR. Similarly, the reports for Colombia and Mexico quantified the impact of those risks on domestic growth. Furthermore, some regional studies, such as the Nordic-Baltic regional financial sector study, proved very useful vehicles to explore economic and financial spillovers within regions and inform bilateral surveillance in more than one country. The report of the external consultants also highlights the potential contribution of regional surveillance, while cautioning against the risk that more regional work crowd out bilateral surveillance.

28. There appears to be a large unmet demand for cross-country analysis informing the policy dialogue. As noted above, much cross-country analysis gets done, but it fails to fulfill key stakeholders’ expectations: only about a third of authorities found cross-country analysis in surveillance to provide new insights to at least “a large extent.” Several factors seem to be at play. First, of the three types of cross-country analysis practiced—narratives drawing lessons from experience in other countries that faced similar issues; comparisons of indicators for a peer group; and quantitative studies—the first is the least frequent, whereas it is highly valued by country officials. Second, in over half the cases, the link between the analysis and the policy recommendations is not clear enough, resulting in recommendations too vague or with little operational content. Finally, the review also found that there is a large untapped potential in the analytical chapters of the WEO and GFSR, in that few Article IV reports draw on the relevant ones in the bilateral policy dialogue. To be persuasive, crosscountry analysis needs to (i) draw clear policy conclusions well-grounded in the analysis; (ii) be based on appropriate comparator(s); and (iii) be sufficiently deep. Ensuring that crosscountry analysis presented meets all three criteria should help better meet expectations and potentially achieve stronger impact.

29. The following steps could help strengthen further the multilateral perspective in bilateral surveillance:

  • Better leveraging the findings of the WEO and GFSR by drawing out the policy implications in bilateral surveillance. The surveillance committee can play an active role to foster progress in cases where risks are judged to be systemic. Regional surveillance can also contribute to bridge the gap, and cross-country work should more often be explicitly designed to inform bilateral surveillance in more than one country.

  • Improving access to and sharing of cross-country knowledge. Functional departments have a key role to play in this effort. In particular, the review process should help point to relevant experiences, as well as good examples of cross-country and spillover analyses.

  • Changing the incentive structure by rewarding cross-country work on the basis of the value-added to the policy dialogue, rather than the degree of sophistication of the techniques used.

Exchange rate assessments20

30. Despite the progress discussed above, exchange rate analysis still needs to be better integrated into the overall macroeconomic assessment and presented more transparently, in some cases backed by further methodological work. The exchange rate case study revealed that despite progress, staffs exchange rate assessments were not always well integrated into the overall macroeconomic assessment. For instance, staff often discussed the need to improve competitiveness in face of large current account deficits, while considering the real exchange rate broadly in line with fundamentals, without further discussion of this possible inconsistency. For surplus countries, the link between potential undervaluation and inflationary pressures was not always adequately discussed. The consultation of country authorities and Executive Directors revealed widespread dissatisfaction with the Fund’s exchange rate analysis, including wariness of the quantitative techniques used by staff. Echoing these views, the review of staff reports also found that too often the assessments are presented as a “black box,” without adequate documentation of underlying assumptions made by staff. Mission chiefs expressed frustration about the lack of adequate analytical tools and guidance. Indeed, difficult methodological challenges remain, such as the global consistency of the assessments undertaken by country teams or the analysis of exchange rates in certain types of economies (e.g., non renewable commodity exporters, large financial centers, and countries with large and sustained capital inflows).

31. Continued attention to candor and evenhandedness is needed in the treatment of exchange rate issues. The review of staff reports and the case study on exchange rate assessments suggested that clarity on exchange rate issues increased on average with the new Surveillance Decision. Most pre-Decision reports did not include a clear bottom line assessment of the exchange rate level, while most post-Decision reports did. However, the review also found that, since the adoption of the 2007 Decision, a very large proportion of exchange rates were assessed “broadly in line with fundamentals” despite large current account imbalances. It also found a tendency to explain away large imbalances through recourse to temporary factors, suggesting a possible bias in staffs assessments. While these, to some extent, may result from methodological issues, they may also reflect “fear of labeling”—as introduced by the 2007 Decision—for the most worrisome cases. The mission chief survey and the long delays in completing a number of Article IV consultations where possible “fundamental misalignment” was being discussed lend credibility to this concern. Finally, the review also noted that in 15 of the 22 cases of post-Decision consultations where staff expressed that the exchange rate was over- or undervalued, the corresponding summings up were less clear-cut in their assessment.

32. Given the steps recently taken to remedy these emerging issues, further progress is within reach without need for new initiatives, but continued efforts are needed. The recently issued guidance on operational aspects of the new surveillance Decision should help strengthen the integration of exchange rate assessments into overall macroeconomic assessments, foster more transparency on the assumptions and techniques used to conduct quantitative assessments, and help achieve more consistent candor.21 This would allow for a greater scrutiny of the assessments and should encourage greater attention to robustness. Furthermore, efforts to disseminate methodological guidance on basic assessment techniques, critical datasets of reference, and best practices should gradually enhance the quality and consistency of exchange rate assessments. As concerns remain about uncertainties in exchange rate analyses using current, state-of-the-art, techniques, more research is underway. For these initiatives to bear fruit, it is important to keep incentives properly aligned, and therefore exchange rate assessments among the monitorable surveillance priorities, though broadened to encompass also assessments of external stability risks.

B. Other Key Areas For Improvement

Communicating effectively22

33. The Fund’s surveillance communication has many assets to build on:

  • The policy dialogue is highly valued. Many country officials interviewed by the external consultants noted that the oral communication added the most value. Both mission chiefs and country authorities expressed overall large satisfaction with the degree of candor of their policy dialogue.

  • Staff reports also appear to present clear and candid assessments overall, albeit not uniformly and to a lesser extent than the policy dialogue. Across different audiences and surveys, Fund communications were considered credible and newsworthy. In spite of the technicality of the topics, the review of reports judged that the main conclusions provided in the executive summary and staff appraisal would be comprehensible to the average college-educated adult in 90 percent of cases. And while the clarity of bottom line assessments was found lacking in a fifth of reports, authorities are almost universally satisfied with the current degree of candor of reports on their own countries. However, Executive Directors are generally more circumspect when assessing candor in reports outside their constituency.

34. But the proliferation of vehicles —mission concluding statements, PINs, staff reports, SIPs—is generally viewed as confusing and counterproductive, although each has an audience. Different audiences have different needs, and therefore different preferences across the various vehicles. Many (e.g., authorities, journalists) find the concluding statement highly effective, particularly compared to the other “ short” vehicle, the summing up/PIN, found inscrutable by most (including Executive Directors). However, there is also an audience for the longer, more thorough vehicles that are the staff reports and SIPs (including authorities and financial market participants). But few outside the Fund understand the difference between these various vehicles or what they add to one another.

35. Improvements in brevity, timeliness, and clarity of key messages would help capture the attention of a broader audience. Most audiences indicated that shorter reports would increase readership and hence impact, though Executive Directors recently reaffirmed that brevity should not be pursued at the expense of analytical persuasiveness. Insufficient timeliness is also an issue for some audiences (primarily authorities and media), and Executive Directors indicated that they welcomed efforts to narrow the gap between the end of the mission and issuance of the staff report. Finally, the clarity and persuasiveness of surveillance messages could be improved. In the review of staff reports, 60 percent of them were judged to lack a clear, small set of priority messages in the relevant country. The frequent use of jargon or overly long sentences was found to hinder communication quality in about one-third of the reviewed staff reports. And only half of the reports (54 percent) were assessed as good or very good in terms of overall communication value.23 Consistently, several officials told the external consultants that the language of staff reports sometimes had to be “revamped” in order to extract salient messages for senior policymakers in a form that would support impact on the policy debate. While Directors were broadly satisfied with the current presentation of staff reports, only 43 percent thought most reports were well-argued and convincing.

36. Changes are already underway to enhance the effectiveness of the written communication of surveillance, but a strategic approach is needed. Significant efforts are underway to make surveillance outputs—in particular staff reports—more accessible to a wider audience, persuasive, concise, and timely. But country-specific strategies developed by country teams—as presented in surveillance agendas—typically focus exclusively on process (e.g., “encourage the authorities to publish reports” or “(staff to) hold press conferences or seminars”), without specificity about messages or audiences.

37. The effectiveness of surveillance communication would be further increased by adopting a more purposeful approach. In particular, this could include:*

  • Encouraging greater candor in staff reporting to the Executive Board, with appropriate safeguards, to ensure candor in the policy dialogue between the Fund and members.

  • Rationalizing and modernizing the key communications vehicles. The publication of end-of-mission statements should be further encouraged and used as a prelude to the publication of post-Board products. Better use should be made of the summary information provided by the executive summary of staff reports and the summing up. An effective and simple step would be to give countries the option of releasing the staff report’s executive summary and the summing up (together) on a country web page, to be augmented with the rest of the staff report when countries consent to its publication. This would eliminate the need for PINs, and clarify the relationship between the different elements potentially released to the public. Separately, summings up should be written in a more reader-friendly way.

  • Tailoring surveillance communication to emphasize a few key messages and match messages and medium to the target audience.

  • Emphasizing context over volume. The instruments that would make the greatest contribution in this area, and could be used selectively, are: easier access by external audience to Fund “experts;” more background briefings focused on key topics; and more use of tailored instruments.

38. Unavoidable trade-offs to such more forceful communication should be acknowledged. Striving for conciseness is admittedly a challenge when simultaneously seeking greater coverage in key areas. This puts more than ever a premium on effective drafting and skillful presentation. Furthermore, focusing on a few clear key messages bears a higher risk of getting things wrong than producing overly hedged, catch-all reports. Acknowledging this is important for setting the right incentives upfront.

Preserving existing strengths

39. Finally, vigilance is required not to backtrack on previously well-established or recently-achieved areas of strength. There is a danger that the greater emphasis on themes identified as deserving greater priority be achieved at the expense of the quality of analysis of other core areas. Some findings in these areas raise red flags. That monetary policy discussions sometimes do not seem to add much value—it is seen as very high for about 40 percent of members (mostly LICs), but limited for another 30 percent (mostly advanced countries)—suggests a need for strengthening, especially in the current environment of growing inflationary pressures. Similarly, the review of staff reports found that short-term fiscal stance recommendations are not always backed by sufficient rationale. Furthermore, the review of recent surveillance agendas also indicated that gains in sharpening the focus of surveillance on core mandate topics may not be firmly entrenched. It is also possible that some of the recent gains (e.g., more attention paid to financial sector issues) in part reflected changes in the economic environment (e.g., heightened financial stress), and thus may be more prone to reversal. Thus continued emphasis on these areas in the review process and periodic monitoring of performance is warranted.

A similar quality of surveillance across countries

40. Some quality dimensions of surveillance reports appear to be affected by factors that should not be relevant. The review of staff reports found that factors such as income level, size, geographic location and exchange rate regime have some influence on rough measures of the sharpness of the focus, the quality of analysis in key areas (financial sector, health check, multilateral perspective), and the candor of surveillance. In general, reports were more likely to meet good practice for: advanced economies; countries with flexible exchange rates; big and very small economies; and countries in Asia-Pacific, Europe, and the Western Hemisphere.24 In many cases, the differences are statistically significant. This is clearly not optimal, even though it is to be expected that more resources be devoted to countries with systemic importance. However, surveillance generally appears of similar quality within each group, as well as between countries with or without a Fund-supported program. In many cases, the differences across groups reflect exogenous constraints, such as the availability of data and analytical material prepared by third parties. Furthermore, the quality of assessment of very small (mostly poor) countries seems at par with the largest (mostly rich) countries.

41. These findings, while not alarming, suggest a need for continued attention. The review process in particular has a key role to play in minimizing differences of treatment not called for by country circumstances relevant to the issues examined. Continued monitoring might also help create positive incentives.

IV. Issues For Discussion

42. Do Directors agree that the evidence collected for this TSR suggests that surveillance adds significant value to members and the public at large, and that significant —albeit uneven—progress has been made toward the monitorable objectives set in the 2004 BSR?

43. Do Directors consider that the discussion in section III.A appropriately identifies key priority areas where further progress would significantly increase the effectiveness of surveillance and section III.B other important areas for improvement? Do Directors agree with the recommendations highlighted in these sections and in Box 1, as elaborated in Supplement 1? What are Directors’ views on the trade-offs flagged in these sections? Do Directors have additional suggestions to further enhance the effectiveness of surveillance?

44. Do Directors consider that the methodology used to conduct this TSR and laid out in detail in Supplement 2 establishes a consistent and robust framework for assessing the effectiveness of surveillance, to be used (subject to continued improvements) in future TSRs?

Annex: A Statement of Surveillance Priorities

45. This Annex presents a proposal for the IMF’s first Statement of Surveillance Priorities (SSP). Attached are: (i) a draft SSP (which would be published on a stand-alone basis once finalized); and (ii) a draft box providing more detail on economic priorities, that would provide further guidance to staff in implementing them. The priorities highlighted in the SSP are intended to guide all surveillance work.

46. The draft SSP draws on recent discussions on multilateral surveillance and TSR findings. As previously discussed: (i) economic priorities are chosen to reflect the main global concerns today, as reflected in the latest WEO, GFSR, and the Board discussion of Common Themes in Systemic Consultations—but taking the longer view corresponding to a three-year horizon; (ii) operational priorities reflect the main recommendations of the TSR.

47. The SSP emphasizes areas requiring priority attention and clarifies responsibilities. These priorities are fully consistent with the permanent guidance in Article IV and the 2007 Surveillance Decision, and do not imply that traditional aspects of surveillance can be neglected. As the Fund responds to changing circumstances, the priorities themselves can be revisited at any time. Finally, the SSP notes that management and staff are responsible for meeting operational priorities, subject to members’ cooperation.

48. Progress reporting on SSP priorities will occur regularly, with detailed assessment of performance as part of the next TSR. Progress reports on actions taken and any readily visible results will be provided at frequent intervals, for example in the context of the proposed semi-annual briefings on policy issues arising from Article IV consultations, and the Managing Director’s reports on the IMF’s policy agenda. Planned actions towards these priorities will be reported in the statements on the Work Program. In-depth assessment of performance will occur at the time of the next TSR.

Issues for discussion25

49. Do Directors agree with the proposed economic and operational priorities, and their discussion, in the draft SSP?

50. Do Directors agree that the final SSP should be published (without the more detailed Box on priorities, which would only appear in the TSR documents, with revisions if necessary to take into account the Board’s discussion)?

SURVEILLANCE PRIORITIES FOR THE INTERNATIONAL MONETARY FUND, 2008-2011 (*)

To foster multilateral collaboration, and provide timely and relevant advice to member countries and the public, IMF surveillance will be guided through 2011 by the following overarching objective and supporting priorities:

Early warnings and rapid response. Provide clear, advance warning of developments and risks in the global economy, to help members take actions needed to support their economies during a period of global adjustment.

Economic priorities

The global economy faces a period of slower growth alongside the challenges of sharp changes in commodity prices, the deflation of financially-driven bubbles, and continuing global imbalances. Policies should aim at achieving sustainable noninflationary growth. In coordination with other International Financial Institutions, the IMF should promote a common understanding of the forces and linkages underlying these challenges; draw key lessons from different experiences to share across the membership; and advise on how best to use policy—in particular monetary, fiscal, exchange rate, and financial sector policies—in support of three specific, intertwined objectives:

Adjusting to global financial and commodity booms and busts. The challenge will be to achieve noninflationary growth at a time when booms and busts are slowing potential output growth, changing the sources of global demand, and providing a boost to headline inflation that could well reverse abruptly. This will require domestically appropriate and globally consistent policy responses to past and potential future rapid changes in commodity prices, and to financial sector de-leveraging and associated volatility in asset prices;

Reducing global imbalances in an orderly manner while minimizing adverse real and financial repercussions;

Optimizing the potential of global capital flows while forestalling disruptive patterns by being attentive to cross-border capital flows, including to LICs, arising from global imbalances, and strengthening the global financial system with emphasis on upgrading regulation and supervision, including across borders.

Operational priorities

Risk assessment. Refine the tools necessary to provide clear early warnings to members. Thorough analysis of major risks to baseline projections (including, where appropriate, high-cost tail risks) and their policy implications should become more systematic.

Real-financial linkages. Improve financial stability diagnostic tools, deepen understanding of linkages, including between markets and institutions, and ensure adequate discussion in surveillance reports;

Multilateral perspective. Bilateral surveillance to be informed systematically by analysis of inward spillovers; outward spillovers (where relevant); and cross-country knowledge (as useful); and,

Exchange rate analysis and external stability risks. Clearer and more robust discussions to be provided in all staff reports, underpinned by further progress in refining assessment methodologies.

The Executive Board has highlighted the above priorities to foster multilateral collaboration and guide IMF management and staff in the conduct of surveillance. These priorities look ahead three years, but may be revised if circumstances warrant. Permanent guidance for surveillance (such as provided in the 2007 Decision on Bilateral Surveillance), traditional areas of strength, and relevant country-specific issues should not be overlooked.

Management and staff are responsible for delivering on the operational priorities, subject to members’ cooperation in line with commitments under the Articles of Agreement. To foster progress toward economic priorities, management and staff are responsible for providing candid high-quality analysis and effective communication. The Managing Director will report: (i) regularly on actions toward priorities and readily visible results; and (ii) at the time of the next Triennial Surveillance Review on progress in attaining these priorities; management’s and staff’s contributions; and factors that impeded progress.

article image

SURVEILLANCE PRIORITIES FOR THE(*) INTERNATIONAL MONETARY FUND, 2008-2011

In pursuit of its mandate to promote international monetary andfinancial stability, IMF surveillance will be guided through 2011 by the following priorities:

Economic priorities

The global economy faces a period of severe financial distress and slower growth alongside the challenges of sharp commodity price changes and global imbalances. The following interrelated policy objectives will be key to return to an international environment more conducive to sustainable noninflationary growth:

  • Resolve financial market distress. Restore stability and minimize the adverse impact of the current crisis in financial markets on the real economy;

  • Strengthen the global financial system by upgrading domestic and cross-border regulation and supervision, especially in major financial centers, and by avoiding the exposure of capital-importing countries, including low-income countries, to excessive risks;

  • Adjust to sharp changes in commodity prices. React to commodity price shifts in domestically appropriate and globally consistent ways, with emphasis on keeping inflationary pressures in check in boom phases and minimizing risks that could arise when prices fall;

Promote the orderly reduction of global imbalances while minimizing adverse real and financial repercussions.

In coordination with other International Financial Institutions, the IMF should promote a common understanding of the forces and linkages underlying these challenges; draw key lessons from different experiences to share across the membership; provide clear advance warnings of risks to global economic and financial stability; and advise on how best to use policy—in particular monetary, fiscal, exchange rate, and financial sector policies—in support of these objectives.

Operational priorities

  • Risk assessment. Refine the tools necessary to provide clear early warnings to members. Thorough analysis of major risks to baseline projections (including, where appropriate, high-cost tail risks) and their policy implications should become more systematic;

  • Financial sector surveillance and real-financial linkages. Improve analysis of financial stability, including diagnostic tools; deepen understanding of linkages, including between markets and institutions; and ensure adequate discussion in surveillance reports;

  • Multilateral perspective. Bilateral surveillance to be informed systematically by analysis of inward spillovers; outward spillovers (where relevant); and cross-country knowledge (as useful); and,

Analysis of exchange rates and external stability risks. In the context of strengthening external stability analysis, integrate clearer and more robust exchange rate analysis, underpinned by strengthened methodologies, into the assessment of the overall policy mix.

The Executive Board has set the above priorities to foster multilateral collaboration and guide IMF management and staff in the conduct of surveillance. These priorities look ahead three years, but may be revised if circumstances warrant. They will guide the Fund’s work within the framework for surveillance provided by the Articles of Agreement and the relevant Board decisions, including the 2007 Decision on Bilateral Surveillance. Moreover, traditional areas of strength (such as fiscal policy and debt sustainability analysis) and relevant country-specific issues should not be overlooked.

The Executive Board is responsible for conducting, guiding and evaluating surveillance in order to ensure the achievement of these priorities. Management and staff are responsible for delivering on the operational priorities, subject to members’ cooperation in line with commitments under the Articles of Agreement. To foster progress toward economic priorities, management and staff are responsible for providing candid high-quality analysis and effective communication. The Managing Director will report: (i) regularly on actions toward priorities and readily visible results; and (ii) at the time of the next Triennial Surveillance Review on progress in attaining these priorities; management’s and staff’s contributions; and factors that impeded progress.

article image
1

Most recently, IEO Evaluation of IMF Exchange Rate Policy Advice, 1999-2005 (May 17, 2007) and Evaluation of the IMF’s Multilateral Surveillance (April 7, 2006).

2

On April 16, 2008, the Board informally endorsed that the TSR would concentrate on reviewing the implementation of surveillance, leaving the review of the Decision to the next TSR and that issues of cost-effectiveness and impact of surveillance would be outside the focus of this TSR, as would non bilateral forms of surveillance.

3

Staff also invited think-tanks and civil society organizations to participate in anonymous surveys, but the low response rates did not allow for data analysis (Supplement 2, chapter III.C).

4

The bulk of the evidence-gathering work was conducted in Spring 2008. The 50-country sample was picked from staff reports discussed by the Board by end-February 2008.

5

A full description of the methodological framework is presented in Supplement 2.

6

Supplement 1, Detailed Thematic Findings; Supplement 2, Background Information and Statistical Appendix; and IMF Surveillance in Europe: Progress in Refocusing, Report by an External Consultant.

7

The TSR papers use the terms “developing economies” and “low-income countries” interchangeably. In practice, the former concept, which is slightly broader, was used to analyze staff reports, while the latter was used in the surveys (Statistical Appendix in Supplement 2 offers a full description of country groupings).

8

See also External Consultant Report.

9

See Supplement 1, Chapter II.

10

See IMF Executive Board Reviews the Fund’s Surveillance (PIN/04/95).

11

See Supplement 1, Chapter I.

12

See Supplement 1, Chapter III.

13

See Supplement 1, Chapters V and VI.

14

Although the 2004 BSR did not establish a benchmark, a comparison of two vintages of SIPs (see Supplement 2) and the survey of mission chiefs confirm that this represents an improvement.

15

See Supplement 1, Chapter IV.

16

See Supplement 1, Chapters II and VII.

17

See Supplement 1, Chapter IV, for elaboration.

18

See Supplement I, Chapter VII.

19

See Supplement 1, Chapters V and VI, for elaboration.

20

See Supplement 1, Chapter III, for elaboration.

21

See Guidance on Operational Aspects of the 2007 Surveillance Decision, August 4, 2008.

22

See Supplement 1, Chapters VIII and IX.

23

Reviewers rated reports on overall communication quality according to the extent to which the reports were drafted with a clear story line; crafted around a relatively small set of specific questions, with important statements clearly substantiated with facts or data, and helpful use of data, charts, and boxes generally; and concise.

*

Some of these recommendations were not endorsed by the Board and will be considered further in the upcoming review of transparency policy.

24

Regional differences possibly reflect in large part correlation with some other differences (e.g., income level).

25

The draft SSP will be discussed in a separate Board meeting shortly after the TSR discussion. A final version to be adopted by Board decision, possibly on lapse of time basis, will be circulated after the SSP Board meeting.

(*)

DRAFT-DIFFERS FROM THE FINAL VERSION ADOPTED BY THE EXECUTIVE BOARD

(*)

FINAL VERSION ADOPTED BY EXECUTIVE DIRECTORS ON OCTOBER 2, 2008

2008 Triennial Surveillance Review - Overview Paper
Author: International Monetary Fund