- International Monetary Fund. Independent Evaluation Office
- Published Date:
- September 2006
© 2006 International Monetary Fund
Production: IMF Multimedia Services Division
Typesetting: Alicia Etchebarne-Bourdin
Figures: Jason Soleil
Multilateral surveillance/[prepared by a team headed by Shinji Takagi and including Thomas Reichmann … [et al.]—[Washington, D.C.]: Independent Evaluation Office, IMF, 2006—(Evaluation report).
Includes bibliographical references.
1. International Monetary Fund—Evaluation. 2. Economic policy. I. Takagi, Shinji, 1953- II. Reichmann, Tomás. III. Series: Evaluation report (International Monetary Fund).
Please send orders to:
International Monetary Fund, Publication Services
700 19th Street, N.W., Washington, D.C. 20431, U.S.A.
Tel.: (202) 623-7430 Telefax: (202) 623-7201
Statement by the Managing Director, IMF Staff Response, and Summing Up of IMF Executive Board Discussion by Acting Chair
The following conventions are used in this publication:
In tables, ellipsis points (…) indicate “not available” and 0 or 0.0 indicates “zero” or “negligible.” Minor discrepancies between sums of constituent figures and totals are due to rounding.
An en dash (-) between years or months (for example, 2005-06 or January-June) indicates the years or months covered, including the beginning and ending years or months; a slash or virgule (/) between years or months (for example, 2006/06) indicates a fiscal or financial year, as does the abbreviation FY (for example, FY2006).
“Billion” means a thousand million; “trillion” means a thousand billion.
“Basis points” refer to hundredths of 1 percentage point (for example, 25 basis points are equivalent to ¼ of 1 percentage point).
As used in this publication, the term “country” does not in all cases refer to territorial entity that is a state as understood by international law and practice. As used here, the term also covers some territorial entities that are not states but for which statistical data are maintained on a separate and independent basis.
Some of the documents cited and referenced in this report were not available to the public at the time of publication of this report. Under the current policy on public access to the IMF’s archives, some of these documents will become available five years after their issuance. They may be referenced as EBS/YY/NN and SM/YY/NN, where EBS and SM indicate the series and YY indicates the year of issue. Certain other documents are to become available 10 or 20 years after their issuance, depending on the series.
Surveillance is one of the most important activities of the IMF, and is central to its efforts to promote global financial stability through international monetary cooperation. Multilateral surveillance can identify economic linkages and policy spillovers between countries, as well as international economic and market developments. It can complement bilateral surveillance by adding global and cross-country perspectives to the analysis of developments in individual countries and in so doing, it can enhance the quality and relevance of the IMF’s policy advice.
What is special about the IMF’s role is that its near-universal membership gives it a uniquely broad perspective, and that all IMF member governments have committed themselves to be part of its system of peer review and oversight. However, to be influential in this environment, multilateral surveillance must bring to bear analytical rigor, clear policy prescriptions, and an active engagement with senior national policymakers and relevant international forums.
Given the substantial cost to the IMF of its multilateral surveillance activities, and the key role that the IMF has in ensuring a more stable world economy, the Independent Evaluation Office conducted an evaluation of the IMF’s multilateral surveillance activities for the period 2000-05. The evaluation separated issues concerning content and quality from issues of use, delivery, and ultimate effectiveness, and evaluated both using data gathered from desk studies of outputs, surveys of stakeholders, and interviews of government officials. This evaluation report recognizes the analytical quality of many individual components of multilateral surveillance. However, it sees considerable scope for improvement, in particular through better integration of financial and macroeconomic dimensions, as well as bilateral and multilateral analysis and policy prescriptions.
The report recommends that the IMF strengthen its role at the center of a more robust global peer review system by establishing a more proactive engagement with relevant intergovernmental groups; enhance the roles of the Executive Board and the International Monetary and Financial Committee in multilateral surveillance; streamline and better focus the products of multilateral surveillance, present shorter and clearer messages, and deliver them more strategically to target groups; and define more clearly the goals of multilateral surveillance and the mechanisms to achieve them.
Thomas A. Bernes
Independent Evaluation Office
This evaluation report was prepared by a team headed by Shinji Takagi and including Thomas Reichmann, Nils Bjorksten, Halim Kucur, and Ramya Ghosh. Additional contributions were received from David Peretz and Misa Takebe. The report was approved by Thomas A. Bernes, Director of the Independent Evaluation Office. Chapters of the report benefited from inputs from David Goldsbrough and Marcelo Selowsky. Administrative support by Arun Bhatnagar, Annette Canizares, and Maria Gutierrez is gratefully acknowledged, as is editorial assistance from David Cheney and Esha Ray.
Abbreviations and Acronyms
African Development Bank
African Department (IMF)
Asian Development Bank
Asia and Pacific Department (IMF)
Asia-Pacific Economic Cooperation Council
BIS Annual Report
Bond, Equities, and Loans database (IMF)
Bank for International Settlements
Central and Eastern Europe
Central African Economic and Monetary Union
Coordinating Group on Exchange Rate Issues (IMF)
Committee on the Global Financial System
Commonwealth of Independent States
Corporate vulnerability utility (IMF)
European Central Bank
Eastern Caribbean Currency Union
Economic Commission for Latin America and the Caribbean (United Nations)
Economic and Financial Affairs (EU)
Economic and Financial Committee (EU)
Economic Policy Committee (OECD)
European Department (IMF)
External Relations Department (IMF)
Fiscal Affairs Department (IMF)
Financial Action Task Force
Foreign direct investment
Finance Department (IMF)
Financial Market Update (IMF)
Financial Stability Forum
Financial Stability Review (Bank of England)
Financial Systems Trends (IMF)
Fiscal or financial year
Group of Three
Group of Five
Group of Seven
Group of Ten
Group of Twenty
General Arrangements to Borrow
Government Accountability Office (United States)1
Gross domestic product
Global Data Source database (IMF)
Global Economic Environment data (IMF)
Global Economy Model (IMF)
Global Fiscal Model (IMF)
Global Financial Stability Report (IMF)
Global Markets Monitor (IMF)
Global Trade Analysis Project
Heavily Indebted Poor Countries Initiative (IMF and World Bank)
International Association of Insurance Supervisors
International Capital Markets Department (IMF)
International Capital Markets Report (IMF)
Independent Evaluation Office (IMF)
International financial institution
International Monetary Fund
International Monetary and Financial Committee (IMF)
International Organization of Securities Commissions
London interbank offered rate
Mean absolute error
Middle East and Central Asia Department (IMF)
Millennium Development Goals (United Nations)
Multilateral Exchange Rate Model (IMF)
Monetary and Financial Systems Department (IMF)
Multi-Region Econometric Model (IMF)
Organization for Economic Cooperation and Development
OECD Economic Outlook
Policy Development and Review Department (IMF)
Public Information Notice (IMF)
Purchasing power parity
Research Department (IMF)
Root mean squared error
Report on the Observance of Standards and Codes (IMF)
Secretary’s Department (IMF)
Use of Fund resources (IMF)
Vulnerability exercise (IMF)
West African Economic and Monetary Union
World Economic and Market Developments (IMF)
World Economic Outlook (IMF)
Western Hemisphere Department (IMF)
Working Party 1 (OECD)
Working Party 3 (OECD)
World Trade Model (IMF)
Summary of Findings and Recommendations
Surveillance is a core function of the IMF, a critical element of its toolkit to promote global financial stability through international monetary cooperation. Multilateral surveillance brings into analysis economic linkages and policy spillovers between countries, as well as international economic and market developments. It complements bilateral surveillance by adding global and cross-country perspectives to the analysis of developments in individual countries. And by exploring options to deal with policy spillovers in a global context, it can enhance the policy advice that the IMF gives to its members.
The IMF is not alone in providing analysis of the world economy. What is special about the IMF is that all governments that belong to the organization have committed themselves to be part of a system of peer review and oversight. The near universal membership of the IMF gives it a unique perspective. At the same time, a number of other government bodies and private sector entities are engaged in surveillance-type activities. For the IMF to have preeminent impact in this environment, it must bring to bear analytical rigor, clear policy prescriptions, and an active engagement with senior national policymakers and relevant forums.
The ultimate test of the effectiveness of multilateral surveillance is its impact on policies in member countries. Are such policies ever modified as a result of IMF advice about linkages and spillover effects, or as a result of discussions or peer pressure in international forums to which the IMF provides analysis and advice? Such an assessment is difficult for several reasons, including the multiple factors that influence a country’s policies. The main focus of the evaluation is therefore more on the content and quality of multilateral surveillance and how effectively policy implications are drawn and communicated.
Among the questions we address are: (1) Do the issues analyzed correspond to the IMF’s unique perspective with respect to economic linkages between countries? (2) Are the issues examined relevant and timely? (3) Does multilateral surveillance enhance the policy advice provided by bilateral surveillance? (4) How well are macroeconomic and capital market surveillance combined in the analysis of relevant issues? (5) Are the messages of multilateral surveillance reaching the intended audience? (6) Are the messages being presented in a way that maximizes their impact?
IMF multilateral surveillance is carried out by various departments and interdepartmental committees. IMF staff uses a wide range of vehicles to analyze economic developments, and the resulting policy messages are conveyed in a wide range of outputs or products (see box). Most of the individual components of the final products are well crafted and feature high-quality analysis; they are useful to particular audiences and fulfill particular needs. The World Economic Outlook (WEO) is especially well regarded. This evaluation, however, finds that a clear and comprehensive strategy is lacking to guide the integration of the components and the delivery of outputs. As a result, multilateral surveillance is not achieving its full potential.
The absence of an overall strategy has meant that the IMF’s multilateral surveillance as a whole is less than the sum of its parts. Outputs give too much weight to providing information on economic developments and prospects, for which the IMF is increasingly only one of many providers. And they give too little weight to analyzing economic policy linkages, in which the IMF has a comparative advantage, and proactively identifying scope for collective action. The current setup for involving the Executive Board limits its contribution to multilateral surveillance. The failure to clarify the operational goals and define the mechanisms to best meet them has resulted in:
a predominantly “bottom-up,” or country-based, approach to policy advice;
a “silo” structure (in which different IMF departments produce different outputs without adequate coordination) that hinders the fuller integration of macroeconomic and capital market approaches;
a proliferation of publications that lack focus; and
an insufficiently proactive engagement with various high-level groups of national policymakers.
The Main Outputs of IMF Multilateral Surveillance
IMF multilateral surveillance is disseminated to various audiences through a number of outputs, described in greater detail in Background Documents (pp. 9-20).1 These outputs include:
Semiannual “flagship” reports: the World Economic Outlook (WEO) and the Global FinancialStability Report (GFSR);
Regional outlooks produced by four of the IMF area departments;
Regular contributions to intergovernmental forums and committees, such as the Group of Seven (G-7), the Group of Twenty (G-20), and the Financial Stability Forum (FSF);
Confidential analyses of periodic IMF-internal exercises: the Coordinating Group on Exchange Rate Issues, the vulnerability exercise, and World Economic and Market Developments (WEMD); and
Internal reports: the semiannual Financial Systems Trends, the semiannual Financial Market Update, and the daily Global Markets Monitor.
Content and quality
Selection of issues for analysis. The products of multilateral surveillance—especially the WEO—have been largely successful in selecting for analysis issues that reflect the IMF’s comparative advantage. The WEO gives roughly equal weight to issues that deal with the spillovers of policies in individual countries and to analyzing and comparing the experience of different countries. Dedicated analysis of exchange rate issues and related spillover effects, however, does not appear frequently—which is surprising given the IMF’s mandate to oversee the international monetary system and the exchange rate policies of its member countries.
Timely identification of relevant issues and global risks. The WEO has also largely succeeded in identifying in a timely way relevant issues for analysis, as measured against the issues subsequently picked up by G-7 and G-20 agendas. In terms of identifying relevant global macroeconomic and financial risks, both the WEO and the Global Financial Stability Report (GFSR) also compare favorably with similar publications of other international and national bodies. This assessment, however, is based on evidence gathered during the relatively calm period of 2000-05, when no major crisis tested the IMF’s “early warning” mechanisms.
Integration of multilateral and bilateral surveillance. The evaluation confirms the findings of the 1999 external evaluation of surveillance that the IMF’s surveillance has a strong bilateral (or country) orientation, so that policy advice and economic forecasts predominantly reflect the views of IMF area departments (the departments—grouped by geographic region—that carry out bilateral surveillance). As a result of this country orientation, multilateral surveillance has not sufficiently explored options to deal with policy spillovers in a global context; the language of multilateral advice is no more based on explicit consideration of economic linkages and policy spillovers than that of bilateral advice. In addition, the dominant bottom-up (or individual country) approach also tends to yield consistently optimistic macroeconomic forecasts for certain regions.
Integration of macroeconomic and capital market analysis. The IEO evaluation notes the insufficient integration between the WEO and the GFSR, a point also emphasized by the recent McDonough Report.1 The evaluation identifies areas where integration could have been desirable but did not take place, largely owing to the “silo” structure of multilateral surveillance.
Use and delivery
Use of the WEO in bilateral surveillance. There is substantial scope for IMF staff to increase its use of multilateral surveillance outputs in bilateral surveillance work. The evaluation finds that most area department economists do not make much use of the WEO in their country work (other than the WEO’s quantitative forecasts of major economic variables). Indeed, only 14 percent of the senior staff surveyed said that WEO topics were discussed with national authorities.
Use of the GFSR in bilateral surveillance. Few area department economists use the GFSR on a regular basis (only 4 percent of the survey respondents used it “regularly” in their country work). Although the GFSR has raised some important longer-term issues, it has not in practice added value to bilateral surveillance beyond the information already available in the markets. It has not adequately distilled the implications of market developments for the IMF’s day-to-day country work.
Presenting the message. Given the variety of tasks assigned to multilateral surveillance products, the documents have tended to be too long and to lack focus. In the case of the main surveillance chapter of the WEO (Chapter I), for example, each component of the analysis may be useful to a particular audience or meet a particular need, but the efforts to meet all the varying demands have expanded the chapter unduly. Indeed, the full WEO document could benefit from considerable streamlining so that its critical messages were more clearly highlighted. As to surveillance notes, they should concentrate on spelling out the consequences of policy spillovers and objectively presenting options for dealing with them.
Reaching the intended audience. Attempts to reach multiple audiences through the same publications have complicated the task of communicating the messages effectively. The wide press coverage enjoyed, particularly by the WEO, indicates that IMF multilateral surveillance messages have a significant potential for influencing public debate. Yet, we were struck by the low readership (both internally and externally) of the WEO itself. Most readers rely on summaries rather than read the document it self. This underscores the need for the products of IMF multilateral surveillance to have a more explicit “customer” focus, with a range of well-communicated products aimed at meeting the diverse needs of various audiences.
Potential for peer pressure. The potential for multilateral surveillance to exert peer pressure on individual country policies is not fully exploited. First, the IMF is not proactively engaged with the G-7, the G-20, or other forums to which it has unique access. Second, the current structure for involving the Executive Board limits the contributions that it (and the International Monetary and Financial Committee, IMFC) can make to multilateral surveillance.
Recommendation 1. Strengthen the IMF’s role at the center of a more robust global peer review system by establishing a more proactive engagement with relevant intergovernmental groups.
The IMF should become more proactive with respect to intergovernmental groups, particularly the G-7 and the G-20. The emergence of these and other government bodies (e.g., the Organization for Economic Cooperation and Development (OECD) and the Financial Stability Forum (FSF)) and regional groupings (e.g., the euro area, the Asia-Pacific Economic Cooperation Council (APEC), and Western Hemisphere Finance Ministers), as well as increasingly sophisticated officials in capitals, have fundamentally altered the context in which the IMF must act. Increasingly, the IMF must draw on its strength (near-universal membership) and comparative advantage to provide leadership to this global system; it must also draw upon the global system’s collective output to strengthen its own policy advice. The IMF has unique access to the G-7, the G-20 (which includes key emerging market economies), and other global and regional forums of senior national policymakers. Because these forums meet frequently, have limited attendance, and involve actual decision makers, they may provide opportunities for a more frank discussion of policy spillovers and possible responses, and for more effective peer pressure than is possible in larger meetings or at meetings of less senior officials. Rather than seeing such groups as competitors, it is critical that the IMF expand its ties to these groups, while at the same time enhancing the involvement of the IMFC.
Management and the Board could consider a number of possible approaches, such as:
establishing a unit dedicated to maintaining constant contact with the relevant officials of systemically important countries, particularly the chairpersons that guide the work of these groups;
giving more attention to the continuity of IMF representations at these meetings to allow it to foster personal relationships of trust; and
focusing surveillance notes on policy spillovers and options for addressing them. The review of recent developments and prospects could be made into an appendix to the note. A one-page, double-spaced summary should accompany these notes, directly targeted at the senior national policymakers.
Recommendation 2. Enhance the roles of the Executive Board and the IMFC in multilateral surveillance.
Executive Directors play many roles in multilateral surveillance, including as conduits for information to the national authorities and in helping prepare the IMFC discussions. The Board’s primary multilateral surveillance discussions currently center on draft WEO and GFSR documents, each of which is too large to serve as the basis for meaningful policy discussions (particularly within a tight time frame). As a result, the Board’s attention is inevitably focused on factual clarifications and drafting suggestions. To address this weakness, the Board could:
— focus its surveillance discussions on a few issues of critical importance, so as to promote free and open discussion. It might identify and agree on key issues for ministers to focus on at the upcoming IMFC meeting;
— endorse, every six months (say, in summer and winter) in the context of a World Economic and Market Developments (WEMD) session, a short statement on the state of the world economy. This statement should be communicated to the public, including through a Public Information Notice (PIN),2 similar to what is done with respect to Article IV consultations with individual countries;
— consider setting up a standing committee to monitor progress on strengthening the IMF’s and the Board’s surveillance activities.
The IMFC should focus on issues related to policy spillovers and scenarios for collective action. To encourage a more frank and focused consideration of these issues, the Executive Board could prepare, ahead of an IMFC meeting, a short statement of the critical issues facing the global economy and their implications for senior policymakers. This statement could be presented to capitals sufficiently ahead of the meeting to allow time for consideration.
Recommendation 3. Improve the content and form of multilateral surveillance outputs through streamlining and more focus on key issues.
To heighten their impact on the global policy debate, IMF multilateral surveillance products should present a much shorter and more focused message, coupled with a more strategic communications policy to better deliver key messages to target groups. In a world with numerous analyses of global macroeconomic developments and prospects, and in which the IMF’s informational advantage has diminished, the IMF cannot expect to affect the policy debate unless it can clearly articulate and skillfully present its messages on the global situation and on the risks and policy implications it poses. Although multilateral surveillance contributes useful pieces to the global policy debate, the lack of clearly stated objectives has led to multiple publications; multitasking by each of these publications, with a resultant loss of focus; and the absence of an institution-wide communications strategy to coordinate the timing and delivery of clearly articulated and critical messages. In our view, the IMF needs to review and reorient its major multilateral surveillance publications. In this connection, the report on the “Macroeconomics of Globalization” recently proposed by the Managing Director provides an opportunity to rethink and, if necessary, streamline or consolidate some of the existing publications.3 It is not our intent to recommend a specific approach as many variations are possible. But the following possibilities might be considered:
— The special topics chapters of the WEO and the GFSR that deal with long-term issues could be separated and included in a new globalization report;
— The WEO could be streamlined and focused on areas where the IMF can provide the greatest value added over information available elsewhere;
— The WEO could make wider use of scenario analysis and cite the results of such analysis more openly in the main text, drawing pointed policy implications where feasible. It could explore integrating emerging methodologies of scenario analysis into the IMF’s existing global economic models;
— A short note should accompany each WEO, assessing critical global economic issues and drawing out their policy implications in a way that would be helpful and accessible to senior national policymakers;
— The timing of publications could be better sequenced to allow the intended audiences sufficient time to absorb them. At present, the WEO and GFSR—which consist of about 500 typeset pages—are delivered almost simultaneously for the spring and fall meetings of the IMFC.
The Executive Board should clarify the scope of regional surveillance. IMF staff has taken various initiatives in recent years with respect to its regional work, partly responding to the call of some Executive Directors. In addition to the surveillance of common currency areas, which has become more formalized, most area departments have begun to produce regional economic outlooks. A large portion of these reports, however, describes recent economic developments and prospects, information which is available elsewhere. While regional outreach has been well received, it is not clear that regional outlooks effectively serve a regional surveillance function—which would require them to concentrate on regional economic interlinkages and policy spillovers. The Board’s clarification should cover the following issues:
— the role of regional outlook publications in the IMF’s overall publications policy;
— whether it makes more sense to reorient some regional studies on the basis of analytical constructs (such as stage of development, small island economies, or some other element of commonality), rather than on the basis of geography defined by the jurisdiction of an area department. If so, the Board should assign responsibility for initiating and producing such cross-country studies;
— whether analytical chapters of regional out-looks—which deal with topical issues—might also be included in the proposed globalization report.
Recommendation 4. Strengthen the structure of multilateral surveillance by clarifying operational goals and defining organizational strategies and accountabilities.
The IMF should clarify the operational goals of multilateral surveillance and define the strategies and mechanisms to achieve these goals, including, for each output, the purpose, the intended audience, who is accountable, and how to measure effectiveness. The lack of clearly stated goals has led to differing views of these goals and their meaning and to an inability to measure effectiveness.
The IMF needs to strengthen the multilateral dimension of surveillance, particularly for “systemically important” countries (that is, countries that have an impact on global financial stability). The need to better integrate bilateral and multilateral surveillance has long been recognized. However, despite all efforts, including repeated Board directives, Policy Development and Review Department (PDR) guidance notes, and the IMF’s internal review process, integration has not been achieved. Although one is tempted to say “just do it,” the failure of earlier efforts over at least the last six years must give reason for pause. There is a need for serious reflection on why the existing incentive structure within the institution has thwarted progress on this front. At a minimum, benchmarks need to be established to measure progress on the integration of financial sector and capital markets work with macroeconomic work, and on the integration of multilateral and bilateral surveillance. Success in meeting these benchmarks should then be a key component of the scheduled 2008 Biennial Review of Surveillance.
To achieve this goal may require more fundamental organizational changes. Such change can be disruptive and should not be undertaken lightly. Many considerations are relevant and the IEO is not in a position to make specific recommendations. However, a number of possible options, ranging from major structural change to a reallocation of accountabilities within the existing organizational structure, have been identified and are worth considering:
— Given the centrality of surveillance, the IMF could establish a Surveillance Department by combining the existing multilateral surveillance functions of the Research Department (RES) and the surveillance functions of the PDR.
— For each systemically important country, the IMF’s Economic Counsellor (and head of RES) could prepare a one-page assessment of the key policy measures that would contribute toward resolving core global surveillance challenges prior to the Article IV consultation discussion at the Board.
— Give sign-off authority to RES—in addition to PDR—on the briefing papers and staff reports for Article IV consultations with systemically important countries, and promote RES’s more active participation in country work. This may call for the creation of a global surveillance division responsible for integrating all aspects of IMF surveillance.
— Make greater use of the internal Surveillance Committee, chaired by management, to form institutional positions on systemically important issues, including exchange rates, and to contribute to management’s accountability to the Board.
As already recognized within the IMF, the work of the International Capital Markets Department, ICM (soon to be merged with the Monetary and Financial Systems Department, MFD)should be reoriented toward informing IMF economists of the macroeconomic implications of market developments and unfolding risks. The evaluation confirms the findings of previous evaluations, as well as the recent McDonough Report, that capital market surveillance is not well integrated into the work of the rest of the IMF. The following steps could thus be considered:4
— The new department to be created by merging ICM and MFD could make greater efforts to distill the macroeconomic implications of its capital market analysis, speak the language of macroeconomists, avoid excessive use of market jargon, and collaborate better with other departments.
— The new department should seek a more cost-effective and targeted way to deliver to its audiences the results of its research and analyses on capital market topics.