Front Matter

Front Matter

Author(s):
International Monetary Fund. Independent Evaluation Office
Published Date:
April 2014
    Share
    • ShareShare
    Show Summary Details

    IEO

    Independent Evaluation Office

    of the International Monetary Fund

    IMF Forecasts

    Process, Quality, and Country Perspectives

    © 2014 International Monetary Fund

    Cover design: Michelle Martin, IMF Multimedia Services

    Cataloging-in-Publication Data

    Joint Bank-Fund Library

    IMF forecasts: process, quality, and country perspectives.– Washington, D.C.: International Monetary Fund, c2014.

    pages.; cm

    At head of title: IEO, Independent Evaluation Office of the International

    Monetary Fund.

    “This report was prepared by an IEO team led by Hans Genberg.”

    Includes bibliographical references.

    ISBN: 978-1-47559-951-0

    1. Economic forecasting. 2. International Monetary Fund. I. Genberg, Hans. II. International Monetary Fund.

    HB3730.I44 2014

    Publication orders may be placed online, by fax, or through the mail:

    International Monetary Fund, Publication Services

    P.O. Box 92780, Washington, DC 20090, U.S.A.

    Tel. (202) 623-7430 Fax: (202) 623-7201

    E-mail: publications@imf.org

    www.imfbookstore.org

    www.elibrary.imf.org

    Contents

    Background Work

    The following Background Papers and Background Documents are available on the IEO website at www.ieo-imf.org.

    Background Papers

    BP/14/01. An Assessment of IMF Medium-Term Forecasts of GDP Growth

    BP/14/02. An Evaluation of Commissioned Studies Assessing the Accuracy of IMF Forecasts

    BP/14/03. The IMF/WEO Forecast Process

    BP/14/04. On the Accuracy and Efficiency of IMF Forecasts: A Survey and Some Extensions

    BP/14/05. IMF Forecasts in the Context of Program Countries

    Background Documents

    1. User Perspectives on IMF Forecasts: Survey Methodology and Results

    2. IMF Training on Forecasting Methods and Forecast Interpretation

    Foreword

    IMF macroeconomic forecasts lie at the core of the IMF’s work. They form the basis for the IMF’s view on the outlook for the global economy as presented in the World Economic Outlook (WEO); they guide its advice in the context of Article IV consultations with member countries; and they underlie the framework of IMF-supported programs. If member country authorities are to have confidence in the IMF’s analysis and advice, it is vital that the underlying forecasts are viewed as well-founded, of high quality, and evenhanded. This evaluation assesses these aspects of IMF forecasts.

    The evaluation finds that the processes and methods used to generate short-term forecasts for Article IV consultations and the WEO are well structured and, in general, appropriately tailored to country-specific characteristics. By and large, country officials have confidence in their integrity, although some believe the forecasting process lacks transparency. Processes to guide the development of medium-term forecasts are found to be less developed than those for short-term forecasts.

    The evaluation shows that the accuracy of IMF short-term forecasts is comparable to that of private forecasts. Both tend to overpredict GDP growth significantly during regional or global recessions, as well as during crises in individual countries. Except for these episodes, IMF forecasts do not show substantial positive or negative biases. Short-term forecasts of GDP growth and inflation made in the context of IMF-supported programs were also unbiased in the majority of cases. However, they tended to be optimistic in high-profile cases characterized by exceptional access to IMF resources. While the IMF has procedures in place to learn from past forecast performance, the evaluation finds that they are not always utilized to their full potential.

    The analysis, evidence, and recommendations in this evaluation report aim to contribute to a better understanding of the macroeconomic forecasting process at the IMF and the quality of IMF forecasts. We hope that the report will provide a useful basis for reflections on ways in which the forecasting process and methods might usefully evolve over time.

    Moises J. Schwartz

    Director

    Independent Evaluation Office

    IMF Forecasts: Process, Quality, and Country Perspectives

    This report was prepared by an IEO team led by Hans Genberg. The IEO team included Carlos de Resende, Francesco Luna, Franz Loyola, and Andrew Martinez. The evaluation was informed by background studies prepared by Charles Freedman, Michael Salemi, and members of the evaluation team. The evaluation benefited from discussions with participants at workshops held in January 2013 and October 2014, and from comments by James Boughton, Jack Boorman, Patrick Conway, Neil Ericsson, John Hicklin, and IEO staff. It also benefited from comments by IMF staff on an earlier version of this report. However, the final judgments are the responsibility of the IEO alone. Arun Bhatnagar, Annette Canizares, and Mari Lantin provided administrative assistance. Rachel Weaving, Roxana Pedraglio, and Esha Ray provided editorial and production management asistance. The report was approved by Moises Schwartz.

    Abbreviations

    ADB

    Asian Development Bank

    BRIICS

    Brazil, Russian Federation, India, Indonesia, China, and South Africa

    CPI

    consumer price index

    DMX

    Data Management for Excel

    EC

    European Commission

    EPA

    ex post assessment

    EPE

    ex post evaluation

    ESR

    External Sector Report

    EU

    European Union

    EWE

    Early Warning Exercise

    FDMD

    First Deputy Managing Director

    FRB

    Federal Reserve Board

    FSB

    Financial Stability Board

    GDP

    gross domestic product

    GFSR

    Global Financial Stability Report

    GPM

    Global Projection Model

    G7

    Canada, France, Germany, Italy, Japan, United Kingdom, and United States

    G20

    a grouping composed of major industrial countries and systemically important developing and emerging market countries

    ICD

    Institute for Capacity Development

    IDFC

    Interdepartmental Forecast Committee

    IMFC

    International Monetary and Financial Committee

    MONA

    Monitoring of Fund Arrangements

    MSI

    Meeting on Surveillance Issues

    OECD

    Organization for Economic Cooperation and Development

    SPR

    Strategy, Policy, and Review Department

    WEO

    World Economic Outlook

    The following conventions are used in this publication:

    • In tables, a blank cell or N/A indicates not applicable,” ellipsis points (…) indicate “not available,” and 0 or 0.0 indicates “zero” or “negligible.” Minor discrepancies between sums of constituent figures and totals are due to rounding.

    • An en-dash (–) between years or months (e.g., 2013–14 or January–June) indicates the years or months covered, including the beginning and ending years or months; a slash or virgule (/) between years or months (e.g., 2013/14) indicates a fiscal or financial year, as does the abbreviation FY (e.g., FY2014).

    • “Billion” means a thousand million; “trillion” means a thousand billion.

    As used in this publication, the term “country” does not in all cases refer to a territorial entity that is a state as understood by international law and practice. As used here, the term also covers some territorial entities that are not states but for which statistical data are maintained on a separate and independent basis.

    Some of the documents cited and referenced in this report were not available to the public at the time of publication of this report. Under the current policy on public access to the IMF’s archives, some of these documents will become available 3 or 5 years after their issuance. They may be referenced as EBS/YY/NN and SM/YY/NN, where EBS and SM indicate the series and YY indicates the year of issue. Certain other types of documents may become available 20 years after their issuance. For further information, see www.imf.org/external/np/arc/eng/archive.htm.

    Executive Summary

    Macroeconomic forecasts are important inputs into IMF bilateral and multilateral surveillance. They form the basis of the analysis and advice contained in Article IV consultations and of the Fund’s view of the outlook for the world economy, as presented in the flagship publications: the World Economic Outlook (WEO), the Global Financial Stability Report (GFSR), and the Fiscal Monitor. The IMF also uses macroeconomic forecasts extensively in other contexts, such as debt sustainability analysis, spillover reports, pilot external balance assessments, and negotiations of IMF-supported adjustment programs, and as the baseline for constructing scenarios and risk assessments for the global economy.

    For member country officials to have confidence in the IMF’s analysis and advice, the underlying forecasts must be viewed as sound, evenhanded, and of high quality.

    This evaluation assesses these aspects of IMF forecasts. Though the forecasting process at the IMF has evolved significantly in the past five years, the assessment deals with current practice. It finds that:

    • The processes and methods used to generate short-term forecasts for Article IV consultations and the WEO are well structured and, in general, appropriately tailored to country-specific characteristics. By and large, country officials have confidence in their integrity. Some officials believe the forecasting process lacks transparency, however—which is consistent with the evaluation team having to spend considerable time and effort to determine exactly how it is structured.

    • Averaged over all member countries and over the period 1990–2011, WEO short-term and medium-term forecasts overpredicted GDP growth and underpredicted inflation. Measured biases in IMF forecasts are highly dependent on the chosen sample period, however. In particular, significant over-predictions of GDP growth tended to occur during regional or global recessions, as well as during crises in individual countries. Except for these episodes, the forecasts did not show substantial positive or negative biases.

    • The accuracy of IMF short-term forecasts was comparable to that of private sector forecasts. This was the case for normal periods as well as for recessions and crises, and for advanced as well as emerging market economies.

    • Short-term forecasts of GDP growth and inflation made in the context of IMF-supported programs were unbiased in the majority of cases. However, they tended to be optimistic in high-profile cases characterized by exceptional access to IMF resources; these cases represented over 80 percent of the dollar amount of IMF resources disbursed. At the first program review (normally about three months into the program), forecast biases were typically reduced or reversed.

    • The IMF has procedures in place to learn from past forecast performance, but these procedures are not always utilized to their full potential.

    Changes in the world economy call for continuous adaptation of the forecasting process and learning by individual forecasters. The evaluation identifies areas where action can be taken to enhance the credibility of the forecasting process and to ensure that high quality is maintained. The recommendations of the evaluation fall into three broad categories. The IMF should:

    • Promote a culture of learning from past forecast performance by introducing a more structured process for implementing and disseminating the recommendations of commissioned studies of forecast performance, and by ensuring that the accumulated knowledge and experience in the institution is effectively incorporated into the forecasting process.

    • Ensure that best practice is followed by providing appropriate guidance to desk economists in forecasting for both the short- and medium term. Attention should focus on how forecast methods should be adapted to economies with different structural features and data availability. The IMF should monitor the consistency of medium-term forecasts across the institution as it does now for the short-term outlook.

    • Enhance transparency by describing the forecasting process in an accessible form, and by making historical forecasts more easily accessible.

      You are not logged in and do not have access to this content. Please login or, to subscribe to IMF eLibrary, please click here

      Other Resources Citing This Publication