- International Monetary Fund. Independent Evaluation Office
- Published Date:
- April 2014
Independent Evaluation Office
of the International Monetary Fund
Process, Quality, and Country Perspectives
© 2014 International Monetary Fund
Cover design: Michelle Martin, IMF Multimedia Services
Joint Bank-Fund Library
IMF forecasts: process, quality, and country perspectives.– Washington, D.C.: International Monetary Fund, c2014.
At head of title: IEO, Independent Evaluation Office of the International
“This report was prepared by an IEO team led by Hans Genberg.”
Includes bibliographical references.
1. Economic forecasting. 2. International Monetary Fund. I. Genberg, Hans. II. International Monetary Fund.
Publication orders may be placed online, by fax, or through the mail:
International Monetary Fund, Publication Services
P.O. Box 92780, Washington, DC 20090, U.S.A.
Tel. (202) 623-7430 Fax: (202) 623-7201
Statement by the Managing Director and the Acting Chair’s Summing Up
The following Background Papers and Background Documents are available on the IEO website at www.ieo-imf.org.
BP/14/01. An Assessment of IMF Medium-Term Forecasts of GDP Growth
BP/14/02. An Evaluation of Commissioned Studies Assessing the Accuracy of IMF Forecasts
BP/14/03. The IMF/WEO Forecast Process
BP/14/04. On the Accuracy and Efficiency of IMF Forecasts: A Survey and Some Extensions
BP/14/05. IMF Forecasts in the Context of Program Countries
1. User Perspectives on IMF Forecasts: Survey Methodology and Results
2. IMF Training on Forecasting Methods and Forecast Interpretation
IMF macroeconomic forecasts lie at the core of the IMF’s work. They form the basis for the IMF’s view on the outlook for the global economy as presented in the World Economic Outlook (WEO); they guide its advice in the context of Article IV consultations with member countries; and they underlie the framework of IMF-supported programs. If member country authorities are to have confidence in the IMF’s analysis and advice, it is vital that the underlying forecasts are viewed as well-founded, of high quality, and evenhanded. This evaluation assesses these aspects of IMF forecasts.
The evaluation finds that the processes and methods used to generate short-term forecasts for Article IV consultations and the WEO are well structured and, in general, appropriately tailored to country-specific characteristics. By and large, country officials have confidence in their integrity, although some believe the forecasting process lacks transparency. Processes to guide the development of medium-term forecasts are found to be less developed than those for short-term forecasts.
The evaluation shows that the accuracy of IMF short-term forecasts is comparable to that of private forecasts. Both tend to overpredict GDP growth significantly during regional or global recessions, as well as during crises in individual countries. Except for these episodes, IMF forecasts do not show substantial positive or negative biases. Short-term forecasts of GDP growth and inflation made in the context of IMF-supported programs were also unbiased in the majority of cases. However, they tended to be optimistic in high-profile cases characterized by exceptional access to IMF resources. While the IMF has procedures in place to learn from past forecast performance, the evaluation finds that they are not always utilized to their full potential.
The analysis, evidence, and recommendations in this evaluation report aim to contribute to a better understanding of the macroeconomic forecasting process at the IMF and the quality of IMF forecasts. We hope that the report will provide a useful basis for reflections on ways in which the forecasting process and methods might usefully evolve over time.
Moises J. Schwartz
Independent Evaluation Office
IMF Forecasts: Process, Quality, and Country Perspectives
This report was prepared by an IEO team led by Hans Genberg. The IEO team included Carlos de Resende, Francesco Luna, Franz Loyola, and Andrew Martinez. The evaluation was informed by background studies prepared by Charles Freedman, Michael Salemi, and members of the evaluation team. The evaluation benefited from discussions with participants at workshops held in January 2013 and October 2014, and from comments by James Boughton, Jack Boorman, Patrick Conway, Neil Ericsson, John Hicklin, and IEO staff. It also benefited from comments by IMF staff on an earlier version of this report. However, the final judgments are the responsibility of the IEO alone. Arun Bhatnagar, Annette Canizares, and Mari Lantin provided administrative assistance. Rachel Weaving, Roxana Pedraglio, and Esha Ray provided editorial and production management asistance. The report was approved by Moises Schwartz.
Asian Development BankBRIICS
Brazil, Russian Federation, India, Indonesia, China, and South AfricaCPI
consumer price indexDMX
Data Management for ExcelEC
ex post assessmentEPE
ex post evaluationESR
External Sector ReportEU
Early Warning ExerciseFDMD
First Deputy Managing DirectorFRB
Federal Reserve BoardFSB
Financial Stability BoardGDP
gross domestic productGFSR
Global Financial Stability ReportGPM
Global Projection ModelG7
Canada, France, Germany, Italy, Japan, United Kingdom, and United StatesG20
a grouping composed of major industrial countries and systemically important developing and emerging market countriesICD
Institute for Capacity DevelopmentIDFC
Interdepartmental Forecast CommitteeIMFC
International Monetary and Financial CommitteeMONA
Monitoring of Fund ArrangementsMSI
Meeting on Surveillance IssuesOECD
Organization for Economic Cooperation and DevelopmentSPR
Strategy, Policy, and Review DepartmentWEO
World Economic Outlook
The following conventions are used in this publication:
In tables, a blank cell or N/A indicates not applicable,” ellipsis points (…) indicate “not available,” and 0 or 0.0 indicates “zero” or “negligible.” Minor discrepancies between sums of constituent figures and totals are due to rounding.
An en-dash (–) between years or months (e.g., 2013–14 or January–June) indicates the years or months covered, including the beginning and ending years or months; a slash or virgule (/) between years or months (e.g., 2013/14) indicates a fiscal or financial year, as does the abbreviation FY (e.g., FY2014).
“Billion” means a thousand million; “trillion” means a thousand billion.
As used in this publication, the term “country” does not in all cases refer to a territorial entity that is a state as understood by international law and practice. As used here, the term also covers some territorial entities that are not states but for which statistical data are maintained on a separate and independent basis.
Some of the documents cited and referenced in this report were not available to the public at the time of publication of this report. Under the current policy on public access to the IMF’s archives, some of these documents will become available 3 or 5 years after their issuance. They may be referenced as EBS/YY/NN and SM/YY/NN, where EBS and SM indicate the series and YY indicates the year of issue. Certain other types of documents may become available 20 years after their issuance. For further information, see www.imf.org/external/np/arc/eng/archive.htm.
Macroeconomic forecasts are important inputs into IMF bilateral and multilateral surveillance. They form the basis of the analysis and advice contained in Article IV consultations and of the Fund’s view of the outlook for the world economy, as presented in the flagship publications: the World Economic Outlook (WEO), the Global Financial Stability Report (GFSR), and the Fiscal Monitor. The IMF also uses macroeconomic forecasts extensively in other contexts, such as debt sustainability analysis, spillover reports, pilot external balance assessments, and negotiations of IMF-supported adjustment programs, and as the baseline for constructing scenarios and risk assessments for the global economy.
For member country officials to have confidence in the IMF’s analysis and advice, the underlying forecasts must be viewed as sound, evenhanded, and of high quality.
This evaluation assesses these aspects of IMF forecasts. Though the forecasting process at the IMF has evolved significantly in the past five years, the assessment deals with current practice. It finds that:
The processes and methods used to generate short-term forecasts for Article IV consultations and the WEO are well structured and, in general, appropriately tailored to country-specific characteristics. By and large, country officials have confidence in their integrity. Some officials believe the forecasting process lacks transparency, however—which is consistent with the evaluation team having to spend considerable time and effort to determine exactly how it is structured.
Averaged over all member countries and over the period 1990–2011, WEO short-term and medium-term forecasts overpredicted GDP growth and underpredicted inflation. Measured biases in IMF forecasts are highly dependent on the chosen sample period, however. In particular, significant over-predictions of GDP growth tended to occur during regional or global recessions, as well as during crises in individual countries. Except for these episodes, the forecasts did not show substantial positive or negative biases.
The accuracy of IMF short-term forecasts was comparable to that of private sector forecasts. This was the case for normal periods as well as for recessions and crises, and for advanced as well as emerging market economies.
Short-term forecasts of GDP growth and inflation made in the context of IMF-supported programs were unbiased in the majority of cases. However, they tended to be optimistic in high-profile cases characterized by exceptional access to IMF resources; these cases represented over 80 percent of the dollar amount of IMF resources disbursed. At the first program review (normally about three months into the program), forecast biases were typically reduced or reversed.
The IMF has procedures in place to learn from past forecast performance, but these procedures are not always utilized to their full potential.
Changes in the world economy call for continuous adaptation of the forecasting process and learning by individual forecasters. The evaluation identifies areas where action can be taken to enhance the credibility of the forecasting process and to ensure that high quality is maintained. The recommendations of the evaluation fall into three broad categories. The IMF should:
Promote a culture of learning from past forecast performance by introducing a more structured process for implementing and disseminating the recommendations of commissioned studies of forecast performance, and by ensuring that the accumulated knowledge and experience in the institution is effectively incorporated into the forecasting process.
Ensure that best practice is followed by providing appropriate guidance to desk economists in forecasting for both the short- and medium term. Attention should focus on how forecast methods should be adapted to economies with different structural features and data availability. The IMF should monitor the consistency of medium-term forecasts across the institution as it does now for the short-term outlook.
Enhance transparency by describing the forecasting process in an accessible form, and by making historical forecasts more easily accessible.