Front Matter

Front Matter

Author(s):
International Monetary Fund. Independent Evaluation Office
Published Date:
March 2013
Share
  • ShareShare
Show Summary Details

© 2011 International Monetary Fund

Production: IMF Multimedia Services Division

Cover: Lai Oy Louie

Cataloging-in-Publication Data

Research at the IMF : relevance and utilization / [prepared by an IEO team led by Ruben Lamdany and Hali Edison]. – Washington, D.C.: International Monetary Fund, 2011.

p.; cm.

At head of title: IEO, Independent Evaluation Office of the International Monetary Fund. Includes bibliographical references. ISBN 978-1-61635-154-0 1. International Monetary Fund – Research – Evaluation. 2. Economics – Research – Evaluation. 3. International finance – Research – Evaluation. I. Lamdany, Ruben, 1954– II. Edison, Hali J. III. International Monetary Fund. Independent Evaluation Office.

HG3881.5.I58 R47 2011

Please send orders to:

International Monetary Fund, Publication Services

P.O. Box 92780, Washington, D.C. 20090, U.S.A.

Tel.: (202) 623-7430 Fax: (202) 623-7201

E-mail: publications@imf.org

Internet: www.imfbookstore.org

Contents

  • Background Papers

    • BP/11/01. Evaluating the Quality of IMF Research: A Citation Study BP/11/02. Review of IMF Research on Monetary Policy Frameworks

    • BP/11/03. Review of IMF Research on Tax Policy

    • BP/11/04. An Examination of the Quality of a Sample of 60 Selected Issues

    • Papers BP/11/05. An Evaluation of the Research Chapters of the IMF’s World Economic

    • Outlook and Global Financial Stability Report

    • BP/11/06. Review of the IMF’s Regional Economic Outlook Reports, 2003–09

    • BP/11/07. Macro-Financial Linkages in IMF Research

The following conventions are used in this publication:

  • In tables, a blank cell or N/A indicates “not applicable,” ellipsis points (…) indicate “not available,” and 0 or 0.0 indicates “zero” or “negligible.” Minor discrepancies between sums of constituent figures and totals are due to rounding.

  • An en dash (–) between years or months (for example, 2008–09 or January–June) indicates the years or months covered, including the beginning and ending years or months; a slash or virgule (/) between years or months (for example, 2008/09) indicates a fiscal or financial year, as does the abbreviation FY (for example, FY2009).

  • “Billion” means a thousand million; “trillion” means a thousand billion.

  • “Basis points” refer to hundredths of 1 percentage point (for example, 25 basis points are equivalent to ¼ of 1 percentage point).

As used in this publication, the term “country” does not in all cases refer to a territorial entity that is a state as understood by international law and practice. As used here, the term also covers some territorial entities that are not states but for which statistical data are maintained on a separate and independent basis.

Some of the documents cited and referenced in this report were not available to the public at the time of publication of this report. Under the current policy on public access to the IMF’s archives, some of these documents will become available five years after their issuance. They may be referenced as EBS/YY/NN and SM/YY/NN, where EBS and SM indicate the series and YY indicates the year of issue. Certain other documents are to become available 10 to 20 years after their issuance, depending on the series.

Foreword

This study evaluates the relevance and utilization of IMF research to member country authorities, to IMF staff and to other stakeholders. It also examines its technical quality and management.

Research is at the heart of innovation and improving policymaking. At the IMF, research contributes to the development and updating of conceptual models and tools that form the basis for its analysis and policy recommendations. IMF research has played an important role on how country authorities think about policymaking, and on furthering global knowledge. High-quality research also contributes to the IMF’s reputation and credibility.

The IMF produces a large body of research, ranging from background studies for bilateral surveillance activities to working papers and external publications dealing with topics of more general interest. This includes a large number of high-quality products, many of which are widely read in member countries and play a significant role in policymaking. This was particularly true for the WEO and GFSR, but also for other publications. At the same time, the evaluation found that there is significant scope to improve the relevance and quality of IMF research, and enhance its utilization. Early consultation with country authorities on research themes, a greater country and institutional context, and clearer standards for quality control are some of the required measures.

The evaluation found that many studies had conclusions and recommendations that did not appear to flow from the analysis and other studies seemed to be designed with the conclusions in mind. Moreover, there is a widespread view among IMF staff that research findings need to be aligned with current IMF policies. These problems are present across all research types, including in working papers and other academic-style research. Because of this, member country authorities and other stakeholders perceive IMF research as “message-driven.” To deal with this problem, IMF Management and its Executive Board need to cultivate an open, independent, and innovative research environment, explicitly encouraging staff to explore differing and alternative views. Creating an environment that encourages candor and diverse and dissenting views, is a recommendation that the IEO put forward previously in its recent evaluation on the IMF Performance in the Run-Up to the Financial and Economic Crisis, as such an environment would also enhance the effectiveness of IMF surveillance. This evaluation also recommends greater consultation and cooperation with country authorities, and an enhanced quality review process—reforms that would bring greater diversity of research methods and perspectives, and ensure that conclusions and recommendations in research papers are better linked than is the case today to actual findings.

Conducting high-quality, policy-relevant research at the IMF is essential for its credibility—both in interactions with country authorities and with the international community more generally. IMF research should play its part in facilitating the understanding of the multiple economic uncertainties that lie ahead. Sound and relevant research would assist the Fund in carrying out its activities under increasingly challenging times. We hope that this evaluation will contribute to further improve the relevance, quality, and utilization of the IMF research.

Moises J. Schwartz

Director

Independent Evaluation Office

Research at the IMF: Relevance and Utilization

This report was prepared by an IEO team led by Ruben Lamdany and Hali Edison. The IEO team included Ita Mannathoko, Charan Singh, Scott Standley, Louellen Stedman, Hugh Young, Alisa Abrams, Jennet Hojanazarova, Larissa Leony, Roxana Pedraglio, and Jérôme Prieur. The team was assisted by contributions from Joshua Aizenman, Robin Boadway, Gerald Caprio Jr., Barry Eichengreen, Petra Geraats, Refet Gürkaynak, Christopher Heady, Miguel Kiguel, Henrik Kleven, Kenneth Kuttner, Peter Montiel, Javier Pérez de Azpillaga, Marcelo Selowsky, Marko Škreb, Andrés Solimano, Yi Sun, and Shinji Takagi. The evaluation benefited from discussions with participants at three workshops held in November 2009, April 2010, and August 2010, and from comments from John Hicklin and Joanne Salop. However, the final judgments are the responsibility of the IEO alone. Sarah Balbin, Arun Bhatnagar, Annette Canizares, and Mari Lantin provided administrative assistance. Rachel Weaving, Roxana Pedraglio, and Esha Ray provided editorial and production management assistance. The report was approved by Moises Schwartz.

Abbreviations

ADV

advanced economy

APR

annual performance review

BIS

Bank for International Settlements

CRP

Committee on Research Priorities

DSGE

dynamic stochastic general equilibrium

ECB

European Central Bank

ECF

Extended Credit Facility

EXR

External Relations Department

FAD

Fiscal Affairs Department

FDMD

First Deputy Managing Director

FSAP

Financial Sector Assessment Program

GEM

global economy model

GFSR

Global Financial Stability Report

HIC

high-income country

HIPC

Heavily Indebted Poor Countries

HRD

Human Resources Department

ICM

International Capital Markets Department

IEO

Independent Evaluation Office

INS

IMF Institute

IT

information technology

LEM

large emerging market

LIC

low-income country

MAE

Monetary and Exchange Affairs Department

MCD

Middle East and Central Asia Department

MCM

Monetary and Capital Markets Department

MFD

Monetary and Financial Systems Department

MIC

middle-income country

MIP

Management Implementation Plan

OECD

Organization for Economic Cooperation and Development

OEM

other emerging markets

OP

occasional paper

PDP

policy discussion paper

RC

Research Coordinator

REO

Regional Economic Outlook

RePEc

Research Papers in Economics

RES

Research Department

SDN

staff discussion note

SIP

selected issues paper

SPN

staff position note

WEO

World Economic Outlook

WP

working paper

Executive Summary

This evaluation assesses research produced at the IMF between 1999 and 2008. It focuses on relevance and utilization, but also examines technical quality and management. Research is defined broadly to capture most analytical publications of the IMF, ranging from surveillance-oriented output, for example, selected issues papers (SIPs) prepared for Article IV consultations and the analytical chapters of the World Economic Outlook (WEO) and Global Financial Stability Report (GFSR), to more academically-oriented output, for example, working papers (WPs) and publications in external journals. These outputs comprised a large body of research, about 650 publications annually, at a cost of about 10 percent of the IMF budget.

The evaluation finds that IMF research was widely read, that it included a large number of high-quality and very useful publications, and that it was appreciated by country authorities and the research community. This was particularly true for the WEO and GFSR, but also for many other publications. Nonetheless, several issues merit attention.

First, the relevance of research was often hampered by lack of early consultation with country authorities on research themes and by lack of sufficient country and institutional context. Also, authorities indicated that some important issues, such as macro-financial linkages and aspects of monetary policy, were not adequately covered. To strengthen relevance, the IMF should conduct a periodic strategic review of the function and uses of its research product lines to establish whether they should be strengthened, redesigned, or discontinued. Consultation with authorities on research topics and discussions of results should become standard practice. Increased and earlier interaction with authorities as well as longer country assignments by mission members would enhance the country and institutional context of research.

Second, the technical quality of IMF research publications was quite diverse. The WEO, GFSR, and external publications were generally of high quality. On the other hand, the quality of SIPs and WPs, which are not subject to a rigorous quality review, was lower and more variable. To enhance quality, adequate time and resources should be allocated to each research project, even if this leads to fewer publications. The review of research products should be strengthened to improve quality and to prevent the publication of low-quality products.

Third, many authorities reported that IMF research was message-driven, and many staff indicated that they often felt pressure to align their conclusions with IMF views. To enhance their quality, reputation, and utilization, working papers should reflect the results of technical analysis even if these are not well aligned with messages in surveillance activities documents.

Finally, there is a need for greater prioritization and coordination of research across the IMF. To this end, Management should designate a senior staff member, the Research Coordinator (RC), to coordinate research activities across the organization, including by setting standards for quality review processes and publication policies, to promote greater openness, and to address other weaknesses identified in this evaluation. The RC should prepare an indicative medium-term research agenda, in consultation with member countries and the Executive Board, and it should report annually to them on its implementation. This medium-term agenda should not be seen as excluding research on other relevant issues.

    Other Resources Citing This Publication