Front Matter

Front Matter

Author(s):
International Monetary Fund. Independent Evaluation Office
Published Date:
March 2013
    Share
    • ShareShare
    Show Summary Details

    © 2011 International Monetary Fund

    Production: IMF Multimedia Services Division

    Cover: Lai Oy Louie

    Cataloging-in-Publication Data

    Research at the IMF : relevance and utilization / [prepared by an IEO team led by Ruben Lamdany and Hali Edison]. – Washington, D.C.: International Monetary Fund, 2011.

    p.; cm.

    At head of title: IEO, Independent Evaluation Office of the International Monetary Fund. Includes bibliographical references. ISBN 978-1-61635-154-0 1. International Monetary Fund – Research – Evaluation. 2. Economics – Research – Evaluation. 3. International finance – Research – Evaluation. I. Lamdany, Ruben, 1954– II. Edison, Hali J. III. International Monetary Fund. Independent Evaluation Office.

    HG3881.5.I58 R47 2011

    Please send orders to:

    International Monetary Fund, Publication Services

    P.O. Box 92780, Washington, D.C. 20090, U.S.A.

    Tel.: (202) 623-7430 Fax: (202) 623-7201

    E-mail: publications@imf.org

    Internet: www.imfbookstore.org

    Contents

    • Background Papers

      • BP/11/01. Evaluating the Quality of IMF Research: A Citation Study BP/11/02. Review of IMF Research on Monetary Policy Frameworks

      • BP/11/03. Review of IMF Research on Tax Policy

      • BP/11/04. An Examination of the Quality of a Sample of 60 Selected Issues

      • Papers BP/11/05. An Evaluation of the Research Chapters of the IMF’s World Economic

      • Outlook and Global Financial Stability Report

      • BP/11/06. Review of the IMF’s Regional Economic Outlook Reports, 2003–09

      • BP/11/07. Macro-Financial Linkages in IMF Research

    The following conventions are used in this publication:

    • In tables, a blank cell or N/A indicates “not applicable,” ellipsis points (…) indicate “not available,” and 0 or 0.0 indicates “zero” or “negligible.” Minor discrepancies between sums of constituent figures and totals are due to rounding.

    • An en dash (–) between years or months (for example, 2008–09 or January–June) indicates the years or months covered, including the beginning and ending years or months; a slash or virgule (/) between years or months (for example, 2008/09) indicates a fiscal or financial year, as does the abbreviation FY (for example, FY2009).

    • “Billion” means a thousand million; “trillion” means a thousand billion.

    • “Basis points” refer to hundredths of 1 percentage point (for example, 25 basis points are equivalent to ¼ of 1 percentage point).

    As used in this publication, the term “country” does not in all cases refer to a territorial entity that is a state as understood by international law and practice. As used here, the term also covers some territorial entities that are not states but for which statistical data are maintained on a separate and independent basis.

    Some of the documents cited and referenced in this report were not available to the public at the time of publication of this report. Under the current policy on public access to the IMF’s archives, some of these documents will become available five years after their issuance. They may be referenced as EBS/YY/NN and SM/YY/NN, where EBS and SM indicate the series and YY indicates the year of issue. Certain other documents are to become available 10 to 20 years after their issuance, depending on the series.

    Foreword

    This study evaluates the relevance and utilization of IMF research to member country authorities, to IMF staff and to other stakeholders. It also examines its technical quality and management.

    Research is at the heart of innovation and improving policymaking. At the IMF, research contributes to the development and updating of conceptual models and tools that form the basis for its analysis and policy recommendations. IMF research has played an important role on how country authorities think about policymaking, and on furthering global knowledge. High-quality research also contributes to the IMF’s reputation and credibility.

    The IMF produces a large body of research, ranging from background studies for bilateral surveillance activities to working papers and external publications dealing with topics of more general interest. This includes a large number of high-quality products, many of which are widely read in member countries and play a significant role in policymaking. This was particularly true for the WEO and GFSR, but also for other publications. At the same time, the evaluation found that there is significant scope to improve the relevance and quality of IMF research, and enhance its utilization. Early consultation with country authorities on research themes, a greater country and institutional context, and clearer standards for quality control are some of the required measures.

    The evaluation found that many studies had conclusions and recommendations that did not appear to flow from the analysis and other studies seemed to be designed with the conclusions in mind. Moreover, there is a widespread view among IMF staff that research findings need to be aligned with current IMF policies. These problems are present across all research types, including in working papers and other academic-style research. Because of this, member country authorities and other stakeholders perceive IMF research as “message-driven.” To deal with this problem, IMF Management and its Executive Board need to cultivate an open, independent, and innovative research environment, explicitly encouraging staff to explore differing and alternative views. Creating an environment that encourages candor and diverse and dissenting views, is a recommendation that the IEO put forward previously in its recent evaluation on the IMF Performance in the Run-Up to the Financial and Economic Crisis, as such an environment would also enhance the effectiveness of IMF surveillance. This evaluation also recommends greater consultation and cooperation with country authorities, and an enhanced quality review process—reforms that would bring greater diversity of research methods and perspectives, and ensure that conclusions and recommendations in research papers are better linked than is the case today to actual findings.

    Conducting high-quality, policy-relevant research at the IMF is essential for its credibility—both in interactions with country authorities and with the international community more generally. IMF research should play its part in facilitating the understanding of the multiple economic uncertainties that lie ahead. Sound and relevant research would assist the Fund in carrying out its activities under increasingly challenging times. We hope that this evaluation will contribute to further improve the relevance, quality, and utilization of the IMF research.

    Moises J. Schwartz

    Director

    Independent Evaluation Office

    Research at the IMF: Relevance and Utilization

    This report was prepared by an IEO team led by Ruben Lamdany and Hali Edison. The IEO team included Ita Mannathoko, Charan Singh, Scott Standley, Louellen Stedman, Hugh Young, Alisa Abrams, Jennet Hojanazarova, Larissa Leony, Roxana Pedraglio, and Jérôme Prieur. The team was assisted by contributions from Joshua Aizenman, Robin Boadway, Gerald Caprio Jr., Barry Eichengreen, Petra Geraats, Refet Gürkaynak, Christopher Heady, Miguel Kiguel, Henrik Kleven, Kenneth Kuttner, Peter Montiel, Javier Pérez de Azpillaga, Marcelo Selowsky, Marko Škreb, Andrés Solimano, Yi Sun, and Shinji Takagi. The evaluation benefited from discussions with participants at three workshops held in November 2009, April 2010, and August 2010, and from comments from John Hicklin and Joanne Salop. However, the final judgments are the responsibility of the IEO alone. Sarah Balbin, Arun Bhatnagar, Annette Canizares, and Mari Lantin provided administrative assistance. Rachel Weaving, Roxana Pedraglio, and Esha Ray provided editorial and production management assistance. The report was approved by Moises Schwartz.

    Abbreviations

    ADV

    advanced economy

    APR

    annual performance review

    BIS

    Bank for International Settlements

    CRP

    Committee on Research Priorities

    DSGE

    dynamic stochastic general equilibrium

    ECB

    European Central Bank

    ECF

    Extended Credit Facility

    EXR

    External Relations Department

    FAD

    Fiscal Affairs Department

    FDMD

    First Deputy Managing Director

    FSAP

    Financial Sector Assessment Program

    GEM

    global economy model

    GFSR

    Global Financial Stability Report

    HIC

    high-income country

    HIPC

    Heavily Indebted Poor Countries

    HRD

    Human Resources Department

    ICM

    International Capital Markets Department

    IEO

    Independent Evaluation Office

    INS

    IMF Institute

    IT

    information technology

    LEM

    large emerging market

    LIC

    low-income country

    MAE

    Monetary and Exchange Affairs Department

    MCD

    Middle East and Central Asia Department

    MCM

    Monetary and Capital Markets Department

    MFD

    Monetary and Financial Systems Department

    MIC

    middle-income country

    MIP

    Management Implementation Plan

    OECD

    Organization for Economic Cooperation and Development

    OEM

    other emerging markets

    OP

    occasional paper

    PDP

    policy discussion paper

    RC

    Research Coordinator

    REO

    Regional Economic Outlook

    RePEc

    Research Papers in Economics

    RES

    Research Department

    SDN

    staff discussion note

    SIP

    selected issues paper

    SPN

    staff position note

    WEO

    World Economic Outlook

    WP

    working paper

    Executive Summary

    This evaluation assesses research produced at the IMF between 1999 and 2008. It focuses on relevance and utilization, but also examines technical quality and management. Research is defined broadly to capture most analytical publications of the IMF, ranging from surveillance-oriented output, for example, selected issues papers (SIPs) prepared for Article IV consultations and the analytical chapters of the World Economic Outlook (WEO) and Global Financial Stability Report (GFSR), to more academically-oriented output, for example, working papers (WPs) and publications in external journals. These outputs comprised a large body of research, about 650 publications annually, at a cost of about 10 percent of the IMF budget.

    The evaluation finds that IMF research was widely read, that it included a large number of high-quality and very useful publications, and that it was appreciated by country authorities and the research community. This was particularly true for the WEO and GFSR, but also for many other publications. Nonetheless, several issues merit attention.

    First, the relevance of research was often hampered by lack of early consultation with country authorities on research themes and by lack of sufficient country and institutional context. Also, authorities indicated that some important issues, such as macro-financial linkages and aspects of monetary policy, were not adequately covered. To strengthen relevance, the IMF should conduct a periodic strategic review of the function and uses of its research product lines to establish whether they should be strengthened, redesigned, or discontinued. Consultation with authorities on research topics and discussions of results should become standard practice. Increased and earlier interaction with authorities as well as longer country assignments by mission members would enhance the country and institutional context of research.

    Second, the technical quality of IMF research publications was quite diverse. The WEO, GFSR, and external publications were generally of high quality. On the other hand, the quality of SIPs and WPs, which are not subject to a rigorous quality review, was lower and more variable. To enhance quality, adequate time and resources should be allocated to each research project, even if this leads to fewer publications. The review of research products should be strengthened to improve quality and to prevent the publication of low-quality products.

    Third, many authorities reported that IMF research was message-driven, and many staff indicated that they often felt pressure to align their conclusions with IMF views. To enhance their quality, reputation, and utilization, working papers should reflect the results of technical analysis even if these are not well aligned with messages in surveillance activities documents.

    Finally, there is a need for greater prioritization and coordination of research across the IMF. To this end, Management should designate a senior staff member, the Research Coordinator (RC), to coordinate research activities across the organization, including by setting standards for quality review processes and publication policies, to promote greater openness, and to address other weaknesses identified in this evaluation. The RC should prepare an indicative medium-term research agenda, in consultation with member countries and the Executive Board, and it should report annually to them on its implementation. This medium-term agenda should not be seen as excluding research on other relevant issues.

      You are not logged in and do not have access to this content. Please login or, to subscribe to IMF eLibrary, please click here

      Other Resources Citing This Publication