Annex 1: Status of the Implementation of Recommendations from the 1999 External Evaluation of Research in the IMF29

International Monetary Fund. Independent Evaluation Office
Published Date:
March 2013
  • ShareShare
Show Summary Details
1. Create a Committee on Research Priorities (CRP) to
  • The CRP was established in November 1999. It identified priorities, encouraged research in selected topics, facilitated discussion and coordination among departments, and sponsored seminars.

assist in strategic planning and to support research activities.
  • The CRP became inactive after 2001. However, memoranda from the Managing Director on work organization continued to refer to the CRP.

  • In 2004 the Research Committee was established with broadly the same mission as the CRP but was not sustained beyond 2006.

2. Introduce explicit departmental targets for staff time allocated to research activities.
  • No evidence was found of explicit Fund-wide targets for dedicated research time, but resources were allocated for research within some departments.

  • For example, RES provided notional guidelines for the department, splitting time equally among operational work, “directed” research, and “self-generated” research. There was evidence that staff time was also explicitly allocated to research in some other departments, e.g., IMF Institute and MCD.

3. Shift the mix of research toward topics that add the most value.
  • In January 2000, the CRP identified Fund-wide research topics. It also asked relevant departments to take stock of existing work on financial markets and developing or transition countries and to propose new work where gaps existed, with the goal of shifting more resources to research in these areas.

  • An interdepartmental Committee on Low-Income Country Work chaired by the First Deputy Managing Director was formed in 2004. Its goal was to ensure internal and external coordination of research, policy development, operational practice, communication, and outreach on low-income countries.

  • During 1999-2008 there was an increase in the coverage of fiscal and financial issues in working papers (WPs) and selected issues papers (SIPs).

  • There was a shift in the country focus of SIPs toward emerging market economies and a small increase in the number of SIPs on ECF-eligible countries. However, the average number of papers per ECF-eligible country remained lower than that for other income groups.

4. Create incentives to improve collaboration among departments and to encourage researchers to contribute to policy work.
  • The Interdepartmental Working Group on Fund Research (WGFR) constituted in 1989 continued to gather and disseminate information on ongoing and planned research projects across the Fund.

  • Some departments recognized a researcher’s service to other departments in the annual performance review (APR).

  • Collaboration among departments took place on specific issues (e.g., G-20 Surveillance Notes, climate change).

  • Several departments have established collaboration sites on their internal web pages.

5. Improve assessment of research quality in the annual performance evaluation system.
  • Quality of research is not part of the APR process in a uniform manner across the institution.

  • The decision on whether and how to recognize quality as part of the APR process is made at the divisional level and may vary across departments and divisions.

6. Give all staff, no matter how junior, opportunities to present their research products to management and the Executive Board.
  • This is not a standard practice at the Fund.

7. Management should give a clear mandate to the Director of the
  • The mandate of the Director of Research and Economic Counsellor remained largely unchanged for the four directors who served during 2001-10.

Research Department to be both an active research leader and economic counsellor to the Fund.
  • The Director’s mandate continues to include the function of Economic Counsellor, and to lead the research of the Research Department, but not the research conducted in the rest of the IMF.

8. Create a more effective performance evaluation system.
  • Overall, changes in the performance evaluation system since 1999 have sought greater differentiation by performance, but in 2008 the system was simplified, limiting performance ratings to four categories.

9. For departments other than Policy Development and Review Department, the Fund should consider how to reduce unnecessary internal review of Fund work and avoid formal written comments where informal communication would be adequate.
  • The review of working papers by EXR was reduced.

  • In most departments research was reviewed primarily by division chiefs. SIPs were reviewed by the mission chiefs. In some cases, internal seminars were used to secure feedback. Inter-departmental reviews occurred mainly with WEO, GFSR, and REOs.

  • The Fund-wide review process was examined several times during the evaluation period, including in 2003-04 and more recently in 2007, when the Working Group on Simplifying the Review Process made recommendations that led to the introduction of a new process in 2009.

10. Encourage participation in relevant external conferences.
  • In 2000 CRP increased the travel budget for staff attendance at outside conferences.

  • Performance appraisals in RES took into account participation in important conferences.

11. Put only the names of significant contributors on Fund publications.
  • According to interviews of department managers, treatment of authorship was mixed. In some departments, everyone associated with a paper was included as an author. But, in other departments a clear effort was made to only mention key contributors as authors.

  • A 2007 department self-evaluation stated that RES adhered to this recommendation.

12. Improve collaboration between World Bank and Fund researchers.
  • A monthly Bank-Fund research seminar was initiated in November 1999. However for the most part, the papers were not based on joint research.

  • A 2006-07 review of Bank-Fund collaboration cited joint work on specific products to which collaborative research may have contributed, such as FSAPs, the HIPC Initiative, and debt sustainability analysis.

13. Introduce more flexibility into hiring procedures for entry-level economists.
  • The Economist Program selection process was modified to attract some candidates with a strong research interest and demonstrated research abilities. Research submissions were part of the selection process and RES contributed to the selection process of these candidates.

14. Consider streamlining the management structure in the Research Department.
  • Management processes and structure in the Research Department were changed several times in the past decade, typically following the appointment of a new department head.

15. Write and disseminate nontechnical summaries of the highest quality and most relevant research.
  • In June 2000, the Research Department began publishing a quarterly research newsletter, the IMF Research Bulletin. This selectively summarizes key components of research done at the IMF and provides a listing of research documents and other research-related activities, including IMF conferences and seminars.

16. Treat working papers as preliminary documents.
  • All papers continued to be authorized by a manager.

  • The disclaimer used on working papers was adjusted after 1999, but not along the lines proposed in the evaluators’ recommendation.

17. Create a new vehicle for nonsenior staff to make presentations to Management and the Executive Board.
  • There was no evidence that a new vehicle was established.

18. Improve dissemination of research to nontechnical audiences outside the Fund.
  • Presentations to authorities of the WEO and GFSR were increased. In recent years, there were also presentations of REOs.

  • However, dissemination of hard copies of research products declined, following budget cuts and a shift in focus to web-based dissemination.

19. Increase the number of research assistants relative to economists.
  • Following the evaluation, the ratio of research assistants to economists initially increased, but it fell back in 2007.

  • Starting in FY2005, the Fund started hiring information management assistants to help with database management.

20. Create an ongoing external review process for research products.
  • No evidence was found of an ongoing Fund-wide external review process for research products.

  • There were isolated examples of departments seeking external reviews. The most prominent example was an academic panel by FAD, convened annually from 2002 to 2008.

21. Monitor progress on implementing recommendations in this report.
  • Eight months after the Board discussion of the 1999 External Evaluation of Research, IMF Management submitted an “Information Note on Follow-Up to the External Evaluation of Research” (EBAP/00/28) to the Board.

  • The Research Department undertook an internal review of the status of implementation of recommendations pertinent to the Department in 2007.

22. Create periodic, general, external reviews of research activities.
  • There was no evidence of an external review of the IMF’s research activities.

This annex was prepared by the IEO to assess the status of the 22 recommendations that were contained in the External Evaluation of IMF Research prepared by Mishkin and others (1999).

    Other Resources Citing This Publication