Comments by the Managing Director on the Report of External Evaluators on the IMF’s Economic Research Activities
- International Monetary Fund
- Published Date:
- April 2000
Executive Board Meeting
September 7, 1999
We have been asked by the Chairman of the Evaluation Group of Executive Directors to comment on the external evaluation of research. Management welcomes the evaluation report, which provides much food for thought. We are grateful to the evaluators for the work they have done, and for providing an opportunity for an in-depth discussion of the role of research in the Fund. The comments that follow are intended to contribute to the Executive Board’s consideration of the subject, but they are not comprehensive. After discussing the evaluation’s rationale for Fund research and its categorization of the types of research that should be conducted in the Fund, we take up the suggestion for a committee to set research priorities and provide funding for collaborative research with outside researchers, along the lines of the World Bank’s Research Committee; we then comment briefly on the evaluation of staff performance and the evaluation group’s remarks on the review process, which was also discussed in the external evaluation of surveillance. We will not comment here on questions of a personal nature.
We of course share the view of the panel that high-quality research is critical to the successful operation of the Fund, in both its surveillance and its operational roles, and share the rationale for that view set out in paras. 24–25. The report defines (para. 21) three categories of Fund research: policy foundation research, which develops basic analytic tools and frameworks (e.g., “Are Currency Crises Predictable?”); policy development research, which creates the broad policy framework (strategy) that guides Fund operations (e.g., “IMF-Supported Programs in Indonesia, Korea, and Thailand: A Preliminary Assessment,” “Dollarization: Implications for Monetary Policy”); and policy analysis research, which provides policy advice in the course of the day-to-day operations (tactics) of the Fund (e.g., “Inflation Targeting in Korea: An Empirical Exploration”).
In considering this categorization, it may help to think of Fund research as operating along a continuum, at one end using current analytic methods to analyze economic developments in the world economy and individual countries, at the other end undertaking abstract research, with no obvious immediate applicability to Fund operations. For convenience, we can divide this line into four categories: (1) analysis of and reporting on current economic events and problems, such as the WEO and International Capital Markets report (ICMR) as well as some of the work done in the context of Article IV consultations that is presented in Recent Economic Developments papers and other background documents; this could be called “surveillance research”; (2) analytic and empirical work on cross-cutting policy issues confronting the Fund in its operations, such as the forthcoming paper on exchange rate systems or the work on dollarization, and many of the papers on capital account liberalization, and on inflation targeting—corresponding roughly to the evaluators’ “policy analysis” and “policy development” research; (3) research on new ways for the Fund to think about or deal with problems, some of which may not yet have been encountered in the Fund’s operations—this appears to correspond to the evaluators’ “policy foundation research”; and (4) abstract research, with no obvious relevance to Fund operations, but which if well done could be published in academic journals. Roughly corresponding to the movement along this line is a shift from research that would be expected or requested by the Executive Board, management, and non-Research Department staff, to research that would most likely be motivated by the Research Department and by individual researchers.
Our own evaluation, shared by the external evaluation of research, is that the Fund’s surveillance-research work, is generally first-rate. We have in mind particularly the WEO, the ICMR, the work presented at World Economic and Market Developments sessions, and the surveillance notes produced for different global and regional forums. Both the external evaluation of research (para. 34) and the external evaluation of surveillance record that this work is very highly regarded outside the Fund as well. The analytic work that serves as an input to country surveillance and operational work is bound to be of more variable quality, but we believe that too is generally at a high level. The evaluators comment that Fund staff should be undertaking less of this work on countries in which good policy research is already being done in central banks or other research organizations. While we understand the motivation for this argument, and agree that Fund staff is likely to add relatively less to a policy discussion with a country where the policy debate is well advanced, we should also recognize that effective Fund surveillance requires the staff to form its own views on issues that will be the subject of Article IV discussions. Although the Fund will be only one voice among many in the policy debate in the advanced countries, Fund staff needs to have a well-informed and analytically sound capacity to undertake a policy dialogue with the authorities, both because such countries are systemically important, and because it is important not to create a dual-track surveillance process. That is why staff will have to continue to undertake surveillance research on such countries. And, having undertaken it, they should publish it.
The evaluators, after commending the Research Department’s surveillance research work, focus their attention on the remainder of Fund research, with which they were less impressed. While it is natural for such an evaluation to focus on perceived weaknesses rather than strengths, the external evaluation’s appraisal does not sufficiently recognize that surveillance research constitutes a large part of the output the Research Department is expected to produce; further, as the Director of Research notes in his comments on the external evaluation, he was explicitly charged on his appointment with shifting the output of the Research Department in the direction of this type of work. This has been done extremely well, and the value of the Research Department’s work in this area is widely recognized among the Fund’s membership, both by members of the Executive Board and by many other officials with whom we have spoken. Management too regards the regular publications of the Research Department and Board briefings by the Director of Research, and the contributions based on this work that the Fund makes to policy discussions in regional and G-7 forums, as playing the central role in the process of multilateral surveillance, and doing so extremely well.
Our second category of research, on cross-cutting issues confronting the Fund in its operations, corresponds to the external evaluation’s policy analysis and policy development categories of research. The external evaluation of research devoted a considerable effort to the evaluation of this research, recording mixed views. Rather than express a view on that appraisal, we want to make three comments that might help set it in perspective. First, most research papers written in any institution or academic department have weaknesses; only a minority of papers that are written are accepted for publication, and of those, only a few have any shelf life. Second, one of the main reasons to write a paper is to gain a full understanding of a topic, which reading alone rarely achieves. It is for this reason that the Fund does not exercise much control over the papers that appear in the Working Paper series. In this regard, we do not favor curtailing the issuance of Working Papers; there should be a strong presumption in favor of publication, not only because researchers who have done the work deserve to have it see the light of day, but also because the staff’s work benefits from being subject to public scrutiny. We do, however, share the evaluators’ concern that such papers be clearly identified as working papers; we should avoid any implication that might inappropriately elevate the status of these papers. Third, we are not sure how to categorize an extremely important part of Fund policy work that may not have been sufficiently taken into account in the external evaluation of research—namely the policy papers, typically coming out of the Policy Development and Review Department, in which Fund staff develop a practical framework for implementing approaches or policies whose general nature has been prescribed or suggested by the Board. Examples include the papers on private sector involvement now before the Board, or those written as the Fund developed the Contingent Credit Line Facility. We regard Fund staff as unparalleled in this particular, critical, skill, best described as policy development work, which frequently involves an iterative process with the Board.
Nonetheless, Fund work in both this second category of cross-cutting operational research, and that on policy foundation work, needs to be strengthened. This requires (1) priority setting and (2) strong researchers and the incentives for them to do the right work. We support the external evaluation’s recommendations both that the balance of Fund research should be tilted toward areas in which the institution has a particular need or should have a comparative advantage, such as cross-country research and that on emerging market and developing countries, and that the Fund should have a more systematic procedure for setting research priorities. We agree too that we should consider establishing a Committee on Research Priorities (CRP). The current Working Group on Fund Policy Advice (WGFPA) serves more as a coordinating committee than as a setter of priorities. The proposed CRP could meet twice a year, and should include senior staff of departments (most of those in the Fund) in which research is demanded or supplied or discussed, and could be chaired by management. If a meeting were to be held soon, it would include on the agenda, for example, research on capital account liberalization, exchange rate systems, private sector involvement in the resolution of financial crises, and aspects of financial sector behavior. We would need also to consider whether the WGFPA, whose membership includes staff more actively involved in research, should continue to exist. If it does, it could include on its agenda a discussion of progress in priority research areas.
We strongly welcome the call by the external evaluation of research for more interaction between Fund staff—not only those in the Research Department—and outside researchers, including more frequent attendance by Fund staff at academic conferences, and by academics at Fund conferences. We need also to seek closer interactions with researchers outside North America. We would like to see more joint research with the World Bank, but would not recommend institutionalizing this effort; coordination is a highly resource-intensive activity. We are also not yet convinced of the value of providing either the CRP or the WGFPA with a budget to fund research by consultants, particularly if the research would not be done in the Fund. Before deciding whether to accept this recommendation, we should both ask what marginal contribution to the creation of the desired knowledge would be made by such financing and carefully evaluate the World Bank’s experience with this approach. We agree that the Fund would benefit from having more visiting researchers, and the process by which they are selected should be uniform across departments and transparent.
Much of the external evaluation of research is devoted, rightly, to what the Fund should do to attract, encourage, and retain high-quality researchers. We believe the incoming Economist Program groups typically contain a good proportion of economists interested in and able to do good research. We should expect graduating Ph.D.s who choose the Fund as a career to tend to be more interested in the applications of economics and in policy issues than those who choose a purely academic career, but that is all to the good since that type of research should be particularly highly valued in the Fund. No doubt in any Economist Program group there will be some more interested in a longer-term research career, and some of these researchers will want to work on topics in the fourth category—those of no obvious immediate relevance to Fund operations. Some research of this type should be permitted, provided the staff member is also doing other work more directly relevant to Fund operations. As the external evaluation recommends, we should make it easier for those interested in and good at research to spend longer in the Research Department, whether on their first round in the department or after a period of mobility. We also need to recognize that it is extremely difficult for a researcher to stay current in his or her field if not continuously involved in it. This implies at least that supervisors should enable such researchers to have more time for research. We note the ideal of setting aside continuous blocks of time suggested by the external evaluation, but there are typically interruptions even in academic life, for instance to teach classes. The problem faced by Fund researchers may not be so much the continuity of time available for research as the shortage of time, given the general work pressures to which staff—particularly the best staff—are exposed. Any initiatives in this area, as well as those discussed in the next paragraph and the next to that but one, would need to take due account of the more general implications that are likely for the Fund’s operations.
The Fund needs also to attract more established researchers, somewhat later in their careers than are incoming Economist Program participants. Some midcareer researchers could come through the visiting scholars program, but we also need the capacity to appoint excellent researchers to a more permanent position at a high rank in the Fund, when such opportunities arise. This reopens the question of finding a mechanism by which outstanding individuals, whether from outside or inside the Fund, who would not be placed in a managerial position, might nonetheless qualify for a B-level appointment. While all these matters are under consideration, we should also bear in mind the Board’s frequently expressed belief, which was supported by the External Evaluation of Surveillance, that we need to attract more midcareer individuals who have had policy experience.
As to the other incentives for research suggested by the External Evaluation of Research, we agree that names of staff who contributed to a paper should be recorded on it, as is increasingly being done now. We would not, however, wish to remove the signatures of the heads of department in which a paper was prepared, for that provides an important element of accountability. As to the suggestion that papers should be presented to the Board by those who prepared them, departmental practices on this point differ. We can see both the pros and cons of the evaluation’s proposal, among the latter the fact that when a policy position is proposed in a paper, we expect the head of the department to be responsible for and willing to defend the position. We have the impression that papers more in the nature of research, for instance some of the recent papers on capital controls, are more often presented by their authors. Possibly this suggestion should be handled on a case-by-case basis.
The research evaluators suggest that annual performance appraisals should include an appraisal of research carried out by the staff member. We believe that is already being done. The evaluation’s more wide-ranging suggestion that the grading on Fund performance appraisals should be tougher is under review, but it should be noted that managers tend to use the salary increase as a more accurate measure of performance than the numerical grade.
The external evaluation of research, like the external evaluation of surveillance, believes too much time is spent on review, and that the reviews are too formal. As noted in our comments on the External Evaluation of IMF Surveillance, we too believe that too many resources are devoted to the review process, and that departments should use the “no comment” option more frequently. It is also likely that review within some departments is too staff-intensive (too many people asked to review a paper, too many meetings to discuss the review), and resources could be saved there. But the review process is a critical input into the formation of a staff policy consensus, and the obligation of the initiating department to note divergences of view among departments when sending a paper to management for clearance ensures that major differences of view can be discussed and a choice among them made. A system of informal oral comments would not work, and would leave open the possibility of disputes about who said what when. Writing down comments helps produce an appropriate degree of seriousness by both reviewer and recipient: comments should be pointed, brief, and written.
We look forward to the Board’s discussion of the external evaluation of research and to following up on the recommendations that emerge from it.