Staff Response to the External Evaluation of the IMF’s Economic Research Activities
- International Monetary Fund
- Published Date:
- April 2000
The staff appreciates this far-reaching and thought-provoking report on research in the Fund. The report provides a useful outside perspective on the IMF’s research and a number of sensible suggestions for improving it. The report should be taken seriously, and it is in that spirit that we record points of both agreement and disagreement and explain our disagreements where they exist.
While many of the recommendations resonated with the staff, departments questioned the basis for others, either finding the evidence marshaled for a change from current practices unconvincing or thinking the report did not pay sufficient heed to the broader objectives that research serves in the Fund or to the constraints under which staff operates. Staff believes that the value of the Fund’s research products needs to be assessed relative to the priorities set by management and the Executive Board and, more fundamentally, by the Articles of Agreement. It is clear that the Fund’s research on various aspects of the economies of industrial countries, such as on structural unemployment, monetary policy frameworks, and public pension plans, makes a major contribution to the surveillance process and the policy debate, whether or not other research on these topics already exists. In this regard, we encourage the Executive Board to provide guidance on whether it agrees with altering priorities in the direction advocated by the external evaluators.
A general concern of staff is that the evaluators seem not to have given adequate attention to key factors that necessarily and desirably affect the orientation and organization of research activities in the Fund. Specifically, beyond their general contribution to knowledge, these activities are supposed to support and contribute to the operational activities of the Fund. A substantial portion of Fund research is oriented toward policy issues faced by individual members. This reflects the fact that the Fund’s operational work in the key areas of surveillance, financial programs, and technical assistance is largely with individual members—rather than with multicountry groupings. Similarly, the organization of resources for research distributes them across the area departments and functional departments, rather than concentrating them in a single department or under the control of one interdepartmental committee. This organization facilitates the linkage between research and the Fund’s more basic responsibilities. As the evaluators suggest, this orientation and organization probably have some adverse effect on the value of Fund research—viewed as an isolated product as seen by other researchers outside of the Fund. However, the contribution from this orientation and organization to other important work receives little or no attention from the evaluators. This is fundamentally an issue of achieving the right balance, which requires weighing appropriately all of the relevant objectives of the Fund’s research activities.
This response to the evaluators’ report is a compilation of the departmental comments. It first covers general issues of methodology and mandate, then considers the details of the evaluation of existing research presented in the report, and finally discusses reactions to the specific recommendations.
Definition of Research and the Scope of the Evaluation
How to define research is a thorny issue, and the choice between a broad and a narrow definition has a great impact on what outputs need to be evaluated and the extent of staff resources devoted to research work. Unfortunately, the report is somewhat ambivalent on the appropriate definition, adopting the broad definition at times, thus including a (favorable) discussion of the World Economic Outlook and International Capital Markets reports, and at times a narrow one, when Working Papers are evaluated from an academic perspective of originality and publishability in a good journal (e.g., para. 40). Several departments noted that some of their “research,” embodied in annexes to consultation reports that also appear as Working Papers, does not pretend to be original research but rather is a summary of the literature serving to focus the consultation discussions and to inform the Executive Board. There is also research that does not (nor is it meant to) produce a publishable product, but it may have an ancillary benefit of helping the researcher to develop his or her knowledge and skills.
The evaluators argue that the Fund does not, and should not, try to produce all the research relevant for its needs. The staff agrees. However, the evaluators go on to say, “it should use relevant research that already exists outside, without duplicating it” (para. 27). In the Research Department, for example, a good deal of work continues to be done on what is known in the research literature as “fundamental equilibrium exchange rates” (FEERs). In view of the Fund’s mandate to exercise “firm surveillance” over members’ exchange rate policies, research on FEERs inside the Fund is highly relevant, if not vital, even if it overlaps with research done outside the Fund. Indeed, it would be senseless to argue that the Fund staff should work only on research issues that nobody else finds interesting or worthwhile. Surely, the evaluators do not suggest this; but barring an absolute standard to avoid all duplication, it remains unclear what the evaluators really mean and how it is to be judged whether Fund research satisfies the evaluators’ criterion.
A related problem for the evaluators was how to situate the research work in the context of the Fund’s other, more operational work. Clearly the terms of reference did not authorize the evaluators to look at all aspects of the Fund’s work and their relative priorities—a daunting task in any case, far beyond the capacity of a three-man team with total resources of six person-months. However, the report does stray far beyond a strict look at research, in particular in considering the interdepartmental review of Fund operational papers, in calling for broad changes in the annual performance review (APR) process, and in recommending substantial changes in the management structure of the Research Department. Staff felt that these aspects of the Fund’s work were not well understood by the evaluators and went well beyond their mandate. In addition, the discussion of the staff mobility requirement (paras. 70–72) does not adequately take into account the other goals of the Fund (in addition to the objective of producing good research) that are relevant when considering changes to this personnel policy.
In contrast, many on the staff thought that the central question of the terms of reference of the evaluators, “whether economic research in the IMF contributes effectively to the Fund’s objectives,” was not adequately assessed and not given a clear answer. In particular, “Are there any major areas of research at present undertaken by the staff whose value added could be regarded as insufficient? Are there major omissions in the Fund’s research agenda?” The nuanced response of the evaluation report seems to be that on some counts many Working Papers and publications are not up to academic standards, but a significant number are; whether those that are not up to academic standards contribute otherwise to the Fund’s overall objectives received little attention. Indeed, the staff believes that even cursory examination of titles and abstracts of Fund research papers confirms that the overwhelming bulk of this research is relevant to the Fund’s mandate, mission, and interests, and that much of this research is, appropriately, quite different from the average academic research paper. Aside from a general reorientation of research activities away from industrial and toward developing countries and transition economies, and the need to do more cross-country and financial sector studies, there are no specific areas identified by the evaluators that the Fund’s staff was judged to have missed. Finally, the dissemination of Fund research could have received considerably more attention, with practical and constructive suggestions for improvement.
Insufficient attention to the other objectives of the Fund in addition to producing research may also have led to judgments about the hierarchical nature of the Research Department. It is worth noting, however, that it is no more hierarchical than other departments, which reflects that the department also has important operational responsibilities (this is discussed in the annex to this staff response, prepared by Mr. Mussa). And the decentralized nature of the working arrangements across departments would make it difficult for the allocation of resources envisaged by the Committee on Research Priorities to work without major organizational changes, desirable though it might be. In fact, some departments felt that decentralization produced more original research of higher quality, and contributed to job satisfaction, relative to the extra bureaucracy embodied in the proposed Committee on Research Priorities. An annex to the report on the experience of the World Bank with an internal market for research might have been useful in this regard.
The Evaluation Process and Criteria
The report’s evaluation of research is based on personal interviews and on an assessment of research output. While the evaluation process is necessarily selective, the external evaluators’ approach suffers from several shortcomings with potentially important implications for the report’s conclusions (or at least the foundation for these conclusions).
First, the interviews with policymakers outside the Fund do not seem to have included any interviews with policymakers in Africa, the Middle East, and transition countries of central and eastern Europe. Overall, the staff is concerned that this lack of regional coverage may affect the external evaluators’ assessment of important segments of the research activities undertaken in the Fund, notably with respect to developing and transition countries. Our information is that in developing and transition economies, Fund research does receive a good deal of attention both inside and outside the official policy community.
Second, in assessing the research output, the report seems to be too selective for arriving at well-founded conclusions. In particular:
The evaluation criteria were slanted toward academic research. This bias reflects a serious misunderstanding about the intended audience of a substantial amount of Fund research. As an illustration, the assessment by the Asia and Pacific Department of the relative usefulness of the inflation targeting papers on Korea and Philippines is opposite to that suggested in Box 1 of the report. In particular, the paper on the Philippines has helped to promote a dialogue among policymakers, illustrating that Fund research needs to be evaluated from a wider perspective than a strictly academic one.
The external evaluators did not assess the outside publication record of Fund staff (this would have covered papers that are never published as Working Papers, but also Working Papers that are eventually published outside the Fund, and cited as such).
The sample of Working Papers that is evaluated could have been broadened to get a better perspective (not focusing almost exclusively on 1998 and covering a larger number of papers). The external evaluators apparently did not assess whether the Fund had produced research that had a substantial impact in its areas of primary responsibility. Because a period of at least three or four years is usually required to assess whether a paper has such an impact, the sample of recent papers examined by the evaluators, which contained only very recent papers, is not really adequate.
The report’s conclusions could have been strengthened by including some comparisons with the research output of other institutions.
Third, the report does not adequately recognize the range of research topics and activities undertaken in the Fund. For example, it suffers from a relative lack of attention to research on transition countries and fiscal issues, which play a central role in fulfilling the Fund’s mandate. It also underplays the already considerable research of the Monetary and Exchange Affairs Department and others on financial sector issues.
Comments on Recommendations
Staff welcomed a number of the recommendations, with many departments suggesting that they deserved serious consideration. While departments commented on many of the 9 key recommendations, relatively few comments were made on the 13 supplementary recommendations. The following highlights the areas of disagreement with the report’s recommendations, in particular with respect to the 9 key recommendations and the 5 more controversial supplementary recommendations.
Recommendation 1: Create a Committee on Research Priorities (CRP) to identify Fund-wide research priorities, provide resources in support of specific research topics, decide on visiting scholars, and hire outside consultants. The external evaluators note that there is already such a committee, the Working Group on Fund Policy Advice, but that their recommendations are intended to make it more effective and extend its scope. Staff thought that these were useful suggestions, but expressed some skepticism about whether this would be an improvement on current practice.
While some departments noted that there could be scope for a somewhat more centralized approach to setting research priorities, others noted the need to strike a balance between centralized decision-making and coordination and the need to shape their own research programs. There was already a high degree of coordination in determining the work program for the Executive Board. Area departments were particularly concerned to maintain the necessary freedom and flexibility to deal with country-specific research in the context of their surveillance activities. Many departments observed that a more centralized approach, while potentially yielding benefits, also entailed significant risks; that is, they were concerned that a more centralized approach might stifle individual initiative and creativity, adversely affect the timeliness of research, and reduce flexibility. Some were concerned that the proposed CRP might only lead to another layer of bureaucracy.
Many departments stressed that the modalities of the proposed committee would need some further fleshing out. A common concern was how the CRP would obtain and allocate resources. Some doubted that it was feasible to set up an independent body with the power to commit resources, unless the committee was comprised of department heads who control resources. Many stressed that it would be essential to maintain substantial resources within departments in order to maintain flexibility and ensure that they could meet their responsibilities toward individual countries and provide the needed support for operational and policy work.
Recommendation 2: Introduce explicit departmental targets for staff time allocated to research. On balance, departments did not display much enthusiasm for this recommendation. Most departments noted that explicit targets on one activity (research) would be difficult to implement without a broader assessment of other competing activities, given the pressures on resources. If it were to be a binding constraint, either additional resources would need to be available to departments or it would have to be accepted by management and the Executive Board that there would be a reduction in other activities.
Recommendation 3: Shift the mix of research toward topics adding the most value—developing country, cross-country, and financial sector research. There was substantial support for the view that more cross-country (including regional) research should be undertaken. However, some departments felt that the case for shifting the mix toward developing countries (to be defined to include transition countries) required more justification than had been given in the report. Others felt that the report was in a way “preaching to the converted,” as there already had been a substantial increase in the number of projects devoted to developing country and financial sector research. In supporting more research in these areas, though, departments were less convinced that the evaluators had come to grips with the types of research that were most beneficial to the Fund or that a shifting mix should imply less emphasis on other types of research. They considered that there should be continuing emphasis on background research that, while hardly original from the academic perspective, resulted in broad syntheses or eclectic interpretations that were valuable in providing a focal point for discussions with the national authorities as well as informing both country and Executive Board policy discussions. Even if the mix were to shift somewhat, the need to support the Fund’s very substantial surveillance requirements similarly meant that country-specific research and research on industrial country issues would remain a major focus of Fund research.
Recommendation 4: Create incentives to improve collaboration and to encourage researchers to contribute to policy work. Research staff would receive credit in the annual performance review for providing service to other departments. A form of internal market would be created to facilitate the participation of functional department staff in policy development and mission work. Among those departments that commented on this recommendation, there was support for recognizing a researcher’s service to other departments, although some asserted that this was already the case, for instance through assessments by mission chiefs, which are typically included in the APR. On the proposal to create an internal market, similarities to the World Bank model were noted, and the latter seemed to have led to a shift from research being done by Bank staff to consultants, which was not viewed as desirable.
Recommendation 5: Improve the assessment of research quality in the APR. All departments commenting on the recommendation supported it. Two departments noted that they already did give explicit attention to the quality of research that was conducted.
Recommendation 6: Give opportunities to all staff to present research products to the Board and management. Some departments felt that the proposal was a good one in principle, but might be difficult to implement given constraints on the Board’s time. Junior economists have already some scope for presenting papers to Fund seminars attended by management and Executive Directors, such as a recent one on China.
Recommendation 7: Role of the Director of Research. Departments did not understand the purpose of the recommendation, since its description of the role of the Director of Research seemed to describe the current situation. See the Annex to this staff statement for an extended discussion.
Recommendation 8: Create a more effective performance evaluation system. Departments noted not only that this area was beyond the evaluators’ terms of reference, but also that the evaluators had only superficially looked at one aspect of the Fund’s performance evaluation system and not the system as a whole. As in most systems, the overall rating is only one element of performance evaluation and differentiation between high and low performers. Outstanding performance is primarily recognized through a “1” rating, and for those doing research, such a rating depends on research quality as well as quantity. This rating can be given to no more than 15 percent of the staff. The large majority of staff are rated “2.” However, within this “2” rating, performance is carefully differentiated and this differentiation is reflected in salary increases of individual staff members. Weaker performers are encouraged to look for positions elsewhere or, at a later stage of a Fund career, when most of the performance problems arise, to take early retirement from the Fund. In the majority of these cases separation is brought about without the necessity for a “3” rating. A “3” rating is normally given in those cases where performance deficiencies require formal performance management, including close performance monitoring, by the department concerned in cooperation with the Human Resources Department. The purpose of this performance management is to help ensure that performance reaches required standards or, failing that, to bring about separation. Another important aspect of the Fund’s system is the initial appointment of all new staff on a two-year fixed-term basis, which allows for careful assessment of performance and the suitability for longer-term Fund employment before conversion to regular staff status. A significant number of the initial fixed-term appointments are not converted to regular status and in most cases nonconversion does not involve a formal “3” rating. In all, 15–20 staff members, both on fixed-term and regular appointments, have been separated annually in the past few years because of weak performance.
Recommendation 9: Reduce unnecessary internal review of Fund work and avoid formal written comment where informal oral communications would be adequate. Staff agreed that the objective of making the review process more efficient was a worthy objective and deserved more systematic study. However, such a study needed to take into account that the Fund had functions in addition to research. Some were skeptical that oral commentary and informal procedures could be effective. For example, it was noted that review was a vehicle for spreading information across departments as well as a fundamental vehicle for quality control. It was suggested also that discipline was strengthened since reviewers had to read carefully and authors were held to account when commentary was written.
Recommendation 13: Introduce more flexibility into the hiring procedures for entry-level economists. Most departments expressed their reservations, and questioned whether it was in the best interests of the Fund to hire staff whose only career goal was research; and they wondered whether creating a special hiring procedure for research-oriented staff risked divorcing the Research Department from the rest of the Fund. They noted that recruitment at the Fund was intended to meet the larger goals of the Fund, not just its research goals. However, the suggestion of job seminars was judged to be useful in some cases.
Recommendation 14: Consider streamlining the management of the Research Department. Departments did not agree with the report that the management structure of the Research Department was unnecessarily hierarchical. Some commentators thought that the evaluators had misunderstood the management structure and others suggested that they had not analyzed the question in sufficient depth and the recommendation needed to be rethought. In their view, it was important to recognize that many non-research types of activities also were performed within the Research Department. These points are discussed in more detail in the annex to these comments.
Recommendation 15: Write and disseminate nontechnical summaries of highest quality and most relevant research. This was the practice previously for Working Papers; only recently were summaries of Working Papers eliminated as being duplicative of abstracts and introductions. The more selective approach advocated, and their circulation to management and the Executive Board, deserves consideration, however.
Recommendation 16: Treat Working Papers as preliminary. Departments were relatively evenly split over the merits of this recommendation. While some welcomed a more informal approach, others felt that there was little to be gained since Working Papers already were subject to relatively minimal scrutiny. Some questioned whether a completely unreviewed paper should be issued under the Fund’s logo as, in their view, any institution was inevitably going to be judged by its products—even when they were accompanied by long disclaimers. For them, some quality and confidentiality screening was viewed as being essential. It was also noted that the evaluation report was somewhat inconsistent, criticizing the quality of Working Papers and the fact that they were a “final product” not leading to further publication, while at the same time advocating the abandonment of the quality control provided by division chiefs’ approval.
Recommendation 17: Create a new vehicle for non-senior staff to make presentations to management and the Executive Board. In principle, staff supported this idea, but questioned whether the Board would find the time.
Beyond its general appraisal of research in the Fund, the evaluators’ report offers comments and recommendations directed specifically at the Research Department (RES)—its role and mandate, operation, organizational structure, and management. It is appropriate for the Director of the Research Department to respond to these aspects of the report. In doing so, it is important to emphasize both that research activities in the Fund extend well beyond the work in RES, and that the activities of RES involve much more than research.
Indeed, RES bears the primary responsibility within the staff for assisting Fund Management and the Executive Board with their responsibilities for multilateral surveillance as mandated by the Articles of Agreement. This work includes preparation of the World Economic Outlook (WEO), the International Capital Markets reports (ICMR), the G-7 and other surveillance notes, the regular World Economic and Market Developments (WEMD) sessions for the Executive Board, policy papers for the Executive Board, and the analysis of exchange rates and of multicountry macroeconomic models. Staff in RES spend significantly more of their time on these and other operational activities (and on closely related research) than they do on research projects selected at staff initiative.
Clearly, an evaluation of RES that seeks to reach broad conclusions about its appropriate role and mandate and to make relevant recommendations about its operations and management needs to look in considerable depth and detail at all of the department’s activities, within the broader context of the institution of which it is a part. This, of course, was not the focus of the evaluators’ report, which is concerned with the research activities of the entire Fund, including part of the activities of RES.
To put the recommendations in context, therefore, it is relevant to take note of the report of the outside evaluation of the Fund’s surveillance activities,1 which was conducted separately but essentially simultaneously with the evaluation of research activities.
The Fund’s published work on multilateral surveillance is widely recognized as being of high quality. We learned of many instances where the WEO was a basic source document and building block for officials engaged in monitoring and forecasting international developments. The ICMR was also highly rated, though clearly less widely known and used. While it appealed to a narrower audience than the WEO, it was particularly appreciated by those interested and/or involved in assessing international financial developments as bringing more analytical substance to the review of issues than is typically found in other coverage available. The same seems true for WEO as well, although it has more direct competition from other public and private publications than does the ICMR. (External Evaluation of IMF Surveillance, Chapter III, para. 47).
Although much less known than WEO and the ICMR, the WEMD sessions also constitute an important element within the process of multilateral surveillance. In these sessions, Executive Board members, selected staff members, and management engage in relatively open and informal discussions of issues. These can range from the most recent developments in the international monetary system to an assessment of vulnerabilities in different countries around the world. Many participants have rated those informal meetings among the most interesting and important of Board meetings, and those eligible to attend are keen to do so. (External Evaluation of Surveillance, Chapter III, para. 54).
In contrast, the main conclusion concerning RES in the evaluation of the Fund’s research activities may be summarized as follows: “A culture shift in the Research Department would raise morale and help to strengthen the contributions of the Department to the policy development process.” Recommendation 14 calls for significant downsizing of the management structure of the department, which would have the effect of eliminating the senior staff responsible for planning, organizing, and supervising most of RES’s work on multilateral surveillance. Recommendation 7 urges explicitly that management alter the mandate of the Economic Counselor and Director of RES to orient the department’s activities toward a much heavier focus on research per se.
In responding to this central recommendation concerning RES, and to the broader assessment of which it is a part, I want to emphasize that I share the position of the evaluators on two key points. First, in accordance with Article XII, Section 4(b) of the Fund’s Articles of Agreement, it is the responsibility of Fund management, under the general policy guidance of the Executive Board, to determine the organization and establish the mandate for the activities of the staff, including the Research Department. Second, among all departments in the Fund, RES ought to have a mandate that puts particular emphasis on what the evaluators call “policy foundation research” and, to a lesser extent, on what they call “policy development research,” and that RES’s research should put relatively less emphasis than that of other departments on “policy analysis research,” which is more directly and immediately linked to the Fund’s operational work.
Interestingly, but not surprisingly, the facts demonstrate that this has indeed been the relative orientation of research in RES in the past few years (and before). Specifically, Table 1 of this Annex reports on the distribution of research papers written in the Fund into three categories: general research that is not country-related; country-related research that examines several countries; and country-related research that focuses on a single country. The distribution of research into these categories in RES is compared with the distribution in all other departments in the Fund. In RES, about half of all research is not country-related; it focuses on more general analytical issues. In other departments, less than 30 percent of research is in this category; and the vast bulk of research is country-related and particularly single-country-related. Also, for country-related research, the ratio of multicountry to single-country-related work is three times as high in RES as the average for all other departments in the Fund. This does not speak ill of other departments. With the resources they have to devote to research, they are using them in a manner that is closely linked to their particular responsibilities. Similarly, as one would and should expect, the resources available for research in RES are effectively deployed in a manner that is qualitatively quite different from that in other departments.
|(Number of papers)|
|IMF minus RES|
|(in percent of research papers of relevant group)|
|IMF minus RES|
|Memorandum item: ratio of cross-country to country-specific|
|IMF minus RES||0.4||0.5||0.6||0.5||0.5|
In my view, if RES is to do its job, within the context of the institution of which it is a part, it must undertake and properly balance three fundamental activities: work on vitally important operational issues, especially those related to multilateral surveillance (including related research); research that concerns the intellectual foundations of the Fund’s activities or assists in understanding and developing the Fund’s approach to dealing with its responsibilities; and other (mainly nonresearch) activities that contribute to the work of the Fund. If RES has no significant operational responsibilities and no more than very limited contact with the Fund’s nonresearch activities, then its research will inevitably tend to lack relevance to the institution’s principal operational concerns; and, relevant or not, its research would tend to be ignored by other departments and the institution as a whole. In this environment where operational work is mainly the responsibility of other departments, it is crucial that RES actively engage in policy debate on key issues facing the institution.
For RES, which has these multiple responsibilities, tension inevitably arises over the allocation of resources among different activities and, as a separate but related matter, the relative importance and prestige that are perceived to be assigned to the different activities carried out by staff in the department. The evaluators of the Fund’s research activities clearly believe that both the allocation of resources in RES and the perceptions concerning their relative importance and prestige should be shifted substantially in the direction of independent research on policy foundations and policy analysis. Before turning to these issues for the future, where to a degree I share the views of the evaluators, it is useful to reflect on the past.
First, I would assert that during the past eight years, the total contribution of RES to the work of the Fund, through its research and other activities, has been massively upgraded—well beyond the increase in resources available to RES. It is true that this massive upgrading has been particularly important in the Department’s contribution to the more operationally oriented activities of the Fund, including through research related to these activities. Research that is not so related has tended to suffer somewhat, and the morale of some staff concentrating on such research has probably been adversely affected. But, the bottom line is the total contribution of RES to the work of the Fund. Here I believe that the evaluations of a representative and well-informed group of people familiar with the work of RES would share a strong consensus that RES makes very important and valuable contributions to the work of the Fund, and has done so increasingly in recent years. Indeed, a recent survey of RES staff clearly reveals that that is overwhelmingly the opinion of staff in RES as well, despite concerns about the curtailment of time available for staff-selected research (see Table 2 of this Annex).
|1.||On average, in the past two-three years (or less, if your tenure in RES has been lower), time allocated across the different tasks I performed was: (denoted by “Actual” below). The time that I would like to allocate across the different tasks I perform is; (denoted by “Desired” below).|
|Individual research work (including training, conferences, and seminar attendance)|
|Directed research work (Board papers, WEO and ICM report writing, etc.)|
|Review work (including short-term policy notes)|
|Missions and miss ion-related work|
|Management of staff and other administrative work|
|2.||The Research Department makes a valuable overall contribution to the Fund’s work.1|
|3.||I am able to complete the tasks assigned to me within normal working hours (40 hours per week).1|
Average of responses where 6=Strongly Agree, 5=Agree. 4=Tend to Agree, 3=Tend to Disagree. 2= Disagree. I=Strongly Disagree. To put the scales in perspective, the average score obtained overall 30 questions was 4.2. The highest and lowest scores obtained were 5.5 and 1.6, respectively.
Average of responses where 6=Strongly Agree, 5=Agree. 4=Tend to Agree, 3=Tend to Disagree. 2= Disagree. I=Strongly Disagree. To put the scales in perspective, the average score obtained overall 30 questions was 4.2. The highest and lowest scores obtained were 5.5 and 1.6, respectively.
Second, this shift in the orientation of the work of RES is not an accident. When I was hired in the summer of 1991, Fund management made clear that they wanted the new Director of RES to increase substantially the effective contribution of the Department to the operational work of the Fund, while sustaining improvements that had been made in the Department’s research efforts. In addition to their own view, management was also clearly expressing the desire of many Executive Directors. My approach to accomplishing this agreed objective was to focus attention on those activities where RES had some leverage to influence the operational work of the Fund. The evaluators note in their report the difficulties in getting area department staff (who have the greatest direct influence on the Fund’s operational work) to make use of research papers written in other departments, including RES. This is not a new problem—and not one that can be made to disappear. I was determined to use and to enhance the WEO, the ICMR (for which RES had just been given primary responsibility), the WEMD sessions, the surveillance notes for the G-7 (later extended to several other groups), and other products produced directly by RES as the principal vehicles for strengthening RES’s contribution to the work of the Fund. This included using these vehicles as effective mechanisms to derive policy impact from an important part of the research done in RES. Active use of the unique access of the Economic Counselor (among all staff in RES) to senior-level policy discussions with Fund management, Executive Directors, senior staff, and key policy officials in member governments was also part of the strategy. RES’s work on the review process was narrowed to a limited group of countries to facilitate a substantial deepening of this work for the countries covered. The intent was threefold: to have a greater impact on policy issues in the context of the Fund’s country work; to enhance the awareness of economists in RES about policy issues that might stimulate their research efforts; and to establish contacts between economists in RES and other departments through interactions in the review process that would prove valuable in research and in career development.
Third, the consequences of the shift in the orientation of work in RES were not inadvertent. Decisions about the allocation of resources have costs as well as benefits. The emphasis on developing and upgrading the WEO, the ICMR, the WEMD sessions, the surveillance notes, and related activities (including research connected with these products), and on the review process have undoubtedly taken staff resources away from more general research that does not have such an immediate link to RES’s operational activities. Indeed, beyond the general shift in emphasis in RES’s activities since 1991, I have taken a number of specific decisions that have tended to heighten these costs, particularly during the past two years. For example, at my instigation, with the concurrence of management and the Executive Board, an Interim WEO on the Asian crisis was produced in December 1997, and in the wake of the Russian and LTCM crises, an Interim WEO/ICMR was produced in December 1998. Such activities diverted staff from other tasks, including especially research. In the environment of financial and economic crises that engulfed many of the world’s emerging market economies and that posed important challenges for the Fund, I believe that balance between benefits and costs from these specific shifts in the use of resources in RES yielded a substantial net benefit to the Fund and its membership.
Also, for many years, I have pressed the development of RES’s work in the capital markets area, sometimes against significant opposition from outside RES, including some members of the Executive Board. This was based on the shared view of key staff in RES that issues relating to financial markets and institutions, including their supervision and regulation, were of vital importance for the Fund (and its members) but were, unfortunately, receiving seriously inadequate attention. At the height of the effort to impress this point on the international community, between the summer of 1996 and the summer of 1997, significant staff time in RES that would otherwise have been available for research was committed to this effort. Again, I believe that the benefits already have, and surely will in time, substantially outweigh the costs; but there have been costs.
In contrast, by choice rather than accident, RES has not been deeply involved in a significant number of important operational and policy issues that plausibly lie in or near the department’s domain of responsibility. For example, in the mass of recent work on reforming the architecture of the international monetary system, RES has focused its efforts on the main conceptual issues and has avoided significant involvement with many papers on the nuts and bolts of specific reforms—papers that lie more within the domains of responsibility and comparative advantage of other departments.
Having considered the past, what of the future? Here I share the views of the evaluators about the direction in which it would be desirable to shift the emphasis of activities in RES in order to best serve the interests of the membership. However, I disagree with the magnitude of the shift that they seem to suggest, and with some of the ways they recommend for achieving this shift.
In terms of the desirable direction of the shift, there is little doubt that substantial increase in RES’s involvement in operationally related work in recent years has cut seriously into the time that RES staff have to devote to independent, staff-selected research. The present situation is, in my view, not sustainable. It will not be possible to attract and retain staff of the very high caliber that is needed to do worthwhile research, especially in the policy foundation category, if the time available to do staff-selected research remains as tightly constrained as it has been recently.
What would be needed to fix this problem? The results of the RES staff survey indicate that economist- and management-level staff in RES spend 28 percent of their time on staff-selected research and would like to spend 39 percent (see Table 2). At the economist level, the figures are 34 percent (actual) and 42 percent (desired); and at the management level, the figures are 13 percent (actual) and 30 percent (desired). These are average figures for the two categories of staff; differences among individuals are substantial. In my view, figures of about 40 percent of staff time for economists and 30 percent of staff time for management-level personnel available for staff-selected research are reasonable and are what is needed to sustain a high-quality research department, with an appropriate balance of activities, that will best serve the interests of membership in the longer term. Of course, amounts of time available for staff-selected research would vary among staff and would vary over time with the ebb and flow of other work.
How might additional staff time for self-selected research be made available? For RES’s work on multilateral surveillance, review work, and other operational activities (and related research), it is possible that, if the world economic situation calms down, some resources will be freed for staff-selected research. However, the clear trend over at least the past eight years, not just recently, has been for ever growing demands on RES to produce operational work (and related research) in the particular areas where it has responsibility; and there is no good reason to expect this trend to reverse. The recommendation of the evaluators of surveillance that the Fund consider quarterly publication of a combined WEO/ICMR is only one example of many calls for more operational work to be produced by RES. The same is also true for the other activities on which RES staff spend their time. In particular, while RES’s supervisory staff reports that they would like to spend 14 percent of their time on management and supervisory work, they report that they actually spend 23 percent. The fact is that the incessant pressures on supervisory-level staff to manage both the substantive work of RES and improve personnel management leave no realistic room to cut the total amount of time devoted to this activity by this part of the staff. The bottom line is that unless Fund management and the Executive Board wish to meaningfully reduce the quantity or degrade the quality of RES’s contributions to operational work, there is no way to reach reasonable and appropriate targets for time available for staff-selected research other than to increase the total available amount of staff time.
How much of an increase in staff resources would be appropriate? This is a matter of judgment that must recognize both budgetary realities and competing uses for Fund staff. To raise the amount of staff time available for staff-selected research by RES economists (which number only about 55) to 42 percent from 34 percent would require ten additional economists, plus relevant support staff, after allowing for modest reductions in other work.2 At the supervisory level, to reach the target of raising time available for staff-selected research from 13 percent to 30 percent, five additional staff would be required. One at least would be required to deal with the increased supervisory load arising from the additional economists and support staff. Four would be required to spread out the existing load of work among supervisory staff so that time for staff-directed research could be raised for all supervisory staff.3 (If the calming of conditions in the global economy does not reduce the total workload, one more would be needed to deal with the problem that RES’s supervisory staff now need to spend substantially more than regular Fund hours to accomplish their work.)
Relative to total Fund staff of about 2,500 and supervisory staff of about 300, increases of ten economists (and five or six support staff) and of five supervisory staff are not large. They are significant, however, relative to planned increases in the size of the Fund staff, and the budgetary cost is not trivial. I do not put these figures forward as a specific proposal. However, I do believe that it is important for Fund management and the Executive Board to understand, in rough quantitative terms, what would be required to move substantially in the direction that both the evaluators and I believe would be desirable.
In this regard, it is particularly important to correct the problem that operational work and various management tasks place such heavy demands on the time of RES’s supervisory staff that they can engage in very little of their own staff-selected research. Although the heavy efforts of RES’s supervisory staff on operational work have helped to protect some of the time of more junior staff for their staff-selected research, the longer-term effect on the intellectual vitality of RES’s senior staff and the effect on morale of all staff engaged in research has been negative. Moreover, the situation where RES’s senior staff have very little time for their own research contributes significantly to the most important problem that the Fund faces in attracting and retaining high-quality staff who want to pursue careers in the Fund where there is some meaningful opportunity to undertake serious and valuable research.
Indeed, the most important problem that the Fund faces in retaining key junior staff who have the ability for and interest in doing high-quality research of relevance to the Fund (especially in the category of policy foundation research) is the very limited opportunities that such individuals see both to advance in their careers in the Fund to the supervisory level based in significant part on the quality of their research and to continue after such advancement to devote a meaningful part of their time to staff-selected research. The plain fact is that only for very few of 300 supervisory positions in the Fund (mainly in RES and one or two other functional departments) is the quality of a staff member’s research and potential for further research a significant factor influencing promotion to the supervisory level. And, once promoted to such a position, the time available for staff-selected research is limited and continues to shrink. The significance of these facts is readily apparent to a number of highly competent junior staff who might want to continue their careers in the Fund with substantial continuing involvement in research but who now see little opportunity to do so.
The remedy to this important problem, in my view, is not to have a research department in the Fund that is devoted almost exclusively to research, with little operational responsibility, and with only a very small supervisory staff. That might be attractive to some junior staff who want to spend only a few years in the Fund, primarily doing research in their own areas of interest, before moving on to careers outside the Fund. It might well result in the production of more academically oriented research papers and a larger number of papers published in academic journals. It is highly doubtful, however, that it would generate more high-quality research on issues of primary relevance to the Fund. Junior staff working in such a research department would, quite naturally, orient their work to the market where they hope to advance in their longer-term careers outside the Fund.
Rather, in my view, the objective of maintaining a team of high-quality researchers who work on issues of vital interest to the Fund—beyond those of the most immediate operational relevance—is to have a research department that has diversified responsibilities for research and for operational work, and that has adequate time available for economist staff at all levels to engage in self-selected research projects. Involvement with research helps to keep the quality of operational work high. Involvement with operational work helps to keep self-selected research focused on issues relevant to the Fund. This issue is one of achieving and maintaining the right balance and of having the resources necessary for that purpose.
Note: This annex was prepared by Michael Mussa, Economic Counselor and Director of the Research Department.
External Evaluation of IMF Surveillance: Report by a Group of Independent Experts (1999).
Taking account of leave, administrative work, and other frictions, an additional economist adds net only about 85 percent of a staff year that is potentially usable for research or operational work. Also, each additional economist needs to be allocated 40 percent of his/her time for staff-selected research, leaving 45 percent of time added by the new economist to reduce the non-staff selected-research workload of other economists.
For supervisory staff, 15 percent of time is also lost to various frictions. If each supervisor is allocated 30 percent of time for staff-selected research, this leaves 55 percent of time added by a new supervisor that is available to relieve work of other supervisors so as to increase their staff-selected-research time.