Chapter 6 A Brief History of the IEO
- Ruben Lamdany, and Hali Edison
- Published Date:
- December 2012
This brief history of the IEO is written to help mark the office’s tenth anniversary. It covers some of the events that led up to the creation of the IEO, the initial vision for the office, and the key events in the IEO’s 10-year history (see Box 6.1 for summary of key events). The focus is on the institutional history rather than on the substance of the IEO’s work, which is covered elsewhere.1 The chapter ends with an assessment of IEO’s successes and of some recurrent institutional issues that continue to be the subject of debate.
In a sense, the genesis of the IEO can be traced back at least 20 years, to the late 1980s/early 1990s when discussions began on its creation. Evaluation offices with greater or lesser degrees of independence have a longer history at the multilateral development banks than at the IMF (see annex). Before the late 1980s, the IMF Executive Board and Management saw independent evaluation as an activity that might be appropriate for development agencies but not for an institution like the Fund. Not until the late 1980s/early 1990s did a few Board members begin to suggest that the IMF too could benefit from having an independent evaluation office. This was the start of a long and difficult process leading to the IEO’s eventual birth in September 2000.
In January 1993, the Board discussed a statement by Managing Director Michel Camdessus and a report by a staff task force that recommended the creation of an evaluation office at the Fund. The report referred to this office as independent, although the task force was recommending that the Director be appointed by and accountable to the IMF Managing Director. Introducing the Board discussion, the Managing Director observed that the proposal answered a request that many Board members had made in the past. There was wide support in the Board from both developed and developing country chairs to create the office, led by Executive Directors2 representing Brazil (then the Dean of the Board) and the United Kingdom, albeit with different opinions on many details. A few Directors noted their opposition or reservations, but in most cases also noted their willingness to join a consensus in favor of setting up the office. At the end of the Board discussion the Australian Executive Director, Ted Evans, suggested that the new office be called the “Independent Evaluation Office.” In concluding the meeting, Camdessus said he saw broad support for the proposal, albeit with varying degrees of enthusiasm, and that he would return to the Board quickly with an amended proposal to meet the concerns expressed by some Board members, so that a final decision could be taken in time to establish the new office in May that year. But in fact the issue was shelved, with Management citing continued lack of Board consensus combined with staffing and resource pressures in the Fund as reasons for lack of further action.
Box 6.1Key Events in the History of Independent Evaluation at the IMF
|1973||The World Bank Executive Board establishes the Operations Evaluation Department.|
|1989–92||Some IMF Executive Directors call for the establishment of an evaluation office at the IMF.|
|1992||Managing Director Michel Camdessus sets up a task force to examine establishing an IMF evaluation office.|
|1993||The Executive Board discusses the task force report’s findings and conclusions and considers the Managing Director’s proposal to establish an evaluation office. An office is not established.|
|1995–96||The Executive Board revisits discussion on how to strengthen the IMF’s evaluation function. It establishes a group of Executive Directors to oversee an ad hoc evaluations process to be re-examined after two to three years.|
|1996–99||Three external evaluations are commissioned by the Evaluation Group of Executive Directors.|
|1999||The Board of Governors Interim Committee reaffirms the importance of IMF independent evaluation.|
|2000||The Executive Board discusses “Review of Experience with Evaluation at the Fund.”|
|The Executive Board approves the establishment of an evaluation office (EVO), to be independent of IMF Management and staff and to operate at arm’s length from the Executive Board.|
|2001||Montek Singh Ahluwalia is appointed the first Director. He changes the department’s acronym from EVO to IEO.|
|The Evaluation Group of Executive Directors is transformed into the standing Executive Board Evaluation Committee created to oversee the IMF evaluation function.|
|2002–05||IEO builds up staff and capacity to undertake evaluations. IEO completes its first seven evaluations.|
|2005||Thomas Bernes is appointed the second Director of IEO. He joins the office in June 2005.|
|IEO subsequently completes three evaluations.|
|2005–06||The Executive Board launches an external evaluation of the IEO, headed by former Executive Director Karin Lissakers. The Lissakers Report contains a number of recommendations approved by the Board to enhance the effectiveness of IEO, including the establishment of a follow-up system for the implementation and monitoring of Board-endorsed IEO recommendations.|
|2007||The Executive Board approves a number of reforms, including frameworks for a forward-looking Management Implementation Plan and an annual Periodic Monitoring Report.|
IEO completes an evaluation on exchange rate policy advice. In June, the Executive Board approves the first Management Implementation Plan (on The IMF and Aid to Sub-Saharan Africa).
|2008–09||The Executive Board approves the first Periodic Monitoring Report in January 2008.|
IEO completes four further evaluations including on IMF governance.
|2009||Moises Schwartz is appointed the third Director of IEO. He joins the office in February 2010.|
|2010||IEO completes two further evaluations.|
|2011||IEO launches three evaluations.|
|In recommending Executive Board approval of the Fourth Periodic Monitoring Report, the Evaluation Committee calls for further enhancements to the evaluation follow-up process. The Evaluation Committee begins the process for undertaking the second External Evaluation of the IEO.|
|IEO marks its tenth anniversary with a conference in December.|
Those who continued to press for the creation of an evaluation office over the next few years saw a hardening of Management’s opposition, possibly reflecting a concern that an independent unit could end up “second-guessing” Management,3 despite mounting external and internal pressures to act. The pressures included an explicit call by G-7 finance ministers (in their background documentation for the June 1995 Halifax Summit) for the creation of an independent evaluation office at the Fund; similar calls from subsequent G-7 summits; and—later in the 1990s, as the Fund’s reaction to emerging market capital account crises came under scrutiny—from other external stakeholders and the nongovernmental organization community.4
Instead, Camdessus suggested a different approach to evaluating Fund activities. At the end of 1994, Management commissioned Sir Alan Whittome to undertake an independent evaluation of Fund surveillance, with special reference to the 1994 Mexican crisis. The resulting report had a very limited circulation but was discussed by the Board in April 1995. In summing up that discussion,5 the Managing Director noted the high quality of the evaluation and said he had come to the view that such use of outside experts would be a less costly, less bureaucratic, and more refreshing approach than setting up a separate evaluation office. In January the following year, Camdessus formally proposed this approach to the Board: Management and the Board would experiment with commissioning a series of independent external evaluations and would review the experience after some two to three years.6 The Board generally endorsed this approach but it also called for the creation of a Board Committee, chaired by an Executive Director, to oversee the external evaluation function. Subsequently it was also agreed that the activities of the Fund’s Office of Independent Audit and Inspection (OIA) would be expanded to conduct more reviews of the Fund’s structure and work practices.
An Evaluation Group of Executive Directors (EG) was convened in July 1996, with terms of reference that were adopted by the Board in September that year,7 and commissioned three external evaluations in the period 1996–99.8
“External Evaluation of the ESAF,” carried out by a group led by Kwesi Botchwey (former Finance Minister of Ghana);
“External Evaluation of Fund Surveillance,” carried out by a group led by John Crow (former Governor of the Bank of Canada); and
“External Evaluation of the Fund’s Research Activities,” carried out by a group chaired by Professor Frederic S. Mishkin (former Director of Research at the Federal Reserve Bank of New York).
These evaluations produced important findings, conclusions, and recommendations. For example, “External Evaluation of Fund Surveillance” (the Crow Group report) urged the Fund to concentrate its resources on the most systemically important countries, and to focus its work on the international aspects of the systemically important countries’ policies—and in particular on the interface of financial sector and macroeconomic policies of the systemically important countries. These conclusions came to be echoed in several subsequent IEO reports and remained valid 10 years later.
However, Executive Directors and many external stakeholders saw shortcomings with the Fund’s reliance on external evaluations. As anticipated, the experience was reviewed by the EG at the end of the three-year trial period; and a report was considered by the Board in early 2000.9Box 6.2 lists the main conclusions on external evaluations commissioned by the EG.
Box 6.2Conclusions on Experience with Select External Evaluations, 1996–99
It took Directors a long time to agree on topics and select evaluators, resulting in significantly fewer evaluations being undertaken than originally expected.
There was an external perception that the Board’s direct involvement in the choice of topics and evaluators had constrained the choice of more sensitive topics and more critical perspectives. There was no process of external consultation on the choice of topics.
Most of the external experts chosen had limited knowledge of the internal workings of the IMF, and criticisms were made both of the quality of some of the analysis and the practicability of some of the specific proposals made.
Once their job was complete the evaluators ceased to be available to help with implementation. There was no follow-up process. And the ad hoc nature of the evaluations made it hard to maintain institutional memory of their findings.
The resource costs of the process, in terms of the time of Board members and staff as well as the direct cost of the evaluators, were substantial.
The EG review also took account of external opinions on independent evaluation at the Fund. At the time there were mounting calls from external stakeholders for the establishment of a permanent independent evaluation office at the Fund. These were fueled in part by widespread criticisms of the Fund’s handling of the late 1990s capital market crises in East Asia and elsewhere. The EG review noted reports by groups of NGOs calling for the establishment of a separate independent evaluation office.10 It also noted the successive calls made by G-7 finance ministers for the Fund to strengthen its processes for external evaluation, and the reaffirmation by the Interim Committee of “the importance of independent evaluations of the Fund’s operations and policies.”11
The EG review considered three options for the future: (1) continuing with the existing arrangements (rejected for the reasons noted in Box 6.2); (2) expanding the capacity of the OIA (an option thought unlikely to produce, or to be perceived to produce, truly independent evaluations given that the OIA reported to Management); and (3) the creation of an independent evaluation office. The EG recommended the third option to the Board: the creation of a new independent evaluation office—which at that point was referred to as the EVO.
Establishment of the IEO
The Initial Vision
In making its recommendations in early 2000, the EG emphasized several features that its members considered essential for an independent evaluation office. In a sense these constituted the initial vision of the founders of the IEO:
“. . . an EVO . . . reporting directly to, but operating at ‘arms length’ from, the Board, and with effective independence from management.”
“. . . [the EVO] must complement existing [self-] evaluation efforts by augmenting the potential scope of evaluation where Fund expertise may be limited. . . .”
“. . . it must enhance the credibility of evaluations to observers outside the Fund. . . . Even if it were internally accepted that current self-evaluation was wholly objective, the perception outside the institution that such bias exists, in and of itself, undermines the ability of the Fund to undertake its work.”
The evaluation office would “need to include a transparent and efficient mechanism for systematic follow-up [of its recommendations].”
It would “benefit from the hiring of staff embodying considerable breadth in their backgrounds and expertise. . . . [In addition to adequate understanding of macroeconomic issues] EVO staff should collectively possess both a broad and demonstrated interest and experience in areas such as public policy, law, economic history, and capital markets. The build-up and retention of evaluation expertise in the EVO would also benefit the Fund.”
“. . . it would need to be large enough to carry out and follow up on a sufficient number of evaluations to derive meaningful lessons to inform the work of the Board . . . but be small enough to force the prioritization of topics and the coordination of its efforts with evaluation underway elsewhere in the Fund.”
It should be “provided with a budget from which it could augment its staffing [where additional experience or perspective is needed] with external consultants and experts to participate in, lead, or even wholly conduct particular evaluations. This would be one channel through which . . . to ensure that external input formed a part of independent evaluation. Where appropriate, external input could also be obtained through public consultations conducted by EVO. . . .”
The EG’s review of experience and its recommendations for establishing an independent evaluation office were largely the work of a small group of Executive Directors led by Thomas Bernes as chair of the EG.12 The EG worked hard both to convince the few remaining doubtful Executive Directors and to craft an outline for an evaluation office. Partly drawing on experience with evaluation in other international financial institutions, the EG proposed the establishment of an evaluation office whose Director would be chosen and appointed by the Board, not Management, and who would have an exceptionally high degree of independence, balanced by strong requirements for consultation and transparency.
Camdessus, who retired in February 2000, suggested that the decision whether or not to go ahead should be left to a Board meeting chaired by his successor, Horst Köhler. The Board discussion of the EG report took place after Camdessus’ retirement, with Köhler as Chairman of the Board. Köhler, who was familiar with independent evaluation from his time as President of the European Bank for Reconstruction and Development (EBRD), was immediately sympathetic to the proposal. The general approach to establishing the EVO was endorsed by the Board in April 2000, and a few days later by the International Monetary and Financial Committee (IMFC).13 Discussions continued over the summer of 2000, with the main features of the office being agreed at Board meetings in August and September 2000.14
Terms of reference (TOR) for the evaluation office were agreed at the September 2000 meeting and subsequently reported to the IMFC15 (see Part IV of this volume for the full TOR). Their key features are:
a mission of promoting learning in the Fund as well as improving the Board’s oversight (thus giving the evaluation office the two standard functions of independent evaluation—learning and accountability—although the latter had been and remained the Board’s main concern);
a work program to be decided by the Director after a broad process of consultation;
a variety of measures to buttress the independence of the Director and staff;
a strong presumption that reports would be published; and
a budget set by the Board separately from the general IMF budget (the main concrete sense in which the evaluation office is accountable to the Board).
Making the Office Operational
At the same time, the Board began a search process to recruit the EVO’s first Director. The process involved drawing up a job description and terms of reference for the Director, the selection of a firm of search consultants, the preparation by that firm of a shortlist of preferred candidates, and finally a choice from that shortlist by the Board. The appointment was to be for a period of four years, renewable for a further three—with the Director then disqualified from further employment with the IMF.
In April 2001, the Board announced the appointment of Montek Singh Ahluwalia as the first EVO Director. Ahluwalia was exceptionally well placed to establish the new office. He had been a member of the Indian Planning Commission and Finance Secretary in the Government of India. He had a reputation of successfully implementing major financial and economic reforms in India, a good knowledge of the Bretton Woods Institutions, and experience in both government and international organizations.
Ahluwalia took up the post in July 2001 and quickly assembled a group of high-quality staff from outside and inside the Fund. His first senior appointment was David Goldsbrough, formerly Deputy Director in the Fund’s Western Hemisphere Department, who became the EVO’s first Deputy Director. This was followed by the appointments from outside the Fund of Shinji Takagi and Marcelo Selowsky, who led two of the three initial evaluations (David Goldsbrough led the third). Ahluwalia was also able to form a strong group of part-time external advisors to help guide and assure the quality of the office’s work. One of his first acts as Director was to replace the EVO acronym with IEO, as a way to emphasize the office’s independence.
In October 2001 the IEO issued its first progress report to the IMFC.16 By this time arrangements were well under way for recruiting staff for all the approved full-time posts. Consultations had also begun with internal and external stakeholders on developing an initial work program, identifying first a wide range of possible studies and then choosing from that menu. At this point the Director expressed the hope that about five projects could be undertaken each year once the Office had reached full capacity.
The last set of decisions in making the IEO operational was taken in August 2002 when the Director in consultation with the Board adopted standard rules and processes for the review and publication of evaluation reports and other documents produced by the IEO. These rules and processes, which constitute the basis for those in place a decade later,17 stipulated that:
The IEO will give units in the Fund whose activity is being evaluated an opportunity to comment on preliminary assessments. The evaluators will incorporate all factual corrections that may surface but are free to take account of or ignore any comments on substantive aspects of the assessment.
When an evaluation report has been completed it will be transmitted to Management and the EG18 and circulated to the Executive Board. At this point, no changes to the document can be made other than purely factual corrections.
IMF Management will be provided an opportunity to prepare written comments.
Reports will be discussed by the Board, and a Summing Up of the discussion prepared.
Reports, if published, will include written comments received from Management and staff, and any IEO responses thereto, along with the Summing Up of the Board’s discussions.19
The First Five Years (2002–06)20
Establishing the Office and Setting the Work Program
The office was fully staffed by early 2002, with an agreed budget for FY200321 (Box 6.3). The Director began a process of consultation on the IEO’s work program shortly after his appointment in July 2001. An initial list of 34 possible evaluation topics was prepared and published for consultation with members of the Executive Board and other interested internal and external groups, including representatives of civil society and academics in Washington, Europe, and Africa. Following these discussions, 15 topics were chosen for the IEO’s initial medium-term program.22 The choice gave priority to topics that had been the subject of controversy or criticism, were of the greatest interest to the Fund’s wide range of member countries, and offered the greatest learning potential.
Box 6.3IEO Initial Budget and Staffing
The initial budgets for IEO (FY2003, FY2004, and FY2005) were set at a level representing around 0.5 percent of the IMF’s total administrative budget. This was, intentionally, significantly less than the percentages of administrative costs accounted for by evaluation offices of other international financial institutions (1.2 percent on average). Initially it was hoped that this level of funding would enable a steady-state level of output equivalent to four standard-size evaluations a year; the possibility was kept open of increasing this number to five later on. But when, later, it became clear that doing so would require a significant increase in the budget, the Board decided that the IEO should decrease the number of evaluations rather than increase the budget.
From FY2004 the budget was sufficient to finance 13 staff positions (the Director, Deputy Director, nine professionals, and two administrative staff, and thus more than the 11 posts originally envisaged), and a greater use of consultants than in the IMF in general. The planned reliance on consultants reflected in part IEO’s shifting needs for different kinds of expertise, and was also intended to strengthen the IEO’s independence and credibility. Staff appointments were—and continue to be—supplemented by a number of contractually employed research assistants.
Initial Evaluations: Setting the Pattern
Three topics were chosen from the list to be evaluated during FY2003: Prolonged Use of IMF Resources; Capital Account Crises; and Fiscal Adjustment in IMF-Supported Programs. Work on all three started in parallel. The mix was deliberate, including one topic—capital market crises—that was bound to raise controversial issues, and two that focused more on internal IMF processes but were thought likely to offer good learning opportunities. There were extensive processes of consultation and checking: the office felt it should be especially thorough in its work on these initial evaluations, knowing they would help establish its reputation and at the same time set precedents for its future work. The first IEO evaluation report—Evaluation of Prolonged Use of IMF Resources—was issued in September 2002 and the next two were completed by August 2003.
For FY2004 a further three topics were chosen for evaluation from the same list, following a further process of consultation. The Director noted that one of the three—an evaluation of the PRSP/PRGF process, involving six in-depth country case studies—was equivalent in scope to two normal evaluation projects. Four more projects were chosen from the list for FY2005 with work on one more to start in FY2006. At that point—towards the end of 2004—11 of the original 15 topics had been selected for evaluation. The IEO embarked on a new round of consultations aimed at identifying a further menu of topics to guide its work over the following years.
Processes that were developed in the course of the first few evaluations set a pattern that continues today.
The chosen project leader and team produce a concept note, setting out the main questions to be addressed and methods to be used, and this is used as a basis for a brainstorming session with a group of knowledgeable external advisors.
Based on this the IEO prepares a draft issues paper which is posted on the IEO website and used as a basis for discussion with the Board, Management, staff, and external stakeholders. A final issues paper is then posted setting out the scope of the evaluation, main questions, methods, and work plan.
Methods used include (internal and external) document reviews, surveys, interviews, statistical analyses, and/or preparation of background papers.
Towards the end of the evaluation a further workshop with external advisors and experts is held to discuss emerging conclusions and recommendations. Often these workshops give rise to further questions and suggestions requiring further work.
A draft evaluation report is prepared and reviewed within the IEO and by selected external advisors.
The revised draft report is then sent to IMF staff for written comment, typically within three to four weeks. These comments are meant to focus on factual errors and inaccuracies. Where appropriate, relevant country authorities are also given an opportunity to correct factual errors.
After careful review, taking account of all comments, the IEO Director approves the final version of the evaluation report, for circulation initially to the Evaluation Committee and IMF Management.
Two IEO evaluations in this early period (Evaluation of the IMF’s Role in PRSPs and the PRGF and Financial Sector Assessment Program) involved reviews of activities where the Fund’s cooperation with the World Bank was very close. In each case, the Bank’s Operations Evaluation Department/Independent Evaluation Group undertook parallel evaluations of World Bank activities, and the two offices collaborated closely on inputs, including through joint country case studies and joint surveys.
In addition to evaluation reports, the IEO established a practice of producing progress reports to the IMFC and regular annual reports. Regular progress reporting to the IMFC began in the fall of 2001. The first full annual report of the IEO, setting out progress in the first two years, was submitted to the Board and published in 2003; it was followed by a report in 2004, and a further report covering the two years 2005–06. The first annual report set a pattern followed in many subsequent reports by including reflections on common themes emerging from IEO evaluations (Box 6.4).
Box 6.4Themes from IEO Evaluations Highlighted in Annual Reports, 2003–06
IEO’s Annual Report 2003 highlighted:
The need for greater candor in IMF surveillance.
The need to deal better with uncertainty in program design and the benefits of contingency planning.
The lesson that underlying domestic political commitment is more important than the specific structure of IMF conditionality.
The Annual Report 2004 noted:
The need for greater clarity about intermediate objectives and performance indicators in program design.
A number of weaknesses in the IMF decision-making process, including the way that the candor of assessments tended to become muted as they are transmitted through the institution, a reluctance to address what should be the alternative strategy if the preferred approach fails, and some questions about the respective roles of Management and the Executive Board.
The Annual Report 2005–06 repeated two of these themes—the need for greater candor in papers submitted to the Board and clarity about objectives and criteria for judging the Fund’s performance—and added a further lesson about the need for the Fund to explain better the rationale for policy advice and program design in particular countries.
Change of Director
Montek Singh Ahluwalia, the IEO’s first Director, resigned in June 2004 to take up a cabinet-level position in the Indian Government, and David Goldsbrough, the Deputy Director, took over as Acting Director while a search was instituted for a new Director. The search process used was essentially the same as that used in appointing the first Director.
On the recommendation of the outgoing Director, one change was made in the terms of reference for the job. It was decided that allowing for an appointment to be renewable after an initial four years could be perceived to compromise the Director’s independence as he approached the point where he might seek reappointment. Thus the IEO’s second Director was offered a nonrenewable term of appointment of six years.
After the Board reviewed and interviewed candidates shortlisted by the chosen executive search firm it was decided to offer the appointment to Thomas Bernes (at the time Secretary of the Development Committee of the World Bank and IMF and former Canadian Assistant Deputy Minister of Finance, and former Executive Director of the IMF), who took up his appointment in June 2005.
The Lissakers Report: Evaluating the Evaluators
As noted above, the IEO’s initial terms of reference provided for an external evaluation of the office after a period, to assess its effectiveness and to consider possible improvements to its structure, mandate, operations, or terms of reference. To do the job the Board chose an independent panel in September 2005, chaired by Karin Lissakers (former U.S. Executive Director at the Fund).23 The panel reported in March 2006.24
The panel concluded that the IEO had served the IMF well, but also identified “certain weaknesses and . . . trends that are cause for concern about its future,” noting the “biggest challenge facing the IEO [as being] to avert the tendencies, pressures, and practices that may push it in the direction of becoming bureaucratized, routinized, and marginalized.” The panel’s specific recommendations are set out in Box 6.5.
These recommendations reflect the panel’s findings, which include the following:
While IEO reports had led to some improvements, both Board and Management had paid too little attention to systematic follow-up.
Management and staff should take an open and constructive approach to the IEO’s findings.
There was a risk of duplication of work with the Fund’s Policy Development and Review Department (now the Strategy, Policy, and Review Department), which had a deliberate policy of working on topics being looked at by the IEO, partly with a view to being ahead of IEO recommendations. The panel saw this as wasteful duplication.
Too many evaluations had focused on process issues and not enough on issues of systemic and strategic institutional importance.
Reports had become too long, and should be made more punchy.
There had been insufficient engagement with governments and other stakeholders—an important way of getting the IEO’s messages across.
Box 6.5Summary of Lissakers Report Recommendations
First, the IEO should address issues that are fundamental to how effectively the IMF is fulfilling its mandate, and its terms of reference should be changed to make this clear. The IEO should be assured full access to information.
Second, the IEO should diversify its staff and contractual mix and make greater use of people of eminence from outside the Fund to lead evaluation teams. Strong outside personalities with limited IMF exposure are likely to bring a fresh perspective and questioning attitude and ensure that the IEO adds value to the array of evaluations already being undertaken within the Fund.
Third, a more systematic approach is needed to follow up the recommendations of the IEO and monitor their implementation. The Board and the Evaluation Committee need to take responsibility and play a more active role in this regard.
Fourth, the IEO’s dissemination and outreach activities need a complete overhaul, particularly to raise the IEO’s profile in developing and emerging economies where the IMF’s role is considered most contentious.
The IMF will reap the full benefits of a strong IEO if the Board plays an active role and senior management takes a constructive and open stance toward the evaluation office. The IEO in turn must be bold—about what it evaluates, how it evaluates, and who it hires to do the job.
The panel also expressed a concern that in one case25 the IEO “had accommodated management and staff sensibilities to the detriment of the information value of its evaluation and its contribution to Board oversight.”
The Executive Board considered the Lissakers Report on April 26, 2006 and endorsed some but not all of these findings (see Part IV of this volume for summary). A few Directors suggested that the practices for submitting the IEO’s draft reports to Management and staff for comment should be reviewed. There was much discussion of the panel’s concern that the IEO’s independence could have been compromised (particularly in the IMF and Argentina report) by pressures from Management to alter draft reports. On this the Summing Up notes that:
Directors discussed extensively the feedback process for draft IEO evaluation reports . . . and its implications for the IEO’s independence, both actual and perceived. They agreed that best practice requires the IEO to solicit comments from staff, management, and other players on its draft reports but, at the same time, to exercise its independent judgment and responsibility on whether to take these comments on board. Thus, any changes introduced by the IEO in the feedback process would be expected to be based on the exercise of best judgment by the IEO, rather than constituting evidence of accommodating management or staff sensitivities. . . .
With one exception—that no changes were made to the policy on the IEO’s access to information26—the recommendations from the Lissakers Report were broadly implemented by the IEO Director and the Board over the following two years, as summarized below.
Evaluation reports became shorter and more focused. A separate retrospective of IEO evaluations to date27 finds that that there was a modest increase in “bite” in post-Lissakers evaluations, with more attention paid to issues of substance and a reduced focus on process, but that both of these trends were partly due to a change in the nature of evaluations. There was also a marked sharpening of presentation and shortening of reports, as the result of a deliberate policy introduced by the incumbent Director.
The Board review process was formalized, with set limits on timing. In early 2007 the Executive Board and Management agreed on guidelines including that the Board’s consideration of an IEO report would generally be scheduled within six weeks of its circulation to the Board, and that Executive Directors would receive any comments from Management and staff at least two weeks before the Board discussion.28
New follow-up mechanisms were introduced. In January 2007 the Board agreed on a new framework for follow-up to IEO reports. This requires:
Management to provide, soon after the Board’s discussion of each IEO report, a forward-looking Management Implementation Plan (MIP) for the recommendations endorsed by the Board; and
Management to present to the Board an annual Periodic Monitoring Report (PMR) on the state of implementation of Board-endorsed recommendations and MIPs.
The first MIP (for the evaluation of The IMF and Aid to Sub-Saharan Africa) and the first PMR were produced in FY2008.
IEO’s outreach and communications strategy was strengthened. Outreach was revamped following the Lissakers Report, and strengthened to the extent possible within the limitation set by the Board: that there would be no increase in the budget for this purpose (see below). Most importantly, the IEO launched a new website in early 2007, giving easy access to all IEO material and reports. Steps were also taken to actively manage the IEO’s email data base and to issue a biannual newsletter. And the number of outreach seminars, both to contribute to ongoing evaluations and to disseminate the results of completed evaluations, was stepped up, often in cooperation with and at the request of third-party organizations.29
Changes were made to the IEO’s human resources policies. In September 2007 the Board approved some minor changes in the terms of reference for the IEO Director and in the terms and conditions of appointment for some categories of IEO employees,30 and in early 2008 the Board agreed to the publication of these revised terms on the IEO website. In May 2008, responding to the Lissakers Report recommendation to strengthen the IEO’s actual and perceived independence, the Board agreed that any IEO project leader hired in the future should have a cooling-off period of 12 months before he or she could be employed as an IMF staff member.
IEO practices regarding Management and staff review of draft evaluation reports were clarified. In December 2007, the Director of IEO confirmed his intention to maintain the practice of allowing Management and staff the opportunity to comment on draft reports, while acknowledging that interaction with IMF staff at this stage of an evaluation could be perceived as potentially compromising the independence of views of the evaluation team. His decision was based on an earlier review initiated by the IEO, and took account of the concerns about IEO’s independence that had been raised during the Board discussion of the Lissakers Report. He proposed that any subsequent material changes reflected in the final version of the evaluation report would be explained to the Board and recorded in an internal memorandum, along with a redlined version to facilitate subsequent internal reviews or external panels reviewing the IEO’s work. He also confirmed that the IEO has no obligation to take on board any such comments received.31
The Second Five Years (2007–11)32
The IEO’s second five years started with the implementation of a number of improvements recommended by the Lissakers Report, as discussed above. Other important developments over the period were a reduction in the IEO’s budget and staffing, following parallel reductions across the IMF; a change in the process used to identify new topics for evaluation; exceptionally high staff turnover; the appointment of Moises Schwartz as IEO’s third Director in 2009, following the retirement of Thomas Bernes that year; and continuing discussions on the process for follow-up on IEO evaluations and ways to strengthen the process of learning from them.
Reductions in the IEO’s Budget and Staff
Initially IEO’s budget was set at a level that was estimated would be sufficient to produce about four evaluations a year. This estimate turned out to be over-optimistic. Throughout the 2007–11 period, the IEO’s approved budgets remained at about 0.5 percent of the IMF’s total administrative budget and its number of approved staff posts remained at 13.33 The IEO experienced high turnover (especially in 2008–09, at the time of the IMF downsizing) and difficulties in recruitment, largely because of the restrictions on length of tenure, which are much more stringent than in other similar organizations. This led to some posts remaining unfilled for long periods, and in some years to a high vacancy level. During the IMF downsizing, the idea of raising IEO resources to permit the preparation of five evaluations a year was explicitly abandoned, and the IEO budget was cut in line with that of the IMF. In fact, most evaluations turned out to be more complex and resource-consuming than anticipated, and output remained at between one or two evaluations a year.
After FY2006, IEO’s budget and staffing began to be reduced in real terms, following parallel reductions across the Fund. Year-on-year budget reductions of around 6 percent in real terms took place over the years up to FY2010. To accommodate these cuts the number of economist positions in the IEO was reduced by one, bringing staff numbers down from 13 to 12. There was a parallel reduction in the office’s expected output. While in 2007 the aim was still to work on three evaluations at any one time, by 2009 the Director spoke of the Office “struggling to produce two evaluations a year,” and by 2011 the budget was thought to be sufficient to continue actively working on two evaluations a year, with fewer than two completed in any year. This was in part also a reflection of the complexity of topics chosen, as noted below.
A New Director
About halfway through the 2007–11 period Thomas Bernes announced, in January 2009, his intention to retire as Director that summer and he left office at the end of July. The Evaluation Committee recommended and the Board agreed to use the same procedure for selecting a new Director as on previous occasions, with a search firm identifying a shortlist of candidates from which the Board would make its choice. The decision to offer the post to Moises Schwartz (former senior Ministry of Finance and Central Bank official in Mexico and former IMF Executive Director representing Mexico and seven other countries), the current Director, was taken in November 2009 and he started work as Director in February 2010.34
Choice of Evaluation Topics
The way evaluation topics are chosen was also changed, recognizing that evaluation priorities were likely to change quite quickly with the evolution of the global economy and the Fund’s work. The new approach was to consult annually on which topics should be chosen from a shortlist that changed as potential new topics emerged.35 Thus in April 2006 the IEO circulated to the Board, and published, a list of 23 possible topics for evaluation that had been identified following discussions with a variety of stakeholders.36 Following feedback on priorities and timing, in July 2006 the Director selected four of these topics to add to the IEO’s future work program.37 Consultations on subsequent topics, which began in 2008, were interrupted by the change in Director in 2009, but not before the topic of IMF Performance in the Run-Up to the Financial and Economic Crisis had been added to the evaluation work program. Work on this evaluation started before Bernes’s departure and was completed in 2010. Following consultations in 2010 on a further menu of 17 possible evaluation topics, two new evaluations were launched,38 and in 2011 an evaluation was launched on a third topic from the list.39
Several of the topics chosen required complex evaluation with many country case studies and/or extensive research on stakeholder attitudes or on practices in other institutions. Their complexity, together with the IEO’s staffing difficulties, contributed to limiting the Office’s output to two or fewer evaluations a year. The selection of complex topics also represented a change from the original intention, which had been to balance (1) evaluations of broad and complex topics, which often allow cross-country comparison, with (2) simpler evaluations, for example those concentrating on experience with Fund-supported programs in a single country. The change in the balance reflected a recognition (one of the messages of the Lissakers Report) that broader evaluations can provide a better opportunity for learning, and that single-country cases with limited broader application make little sense when the office is only carrying out two or three evaluations a year. It may also partly reflect an assumption that the IMF’s self-evaluation processes provide an effective way to evaluate its approach in individual countries.40
Conduct of Evaluations
While the evaluation process remained essentially unchanged over this period, at times the IEO had to adapt given its own circumstances or the characteristics of certain evaluations. For example, the evaluations of The IMF and Aid to Sub-Saharan Africa and IMF Involvement in International Trade Policy Issues were led by external consultants rather than by IEO staff members. Also, for the evaluation of IMF Performance in the Run-Up to the Financial and Economic Crisis, the IEO constituted an advisory group of eminent persons that discussed a preliminary version of the report and prepared a summary of its views that was published alongside the final report.
Presentation of Reports
The Lissakers Report had recommended that the IEO produce shorter reports, and the average number of pages fell by more than half between the IEO’s first and second five-year periods (pre- and post-Lissakers Report). At the same time the IEO experimented with a variety of approaches for accommodating additional background material.41
The numbers and presentation of evaluation findings and recommendations varied widely across the evaluation reports produced in 2007–11. Numbers of main recommendations ranged from 2 to 14 and numbers of sub-recommendations from 0 to 25. The translation of evaluation findings into recommendations also varied;42 one approach was to frame only a few recommendations at a general level, while another was to list a (usually larger) number of specific actions to address identified concerns. There are arguments for and against both approaches, as discussed in the IEO’s Annual Report 2011:
In designing its recommendations the IEO faces important trade-offs regarding the degree of specificity. The IEO can provide general recommendations to complement its conclusions, leaving it to IMF Management to propose specific actions to effect change. This approach has the advantage of focusing attention on big-picture goals and allows the Board to endorse the direction of needed reforms while allowing Management the flexibility to propose how best to pursue these goals and to present specific actions in the MIP to achieve them. On the other hand, this approach makes it very difficult for the Board to assess the extent to which Management’s proposed actions would address the goals endorsed; further, often these actions are such that it is difficult for the Board to monitor their implementation.
Alternatively, the IEO can recommend specific actions to address goals and concerns raised by its evaluations. Detailed IEO recommendations have the advantage that they are more likely to be closely aligned with the conclusions of the evaluation and easier for the Board to monitor. But this approach may diminish Management and staff ownership of the implementation plan, and would not make full use of their greater institutional knowledge and their ability to integrate the implementation of Board-endorsed competing approaches. . . .
Annual Reports and Common Themes
IEO annual reports continued the practice of highlighting common themes from evaluations conducted. Box 6.6 shows the common themes as summarized in the Annual Report 2009.
Box 6.6Common Themes Noted in the IEO Annual Report 2009
The 2009 report noted that themes emerging from earlier evaluations emphasized the need for:
Better management of institutional change at the IMF.
Greater clarity about the goals of various IMF initiatives and a properly aligned external communications strategy.
Strengthened partnership between the IMF and partner institutions and donors.
Clearer metrics for the assessment of the impact of IMF’s policy advice and whether the IMF is meeting its commitments to countries.
The IMF to be more explicit about who is accountable for what and to whom.
The report added a number of lessons for the IEO itself from the emerging financial and economic crisis:
The need to be more pointed in challenging the evenhandedness of Management in dealing with members.
The need to examine more critically the Fund’s ability to “speak truth to power.”
The need to be bolder in highlighting downside risks.
The need to do more to encourage follow-up on evaluation findings.
Staff Turnover and Other Staffing Issues
High staff turnover became a critical issue for the IEO during its second five years. For example, during the two-year period before 2009, there were 11 departures from the 7 nonmanagerial economist positions. Because of high staff turnover, over the period FY2009–11 staff numbers in management and economist positions were on average one or more below budget.
The continued turnover and difficulties in recruitment imposed significant costs on the IEO. Factors that contributed to the high turnover included the impact of the employment conditions, such as the six-year term limit on regular IEO staff positions, that were imposed initially on the IEO to strengthen its perceived independence. These employment conditions could also have affected the IEO’s ability to attract high-quality staff. Following a 2009 IEO review of its employment policies, in April 2010 the Board approved changes to address these issues and enhance IEO’s technical excellence and institutional knowledge while maintaining its diversity and freshness of perspectives, and to strengthen its actual and perceived independence (Box 6.7).
Box 6.7Changes in IEO Employment Policies Approved in April 2010
The possibility, in limited circumstances, to extend the term limit for senior employees and economists from 6 to 12 years.
The possibility of hiring some research assistants/officers as noncontractual employees for up to six years.
The adoption of a policy whereby employees hired from the IMF would lose their guaranteed right of return to the Fund if they remain at the IEO for more than 6 years.
An expansion of the categories of IEO staff subject to a 12-month “cooling-off” period before they can join Fund staff.
Follow-Up on IEO Reports
Experience with the follow-up mechanisms that were introduced in 2007 (see the section “The Lissakers Report: Evaluating the Evaluators” above) has been mixed. The first PMR,43 which reviewed the status of implementation of all Board-endorsed recommendations from the first 10 IEO evaluations, was discussed by the Board in January 2008. In the course of that discussion the Board asked the IMF staff to produce well-defined benchmarks or other measurable criteria to monitor progress in implementation. When the Evaluation Committee came to review the Third PMR in October 2009, it noted that all key benchmarks had been met or were on track for timely completion; but it also noted that in some cases more needed to be done to achieve the broader policy objectives underlying specific IEO recommendations, and that the PMR process might need to be revised. Later, the Committee again broached the possibility of improving the PMR process: in July 2011, discussing the Fourth PMR, the Committee recommended and the Board agreed that the coverage of PMRs could be expanded to cover broader policy objectives as well as specific recommendations, and by describing the status of past implementation plans. The Committee also pointed out that the process for endorsing IEO recommendations needed to be strengthened, including by reforming the Board Summing Up process to improve clarity and accuracy.
How to improve the follow-up on Board-endorsed recommendations has been a recurring question throughout the five years since the Lissakers Report. There are several difficulties in dealing with this issue. First, there is the strength or otherwise of incentives provided by the Board, starting with closer monitoring, to enhance the commitment of Management and staff to implementing these recommendations. Second, some categories of IEO findings and recommendations by their nature are unlikely to be amenable to monitoring follow-up by tracking specific benchmarks. Examples are recommendations that are pitched at a general or broad level, leaving Management to propose the detail of implementation (see discussion of this issue in the above section). Finally, some recommendations require actions by country authorities or Board members, leaving Management constrained in its ability to propose or implement an action plan. For example, some of the recommendations from the evaluation of IMF governance are directed to the Fund’s governors.44
Over the past decade, the IEO has followed closely the vision for its creation. Thus far the results have met, if not exceeded, the initial expectations. The IEO has produced what are generally acknowledged to be independent, objective, and high-quality evaluations of complex and sometimes sensitive issues. Its work has been held up as an example of the kind of evaluations that should be carried out elsewhere. Many of the IEO’s recommendations have led to practical improvements in the Fund. Another measure of success is that some of the IEO practices that have led to its exceptional degree of institutional independence have been replicated elsewhere.45
To ensure the IEO’s continued success will (as recommended in the Lissakers Report) require sustained efforts and support by the Board, and willingness by IMF Management and staff to engage constructively. For the Board, in addition to protecting the IEO’s independence and ensuring that it has sufficient resources, this means working with Management on ways to strengthen the follow-up on IEO recommendations. For Management and staff, it means further efforts to strengthen the institution’s learning culture to better incorporate IEO lessons into IMF operations and practices. These issues will no doubt continue to be debated in the years ahead, informed also by the second external evaluation of the IEO that is now under way.
|1970||The Inter-American Development Bank (IDB) establishes an evaluation function.|
|The World Bank President creates the Operations Evaluation Unit within the Programming and Budgeting Department.|
|1973||The World Bank Executive Board establishes the Operations Evaluation Department (OED).|
|1975||The World Bank-International Finance Corporation Joint Audit Committee establishes an evaluation subcommittee to review project evaluation reports and assess the project evaluation system.|
|1978||The Asian Development Bank (ADB) establishes a Post-Evaluation Office.|
|The International Fund for Agriculture and Development (IFAD) establishes an internal evaluation function as part of the Monitoring and Evaluation Division, reporting to an Assistant President.|
|1980||The African Development Bank (AfDB) Evaluation Unit is set up in the Research and Planning Division.|
|1987||The AfDB Evaluation Unit becomes the Operations Evaluation Office, reporting to the President.|
|1990||The Islamic Development Bank (IsDB) establishes the Operations Evaluation Office under the authority of the Adviser for Operations Evaluation and Audit, headed by a Director who reports to the President.|
|1992||The European Bank for Reconstruction and Development (EBRD) establishes the Project Evaluation Office.|
|1994||IFAD establishes the Office of Evaluation and Studies.|
|The IDB former Ex-Post Evaluation Unit and the External Review and Evaluation Office merge to create the Evaluation Office.|
|The World Bank Committee on Development Effectiveness (CODE) is established as a standing committee of the Executive Board.|
|1995||The European Bank for Investment (EIB) establishes the Operations Evaluation Department.|
|The AfDB Operations Evaluation Office is upgraded to a department whose Director reports to the Board of Directors and administratively to the President.|
|The Informal Subcommittee of CODE is established to consider assessments submitted by OED and whether adequate follow-up action has been taken on CODE-endorsed OED recommendations.|
|The International Finance Corporation Operations Evaluation Unit is reorganized as a department, the Operations Evaluation Group (OEG).|
|1996||The Evaluation Cooperation Group (ECG) is established. Founding members are evaluation offices from the AfDB, ADB (Secretariat), EBRD, IDB, and the World Bank Group. Subsequent members are evaluation offices from the EIB, IMF, IFAD and IsDB. The Organization for Economic Cooperation and Development–Development Assistance Committee and United Nations Evaluation Group are permanent observers.|
|1999||The IDB Office of Evaluation and Oversight (OVE) is created to be independent of Bank Management, reporting solely and directly to the Board of Executive Directors.|
The ADB Post-Evaluation Office becomes the Operations Evaluation Office. In December 2000, the Board of Directors establishes the Development Effectiveness Committee.
|2000–01||The Independent Evaluation Office of the IMF (IEO) is created to be at arm’s length from the Board and becomes operational in July 2001. A Board Evaluation Committee is created in November 2001.|
|2002||The World Bank Multilateral Investment Guarantee Agency (MIGA) evaluation office (OEU) is established. The office transmits its reports to the Board through a Director General.|
|2003||IFAD creates an independent Office of Evaluation whose Director is responsible to the Executive Board.|
|2004||The ADB Operations Evaluation Office becomes the independent Operations Evaluation Department, reporting to the Board of Directors through the Board’s Development Effectiveness Committee.|
|2005||The EBRD Project Evaluation Office becomes an independent evaluation office, headed by a Chief Evaluator who is responsible directly to the Board.|
|2006||OED, OEG, and OEU merge. The World Bank Group Independent Evaluation Group (IEG) is led by a Director-General who appoints three Directors in consultation with management and the CODE chair.|
|2009||The ADB Operations Evaluation Department is renamed the Independent Evaluation Department by the Board of Directors to enhance the independence and effectiveness of the evaluation function.|
|The IsDB Operations Evaluation Office becomes the Group Operations Development Department. The Director reports to the Board through the Audit Committee and to the Chairman of the IsDB Group.|
|The EIB Operations Evaluation Department is put under the responsibility of the Inspector General who performs independently and is accountable to the President and to the Board Management Committee.|
See Joanne Salop, “IEO Retrospective: Ten Years of Independent Evaluation at the IMF,” Chapter 8 in this volume.
The IMF Executive Board currently comprises 24 Executive Directors who represent the Fund’s 188 members.
These concerns seem to have been shared by the Managing Director, Michel Camdessus, and his First Deputy Managing Director, Stanley Fischer. After his retirement, however, Camdessus became broadly supportive of the work of the IEO. For example, speaking at a conference in 2007 on the IEO report on The IMF and Aid to Sub-Saharan Africa he wholeheartedly endorsed the IEO’s recommendations.
Pressure mounted from external stakeholders, particularly as other international organizations established or strengthened their evaluation units. In particular, in the 1990s the evaluation offices of several international financial institutions founded the Evaluation Cooperation Group (ECG) to align and coordinate independent evaluation practices across their institutions.
“Summing Up by the Chairman: Mexico—Report on Fund Surveillance, 1993–94” (SUR/95/35, April 7, 1995).
“Statement by the Managing Director on the Development of the Evaluation Functions in the Fund—Executive Board Meeting, January 10, 1996” (BUFF/95/125, November 28, 1995).
See “Review of Experience with Evaluation in the Fund,” Appendix I, March 14, 2000, available at www.imf.org.
The reports of these evaluations, the Managing Director’s statements and staff responses, and summaries of Board discussions are available at www.imf.org.
See “Review of Experience with Evaluation in the Fund,” March 14, 2000, available at www.imf.org. In addition to the three external evaluations that were commissioned by the EG, the report considered several other evaluations carried out over the same period, including the “Review of the Resident Representatives Program,” conducted in-house by the OIA, and other self-evaluations.
See Jacques J. Polak, “IMF Study Group Report: Transparency and Evaluation,” Center of Concern, April 1998; and Angela Wood and Carol Welch, “Policing the Policemen—The Case for an Independent Evaluation Mechanism for the IMF,” Bretton Woods Project and Friends of the Earth-US, April 1998.
Interim Committee Communiqué, September 26, 1999.
The EG also produced a subsequent report, “Making the IMF’s Independent Evaluation Office (EVO) Operational: A Background Paper” (EBAP/00/84, 7/19/00). This was subsequently revised in the light of Board discussion and made available at www.imf.org on August 7, 2000.
IMFC Communiqué, April 16, 2000.
See “Making Fund’s Independent Evaluation Office Operational, Executive Board Meeting 00/81, August 3, 2000” (EBM/00/81–2, April 5, 2001), and “Independent Evaluation Office—Establishment and Terms of Reference—Report to the International Monetary and Financial Committee, Executive Board Meeting 00/94, September 13, 2000” (EBM/00/94, May 24, 2001), respectively.
“IMF Executive Board Report to the IMFC on the Establishment of the Independent Evaluation Office (EVO) and its Terms of Reference,” September 12, 2000, available at www.imf.org.
“Progress in Making the Independent Evaluation Office (IEO) Operational,” October 31, 2001, available at www.imf.org.
The section “The Lissakers Report: Evaluating the Evaluators” discusses the 2007 update to these rules and procedures, and there have continued to be some updates over the years. For a detailed explanation of the evaluation cycle, see Alisa Abrams and Ruben Lamdany, “Independent Evaluation at the IMF: Understanding the Evaluation Cycle,” Chapter 7 in this volume.
As a result of the establishment of the IEO, the Evaluation Group of Executive Directors was reconstituted in November 2002 as the Evaluation Committee, a standing committee of the Board.
There is presumed publication of IEO reports; but publication needs to be approved by the Board. So far, all IEO reports have been published.
The main sources for this section are the IEO annual reports for the years 2003, 2004, and 2005–06.
The IMF financial year runs from May 1 to the following April 30. FY2003 refers to the period May 1, 2002 to April 30, 2003.
IEO Annual Report 2003, p. 3.
The other two members were Ishrat Husain, Governor of the Central Bank of Pakistan, and Ngaire Woods, Director of the Global Economic Governance Programme at Oxford University.
“Report of the External Evaluation of the Independent Evaluation Office,” March 29, 2006 (Lissakers Report), available at www.ieo-imf.org/ieo/files/evaluationofieo/032906.pdf. The Summing Up of the Executive Board discussion of the report may be found in Part IV of this volume.
The IMF and Argentina, 1991–2001, 2004, available at www.ieo-imf.org.
As detailed in a 2002 memorandum from the Managing Director, IEO has the right to obtain all information except to the extent that the information is covered by attorney-client privilege or falls in the “zone of privacy” with respect to confidential communications of the Managing Director and Deputy Managing Directors’ office with persons or institutions outside the Fund and within and between their immediate offices or between Executive Directors and their authorities and within and between their office. The Director of the IEO is also to be granted access to side letters on the same terms as those that apply to the Executive Board.
Salop, Chapter 8 in this volume, contains an analysis of the extent to which the recommendations about the content of evaluations and evaluation reports have been implemented.
If specific circumstances require a departure from this guideline, Management must explain the reasons at a special Board meeting and consult with the Chair of the Evaluation Committee.
The IEO’s Annual Report 2007 lists 23 outreach activities between May 2006 and April 2007. Sixteen took place in the following year.
The most significant elements were to exclude administrative and staff assistants from the six-year term limit for regular IEO staff positions and to remove the constraint of a minimum initial appointment of two years for noncontractual employees.
As reflected in the Summing Up of that discussion, the Board reaffirmed the IEO’s original TOR and the 2002 memorandum providing that Management and staff may not insist on any changes to evaluation assessments.
The main sources for this section are IEO annual reports for the years 2007 to 2011.
This figure includes the Director, a Deputy Director, and two senior project leader employees, as well as seven economists and two administrative assistants. During this period, research assistants and research officers continued to be engaged on a shorter-term, contractual basis.
Two successive Deputy Directors acted as Director in the interim period: John Hicklin and Ruben Lamdany.
For example, the case for an evaluation of the Fund’s performance in the run-up to the 2008 financial and economic crisis could not have been anticipated until the crisis occurred.
See “Possible Topics over the Medium Term” (May 8, 2006), available at www.ieo-imf.org/ieo/files/workprogram/050806.pdf.
These were “Aspects of IMF Corporate Governance—including the Role of the Board,” “The IMF’s Interactions with Its Member Countries,” “The Fund’s Research Agenda,” and “The Fund’s Approach to International Trade Issues.” See “Final Work Program for FY2007 and Beyond,” at www.ieo-imf.org/ieo/files/workprogram/wp072606.pdf.
“International Reserves: The IMF’s Advice and Country Perspectives” and “The Role of the IMF as Trusted Advisor.”
“Learning from Experience at the IMF: An IEO Assessment of Self-Evaluation Systems.”
The only examples of single-country evaluations were The IMF and Argentina, 1991–2001, which itself was quite complex and controversial, and IMF Support to Jordan, 1989–2004. The latter is generally regarded as not being as useful as others, in part because it was seen as addressing a topic that IMF staff could approach in a similar fashion.
See Salop, Chapter 8 in this volume.
See Salop, Chapter 8 in this volume.
“Periodic Monitoring Report on the Status of Board-Endorsed Recommendations and Monitoring Implementation Plans” (December 3, 2007), available at www.imf.org/external/np/sec/pn/2008/pn0825.htm.
See Louellen Stedman, “IEO Recommendations: A Review of Implementation,” Chapter 9 in this volume.
Evaluation offices in other international organizations and in certain governments have emulated arrangements that help ensure IEO’s independence. Examples include the World Bank’s Independent Evaluation Group, and the Independent Commission for Aid Impact established in the United Kingdom in 2011 to evaluate its aid programs, which reports directly to Parliament.