Chapter 4 IEO: Reminiscences of the Early Years

Ruben Lamdany, and Hali Edison
Published Date:
December 2012
  • ShareShare
Show Summary Details

I had hoped to participate in the Ten Years Conference but official commitments in India prevented me from doing so. I am therefore especially grateful to the organizers for inviting me to contribute a personal statement for the conference volume.

The Background

David Peretz’s excellent paper on the history of the IEO (see Chapter 6 in this volume) points out that the need for independent evaluation of the activities of the IMF had begun to be voiced by some IMF Executive Directors from 1989 onwards. A start was made by commissioning external reviews of particular aspects of the Fund’s activities, but the idea of a permanent evaluation office did not gain traction. Peretz points out that both Michel Camdessus and Stanley Fischer had reservations because they felt that a permanent office would constantly “second guess” Management. This was a valid concern, but we need to ask why the same shareholders found it perfectly reasonable for the World Bank, across Nineteenth Street, to have an evaluation office but not the Fund.

The asymmetry is probably best explained by the fact that development agencies were viewed as using taxpayers’ money to finance development projects, and the compulsions for independent evaluation in their case arose from the need to persuade skeptical taxpayers that their money was well spent. Central banks also deploy vast resources when they have to, but because these funds are not voted by legislatures, there has never been a comparable tradition of independent scrutiny of their activities. It is reasonable to conclude that it is because the IMF was seen more as the natural counterpart of central banks than of development agencies that it was able to ward off external evaluation.1 The push for evaluation also has much to do with the culture of transparency. The culture of central banking has been quite different, tending even to glorify a degree of opacity. This is best exemplified by Alan Greenspan’s much-quoted remark to a Senate committee in 1987, “If I seem unduly clear to you, you must have misunderstood what I said.”2

The Fund, like any important institution, has always had its fair share of critics of its lending policies in individual countries and of the conditionalities attached to these policies. However, the watershed event that ultimately led to the establishment of an evaluation office at the Fund was the East Asian crisis of 1997. Fund surveillance was criticized because several emerging market economies, which until then were viewed as poster boys of good economic policy, collapsed without any early warning from the Fund. The Fund’s lending policies too came under unprecedented criticism. The Indonesia program was the most heavily criticized, because of the very large number of conditionalities that to many observers appeared irrelevant for restoring macroeconomic stability. The Korea program was severely criticized in Korea at the time, because the Fund was felt to have responded much more slowly and with much less money than in Mexico, with the result that what could have been contained as a liquidity crisis became something much bigger, necessitating painful structural reform.3 Malaysia publicly rejected the Fund’s policies and resorted to heterodox imposition of some controls on capital outflows, and appeared to do better than the countries that followed the Fund’s advice. Nor was criticism limited to the developing countries. Influential critics in the United States viewed the Fund as the cause of the crisis because it promoted moral hazard—among them was George Schultz, former U.S. Secretary of Treasury and State, who even advocated abolishing the IMF!

East Asia in 1997, foreshadowed by Mexico in 1994, signaled the emergence of a new form of crisis originating in the capital account.4 Open capital accounts and global financial integration meant that changes in perception about fundamentals, which might or might not emanate from underlying weaknesses, could trigger large capital outflows, often disproportionate to the underlying weakness. Lack of knowledge about individual emerging markets, combined with herd behavior on the part of investors, could provoke contagion. The speed and scale of capital flow reversals could be highly disruptive, requiring much larger liquidity than in the traditional current account crises and also requiring it to be provided very quickly.

The Fund had neither the resources nor the procedures to deal with these crises. In the Mexican case, Fund resources had to be supplemented by resources from major industrialized countries, and this was repeated in East Asia, with the Fund negotiating additional amounts to be contributed by other countries and also international financial institutions.5

To its credit, the membership of the Fund recognized the new reality and quickly approved a 50 percent increase in quotas in January 1998 in the Eleventh Quota Review, the first quota increase since 1990. This was supplemented by the establishment of the New Arrangements to Borrow later in 1998. However, the quota increase could only become effective after approval by the U.S. Congress, where it faced considerable resistance for a variety of reasons. In the end, prompted by the Russian default in mid-1998 and its repercussions on Wall Street because of the Long Term Capital Management affair, a compromise was reached in the U.S. Congress and the quota increase was approved in October 1998, along with a list of reforms of the Fund. The establishment of an Independent Evaluation Office was one of the elements of the reform package envisaged at the time. The IMF Board finally decided to establish the evaluation office “independent of Management” and “at arm’s length” from the Board in 2000.

I was serving as a Member of the Planning Commission in India while these events were unfolding and I recall receiving a phone call in late 2000 from a professional head hunter informing me that the IMF was launching a search for the first director of the newly created Independent Evaluation Office, and inviting me to apply. I was told that the appointment was to be made by the Board and not Management, and that they would first prepare a short list of five or six who would then be interviewed by the Board. I thanked them for thinking of me, but indicated politely that I was not interested in leaving my job in India. Several weeks later I received a second phone call, this time to say that they had short listed five very good candidates for interview but several people they had consulted outside the Fund suggested that I should be persuaded to agree to have my name added to the short list. This was clearly more flattering than the earlier invitation to apply for the job, and after some consultation with my wife, I agreed.

The Fund has always prided itself on being discreet, and in keeping with this tradition the Board decided that to avoid the publicity that would inevitably be generated if all candidates arrived to be interviewed in Washington, the candidates would be interviewed by video conference. One candidate happened to be based in Washington, but the Board decided that in order to ensure a level playing field, he too should be interviewed by video! I was delighted to be informed a few days later that the Board had decided to offer me the appointment. I accepted the offer and began to think with some trepidation of the challenge of setting up the new evaluation office.

I have been fortunate in being appointed to several senior positions in the Indian government during my career, but in all these cases I was appointed to an office that already existed, with staff already in place, and vacancies filled through an established system of civil service appointments. As the IEO’s first Director, I faced the daunting challenge of filling in an organization chart that was entirely empty, except for the box at the top which had my name. The terms of reference envisaged an internationally recruited staff whose recruitment was to be done entirely by the Director, subject only to the condition that a majority of the staff had to be from outside the IMF. I could recruit staff drawn from the IMF, but these had to be a minority, and indeed I could if I wished have no IMF staff at all.

My first important decision was the choice of a Deputy Director. Having been brought in from outside, I decided I needed a Deputy Director from within the Fund staff. I had known David Goldsbrough, then Deputy Director in the Western Hemisphere Department, from the days when he was the Division Chief on the Fund team that negotiated the IMF arrangement with India in 1991. I had led the Indian side in that negotiation and I had been very favorably impressed by his professionalism, insight, balanced judgment, coolness in discussions, and most importantly, his intellectual openness to consider a differing point of view. I contacted David on my second or third day in Washington and asked him if he would be interested in joining the IEO as Deputy Director. David said he would like to think it over and I was delighted when the very next day he told me he would be willing to come. We quickly sealed the deal. I was deeply impressed by his willingness to join the IEO on a lateral move since he was widely regarded as Director material in the Fund. He clearly came to the IEO because he felt the job was potentially important, and would strengthen the organization. His contribution to the subsequent performance of the IEO was immense.

My second important decision was to recruit just under half of the IEO staff from within the Fund. Some nongovernmental organizations expressed concern about this decision as they felt that such a large involvement of Fund staff would reduce the credibility of the evaluations. I knew the Fund’s reputation had suffered greatly in East Asia, but I have no doubt in retrospect that I made the right decision. Critical evaluation of the Fund’s performance was absolutely necessary, but the criticism had to be constructive and based on an adequate understanding of the constraints under which Fund teams must perform. Besides, our criticism and recommendations had a much better chance of acceptance by Management and staff if the evaluation teams were aware of the constraints under which the Fund works, especially in situations of crisis management.

We quickly assembled the senior team consisting of Shinji Takagi from Osaka University and Marcelo Selowsky who had worked for many years at the World Bank, and then selected others to achieve the overall balance between insiders and outsiders. No director can function without a first-rate office manager and in Annette Canizares I was fortunate to have one of the best.

Some Issues of Strategy

From the very beginning, we had decided that IEO evaluations should be forward looking, focusing primarily on lessons to be learnt from the experience being evaluated, and not backward looking, fixing responsibility for mistakes made. Both activities are important, but they require very different types of analysis and in the latter case also much higher levels of proof. I felt that fixing responsibility is something that Management should do if it wanted to.

Drawing valid lessons from past experience is not as easy as it sounds. One can generally tell in retrospect what went wrong, but to explain why it went wrong requires an agreed analytical understanding of the way an economy functions. It is only on this basis that one can identify which particular actions were responsible for the problem being studied. I recall discussing with Shinji Takagi that it was not enough to discuss with different participants and come up with an analysis that is internally consistent. We had to keep in mind what I called the “Rashomon effect,” after the Akira Kurosawa film Rashomon, which is built around the theme that many different narratives can be consistent with the same set of observed facts.

If it is difficult to be sure exactly why something happened, it is even more difficult to assert the counterfactual of what might have happened if something different had been done. Yet the counterfactual is an essential part of any evaluation. Once again, one’s conclusion depends critically on the implicit model of the economy one has in mind, and small differences in the implicit model can lead to very different conclusions. Nor do these differences need to be based on large politico-ideological differences. One need only look at the widely different views currently being expressed by established economists, of equal distinction, about the likely effect of fiscal austerity on employment and growth in industrialized countries.

Some of the criticism of the Fund’s crisis management recommendations in developing countries, which was often dismissed as merely ideological, could be validly defended as following logically from somewhat different specifications of the underlying model, and especially from different assumptions about the lags in response. Lags are particularly important since they are often longer than assumed by the people advocating particular policies. This is often the reason why the beneficial effects projected from particular policies do not materialize in the period intended. Excessive optimism about the effects of policy intervention, based on an underestimation of the lags involved, can be particularly dangerous in a world of fluid capital flows where revival of confidence is essential for the restoration of stability. The perceived failure of a program to deliver results in the period expected can actually make things worse.

To deal with these issues satisfactorily, the IEO needed high-quality expertise, and we could not possibly expect to have all the needed expertise in-house, given our small size. I was therefore convinced that we would need substantial involvement of outside consultants who had special expertise in the themes being studied or the countries involved. In the IMF unlike in the World Bank, the idea of bringing in outsiders was not part of the culture; the Fund did not use external consultants in either its surveillance or its lending operations. The Executive Board, to which I reported, was highly supportive of the IEO on all matters, but several members had doubts about the proposal for involving consultants. I was told that if we needed more resources, we should ask for more staff positions rather than relying on what some members clearly felt would be “fly by night” consultants. In the end, the Board gave us the benefit of the doubt and approved a budget that would enable us to use consultants for individual evaluations. I recall the contribution to our early reports made by consultants such as Jeffrey Frankel, Nouriel Roubini, Stephen Grenville, David Peretz, and Alfonso Bevilaqua. Their involvement enriched the quality of our evaluations and lent credibility to our recommendations.

Being the last evaluation office to be established in the family of international financial institutions gave the IEO an important advantage. We were able to develop state-of-the-art procedures to achieve the objective of independence from Management. IEO evaluation teams were given full access to Fund staff and to internal Fund documents, excepting only internal notes used by Deputy Managing Directors to brief the Managing Director, or notes of meetings of the Managing Director with representatives of member governments. These were viewed as internal communications of Management that deserved to be excluded. However, notes from staff to Management and instructions from Management to staff were made available. Draft evaluation reports were first shown to the Fund staff and their comments obtained and taken into account, especially on issues of factual accuracy. Once a report had been approved by the IEO Director, it was sent simultaneously to Management for comments and to Board members for information. The report was not changed thereafter based on comments received from Management. Instead, the comments of Management, and the comments of the IEO Director on Management’s comments, were both placed before the Board, along with the report for discussion. All these documents were subsequently included in the published version of the report, along with the Chairman’s summary of the Board discussion.

The First Four Evaluations

Choosing subjects for evaluation was not easy because we were writing on a clean slate and there was a wide range of subjects that critics of the Fund wanted studied. We adopted a deliberately consultative procedure, putting out a set of possible subjects in the public domain and inviting suggestions from a wide set of stakeholders on which evaluations should have priority for early adoption. I recall discussing various possible subjects with former Managing Directors Jacques de Larosière and Michel Camdessus; with academics such as Martin Feldstein, Peter Kenen, Fred Bergsten, Richard Portes, Stephany Griffith-Jones, Allan Meltzer, Yung Chul Park, and Takatoshi Ito; with former and current government officials such as Larry Summers, Ted Truman, Mervyn King, Eisuke Sakakibara, Pedro Malan, and Arminio Fraga; and also with nongovernmental organizations such as Oxfam, the Bretton Woods Project, Jubilee 2000, and many others.

The first four IEO evaluations were deliberately balanced between multi-country studies of thematic issues and detailed studies of individual countries. The first report, Prolonged Use of IMF Resources (IEO, 2002), was a thematic report that dealt with the phenomenon of countries repeatedly accessing Fund resources. Prolonged use ran counter to the normal expectation that countries should resort to the Fund only periodically, to deal with crises. Its prevalence raised relevant questions about whether the design of these programs was therefore internally flawed, leading to repeat lending. The second report, The IMF and Recent Capital Account Crises: Indonesia, Korea, Brazil (IEO, 2003a) dealt with recent capital account crises in three different countries. The third, Fiscal Adjustment in IMF-Supported Programs (IEO, 2003b) was another thematic report, dealing with the issue of whether IMF programs took a cookie-cutter approach to fiscal adjustment, giving too much emphasis to fiscal adjustment, as summarized in the witticism that the Fund staff thought IMF stood for “It’s Mainly Fiscal.” The fourth report, The IMF and Argentina, 1991–2001 (IEO, 2004), was a single-country case study that took an in-depth look at the IMF’s long engagement with Argentina, its enthusiastic support for Argentina’s currency board, the reasons for the collapse of this arrangement, and what this implied about the IMF’s surveillance in the run-up to the crisis and about the Fund’s crisis management.

I was actively involved in overseeing the preparation of all four reports and piloted the first three through the final stage of Board discussions. The Argentina study was approved by me as Director, but was actually discussed in the Board only after I left. This was because the change of government in India, following the general elections in May 2004, led to my being offered the position of Deputy Chairman of the Indian Planning Commission, an offer I could not resist. I apologized profusely to the Board for leaving prematurely and returned to India in June 2004.

The Argentina case study was of special interest because several observers had attributed the East Asian crisis to the fact that the countries involved were defying the impossible trinity. They were seeking to pursue simultaneously all three goals—of an independent monetary policy, an open capital account, and exchange rate stability—whereas the impossible trinity proposition held that only two of these three could be achieved and at least one must therefore be surrendered. The IMF did not want to give up open capital accounts, nor did it want to downplay the desire for exchange rate stability. The only solution under the impossible trinity was to give up monetary independence and Argentina was an example of a country that had made precisely that choice by adopting a currency board arrangement. In a public celebration of this apparent success, the Managing Director had invited President Carlos Menem of Argentina to address the Bank-Fund Annual Meeting in Washington in 1998. A little more than three years later, the currency board arrangement collapsed.

The IEO evaluation established that the collapse was primarily due to the fact that the fiscal situation was allowed to run out of control. It also established that the Fund was aware of the fiscal problem but never pointed out the catastrophic effect the problem would have in making the currency board unviable. In fact, the staff never made an assessment of whether the currency board itself was a suitable arrangement for Argentina, given the country’s fiscal circumstances. The inconsistency in Argentina—of lack of fiscal control simultaneously with exchange rate rigidity—bears some eerie resemblances to the problems facing some countries in the euro zone periphery today.

One can never be a credible judge of one’s own performance—that is best left to independent evaluation—but I do feel the IEO made a good start as the “new kid” on the evaluation block. The Executive Board received the early reports well, as did external stakeholders. IMF Management too went out of its way to be open minded in accepting the need for some rethinking. There was absolutely no interference with our work, and there was a very welcome openness to accepting many of our recommendations. There were sensitivities on some points. For example, there was concern that the timing of the Argentina evaluation, so soon after the crisis, might complicate ongoing negotiations with a new Argentine government. This was a reasonable concern but equally I felt it would have been odd if the IEO had not evaluated what had been one of the most controversial IMF-supported programs. Early evaluations do pose potential problems of sensitivity, but delaying evaluation runs the risk of making it stale. We decided we would go ahead, while strictly observing our mandate not to interfere in ongoing programs. It is to the credit of Management that they did not attempt to interfere.

Similar problems arose in our efforts to disseminate the results of evaluations of the crises in Indonesia, Brazil, and Argentina. Dissemination of evaluation results was an important part of the IEO mandate and was to be done through seminars in the countries studied. Dissemination did not present any problems in the case of the thematic evaluations covering many countries, but regarding individual country case studies it was felt that holding seminars on controversial issues might interfere with the delicate process of rebuilding the Fund’s relationship with these countries. As a result, plans to disseminate the findings of the reports in the countries themselves were scaled down. In retrospect, I feel this was not unreasonable: the reports were available to all concerned in the countries involved, and there was little to be gained by courting unnecessary controversy.

More Recent IEO Studies

I am particularly pleased to see that the IEO has gained in strength since its early years. Under the leadership of my immediate successor Thomas Bernes, and now under Moises Schwartz, it has matured into an important and accepted part of the IMF’s governance system. It has produced several important reports that have helped initiate new directions of policy in the IMF and also contributed to an understanding of the complexities of the global economy and their implications for policy. Two examples of reports produced after I left, which I feel were particularly influential, deserve specific mention.

The IEO report on The IMF’s Approach to Capital Account Liberalization (IEO, 2005) has helped to bring clarity in an area where there has been much controversy and also some misrepresentation by critics. The Fund has often been accused of pushing countries to liberalize the capital account. The IEO report reveals a more nuanced reality. It establishes that the IMF never explicitly introduced capital account liberalization as a conditionality. This is an important point though not altogether surprising—since to require liberalization, as the report notes, would have been contrary to the Articles of Agreement which do not allow the IMF to interfere with policies aimed at controlling the capital account. As the report puts it, “Throughout the 1990s, the IMF undoubtedly encouraged countries that wanted to move ahead with capital account liberalization and even acted as a cheer leader when they wished to do so, especially before the East Asian crisis. However there is no evidence to suggest that it exerted significant leverage to push countries to move faster than they were willing to go.”

That said, the IEO report also points out some degree of culpability, because “the IMF pointed out the risks inherent in an open capital account as well as the need for a ‘sound financial system,’ but these risks were insufficiently highlighted and the recognition of the risk and preconditions did not translate into operational advice on pace and sequencing until later in the 1990s.” This seems very much like a doctor mentioning the need for weight control, but avoiding specific references to cardiac risk or prescribing specific courses of action. Looking back, the position taken by the IMF on the liberalization of the capital account was broadly in line with a stream of thinking that was never widely accepted in the developing world, but was dominant among academics in the United States with some distinguished exceptions, most notably John Williamson. Efforts were being made in the mid-1990s to push for an amendment of the Fund’s Articles that would give the Fund a role in liberalizing the capital account, and the Managing Director Michel Camdessus was known to favor such a change. But the amendment was stoutly resisted at the time by most developing countries and the move died a natural death after the East Asian crisis.

The Summing Up by the Acting Chair of the Board’s discussions of the IEO report on capital account liberalization conveys the different points of view prevailing in the Board. It records broad acceptance of the report while noting that the Fund’s thinking was itself evolving. Directors agreed that the Fund has an inherent responsibility to its members to analyze the benefits and risks involved in a world of open capital markets and provide practical, sound, and appropriate policy advice to members on those issues. The Directors could not agree on the merit of an Executive Board statement clarifying the elements of agreement on capital account issues. However, they noted that they would have an opportunity to come back to this issue in the context of the ongoing strategic review. And so they did.

The IMF’s official position has moved considerably since then to the point where the IMF now explicitly recognizes that there may be conditions where control over the capital account is actually desirable. It is of course impossible to say whether the IEO evaluation led directly to this change, or whether the change would have happened in any case because of the ongoing evolution of thinking. However, it is probably fair to say that the IEO report must have had a powerful effect on the process, especially strengthening the position of developing countries, which always had reservations on this subject. No evaluator should want more.

The recent IEO evaluation of IMF Performance in the Run-Up to the Financial and Economic Crisis (IEO, 2011) has also made valuable contributions to understanding the reasons why vulnerabilities were allowed to build up in the financial sector in industrialized countries without being spotted. This failure is all the more striking because one of the conclusions of the earlier study of capital account crises (IEO, 2003a) was that financial sector vulnerabilities could lead to crises and so such vulnerabilities should be carefully watched. This conclusion had led to the Fund’s introduction of the Financial Sector Assessment Program and of Reports on the Observance of Standards and Codes as part of surveillance, but it seems these exercises remained focused on developing countries.

The IEO’s conclusion in that report that the IMF suffered from “groupthink” is particularly insightful. Obviously it was unthinkable that these problems would arise in the heart of the financial system of the industrialized world. And yet it is not as if there were no dissenting voices. Raghuram Rajan, then Chief Economist of the IMF—in a paper contributed to the Jackson Hole Conference in 2005—had specifically argued that financial liberalization in industrialized countries, rather than generating more efficient forms of intermediation, may actually have led to a buildup of vulnerabilities that could create serious problems in a crisis. He was roundly criticized—as he himself has put it—for being a Luddite! Group-think was firmly embedded in a system of cognitive dissonance that would screen out dissenting views not only in the IMF but in a much wider spectrum of the profession. The IEO report on the IMF and the 2008 crisis has received wide publicity, including explicit and favorable mention in G-20 Summit documents. This is surely an acknowledgement at the Summit level of the usefulness and acceptability of the evaluation reports of the IEO.

I have no doubt that we are all better off for what the IEO has done to subject macroeconomic management issues in a variety of countries, and the Fund’s role in these situations, to candid post facto appraisal. It has been a privilege to have had some role in the early years of the organization. It is a measure of my conviction of the value of independent evaluation as a guide to policy that we are about to establish a similar Independent Evaluation Office in the Planning Commission. I hope it proves at least as successful.


Acknowledgements are due to David Goldsbrough, Marcelo Selowsky, and Shinji Takagi for comments on a first draft while absolving them of any blame for shortcomings that remain.

In reality the IMF did use taxpayers’ resources—which is precisely why evaluation became inevitable, as explained later in this chapter.

Quoted in Guardian Weekly, November 4, 2005.

It is a different matter that the Korean government and Korean society bit the bullet with determination and quickly overcame the crisis and emerged stronger.

Managing Director Michel Camdessus famously called the Mexican crisis the first financial crisis of the twenty-first century. See Fischer (2001).

These negotiations were especially cumbersome in the case of Korea since they occurred over Christmas.

    Other Resources Citing This Publication