Back Matter

Back Matter

Author(s):
International Monetary Fund. Independent Evaluation Office
Published Date:
August 2011
    Share
    • ShareShare
    Show Summary Details
    ANNEX

    1 Timeline of Relevant Events

    2004

    • U.S.: Federal Reserve raises interest rates for first time in four years (June)

    • Basel Committee on Banking Supervision issues Basel II standards (June)

    • IMF: Rodrigo de Rato becomes Managing Director (MD) (June)

    • IMF: Biennial Surveillance Review message: financial sector and markets analysis not integrated into bilateral surveillance (July)

    • U.S.: Annual rate of increase for home prices peaks at over 20 percent (July)

    • U.S.: SEC suspends net capital rule for the “big five” investment banks (August)

    2005

    • U.S.: Greenspan notes signs of froth in local markets and calls home prices unsustainable (June)

    • IMF: Rajan Jackson Hole speech notes financial development has made the world riskier, incentives in financial sector are skewed; paper predicts low probability high-cost downturn (August)

    2006

    • IMF: MD directs that analysis of financial sector and balance sheet vulnerabilities be integrated into staff reports (March)

    • IMF: DMD says Fund work moving to less of firefighter and more of (preventive) doctor (June)

    • U.S.: Home prices peak (July)

    • U.S.: Federal Reserve maintains interest rates for first time after two years of increases (August)

    • U.K.: Daily Telegraph predicts credit crunch (September)

    • IMF: ICM/MFD merger effective to form new department, MCM (December)

    January–June 2007

    • U.S.: Subprime mortgage market collapses

      • —Freddie Mac announces no purchase of risky subprimes/MBS (February)

      • —Beginning of forced sale of Countrywide and other mortgage lenders (February)

      • —New Century Financial Corp. files Chapter 11 (April)

    • IMF: Multilateral Consultation on Global Economic Imbalances (April)

    • IMF: Executive Board reviews 1977 Decision on Surveillance (January–June)

    • U.S.: Moody’s downgrades 100 subprime-backed bonds (June)

    • U.S.: Bear Stearns suspends redemptions on hedge funds (June)

    July–December 2007

    • U.S.: Financial sector under stress

      • —S&P credit watch on 612 securities backed by subprime mortgages

      • —Bear Stearns liquidates two MBS hedge funds (July)

      • —American Home Mortgage Invest. Corp. files Chapter 11 (August)

    • Global financial markets show signs of stress; diminished liquidity in interbank markets (August–September)

      • —BNP Paribas halts redemptions on three funds

      • —European Central Bank (ECB) injects €95 billion into market

      • —U.S. Federal Reserve reduces discount rate (August), federal funds rate (September)

    • U. K.: Bank of England liquidity support for Northern Rock

    • IMFC: strong fundamentals, robust emerging markets/developing country growth (October)

    • IMF: Dominique Strauss-Kahn becomes MD (November)

    • U.S. FOMC initiates temporary swaps for six months (ECB, SNB) (December)

    • MD: Messages (December)

      • —External: Fund has key role to play in “credit crunch”

      • —Internal: cut $100 million, move ahead with downsizing effort

    January-August 2008

    • IMF: Working Group on Financial Crises of the Future headed by FDMD (January)

    • U.K.: Northern Rock taken into state ownership by U.K. Treasury (February)

    • U.S.: Federal Reserve announces financing for JPMorgan Chase to acquire Bear Stearns (March)

    • IMF: Executive Board discusses turmoil; approves new income model (March), administrative restructuring/downsizing (April)

    • U.S.: Government auctions, existing swap lines increased (May)

    • IMF: Public release of MD statement noting shift from internal issues to focus on key global economic and financial concerns (July)

    • U.S.:

      • —Office of Thrift Supervision places IndyMac into receivership (July)

      • —Temporary authorization to purchase Fannie/ Freddie equity if needed (July)

      • —FOMC announces “downside risks to growth have increased appreciably” (August)

    September 2008

    • U.S.: Critical financial market events

      • —Fannie/Freddie placed in government conservatorship (September 7)

      • —Lehman Brothers files for bankruptcy (September 15)

      • —Government provides emergency loan to AIG (September 16)

    • Money market run begins/credit markets freeze (September 16–17)

    ANNEX

    2 Factors That Contributed to the Crisis According to IMF Staff

    This annex is drawn from the IMF’s own ex post analysis. According to IMF staff, the following factors contributed to the crisis:35

    Macroeconomic forces. A long period of high growth, low real interest rates, and limited volatility led to excessive optimism about the future, pushed up asset prices and leverage, and prompted a search for yield and an underestimation of risks.

    • Monetary policy. Short-term interest rates were low, reflecting accommodative monetary policy. Central banks and financial regulators largely focused on inflation and aggregate activity, thereby paying insufficient attention to the buildup of systemic risk associated with rapid asset price increases (particularly in housing markets) and growing leverage.

    • Global imbalances. These too played a role in the buildup of systemic risk. High saving in Asia and oil-surplus countries had as their counterpart large capital inflows to the United States and Europe. This contributed to low long-term interest rates, underpinning the rise in asset prices, leverage, a search for yield, and the associated creation of riskier assets.

    Global architecture. A fragmented surveillance system compounded the inability to see growing vulnerabilities/risks. Multilateral coordination and collaboration lacked sufficient leadership to achieve the needed response to systemic risks. On financial regulation, there were no ex ante rules governing cross-border resolution or burden sharing. The absence of broad liquidity insurance implied an inadequate international response when interbank markets around the world froze up.

    Financial system. New structures and new instruments were riskier than they appeared. A presumption that these instruments dispersed bank risk ignored the larger fact that risk remained concentrated in entities linked to the core banking system. Market discipline failed amid the prevailing optimism, due diligence was outsourced to credit rating agencies, and a financial sector compensation system based on short-term profits reinforced risk-taking.

    • Regulatory perimeter. A lightly regulated and generally unsupervised shadow banking system in the United States had grown as large as the formal banking system. Banks evaded capital requirements by pushing risk to affiliated entities in the shadow system. Regulation was not equipped to see risk concentration and the flawed incentives behind the financial innovation boom. There were shortcomings in consolidated supervision and underwriting standards.

    • Market discipline. Due diligence—in assessing counterparties and collateral—failed. Supervisory and regulatory incentives led to too much reliance on credit ratings whose methodologies were inadequate and inappropriate when applied to complex structured products, and thereby failed to capture the risks. Ratings agencies were also subject to conflicts of interest. Market discipline was eroded by the “too big to fail” nature of the largest most interconnected institutions. The complexity and opacity of structured credit instruments undermined market discipline. Risk management practices of many financial institutions were deficient, reflecting shortcomings in judgment and governance: the users of risk management models used poor business judgment, and warnings by risk managers were sometimes ignored or underestimated by senior management.

    • Pro-cyclicality. A constellation of regulatory practices, (fair value) accounting treatment of structured products, ratings, and incentives magnified the credit boom and exacerbated market turbulence. Some recent regulatory initiatives (such as Basel II) may have also intensified pro-cyclical behavior.

    • Information gaps. Financial reporting was inadequate, understating the risks borne by the reporting entities. There were extensive gaps in regulators’ and markets’ data and understanding of underlying risks. These included risks embedded in complex structured products, the degree of leverage and risk concentration in systemically-important financial institutions, the difficulty of assessing liquidity and counterparty risk, and on-balance-sheet risks and links with off-balance-sheet risks. Shortcomings in valuation models and practices played a role.

    • Crisis management. Cross-border differences in emergency liquidity frameworks and inadequacies in crisis management frameworks, including deposit insurance, played a role in propagating the crisis.

    ANNEX

    3 Country Coverage

    The countries/economies covered by the evaluation are: Argentina, Australia, Austria, Belarus, Bosnia and Herzegovina, Brazil, Canada, China, Colombia, Costa Rica, El Salvador, European Union, France, Germany, Guatemala, Hungary, Iceland, India, Indonesia, Ireland, Italy, Jamaica, Japan, Korea, Latvia, Luxembourg, Mexico, Poland, Romania, Russia, Saudi Arabia, Serbia, Seychelles, South Africa, Spain, Switzerland, Turkey, Ukraine, United Kingdom, and United States.

    This list includes the G-20, and those countries (excluding low-income countries) that initiated a new IMF arrangement, including contingent commitments under an FCL, in the aftermath of the crisis (through 2009). Also included are financial centers such as Luxembourg and Switzerland, and countries such as Ireland and Spain that had vulnerabilities similar to those that precipitated the crisis in the United States and the United Kingdom.

    ANNEX

    4 Early Analysis and Diagnosis of Factors Leading to the Crisis

    A number of analysts outside the IMF pointed to the vulnerabilities and policy shortcomings that eventually led to the crisis. The following briefly reviews some of these contributions through 2006.

    Warning about the prospect of a housing market collapse:

    • Illustrating the entrenched nature of home price speculation by viewing the ongoing appreciation in historical perspective (Shiller, 2005);

    • Predicting recession via asset price adjustment (Krugman, 2006; Richebacher, 2006);

    • Linking unsustainable household balance sheets to a dramatic reversal of household spending (Parenteau, 2006).

    Forecasts linking a housing market collapse to financial implosion:

    • Recognizing that an asset bubble backed by unsupportable subprime mortgages could not endure (Burry, 2005, as described in Lewis, 2010),

    • Probing where the mortgage risk was located and the repercussions for the institutions holding it after the prospective housing bust (Roubini, 2006).

    Highlighting regulatory shortfalls and ensuing risks:

    • Warning about the need to strengthen disclosure requirements and oversight over OTC derivatives (Commodity Futures Trading Commission, 1998)

    • Warning about the risks and conflicts of interest inherent in using private credit ratings to measure loan quality as the basis for lowering capital requirements (Shadow Financial Regulatory Committee, 2000).

    • Highlighting an array of risks arising from the evolving nature of structured finance (summary of proceedings from conference organized by the IMF Institute, 2005).

    Urging monetary policy to take account of its impact on credit expansion and asset prices and warning of the drawbacks of not doing so (Borio and Lowe, 2002; Borio and White, 2003).

    But some within the IMF also were quite prescient regarding the evolving risks and vulnerabilities, as evidenced by the contributions below through 2006.

    Pointing to risks in the evolution of financial markets:

    • The IMF’s Economic Counsellor warned in his personal capacity that the evolution of financial development and the nature of compensation incentives for investment managers were driving the financial system toward increased risk, which ultimately could freeze the interbank market and lead to a full-blown financial crisis (Rajan, 2005a and 2005b);

    • “Liquidity shortage as a potential amplifier for market price shocks was a major ‘blind spot’ and will need to be at forefront of all future effort to further improve the global financial architecture” (GFSR, 2005);

    • “Historically the most important risk for financial markets in good times is complacency. Current risk premiums leave little or no room for asset valuation errors” (GFSR, 2005);

    • The cyclical and structural shift in global financial markets could “become hazardous to financial stability” (GFSR, 2005);

    • A combination of low risk premiums, complacency, and untested risk management systems dealing with complex financial instruments could become hazardous to financial markets. The proliferation of complex, leveraged financial instruments (such as credit derivatives and structured products) made liquidity risk increasingly relevant (GFSR, 2005).

    Highlighting regulatory shortfalls and ensuing risks:

    • From unregulated OTC derivatives, including those relating to the liquidity consequences of the unraveling of derivative contracts. “There could be a tsunami of credit evolving into a perfect storm …,” as he warned of counterparty risk and evaporating liquidity (Schinasi, 2006);

    • “… credit risk which appears to have left the banking system may in fact turn out not to have done so” (Executive Board member’s statement on the GFSR, 2006).

    Forecasts linking a housing market collapse to financial implosion:

    • The “longer [asset bubbles unjustified by fundamentals] persist, the greater the potential for disruptive corrections” (GFSR, 2004).

    Warning about the prospect of a housing market collapse:

    • “particularly concerned” about buoyant property prices in the United Kingdom, Australia, Ireland, and Spain, and to a lesser degree in United States and New Zealand (WEO, 2004);

    • “heightened concerns” about an asset price bubble and a sharp correction thereof (WEO, 2004);

    • Concern about the possibility of a synchronized downturn with significant adverse effects (WEO, 2004).

      Sketching out the contours of a systemic financial crisis in the context of global imbalances (background paper for the multilateral consultation by team of IMF financial experts, 2006):

      • “… the adjustment of the global imbalances poses financial sector risks. Global imbalances have counterparts in the sectoral balance sheets and the portfolios and risk exposures of financial institutions. A disorderly adjustment would likely impact on the sectors where the banks are most heavily exposed … Assessing the behavior of capital markets under a disruptive scenario is … challenging … as it entails financial products and markets that have yet to be tested under global systemic distress. These effects have not been factored into the subsequent analysis of risks to the banking systems, but merit attention during the multilateral consultations.”

      • “… concern[ed] about the increased use of nontraditional mortgage products for which default histories were limited … while the historical loss experience on mortgages has generally been low, the growth of innovative mortgage instruments has increased potential risks. A significant correction in house prices combined with a slowing economy could result in a significant increase in delinquencies on loans to households as well as commercial real-estate loans. To the extent that nontraditional mortgage products may not be completely understood by borrowers, an environment of higher interest rates may trigger reputation and litigation risks to banks.”

      • “… In several countries, banks and other financial institutions are heavily exposed to the housing market, including to the U.S. mortgage market through investments in mortgage backed securities. Since the ultimate effects of risk transfer across institutions and sectors are largely unknown, it is also possible that counterparty risk and unwarranted risk concentrations could lead to financial contagion, amplifying the costs of a disruptive scenario.”

    ANNEX

    5 Weaknesses in FSAPs in Advanced Economies

    This annex describes the main factors that contributed to a mixed record in the quality and usefulness of FSAPs in advanced countries. It draws on the FSAPs of advanced countries during 2004–08 and staff’s 10-year retrospective of the FSAP (IMF, 2009b).

    Lack of candor and clarity. This seems to have been more of a problem in the FSAPs for advanced than for other countries, as some of the IMF’s assessments for emerging markets were pointed and direct about risks and vulnerabilities. According to IMF (2009b), lack of candor and clarity “might be symptomatic of a desire of team members to avoid conflict with national officials.” The typical tendency was to present a “balanced” view, beginning with a positive statement before acknowledging any risks.

    Inadequate or lack of coverage on topics relevant to the crisis. Coverage of liquidity risks, crisis preparedness, bank resolution, and external funding risk seemed less consistent in the FSAPs for advanced countries than for emerging markets. To assess liquidity risks, for example, FSAPs sometimes reviewed only the central bank’s liquidity management instruments. Some aspects of capital markets that should have received attention in advanced countries—asset securitization, commercial paper, and short-term funding markets—were not routinely covered.

    Stress test weaknesses. According to IMF (2009b), “stress tests … did not provide significant insights regarding the crisis.” Reasons include: specifying shocks that were not sufficiently severe (reflecting, in part, the sensitivity of country authorities and the difficulty in “thinking the unthinkable”); missing important sources of instability—liquidity risks, concentration of exposures in real estate, off-balance-sheet exposures; working with inadequate data, particularly regarding off-balance-sheet exposures and balance-sheet interconnectedness; as well as methodological challenges in modeling liquidity risk, contagion channels, second-round effects, nonlinearities, and correlation across portfolios.

    Failure to integrate multilateral perspectives. The FSAPs for most countries did not discuss the global macroeconomy nor the developments taking place in countries with strong economic ties to the subject country. They typically focused on domestic issues and scenarios and did not look at cross-country risks or spillovers, crosscutting issues, or global economic risks. In fact, in those instances where global risks were considered, the scenario was the impact from a disorderly collapse of the dollar in line with the IMF’s focus, which is not the way the crisis impacted financial sectors.

    Reassuring messages that induce complacency. Among the key messages from advanced county FSAPs in the run-up to the crisis were: “the outlook for the financial system is positive;” “financial institutions have sufficient cushions to cover a range of shocks;” “the diversification of sources of foreign wholesale funding is a source of strength;” “stress tests … suggest that the financial system as a whole is well positioned to absorb a significant fall in housing prices;” “the financial sector is generally sound and should be resilient to large, but plausible shocks;” “no weaknesses that could cause systemic risks were identified.”

    ANNEX

    6 Conclusions/Recommendations from Previous Reports and Evaluations

    Whittome Report on Fund Surveillance of Mexico (1995)

    • IMF culture does not encourage frank discussion of risks. Staff in habit of second-guessing Management and Board.

    • Managing Director must insist that analysis be pertinent, pointed, and take responsibility for degree of “political” understanding that should be allowed to affect the staff’s conclusions.

    External Evaluation of Fund Surveillance (1999)

    • Fund should place greater emphasis in surveillance on financial sector and capital markets issues.

    • Need greater linkage between bilateral and multilateral surveillance.

    • The Board, Management, and senior staff should attempt to alter the incentive structure by making it clear that they will, if necessary, back up staff who give frank advice.

    • Surveillance should devote more time to identification and analysis of alternative policy options.

    • More financial sector expertise; more policy expertise (such as through secondment or interchange programs); and more outside experience in general to mitigate against insularity, conformity, and lack of hands-on experience.

    Lipsky Report (2001)

    • Focus, expertise, and support on financial sector/ capital markets issues should be enhanced.

    • Weak linkages between multilateral surveillance of capital markets and the Fund’s core bilateral surveillance activities.

    • More effort needed by area departments to follow financial market developments in countries.

    • Active role of Fund Management in making financial sector work more effective. Requires “clear-cut support of senior management” to overcome “natural institutional inertia.”

    McDonough Report (2005)

    • Provide incentives for interdepartmental collaboration to increase cross-fertilization between traditional macroeconomics and financial/capital market issues; overcome silo mentality that is reducing the IMF’s overall effectiveness and influence. Requires clear direction from Management and Executive Board.

    • Fundamental change of orientation and mindset required for all departments, Management, and Executive Board with incentive structures to reward collaboration and penalize silo behavior, set clear objectives on what is expected in terms of integrating financial issues into surveillance. Sustain follow-up to ensure accountability.

    • Clear guidance, continuous monitoring, and direct, regular, continuous, and visible engagement and leadership by the Managing Director and the Fund’s senior leadership are required.

    • Fundamental mind-set change in how the Fund thinks about financial issues. Put financial issues at the center rather than the periphery. Area departments yet to fully embrace the need to change the traditional macro focus and elevate financial issues to a central role in their work. Teams still comprise traditional macroeconomists who lack the necessary comfort level or expertise on financial issues.

    • Departments set their own agendas and priorities. Systematic collaboration is exception rather than rule, and largely limited to calendar-driven events. Problems symptomatic of broader “silo” mentality across departments impeding cooperation, and incentive structure rewards looking up (to Management and the Board) rather than across the institution. Internal silos can only be overcome with strong management.

    • Having two separate publications (WEO and GFSR) raises questions of overlap and efficiency, and does little to reinforce an integrated view of the links between global macro and financial developments. The GFSR is not widely read or used by staff within the organization, and does not play a significant role in country work.

    IEO Evaluation on the Financial Sector Assessment Program (2006)

    • Improve the quality and impact of FSAPs through clearer prioritization of recommendations; improved stress-testing analysis; and more systemic inclusion in the analysis of cross-border, financial sector linkages.

    • Strengthen links between FSAPs and Article IV surveillance by mainstreaming FSAPs and follow-up work into regular surveillance activities. Strengthen the internal review process to ensure that key messages on macro-financial stability are fully reflected in Article IV.

    • Management should clearly signal to the Board those countries that it sees as the highest priorities for FSAPs and Updates, irrespective of whether these countries have volunteered.

    • Utilize financial sector expertise (especially in MFD and ICM) more effectively in the surveillance process.

    IEO Evaluation of Multilateral Surveillance (2006)

    • Enhance role of Board and IMFC in multilateral surveillance.

    • Improve content/form of multilateral surveillance outputs through streamlining and more focus on key issues.

    • Strengthen multilateral surveillance by clarifying operational goals, organizational strategies, and accountability. Clarify scope of regional surveillance.

    • Integration between WEO and GFSR and bilateral and multilateral surveillance (silo structure; bottom-up approach; too many products, too little focus).

    2008 Triennial Surveillance Review (September 2008)

    • Need to strengthen risk assessment (connect dots), highlight unknowns, think the unthinkable, guard against tail risks, incorporate risks at multilateral/ regional level.

    • Better integrate macroeconomic and financial sector surveillance.

    • Do better cross-border inward/outward spillover analyses, cross-country analyses, exchange rate analyses.

    • Pay attention to effective communication; preserve existing strength.

    ANNEX

    7 How Did Country Authorities View the IMF’s Performance?36

    The country authorities who were interviewed were almost unanimous in the view that the Fund failed to warn sufficiently about the risks and vulnerabilities that led to the crisis.37 However, few of them blamed the Fund or the individual mission teams for this failing. They admitted that most observers (including themselves and their fellow authorities) had also been overly comforted by the prolonged benign global environment. As one interviewee put it, “Neither we nor the IMF staff exercised imagination.” The few outside voices that had expressed grave concerns (William White and Nouriel Roubini were among the most frequently cited) were typically not heeded in this seemingly “new paradigm” of a more stable global financial system, underpinned by innovation and risk dispersion.

    Despite the Fund’s failure to warn of the impending crisis, country authorities, in most cases, had much positive to say about the Fund and the bilateral surveillance process. Among the positives were a high general regard for Fund staff competency and analysis. The authorities felt that discussions with mission teams were usually candid, constructive, and of high quality, bringing useful and independent third-party views to the policy debate. Furthermore, most of those interviewed believed that the Fund’s financial sector analysis had improved significantly over the years, and they had a generally high regard for the Financial Sector Assessment Program (FSAP) in particular. FSAPs had often been the catalyst to strengthen countries’ financial sector policies, including spurring countries to do their own stress testing and move toward international best practices in supervision and regulation.

    At the same time, country authorities provided many criticisms regarding the Fund’s performance prior to the crisis. The subjects ranged from analytical weaknesses to political biases, the surveillance process, and organizational problems.

    On the analytical front:

    • The Fund’s general mindset that markets know best and financial innovation reduces risks would have made it difficult for the staff to see the buildup of systemic risks.

    • Bilateral surveillance typically focused primarily on domestic policies and vulnerabilities, offering little analysis of spillovers and contagion (even in the case of small, open economies).38 Where there was some discussion on spillovers or contagion, the Fund usually saw the problems as arising from emerging markets, not from the advanced economies.

    • Notwithstanding improvements over the past decade, the Fund still had not adequately linked macroeconomic with financial sector analysis. This inadequacy was reflected in the heavy reliance on models that to date have been unable to adequately capture macro-financial linkages.39

    • Balance sheet analysis was infrequently employed. Furthermore, when it was used, it was sometimes done incorrectly.

    • While the IMF had performed no worse than others in foreseeing the crisis, it had not used its comparative advantage in analyzing cross-cutting global issues and identifying risks.

    • More use of cross-country analysis (particularly on countries that were facing similar issues) might have helped in identifying common vulnerabilities.

    On political biases:

    • A repeated theme was the apparent lack of evenhandedness in how the Fund treats its largest shareholders versus all others. Many country authorities believed that the Fund offered much more hardhitting critiques of the policies of emerging markets and smaller advanced countries. Meanwhile, even when there were obvious commonalities in vulnerabilities with smaller countries, the large advanced countries were given the benefit of the doubt that their policymakers, supervisors, and regulators would be able to steer their economies through any rough patches. The 2007 Decision on Bilateral Surveillance only heightened this sense of unequal treatment. This perception also came out clearly in the survey of country authorities for the IEO’s evaluation of IMF Interactions with Member Countries; for example, 86 percent of survey respondents from large emerging markets said that surveillance was in the interest of the “largest IMF shareholders.” In particular, some felt that the IMF was insufficiently critical of the policies of a major shareholder.

    On the surveillance process itself:

    • A number of country authorities recognized that the Fund had identified many of the risks and vulnerabilities but typically presented these in a “laundry list of warnings, with no prioritization.” That is, all Fund staff reports had the usual economist approach of “on the one hand (with list of economic positives first—which sets the tone), followed by on the other hand (with list of downside risks).” They asked how one should respond to such a wide-ranging list of risks, listed with no sense of probabilities nor urgency.

    • Policy recommendations were often obvious (e.g., tighten fiscal policy, pursue a credible and sound monetary policy, or strengthen supervision) but lacked specificity about how to implement them. According to one interviewee, “interactions on the Article IV often feel like just any other meeting I have with all those international institutions, too formulaic.”

    • As for the value added by the Executive Board to bilateral surveillance, nearly all felt this was minimal at best, as the Board’s contributions were usually very belated (coming months after the mission team’s concluding statement had been presented to the country authorities) and often superficial (e.g., Summings Up were typically a fairly generic reiteration of the staff report).

    On organizational problems:

    • The high turnover of staff on mission teams was often cited. This implied a considerable loss of country knowledge and a constant training of new mission members to understand country specifics, history, and culture, all of which are very important for providing relevant policy advice and gaining traction.

    • The turnover problem was worsened by the IMF’s restructuring exercise, which was conducted precisely when the crisis was taking hold. In some cases, the restructuring caused countries to experience a complete turnover of mission members or even periods with no mission chief.

    • Finally, the more general issue of staff resources was also reflected in the very infrequent FSAP updates. More continuous follow-up on financial sector issues might have better illuminated the problems ahead of time.

    Interviewees also raised some issues that could be interpreted as having aspects which were both positive and negative regarding the Fund’s performance:

    • In almost everyone’s view, the Fund must walk a very fine line between highlighting the risks of a crisis and actually precipitating one. For this reason, more sensitive messages would sometimes be communicated privately and orally to the authorities. However, on occasion, the authorities on the receiving end of such messages admitted that they did not remember what was said, because the only documented views of the mission were in the concluding statement.

    • Many of the authorities agreed that the Fund teams clearly highlighted the domestic vulnerabilities and risks … but said that those were obvious to everyone.

    • Finally, while the WEO and GFSR40 pointed to many of the pertinent risks and vulnerabilities and were generally held in high regard, policymakers did not notice any warnings regarding an impending crisis. This was widely attributed to the overall upbeat banner messages that typified these documents in the run-up to the crisis.

    ANNEX

    8 Area Department Survey of Staff

    In a self-evaluation of what went wrong, one of the area departments whose countries were most affected by the crisis conducted a survey of staff in October 2009. The following highlights the staff’s views on some of the issues most relevant to this evaluation.

    On the substance of country work, just under half the staff members thought that the area department was strong or very strong in assessing vulnerabilities. The toolkit for macro-financial analysis was often cited as an analytical impediment.

    On where the area department should place the priority in country work, a staggering 98 percent of staff thought it important or very important to prioritize work on vulnerabilities and crisis risks, more so than even fiscal or monetary policy. Write-in responses to the question of priorities repeatedly stressed the need to do more work on cross-country linkages, spillovers, and integration of regional with country-specific perspectives.

    On the main problems in the area department’s surveillance work and ways to fix them:

    • “relations with the authorities. We do not have the incentives to be too critical, especially publicly and to differ substantially. More support from the front office/management, less pressure to make authorities happy, more consistent ‘ruthless truth telling’ across all countries, not just a few.”

    • “more formal and informal communication with functional departments, mainly MCM …” “The main problem is how to bring value added to large economies, which have large staffs of highly trained economists. The solution is to focus on the Fund’s comparative advantages, namely crosscountry work, spillovers, and global consistency.”

    • “no courage to take on countries, especially G7. For years we praised [a large systemic country] for its policy framework and now we have egg on our face.”

    On leadership and communication, just under half the economists agreed with the statement “your ideas and opinions are considered and listened to.”

    On incentives, fewer than a third agreed that the area department gets its voice heard in the interdepartmental review process for policy papers, the WEO, the GFSR, etc. As for why this is the case, almost half believed that incentives (e.g., one gets little credit for good comments) were a serious hurdle. Meanwhile, almost three-quarters of respondents agreed with the statement that “cross-country work faces several constraints, including managerial complexity, incentives, resources, and priority of bilateral relations.” Incentives, for example, were cited by about 85 percent of respondents as a hurdle to producing cross-country work.

    On the silo nature of the Fund, only one-fifth of survey respondents agreed that there was sufficient learning from peers across country teams (and this lack of collaboration within a department bodes poorly for across-department collaboration).

    On intellectual leadership, well over half the respondents felt that department managers had not provided the intellectual leadership to get the job done to a high standard. Some respondents felt that the Fund’s downsizing exercise had impeded the ability to provide intellectual leadership. For example, one respondent wrote that “the [conjuncture] of the restructuring and the crisis has had disastrous consequences on the leadership provided by the department.”

    References

    Statement by the Managing Director Staff Response The Chairman’s Summing Up

    Statement by the Managing Director on the Independent Evaluation Office Report on IMF Performance in the Run-Up to the Financial and Economic Crisis

    Executive Board Meeting

    January 26, 2011

    1. I thank the IEO for putting forward many constructive ideas, which I broadly endorse. The failure of the Fund to warn about a systemic crisis in a sufficiently early, pointed, and effective way is a humbling fact that the institution has been frank about acknowledging and prompt about responding to. Indeed the focus the reform agenda being implemented is precisely on strengthening surveillance and financing for systemic stability.

    2. Since the IEO’s recommendations are at a high level of generality, it is incumbent on us to ask how they can be made actionable within competing work program priorities and budgetary constraints. The reforms in train since the onset of the crisis—the early warning exercise, the vulnerability exercise for advanced economies, G20 MAP inputs, integration of WEO-GFSR messages, mandatory financial stability assessments for the systemic countries, and cross-country and spillover reports to name only a few—will go a long way to enhance the candor and traction of surveillance, and arguably already have done so. In thinking about the scope for further progress, I would like to highlight a few points.

    3. First, on the promotion of more diverse and dissenting views, we should consider carefully the idea of allowing direct inputs by eminent outside experts into systemic surveillance—e.g., WEO, GFSR, Article IVs of systemic countries. This would make available to the Board independent and real time critiques of staff and member countries’ policy positions. For instance, outside experts, including those known to hold differing views, could from time to time provide commentary on staff reports on systemic issues and countries during the circulation period prior to Board discussion. Surveillance is peer review, but peer review can be made more effective with outside input, increasing the imperative, as the IEO puts it, “to speak truth to power.”

    4. Second, there are several IEO recommendations aimed at allowing staff to “connect the dots” better. Integrating the analysis of the WEO-GFSR is certainly a crucial dimension of that task, but does not necessarily equate to merging the documents. Incorporating financial stability assessments in Article IVs is another one on which I expect efforts will be intensified following the Board’s recent decision on FSAPs. And yet another important initiative is the new Fiscal Monitor, which will allow a much closer look at emerging fiscal risks. The rising number of cross-departmental products—thematic reports, spillover reports, recent work on capital flows—are all signs of important progress of staff reducing silos, and on which we have to build. Meanwhile, there should be no doubt that we are committed to fostering staff diversity in all its dimensions, including diversity of opinion.

    5. Third, on the delivery of clear messages on risks and vulnerabilities beyond those in the WEO-GFSR, we are now doing more via the early warning exercise to the Board and the IMFC in restricted settings. This is also increasingly the case in Article IVs, as exemplified by the recent Euro Area mission concluding statement. Nevertheless, we should think about doing more, perhaps even putting out large parts of the EWE in the public domain, with appropriate commentary and safeguards.

    6. I look forward to hearing your views on the important issues raised in the IEO report.

    Staff Response to the Independent Evaluation Office Report on IMF Performance in the Run-Up to the Financial and Economic Crisis

    Executive Board Meeting

    January 26, 2011

    This report nicely complements IMF analysis about the failure of Fund surveillance to adequately anticipate and warn about the global crisis. While it could have been more specific in certain areas and staff have concerns about some methodology, inferences, and factual errors, none of this detracts from the correctness of the report’s recommendations.

    General Points

    1. On the bottom line, two points should be acknowledged up front.

    • First, the IEO correctly identifies the Fund’s failure to call attention to the buildup of vulnerabilities and risks in the global financial system. This is in line with recent IMF work, such as the 2009 paper on the Initial Lessons of the Crisis, the 2008 Triennial Surveillance Review, and the recent review of the Fund’s Mandate. These papers came to similar conclusions and proposed reforms.

    • Second, it is true that much remains to be done to implement remedies, many of which are well in train but whose effectiveness remains to be seen. The IEO report briefly highlights a few of these, such as the inclusion of advanced economies in the Vulnerability Exercises (VEs), the launching of the Early Warning Exercise (EWE), more research on macro-financial linkages, the introduction of spillover reports for systemic economies, and the move to mandatory stability assessment modules under FSAPs for systemically important financial centers. The larger challenge will be to ensure that this new work consistently finds its way, as it lately has done, into frank discussion of vulnerabilities and responses—at least in private when it is not possible to do so in public.

    2. On the core issue of why the Fund failed to predict the crisis, the report examines institutional factors, some of which ring true and others that warrant more reflection. On one side, despite progress, it is clear that we need to enhance our capacity to better “connect the dots” between financial and macroeconomic surveillance and between multilateral and bilateral surveillance. On the other side, the report ultimately ascribes the failure to warn about the crisis to “groupthink,” which is as much a description as an explanation. The report could have looked more at the extent to which staff considered contrarian views (arguably, they did) and how they judged these positions against the much larger evidence marshaled by the mainstream (clearly, they judged incorrectly). This also speaks to the IEO recommendation to increase financial expertise and staff diversity—which undoubtedly is correct, and indeed a goal of the institution, but does not follow from the pre-crisis experience: the vast majority of financial experts, from a diversity of countries and backgrounds, also failed to see the crisis coming. (Ironically, the prescient individuals cited by the report are from remarkably undiverse backgrounds—i.e., macroeconomists with PhDs from U.S.-U.K. universities.) That said, the recommendation to access thoughtful and diverse opinion is a very important one, and one that we return to below.

    3. Staff also have concerns about some other aspects of the report, including the dismissal of the role of data gaps. Lack of information about off-balance-sheet exposures, risks housed in the shadow banking sector, interconnections (national and international), and bank-specific balance sheet data severely hampered real-time analysis. Such data could have said a lot about core issues—such as whether securitization was dispersing risk or concentrating it. Indeed, this subsequently led to significant multilateral initiatives on filling data gaps and exploring financial networks. Moreover, the resistance of supervisors to share relevant data on globally important institutions should have been emphasized as an important finding. Finally, the report is not without misrepresentations. For example, a closer reading of the text around the selective quotes on the United Kingdom and the United States makes it clear that staff did cite many of the vulnerabilities seen over the crisis, but incorrectly judged their systemic importance. Similarly, the claim that the IMF had been recommending faster capital account liberalization in India is at odds with the documented record and even with the IEO’s own 2005 report on The IMF’s Approach to Capital Account Liberalization.

    Recommendations

    4. Staff strongly agrees with the thrust of the IEO’s five recommendations. While there has been a comprehensive program of reform since 2008, it is clear that further improvements could be made. Staff welcomes this opportunity for the Board to discuss where best to focus these efforts, while being mindful of the current budgetary environment. Many of these could be taken up in the forthcoming Triennial Surveillance Review (TSR).

    Recommendation #1: “Create an environment that encourages candor and diverse and dissenting views.”

    5. Staff agree that more can be done to seek alternative or dissenting views. In particular, the Board may wish to consider the case for direct interactions with eminent outside analysts, especially to present contrarian views, in both bilateral and multilateral surveillance. It should be noted, nonetheless, that much is already in train on enhancing outreach efforts by Fund Management and staff. These include the establishment of regional advisory groups, dedicated units within departments that focus on outreach and liaison, and heightened outreach to external stakeholders such as in the context of bilateral and multilateral surveillance (e.g., related to the WEO, GFSR, EWE, and the VEs) and reviews of and reforms to IMF policies and facilities.

    6. We also agree that broadening financial sector expertise of IMF staff is important. Efforts need to continue in hiring financial sector experts and managing their career progression once in the Fund. In particular, consideration should again be given to developing a non-management promotion (“guru”) opportunities for these experts, while being mindful of budgetary considerations.

    7. The recommendation that “Summings Up of Board discussions better reflect areas of significant disagreement and minority views” warrants further discussion. This is more an issue for the Board than for staff, but clearly any change in this area would involve a re-think about the nature and purpose of Summings Up, which by design emphasize consensus building. The recommendation would benefit, though, from empirical backing, as the examples of Summings Up provided in the background paper (Switzerland and the United States) suggest that pervasive concerns about the outlook were acknowledged by the Board.

    8. A separate risk assessment unit may not be useful, given overlap with other initiatives. As described in the report, the unit would report “directly to Management, with the purpose of developing risk scenarios for systemically important countries and analyzing tail risks for the global economy.” With the advent of an Early Warning Exercise that includes advanced economies, this recommendation has effectively been implemented. Still, enhanced outreach to disseminate risk assessments under the EWE could be considered. In addition, the IEO should have made more concrete the vague proposal for staff “to challenge their own preconceptions … and frankly disclose the limitations of data and technical tools ....”

    Recommendation #2: “Strengthen incentives to speak truth to power.”

    9. This is a valid if exceedingly difficult issue for any international agency, and the IEO findings are relevant for both the staff and the Board. At a minimum, we must be ready to speak truth to power in private when financial stability is at stake and where there is a concern about triggering an adverse market reaction. This arguably has been done over the past two years since the onset of the crisis, and will need to be carried forward consistently. The upcoming Triennial Surveillance Review will examine the promotion of effective surveillance, including how best to present views that challenge those of the authorities. We agree on the need to conduct such regular self-assessments with input from both authorities and outsiders, as has increasingly been the case in recent years (the Fund’s Mandate, medium-term strategy, conditionality). Internally, the review process has been strengthened to challenge initial staff positions with broader perspectives (see below).

    Recommendation #3: “Better integrate financial sector issues into macroeconomic assessments.”

    10. The emphasis on the importance of the recent changes to the Fund’s work on financial sector issues is welcome. In addition to the reforms of the FSAP, the Fund has taken other measures since the crisis such as additional hiring and better integration of financial sector experts, enhanced analysis of financial sector risks and surrounding policy issues in both multilateral and bilateral surveillance, the new macro-financial unit in the Research department, and significantly more resources to research and surveillance on financial markets and large and complex financial institutions.

    11. The report makes some useful recommendations on further changes to the FSAP. In particular, staff would welcome a discussion of the possibility of conducting mandatory financial stability assessments every three years, an approach that did not command sufficiently broad support when last taken up by the Board.

    12. The case for MCM sign-off on surveillance papers could have been better justified. An additional layer of sign-off responsibility runs counter to the progress achieved in recent years to streamline the review process and may actually increase the pressure to conform. Moreover, the next crisis may not be a financial one, so it could be equally argued that other departments, such as FAD, should sign off on surveillance papers. At any rate, the real issue is MCM engagement on financial sector issues in Article IV consultations for systemic cases, which has been increasing markedly and without recourse to added bureaucratic processes.

    Recommendation #4: “Overcome silo behavior and mentality.”

    13. Despite recent progress, more can be done to foster cross-departmental collaboration. The new internal review process—with shorter, more focused policy notes (instead of briefing papers)—allows Fund departments to bring diverse multilateral and crosscountry perspectives into country papers at an earlier stage. Many other initiatives also have been introduced since the crisis, including upcoming spillover reports written by cross-departmental teams, regional studies divisions in area departments, the vulnerability exercise for advanced countries and the early warning exercise, and weekly cross-departmental surveillance meetings led by the economic and financial counselors. Staff would have appreciated more specific suggestions from the IEO on furthering collaboration.

    Recommendation #5: “Deliver a clear, consistent message to the membership on global outlook and risks.”

    14. The integration of the WEO and GFSR is being strengthened. Recent efforts include joint forewords and a new statement by the Managing Director that seeks to integrate themes. The case for full merger of the two documents is not clear cut, and some fresh analysis to justify the IEO’s proposal for it would have been more useful than repeating the call for integration.

    15. The recommendation to be ready to err more often in the direction of emphasizing risks and vulnerabilities in systemic cases needs to be thought through more carefully. The recommendation flows from the IEOs finding that cognitive biases affected the IMF’s ability to predict the crisis. The question, however, is how to balance the potential to miss crises (Type I errors) against warnings about crises that never occur (Type II errors). The problem with the IEO’s approach is that it could stoke bureaucratic impulses to pro-forma recitation of risks, thus increasing false alarms and reducing the traction of Fund surveillance.

    The Chairman’s Summing Up IEO Evaluation of IMF Performance in the Run-Up to the Financial and Economic Crisis

    Executive Board Meeting 11/8

    January 26, 2011

    Executive Directors concurred with the general thrust of the IEO evaluation and recommendations. They considered that the report provided a balanced assessment of the failure of Fund surveillance to adequately anticipate and warn about the global crisis, consistent with the Fund’s own reports that acknowledged these shortcomings. Directors noted that the reform initiatives undertaken since the onset of the crisis—the early warning exercise, the vulnerability exercise for advanced economies, inputs into the G20 Mutual Assessment Process, integration of WEO-GFSR messages, mandatory financial stability assessments for systemic countries, and cross-country and spillover reports—will help enhance the candor and traction of surveillance. Nevertheless, Directors agreed that further actions should be considered.

    Key IEO Findings

    Directors broadly agreed with the IEO findings on the factors that had contributed to the failure to identify risks and give clear warnings in the run up to the global financial crisis. They stressed, in particular, the need to further enhance capacity to better “connect the dots” between financial and macroeconomic surveillance and between multilateral and bilateral surveillance. Directors also agreed that more should be done to access thoughtful and diverse opinions within the Fund and from outside experts. They noted that, in addition to bolstering analytical capacity, efforts should be made to improve the institutional culture to encourage creative thinking and alternative views. A few Directors stressed that a broader framework that will help enhance the effectiveness of Fund surveillance should be considered and developed.

    Many Directors cautioned that a lack of data and information before the crisis was a concern, as it hampered real-time assessments. A number of Directors considered, in particular, that the reluctance of financial supervisors to share relevant data on globally important institutions hindered IMF analysis. A number of Directors also stressed that some realism is needed in what to expect of Fund surveillance, given that, ultimately, the responsibility for adhering to and implementing stability-oriented policies lies with the respective authorities.

    IEO Recommendations

    Directors broadly endorsed the IEO recommendations, particularly to help strengthen the IMF’s institutional environment and analytical capacity. At the same time, they expressed a range of views on the appropriateness and suitability of some specific suggestions on how to implement the recommendations. Some Directors suggested a follow-up report by the Executive Board to the IMFC. Directors underscored that further analysis and discussion were warranted in some areas to make the recommendations actionable within competing work program priorities and budgetary constraints, and many suggested considering other responses which could complement the IEO recommendations, including by directly tackling issues of internal culture and institutional values.

    Directors generally agreed that more should be done to seek alternative or dissenting views, and a number of Directors were of the view that direct interactions between the Board and eminent outside analysts could be enhanced. Directors noted that, since the crisis, the Fund has heightened outreach efforts, including engaging stakeholders with different perspectives.

    Directors supported the recommendation to broaden the diversity of staff, including their educational background and skill mix. They welcomed ongoing efforts to hire financial sector experts and to manage their career progression once in the Fund. A number of Directors did not support the IEO’s recommendation to create a new risk assessment unit, given the overlap with other recent initiatives, but instead encouraged enhanced outreach to disseminate risk assessments under the early warning exercise. Many Directors also noted the value of having more granular summings up of Board discussions to reflect important minority views, without losing track of the objective of building consensus.

    Directors agreed that incentives needed to be strengthened to ensure the Fund “speaks truth to power,” while noting that this was an exceedingly difficult issue for any international agency. Directors looked forward to the upcoming Triennial Surveillance Review, which will examine the promotion of effective surveillance. Directors supported the proposal for the IMF to continue to conduct regular self-assessments with input from both authorities and external stakeholders.

    Directors welcomed the IEO’s positive appraisal of the recent changes to the FSAP, and felt that it would be useful to have further discussion of possible enhancements. Some Directors also emphasized the need to continue increasing MCM engagement in Article IV consultations for systemic cases.

    Directors stressed that, while more could to be done to foster cross-departmental collaboration, recent initiatives, such as the new internal review process, should be given time before changes are considered. Directors also considered it crucial that the analyses of the WEO, GFSR and the Fiscal Monitor deliver a consistent message. A number of Directors cautioned that the recommendation to err more often towards emphasizing risks and vulnerabilities could lead to more false alarms and thereby reduce the credibility and traction of surveillance.

    To conclude, today’s discussion highlights the need to continue efforts to overcome shortcomings of Fund surveillance. Management and staff will give careful consideration to the views expressed by Directors in formulating the implementation plan.

    IMF Performance in the Run-Up to the Financial and Economic Crisis:

    IMF Surveillance in 2004–07

    IMF (2009c); “The Recent Financial Turmoil—Initial Assessment, Policy Lessons, and Implications for Fund Surveillance,” April 9, 2008.

    The views expressed here are based on interviews with country authorities as well as some regional and international institutions.

    The results from the survey undertaken for IEO’s evaluation of IMF Interactions with Member Countries (IEO, 2009) indicate that only a minority of advanced and emerging market officials thought the IMF did a good job of alerting member countries about imminent external risks. While a majority of the country authorities rated the IMF’s performance highly on various aspects of interactions, two areas stood out in which only a minority thought the IMF had performed well: (i) presenting alternative scenarios and addressing “what if?” questions and (ii) bringing quickly to the authorities’ attention the implications of changing external conditions.

    Similarly, in the IEO evaluation of IMF Interactions with Member Countries (IEO, 2009) a majority of respondents to a survey of country authorities wanted a greater IMF contribution to spillover analyses, yet did not rate the IMF highly for its effectiveness in this area.

    The survey of country authorities undertaken for the IEO’s evaluation of IMF research (IEO, 2011, forthcoming) found that while a majority of country authorities thought that IMF selected issues papers were somewhat or very useful in informing the policymaking process, in those instances where they were not deemed “very useful,” the most frequently cited reasons were that the analytical framework was not suited to the realities of the country or that the research was too theoretical with little practical applicability.

    Many of the interviewees admitted that they only had time to read the documents’ Executive Summaries. While many did not read the GFSR due to its more technical nature, those involved with financial stability issues did read it.

      You are not logged in and do not have access to this content. Please login or, to subscribe to IMF eLibrary, please click here

      Other Resources Citing This Publication