Chapter 6. Key Findings and Recommendations

International Monetary Fund. Independent Evaluation Office;International Monetary Fund. External Relations Dept.
Published Date:
September 2016
  • ShareShare
Show Summary Details

116. This chapter presents a summary of key findings and draws lessons from the evaluation, and proposes five recommendations to help improve the IMF’s effectiveness.

A. Key Findings and Lessons


117. While the IMF’s pre-crisis surveillance for the most part identified the right issues, it did not foresee the magnitude of the risks that would become paramount in the crisis to follow. Lack of analytical depth, rigor, or specificity and the failure to highlight sufficiently the need for stronger remedial action in a currency union were among the factors that undermined the quality and effectiveness of surveillance. At the euro area level, IMF staff’s position was often too close to the official line of European officials, and the IMF lost effectiveness as an independent assessor. The IMF’s surveillance of the euro-area financial regulatory architecture was generally of high quality, but, along with most other observers, IMF staff missed the overall build-up of banking system risks in some countries. Following the onset of the crisis, however, IMF surveillance successfully identified many unaddressed vulnerabilities, pushed for aggressive bank stress testing and recapitalization, and articulated a vision of banking union.

118. The weaknesses of IMF surveillance in the euro area echoed the larger problem of IMF surveillance in advanced economies identified by the IEO’s 2011 evaluation of IMF Performance in the Run-Up to the Financial and Economic Crisis. That evaluation identified several factors at play, including “a high degree of groupthink, intellectual capture, a general mindset that a major financial crisis in large advanced economies was unlikely, and incomplete analytical approaches” (IEO, 2011). These factors were compounded in the case of the euro area by a “Europe is different” mindset that encouraged the view that surveillance was largely the responsibility of euro area institutions and authorities, that large national current account imbalances were little cause for concern, and sudden stops could not happen within a currency union that issues a reserve currency.

Decision making

119. In May 2010, the IMF Executive Board approved a decision to provide exceptional access financing to Greece without seeking preemptive debt restructuring, even though its sovereign debt was not deemed sustainable with a high probability. In coming to this decision, there was no open and early discussion of the pros and cons of all options available to the IMF. While the risk of contagion was an important consideration, contagion outcomes under different scenarios, especially the adverse consequences of not restructuring debt, were neither rigorously quantified nor thoroughly discussed within the institution. Irrespective of the merit of the final decision, weaknesses in the decision-making process created the perception that the IMF treated Europe differently. The procedure used for Greece was essentially repeated for Ireland and Portugal. In these decisions, the role of the Board was at best perfunctory. While this lack of Board involvement is not a new finding, it was more problematic in these cases.

120. The revision of the 2002 exceptional access framework, also made in May 2010, did not receive the customary careful review and deliberation by the Board. A modification of the framework was necessary to allow exceptional access financing to go forward, but the modification process was not transparent. Staff did not forewarn the Board, and the proposed amendment was embedded in the SBA request document. There was no follow-up on the revision until 2014 (IMF, 2014b), even though at least one Executive Director during the May 2010 meeting requested an early return to a thorough discussion of the implications of the systemic exemption for the IMF’s work. This was a departure from the IMF’s usual process of deliberation, where decisions of such import receive careful review so that intended and unintended consequences as well as implications for the future work of the IMF are clearly understood.

Working with European partners

121. The troika arrangement in most instances proved to be an efficient mechanism for conducting program discussions with national authorities at the staff level in a situation where there was more than one conditional lender. Even so, given the multiple layers of decision making in the euro area, the IMF lost its characteristic nimbleness and agility as a crisis manager. Because the European Commission represented the Eurogroup (and thus euro area governments), the troika arrangement potentially subjected the IMF staff’s technical judgements to political pressure from an earlier stage than is normally the case. The IMF had no established principles for joint lending operations. Nor was there a clear demarcation of responsibilities, an agreed policy on the sharing of confidential information, a mechanism to address differences of view, or a unified analytical or conditionality framework.

122. Lack of preparation was part of the problem. The IMF, having considered the possibility of the use of Fund resources by a euro area member as “extremely unlikely” (IMF, 1998), had never articulated how currency union considerations should be incorporated in program design. There had been no discussion of how the IMF could address macro-critical policies that were under the control of regional institutions like the ECB or whether it could set conditionality on such an institution. Had IMF management and staff discussed the implications of euro area membership for program design with the Executive Board at an earlier stage—in early 2009 when staff first informally explored the idea of a precautionary arrangement with the Irish authorities, or even in the six months prior to Greece’s request—staff would have had a better understanding of specific constraints that they would, or should not, have to accept in an IMF-supported program for a euro area member.

Program design and implementation

123. The IMF-supported programs in Greece and Portugal incorporated overly optimistic growth projections. While these have been a feature of many IMF-supported programs (IEO, 2014a), more realistic projections would have made clear the impact of fiscal consolidation on growth and debt dynamics. They would have also allowed the authorities to prepare accordingly, or persuaded the European partners to consider additional and more concessional financing, while preserving the IMF’s credibility as an independent, technocratic institution.

124. Lessons from past crises were not always applied. For example, the lesson from earlier capital account crises—that it is difficult to reassure private creditors with a high-risk program—was not applied in the SBA for Greece, as the IMF staff came to acknowledge more than five years later (IMF, 2016). Another lesson from the capital account crisis programs of the late 1990s was the need for a contingency plan. Although some in the staff internally raised the issue, the SBA for Greece, high-risk as it was, included “no Plan B” (IMF, 2010e). Structural conditionality is an area where the IMF staff did not prevail over the European partners in applying lessons learned from the Asian crisis: namely, that imposing a long list of structural conditions without prioritization in defiance of the crisis countries’ implementation capacity would be counterproductive.

125. There were instances where IMF staff shone technically, and many officials have expressed a positive assessment of the Fund’s overall contribution to crisis management. Even so, the IMF’s performance was uneven. For example, program design and implementation (including the pace of fiscal adjustment and flexibility for the operation of automatic stabilizers) was close to exemplary in Ireland but severely wanting in Greece. Financial sector work was first-rate in Spain but inadequate in Portugal. The patchy availability of staff with germane experience and expertise may have been among the reasons for this uneven performance.

Accountability and transparency

126. The IMF’s handling of the euro area crisis raised issues of accountability and transparency, which helped to create the perception that the IMF treated Europe differently. The fact that a good fraction of the Executive Board—and more broadly of the IMF’s membership—was not fully kept informed during the crisis undermined the Board’s oversight function and only served to reinforce this perception.

127. Delays in completing internal reviews involving euro area programs did not help dispel the perceived lack of transparency. Preparation of the Board paper reviewing IMF-supported programs during the global financial crisis (including the euro area programs) was delayed for well over a year despite repeated requests by the IMFC. Preparation of the ex post evaluation of Portugal’s EFF-supported program, which should have been completed by June 2015, was still ongoing as of May 2016.

128. As noted at the beginning of this report, the IEO, in conducting this evaluation, faced a lack of clarity in its terms of reference regarding what it could or could not evaluate. Lack of documentation was a serious problem (as some sensitive documents were prepared outside the regular, established channels), while it took the IEO more than a year to obtain some available documents. The evaluation was also hampered by the lack of a clear protocol on the modality of interactions between the IEO and IMF staff.73

B. Recommendations

129. The IMF has drawn many lessons from the recent experience (Box 8). New initiatives related to surveillance and program design are underway, including through the established periodic surveillance and conditionality review processes.

130. Table 3 lists five recommendations for Board consideration, which are expounded with examples in this section. The recommendations are focused on broad governance issues, which the IEO believes were at the root of the problems identified by this evaluation. Many of the problems were caused by procedural inadequacies, which prevented the IMF’s crisis management experience and best technical judgments from being fully applied in the face of political pressure or expediency. Internal documents clearly indicate that many on the staff and the Executive Board were aware, at the time, of the issues raised in this evaluation.

Table 3.IEO Recommendations for Board Consideration
Recommendation 1The Executive Board and management should develop procedures to minimize the room for political intervention in the IMF’s technical analysis.
Recommendation 2The Executive Board and management should strengthen the existing processes to ensure that agreed policies are followed and that they are not changed without careful deliberation.
Recommendation 3The IMF should clarify how guidelines on program design apply to currency union members.
Recommendation 4The IMF should establish a policy on cooperation with regional financing arrangements.
Recommendation 5The Executive Board and management should reaffirm their commitment to accountability and transparency and the role of independent evaluation in fostering good governance.

Recommendation I:The Executive Board and management should develop procedures to minimize the room for political intervention in the IMF’s technical analysis.

131. A similar recommendation was found in the very first evaluation completed by the IEO in 2002, on the prolonged use of IMF resources (IEO, 2002), indicating that the problems identified by the present evaluation are not new. While a range of views exist on the role political judgments should play in the IMF’s decision making, there is a broad consensus that, if political judgment is to be exercised at all, it should take place at the level of the Executive Board in a transparent manner. The credibility of the IMF comes from the technical competence and independence of its staff, and the Managing Director must ensure that its technical work is protected from political influence.

132. Such procedures could involve several elements. For example, the staff could be incentivized to produce rigorous analyses based on realistic program assumptions, and to be transparent in explaining how it came to a particular conclusion. When analytical concepts (e.g., the risk of adverse systemic effects) are placed at the center of decisions, there must be a presumption that they are supported with a clear analytical framework for assessment. When high risk programs are presented for approval, the Executive Board might be provided with alternative options and tradeoffs or an explanation of why a proposed decision was preferred to the other alternatives. Likewise, when the IMF is collaborating with another conditional lender, the Board might be informed as to whether there are areas of disagreement and, if so, how the differences are being (or proposed to be) resolved.

Recommendation 2: The Executive Board and management should strengthen the existing processes to ensure that agreed policies are followed and that they are not changed without careful deliberation.

133. The Board should strengthen the existing processes to ensure that agreed policies are adhered to, that those policies are not changed without full and formal deliberation, and that any necessary corrective action is taken in a timely manner. Management, on its part, should consult the Board early when changes in policy are warranted and should not wait until a formal review is scheduled to discuss needed changes. The reformed exceptional access framework adopted in January 2016 leaves room for discretion in circumstances where debt is assessed to be sustainable but not with a high probability, allowing for a range of options that could meet the prescribed requirements. This puts a greater onus on the Board to ensure that all future requests for exceptional access, particularly where debt is not assessed to be sustainable with a high probability, are properly justified and that financing commitments from other sources can be credibly substantiated.

Box 8.Selected Lessons Drawn by IMF Staff on Recent Crisis Management in the Euro Area Crisis1

Sovereign Debt Restructuring (IMF, 2013b)

  • Debt restructuring has often been too little too late and failed to reestablish debt sustainability in a durable way.

  • While the costs of delaying restructuring are well recognized, authorities’ concerns about financial stability and contagion could exert pressures to delay; delays were sometimes caused by the incentives of official creditors who have interest in accepting, and pressuring the Fund to accept, sanguine debt sustainability assessments.

  • Providing large-scale financing without debt relief would only postpone the need to address the debt problem; the appropriate response would be to deal with the contagion effects of restructuring head-on, for example by establishing adequate safeguards.

  • Action is likely required to increase the rigor and transparency of debt sustainability assessments, explore ways to prevent the use of IMF resources to simply bail out private creditors, and alleviate the costs associated with restructuring.

Ex Post Evaluation on Greece (IMF, 2013c)

  • Better tailoring of Fund lending policies to the circumstances of monetary unions, given the large structural component of programs when exchange rates are fixed.

  • Avoiding undue delays in debt restructuring.

  • More attention to the political economy of adjustment.

  • More parsimony in fiscal structural reforms, and a more hands-on approach to technical assistance.

  • More streamlining in the troika process to reduce the burden on the authorities.

Ex Post Evaluation on Ireland (IMF, 2014b)

  • Country ownership is key.

  • Set realistic targets and meet them. While always an important objective, meeting program targets is particularly relevant for reassuring private capital markets.

  • In a banking crisis, take strong actions upfront—credible asset quality and liquidity assessments and a well-capitalized banking system are critical.

  • Focus conditionality on key challenges.

  • Communicate effectively. While technical expertise is vital for the right diagnosis and in identifying the appropriate policy response, communicating the strategy is also of critical importance.

  • Be proactive and closely engaged.

Crisis Program Review (IMF, 2015c)

  • The systemic exemption provided the euro area with time to build firewalls, but it could not on its own prevent contagion.

  • Internal devaluation, which relies on domestic price adjustment, proved hard to achieve within a short period, and the desired recovery in growth and exports did not materialize for most countries.

  • Concerns about bank–sovereign linkages and cross-border contagion sometimes delayed or limited public debt restructuring, adversely affecting growth and credit intermediation; where public debt is high, timely debt restructuring may also be needed.

  • The growth payoffs from structural reforms in the near term appear to have been modest and less than envisaged; this should be reflected in realistic and prudent program assumptions.

  • Clearer operational guidance for the IMF’s interaction with regional financing arrangements would be helpful for delineating responsibilities; when necessary, commitments on prospective implementation of necessary union-wide policies should be sought; alternatively, program design would need to be based on larger adjustment and financing, or IMF involvement be postponed.

1 While some of these papers draw a number of lessons from a broader set of crisis experience, only those relevant to the euro area are noted here.

134. In this connection, the Executive Board should draw lessons from the implementation of the exceptional access policy during the euro area crisis, especially the extent and timeliness of the information provided and the policy issues presented during informal sessions. The Board should consider why and how it came to be that there were gaps in information provided, and whether any information asymmetry among Executive Directors would be an issue of concern should similar lapses occur in the future.

Recommendation 3: The IMF should clarify how guidelines on program design apply to currency union members.

135. The IMF has long recognized that program design and conditionality for countries that are members of currency unions need to differ from that for countries with a flexible exchange rate and an independent monetary policy (see IMF, 1994a). Policy responsibilities in a currency union are split between national and union-level authorities. The implications of this split for the conduct of Article IV consultations are explicitly considered in the various IMF surveillance decisions and corresponding guidance notes to staff. But the 2002 Conditionality Guidelines (IMF, 2002b) and the Revised Operational Guidance Note to IMF Staff (IMF, 2014a) do not explain how IMF-supported programs will approach the split of policy responsibilities in a currency union from the standpoints of program design and conditionality.

136. The IMF should conduct a comprehensive review and formal discussion of its lending approaches to members of currency unions. Issues that need to be clarified include the following: (i) To whom does/should the IMF owe its primary responsibility in lending to a currency union member—the borrowing country alone, the union as a whole, or the global financial system? (ii) How should the IMF balance the objectives of lending in the member’s best interests and avoiding measures that harm systemic stability? (iii) Is there a scope for lending directly to a currency union without amending the Articles of Agreement? (iv) What would be the circumstances and modalities for setting conditionality on union-level institutions? (v) What is the appropriate role of the regional central bank or other union-level institutions during program discussions with a member country? (vi) What options are available to the IMF to effect changes in union-level policies that may be necessary for the success of the member’s program?

137. Introducing an explicit treatment of issues germane to countries in a currency union would bring existing conditionality guidelines into conformity with surveillance policy and practices and would also promote more evenhanded treatment of members in different currency unions.

Recommendation 4: The IMF should establish a policy on cooperation with regional financing arrangements.

138. Such a policy could be expected to protect the IMF’s technical judgements from political influence. With respect to its engagement in the euro area, the 2012 Treaty Establishing the European Stability Mechanism (ESM) provides for the ESM to “cooperate very closely” with the IMF in providing support to a euro area member.74 Written principles on joint lending operations agreed by the IMF Managing Director with the Head of the ESM, and endorsed by the respective Boards, would provide clarity for all parties and enhance the legitimacy of IMF–ESM cooperation.

139. Areas where clarity could be provided include: (i) sharing of confidential information; (ii) procedures to address differences of view at the mission level and up; (iii) avoidance of cross-conditionality and inconsistent conditionality, especially in overlapping policy areas such as fiscal policy, financial sector restructuring, and structural reforms, and, in the event one institution were to decide to proceed without the others, mutual understanding of the conditions for such action including the scope for informal communication prior to taking the formal decision; (iv) efforts to reduce the burdens placed on country authorities by large mission teams and duplication of information requests; (v) implications for the actions of the other institution of overdue obligations, or arrears, to one institution by a borrowing country;75 and (vi) conditions for requests to the IMF to provide technical assistance, such as took place in the case of Spain, and the modalities to be used by the IMF.

140. Similar agreed cooperation principles adapted to the circumstances of each regional financing arrangement (RFA) would also prove useful for the IMF’s possible program involvement with such arrangements. The IMFC called for the development of such principles in the spring of 2011 (IMFC, 2011) and the G20 endorsed six nonbinding broad principles for cooperation in the fall of that year, but no related formal discussion has taken place at the Executive Board.76 In October 2014, Executive Directors, in discussing the IEO evaluation of the IMF Response to the Financial and Economic Crisis (IEO, 2014c), “generally supported the [IEO] recommendation to develop guidelines for better structuring engagements with other organizations and clarifying the IMF’s roles and accountabilities, to further safeguard the IMF’s independence and help ensure uniform treatment to all member countries,” while noting the need to remain “flexible and pragmatic to allow adaptation to specific circumstances” (IMF, 2014c). As noted by the staff (IMF, 2015c), this is an opportune time for the Executive Board to discuss formally the G20 principles for cooperation between the IMF and RFAs and to develop principles tailored to each RFA. Any agreed collaboration principles would usefully be supplemented by operational guidelines for IMF staff to help promote their consistent application.

Recommendation 5: The Executive Board and management should reaffirm their commitment to accountability and transparency and the role of independent evaluation in fostering good governance.

141. Management, staff, and the Board should avoid actions that could be seen as hindering evaluation efforts; this could lead to missing valuable learning opportunities as well as potentially damaging the IMF’s credibility. Ex post evaluations under the Fund’s exceptional access policy should continue to be prepared in accordance with the guidelines and on time. In addition, the Board should establish or reaffirm clear guidelines on: (i) how to keep records of the process by which important program-related decisions are made at the staff and management levels; (ii) the preparation and retention of the records of informal Board meetings; (iii) the IEO’s access to confidential documents when there is ongoing sensitivity and with what time lags such documents should be made available; (iv) the modality of interactions between the IEO and IMF staff; and (v) how the IEO could assist the IMF to draw timely lessons by providing greater clarity on its terms of reference regarding what it can or cannot evaluate. In this connection, the IEO is already working with staff to address some of the problems it has faced in conducting this evaluation. The IEO welcomes the staff’s initiatives to develop a clear protocol for its engagement with IMF staff in future evaluations.

    Other Resources Citing This Publication