Since May 2016, the IEO has completed two evaluations—The IMF and the Crises in Greece, Ireland, and Portugal and The IMF and Social Protection—and one evaluation update—Multilateral Surveillance: Revisiting the 2006 IEO Evaluation. The IEO has also launched three new evaluations—which will analyze the Fund’s role on fragile states, its financial surveillance activities, and its advice on unconventional monetary policies—and two evaluation updates—which will look into the Fund’s exchange rate policy advice and structural conditionality. In addition, outside its regular work program, the IEO has published two books: The International Monetary Fund and the Learning Organization: The Role of Independent Evaluation (Schwartz and Rist, 2016) and Background Papers for the IMF and the Crises in Greece, Ireland, and Portugal (Schwartz and Takagi, 2017).
The IMF and the Crises in Greece, Ireland, and Portugal
The IEO released its report on The IMF and the Crises in Greece, Ireland, and Portugal on July 28, 2016. The crises that hit several euro area countries between 2010 and 2013, coming so soon after the global financial and economic crisis and occurring in a common currency area comprising advanced and highly integrated economies, posed extraordinary challenges to European and world policymakers. The evaluation examined the effectiveness of the IMF’s surveillance and crisis management in the euro area1 to draw lessons for the future and to enhance transparency. The evaluation covered the 2010 Stand-By Arrangement with Greece, the 2010 Extended Arrangement with Ireland, and the 2011 Extended Arrangement with Portugal.
The evaluation found that, for the most part, the IMF’s euro area surveillance identified the right issues during the pre-crisis period but did not foresee the magnitude of the risks that would later become paramount. The IMF’s surveillance of the financial regulatory architecture was generally of high quality, but staff, along with most other experts, missed the buildup of banking system risks in some countries. In general, the IMF shared the widely-held “Europe is different” mindset that encouraged the view that large imbalances in national current accounts were little cause for concern and that sudden stops could not happen within the euro area. Following the onset of the crisis, however, IMF surveillance successfully identified many unaddressed vulnerabilities, pushed for aggressive bank stress testing and recapitalization, and called for the formation of a banking union.
The report found several issues with the way decision making was managed by the IMF. In May 2010, the IMF Executive Board approved a decision to provide exceptional access financing to Greece without seeking preemptive debt restructuring, even though its sovereign debt was not deemed sustainable with a high probability. The risk of contagion was an important consideration in coming to this decision. The IMF’s policy on exceptional access to Fund resources, which mandates early Board involvement, was followed only in a perfunctory manner. The 2002 framework for exceptional access was modified to allow exceptional access financing to go forward, but the modification process departed from the IMF’s usual deliberative process whereby decisions of such import receive careful review. Early and active Board involvement might or might not have led to a different decision, but it would have enhanced the legitimacy of any decision.
“The IMF’s handling of the euro area crisis raised issues of accountability and transparency.”
On the collaboration with the European partners, the report’s assessment was mixed. The IMF, having considered the possibility of lending to a euro area member as unlikely, had never articulated how best it could design a program with a euro area country, including conditionality on policies under the control of regional institutions. In the circumstances of these programs, where there was more than one conditional lender, the troika arrangement (in which the Fund worked with the European Commission and the European Central Bank) proved to be an efficient mechanism in most instances for conducting program discussions with national authorities, but the IMF lost its characteristic agility as a crisis manager. And because the European Commission negotiated on behalf of the Eurogroup, the troika arrangement potentially subjected IMF staff’s technical judgments to political pressure from an early stage.
Program design and implementation were not ideal. The IMF-supported programs in Greece and Portugal incorporated overly optimistic growth projections. More realistic projections would have made clear the likely impact of fiscal consolidation on growth and debt dynamics, and allowed the authorities to prepare accordingly or persuaded European partners to consider additional—and more concessional—financing while preserving the IMF’s credibility as an independent, technocratic institution. Lessons from past crises were not always applied, for example when the IMF underestimated the likely negative response of private creditors to a high-risk program. The IMF’s performance was uneven although there were instances where IMF staff shone technically and made an important overall contribution.
Finally, the IMF’s handling of the euro area crisis raised issues of accountability and transparency, which helped create the perception that the IMF treated Europe differently. Moreover, conducting the evaluation proved challenging. Some documents on sensitive issues were prepared outside the regular, established channels; the IEO faced a lack of clarity in its terms of reference on what it could or could not evaluate; and there was no clear protocol on the modality of interactions between the IEO and IMF staff. The IMF did not complete internal reviews involving euro area programs on time, as mandated, which led to missed opportunities to draw timely lessons.
Main Recommendations
The Executive Board and management should develop procedures to minimize the room for political intervention in the IMF’s technical analysis;
The Executive Board and management should strengthen the existing processes to ensure that agreed policies are followed and that they are not changed without careful deliberation;
The IMF should clarify how guidelines on program design apply to currency union members;
The IMF should establish a policy on cooperation with regional financing arrangements;
The Executive Board and management should reaffirm their commitment to accountability and transparency and the role of independent evaluation in fostering good governance.
The Executive Board discussed the report on July 19, 2016. Executive Directors welcomed the report and appreciated the accompanying statement by the Managing Director. They agreed that the evaluation’s findings provided valuable insights and lessons for handling crises in members of currency unions. Directors underscored that the work of the IEO continues to play a vital role in enhancing the learning culture within the Fund, strengthening the Fund’s external credibility, and supporting the Executive Board’s oversight responsibilities.
Directors broadly shared the general thrust of the IEO’s main findings and broadly endorsed its recommendations, with some caveats. Directors recognized that, while the Fund needs to learn from the experience of the three euro area crisis programs, it was important to acknowledge the difficult and unprecedented circumstances prevailing at the time. Directors also noted that the uncertainty and fear of contagion were acute given the backdrop of the global financial crisis. They emphasized that the Fund’s performance in these crisis cases must be assessed in this broader context as it navigated uncharted territory. Against this background, Directors considered that the Fund-supported programs had succeeded in buying time to build European firewalls, preventing the crisis from spreading, and restoring growth and market access in Ireland and Portugal. They observed that the political economy of the Greek crisis was unique and complex. Directors generally viewed the unprecedented troika arrangement as efficient overall, while the need to coordinate and reach common ground with the European partners might have affected the Fund’s agility as a crisis manager, and gave rise to criticism that its decision-making process lacked transparency.
The full IEO report, along with a statement by the Managing Director and the Acting Chair’s Summing Up, is available on the IEO website at
The IMF and Social Protection
The IEO released its evaluation of The IMF and Social Protection on July 24, 2017, following the Executive Board’s discussion of the report on July 19. The IMF has stepped up its attention to social protection over the past decade as it has dealt with the aftermath of the global financial crisis and addressed concerns from the impact of food and fuel price shocks and broader stresses on low-income groups and the most vulnerable. The evaluation examined the IMF’s involvement in social protection across the membership, in the context of surveillance, programs, and technical assistance, as well as its collaboration with other institutions in this area.
The report found widespread IMF involvement in social protection across countries although the extent of engagement varied—with high-quality work in some cases, but more limited treatment in others. This cross-country variation to some degree reflected an appropriate response to country-specific factors, in particular an assessment of whether social protection policy was “macro-critical,” and the availability of expertise from development partners or in the country itself. However, idiosyncratic factors also seem to have played a part, as staff had different views on what kind of work they were expected to do in this area, as well as varying degrees of interest in these issues. At times it seemed that attention to social protection in surveillance devolved into a box-ticking exercise as staff tried to pay due attention to an increasingly broad range of policy issues.
In the program context, the report noted that the IMF invariably emphasized the need to mitigate potential adverse effects of program measures on the most vulnerable and generally worked well with development partners to address social protection concerns. However, authorities sometimes found the IMF to be insufficiently attuned to local conditions, and the IMF’s efforts to incorporate social protection concerns in program design and conditionality in some cases met with implementation challenges due to local capacity constraints and differences in country commitment.
“…widespread IMF involvement in social protection…with high-quality work in some cases, but more limited treatment in others.”
The report found that IMF-World Bank cooperation on social protection generally worked well, allowing the Fund to draw effectively on Bank expertise in this area. However, while the IMF’s preferred approach of targeting social protection to the poor and vulnerable was aligned with the World Bank’s approach, it meshed less well with the rights-based approach to social protection espoused by the International Labour Organization and UN agencies which emphasizes universal benefits and targeting by category (e.g., demographic group) rather than income. This difference in viewpoints posed a challenge to IMF collaboration with such agencies and the report pointed out that it could complicate Bank-Fund collaboration going forward as the World Bank moves to adopt the goal of universal social protection.
The difference in viewpoints also affected how civil society organizations perceived the IMF’s commitment to social protection and the UN Sustainable Development Goals. The report observed that efforts by the IMF’s external communications to emphasize the Fund’s “human face” did not always convince stakeholders, especially civil society, in part because of heightened expectations.
Main Recommendations
The IMF should establish a clear strategic framework to guide its involvement in social protection among multiple competing priorities;
The IMF should provide tailored policy advice based on in-depth analysis (ideally drawing on work by development partners or country authorities) when social protection is determined to be a strategic priority;
The IMF should find more realistic and effective approaches to program design and conditionality to ensure that adverse impacts of program measures on the most vulnerable are mitigated;
The IMF should realistically explain its approach to social protection issues in external communications;
The IMF should engage actively in inter-institutional cooperation on social protection
Executive Directors welcomed the report. They agreed with the need to refine the Fund’s approach to social protection and joined the Managing Director in supporting all the evaluation’s recommendations. At the same time, Directors underscored the need to be mindful of the Fund’s mandate to engage only in macro-critical areas while bearing in mind its resource constraints and comparative expertise. IMF management will work with staff to implement the recommendations based on a Management Implementation Plan anticipated to be presented to the Board within six months.
The full IEO report, along with a statement by the Managing Director and the Acting Chair’s Summing Up, is available on the IEO website at
Multilateral Surveillance: Revisiting the 2006 IEO Evaluation
On March 8, 2017, the IEO published the report Multilateral Surveillance: Revisiting the 2006 IEO Evaluation.2 The 2006 IEO evaluation of IMF Multilateral Surveillance commended many aspects of the IMF’s work, but was concerned about the absence of an overall strategy that rendered the whole less than the sum of the parts. International policy linkages and spillovers were not sufficiently explored, dedicated analysis of exchange rate issues was too limited, forecasts and policy advice were excessively country-driven, and there was insufficient integration, both between multilateral and bilateral surveillance, and between macroeconomic and financial sector analyses. The potential to influence countries’ policies was not fully exploited, and using the same publications to reach multiple audiences had undermined communication and limited readership.
The revisit found that IMF multilateral surveillance has undergone significant reforms over the past decade, as the global financial crisis served as a catalyst for many reforms proposed by the 2006 IEO evaluation. The legal framework for surveillance was strengthened by the 2012 Integrated Surveillance Decision, which provides a more comprehensive basis for conducting IMF surveillance. New products and activities closed gaps in pre-crisis analysis in areas such as vulnerabilities and spillovers in advanced economies, while the analysis of exchange rates and their consistency with external positions and economic fundamentals was refined. These new reports and activities were appreciated by authorities in member countries and other stakeholders. Greater IMF involvement in global policy deliberations was aided by the IMF’s effective support to the G20, while the Early Warning Exercise enabled a more structured discussion of risks.
At the same time, the report noted that the expansion of overlapping multilateral surveillance products limited policymakers’ ability to absorb the information and analysis, and complicated the IMF’s efforts to ensure consistency across the various products. Instances of different and sometimes contradictory messages across the expanded menu of reports pointed to the need for stronger interdepartmental cooperation. In this context, recent efforts at consolidation and streamlining offered promise.
The International Monetary Fund and the Learning Organization: the Role of Independent Evaluation
This book coauthored by Moisés Schwartz and Ray Rist (2016) reflects on the IEO’s mission and institutional relationship with the IMF. Most of all, it focuses on the contributions that the IEO has made, and should be making, to improve the Fund’s effectiveness. It concludes that, while the Fund has benefited extensively from the IEO’s activities in terms of accountability and credibility, it has not yet exploited the full potential of independent evaluation. In order to achieve that, the Fund should become a learning organization, which would allow it to be more receptive to critical and sometimes unwelcome advice, and recognize the IEO as a true partner. Thus, the Fund needs to perceive independent evaluation as a learning mechanism. Tough some progress has been made, the Fund still needs to go some way before a strong evaluation culture is widespread and established within the institution. This is a joint challenge for the Fund and the IEO, but the authors conclude by calling on Fund’s management to take a more active role to enable independent evaluation to bring the IMF closer to the ideal of a learning institution.
Outreach and Communication
Outreach is critical to achieving the IEO’s objectives. The IEO has increased outreach to authorities and civil society in member countries, as well as the Board, management and IMF staff, in line with the recommendations of the external evaluations of the IEO. These activities are critical to enhancing transparency and a better understanding of the IMF, to fostering institutional learning, and to supporting Board oversight—IEO’s key mandates. Outreach is also important for publicizing and encouraging discussion of the IEO’s work and thereby increasing its impact. It is also very useful for receiving feedback and gathering information on what issues are more interesting for a broad range of stakeholders. To this end, the IEO organized or participated in numerous events during the period under review. These are listed on page 15.
The IEO also actively uses its website, along with email communication with subscribers, to disseminate its work and to solicit public comments on ongoing, future, and completed evaluations. The website (www.ieo-imf.org) serves as a repository of all IEO work.
Budget and Staffing
In FY2017, the IEO expended 93.5 percent of its approved budget amount (see “Administrative Budget” below for details about the IEO’s budget and expenditures). The decrease in spending was largely attributable to delays in filling vacancies and advancing new projects, in anticipation of the arrival of the new IEO Director. Following a thorough review of its budget implementation and projected temporary reductions in expenditures, at the beginning of the 2017 calendar year the IEO returned $200,000 of its funds to IMF’s central budget.
In March 2017, the Executive Board approved the IEO FY2018 budget proposal of $6.2 million, representing a zero real growth over FY2017 budget. This budget, along with a carryover of unspent funds from FY2017 of up to 5 percent of the authorized FY2017 budget, will allow the IEO to meet the demands of its FY2018 work program. The foreseen FY2018 work program includes the completion of one ongoing evaluation, continuing work on two new ones, and the completion of two evaluation updates.
On July 31, 2017, there were fifteen staff positions (including the Director) at the IEO. Several staff members moved on and new ones were recruited during the period under consideration. The IEO maintains a diverse group of professionals, of whom more than half come from outside the Fund. In addition, the IEO continues to rely extensively on external consultants to bring expertise and fresh perspectives to its evaluation work.
Previously, the IEO had presented IMF Response to the Financial and Economic Crisis (IEO, 2014a) and IMF Performance in the Run-Up to the Financial and Economic Crisis: IMF Surveillance in 2004–07 (IEO, 2011). See also Background Papers for the IMF and the Crises in Greece, Ireland, and Portugal (Schwartz and Takagi, 2017), which collects the extensive background material prepared for this evaluation.
This report is part of an IEO series that revisits past evaluations to assess their relevance after about a decade.
