Summaries of IEO Evaluations, 2002–11
- Ruben Lamdany, and Hali Edison
- Published Date:
- December 2012
1. Evaluation of Prolonged Use of IMF Resources (2002)
This evaluation assesses the causes and consequences of countries having repeated access to IMF financing. For the report, any country engaged in an IMF-supported program for at least seven of the previous ten years is classified as a prolonged user in that year. Under this definition, 51 of the 128 countries that borrowed in the period 1971–2000 were prolonged users at some point. Prolonged use has increased over time and is persistent; few countries “graduate” from such use. While most of the increase involved low-income countries eligible for IMF concessional financing, the bulk of financial commitments to prolonged users came from nonconcessional resources.
The evaluation identified five factors contributing to prolonged use: (i) a broadening of the rationale for IMF program involvement in recognition that many balance of payments problems, especially in low-income countries, arose from deep-seated structural problems that needed more time for adjustment; (ii) a demand for IMF lending as a “seal of approval” by many donors and creditors; (iii) the extension of programs because of judgments that IMF surveillance is insufficient for signaling the soundness of the macroeconomic framework or promoting desired policy changes; (iv) weaknesses in program design and implementation, including over-optimistic growth projections, too broad conditionality on structural policies, and insufficient attention to the domestic ownership of core policy commitments; and (v) lack of effective follow-up within the IMF on already approved elements of a strategy to reduce prolonged use. One reason for this lack of follow-up was the absence of an explicit definition of prolonged use, which made it difficult to enforce due diligence requirements for identified cases.
The evaluation recognizes that prolonged use can be justified in cases where the simultaneous challenges of macroeconomic adjustment, structural reform, and institutional development take considerable time to resolve. However, prolonged use can also hinder the development of robust domestic policy formulation processes, undermine country “ownership,” and lead to pressure to agree to a series of weak programs to unlock other sources of financing. IMF surveillance can also be “crowded out” by short-term program issues.
The evaluation makes a number of recommendations covering institutional arrangements, program design, surveillance, and internal IMF governance that would help reduce prolonged use. First, the IMF should adopt a definition of prolonged use as a trigger for enhanced due diligence actions, including systematic ex post assessments of previous programs and forward-looking consideration of “exit” strategies. It should also provide credible alternatives to IMF lending arrangements as a signaling device to donors and creditors. Greater selectivity in program content, aligned with steps to maximize ownership, are also needed, along with better tailoring of the program timeframe to the foreseeable length of a country’s adjustment needs. A greater operational separation between surveillance and program activities is also needed in prolonged use cases.
2. The IMF and Recent Capital Account Crises: Indonesia, Korea, Brazil (2003)
The evaluation examines the crisis management role of the IMF in Indonesia (1997–98), Korea (1997–98), and Brazil (1998–99). Among the major findings are:
Surveillance was more successful in identifying macroeconomic vulnerabilities than in recognizing the risks in financial sector and corporate balance-sheet weaknesses. Even when the diagnosis was broadly accurate, insufficient candor limited the impact.
Macroeconomic outcomes turned out to be very different from program projections, because insufficient attention was paid to (i) the possibility of a large depreciation and (ii) the balance-sheet effect of currency depreciation.
In view of output developments and the initial stock of debt, fiscal tightening was not warranted in Indonesia and Korea. In Brazil, it was appropriate because fiscal sustainability was a major issue.
Monetary policy was initially set tight, in recognition of the trade-off between higher interest rates and a weaker exchange rate. However, the experience is mixed and does not provide definitive evidence on the stabilizing effect of high interest rates.
The official package for Korea was inadequate because of ambiguity over the “second line of defense,” which contributed to the program’s failure. When major countries decided to involve the private sector, the IMF played a useful facilitating role.
The Indonesian and Korean programs were heavily loaded with conditionality in structural reform. Measures to rehabilitate the financial sector were necessary, but many other measures were not essential. The experience suggests that a successful bank closure and restructuring program must include a comprehensive and well-communicated strategy in which transparent rules are consistently applied.
A program for restoring confidence must include a strategy to communicate the logic of the program. None of the three programs initially contained such a strategy.
While the close involvement of the Executive Board and the major shareholders was proper and necessary, frequent contacts at multiple layers unnecessarily subjected staff to micromanagement and political pressure.
Article IV consultations should take a “stress-testing” approach to the analysis of a country’s exposure to a potential capital account crisis.
Additional steps should be taken to increase the impact of surveillance, including through making staff assessments more candid and more accessible to the public, and providing appropriate institutional incentives to staff.
Since restoration of confidence is the central goal, the IMF should ensure that the financing package, including all components, should be sufficient to generate confidence and also of credible quality.
3. Fiscal Adjustment in IMF-Supported Programs (2003)
Fiscal adjustment plays a key role in IMF-supported programs, and it has also been the subject of some controversy in two dimensions. First, the quantitative dimension: does the IMF’s approach to fiscal adjustment reflect a “one-size-fits-all” approach leading to an unnecessary economic austerity? Second, the qualitative dimension: could the efficiency, sustainability, and equity of that adjustment be improved by an alternative mix and sequence of revenues and expenditure policies? The evaluation is based on a cross-section analysis of 133 programs and a detailed examination of 15 programs.
The evidence does not support the view that IMF-supported programs adopt a one-size-fits-all approach to fiscal adjustment. In fact, there is significant variability in program design. About 40 percent of programs targeted a larger current account deficit (as a percent of GDP) and about one-third targeted an increase in the primary deficit and primary spending. Program targets are often revised during implementation, suggesting some flexibility.
Further, in spite of significant variability, the cross-country evidence does not show that growth rates decline systematically, although the capital account crisis cases are notable exceptions. There is, however, the possibility of a contractionary bias in the fiscal design because programs tend to be over-optimistic in projecting the recovery of private spending, thereby targeting stronger fiscal adjustment than necessary. Programs are reluctant to project a slowdown or negative growth—hence the possibility for some countercyclical fiscal policy is rarely discussed explicitly.
Program documents often do not clearly explain the rationale for the magnitude and pace of the fiscal adjustment and how the proposed adjustments fit into other assumptions of the program. This reduces the transparency of the program and the ability to identify the critical assumptions that need to be monitored for possible midcourse corrections. Sustained progress in structural fiscal reforms that improve resilience is often not satisfactory, and Article IV consultations are not sufficiently candid in flagging policy inaction. A core issue is how to match the shorter time frame of a program with the longer time frame often necessary to carry out these reforms.
There is no evidence that IMF-supported programs, in and of themselves, have an adverse effect on aggregate social spending. However, maintenance of aggregate spending may not be sufficient to protect the most vulnerable groups under crisis situations.
Programs documents should explain much better the rationale for the magnitude and pace of the fiscal adjustment and how it is linked to other program assumptions.
The internal review should give more emphasis to the earlier stages of the process.
Programs should give more importance to the formulation and implementation of key fiscal structural reforms—even if they cannot be fully implemented during the program period.
Surveillance should provide such a longer term road map of reform and assess progress.
Clearly delineate the operational framework in which social issues will be addressed within program design in non-PRGF countries. This could include encouraging countries to identify critical social programs they wish to protect in case of shocks.
4. Evaluation of the IMF’s Role in Poverty Reduction Strategy Papers and the Poverty Reduction and Growth Facility (2004)
This evaluation, conducted in parallel with one by the World Bank’s Operations Evaluation Department, assessed the role of the IMF in the PRS process and the extent to which the PRGF is living up to its key features. It concludes that while the PRS approach has the potential to encourage the development of country-owned and credible long-term strategies for growth and poverty reduction, actual achievements thus far have fallen considerably short of potential. This outcome is attributed, in part, to shortcomings in the design of the initiative, including a lack of clarity about the role of the IMF.
Participation in the formulation of PRSPs is more broadly based than in previous approaches, although it was typically not designed to strengthen existing domestic institutions for policymaking and accountability (e.g., through parliaments). Results in terms of ownership are mixed, with the least change in macroeconomic policy areas where there is relatively strong ownership in the narrow circle of official stakeholders but much less among other domestic stakeholders. Most PRSP strategies are an improvement over previous efforts, in terms of providing greater poverty focus, a longer-term perspective, and some orientation toward results. However, most PRSPs still do not provide a strategic road map for policymaking, especially in the area of macroeconomic and related structural policies, often avoiding difficult strategic choices. Insufficient attention was given to strengthening implementation capacity, including in the areas of IMF competence. Budgetary processes remain weak, and the linkages between the PRSP, medium-term expenditure frameworks, and budgets are generally poor.
The effectiveness of the IMF contribution has varied considerably, with marked difference between “good” and “average” practice. IMF staff typically did not participate actively to inform the policy debate on macroeconomic policy issues during the PRS formulation process and made limited contributions to understanding country-specific micro-macro linkages. Success in embedding the PRGF in the overall growth and poverty reduction strategy has been limited, partly reflecting shortcomings in those strategies. Nevertheless, program design under the PRGF has incorporated greater fiscal flexibility to accommodate aid inflows, and the IMF has been more open to considering alternative, country-driven policies. There is no evidence of generalized “aid pessimism” or a systematic “disinflation” bias.
The report makes two broad sets of recommendations. The first set recommends greater flexibility in the PRS approach to fit better the needs of countries with different administrative capacities and political systems as well as a shift in emphasis from the production of documents to the development of sound domestic policy formulation and implementation processes. The second set aims to improve IMF effectiveness by clarifying what the PRS approach means for IMF operations and strengthening accountability on what the IMF itself is supposed to deliver, guided by the priorities emerging from the PRS process.
5. The IMF and Argentina, 1991–2001 (2004)
The evaluation examines the role of the IMF in Argentina during 1991–2001. Among the major findings are:
Surveillance underestimated the vulnerability inherent in the steady increase in public debt, when much of it was dollar-denominated and externally held, and did not consider exit strategies when meaningful progress in structural reform was not forthcoming.
In late 2000, the IMF increased its commitment of resources by viewing any exchange rate or debt sustainability problem as manageable with strong policy action. This may have worked, but the authorities proved unable to implement the policies as agreed.
Even when two successive Ministers of Economy resigned in March 2001, and the new Minister began to take a series of controversial and market-shaking measures, the IMF continued to support the existing policy framework.
By mid-2001, it should have been clear that the initial strategy had failed and that Argentina’s exchange rate and public debt could not be considered sustainable. However, the IMF did not press the authorities for a fundamental change in the policy regime.
The decision to call the program off-track in December 2001 was fully justified, but the way it was done limited the ability of the IMF to continue to be engaged with Argentina.
An earlier shift in the IMF’s strategy could have mitigated the costs because Argentina’s economic health would have deteriorated that much less and more resources would have been available to moderate the inevitably painful transition process.
The Argentine experience reveals weaknesses in the IMF’s decision-making process: (i) contingency planning was insufficient; (ii) from March 2001 on, the IMF accepted a less cooperative relationship with the authorities; (iii) the IMF paid little attention to the risks of giving the authorities the benefit of the doubt beyond the point where sustainability was in question; and (iv) the Executive Board did not fully perform its oversight responsibility.
The evaluation suggests six sets of recommendations, the major points of which include:
IMF surveillance needs to be strengthened further, by making medium-term exchange rate and debt sustainability the core focus. Systematic discussion of exchange rate policy must become a routine exercise on the basis of candid staff analysis.
The IMF should have a contingency strategy from the outset of a crisis. Where the sustainability of debt or the exchange rate is in question, the IMF should indicate that its support is conditional upon a meaningful shift in the country’s policy.
The IMF should refrain from entering a program relationship with a country when there is no immediate balance of payments need and there are serious political obstacles to needed policy adjustment or reform.
To strengthen the role of the Executive Board, procedures should be adopted to encourage: (i) effective Board oversight of decisions under Management’s purview; (ii) provision of candid and full information to the Board on all relevant issues; and (iii) open exchanges of views between Management and the Board on all topics.
6. IMF Technical Assistance (2005)
The evaluation examines the technical assistance (TA) provided by the IMF to its member countries. It is based on desk reviews of a sample of countries, cross-country data on TA, and in-depth country case studies with country visits. The evaluation unbundles TA into three stages—priority setting; delivery; and monitoring and evaluation of impact. Key findings include:
Prioritization: Seventy percent of IMF TA is directed to countries with per capita income below $1,000. The volume of TA provided to countries is also positively associated with having a Poverty Reduction and Growth Facility (PRGF) or Extended Fund Facility (EFF) supported program. However, there is a weak link between TA priorities and Poverty Reduction Strategy Papers (PRSPs) or key policy issues identified in Article IV consultations. In most cases, the PRS process has still not been able to clearly identify major capacity-building needs to be taken up by TA. Thus, TA activities do not appear to be guided by a medium-term country-based policy framework.
Delivery: Country officials have generally been satisfied with the resident experts provided by the IMF, particularly their hands-on role. However, there is little involvement of the authorities in the preparation of their TA. This reduces their ownership and often masks important differences in expectations between authorities and staff.
Monitoring and impact evaluation: There has been progress in enhancing the technical capacity of the agencies receiving IMF TA. Significant variability was found in whether agencies have made full use of the increased capacity in order to have an impact on the ground. It is critical that the IMF should understand fully what prevents agencies from doing so. Part of the problem is that IMF documentation and reporting does not clearly unbundle and track the different stages of progress and hence limits the discussion with the authorities. Frequently, political interference or lack of support by the authorities prevents agencies from using the knowledge transmitted by TA effectively. Indeed, the case studies suggest that resistance by vested interests may mount as these agencies improve their ability to act. Such constraints are not candidly reported and discussed with the authorities.
The IMF should develop a medium-term country policy framework for setting TA priorities, incorporating country-specific strategic directions and linked to more systematic assessments of factors underlying past performance.
The IMF should develop more systematic approaches to track progress on major TA activities and to identify reasons behind major shortfalls.
The IMF should emphasize greater involvement by the authorities and counterparts in the design of TA activities and arrangements, as a signal of ownership and commitment.
TA experts should make stronger efforts to identify options and discuss alternatives with local officials prior to drafting TA recommendations.
The program of ex post evaluations of TA should be widened and more systematic procedures for disseminating lessons put in place.
Prioritization filters should be discontinued or replaced by ones that would more effectively guide TA allocation.
7. The IMF’s Approach to Capital Account Liberalization (2005)
The evaluation reviews the IMF’s policy advice to emerging market economies on capital account liberalization and related issues during 1990–2004. Among the major findings are:
In multilateral surveillance, the IMF’s analysis emphasized the benefits of greater access to international capital flows, while paying less attention to the risks inherent in their volatility. As a consequence, its policy advice was directed toward emerging market recipients of capital flows, and focused on how to manage large capital inflows and boom-and-bust cycles; little policy advice was offered on how source countries might help to reduce the volatility of capital flows on the supply side.
In country work there was apparent inconsistency in the IMF’s advice. Sequencing was mentioned in some countries but not in others; the intensity of advice on capital account liberalization differed across countries or across time; and a range of views was expressed on use of capital controls. Analysis of country documents suggests that this largely reflected reliance on the discretion of individual IMF staff members.
The lack of a formal IMF position on capital account liberalization gave individual staff members freedom to use their own professional and intellectual judgment in dealing with specific country issues.
In more recent years, somewhat greater consistency and clarity has been brought to bear on the IMF’s approach to capital account issues. While the majority of staff members now appear to accept the new paradigm (the so-called “integrated approach” that emphasizes sequencing and preconditions), there continues to be some uneasiness with the lack of a clear position by the institution.
On the basis of these findings, the evaluation makes two sets of recommendations:
There is a need for more clarity on the IMF’s approach to capital account issues. Possible steps could include: (i) clarification by the Executive Board of the scope of IMF surveillance on capital account issues; (ii) development of an operationally meaningful indication of the benefits, costs, and risks of capital account liberalization as well as moving at different speeds; (iii) a statement by the Board clarifying the common elements of agreement on capital account liberalization, in order to provide staff with clear guidance on what the IMF’s official position is.
The IMF should give greater attention to the supply-side factors of international capital flows and what can be done to minimize the volatility of capital movements. Building on recent initiatives, the IMF should provide analysis of what can be done to minimize the volatility of capital flows by operating on the supply side.
8. IMF Support to Jordan, 1989–2004 (2005)
The report assesses the extent to which the IMF contributed to tackling Jordan’s major macroeconomic challenges during the period of the country’s engagement in IMF-supported programs. Jordan “graduated” from reliance on such programs in July 2004 after 15 years of almost continuous involvement. The main focus of the evaluation was on the effectiveness of the IMF-supported programs to Jordan. Jordan was on the verge of defaulting on its external debt payment obligations following a severe balance of payments crisis during late 1980s. In addition to high external debt, twin deficits, rigid fiscal structure, and structural impediments to growth, Jordan was susceptible to external shocks in the region.
The report’s overall assessment of the IMF’s role in Jordan was that it had been moderately successful. The IMF helped the authorities to address macroeconomic stabilization challenges successfully, but some of the main structural rigidities that underlay the financial crisis that led Jordan to its first IMF-supported program still remained, especially on the fiscal side. The evaluation found that most staff papers to the Board on Jordan did not provide a clear rationale for the magnitude and composition of targeted adjustment. This made it difficult for the Board to make judgments on the factors underlying any subsequent failures to achieve key objectives, and on the appropriateness of any program modifications. A greater focus on public expenditure policy to advice on the major expenditure cuts envisaged in the early programs would have been desirable. In general, there was not much difference in the macroeconomic policy discussions undertaken during Article IV consultations and those in program discussions.
The report highlighted broad lessons relevant to IMF operations in other countries and two specific lessons for the IMF in Jordan.
Lessons with broad applicability:
The underlying rationale for key program design elements should be explained clearly in Board papers.
There is need for more candor in staff report assessments of risks to programs.
The programs need to be set in an explicitly longer-term perspective.
Fiscal reforms should be emphasized at an early stage of formulation of institutional reforms.
The IMF and the World Bank need to set clear objectives signaling what the needs and obligations of each institution are on specific items in the program.
Timetables need to take into account the political economy aspects.
There is need for wider dissemination of IMF TA reports to have a more informed public discourse on policy advice.
IMF’s future role in Jordan:
There is need to help the authorities design a macroeconomic framework to manage a decline in grants and preserve fiscal sustainability.
There is need to help design strategies to tackle Jordan’s key remaining fiscal rigidities.
9. Financial Sector Assessment Program (2006)
This evaluation assesses the IMF role in the FSAP along with a parallel report by the World Bank’s Independent Evaluation Group. The evaluation concludes that the FSAP significantly improves the IMF’s ability to conduct financial sector surveillance and to understand the key linkages between financial sector vulnerabilities and macroeconomic stability. Key features to be preserved going forward include an integrated approach to assessing financial sector vulnerabilities and development needs; an institutional linkage to surveillance; and a mechanism to coordinate IMF and World Bank inputs.
Despite these achievements, the initiative is at a critical crossroads and some of the gains could be eroded without significant modifications to address two related sets of problems. First, financial sector assessments (FSAs) have not been fully integrated as a regular part of IMF surveillance. Second, there are serious doubts that current incentives for participation and associated priority-setting will ensure adequate coverage of systemic and vulnerable countries. Moreover, while the evaluation concludes that the overall average quality of the FSAP exercises is quite high, several shortcomings are identified. Most notably, insufficient attention has been paid to cross-border financial linkages and their potential consequences. In addition, many FSAPs inadequately prioritize the urgency of their recommendations, which hampers effective follow-up by both surveillance and technical assistance.
The evaluation’s recommendations focus on three key themes. First, changes are needed in how country choices for FSAs are made and in how those assessments are integrated into IMF surveillance. The approach proposed by the report contains three mutually supporting recommendations: sharper criteria for priority-setting, with choices for country-specific strategies across a range of modalities for financial sector surveillance; strengthened incentives for comprehensive assessment exercises to foster coverage of countries of systemic importance, albeit within a still voluntary framework for the FSAP; and strengthened links between FSAPs and Article IV surveillance.
Second, steps are needed to improve further the quality of the FSAP and strengthen its impact. These would include clearer prioritization of recommendations and candid discussion of potential consequences; more systematic inclusion of cross-border financial sector issues; and greater involvement of the authorities in the overall process. Changes are also needed in the organization of IMF mission activities to utilize scarce financial sector expertise more effectively in the surveillance process.
Third, the evaluation found that the use of joint IMF–World Bank teams (as well as outside experts) enhanced the depth of analytical expertise and credibility of the findings. But if strengthened incentives for participation are successful, more concrete guidelines for setting priorities and contributing resources will be required, with the IMF taking the lead where significant domestic or global stability issues are present.
10. Multilateral Surveillance (2006)
The evaluation examines the effectiveness of multilateral surveillance. Among the major findings are:
The outputs of multilateral surveillance are largely successful in analyzing topical issues in ways that reflect the IMF’s comparative advantage. However, they give insufficient coverage to exchange rate issues, which is surprising given the IMF’s mandate.
The WEO has largely succeeded in identifying relevant issues and global risks in a timely way. In terms of identifying global macroeconomic and financial risks, the WEO and the GFSR compare favorably with similar publications of other bodies.
IMF surveillance has a strong country orientation. As a result, multilateral surveillance has not sufficiently enriched the policy advice offered through bilateral surveillance by identifying scope for collective action.
The WEO and the GFSR are insufficiently integrated. This owes largely to a “silo” problem in the IMF’s internal organization in which different departments pursue their individual mandates with insufficient attention to the organization’s overall mission.
The WEO and the GFSR enjoy wide media coverage but they reach senior policymakers only indirectly. This is understandable. Still, much shorter G-7 and G-20 surveillance notes are also rarely read by policymakers themselves.
Interviews with the senior country officials suggest that multilateral surveillance, if effectively conducted and communicated, is able to provide valuable input into policymaking.
On the basis of these findings, the evaluation makes four key recommendations:
Define more clearly the goals of multilateral surveillance and the mechanisms to achieve them. Multilateral perspectives must be better integrated into bilateral surveillance.
Engage more deeply with such intergovernmental groups as the G-7 and the G-20, as they provide opportunities for a more frank discussion of policy spillovers and possibilities for collective policy action and for more effective peer pressure.
Enhance the roles of the Executive Board and the IMFC in multilateral surveillance. The Board should identify key global policy actions while the IMFC should focus on policy spillovers and opportunities for collective action.
Key multilateral surveillance products must be more “customer” focused, by streamlining their content and targeting them more strategically at various audiences. Surveillance notes should concentrate on spelling out the consequences of policy spillovers and identifying options for dealing with them.
11. The IMF and Aid to Sub-Saharan Africa (2007)
This evaluation examined the IMF’s role and performance in the determination and use of aid to low-income countries in Sub-Saharan Africa (SSA). The time period of the evaluation (1999–2005) saw improving performance in much of SSA and increasing aid levels as debt relief gained momentum and donors introduced multi-donor budget support. It also saw the IMF replace the Enhanced Structural Adjustment Facility (ESAF) with the Poverty Reduction and Growth Facility (PRGF) as its primary lending instrument in low-income countries.
A recurring theme of the evaluation was the perceived disconnect between the IMF’s rhetoric on aid and poverty reduction and its practice at the country level. Underlying this theme was a larger issue of attempted—but ultimately unsuccessful—institutional change that the evaluation explored in explaining its findings. When introduced, the PRGF was intended to involve a new way of working for the IMF, with programs based on country-owned measures geared to promoting poverty reduction and growth. However, institutional developments within the Fund, including limited support from the Executive Board, which was deeply divided on the Fund’s role in low-income countries, and turnover in top Management, prevented the actualization of significant change. The result was widespread ambiguity and confusion—both internally and externally—about what the Fund’s policies and practices actually were in the areas under study.
PRGF-supported macroeconomic policies generally accommodated the use of incremental aid in countries with high stocks of reserves and low inflation; in other countries additional aid was programmed to be saved to increase reserves or to retire domestic debt.
PRGFs neither set ambitious aid targets nor identified additional aid opportunities, including in country circumstances in which absorptive capacity exceeded projected aid inflows.
Of the key features distinguishing the PRGF from the ESAF, fiscal governance (a long-standing IMF mandate) was far more systematically treated than other (newer) elements, such as the use of social impact analysis or the pro-poor and pro-growth budget provisions.
IMF communications on aid and poverty reduction contributed to the external impression that the IMF committed to do more on aid mobilization and poverty-reduction analysis.
The IMF missed opportunities for communicating with a broader audience in SSA.
The Executive Board should clarify IMF policies on the underlying performance thresholds for the accommodation of additional aid, the mobilization of aid, alternative scenarios, poverty and social impact analysis, and pro-poor and pro-growth budget frameworks. IMF Management should provide clear guidance to staff on these policies, including what is expected with respect to working with the World Bank and other partners. The External Relations Department should ensure the consistency of institutional communications with Board-approved operational policies and IMF-supported operations.
IMF Management should establish transparent mechanisms for monitoring and evaluating the implementation of the clarified policy guidance.
Management should clarify expectations and resource availabilities for resident representatives’ and mission chiefs’ interactions with local donor groups and civil society.
12. IMF Exchange Rate Policy Advice (2007)
The main finding is that, over the 1999–2005 period, the IMF was simply not as effective as it needs to be to fulfill its responsibilities to exercise surveillance over the international monetary system and members’ exchange rate policies.
The reasons for the IMF’s failing to fully meet its core responsibility were many and complex. Among these reasons were: a lack of understanding of the role of the IMF in exchange rate surveillance; a failure by member countries to understand and commit to their obligations to exchange rate surveillance; a strong sense amongst some member countries of a lack of evenhandedness in surveillance; a failure by Management and the Executive Board to provide adequate direction and incentives for high-quality analysis and advice on exchange rate issues; and the absence of an effective dialogue between the IMF and many countries.
Recommendations require action by IMF staff, Management, the Board and authorities:
First, the rules of the game for the IMF and its member countries need to be clarified, with a revalidation of the fundamental purpose of surveillance. At the same time, practical policy guidance should be developed on key analytical issues, including on the stability of the system (flowing from periodic Executive Board policy review of the stability of exchange regimes and exchange rates), and on the use and limits of intervention.
Second, Management should give much greater attention to ensuring effective dialogue with authorities, by developing a strategic approach to identify opportunities for improvement, and by adjusting incentives to raise controversial issues.
Third, actions are required to deal with problems in implementing existing policy guidance. These should: resolve inconsistencies and ambiguities over the issue of regime classification; back up advice on exchange rate regimes more explicitly with analytical work; improve assessments of the exchange rate level, including by developing needed analytical work; pursue the problems of data provision for surveillance; and give incentives to implement guidance for the integration of spillovers into bilateral and regional surveillance.
Fourth, the management of staff’s work should be improved. Better focus is needed on the analytical work on exchange rates, including by clarifying responsibility and accountability for exchange rate policy issues, and reconsidering the structure of staff teams.
Fifth, the issues of confidentiality and Executive Board oversight over policy discussions should be clarified. An understanding is needed on what would, and what would not, be revealed to the Board. Confidential discussions about contingent policy actions should be a regular feature of dialogue with member countries, while establishing a way for the Board to exercise its oversight responsibilities in this area.
Finally, since action on global imbalances has not been fully explored, the opportunities for potential multilateral concerted action deserve to be a key strategic management focus.
13. Structural Conditionality in IMF-Supported Programs (2007)
This evaluation examines factors influencing the effectiveness of IMF structural conditionality in bringing about structural reform, and assesses the impact of the streamlining initiative launched in 2000 and of the 2002 Conditionality Guidelines. These aimed at reducing the volume and scope of structural conditionality by requiring “parsimony” in the use of conditions and stipulated that conditions must be “critical” to the achievement of the program goals.
The evaluation finds that during the period 1995–2004 there was extensive use of structural conditionality in IMF-supported programs, with an average of 17 conditions per program/year. Most of these conditions had little structural depth and only about half of them were met on time. Compliance was only weakly correlated with subsequent progress in structural reform. Ownership of the reform program by the economic team and by the line ministries in charge of the specific measures was necessary both for compliance and for continuity of the reform. Compliance and effectiveness were higher in the areas of IMF core competency, such as public expenditure management and tax-related issues, and lower in areas such as privatization and reform of the wider public sector.
The streamlining initiative did not reduce the volume of conditionality, partly because structural conditions continued to be used to monitor other initiatives such as donors’ support programs and the European Union (EU) accession process. But it helped to shift the composition of conditionality toward IMF core areas and new areas of basic fiduciary reform. At the same time, the IMF moved away from controversial areas where it had little impact and that largely fell within the World Bank’s areas of expertise. Nonetheless, Fund arrangements still included conditions that seem not to have been critical to program objectives.
Recommendations include reaffirming the need to reduce the volume of structural conditionality. As a practical first step, a notional cap could be set, possibly at four or five conditions per year—half the current average for performance criteria and prior actions. The use of structural benchmarks should be discontinued and measures with low structural content should not be part of conditionality. Normally, conditionality should be restricted to the core areas of IMF expertise. In other critical areas such as the wider public sector, the IMF should play a subsidiary role to that of the World Bank, which has greater expertise in these areas. Explicit Board guidance would be needed when reforms in noncore areas are deemed critical but effective cooperation with the Bank is unlikely to crystallize in time. The Fund should develop a monitoring and evaluation framework linking conditions to reforms and goals, which would provide a more robust basis for assessing programs results. Program documents should explain how the proposed conditionality is critical to achieve explicit objectives. For PRGFs, in particular, program requests should be accompanied by an operational roadmap covering the length of the program, explaining the proposed reforms, their sequencing, and expected impact.
14. Governance of the IMF: An Evaluation (2008)
This evaluation assesses the degree to which Fund governance is effective and efficient, and whether it provides sufficient accountability and channels for stakeholders to have their views heard. The focus is on institutional structures as well as on the formal and informal relationships between the Fund’s main bodies of governance: the Executive Board, Management, and the International Monetary and Financial Committee (IMFC).
For much of the past six decades, gradual reforms in its governance allowed the Fund to remain relevant in a changing world economy. But the reforms have not kept pace with changes in the environment in which it operates. Today, the institution’s legitimacy and relevance are being questioned. Much attention has recently been focused on quotas and voting power, but broader governance reform also holds the potential to strengthen the Fund’s legitimacy, accountability, and effectiveness.
Overall, effectiveness has been the strongest aspect of Fund governance, allowing fast and consistent action particularly in times of systemic crisis. On the other hand, accountability and voice have been its weakest aspects, which if left unaddressed would likely undermine effectiveness over the medium term. The evaluation has four broad conclusions and recommendations, and it proposes a series of detailed measures specific to each of the main governance bodies.
First, there is a lack of clarity on the respective roles of the different governance bodies, and in particular between the Board and Management. To strengthen the IMF’s effectiveness and to facilitate accountability, the roles and responsibilities of each of its governance bodies need to be clarified with a view to minimizing overlaps and addressing possible gaps.
Second, the Fund needs more systematic ministerial involvement. The IMFC, as an advisory body, lacks a mandate for setting strategic directions and providing high-level oversight of the institution. To fulfill these functions, the evaluation calls for the activation of the Council, as contemplated in the Articles of Agreement, which should operate with a high degree of consensus, perhaps through the use of special majorities.
Third, the Board’s effectiveness is hindered by excessive focus on executive, rather than supervisory, functions. The Board should reorient its activities towards a supervisory role, playing a more active part in formulating strategy, monitoring policy implementation to ensure timely corrective actions, and exercising effective oversight of Management. To this end, the Board would need to change many of its working practices, shifting away from executive, day-to-day operational activities, including through more delegation to committees and possibly to Management.
Finally, a framework needs to be put in place to hold Management accountable for its performance. Work is under way to set up such a framework, which should specify criteria and a process for regular assessments.
15. IMF Involvement in International Trade Policy Issues (2009)
Trade policy occupies an unusual and at times problematic place in the work of the IMF. Few would dispute that trade policies of IMF members have strong influences on macroeconomic stability. However, trade policies are often seen as peripheral to the IMF’s core competency. This leaves scope for a range of views on the proper role for the IMF in advising on trade policy. Also, the IMF’s orientation toward unilateral trade liberalization has stoked the debates on whether such liberalization is always in a country’s own interests and whether preferential trade agreements are harmful. Added to these debates are charges that the IMF has pressed harder for liberalization in borrowing countries than in countries with which it has a surveillance-only relationship.
This evaluation, which examines the IMF’s involvement in trade policy issues during 1996–2007, addresses five questions. What is the nature of the IMF’s mandate to cover trade policy? Did the IMF work effectively with other international organizations on trade policy issues? Did the Executive Board provide clear guidance to staff on the IMF’s role and approach to trade policy? How well did the IMF address trade policy issues through lending arrangements and surveillance? Was IMF advice effective?
The evaluation finds that the IMF’s role in trade policy has evolved in some desirable and some less desirable ways. In its general streamlining after 2000, the IMF scaled back its involvement in traditional trade policy issues (tariff and nontariff barriers to merchandise trade), especially in the context of conditionality. This is welcome as average tariffs in most countries had fallen to relatively low levels, conditionality often did not achieve lasting changes in trade policy, and the pressure for unilateral liberalization especially through conditionality created tensions with multilateral negotiations in the World Trade Organization.
But in other respects the IMF’s scaling back on trade policy advice came at the cost of constructive roles in trade issues central to financial and systemic stability. Three such gaps stand out. First, the IMF has not clearly enough defined or pursued a role vis-à-vis trade in financial services—an area where its perspective is essential. Second, fairly active interest of IMF researchers in macroeconomic and systemic effects of preferential trade agreements has not adequately filtered into bilateral and multilateral surveillance. Third, the IMF has not given due attention recently to global effects of trade policies (such as high agricultural tariffs and subsidies) in systemically important countries.
The evaluation recommends several ways to use the limited resources the IMF can devote to trade policy to fill these gaps. More active interinstitutional cooperation, backed by formal interactions, is essential. Also, however, the IMF needs a small repository for in-house expertise—a division solely devoted to trade policy—to be the locus of such cooperation and to help identify trade policy issues in which the IMF should be involved. Finally, regional and global implications of trade policy developments should be explored in depth periodically in World Economic Outlook and Regional Economic Outlook exercises. The Board should regularly review and give guidance on the IMF’s role in trade policy issues.
16. IMF Interactions with Member Countries (2009)
This evaluation assesses the degree to which IMF interactions with member countries were effective and well managed in 2001–08, with particular attention paid to 2007–08. In spite of relatively high perceptions of overall effectiveness in some country groupings, there is evidence of lack of agreement between authorities and staff on the scope of interactions in some cases, and of widely varying effectiveness in particular roles. Interactions were effective in program and technical assistance context, and, in general, in contributing to a good exchange of views and in providing objective assessments. However, in other areas, including in the international dimensions of surveillance, effectiveness and quality were not rated highly.
Evidence suggests that interactions were least effective with advanced and large emerging economies. They were most effective with PRGF-eligible countries, and, to a lesser extent, with other emerging economies. Particularly troubling was the continuing strategic dissonance with large advanced economies, especially about the Fund’s role in international policy coordination, policy development, and outreach. The authorities did not give the Fund high marks in these areas. Neither did staff, who nevertheless aimed to do more. The evidence also points to limited effectiveness with large emerging economies, many of whom saw the surveillance process as lacking value and/or evenhandedness.
The evaluation found that outreach with stakeholders beyond government contributed little to the effectiveness of interactions. The Fund’s transparency policy did less than staff had hoped to increase traction, as some authorities blocked timely dissemination of mission findings. Dissemination initiatives designed to gain influence in domestic policy debates by repositioning the Fund as an informed analyst—and distancing it from the negative legacy of past engagement—remain work in progress.
The evaluation found that interactions were undermanaged, although some individuals managed particular interactions very well. The Fund was ineffective in enhancing traction with surveillance-only countries. The Fund paid too little attention to the technical expertise and other skills that might have added value, and neglected to manage pressures that staff felt to provide overly cautious country assessments—a finding of major concern, especially in systemically important countries. In PRGF-eligible countries, attractive financing, debt relief, and strong links to donor funding made for an abundance of traction. But in some cases it also led to what authorities perceived to be arrogant and dictatorial staff behavior—though they saw evidence of progress in recent years. Staff incentives and training largely ignored interactions, and responsibilities for relationship management were not clear.
The following recommendations aim at enhancing the effectiveness of interactions:
To make the Fund more attractive to country authorities and promote traction: (i) improve the quality of the international dimensions of the Fund’s work; (ii) recruit specialist skills and bring more experts on country visits; (iii) articulate menus of products and services for emerging-market and advanced economies; (iv) replace country surveillance agendas with strategic agendas.
To improve the effectiveness of outreach: (v) clarify the rules of the game on outreach; and (vi) decide how to handle the Fund’s negative reputational legacy in countries where it is a factor undermining interactions, and equip staff with the skills and resources to follow through.
To improve the management of interactions: (vii) develop professional standards for staff interactions with the authorities; (viii) increase mission chief and staff tenure and training, and improve incentives for interactions; and (ix) clarify relationship management responsibilities and accountabilities.
17. IMF Performance in the Run-Up to the Financial and Economic Crisis: IMF Surveillance in 2004–07 (2011)
This evaluation assesses the performance of IMF surveillance in the run-up to the global financial and economic crisis and offers recommendations on how to strengthen the IMF’s ability to discern risks and vulnerabilities and to warn the membership in the future. It finds that the IMF provided few clear warnings about the risks and vulnerabilities associated with the impending crisis before its outbreak. The banner message was one of continued optimism after more than a decade of benign economic conditions and low macroeconomic volatility. The IMF, in its bilateral surveillance of the United States and the United Kingdom, largely endorsed policies and financial practices that were seen as fostering rapid innovation and growth. The belief that financial markets were fundamentally sound and that large financial institutions could weather any likely problem lessened the sense of urgency to address risks or to worry about possible severe adverse outcomes. Surveillance also paid insufficient attention to risks of contagion or spillovers from a crisis in advanced economies. Advanced economies were not included in the Vulnerability Exercise launched after the Asian crisis, despite internal discussions and calls to this effect from Board members and others.
Some of the risks that subsequently materialized were identified at different times in the Global Financial Stability Report, but these were presented in general terms, without an assessment of the scale of the problems, and were undermined by the accompanying sanguine overall outlook. These risks were not reflected in the World Economic Outlook or in the IMF’s public declarations. The IMF did appropriately stress the urgency of addressing large global current account imbalances that, in the IMF’s view, risked triggering a rapid and sharp decline in the dollar that could set off a global recession. But the IMF did not link these imbalances to the systemic risks building up in financial systems.
The IMF’s ability to detect important vulnerabilities and risks and alert the membership was undermined by a complex interaction of factors, many of which had been flagged before but had not been fully addressed. The IMF’s ability to correctly identify the mounting risks was hindered by a high degree of groupthink, intellectual capture, a general mindset that a major financial crisis in large advanced economies was unlikely, and inadequate analytical approaches. Weak internal governance, lack of incentives to work across units and raise contrarian views, and a review process that did not “connect the dots” or ensure follow-up also played an important role, while political constraints may have also had some impact.
The IMF has already taken steps to address some of these factors, but to enhance the effectiveness of surveillance it is critical to clarify the roles and responsibilities of the Board, Management, and senior staff, and to establish a clear accountability framework. Looking forward, the IMF needs to (i) create an environment that encourages candor and considers dissenting views; (ii) modify incentives to “speak truth to power;” (iii) better integrate macroeconomic and financial sector issues; (iv) overcome the silo mentality and insular culture; and (v) deliver a clear, consistent message on the global outlook and risks.
18. Research at the IMF: Relevance and Utilization (2011)
This evaluation assesses research produced at the IMF between 1999 and 2008. It focuses on relevance and utilization, but also examines technical quality and management. Research is defined broadly to capture most analytical publications of the IMF, ranging from surveillance-oriented output, for example, selected issues papers (SIPs) prepared for Article IV consultations and the analytical chapters of the World Economic Outlook (WEO) and Global Financial Stability Report (GFSR), to more academically-oriented output, for example, working papers (WPs) and publications in external journals. These outputs comprised a large body of research, about 650 publications annually, at a cost of about 10 percent of the IMF budget.
The evaluation finds that IMF research was widely read, that it included a large number of high-quality and very useful publications, and that it was appreciated by country authorities and the research community. This was particularly true for the WEO and GFSR, but also for many other publications. Nonetheless, several issues merit attention.
First, the relevance of research was often hampered by lack of early consultation with country authorities on research themes and by lack of sufficient country and institutional context. Also, authorities indicated that some important issues, such as macro-financial linkages and aspects of monetary policy, were not adequately covered. To strengthen relevance, the IMF should conduct a periodic strategic review of the function and uses of its research product lines to establish whether they should be strengthened, redesigned, or discontinued. Consultation with authorities on research topics and discussions of results should become standard practice. Increased and earlier interaction with authorities as well as longer country assignments by mission members would enhance the country and institutional context of research.
Second, the technical quality of IMF research publications was quite diverse. The WEO, GFSR, and external publications were generally of high quality. On the other hand, the quality of SIPs and WPs, which are not subject to a rigorous quality review, was lower and more variable. To enhance quality, adequate time and resources should be allocated to each research project, even if this leads to fewer publications. The review of research products should be strengthened to improve quality and to prevent the publication of low quality products.
Third, many authorities reported that IMF research was message-driven, and many staff indicated that they often felt pressure to align their conclusions with IMF views. To enhance their quality, reputation, and utilization, working papers should reflect the results of technical analysis even if these are not well aligned with messages in surveillance activities documents.
Finally, there is a need for greater prioritization and coordination of research across the IMF. To this end, Management should designate a senior staff member, the Research Coordinator (RC), to coordinate research activities across the organization, including by setting standards for quality review processes and publication policies, to promote greater openness, and to address other weaknesses identified in this evaluation. The RC should prepare an indicative medium-term research agenda, in consultation with member countries and the Executive Board, and it should report annually to them on its implementation. This medium-term agenda should not be seen as excluding research on other relevant issues.