- International Monetary Fund. Independent Evaluation Office
- Published Date:
- October 2005
The purpose of this evaluation is to examine technical assistance (TA) provided by the IMF to its member countries and to derive operational recommendations that can enhance the contribution of TA to the overall IMF mandate.
The stated objectives of IMF TA were defined in the most recent policy statement on the subject. “In accord with the IMF’s first Article of Agreement, the objective of the IMF’s TA program is to contribute to the development of the productive resources of member countries by enhancing the effectiveness of economic policy and financial management. The IMF’s TA program works to achieve this objective in two ways. First, much of the IMF’s TA seeks to support the efforts of members to strengthen their capacity— in both human and institutional resources—to formulate and implement sustainable, growth-oriented and poverty-reducing macroeconomic, financial, and structural policies. Second, the IMF assists countries in the design of appropriate macroeconomic and structural policy reforms, taking account of the lessons learned by other countries in addressing similar economic policy concerns.”1
Thus, the definition of TA used within the IMF is quite wide. In practice, it includes not only activities to support long-term institution building in countries themselves, but also fact-finding and analytical activities that allow the IMF to advise countries on specific policies and in the design of IMF-supported programs. This wide definition presents special analytical challenges, both to identifying TA priorities as well as to evaluating its impact.
The IMF devoted about $80 million of direct resources to TA in FY2004 (excluding training), which is about 10 percent of the total administrative budget.2 The pattern of IMF TA activities has evolved considerably over time. In the early 1990s, a significant share of TA resources was allocated to the transition countries to help develop the policies and institutions for a market economy. Since the late 1990s, a greater share has focused on countries receiving assistance under the PRGF, including for improving governance and for capacity building. Crisis prevention and resolution, TA to post-conflict countries, and TA in connection with regional arrangements (such as the new regional technical assistance centers), have also received increased attention.
Issues to Be Addressed
To assess whether IMF TA is achieving its stated objectives and, if necessary, to identify corrective steps, it is useful to unbundle the TA process into three stages:
- Prioritization and resource allocation. What are the factors determining the allocation of TA resources among countries? How are TA priorities selected at the country level and how well are they related to broader development strategies in the country to assure relevance? How is the balance between short-term TA (often to support the policy advice of the IMF) and TA to support (long-term) institution building established?
- TA delivery. What can be learned about specific modalities of TA delivery and ways to interact with authorities during the implementation period so as to increase the effectiveness of TA?
- Impact and evaluation. What has been the impact of TA and how is it tracked and monitored, including through ex post evaluation? What are the key factors that contribute to TA effectiveness?
How is TA allocated among countries and priorities identified within countries?
What patterns can be found in the allocation of IMF TA resources among countries? Are they responsive to indicators of low institutional capacity? How sensitive is the level of TA to different types of financial arrangements with the IMF? Are TA allocations to countries dominated by the level of external financing from donors (for example, trust funds)? The answers to these questions will help identify the major forces shaping cross-country allocations of TA.
Any process for allocating TA resources across competing uses will need to decide how priorities are set between and within countries and how resources are reallocated on the basis of evidence on outcomes (track records). The process would need the following two components:
- An identification of the broad areas where the IMF would seek to maintain or develop “core competencies” in TA in the light of its small size relative to global TA efforts and the need to closely complement its other instruments, namely, lending in support of adjustment, surveillance, and related diagnostic activities, such as ROSCs and FSAPs.
- A mechanism for prioritizing TA activities within these broad areas. Since the IMF’s definition of TA includes both short-term diagnostic activities and longer-term institution building, a major challenge will be to decide on the appropriate mix between the two types of activity in each country, and how they interact over time with each other and with other IMF instruments.
A key question to be addressed is how well this process is working today. Is TA provided to a country anchored in a medium-term strategy from which TA priorities can be coherently derived? Are Article IV consultations and the PRS process (in the case of low-income countries) providing such an integrated view? Or is TA the result of a process driven by the separate needs of programs, ROSCs, and FSAPs? Given budget constraints, how are trade-offs managed in practice? How well are the authorities involved in the process?
How effective has been the process of TA delivery?
Achieving the final objectives of TA will also depend on the effectiveness of the means of delivery. Several questions can be raised: (1) How clear are the initial TORs and how involved are the authorities in their preparation? (2) Is there clarity at the outset regarding how progress will be measured? (3) What are the features for a successful resident expert? (4) Is the mixture between short-term missions and long-term resident experts (LTEs) appropriate given country circumstances? Should LTEs be in charge of monitoring program conditionality in their areas of expertise? (5) How effective has been the dialogue with the authorities during the delivery period in enhancing the ownership of TA recommendations?
What was the impact of TA and what factors contributed to its effectiveness? How was this tracked?
An assessment of the impact of TA and the factors influencing it—both country factors and those under the control of the IMF—is critical in allocating TA in the future and improving its design. Several questions emerge: (1) What was the impact of TA and what are the key factors determining impact? (2) How well is the IMF tracking intermediate and final outputs? (3) Does the IMF distinguish between the improved technical know-how of agencies receiving TA and their subsequent ability to apply that knowledge by carrying out their duties—the latter requiring political support from the authorities? (4) How candid is the reporting to headquarters when effectiveness is impaired by political interference?
Methodology and Empirical Basis
The evaluation is organized around the set of three questions raised above—which also defines three consecutive stages of the results chain determining the impact of TA. They are addressed in this evaluation by using different types of empirical evidence, namely:
- Internal IMF cross-country data on TA field activities (derived from the Travel Information Management System (TIMS)) to examine trends and patterns in the allocation of TA among countries and identify the forces behind these patterns.
- A sample of 10 low-income countries to study the link between the PRSP and the identification of TA needs.
- A sample of 18 countries to assess the link between Article IV surveillance documentation and TA activities.
- A sample of 12 technical assistance consultation (TC) exercises that were piloted in an effort to enhance the link between Article IV surveillance and TA activities in a country.
- In-depth analysis of 6 countries, including field visits to elicit views from local counterparts and authorities (Cambodia, Honduras, Niger, Ukraine, Yemen, and Zambia).
- Interviews with IMF staff, including TA and training coordinators in both functional and area departments, as well as staff and consultants involved in TA activities in the countries above in the period examined.
- A review of findings of 11 internal and external evaluations of thematic or regional TA provided by the IMF.3
Figure 1.1Stages and Results Chain of TA Activities Supporting Institution Building
Figure 1.1 describes how this evidence is used throughout the different stages of the evaluation. As we move to understand the impact of specific TA activities and its determinants, the evaluation relies on the in-depth case studies and qualitative evidence drawn from interviews. The measures used to assess impact, and their limitations, are discussed in more detail in Chapter 6.
The report is organized as follows: Chapter 2 reviews briefly the results of, and follow-up to, the 1999 Fund-wide TA evaluation. Chapter 3 examines trends and patterns in the allocation of TA resources across subjects and countries; Chapter 4 examines how well the identification of TA priorities at the country level is working; Chapter 5 examines the process and modalities of TA delivery; and Chapter 6 examines the impact of TA, how progress is being tracked, and factors that may impair progress. At the end of each chapter, recommendations are presented.
This evaluation does not assess the role of the new RTCs because their experience is too recent for an effective evaluation. A review of preliminary experiences is at present being undertaken by OTM.
Direct cost of TA includes (1) all missions headed by TA departments, except missions in connection with ROSCs, FSAPs, etc.; (2) follow-up TA missions to implement the recommendations of these surveillance activities; (3) resident experts hired by the TA departments; and (4) TA activities undertaken at headquarters by TA departments.
These evaluations include: “External Evaluation of Technical Assistance Provided by the IMF’s Monetary and Exchange Affairs Department” (IMF, 1996); “Fiscal and Monetary Management Reform and Statistical Improvement: Pacific Financial Technical Assistance Centre, Report of the Evaluation Team” (IMF, 1997); “Review of Technical Assistance Provided by the Revenue Administration Division of the Fiscal Affairs Department in Fiscal Year 2000” (IMF, 2000d); “An Assessment of FAD Technical Assistance in Establishing the Treasuries in the Baltic Countries, Russia, and Other Countries of the Former Soviet Union” (IMF, 2000a); “Review of the Technical Assistance by the Legal Department” (IMF, 2001c); “Report of the Task Force on the Review of the Monetary and Exchange Affairs Department” (IMF, 2002b); “External Evaluation of Technical Assistance in PEM in Francophone African Countries” (IMF, 2003b); “Statistical Capacity Building: Case Studies and Lessons Learned” (IMF, 2003h); “Public Expenditure Management Reform in Anglophone African Countries: An Assessment of FAD Technical Assistance” (IMF, 2003g); “Mid-Term Review of Caribbean Regional Technical Assistance Centre” (IMF, 2003f); and “Internal Assessment of FAD’s TA in Public Expenditure Management: The Mainly Anglophone African Countries” (IMF, 2003d).