IEO Annual Report 2005-06
Chapter

Appendix 10 Evaluation of IMF Technical Assistance: Recommendations, Board Response, and Subsequent Follow-Up

Author(s):
International Monetary Fund. Independent Evaluation Office
Published Date:
January 2007
Share
  • ShareShare
Show Summary Details
IEO RecommendationsExecutive Board Response1Task Force Recommendations2Follow-Up3
Prioritization and resource allocation
The IMF should develop a medium-term country policy framework for setting TA priorities, incorporating country-specific strategic directions and linked to more systematic assessments of factors underlying past performance.Directors endorsed this recommendation and generally supported the managerial approaches suggested for carrying it forward. In particular, most Directors agreed that in low-income countries, the PRSP should increasingly serve as a vehicle for identifying medium-term TA needs and improving coordination among agencies, although in a number of cases, the focus of PRSPs will need to be sharpened to carry out this role effectively. In other countries, Directors stressed the importance of developing country-centered frameworks for identifying TA needs and noted that a variety of approaches—possibly including greater use of Article IV consultations to assess needs—may be appropriate.Short TA Country Strategy Notes (TACSNs) should be prepared for “intensive IMF TA users” and systemically important TA countries. With a strategic perspective developed in their surveillance or use of Fund resources activities, area departments would identify the issues/ problems to be addressed by TA. TACSNs would present the IMF’s TA strategy to the authorities, and seek their input into that strategy. TACSNs should provide area and functional departments with a solid information base to further their dialogue on TA prioritization decisions. In view of the possible significant resource implications of this proposal and an inconclusive earlier experience with technical consultations, the proposal would be launched on a pilot basis.Area departments already include TA monitoring on their resident representative terms of reference when applicable.4
The TACSN pilot was completed in April 2006. An evaluation of experience with the TACSN pilot is currently ongoing, with a view to preparing a possible guidance note on the form and content for a next set of TACSNs under guidance from the Technical Assistance Subcommittee of the Medium-Term Strategy Committee on Capacity Building.
Tracking progress and reasons for shortfalls
The IMF should develop more systematic approaches to track progress on major TA activities and to identify reasons behind major shortfalls.Directors supported the recommendation that at the outset of major TA activities, staff and authorities should agree on measurable indicators of progress covering all major stages of the TA life cycle. Better tracking includes careful explanation of shortfalls in execution, and candid staff reporting on obstacles to progress. However, on these endeavors staff should avoid imposing conditions on members without their consent. While supporting the view that implementation records should be an important guide in weighing TA requests, a number of Directors cautioned against using tracking indicators mechanistically for TA resource allocations.The Task Force notes the IEO’s view on the TA Information Management System (TAIMS) that was being developed as “an opportunity to systematically improve tracking and monitoring of TA on a IMF-wide basis,” and a “vehicle through which enhanced monitoring practices become unified and more transparent across the institution.” Agreeing with that view, it notes that TAIMS is expected to systematically include clear ex ante set of progress indicators and identified risk factors, to record outputs and other measurable indicators, and to provide for their monitoring.In the context of the Task Force on Performance Indicators, staff is seeking to identify different types of TA outputs for the purpose of tracking and monitoring performance.
TAIMS now generates monthly mission activity reports.
TAIMS is being extended to the IMF’s regional TA centers, and further work is ongoing on enhancing TAIMS as a monitoring tool. In particular, the Working Group on Monitoring and Evaluation has been reconvened to explore and pursue the next steps in TAIMS’s development.
Interaction with authorities and local experts
Greater involvement by the authorities and counterparts in the design of TA activities and arrangements for follow-up should be emphasized as a signal of ownership and commitment.Directors concurred that greater involvement and ownership by the recipient authorities and discussion of options are crucial to greater TA effectiveness. They welcomed the proposals for more participation by country authorities in drawing up terms of reference (TORs) while systematically seeking their tangible commitments to the contemplated TA strategy or advice early on. However, a number of Directors cautioned that these indications of commitment should not be treated as conditionality for access to TA.TA departments should further strengthen the dialogue with country authorities when drafting TORs for short- and long-term experts, and systematically seek their inputs. For enhancing the likelihood of implementation, fostering lessons to be learned, and disseminating best practices, those departments should systematically seek the authorities’ consent to disseminate TA reports among the country’s technical staff, IMF Executive Board, and selected donors. Broader circulation and availability of the TA reports could help in building consensus around proposed reforms and strengthen the authorities’ resolve to follow up on recommendations.TA departments have made efforts to enhance the dialogue with the authorities, including efforts to clearly lay out the authorities’ and TA departments’ expectations for TA operations. Actions suggested by the Task Force are being implemented by all departments. The preparation of TACSNs, in consultation with country authorities, would also contribute to the fulfillment of this objective.
Enhancing identification of options
Stronger efforts should be made by TA experts to identify options and discuss alternatives with local officials prior to drafting TA recommendations.When relevant, TA reports should record discussions with the authorities on alternative policy options, the considerations leading to them, and when there were differences of views summarize the authorities’ views on key issues and recommendations.Actions recommended by the Task Force are being implemented by TA departments. One department reports engaging with authorities in discussions of preliminary findings and alternative responses before drafting reports.
Ex post evaluations
The program of ex post evaluations of TA should be widened and more systematic procedures for disseminating lessons put in place.Directors highlighted the importance of ensuring that the scope of ex post evaluations be carefully considered and integrated into a broader strategy to make TA delivery more effective. They concurred that external evaluations are a useful tool to enhance accountability and provide a fresh perspective, and that the Office of Technical Assistance Management (OTM), in collaboration with other departments, should continue to prepare and update its program of ex post evaluations and to assess shifts in TA demands across subject areas.For larger TA projects, it is recommended that functional departments produce standardized self-assessments within three months of the conclusion of the project and include them in TAIMS. These self-assessments could be used as input to institutional lesson-learning and for ex post evaluations. To assist in this effort, it is proposed that OTM manage a TA self-assessment/evaluation knowledge base that could be incorporated into future TA and made available to all IMF staff to maximize its potential benefits.As a first step to designing a standardized database on evaluations, TA departments have listed their existing evaluation practices and completed evaluations. The IMF-wide TA evaluation program has been completed for 2006, and the respective Board papers have been issued (see www.imf.org/external/np/pp/eng/2006/071206.htm).
Strategic decisions and trade-offs
The prioritization filters should be discontinued or replaced by ones that allocation. Either would more effectively guide TA course of action involves strategic decisions on trade-offs that need to be taken explicitly.While Directors concurred that the case for discontinuing the current filters was strong, several of them observed that elements of this approach might usefully be preserved. Prioritization of TA resources should flow from a shared vision of the IMF’s overall medium-term objectives—reflecting its core competencies—while at the same time retaining the flexibility to respond to the urgent needs of members. Directors agreed that the IEO’s recommendations would entail more staff-intensive approaches to providing TA, including greater collaboration with country authorities and other TA providers. Going forward, Directors welcomed management’s proposal to ask staff to make concrete proposals on how to implement the IEO recommendations, taking into account budgetary costs, and implications for work practices and TA delivery, and looked forward to future discussion of these proposals, including on prioritization, before their implementation.It is recommended that the current set of prioritization filters—deemed ineffective—be discontinued as the proposed remedial actions encompass the key ingredients for an effective prioritization of TA resources. If fully implemented, they would form a basic accountability framework for all the stages of the TA life cycle—prioritization, delivery, and monitoring and evaluation—against which the effectiveness of IMF TA can be gauged. To make it viable, however, a number of management and organizational challenges must be overcome, while its linkage to the efficient allocation of TA resources—the budget perspective—needs to be developed. The provision of adequate incentives for staff to implement the proposals, in particular for area department staff and resident representatives, is critical.Prioritization filters were discontinued. In their place, the TA Subcommittee has recommended a closer integration of the resource allocation plans (RAPs) with the budgetary processes of both functional and area departments. To that end, and given the nexus between the RAPs, TACSNs, and TA budgets, the TA Subcommittee has decided to set up a Working Group on Technical Assistance Resource Planning. The overall remit of the working group is to make recommendations with a view to improving procedures for allocating TA resources in a collaborative manner that is consistent with the strategic objectives of the IMF and country authorities. It is expected that the working group will present its initial findings and recommendations to the TA Subcommittee by the end of October 2006.
It is important that these proposals also feed into mechanisms that result in an efficient allocation of TA resources across countries, sectors, and policy initiatives on an evolving basis. Thus, beyond the annual exercise, resource allocations for TA through the Medium-Term Budget Framework will need to be informed by strategic decisions, which in turn, will require that emerging trends across departments be identified in a timely way.

This column summarizes the reaction of the Executive Board on each recommendation as reported in the Summing Up by the Acting Chair. Although care has been taken to ensure accuracy, readers are invited to refer to the full text of the summary of the discussion, which is included in the published version of the report and can be accessed from the IEO website (www.imf.org/external/np/ieo/2005/ta/eng/013105.htm).

The Task Force was created by management on March 31, 2005, in response to the Executive Board discussion of the IEO’s evaluation. The Task Force report was discussed by the Executive Board on July 27, 2005.

The description of follow-up is intended to provide a factual indication of any additional steps taken since the Board discussion of the evaluation report. It is not intended to be an evaluation of any follow-up by management or the Executive Board. Where staff internal assessments have been made of relevant issues, these are reported but have not been subject to any independent confirmation by the IEO.

Terms of reference for resident representatives in countries that are major TA recipients should list among their responsibilities an active involvement in TA implementation, follow-up on TA activities by other providers in areas of interest to the IMF, as well as coordinating activities where those providers are receptive to this approach.

This column summarizes the reaction of the Executive Board on each recommendation as reported in the Summing Up by the Acting Chair. Although care has been taken to ensure accuracy, readers are invited to refer to the full text of the summary of the discussion, which is included in the published version of the report and can be accessed from the IEO website (www.imf.org/external/np/ieo/2005/ta/eng/013105.htm).

The Task Force was created by management on March 31, 2005, in response to the Executive Board discussion of the IEO’s evaluation. The Task Force report was discussed by the Executive Board on July 27, 2005.

The description of follow-up is intended to provide a factual indication of any additional steps taken since the Board discussion of the evaluation report. It is not intended to be an evaluation of any follow-up by management or the Executive Board. Where staff internal assessments have been made of relevant issues, these are reported but have not been subject to any independent confirmation by the IEO.

Terms of reference for resident representatives in countries that are major TA recipients should list among their responsibilities an active involvement in TA implementation, follow-up on TA activities by other providers in areas of interest to the IMF, as well as coordinating activities where those providers are receptive to this approach.

    Other Resources Citing This Publication