Technical Assistance Evaluation Program - Findings of Evaluations and Updated Program - Mid-Term Evaluation of Middle East Regional Technical Assistance Center
Author:
International Monetary Fund
Search for other papers by International Monetary Fund in
Current site
Google Scholar Close
Publication Date:
04 Apr 2008
eISBN:
9781498334730
Language:
English
Keywords:
PP; steering committee; CARTAC program; work plan; CARTAC activity; governance practice; CARTAC training programs
Download PDF (1.9 MB)

CARTAC, the second of the regional technical assistance centers, was created with singular emphasis on ownership of technical assistance by the beneficiary countries. To this end, it was structured as a UNDP project with the IMF as Executing Agency and with a Steering Committee empowered to give strategic guidance to the program and select its senior staff from short lists provided by the IMF. With the spread of the RTAC modality, the IMF has sought to bring the Centers' activities within the ambit of overall resource planning for technical assistance, ensure consistency with the institution's view on priorities for technical assistance in the countries concerned, and tighten quality control through backstopping. This has created the potential for conflict with the relative independence that CARTAC has enjoyed from its inception. The conclusion in this report, however, is that alignment with the IMF does not necessarily undermine country ownership and that the Steering Committee can play a pivotal role in defusing any tension that may arise.

Abstract

CARTAC, the second of the regional technical assistance centers, was created with singular emphasis on ownership of technical assistance by the beneficiary countries. To this end, it was structured as a UNDP project with the IMF as Executing Agency and with a Steering Committee empowered to give strategic guidance to the program and select its senior staff from short lists provided by the IMF. With the spread of the RTAC modality, the IMF has sought to bring the Centers' activities within the ambit of overall resource planning for technical assistance, ensure consistency with the institution's view on priorities for technical assistance in the countries concerned, and tighten quality control through backstopping. This has created the potential for conflict with the relative independence that CARTAC has enjoyed from its inception. The conclusion in this report, however, is that alignment with the IMF does not necessarily undermine country ownership and that the Steering Committee can play a pivotal role in defusing any tension that may arise.

2 Background

1. Towards the end of the decade of the 1990s, there was collective recognition among the small states comprising the Caribbean Community and Common Market (CARICOM) that fiscal disequilibrium and unsustainable debt would severely limit scope for adapting to the challenges arising from the economic cost of frequent natural disasters, the loss of traditional trade preferences and other structural changes underway. As a result, the authorities, through the CARICOM Council of Ministers of Finance and Planning (COFAP), approached the IMF to provide technical assistance to enable countries to develop home-grown stabilization and growth programs in the short-term, while building the capacity to undertake such activity independently in the future.

2. The Fund and COFAP agreed that the creation of a regional technical assistance center similar to a facility in operation in the Pacific area, may be an ideal modality for satisfying the regional need for technical assistance. This opportunity for functional cooperation was embraced by Caribbean authorities. At the formal launch of CARTAC in November 2001, both the Prime Minister of host country Barbados and the IMF's Managing Director emphasized the strong political desire among participating countries1 for 'ownership' of the technical assistance program.

3. CARTAC is funded by cash grants and in-kind contributions from bilateral and multilateral sources and structured as a UNDP project with the IMF as Executing Agency. Day-to-day management is in the hands of a Program Coordinator who reports semi-annually to a 14-member Steering Committee, comprising representatives of donors, regional organizations and beneficiary countries, which sets the strategic direction and approves the rolling Work Program. The Steering Committee is also responsible for final selection of the Program Coordinator and the long-term advisers from ranked short lists provided by the IMF.

4. The first phase of the CARTAC program lasted through the end of 2004 and attracted total contributions of $13.4 million. The second phase, scheduled for the period 2005 - 2007, received pledges totaling $15.4 million (see Table 9 for details). CIDA and DFID, the principal bilateral donors, recognize CARTAC as an ideal vehicle for implementing key principles of the Paris Declaration on Aid Effectiveness, such as program ownership by users, pooling of assured multi-year financing to support broad program goals, emphasis on accountability through agreed measurable results and exploiting complementarity with other technical assistance programs.

5. CARTAC's core competencies are in the areas of prudential regulation of financial institutions (FSS), public expenditure management (PEM), revenue policy andadministration (TAX), macro-economics (MAC) and related statistics (STA). These are areas in which the Executing Agency is a global leader and is, therefore, able to provide 'backstopping' from its headquarters staff to ensure that CARTAC's work meets the highest professional standards. Within these areas, specific technical assistance is delivered in response to demand by participating countries, with due regard to taking advantage of possibilities for regional approaches.

6. As emphasized in the report that proposed extension of the project to a second phase "the goal of CARTAC is to assist member countries to build technical capacity so as to effect improved macroeconomic management, thereby achieving stable macroeconomic environments for sustained growth." The document also noted the conclusion of the first mid-term review that CARTAC had successfully laid the foundation to pursue this goal, its "main strengths [being] its prompt and flexible response to requests for assistance from countries in the region; knowledge of cross-regional issues; generation of synergies between technical areas covered; its proximity to country counterparts; capacity to closely follow-up on implementation of recommendations and provide additional assistance when required; the grant nature of the resources, which are not tied to conditions; and its perceived independence from multilateral institutions and bilateral donors."2 There was a clear need, however, to articulate a logical framework to underpin the development of indicators that would enable systematic evaluation of progress towards achieving program goals. The Steering Committee appointed a subcommittee in March 2005 to work on this issue and finally agreed on a framework later in the year. Equally, there has been much discussion of the extent to which CARTAC should be proactive in shaping demand in order to encourage a more coherent and strategic use of scarce technical assistance resources.

7. The project extension report recommended a review of the program within 18 months of the start of the second triennium. The Steering Committee awarded the consultancy to two independent consultants from the region - Messrs. Osborne Nurse and Euric Bobb. This report presents their assessment of CARTAC's performance, with particular focus on results achieved to date and efficiency in programming the use of resources.

3 Mid-Term Review - Key Issues in the Review

3.1 Methodology

8. The Terms of Reference (ToR) for the Second Mid-Term Review (see Appendix 1) focuses on assessing CARTAC's contribution in addressing the region's problems and capacity constraints in each of the five functional areas of technical assistance - financial sector regulation and supervision, macro-economic programming and policy analysis, public expenditure management, revenue policy and administration, and statistics. The assessment is required to focus on three key issues:

  • a. Program efficiency, effectiveness and synergy gains;

  • b. Organizational and operational efficiency and effectiveness; and

  • c. Sustainability.

9. The ToR identifies certain specific questions to be addressed including issues such as the responsiveness to the region's needs, the extent of country ownership of the technical assistance that is being delivered, the timeliness of delivery of technical assistance, the contribution of the Centre to enhanced regional integration, and CARTAC's organizational and governance arrangements.

10. CARTAC has completed about 4.5 years of operation. While the rolling Work Program and semi-annual Activity Reports to the Steering Committee are replete with details of activities in each of the five functional areas, sufficient time has elapsed to begin searching for sustainable results from CARTAC's work. The review team has sought to define results attributable, at least in part, to CARTAC by using three approaches:

  • Analysis of the logical framework approved by the Steering Committee in 2005 (see paragraphs 38 - 44 and Table 4);

  • Discussions with authorities in countries and institutions that account for over 60 per cent of CARTAC's technical assistance effort and also with representatives of key donors and the Executing Agency (see Appendix 4 for List of Persons Interviewed); and

  • Conducting an e-mail survey covering key personnel in all participating countries. (See paragraphs 114-115 and Appendix 2).

3.1.1 The Questionnaire

11. The reviewers developed a simple questionnaire that focused on obtaining views of participants in CARTAC programs on the quality and sustainability of CARTAC's methods and results and to give an indication of future areas on which the Centre should focus. A sample of the questionnaire is attached as Appendix 2.

3.1.2 Countries Visited

12. The reviewers visited Barbados, Antigua and Barbuda and St. Kitts and Nevis as well as Washington DC, meeting and interviewing several CARTAC stakeholders in each of these locations. They also conducted interviews in Trinidad and Tobago, their home base. In addition, the reviewers sought to visit Dominica but were frustrated by airline delays. However, most planned interviews were done by telephone. See list of persons interviewed at Appendix 4.

3.2 Developing the Technical Assistance Program

13. CARTAC is designed to serve twenty (20) countries in five (5) key functional areas. Given the resource constraint, management and the Steering Committee inevitably face tough choices about the strategic content of the rolling Work Program. The first mid-term report describes how the strategic outline of the program was originally developed. Essentially, on the invitation of participating countries long- term advisers conducted diagnostic reviews to ascertain priorities. Where there was an already existing country program framework, such as Guyana which was a HIPC country with a poverty reduction strategy and on-going Poverty Reduction and Growth Facility, CARTAC fit its activities into this framework. In the OECS sub-region, the Eastern Caribbean Central Bank and the Eastern Caribbean Currency Union Monetary Council were pursuing harmonized tax reform, macro-economic programming and strengthening of the financial sector which also facilitated the identification of potential technical assistance that could be provided by CARTAC. Some program choices were influenced by decisions taken at a regional level. For example, the CARICOM Standing Committee of Caribbean Community Statisticians (SCCS) emphasized the need to update national accounts and price statistics using best practice methodology. This influenced the shape of technical assistance in the Statistics area. In other cases, CARTAC and the authorities developed priorities on an ad hoc basis for the areas covered by CARTAC's mandate.

14. The technical assistance program in the second phase built on the activities started in the first phase. Many of these clearly required a longer period of assistance to begin to show results in terms of the creation of institutional capacity and the required human resources to ensure successful operation. This thread of continuity is clearly evident from reading semi-annual Work Programs and the subsequent Activity Reports provided for the meetings of the Steering Committee. Moreover, a needs assessment conducted in preparation for the second phase showed rising interest in receiving support for debt management, pension reform, supervision of non-banks and insurance companies, and capital markets. In addition, a comparative advantage of the CARTAC vehicle is the ability to respond rapidly to changing priorities. Accordingly, the Program Coordinator with the approval of the Steering Committee is able to shift resources among functional areas depending on the urgency of requests. An outstanding example of CARTAC's responsiveness is the shift of resources during 2006 to facilitate rapid development and implementation of value added taxes in Antigua & Barbuda, Belize, Dominica and St. Vincent & the Grenadines.

15. A demand-driven system characterized by rapid response is highly valued by the beneficiary countries. However, it is constrained by the requirement of consistency with CARTAC's mandate, scope for synergy through a harmonized regional approach and the availability of resources. In 2004, the Program Coordinator discussed with the Steering Committee a system of "prioritization filters" used by CARTAC management as a tactical instrument in determining how to respond to requests. The filters are designed to screen requests at three levels:

  • Mandate - does the request fall within the core functional areas defined in the program mandate

  • Strategic objectives - considerations such as contribution to regional integration, responsiveness to country priorities, commitment to implementation, country coverage across CARTAC's membership, support for countries confronting fiscal, debt or financial crises

  • Operational - congruence with the modes of TA for which CARTAC was designed

16. First, CARTAC ensures that the request falls within its five core areas. Hence, requests relating, for example, to support for social sector statistics, generic information technology, human resource management, payment systems, etc are steered to other donors that may be offering assistance in these areas. When requests are related to a CARTAC core area, such as the application of information technology for public expenditure control, steps are taken to ensure harmonization with parallel efforts such as the CIDA-funded program for expenditure management (ECEMP) in the OECS area.

17. CARTAC has embraced regional synergy as a valuable tactic to optimize the use of its scarce resources. Within the mandate, the preference is to emphasize activities that have application across the region. CARTAC would effectively develop and pilot an activity in one jurisdiction, then expand its application, with adjustments as necessary, to others. For example, it developed a manual for on-site examination of credit unions in the Bahamas that now has been provided in electronic form to Belize, Dominica, Guyana, Montserrat and Trinidad and Tobago. Similarly, through CARTAC the Supervisor of Insurance in Barbados obtained four inspectors (funded by the Canadian Government) to work alongside and train staff in on-site inspection of insurance companies. Subsequently, CARTAC covered the expenses of three (3) inspectors from Barbados to assist colleagues in St. Lucia with the inspection of insurance companies in that country. With a clear signal of a need expressed by several jurisdictions, the first step is usually to organize a workshop of interested parties to develop a common understanding for proceeding on a cost effective regional and harmonized basis. Exploratory missions to Haiti in 2004 and Jamaica in 2006 reflect concern about achieving broader country coverage across the membership.

18. CARTAC has undertaken or assisted with diagnostic assessments in several countries as a first step towards agreeing on priorities for specific support in the future. In implementing TA it has emphasized building skills and strengthening institutions through mechanisms such as on-the-job training, workshops and attachments. As far as availability of resources is concerned, CARTAC sees itself as a purveyor of short term assistance and, hence, rarely takes on a commitment to provide assistance for longer than six (6) months. For example, despite its clear interest in improving budget management CARTAC did not accept a request from one jurisdiction for assistance in moving from cash to accrual accounting. This may have required assistance over a period of four (4) years. Nonetheless, it provided exposure to the requirements for accrual accounting by arranging contact with Cayman Islands officials who were already on this trajectory.

19. During the second phase of CARTAC, as in the first, most of the Centre's activities continue to be in the TAX and PEM areas, within which the focus has been primarily on VAT implementation in the former area and cash management systems in the latter.

3.3 Country Ownership and the IMF's Role - Shifting Balance?

20. The modality of technical assistance delivery through field-based regional centers now operates in the Pacific, the Caribbean, Africa and the Middle East. Distinct from the other centers which are IMF projects, CARTAC is a UNDP project for which the IMF is the Executing Agency. One major donor indicated to the reviewers that it pressed for this unique arrangement in order to underline that CARTAC is distinct from the IMF, to emphasize the desirability of larger and firmer funding commitments and to help focus the Centre on the poverty and gender implications of its advice. Many Caribbean governments, wary of any perception of IMF conditionality, saw CARTAC as an independent vehicle to strengthen their ownership of the technical assistance program, particularly the development of home-grown stabilization and growth plans. As pointed out earlier, this aspect was highlighted at the launch of CARTAC, by both the Fund and the host government.

21. In general, we found that Caribbean authorities claim full ownership of their programs with CARTAC. In a few instances, it was pointed out that authorities had even decided to pursue their priorities through other means rather than follow contrary indications from CARTAC. Of course, demand for particular services is often stimulated by exposure to best practice in seminars, workshops or in discussion with long-term advisers. But countries have pursued major initiatives such as the introduction of a personal income tax in Antigua and Barbuda or financial programming in various OECS countries under the SATAP arrangement, from their own enlightened decisions and not as a result of conditions imposed in exchange for financial support. However, during interviews in the region the review team on many occasions listened to expressions of concern that CARTAC may be drifting from its moorings.

22. The distinction between country ownership and country obligation is not always crystal clear. Among its prioritization filters, CARTAC emphasizes a strategic objective of preference in allocating technical assistance to countries undergoing fiscal adjustment/debt programs and financial crisis prevention. This criterion applied ideally to Dominica, a country that has benefited directly (that is, not including benefits from regional or sub-regional programs) from 29 per cent of CARTAC's activities since May 2004. On the face of it, this seems a disproportionate effort by CARTAC given Dominica's relative size, but the country has been engaged in IMF arrangements starting with a Stand-by in August 2002 and continuing with a PRGF from December 2003. CARTAC assisted in areas where Dominica had to meet structural benchmarks, such as introduction of a VAT, medium-term budgeting framework and field inspection of a credit union. For the latter activity, CARTAC even provided services of a long-term adviser and short-term consultants for field work Before this year, Jamaica, a country that relatively recently emerged from many years of IMF programs, showed little interest in requesting technical assistance from CARTAC.

23. Changes underway at the IMF may well sharpen the sense that the program is being increasingly driven from outside the region, at least among those already expressing concern. Functional departments at Fund headquarters look at RTACs through the prism of the institution's strategic view of technical assistance priorities and its overall TA resource allocation plan. Often, it is necessary to directly complement activity taking place at a regional centre. For example, CARTAC work on VAT, fiscal responsibility law and legislation to regulate money transfer services benefited from assistance from the IMF's legal department. Moreover, functional departments decline to provide quality control of TA in response to requests outside their mandate and expertise, such as social statistics and social security systems. Thus, CARTAC finds it impossible to respond to such requests. On a strictly administrative level, recent changes tighten the guidelines for hiring of short-term consultants by requiring that they be listed on the IMF's rosters and that contract data be entered ex ante in the database (TIMS) at IMF headquarters. This has a clear advantage of integrating all technical assistance costs and program content in a single database, but could be viewed as hampering response time, should the signoff on contracts take significantly longer than under previous arrangements.

4 Program Activity

24. Program activities and performance for CARTAC have been analyzed for the period May 2004 to March 2006 and therefore cover the last months of the Phase I program as well as the first fifteen months of the Phase II program. The starting date was selected in an attempt to provide coverage of the Centre's activities for most of the period after the first mid-term review.

25. During the period reviewed the Centre reported some 230 missions, 218 of which included regional or foreign consultants and 74 of which were training activities. In addition 57 persons were supported on 65 attachments for a total of 340 days, and some 1,627 persons attended CARTAC sponsored training and workshop sessions. (See Table 1).

Table 1:

CARTAC Activity Summary

article image

26. Some 60% of these activities were conducted in OECS member countries and the UK Dependencies. A further 9% were conducted directly with the Eastern Caribbean Central Bank (ECCB) or the OECS Secretariat, and 12% were classified as Regional or Sub-Regional in nature. (See Table 2).3

Table 2:

CARTAC Activity by Country and Institution

article image
Notes:
article image

27. CARTAC activity was primarily concentrated in the TAX (99 = 37%) area. FSS, PEM and STA reported similar levels of activity at roughly 20% each, while the MAC area undertook the fewest at 14 missions (5%). At the country level, Dominica (25%), Antigua and Barbuda (16%) and St. Kitts and Nevis (9%) enjoyed the largest individual shares of this activity. Taken together with the 9% attributed to the ECCB and OECS, almost 60% of CARTAC activity is accounted for in these areas.

4.1 Program Effort and Achievements

28. The data indicate that CARTAC has been quite busy conducting the considerable activity indicated above. It is important, however, to try to discern the coverage and impact of this activity in individual countries and in the region as a whole.

29. CARTAC programs may be viewed as country programs and as functional programs. The country program view allows a perspective on the broad range of activities conducted in specific countries. Dominica, Antigua and Barbuda, and St. Kitts and Nevis are the three territories with the most extensive CARTAC interventions, such interventions taking place in each of the areas of specialization. (See Table 3 and Box 1).

Table 3.

CARTAC - Principal Country Programs

article image
Source: CARTAC Reports

The Antigua and Barbuda Program

CARTAC has been assisting Antigua and Barbuda to control the fiscal imbalance that has long threatened to stall economic growth because of public debt ballooning to unsustainable levels. The authorities requested assistance with:

  • (a) Revenue policy and administration;

  • (b) Public expenditure control;

  • (c) Public Sector Investment Programming; and

  • (d) Macro-economic and financial programming

The Minister of Finance and Economy is unequivocal that the tax reform program could not have been implemented without the assistance of CARTAC. He also emphasizes that not only has CARTAC been exceptionally responsive and timely, but that unlike others it is a very good listener. With support from the Fund's Fiscal Affairs and Legal departments, CARTAC provided short-term consultants to work with the government to develop a personal income tax (PIT) and a value added tax (Antigua and Barbuda Sales Tax - ABST) from scratch. In the case of the PIT a consultant spent 3 months training staff, assisting with public outreach, and generally getting the machinery in place for a tax that was introduced in April 2005 after a lapse of almost three decades, and barely six months after CARTAC's first meeting with the Minister. The CIDA-funded Eastern Caribbean Economic Management Program (ECEMP) collaborated with CARTAC by assisting with the automation of registration and collections. CARTAC arranged attachments with the Inland Revenue Department in Barbados and sponsored visits from a tax official from Barbados to help with the collections system. Of a total of 18 staff trained under the program, only 2 have been moved to other duties. In full implementation the PIT is expected to yield $40 million annually; in the first year collections reached $30 million or about two-thirds of all taxes on income in the previous year.

The ABST will also be introduced on a fast track. It is essentially following the same trajectory as the PIT with mentoring and coaching from a short-term consultant who has been visiting for periods of three weeks since November 2005, training in audit techniques by a regional consultant and support for the implementation team on matters of registration from a staff member who works with the General Sales Tax in Jamaica. The ABST is replacing a miscellany of customs and excise duties previously administered by the Customs Department. This department is now receiving support from CARTAC in valuation, classification and audit with a view to adopting ASYCUDA ++.

Antigua and Barbuda is seeking to eliminate its chronic problem of over-spending its budget. Since November 2005, CARTAC has provided a visiting short-term consultant who is assisting with the introduction of a cash management information system that is planned to be fully operational in all departments no later than the end of 2007. Related to the general problem of expenditure control is the effort to develop a system to manage the public sector investment program (PSIP). CARTAC assisted with identifying problems and setting up a mechanism for the evaluation and screening of proposals at a technical level prior to submission to the Cabinet for consideration. The authorities express great satisfaction with the work of the CARTAC long-term adviser. Interestingly, they were unable to take advantage of opportunities for attachments that the adviser was keen to arrange, because of the small staff.

The final piece of the comprehensive economic management reform program supported by CARTAC relates to developing a capacity for analyzing the effects of different policies. Through the Structural Adjustment Technical Assistance Program (SATAP) a short-term consultant has worked with Antigua and Barbuda since 2004 for defined periods. In 2006, the consultant visited in January, March and June when the country completed work on its home-grown structural adjustment program. Slow progress in this area highlights the ever present constraint of lack of suitable counterpart staff that prejudices achieving sustainable results through technical assistance. In this case, there is only one staff member working with the consultant and available to carry forward work on a routine basis.

30. CARTAC may also be viewed as a source of regional public goods, developing programs that foster broad regional benefits and are available to all member countries to be used for their individual benefit. In this regard, almost one-quarter of all CARTAC activities have had some regional dimension. Approximately seventy-five percent of this regional activity has comprised work at the OECS and ECCB level.

4.2 Program Results and Impact

31. The most identifiable results and impact of CARTAC's work during the period under review are demonstrated in the TAX and the PEM areas, particularly in Antigua and Barbuda, St. Kitts & Nevis and Dominica - where CARTAC activities were conducted in the framework of the implementation of an IMF poverty reduction and growth facility.

32. As indicated in Table 2, the largest proportion (37%) of CARTAC's activity was conducted in the TAX area. There have been three main themes to this activity:

  • the modernization of customs administration systems, including improved utilization of the ASYCUDA software systems;

  • the review of taxation policy and improved administration of tax departments; and

  • the introduction of Value Added Tax (VAT) systems to replace revenue that is likely to be lost with the full implementation of the CARICOM Single Market and Economy (CSME).

33. Significant results have been achieved in each of these areas particularly in Antigua and Barbuda and St. Kitts & Nevis. In the latter country, tighter administration - including parallel effort by the Crown Agents paid directly by the government - resulted in a 20 per cent increase in revenue collections. In Antigua and Barbuda, the outstanding result has been the successful implementation of a new system of personal income tax in a country where there was no taxation of personal income for almost three decades. Collections from this new personal income tax have reportedly exceeded original estimates, but a sustained effort to strengthen administration will be needed to build a track record over several years. In addition, new VAT systems are in different stages of being introduced in several countries, including Antigua and Barbuda, Belize, Dominica, Grenada, Guyana and St. Vincent and the Grenadines. (See Box 2).

CARTAC's VAT Implementation Strategy

Several countries in the Eastern Caribbean face severe revenue losses from the structural decline of sugar and banana production. At the same time, the move towards full participation in the CARICOM Single Market and Economy (CSME) with its relatively low Common External Tariff (CET) regime makes it imperative for the OECS countries in particular, but also Belize and Guyana, to find alternative revenues in a widely based consumption tax. This is the background for CARTAC's focus on the development of a reasonably standardized approach to the legislative and administrative aspects of a value added tax to be adopted by countries in the region with adjustments as necessary to fit local circumstances. In the case of OECS countries, CARTAC helped in building the political support for VAT through its funding of the Tax Commission of eminent persons appointed to report to the ECCU Monetary Council. The Commission strongly endorsed the adoption of VAT as the basis of a broad-based consumption tax.

The urgency of implementing a value added tax in some cases led CARTAC advisers to adopt a very aggressive timetable with perhaps only 60 per cent of technical assistance completed before formal launch of the tax in a period of about 12 months, rather than the more measured approach of a 2-year preparatory period that would normally be recommended by fiscal experts. Belize has experienced the fastest implementation - August 2005 to July 2006 - but in that case such a system was being introduced after previous experience with a system that was cancelled on account of a number of implementation problems and loss of political support.

The speed of implementation of VAT provides an example of the balancing of interests by CARTAC between users' demands and the need to convince backstoppers that quality would not be compromised, nor reputation put at risk.

CARTAC has adopted a strategy that is quite intensive in the use of technical assistance and training resources utilizing short-term consultants to provide training and prepare publicity and public education programs. It draws on experience of practitioners in the region and depends on attachments in jurisdictions such as Barbados that already have several years of experience in operating a VAT. The functional and legal departments at Fund headquarters also play pivotal roles.

With CARTAC's assistance a value added tax was implemented in Dominica (March 2006, in keeping with the structural performance target in the PRGF) and Belize (July 2006). Antigua and Barbuda and St. Vincent and the Grenadines are expected to introduce the tax in January 2007. Work has also progressed in Grenada, Guyana and St. Lucia.

The VAT implementation program is considered the finest example of CARTAC's success in delivering relevant, practical technical assistance with a regional focus in response to a clear demand from clients that benefited from exposure to best practices in the use of this method of taxation under CARTAC's auspices.

34. In the Public Expenditure Management (PEM) area, which accounted for 19% of activity, the effort has been focused on the improvement of expenditure control systems primarily through the use of software systems - SMARTSTREAM, which has been supported by the ECEMP Program, FREE BALANCE in Antigua and Barbuda, FITRIX in St. Kitts & Nevis. CARTAC has also supported the creation of a regional association of public expenditure managers - the Caribbean Public Finance Association (CaPFA) providing support for meetings of the Association's Board of Directors and conducting training seminars in conjunction with Board Meetings. The creation of this regional association has provided the public finance managers with a forum for networking that has proven useful in supporting a program of attachments for staff of the relevant departments.

35. The FSS area where there have been two long-term advisers has accounted for almost 20% of activity. Much of the activity in this area has been focused on Dominica and on the ECCB and OECS. One focus of the FSS activity has been on the consideration and development of single regulatory authorities (SRUs) to centralize the prudential regulation of non-banking institutions such as credit unions, domestic insurance companies, money remittance operations, mortgage finance institutions and in some cases, offshore financial institutions. Single regulatory units have been established in St. Kitts & Nevis and Antigua and Barbuda and are in discussion in other countries. Considerable FSS effort has been expended in the conduct of on-site inspections of credit unions in Dominica and Montserrat, and of a development bank in Dominica. While these activities have produced the required output, their value added from a capacity building and sustainability perspective is still to be ascertained. A major exercise has also been initiated in the area of capital markets supervision with a regional effort being undertaken to develop a manual of procedures for the supervision and examination of broker- dealers and other market actors.

36. The focus for STA (19% of activity) has primarily been on the development of supply and use tables for the preparation of national accounts and the rebasing of such accounts to the year 2000. In addition, STA effort has focused on the development of price indices, including import/export price indices. The program in the statistics area has a long-term perspective and is being implemented in national statistics departments that have limited staff resources, both in quality and number, and it may be some time yet before the full impact of the effort bears fruit. Trinidad and Tobago's experience underlines the importance of working with a good staff in order to enhance the chances of achieving sustainable results. In this country, the Central Statistical Office, based on knowledge acquired from working over time with the CARTAC short-term consultant and the manual developed in that exercise, is now, on its own, well advanced in preparing supply and use tables for the 2005 national accounts estimates.

37. The MAC area initiated the lowest level of activity over the period, much of it under the SATAP Program (See Box 3) for providing assistance to the OECS countries in the development of "home-grown" fiscal and macroeconomic adjustment programs. While a few countries have completed home-grown programs, the reviewers do not see evidence of durable capacity installed to continue this work on a routine basis without significant support from short-term consultants.

CARTAC's Work with SATAP

Member countries of the Eastern Caribbean Currency Union decided at a Monetary Council meeting in 1997 to pursue benchmarks towards fiscal convergence by 2007. With the passage of time and increasing recognition of the importance and urgency of significant fiscal adjustment, they turned to CARTAC for technical assistance in building the machinery and skills in each country to develop the required macro-economic projections and home grown adjustment programs. CARTAC responded promptly and intensively by making available short-term consultants to work on regular rotational assignments with small teams of officials in the individual countries. Since April 2004, short-term consultants undertook 6 missions to ECCU countries for a total of 157 working days. Consultants served as coaches and mentors in helping officials to develop baseline scenarios, explore the impact of alternative policies and formulate systems for monitoring progress against benchmarks. In addition to on-the-job coaching, officials had the opportunity to attend various training programs on macro-economic and financial programming arranged in conjunction with the IMF Institute.

A unit was established in the Eastern Caribbean Central Bank to assist country officials in monitoring and reporting on progress. This unit benefited from special financing from DFID.

The six (6) ECCU countries have developed macro-economic programs and there is a regular system of monitoring anchored in the unit at ECCB. However, progress with building capacity varies. In Antigua and Barbuda, for example, reviewers identified only one (1) person in the Budget office functioning as a counterpart to the short-term consultant although the formal structure shows a Policy Unit in the Ministry of Finance and Economy. On the other hand, in St. Kitts and Nevis there are six (6) persons working on monitoring the program. Another source of concern, noted in the IMF's 2005 Article IV consultation report for the ECCU, is that progress at the technical level is not matched by effective engagement of policymakers in the discussion of alternative scenarios, with the notable exceptions of Antigua & Barbuda and St. Kitts & Nevis where there has been consistent engagement at the ministerial level.

38. The reviewers used the approved log frame to ascertain what results may have been achieved by the technical assistance program since its inception in late 2001. From the outset, it should be recognized that technical assistance, though valuable, is a small variable in an enterprise whose overarching goal is "improved governance in the areas of macro-economic, fiscal and monetary policies and practices as a basis for improved economic growth and poverty reduction in the region."4 As CARTAC itself recognizes, other factors such as political support for reform, the quality of the public service, natural disasters and exogenous economic events deeply influence the probability of success of the technical assistance input. However, it is imperative to make a systematic effort to see whether scarce grant resources are being used in a manner that brings sustainable benefits to the region.

39. The log frame presented in Appendix 3 states the agreed expected results in each of the functional areas of the program and traces the outputs and actual results associated with the myriad activities undertaken by CARTAC. Clearly, there will always be problems of attribution. However, in a demand driven program characterized by country ownership, the technical assistance input can be associated with the results achieved.

40. The indicator of success of the improved economic governance described in the program objective is the creation of sustainable capacity in the form of viable institutions and cadres of skilled staff, to undertake the tasks of economic management. Important as they are, inspection reports on financial institutions, draft legislation and feasibility studies, and even the formal establishment of a new unit in the public service only reach the level of outputs of technical assistance activity. CARTAC has produced many outputs, but they cannot be considered results of lasting impact unless there is follow-up action to install permanent capacity. For example, a supervision report on a weak deposit-taking company must be followed up by the institutionalization of prudential procedures and other steps to ensure a healthy on-going enterprise before we can claim results for the technical assistance activity. Under-staffed Single Regulatory Units cannot yet be considered as results.

41. CARTAC appears to have achieved, or contributed critically to, the following results so far:

  • Implementation of a VAT in Dominica and Belize;

  • Introduction of a personal income tax in Antigua and Barbuda;

  • Improved tax administration and budget management in St. Kitts and Nevis, resulting in reduction of the budget deficit;

  • Contribution to economic stabilization in Dominica; and

  • Creation of capacity in Trinidad and Tobago's Central Statistical Office to prepare national accounts in accordance with best practice.

42. The reviewers believe that these are important results, but with the exception of the Trinidad and Tobago case, and to some extent Dominica, are all in the TAX and PEM functional areas. The program has also produced many other useful outputs in these and other areas, such as the nascent macro-economic policy units in some OECS countries under the SATAP program (see Box 3), draft insurance and money services legislation, first steps for a PSIP process in Antigua and Barbuda, support for customs in several countries in the introduction of ASYCUDA++, offshore banking legislation in St. Vincent & the Grenadines and an updated and formalized inspection manual for credit union supervisors in the Bahamas. Such outputs can be turned into program results in the future with political support for building the institutional capacity. In this connection, the Steering Committee may be considered as a forum for peer review that may serve to nudge users of CARTAC technical assistance to complete required actions.5

43. The reviewers did not find baseline information about the numbers and quality of staff in the public sector offices with which CARTAC has been working. While it would be difficult, if not impossible, to measure capacity building from a baseline, it could be useful to develop a system of tracking public officials trained under CARTAC auspices, whether on-the-job, on attachment or in formal seminars and workshops, in order to judge in future whether the skills developed become the pillars of the institutions of economic governance.

44. Table 4 below summarizes the CARTAC program and key impacts and results. A more detailed analysis has been undertaken for each area of CARTAC activity and is presented in Appendix 3.

Table 4 :

Summary of CARTAC Results March 2004 - March 2006

article image

4.3 Capacity Building

45. One of the key results expected from the creation of CARTAC as a regional technical assistance centre was that its operations would contribute significantly to the building and development of capacity within the region to design, implement, operate and maintain the systems that would have been introduced through its activities. Such capacity building would take place in a number of ways, including the conduct of formal training and workshop sessions, the arrangement of professional attachments for key personnel, hands-on guidance on the job and the use of regional consultants.

46. All of the persons and institutions whom we interviewed openly praised the high quality and extent of hands-on work conducted by CARTAC consultants and long-term advisers during field missions, and its impact on the ability of their staff to understand and operate the systems that have been put in place. This is important feedback and substantially justifies the design and operational strategy of the Centre.

47. The Centre conducted 74 formal training and workshop sessions over the period under review. Thirty two percent of these - 24 - were conducted in the public expenditure management area and approximately thirty percent - 22 - in the revenue policy and administration area. The first mid-term review revealed that over the three six-month periods covered by that review, a total of 49 training activities were conducted, forty three percent - 21- in TAX and twenty nine percent - 14 - in PEM6. In short, there is an upward trend in formal training activity, but it continues to be heavily concentrated in the two public finance areas.

Table 5:

CARTAC Training Programs - Participants by Functional Area and Period

article image

48. The second approach to capacity building relates to the organization and engagement of staff in the various countries to professional attachments in other countries. In the period under review, CARTAC arranged 360 days of attachments in 65 programs in which 59 persons participated, compared to 11 attachments in the first review period. As in the first period, the TAX area accounted for the lion's share of the attachments organized, undertaking forty seven such attachments for thirty nine persons from 10 countries over 207 days. See Tables 6 and 7 below. Detailed information shows, however, that the typical attachment lasts only 4 to 5 days. While pressure of day to day work in both the receiving and sponsoring countries may be responsible for such short-stay assignments, it is doubtful that much could be learned in such limited time, even should some attachments involve repeaters.

Table 6:

Analysis of CARTAC Attachments by Functional Area and Beneficiary Country May 2004 - March 2006

article image
Source: CARTAC
Table 7:

CARTAC - Attachments by Reporting and Length of Stay

article image
Source: CARTAC
Table 8:

Composition of the CARTAC Steering Committee

article image

5 Program Management

5.1 Governance

49. CARTAC's governance practice is the key characteristic that distinguishes it from the other RTACs that the Fund supports in the Pacific, Africa and the Middle East. While all RTACs have a Steering Committee, in the case of CARTAC it has been more deeply involved in literally steering the program. Member countries, the donors, the IMF as Executing Agency, the UNDP as project manager and CARICOM are represented on the Steering Committee.

5.1.1 The Steering Committee

50. The Steering Committee currently comprises the following members:

51. The Program Coordinator acts as Secretary to the Steering Committee.

52. The Steering Committee's primary mandate is to set the strategic direction for the Center and to review and monitor its work and achievements. Consequently, the SC meets every six months usually in March/April and September/October, and reviews the work plans and accomplishments and, when necessary, decides on the appointment of the Program Coordinator and the long-term advisers.

53. As identified in the First Mid-Term Review the strong mandate that was given to the SC, reflecting in part the concern of some member countries that the IMF would use CARTAC to drive the policy agendas of the countries, does generate tensions and raise certain concerns that have to be managed.7

54. Key areas in which tension may arise are:

  • proactive use of the program by the IMF to set the direction of technical assistance and, ultimately, determine the policies of member countries that accept assistance;

  • Differing views of the SC and IMF about priorities; and

  • Staffing decisions

55. This observation has become more acutely pertinent with recent changes in the Fund's view of the work of the RTACs in the context of its overall technical assistance program.

56. In the past, CARTAC has successfully mitigated those concerns of beneficiary countries through the strength of purpose with which the SC maintains and plays its role and through its direct and immediate responses to member country demands. In the IMF's recent administrative procedures, CARTAC's work plan is required to fit into the Fund's overall technical assistance program, its selection and appointment of short-term experts is subject to Fund review and approval, and its decisions on mission objectives and timing are also subject to Fund review.

57. These arrangements have an overarching rational justification to improve the coherence of all Fund technical assistance. At face value, however, they may appear to reduce the independence that the Center has had for the first four (4) years of its life and to restrict its flexibility, if not managed effectively. The reviewers understand, nonetheless, that these recently formalized procedures should be neutral since they merely codify existing practice. It may, however, yet be too early to make definitive judgments on the effects of these changes on CARTAC's effectiveness.

58. The First Mid-Term Review also raised some issues that ought to receive focused attention from the Steering Committee and that speak to the broader question of governance of the Center.

59. The first is the question of coherence of the Center's program of work. During the first review, this was a matter of concern to some members of the Steering Committee who expressed the view that the form of reporting did not allow them to readily see how particular activities fit into the "big picture". The current reviewers share a similar concern.

60. The Center's reporting mechanism still emphasizes the detailed activities conducted by the advisers and short-term consultants. At times it is difficult to discern an overall strategy and coherence, as well as expected and achieved results and impact. The Steering Committee sought to address these concerns by encouraging the introduction of a form of logical framework and the use of prioritization filters (see paras. 15 - 18 and para. 22) as mechanisms for managing the work program. It is interesting to note that the departing long-term TAX adviser was the first to use the log frame to demonstrate, with some considerable success, the cohesiveness of the work of that area which has achieved most of the apparently sustainable results associated so far with CARTAC's work (see para. 40). Other areas have not undertaken as comprehensive a structuring of their work and should do so as quickly as possible.

61. It is especially critical that attention be paid to the coherence and priorities of the Center's program at this time when the level of funding that was originally expected is unlikely to be met and in the future where other considerations may affect the ability of the Center to obtain adequate funding for a third phase.

5.1.2 The IMF as Executing Agency

62. From a technical point of view, the IMF has continued to do an excellent job of selecting potential long-term advisers, assuring the high technical quality of the Center's work, offering access to key headquarters resources and in providing administrative support to the Center.

63. The challenge for the Fund is that of finding ways in which to continue to provide the technical and administrative support while ensuring that it does not trample upon or destroy the elements of the Center that distinguish it from the other RTACs and that contribute to the very high level of user satisfaction that has been a feature of the Center since its inception.

64. The challenge for the Steering Committee and the user members is that they must recognize that the Center is inevitably a part of the Fund's technical assistance delivery strategy, that it uses Fund headquarters resources and that it poses some potential for reputational risk for the Fund. In these circumstances, the Fund will naturally want to exercise considerable control or influence over the direction that the Center takes, the priorities that it pursues and the quality of its work.

65. The Steering Committee and the Fund will need to devote more attention to the effective management of these issues.

5.1.3 The UNDP's Role

66. Since the inception of the Center, the UNDP's role has been somewhat narrower than had been initially envisaged, since it was mutually agreed that many specific projectsupport functions fell properly to the Executing Agency8. In practice, UNDP has been mainly responsible for negotiating donor agreements, receiving and managing contributions, making disbursements to the IMF as required, and validating the budget and the accounts for the Program. This narrower role was formalized in the Project Extension Document.9 UNDP is paid a fee of 5 per cent for these functions which are carried out by the field office in Barbados and the Regional Bureau for Latin America and the Caribbean in New York.

67. There is an increasingly widely held view among stakeholders that the UNDP does not add the expected value to the governance and financial management of the Center. This is demonstrated in the current circumstances of potential funding shortfall where the Program Coordinator and the IMF's Office of Technical Assistance Management (OTM) rather than the UNDP devote considerable time to finding solutions. Similarly, there have been inordinate delays in submitting final budget figures to the IMF that, hopefully, may be avoided in future now that the UNDP's financial software is said to be compatible with the IMF's and in view of half-yearly consultations it has initiated with CARTAC.

5.1.4 CARTAC as a Legal Entity

68. The ToR requires reviewers to consider whether CARTAC should be converted to a legal entity, should the program enter a third phase. This issue seems to have surfaced primarily because of some of the inefficiency experienced with financial management of the program and an expectation that it may be easier to deal with some donors, particularly multilateral donors.

69. The principal reaction of multilateral donors during interviews was difficulty in reconciling the permanence of legal status with the view that such a TA program should have a definite sunset provision. In the case of the IDB, Charter restrictions require it to channel resources to OECS countries through the CDB. Similarly, under the European Commission's policies funds for regional programs are managed through CARIFORUM10 and neither of the major bilateral donors seem to have difficulty dealing with CARTAC as currently structured. Further, the reviewers have seen no evidence to suggest that there is consideration of spinning off other RTACs which are Fund projects.

70. The primary advantage of investing CARTAC with a legal personality would be the potential for earning income as an implementing agency for projects funded by others within its mandate. For example, the IDB has operations in Barbados (Modernization of Customs, Excise, VAT; Modernization of the Barbados Statistical Service), Dominican Republic (financial sector reform), Guyana (Value Added Tax) and Suriname (Modernization of Tax Administration) that fall within the core mandate of CARTAC where there could well be opportunities for collaboration in a way that generates revenues (or at least reimbursement). On the downside, however, there will be uncertainty in projecting such revenue and, more importantly, too much time may be devoted to countries where there is income earning opportunity to the detriment of others.

5.2 Backstopping by the Fund

71. As part of its systems for establishing quality assurance and protecting the Fund against the potential for reputational risk posed by the Center, the Fund operates what is called a system of "backstopping". This has been explained in detail in the guidelines on administrative procedures recently issued to the Center.

72. Essentially, backstopping involves the Fund ensuring that the work of the Center is integrated into the Fund's technical assistance strategy, meets its standards for quality and is not undertaken in areas outside of the core competences of the Fund. The process involves the designation of a headquarters "Backstopper" selected from the relevant core area for each of the long-term advisers. The LTAs then work with this person, submitting their own back-to-office reports and draft reports of short-term experts within their areas for review and comment, and obtaining clearance for missions and approval of short-term consultants.

73. Again, while the administrative logic of this system is impeccable, the issues are quite subtle as discussed above and the Fund and the Steering Committee have to work in the most transparent manner to ensure that the original intent and design of the Center is not compromised.

5.3 Management and Administration Procedures

74. CARTAC is managed on a day-to-day basis by a Program Coordinator (PC) who is normally recruited from the IMF and whose expenses are met as part of the Fund's inkind contribution as Executing Agency. Until March 2006, the program was administered fully by the PC. Based on feedback from users, donors in the region and the Fund, the PC developed the work program and resource allocation for the approval of the Standing Committee and then implemented the approved program directly, including agreeing and approving contracts for, and expenditures relating to, short term consultants.

75. The performance of these functions by the Coordinator facilitated and contributed to the Center's ability to respond to requests by user countries in an extremely prompt andefficient manner. Consequently, the First Mid-Term Review confirmed that an important characteristic of the Center was precisely its ability to respond quickly to requests.11 During the current review, users unanimously praised CARTAC for its prompt service that sets it apart from other providers of technical assistance in the region.

5.3.1 Current Administrative Procedures

76. In March 2006, the administrative procedures applicable at CARTAC were changed to ensure that its work program, delivery and use of resources were fully coordinated with the IMF's own technical assistance program and in accordance with the procedures utilized by the Fund in the management of other Regional Technical Assistance Centers (RTACs) in the Pacific, Africa and the Middle East.

77. The new approach arises from general principles relating to RTAC operations discussed by the Executive Board in its Review of the Fund's Regional Technical Assistance Centers. These principles are:

  • RTAC activities are complementary to other forms of Fund Technical Assistance (TA) and are therefore an integral component of the Fund's overall TA program.

  • RTAC activities should be more closely integrated with the Fund's TA program and appropriate quality control and accountability for all TA activities delivered by the RTACs should be ensured.

  • Regarding the Fund's internal organizational structure relating to the RTACs, area departments should have a strategic role in defining the overall TA priorities of the RTACs, and functional departments should be responsible for the technical aspects of the Centers' work. At the same time, it is important to find an appropriate balance that preserves the advantages of the RTAC delivery modality while ensuring the quality of TA.

78. On this basis, the Board decided to fully integrate the RTACs, including CARTAC, into the Fund's TA program. This means that the Program Coordinator must now communicate with Fund headquarters for all administrative and operational issues, in particular:

  • formulation of the RTAC work plan (and any subsequent revisions or updates);

  • recruitment of short-term experts; and

  • administrative matters related to long- term advisers (LTA).

79. In addition, the technical backstopping of all mission-related work of the RTACs is provided by the relevant functional divisions, including the evaluation of consultants recruited by the RTACs. For each topical area, a coordinator is assigned to cover each RTAC, as appropriate. The RTAC LTAs send all mission-related documents, prepared in accordance with guidelines and procedures followed at the respective headquarters departments, to the designated coordinators.

80. The LTAs are responsible for entering mission information in the Fund's Technical Assistance Information Management System (TIMS), and for maintaining the requirements of this database. In addition, mission travel and expenditure is required to be entered in the travel information management system and approved by the relevant headquarters department.

81. These changes in the administration of CARTAC are set out in specific instructions given to CARTAC in March 2006 and mean that:

  • all short-term consultant missions entered at the TA Centre's will be for persons included in the relevant functional department's roster of experts;

  • missions entered by RTAC Office Managers will be approved by the corresponding functional departments;

  • all Proforma contracts entered by the RTAC Office Managers will be approved by the corresponding functional departments. The short-term technical assistance consultants are selected and hired by the functional departments, a practice which is consistent with the functional department's core responsibility for vetting experts and maintaining a high quality expert roster. Functional departments may decide to delegate to the RTACs the selection and hiring of short-term consultants.

  • In establishing these procedures, Fund Directors "acknowledged that the RTAC model carried inherent tensions between Fund control over TA priorities and delivery modalities, and countries' ownership and donor interests, and that it was important to find an appropriate balance that preserved the advantages of the RTAC model while ensuring proper accountability and quality control of the TA delivered". The Board review suggested that, in making the necessary adjustments to face the underlying challenges of the RTAC model, it would be important to preserve existing effective practices, and to avoid rigid one-size-fits-all solutions.

82. The Note goes on to say that "the guidance provided in this note applies to the Fund's existing RTACs,12 except for CARTAC, where noted. It also applies to any new RTACs that might be established in the future. This note recognizes that the CARTAC Steering Committee has adopted, in some cases, practices that are different from those of the other RTACs. This reflects the fact that CARTAC is a UNDP project that was established at the initiative of the CARICOM Council of Ministers of Finance and Planning (COFAP) at their meeting in September 199913. Thus, from the beginning, CARTAC's Steering Committee was assigned a prominent, hands-on role in its governance. In contrast, the other RTACs were established at the initiative of the Fund and have the character of Fund technical assistance projects".

83. These new procedures make little operational distinction for the status of CARTAC, except for recognizing the role of the Steering Committee in the recruitment of the Project Coordinator and the long-term advisers. The procedures also clearly define CARTAC activity, as well as the activity of all RTACs, as important parts of the Fund's TA delivery process with important implications, in the case of CARTAC, for ownership, for the operations of the Centre and for its funding.

5.4 Funding and Expenditure Allocation

84. The second phase of CARTAC began in 2005 and is expected to expire in 2007 with estimated total funding of US$ 15.4 million, inclusive of in-kind contributions of $3.7 million, and initial expenditure budget of $14.2 million. As a result of a number of disappointments, including the cancellation of some $1.0 million of USAID contributions and the likelihood that the IDB will not fully meet its commitment of $650,000, the budgets for the project have had to be revised. As of August 2006, the revised budgets indicate pledged cash contributions of $10.5 million and cash carried over from Phase I of $2.6 million yielding a total of $13.1 million available to meet direct project costs. This is in addition to the in-kind contributions of $3.7 million which have remained unchanged.

85. Revised direct project expenditures are now $13.7 million, inclusive of overhead fees due to the UNDP and the IMF of 5% and 10%, respectively.

5.4.1 Contributions

86. The total resources that were pledged for the second phase of CARTAC for the period 2005 - 2007 included $11.7 million of cash pledges in addition to $3.7 million of in-kind contributions. With the cancellation of the USAID pledge and the likelihood that IDB support may not exceed $200,000, it is now estimated that total cash pledges will amount to $11.4 million, of which $10.5 million is currently confirmed. In addition to these pledges, the project has access to some $2.6 million of unspent balances from the Phase I project for a total of $13.1 million being confirmed as available for direct project spending.

87. This sum compares with the planned work program for the three year period which was originally estimated to cost $14.2 million and has since been revised in August 2006 to $13.7 million. This yields a projected shortfall of some $652,000 should the pledges from the IADB, USAID and the EU not materialize. However, in the event that these pledges are fulfilled, the program will have a very small carry over of some $262,000 at its completion in 2007. Table 9 below summarizes the current status of cash pledges and payments as at August 2006.

Table 9.

CARTAC Estimated Funding - Phase II 2005 - 2007 (US Dollars)

article image

88. Although the concerns about a serious financial crunch that occupied the attention of the Steering Committee at its last meeting now appear to have receded significantly, the projected financial outturn still has significant implications for the structure and execution of the planned program. Firstly, there is the likelihood that CARTAC will not have spare capacity to respond quickly to any bunching of unforeseen demand, without reducing the planned program. Secondly, as the Chairman of the Steering Committee points out, an insignificant carry over will be likely to seriously affect the transition to a third phase, if one is approved, as there is always a time lag between commitment and disbursement by donors.

89. With the recent revision of the budget, and the consequent removal of the high level of anxiety about the Centre's financial position, it may be possible for the Project Coordinator to spend comparatively less time on this issue.

5.4.2 Budgetary Allocations

90. The revised CARTAC budget for the three year duration of the project calls for the expenditure of $13.76 million, inclusive of overhead project management fees. This includes $3.9 million in project staff and regional travel, $5.7 million in short term consultant expenses and $2.2 million in training.

Table 10:

CARTAC - Estimated Program Expenditure 2005 - 2007 (US Dollars)

article image
Source: CARTAC Revised Budget August 2006

91. Over the life of the project therefore, some 80% of total expenditures, including 33% on long term staff and 47% on short term experts, will be allocated to project personnel and short-term experts and 18% to training expenses. (Table 11)

Table 11:

CARTAC Phase II - Budget Allocation

article image

92. Annual project expenditure is expected to peak at $5.0 million in 2006, some fifty per cent higher than the peak expenditure of $3.4 million in 2003 during Phase I. (See Table 12).

Table 12 :

CARTAC Actual and Planned Program Expenditure 2001 -2007 (US Dollars)

article image
Source: CARTAC Revised Budget August 2006

93. In the context of the avowed objective of CARTAC to build sustainable capacity within the region, the pattern of expenditure on training, including attachments and support for seminar participants may be examined to determine the relative importance of this item in the overall budget of the Centre.

94. Annual expenditure on training has increased from $715,000 in 2004 - the year in which the Centre apparently began placing heavy stress on workshops, to $809,000 projected for 2007, after a significant decline in 2005. By 2007, training, which accounted for 29% of expenditure in 2004, will only account for 19%. In practice, this apparent decline in relative importance may be less significant that it appears since much of the technical assistance effort provided by short term consultants includes significant inputs of on-the-job training that is not accounted for separately. However, while expenditure on training is projected to total $2.7 million over the period 2005 -2007, only $300,000 of this is to be spent on attachments, which has also received very high marks from clients as an effective method for developing the required capacity.

The Centre may wish to consider subjecting its proposed allocation of training resources to a further review.

95. The priorities reflected in the planned allocation of the Center's resources over the project period 2005 to 2007 may be discerned by reviewing the relative projected expenditures in each of the core areas of focus.

96. The plan calls for approximately 33% of total expenditure on short-term consultants to be allocated to the TAX area with another 24% in the PEM area, 18% in MAC, 16% in FSS and 10% in STA. Our analysis of results and impact suggests that the most promising areas for yielding results are in the TAX and PEM areas, and that in both the MAC and STA areas likely results have a much longer gestation period. Moreover, the allocation of 16% of resources to the FSS area still does not appear to justify the allocation of two long term advisers, even if the work program does include important issues related to CSME, cross border financial flows and the implications of conglomeration for financial supervision. In this regard, the Center may wish to consider subjecting its planned program for the rest of the term of the project, to further review and consideration.

Table 13:

CARTAC - Planned Allocation of Short Term Expert Expenditure

article image

5.5 Utilization of Consultants

97. The Center's approach to the utilization of consultants is related as well to its strategy for achieving sustainable capacity building results.

98. The resources available to CARTAC to meet the demand by member countries for development and support include the long-term advisers, foreign and regional short-term experts, and to some degree, additional mission resources that may be provided by the Fund from time to time.

99. The focus of this analysis will necessarily be on the utilization of foreign and regional consultants, since long-term advisers are occupied full time at the Centre and their direct mission work as well as their management and supervision of short-term experts are fully covered.

100. However, the utilization of short-term experts potentially contributes significantly to the building of capacity from a number of perspectives. Firstly, to the extent that the STEs work directly with the target staff and institutions in their missions, there is a degree of training, development and transfer of technology that takes place to the benefit of the staff members involved and of the institutions. Secondly, the use of regional STEs addresses the question of capacity building from a wider perspective. Many of the countries that are being served by CARTAC are extremely small and their institutions are not generously staffed. Moreover, there is a significant degree of movement among staff, resulting in the loss of trained personnel from one department to another and sometimes outside of the public sector and outside of the country altogether. Since critical mass of skilled resources will be difficult to achieve in each of these small countries, an important strategic approach to developing critical mass in the region would come from a focus on the use of regional consultants to whom governments may naturally turn in the future when a donor-funded provider such as CARTAC is no longer present in the region.

101. As indicated above, the CARTAC Phase II project estimates that expenditure on STEs would total some $5.7 million over the project period at a rate of roughly $1.9 million a year, with 2006 showing the highest allocation of $2.8 million.

5.5.1 Foreign Consultants

102. For the period May 2004 to March 2006, CARTAC utilized the services of 109 individual foreign experts14 for a total of 2,479 consulting days, including 2,218 days of technical assistance, 66 days in studies and 194 days in training activities.

103. Included among the register of foreign consultants are several from government institutions in the US, Canada and elsewhere who were not paid professional fees. There were approximately 28 such missions providing 225 days of services, of which 83 were technical assistance, 50 in legislative drafting and the remainder in training.

Table 14:

Foreign Consultants Usage May 2004 - March 2006

article image

5.5.2 Regional Consultants

104. The utilization of regional consultants has been significantly less than that of foreign consultants. Over the period May 2004 to March 2006, CARTAC utilized the services of 82 regional consultants in 88 missions for a period of 1,289 days of effort or about 34 percent of total consulting days provided to users. Of these 88 missions, 42 utilizing 192 days were missions in which no fees were paid, since the experts used were primarily public servants on leave from substantive positions.

105. Regional expert utilization has been smallest in the FSS and MAC areas and is most remarkable when compared to the use of foreign experts in these areas. In the year from April 2005 to March 2006, the FSS area utilized the services of 22 foreign experts in 28 missions covering 531 days. In the same period it used only 5 regional consultants in 5 missions for 40 days.

106. The MAC area utilized the fewest number of consultants of all the areas of specialization, utilizing only 1 regional consultant for 9 days and 10 foreign experts in 17 missions over 459 days for the entire period May 2004 to March 2006. PEM and STA also used relatively small numbers of both regional and foreign experts. PEM utilized 29 regional experts for 85 days and 12 foreign experts for 107 days while STA utilized 6 regional experts for 99 days and 23 foreign experts for 313 days over the entire period under review.

107. The TAX area used the greatest number of experts and consulting days in the period under review, utilizing a total of 40 regional consultants in 44 missions covering 1,051 days as well as 25 foreign experts in 30 missions covering 841 days, demonstrating more balance and greater readiness to tap regional skills than is the case in the other areas.

108. The general picture that emerges is that CARTAC has not taken full advantage of the opportunities that may be available in the region to achieve the highest possible level of capacity building through the utilization of regional consultants15. While such regional resources may indeed be scarce, the lack of readily available rosters at the Center of both the currently available resources and the potential for such resources in the future suggests that this aspect has not received the fullest possible attention.

109. With recent changes in administrative procedures, it is now a requirement that consultants engaged by CARTAC be entered on the IMF roster before they could be used by the Center. It is not clear what formal procedures have been established for inclusion in the roster, but achieving such registration in the past has eluded some of the more experienced regional consultants. To the extent that this pattern continues the relative imbalance in the use of foreign and regional experts is likely to continue and an important avenue and element of capacity building will have been insufficiently explored.

Table 15

Regional Consultant Usage May 2004 - March 2006

article image

6 Findings

6.1 Sustainability

110. The training and retention of cadres of professionals across the countries served by CARTAC will be a key test of sustainability of the technical assistance provided under the program. The reviewers studied the files containing feedback from participants in the various seminars and workshops organized by CARTAC. The response of participants was very positive, with virtually every activity scoring 4+ on a 5-point scale. CARTAC does not maintain contact with participants in a systematic way, so there are no tracer studies that would enable an assessment of the extent to which classroom training has translated into better job performance. The reviewers did, however, find anecdotal and impressionistic evidence that on-the-job training was probably more effective than seminars and workshops. In most agencies across all the countries visited staff praised the approach of short-term consultants who worked alongside and guided them for periods of several weeks, sometimes with follow-up visits to bolster the application of new techniques. We also got very positive feedback about the efficacy of attachments to partner agencies in the region and the tremendous value of working visits by regional practitioners familiar with the constraints of working in the public sector in the region.

111. CARTAC's management recognizes the value of attachments and peer exchanges as a key to capacity building in the region on a sustainable basis. In fact, this approach is said to have evolved from an intuitive to an explicit model of capacity building. Professionals in the region consider the support given to professional associations such as the Caribbean Public Finance Association (CaPFA), Caribbean Tax Association (CATA), Caribbean Organization of Tax Administrators (COTA) and the Caribbean Association of Insurance regulators (CAIR) as a key to building networks and sustaining exchanges and development of professional expertise in the region. It is noteworthy, however, that the use of "attachments" as a tool for building capacity was heavily skewed towards the revenue policy and administration area which accounted for 66 per cent of the number of persons and 58 per cent of number of days of attachment under CARTAC auspices from September 2003 to March 2006. This approach should not simply reflect preferences of individual long-term advisers, but should really be embraced as CARTAC's own model, generalized across all areas of assistance and used more intensively by the program.

112. Building viable institutions and adequate human resources is a long-term task, especially in small countries characterized by administrative diseconomies and small pools of professionals. While there are initial results from CARTAC technical assistance (see Boxes on Antigua and Barbuda, VAT) there is much more to be done to achieve CARTAC's overarching goal of "improved governance in the area of macro-economic, fiscal, and monetary policies and practices as a basis for improved economic growth and poverty reduction in the region." This points to the need for examining CARTAC's financing as a dimension of sustainability.

113. Experience with the second phase strongly suggests that CARTAC has to diversify its sources of funding and adopt ground rules that ensure predictability of funding. The reviewers gleaned from discussions with World Bank and IDB staff that their institutions shy away from financing successive phases of a TA program and that any appetite to contribute to yet another phase of this program could be affected to the extent that there is any flavor of supporting the Fund's technical assistance budget. In the case of bilaterals, there is a strong case for applying the principles of the Paris Declaration on Donor Harmonization to ensure pooling and predictability of resources. In the wider context of economic integration in the Caribbean region, CARTAC may be viewed as a regional public good deserving of substantial contributions from better-off countries in the region. CARTAC itself should also be more proactive in seeking opportunities to be involved where financing is available under other programs in support of activities that fall within its mandate, such as its work in 2004 on economic governance in Haiti on a grant from the World Bank in its Low Income Countries Under Stress (LICUS) program.

6.1.1 Views of CARTAC Clients - Questionnaire

114. An e-mail questionnaire was sent to 78 persons in 22 countries and the ECCB derived from a list provided by CARTAC. Responses could have been returned by e-mail or fax. During interviews, the reviewers also indicated that it would be very helpful to get responses to the questionnaire in order to have a systematic basis for testing some of the face-to-face responses to questions. The reviewers received only 23 acknowledgements to the questionnaire along with several indications that the list was not up to date.16 Of the 23 acknowledgements received, 18 persons from 9 countries completed the questionnaire.

115. Given the very positive views of CARTAC's work and the emphasis on ownership that we heard during interviews, the level of response to the questionnaire was very disappointing. Although such responses as we received endorsed the overall favorable impression of the program, we do not see any merit in detailed analysis of the responses received since no valid conclusions can be drawn from the exercise.

6.2 Ownership and Governance

116. In general, Caribbean authorities express satisfaction with and claim full ownership of the technical assistance programs undertaken by CARTAC, despite some concerns that the IMF may be incrementally increasing its influence on the direction of the Center's work through the backstopping procedure and the new administrative regime. On the other hand, the Steering Committee (SC) has continued to exert sufficient authority and influence so that none of the beneficiary countries that the reviewers contacted feels fundamentally fearful of loss of control and ownership.

117. As designed, and as has been the case from the very inception of the Center, the Steering Committee has been doing an excellent job of governance for the Center, making sure that it has remained on mission, that work is of the required quality and driven by the demands of the member countries and that the advisers and short-term consultants used by the Center are of high quality.

118. The Steering Committee has, however, been seeking a better understanding of some strategic issues such as the direction of the Center's work, the results achieved, the impact and sustainability of such results, and the building of capacity in the region. The reviewers consider that while the necessary data is provided to the Steering Committee at its six-monthly meetings, the data has not routinely been submitted as management summaries and analyses that would allow the SC to easily monitor these strategic issues. The reviewers believe that reporting to the SC should increasingly emphasise the analysis and extraction of trends, patterns, results and impact of the work. This would facilitate a critical process of continuous monitoring by the SC and supplement the program evaluation undertaken in mid- term reviews.

119. In respect of the concern that the IMF may be slowly increasing its influence on the Center, the SC must be aware of the context in which the Fund is operating and of the pros and cons of its more hands-on management of the technical assistance that is being provided through CARTAC. There are many management and quality control "pros" in favor of the IMF approach. The only "con" relates to the original design emphasis on regional ownership. In this regard, the SC has to work with user countries to mediate fear of IMF influence that partly informs the concern about potential loss of ownership. The nature and quality of the CARTAC interventions that are so well received derive precisely from the attention that the Fund pays to the quality of the Center's work. Moreover, it would help to assure users that the March 2006 Operational Guideline merely formalizes established practice and that senior IMF management has confirmed in writing the Fund's strong commitment to protecting the demand-driven character of CARTAC (country ownership) and the established role of the SC.

6.3 Funding and related issues

120. The Center has experienced a variety of circumstances with respect to the process of receiving pledges and collecting on commitments. On one extreme, USAID unexpectedly cancelled its commitment of US$1.2 million early in 2006. At the other, CIDA has made its contribution in full and at the beginning of the Program period. Between these extremes lie a number of different circumstances - the IDB being unlikely to honor more than half of its commitment of $650,000, the World Bank needing more than a year to work out the details of the modalities of meeting its commitment and a level of uncertainty as to how the commitment of the European Union would in fact be made and channeled to the Center. In these latter cases there has been no indication of unwillingness to support the Center, but the practical issues of providing the funds lead to concerns about cash flows.

121. In this regard, and anticipating that similar funding issues will always be a feature of efforts such as CARTAC, the Center needs to be able to exercise even more flexibility than it has done to facilitate the execution of a balanced program. This, for example, raises the question of whether there is a continuing need for two long term FSS advisers at this time, and estimating the period for which the elevated demand for resources in the TAX area is projected to continue.

122. A related issue is whether there continues to be strong value added reasons for maintaining the current role of the UNDP. In other RTACs, the IMF is responsible for many of the functions undertaken by the UNDP for CARTAC. However, there is no obvious substitute for the UNDP umbrella that protects the regional sense of ownership. We think that CARTAC should continue as a UNDP project and that in addition to steps already taken to improve efficiency, UNDP should take the lead in stimulating donor support for continued financing of the program.

6.4 Management

123. The reviewers have found that the Centre has maintained its reputation for quick and prompt response despite the recent introduction of new administrative procedures which appear to have added a few days to the process of recruiting short term consultants and in approving mission travel. During the initial period immediately following the change in procedures, it appears that recruitment and mission approvals took somewhat longer. However, owing to the pragmatic approach of OTM and the functional departments the time taken by these procedures has settled to an acceptable period consistent with the Center's reputation for quick response.

124. While the Center has developed its well-earned reputation for speedy and appropriate response, the reviewers have found that in focusing on delivery it has not given due attention to issues of management reporting. For example, while the Coordinator provides the Steering Committee with fairly comprehensive and detailed reports on the activities of the Center at each of the six-monthly Steering Committee meetings, these reports pay less attention to deriving inferences, patterns, directions and summaries from the reported activity. Thus in the April 2006 meeting when the Steering Committee Chairman raised questions about the utilization of regional consultants, there appeared to be no analyses presented from within the detailed data that was made available to the Committee.

125. Similarly, there appeared to be insufficient focus on the overall strategy followed by the Center, attention being directed more on the immediate demands in each of the five areas of specialization and support. As a consequence, it was not immediately clear what impact the demand for substantial additional resources in the TAX area may be having on the Center's - and the long-term advisers' - goals in each of the other four areas.

126. Finally, the reviewers could not readily find evidence of the existence of rosters of short-term consultants - both foreign and regional - in each of the areas, nor could they find similar registers of key Center contacts in each country17.

127. These issues have been raised as management issues which are capable of relatively simple solution and which would contribute significantly to the future effectiveness of the Center.18

6.5 Program Efficiency, Effectiveness and Synergy Gains

128. The efficiency and effectiveness of the Center may be evaluated in several dimensions, including technical, administrative, financial, impact and sustainability and synergy with other regional programs. Each of these will be examined to provide a basis for an overall assessment of the Center.

6.5.1 Technical Considerations

129. The technical design, execution and focus of the Center have been excellent. Program execution has received high marks for the technical competence of both the long-term advisers and the short-term experts.

130. The reviewers are satisfied that the element of ownership of the Center's programs by the member countries has been very well maintained, even considering that CARTAC has had to play a role in bringing to the attention of the authorities new possibilities and best practices in several areas.

131. In this regard, the Center's support of the Panel of Experts on Taxation systems in the OECS countries and conduct of the Survey of Caribbean Tax Systems played a very important role in helping to define and guide the Center's excellent program in the area of revenue policy and administration and to some extent in the public expenditure management area. It is no surprise, therefore, that these two areas emerge as the most effective in CARTAC's work program.

132. The Center has maintained as well its clear focus on its mandate and the technical capability that it can bring to bear on its program, leading at times to disappointment that quite critical areas - such as social statistics and pension reform - are not being covered in the work program.

133. In all technical areas, the Center has performed extremely well.

6.5.2 Administrative Efficiency

134. The administration of the Center's work program and the execution of its missions have also been excellent. Reviews on the responsiveness and efficiency of the organization have been uniformly good.

135. The only concern for the reviewers arises in respect of the effectiveness with which the prioritization filters have been utilized and the implementation of the agreed methodology of work programming and monitoring. The proposed model for programming and monitoring has been tested by application to the revenue policy and administration area, but has not apparently been utilized by other areas to date.

136. The importance of the use of the prioritization filters is emphasized in the current situation of concern for the cash flow and overall funding of the Center. The reviewers believe that an effective strategy of setting priorities would allow the Center to define the expected length of the program in the TAX area, the resources that are to be devoted to it and the highest priorities in the other areas that must be undertaken simultaneously. Such an approach would also, in our view, lead to the conclusion that there may be no need to hire a second LTA in the FSS area after the imminent expiration of the contract of one of the advisers. Similarly, fundamental questions surface about the contributions of the position of long-term adviser in the MAC area.

6.5.3 Financial

137. The principal concern that arises in the financial area is that of ensuring collection of the cash pledges that have been made. Although in the view of the reviewers there does not appear to be a serious cash flow problem for this year, the 2007 program and transition to a potential phase three are likely to be substantially affected if all outstanding commitments are not collected. A related concern is the difficulty and delays experienced in 2004 and 2005 among CARTAC, IMF and UNDP to have final budgets authorized.

138. A second concern in this area is primarily in relation to the financial model that may be followed should the program receive a further extension. The reviewers strongly recommend that the funding model that has been observed by DFID and CIDA, which is consistent with the principles of the Paris Declaration on Aid Effectiveness become the model of choice for CARTAC and that all bilateral donors commit to work within this framework of pooling assured multi-year financing attuned to the broad goals of the program.

6.5.4 Impact and Sustainability

139. A significant part of the interventions initiated under the CARTAC program has such long maturity horizons that it remains difficult at this time to be specific and clearly identify their impact and sustainability after almost five years of the Center's operations. There are a number of reasons for this.

140. Firstly, much of CARTAC's work requires specific decisions by member governments to adopt policy recommendations and endorse and pass the legislative changes that are required in many instances. Where governments have been prompt to adopt the relevant policy positions - the implementation of VAT systems and certain other tax reforms are the clearest examples - the impact of the Center's work is clearly visible. In addition, the implementation methodology adopted by the Center, involving hands-on work by both long and short-term experts, enhances the probability that many of the changes will be sustainable. In responses obtained from the interviews there is a strong sense among member countries that the Center's work has led to sustainable changes in staff skills and experience and in some cases in organization structures.

141. The impact and sustainability of the Center's work is also negatively affected by the scarcity of resources, both in size and skill, available to member governments. This scarcity is exacerbated by the incentives that are available to trained persons to move to more lucrative jobs and careers in the private sector and outside of the region.

142. In recognition of this latter fact, the reviewers are concerned that CARTAC has not given adequate attention to the strategic role that may be played by regional consultants. A strategy to aggressively use regional consulting expertise will go a long way towards broadening the readily available resource base in the region, particularly for the smaller countries. We believe that the Center should, in the last stages of the current phase, implement such a strategy and ensure the fullest possible use of regional expertise in all of its areas of concentration.

6.5.5 Synergy

143. The ability of the Center to work in conjunction with other agencies has been another area of strength and source of its excellent reputation. CARTAC has worked with ECEMP, the ECCU and the ECCB, CARICOM and other agencies, fitting into the programs of such agencies and in some cases providing the critical resources to help such programs achieve their objectives.

6.5.6 Overall Evaluation

144. Our assessment of the overall performance, effectiveness and impact of CARTAC is that it has been very good. There are indeed certain areas that we have mentioned in the analysis to which attention should be paid to ensure improved outcomes and sustainable results for the Center. One key such area is the development of a focus on results and impact more so than on activity. This would require more strategic thinking and a revision of the approach to reporting that is currently in use.

145. With appropriate changes in the areas that have been identified, CARTAC is well on the way to completing its already substantial justification of enhancing the region's capacity for solid economic governance.

7 Recommendations

146. We recommend that consideration be given to continuing the program into a third 3-year phase. CARTAC is a highly valued partner of countries in the Caribbean seeking to enhance capacity for sound economic management. Users stress its closeness, readiness to listen, rapidity of response and practical advice as key characteristics that have enabled this program of technical assistance to gain high acceptance and to be embraced as a key promoter of the interests of countries striving to cope with structural changes. Beneficiaries believe that there is more useful work for CARTAC to do.

147. Financing in phase 3, particularly from bilateral donors, should be invited on the basis of the principles of the Paris Declaration on Aid Effectiveness, specifically:

  • Pooling of cash pledges

  • Assured multi-year funding, preferably with front-loaded disbursements to the program

  • Accountability based on results to be tracked by indicators developed within an agreed logical framework of program goals.

148. The sources of financing the program should be diversified. Since it is perceived as a valuable regional public good, a starting point may be to seek reallocation of resources already directed to the region. In particular, approaches should be made to Venezuela and Trinidad and Tobago. Moreover, the program should explore ways for beneficiaries to increase their contribution while still not paying anywhere near full cost for technical assistance. Among possibilities are an increase in the flat contribution paid by all beneficiaries and/or some partial recovery from users based on their individual demand for technical assistance.

149. CARTAC should give greater emphasis to attachments and use of regional consultants as means of building human resource capacity in the region for 'best practice' economic management. Building credible public institutions staffed by adequate numbers of suitably trained people is a long-term endeavor. CARTAC has made good progress in creating networks and fostering a climate for on-the-job training. However, there needs to be more emphasis on attachments as an institutional priority across all functional areas. This would require beneficiaries to cooperate by releasing staff for longer periods than the 4 or 5 days that is now typical for attachments. CARTAC under the guidance of the Steering Committee should also embrace a wider concept of building capacity in the region to include more intensive use of regional consultants than has been the case to date. Regional consultants constitute a pool of human resources that can underpin common services in the area of economic management on a sustainable basis for countries that may be too constrained to keep highly skilled staff on the public payroll.

150. The Steering Committee should continue to insist on management reporting in a manner that facilitates the tracking of sustainable results from CARTAC's activities. In this connection, the Steering Committee can play a pivotal role to ensure that while CARTAC, like any other RTAC, fits within the strategic framework of overall IMF technical assistance that its work continues to be seen as reflecting the enlightened demands of its users and not the imposition of an agenda. In any event, it bears emphasizing that independent evaluation of the Fund's technical assistance has strongly recommended that it be framed within country-driven priorities.

151. By the inception of a third phase, there will be a sufficiently long track record to be stricter in using performance in implementing TA advice as an important criterion for allocating scarce resources, except of course for countries where CARTAC has had minimal activity in previous phases.

152. Major bilateral donors emphasize the importance to their constituencies of a focus on poverty reduction, already incorporated as an overarching goal in CARTAC's logical framework. The Steering Committee should take ownership of this strategic concern by taking steps to ensure that it is highlighted in management reporting based on the logical framework.

153. Based on demand and performance, the reviewers see a strong case for switching resources to public finance embracing revenue, expenditure, investment programming and debt management functions. We think that financial sector supervision does not require two full-time advisers. In practice, the PC oversees work of short-term consultants in the MAC area, therefore consideration should also be given to eliminating the position of MAC long-term adviser, particularly if the resources are fungible.

154. CARTAC should become proactive in forging relationships outside the English-speaking Caribbean where it has developed strong ties over the last few years. Haiti and Suriname are members of CARICOM and the Dominican Republic is a participant in the CARTAC program. To live up to its name, the Caribbean Area Technical Assistance Center ought to give growing attention to demand from outside the English-speaking Caribbean.

155. CARTAC should not become a legal entity. Acquiring a legal personality would not of itself improve prospects of additional financing. Concerns about administrative efficiency should be addressed in a direct and transparent manner with the UNDP. Moreover, creating a legal entry may imply an open-ended technical assistance program with the attendant risk of beneficiary countries falling into a comfortable habit of seeking extensions of TA support rather than seriously undertaking the job of building viable institutions.

APPENDICES

Appendix 1 : Second Mid-Term Review - Terms of Reference

Terms of Reference

Background and Purpose

The Caribbean Regional Technical Assistance Centre (CARTAC) is a regional resource, based in Barbados, which provides technical assistance and training in core areas of economic and financial management at the request of its 20 participating countries and territories19. The CARICOM Council of Ministers of Finance and Planning (COFAP) took the decision to establish the Centre in September 1999. It became operational in November 2001. CARTAC's mission is to "enhance the institutional and human capacities of the countries in the Caribbean region to achieve their macroeconomic, fiscal, and monetary policy objectives".

Many countries in the region face similar problems in meeting the standards of economic and financial governance expected of them by their citizens, and by domestic and international investors. CARTAC was created to help develop skills in the specialized areas required to design and implement measures to meet these standards at both the national and regional levels.

CARTAC is a multi-participant, pooled mechanism, one of the first in the region. Its three largest contributors are the Canadian International Development Agency (CIDA), the Department for International Development (OFID) and the International Monetary Fund (11%/IF), with each contributing over 20 percent of CARTAC's funding needs. The European Union (EU), Inter-American Development Bank (IDE), Ireland, the United Nations Development Programme (UNDP), the United States Agency for International Development (USAID), and the World Bank (WB) are also contributors, In addition, the Caribbean Development Bank (CDB) has seconded a full-time economist to the Centre. The Government of Barbados finances the costs of CARTAC's office facilities, whilst the other 19 beneficiary countries make annual contributions to the Project.

CARTAC operates as a cost-shared, UNDP project with the IMF as executing agency. Its priorities are set by a Steering Committee, which is chaired by the Governor of the Central Bank of Barbados and includes representatives from five other participating countries, two representatives from the bilateral donors, and six from multilateral agencies. The 20 countries and territories served by CARTAC, include Anguilla, Antigua and Barbuda,

CARTAC provides technical services in five core areas, namely:

  • Public finance management (PPM);

  • Tax/customs policy and administration;

  • Financial sector regulation and supervision, including off-shore financial operations;

  • Economic and financial statistics; and

  • Macroeconomic programming and policy analysis.

CARTAC was initially established for a three-year period (200 1-2004, called CARTAC 17). At its September 2002 meeting in St Kitts, CARTAC's Steering Committee agreed that an independent review of CARTAC's performance should be undertaken during 2003 to help make decisions on its future. The review, which was completed in August 2003, was favourable.

Based on a positive review, a recommendation to fund a follow-on phase for CARTAC was approved at the October 2003 meeting of the Steering Committee. The Steering Committee also requested all donors to continue finding for a further three year period, starting in 2005.

Sufficient funding was subsequently identified and CARTAC was extended for a second phase (2005-2007, called CARTAC II), In its second phase. demand for CARTAC services has strengthened, and it has increasingly sought to develop projects that are in large part transportable from one CARTAC client country to another, for example in the introduction of value-added taxes (VATs), and the strengthening of supervision of credit unions. CARTAC has also increasingly sought to emphasise attachments of staff from one country in the region to another, in the hope of not only providing relevant hands-on training, but also to increase networking within the region, and to provide a more permanent element to CARTAC's capacity-building efforts.

Reasons for the review

This mid-term review is being undertaken to review CARTAC's performance and results to date, including lessons learned, in order to determine the best modalities concerning CARTAC's work and mandate and promote eater accountability for performance. The review should pay special attention to the outcomes of CARTAC's work programs as identified in the log-frame analysis (see attached). Since a third phase of CARTAC is being contemplated, it will be important to look at achievements, impact and lessons learned The mid-tern' review will ascertain results to date and will help the Steering Committee determine optimal strategies for the project's continuation.

Issues to be addressed by the review

Program impact, effectiveness and synergy gains

The contribution of the Centre in addressing the region's problems and capacity constraints in each of the five functional areas covered (public finance management, tax and customs reform, financial sector regulation and supervision, macroeconomic programming and policy analysis, and economic and financial statistics) will be assessed. This should be done by focusing on CARTAC's outcomes, as well as its operational efficiency and effectiveness, and gains resulting from synergies between its operations and those of other donors in the region.

The primary issues in this section should include:

  • i. The results, including effectiveness and impact, of CARTAC's overall TA program, vis-à-vis CARTAC '5 results-based log-framework. This should include an evaluation of outcomes, and preliminary impact of one or two of CARTAC's areas of TA delivery;

  • ii. To what extent has the timeliness of delivery and follow-up been enhanced. Needs and requests have been responded to promptly;

  • iii. The extent that CARTAC has proved responsive in adapting to the region's changing needs. In doing so, the following questions should be addressed:

  • a. Does a purely responsive approach to TA demand remain appropriate;

  • b. Should the balance of resources between program components be reconsidered; and

  • c. Should the prioritization filters adopted by the Steering Committee be reconsidered

  • iv. The extent of country ownership of TA delivered by CARTAC;

  • v. The extent to which CARTAC has contributed to improved integration of TA delivery by national and multinational development partners, with fewer overlaps observed, and the extent to which CARTAC has fostered improved donor cooperation and coordination (progress with respect to items iii-iv should be considered in the context of the objectives of the Paris Declaration); and

  • vi. CARTAC's contribution to enhanced regional integration and knowledge- sharing through workshops, attachments, collegial coordination, helping to develop regional projects, sharing project outputs within the region, and other mechanisms. The review team may also make recommendations of steps CARTAC might take to improve the sustainability of knowledge-sharing aspects of its assistance to CARTAC member countries including possibly the sharing of output with other similar TA delivery mechanisms.

Organizational efficiency, effectiveness and sustainability

The mid-term review will also be expected to comment on the organizational efficiency and effectiveness of CARTAC, as well as the desirability of a further extension of CARTAC and its sustainability. The primary issues in this section would include the following:

  • i. The efficiency of the harmonized/pooled funding mechanism, and whether CARTAC should accept all funding offers, even funds that are tied and have considerable conditionality;

  • ii. The role and effectiveness of the Steering Committee, individual governments, donors, regional bodies, and the IME in determining CARTAC's direction and work plans;

  • iii. The role and effectiveness of the UND?, including accepting, managing, and accounting for the cost-sharing contributions;

  • iv. The role and effectiveness of the IMF as CARTAC's "executing agency";

  • v. The role and effectiveness of the Coordinator in managing CARTAC's work programme and the CARTAC office;

  • vi. Should a third phase of CARTAC be encouraged, and if so, the feasibility and desirability of both changing the legal status of the Centre, e.g., make it a legal entity, and reviewing the composition of the Steering Committee; and

  • vii. If policy changes are appropriate during a triennium, what modalities should be used for effecting those changes.

The review should record any significant lessons that can be drawn from the experience with the second phase of CARTAC, highlighting anything that worked well and that should be emphasised, and anything that has worked badly and should be avoided in future. The reviewers will take note of any suggestions received during the course of the review on the future direction of CARTAC's work areas and operational modalities- The reviewers should also note whether they identified any needs in the region that are not currently being met by any organization and are either included in or closely related to the topics covered in CARTAC'S Programme Document.

Review Process

The Review Team will be retained to carry out the mid-tern' review and preside over the selection process to fill the consultancies under this mandate. The Working Group will both present a short-list of candidates to the Steering Committee for their approval, and oversee the work of the review team.

Field Mission

The review will include meetings with a broad cross-section of CARTAC constituents and stakeholders and with funding agency personnel. The fieldwork is expected to be three and one half (3,5) weeks in duration. A briefing with funding agencies, the Chairman of CARTAC's Steering Committee, and the CARTAC Coordinator will take place in Barbados before other visits in the region. At some point the review will include a visit or consultation with relevant headquarters personnel in the IME, UNT)P and World Bank.

Mid-Term Review Revert

The review team will prepare a mid-term review report that puts forward the reviewers' findings, recommendations and lessons learned. The report will be:

  • i. Prepared in English only;

  • ii. Submitted electronically and in hard copy format.

Level of Effort (LOE)

The proposed work under this mandate is expected to be approximately thirty-three (33) days per consultant. It is estimated that one (1), five day, week will be required for preparatory considerations; three and a half (3.5) weeks including travel for field work; and two (2) weeks for report writing at home base.

Reviewers' Qualifications

It is proposed that the mid-term, review would be carried out by two (2) senior consultants, one with a solid background in economics, governance, and technical assistance, and the other with skills in technical assistance and project evaluation. Preference will be given to experts with experience in the region.

The first resource should have a background in economics, public sector fiscal operations, and public sector management, or financial sector regulation and supervision (of banks and non-bank financial institutions) and be able to provide professional economic capacity development and evaluation services. In addition, the proposed resource must have a background in economic policy reform/development.

The second resource should have a strong background in financial sector regulation and supervision (of banks and non-bank financial institutions) or public sector fiscal operations, and public sector management,. The proposed resource must be able to provide expert advice in the areas of financial policy, evaluation and regulation. The duration of this mandate will for a 33 day period between June 1 and September 15, 2006.

Envisaged Work Program

1. Preparatory work and development of work program May;

2. Field investigations—June/July; and

3. Reporting and presentation:

  • a. Preliminary report August;

  • b. Final report early-mid-September,

Appendix 2: Questionnaire

Appendix 3: Analysis of CARTAC Program Activities

article image
article image
article image
article image
article image
article image
article image

Appendix 4: List of Persons Interviewed

article image
* by telephone

Appendix 5: CARTAC Attachments

article image

Appendix:6 CARTAC Training Activities

article image

Appendix 7: CARTAC Foreign Consultant Usuage

article image

Appendix 8: CARTAC Regional Consultant Usage

article image
1

Anguilla, Antigua & Barbuda, The Bahamas, Barbados, Belize, British Virgin Islands, Cayman Islands, Dominica, Dominican Republic, Grenada, Guyana, Haiti, Jamaica, Montserrat, St. Kitts & Nevis, St. Lucia, St. Vincent & The Grenadines, Suriname, Trinidad & Tobago, Turks & Caicos Islands.

2

UNDP/IMF "Strengthening Economic and Financial Management in the Caribbean region: Caribbean Regional Technical Assistance Centre (CARTAC) Extension, 2005 - 2007" p.10

3

We are not suggesting that CARTAC bears full responsibility for translating outputs into results. The Paris Declaration on Aid Effectiveness is grounded in mutual accountability of donors and "partner countries." Of particular relevance in this context is the commitment of partner countries to "undertake reforms, such as public management reform, that may be necessary to launch and fuel sustainable capacity development processes." Paris Declaration, p. 4

4

First Mid-Term Review of Caribbean Technical Assistance Centre (CARTAC) August 2003, page 7 (http://www.imf.org/external/np/pp/eng/2003/080103 .htm).

Source: CARTAC

5

First Mid-Term Review of Caribbean Technical Assistance Centre (CARTAC) August 2003, pages 26-27 (http://www.imf.org/external/np/pp/eng/2003/080103.htm)..

6

"a Program Support Unit (PSU) for the Centre would be housed at the UNDP in Barbados, and would be responsible for the logistics and procurement of training venues, travel of experts and training participants, equipment procurement, and payment of appropriate allowances" UNDP Programme for the Caribbean regional Technical Assistance Centre, Programme Support Document RLA/00/02, p. 14.

7

UNDP/IMF "Strengthening Economic and Financial Management in the Caribbean region: Caribbean regional Technical Assistance Centre (CARTAC) Extension, 2005 - 2007" p.25.

8

Forum of Caribbean States established in 1992 during Lome IV negotiations to create a vehicle that permitted the European Commission to cover Haiti and Dominican Republic (not members of CARICOM at that time) in regional funding through resources of the 7th European Development Fund (EDF).

9

Ibid. page 37

10

PFTAC, METAC, and the East and West AFRITACs.

11

The counts of both foreign and regional of experts utilized are calculated on a per period basis and ignore the fact that the same individual consultants could have been utilized in different time periods. Consequently, the tally of both foreign and regional consultants utilized is somewhat overstated.

12

The Reviewers have been provided with a Roster of Consultants used by the Center as at September 13, 2006. This lists 126 foreign and regional consultants, of which 46 have been identified as having been used. This compares with a total of 76 foreign and 58 regional consultants reported to the Steering Committee as having been used between May 2004 and March 2006. FSS also reports that they utilized some 36 consultants that were not identified in the Reports for the period under review.

13

The reviewers had requested lists of contacts from each Adviser and received only one response and the list from the office that was eventually utilized.

14

The list of key contacts that was presented to the reviewers, for example, appeared incomplete and was not up to date, as reviewers discovered when identifying persons to whom the questionnaire should be distributed in some countries.

15

It appeared to the reviewers that some of the incoming long-term advisers may not have had fully up to date contact and consultants listings, especially for regional consultants.

16

The 20 countries and territories served by CARTAC, include Anguilla, Antigua and Barbuda, The Bahamas, Barbados, Belize, British Virgin Islands, Cayman Islands, Dominica, Dominican Republic, Grenada, Guyana, Haiti, Jamaica, Montserrat, St. Kitts and Nevis, St. Lucia, St. Vincent and the Grenadines, Suriname, Trinidad and Tobago and the Turks and Caicos Islands.

17

The list of key contacts that was presented to the reviewers, for example, appeared incomplete and was not up to date, as reviewers discovered when identifying persons to whom the questionnaire should be distributed in some countries.

18

It appeared to the reviewers that some of the incoming long-term advisers may not have had fully up to date contact and consultants listings, especially for regional consultants.

19

The 20 countries and territories served by CARTAC, include Anguilla, Antigua and Barbuda, The Bahamas, Barbados, Belize, British Virgin Islands, Cayman Islands, Dominica, Dominican Republic, Grenada, Guyana, Haiti, Jamaica, Montserrat, St. Kitts and Nevis, St. Lucia, St. Vincent and the Grenadines, Suriname, Trinidad and Tobago and the Turks and Caicos Islands.

Same Series

Other IMF Content

Other Publishers

  • Collapse
  • Expand
Cover Policy Papers
Technical Assistance Evaluation Program - Findings of Evaluations and Updated Program - Mid-Term Evaluation of Middle East Regional Technical Assistance Center
Author:
International Monetary Fund
Volume/Issue:
Volume 2008: Issue 021
Publisher:
International Monetary Fund
ISBN:
9781498334730
ISSN:
2663-3493
Pages:
89
DOI:
https://doi.org/10.5089/9781498334730.007