Midterm Review of the Caribbean Regional Technical Assistance Centre (CARTAC)

The mid-term review was undertaken to help funding and implementing agencies foster a greater level of understanding of the Caribbean Regional Technical Assistance Center's (CARTAC) work. It ascertains activities to date and should help the Steering Committee determine optimal strategies for the Center’s continuation.

Abstract

The mid-term review was undertaken to help funding and implementing agencies foster a greater level of understanding of the Caribbean Regional Technical Assistance Center's (CARTAC) work. It ascertains activities to date and should help the Steering Committee determine optimal strategies for the Center’s continuation.

Chapter 1 Introduction

1.1 Overview of Project

The Caribbean Regional Technical Assistance Centre (CARTAC) is a regional resource, based in Barbados, which provides technical assistance in core areas of economic and financial management at the request of its participating countries. CARTAC was created as a result of a Caribbean Community (CARICOM) Council of Ministers of Finance and Planning (COFAP) decision in September of 1999. CARTAC became operational in November 2001 and the current phase of donor support for the Centre will come to an end at the end of 2004 or the beginning of 2005.

Countries in the region face similar problems in meeting standards of economic and financial governance expected of them by their citizens and by investors. CARTAC was created to help develop skills and institutional capacity in the specialised areas required to design and implement measures to meet these standards at the national, regional and international levels. Countries participating in CARTAC are: Anguilla; Antigua & Barbuda; The Bahamas; Barbados; Belize; British Virgin Islands; Cayman Islands; Dominica; Dominican Republic; Grenada; Guyana; Haiti; Jamaica; Montserrat; St.Kitts and Nevis; St. Lucia; St. Vincent & The Grenadines; Suriname; Trinidad & Tobago; and Turks and Caicos Islands.

CARTAC operates as a UNDP project (UNDP RLA/01/011). It is financed from contributions from bilateral and multilateral donors, from payments-in-kind, and from annual contributions from participating countries. The principal contributors, in addition to each of the above listed participating countries, are: Barbados; CDB; CIDA; DFID; EU; IBRD; IDB (pledged); Ireland; IMF; UNDP; and USAID.

The IMF is the executing agency. The IMF has provided an IMF staff member as the Program Coordinator, has contracted the long-term Advisors, has helped set up the office and computer systems in Barbados, has provided backstopping for the long-term Advisors, and has helped source short-term technical advisers. In addition to funding the Program Coordinator, the IMF also funds two of the assistants in the CARTAC office.

The priorities of CARTAC were outlined in the Project Document and have been confirmed by a Steering Committee consisting of six representatives from participating countries, five representatives from the bilateral and multilateral agencies, and one each from CARICOM and the CDB. The Steering Committee meets every six months, with every third meeting being convened in plenary session.

The mission of CARTAC is to enhance the institutional and human capacities of countries in the Caribbean region to achieve their macroeconomic, fiscal and monetary policy objectives. It meets its mission by providing technical services in four core areas, namely: (a) Public Expenditure Management; (b) Tax/Customs Policy and Administration; (c) Financial Sector Regulation and Supervision; and (d) Economic and Financial Statistics.

1.2 Purpose of Mid-Term Review

The mid-term review was undertaken to help funding and implementing agencies foster a greater level of understanding of CARTAC’s work. The mid-term review ascertains activities to date and should help the Steering Committee determine optimal strategies for the Centre’s continuation. The terms of reference for the mid-term review are presented as Appendix A.

1.3 Methodology

The field work and report writing for the mid-term review were conducted by James Bucknall and Neville Grant over a two-month period from mid June 2003 to mid August 2003. The methodology of the mid-term review consisted of:

  • 1. A review of the minutes of Steering Committee meetings;

  • 2. A review and analysis of CARTAC’s periodic work plans and activity reports that are presented to the Steering Committee;

  • 3. Several discussions with CARTAC’s Program Coordinator, the long-term Advisors, and the administrative and financial assistants;

  • 4. Personal interviews with officials of central banks, ministries of finance, and other government departments in: (i) The Bahamas; (ii) Barbados, (iii) Dominica; (iv) Grenada; (v) Guyana; (vi) St Kitts and Nevis; (vii) St. Lucia, (viii) St. Vincent and The Grenadines, and (ix) Trinidad & Tobago;

  • 5. Telephone and/or e-mail interviews with representatives of several of the other 11 participating countries;

  • 6. Interviews with regional organisations including CARICOM, CDB and ECCB in Barbados, Guyana and St. Kitts and Nevis;

  • 7. Interviews with representatives of CIDA, DFID, EU, UNDP and USAID in Barbados;

  • 8. Interviews with officials of IMF, IDB, World Bank and UNDP in Washington, DC and New York;

  • 9. A review of office procedures, filing systems and financial accounts in Barbados;

  • 10. A detailed study of 12 official activity files and 12 back-up working files in the CARTAC office in Barbados;

  • 11. A review of 12 examples of monthly long-term Advisor reports to Washington; and

  • 12. A review of 16 Back to Office Reports prepared by the long-term Advisors and short-term consultants.

A list of people met is presented as Appendix D. A list of people contacted by email and telephone is presented as Appendix E.

1.4 Mid-Term Review Team

The Steering Committee agreed that the international arm of a government consulting agency should undertake the mid-term review. Consulting and Audit Canada (CAC) was selected as the most appropriate body.

Consulting and Audit Canada (CAC) is a Special Operating Agency of the Canadian Government. Its services are available to federal government organisations and other public sector and international organisations. Internationally, CAC has provided consulting and audit services to a wide range of national governments and international organisations. It specialises in providing services to improve the quality of the public sector and to evaluate programs designed to strengthen public sector economic and financial management. CAC ran two separate competitions to select the reviewers. James Bucknall won the economics specialist competition and Neville Grant won the financial services specialist competition.

James Bucknall has a PhD in economics and worked for 14 years in the federal Ministry of Finance in Ottawa, Canada. Since 1984 he has specialised in designing, executing, monitoring, evaluating and providing technical assistance to economic management projects in 32 countries in Africa, Asia, and the Caribbean region. In the late 1990s he was the Financial Secretary of the Government of Sierra Leone.

Neville Grant is President of G.N. Grant and Associates, a financial consulting firm in Ottawa, Canada. He served as Managing Director of the Cayman Islands Monetary Authority; Advisor, Bank Supervision, Bank of Zambia; and Director, Bank Supervision in Canada. He has also worked as a Financial Sector Advisor with the International Monetary Fund in Washington D.C.

1.5 Organisation of the Report

There are four substantive chapters in this report. Each chapter has a section entitled summary findings. Recommendations are also included in each chapter at the appropriate location.

Chapter 2 contains an analysis of CARTAC’s main activities. The principal topics discussed include: (a) the strategic approach followed by CARTAC in developing its program; (b) an analysis of the main countries in which CARTAC has been most active; (c) feedback on CARTAC’s activities from the countries that were visited; (d) feedback from several countries that were not visited; (e) the responsiveness of CARTAC to the needs of the participating countries and the appropriateness of CARTAC’s activities; and (f) an analysis of CARTAC’s cost and outputs.

The management and administration of the CARTAC office in Barbados is discussed in chapter 3. Subjects covered include: (a) the CARTAC budget; (b) CARTAC’s systems of accounts; (c) general administration procedures; and (d) the quality and appropriateness of the three systems of reports.

The subject of CARTAC governance is reviewed in chapter 4. This chapter reviews the role of the UNDP, the IMF, the Steering Committee, and the Technical Panels.

Some of the remaining, more macro issues, are reviewed in chapter 5. These include: (a) regional approach to technical assistance; (b) capacity building and sustainability; (c) the question of whether CARTAC should be reactive or proactive; and (d) participating country support.

Finally, the principal findings and recommendations are summarised and presented in chapter 6.

Chapter 2 Analysis of Cartac Activities

2.1 Introduction

The purpose of this chapter is to discuss : (a) the strategic approach employed by CARTAC in determining its work-plan; (b) the regions and countries that CARTAC is most active in; (c) the overall reactions of countries that are making extensive use of CARTAC services; (d) the reasons why certain countries are not using CARTAC’s resources as much as others; (e) the responsiveness of CARTAC to the changing needs in the region; and (f) the cost efficiency of the inputs and the value of the outputs to date.

2.2 Strategic Approach

When the CARTAC long-term Advisors started with CARTAC in late 2001 and early 2002, each one, when requested by the participating government, visited the country and conducted a diagnostic review of the issues. In many countries, Guyana for example, the government had a well developed PRSP/PRGF program framework and all of CARTAC’s activities fit into this framework. In other countries, the national program was less well developed and CARTAC and the host government worked on the program together. The OECS is an example of this latter approach where all of CARTAC’s work is guided by the OECS and the ECCU Monetary Council’s development strategy which includes fiscal convergence, tax reform, and the financial and statistical sectors. CARTAC contributed significantly to both the development of the strategy as well as to its implementation.

Following the diagnostic phase and the development or confirmation of the participating country’s strategic plan, an overall work plan was developed and the work was divided up between the potential participants whether they be the government itself, multilateral institutions, bilateral donors or CARTAC. Where a particular issue required longer term inputs and where other donor projects were involved, then these issues were worked on by the donor projects. In those cases where no other donor was involved and where the nature of the issue fits with CARTAC’s mandate, then CARTAC became involved. The nature of this involvement included technical advice from the long-term Advisor, a specific training activity or seminar, short-term experts and consultants or in coming or out going professional attachments.

Most of CARTAC’s longer term work fits within the above logical and coherent framework, but not all of it. There are cases where specific problems arise which are best resolved by CARTAC, as opposed to one of the other donors, because CARTAC is flexible and can react rapidly. An example of this type of activity would include the development and support of the Cash Management Unit in Dominica. While not part of the overall strategic plan, this intervention solved a particular issue in a timely manner. Other examples would be CARTAC, by financing an IT expert from Barbados, helping Dominica resolve some of its Smart Stream software PEM reporting problems, and, by coordinating with other donors, assisting in obtaining financing for rolling out Smart Stream to all Dominican government agencies. Other CARTAC activities may be more regional in nature. These would include the regional training programs, the OECS tax policy study, and the Smart Stream Users Conference.

2.3 Analysis of Cartac Activities

CARTAC’s mandate permits it to work in four main areas in 20 different countries. The following tables show the principal countries in which CARTAC is working. The data is developed from material that has been presented to the Steering Committee at its meeting in February of 2003. The data from which these tables were developed are presented as Appendix C.

Table 1 shows the activities that are reported by each long-term Advisor, in each country, in each of the three six-month periods that CARTAC has been in operation. Certain general conclusions can be drawn from the data presented in this table.

  • The largest number of long-term Advisor activities is found within the OECS, ECCB and OECS countries. Some 25 activities were in OECS and ECCB and were classified as sub-regional. Another 36 were in individual OECS countries.

  • The second largest number of long-term Advisor activities, 23, were classified as regional, i.e. were not associated with any one individual country or sub-region but were for the whole region.

  • The third largest number of long-term Advisor activities was recorded for Barbados followed closely by Guyana.

  • The remaining activities were spread out over the region with five countries participating in one or less activities.

  • More activities were recorded in Financial Services but this likely reflects that there were two long-term Advisors working in this area.

  • By and large, if CARTAC was active in one technical area in a country or region, it was also active in other technical areas in that country or region.

Table 1

Long Term Advisor Activities by Country and Region

article image

Key to Headings

  • 1 Advisor activities undertaken October 2001 – February 2002

  • 2 Advisor activities undertaken March 2002 – August 2002

  • 3 Advisor activities undertaken September 2002 – February 2003

Attachment 2 of the February 2003 Steering Committee minutes provided a summary of all of the training activities and seminars, professional attachments, regional short-term experts and international short-term experts. Table 2 shows how these various activities were spread out over the 20 participating countries. The following conclusions can be drawn from these data:

  • The three largest categories of participant are OECS sub region, regional and the individual OECS countries.

  • After the above three areas, the next largest participant countries are Barbados and Guyana.

Table 2

Training and Consulting Activities by Country

article image

Key to headings

  • Activity 1 Training Activities & Seminars

  • Activity 2 Professional Attachments

  • Activity 3 Short-Term Experts – Regional

  • Activity 4 Short-Term Experts – International

It should be noted that representatives of all countries attended some of these training sessions.

The next table in this section, Table 3, weights the training and seminars by showing both the number of participants and the person-days of training (number of persons trained times the length of the training session). This table shows that:

  • 1,613 have undertaken training or attended a seminar provided by CARTAC;

  • The total number of person-days of training is 4,610; and

  • Some 90% of the training was in PEM, Tax and Financial Services.

Table 3

Training and Seminars by Technical Area

article image

Headings:

  • 1. Number of People Trained

  • 2. Total Person-Days of Training

Table 4 shows the number of professional attachments, regionally based short-term experts and internationally based short-term experts and the person days of these experts by the four categories of advice provided by CARTAC. Based on this analysis one can observe:

  • More short-term experts have been used in Tax and Financial Services than in PEM and Statistics;

  • Regional short-term experts are used more often then international experts

Table 4

Professional Attachments and Short-Term Experts

article image

Headings:

  • 1. Number of Activities

  • 2. Number of Person-Days

2.4 Views of Countries Visited

Based on this analysis the reviewers decided to concentrate their in-country interviews to those countries in which CARTAC had been most active, i.e. Barbados, Guyana, Trinidad and Tobago, St. Kitts, Dominica, St. Lucia, St. Vincent and The Grenadines, Grenada and The Bahamas. This section summarises some of the more important findings derived from these interviews. No attempt is made here to describe individual projects or the response of individual countries to these projects. This information is described in fine detail in the Back to Office Reports, the Monthly Reports of the Advisors to the IMF in Washington, and the Work Plans and Activity Reports. The following comments apply to CARTAC’s work in public expenditure management, tax and customs reform, financial sector regulation and supervision, and economic and financial statistics.

All of the participating countries spoke very highly of the need for and modus operandi of CARTAC. From the point of view of the individual participating countries, CARTAC is a success. Almost everyone spoke of CARTAC’s openness to requests, the speediness of its response, the quality of its inputs and its key strategic support at crucial junctures. CARTAC has, in the view of the participating countries, filled an important role in the Caribbean. Several countries mentioned that the Council of Ministers of Finance and Planning of CARICOM had publicly expressed its support of CARTAC activities.

Participating countries were impressed at how quickly and how well CARTAC had become integrated into the region. As a result of this integration, CARTAC did not duplicate the work of other multinational or bilateral agencies and their projects. Rather CARTAC has been able to complement the work of other agencies and is able to use both its ability to move quickly and its technical capacity to fill a niche that other agencies are unable to fill. Participating countries all agreed that CARTAC’s activities were all more effective when the country had a well developed strategy and action plan which was owned by the country. This action plan could have been developed prior to the creation of CARTAC or with the assistance of a CARTAC long-term Advisor. Regardless of the genesis, the important point is that there should be an action plan and that plan must be owned and supported by the participating country.

Many of the countries in the region face common issues yet, for a variety of reasons, the skills and lessons learned in one country are not easily applied in others. CARTAC has been able to overcome this problem in a number of ways such as creating the PEM Network, holding the Smart Stream Users Conference and the secondment of experts from one country to another. Participating countries appreciate the fact that CARTAC’s advice is practical and hands-on. There is a general sense among the participating countries that the region is good analytically but that it is sometimes lacking in its implementing ability. CARTAC is seen as useful in bridging this gap between theory and implementation.

The consensus is that where a regional approach is required, it should be followed. Likewise, when a national approach is required then that is the way things should be done. Regional courses and seminars are much appreciated as they are more efficient, allow participants to network with practitioners in other countries, and permit the sharing and spread of best practices. At the same time, even though there is a region wide need for the Stabilisation and Adjustment Technical Assistance Program (SATAP) in the OECS, each country accepts that the issues it faces are somewhat different from it neighbours and each country must develop its own unit.

The Program Coordinator and the individual long-term Advisors are all very much appreciated by participating countries. Many participants made a special point of expressing their gratitude and appreciation of the former PEM long-term Advisor. Participating countries gave their full endorsement to all of CARTAC’s long-term Advisors and gave praise to the short-term resources that were engaged by CARTAC for training, seminars and short-term consultancies. Many people interviewed singled out one or more of the long-term Advisors for special praise. They found them open to ideas, cooperative in developing activities or a program of activities, prompt to find resources and always ready to help out over the phone or through emails. The reviewers were shown several email strings that support the depth of the long-term Advisors backstopping.

Participating countries felt that all interventions were made with due regard to costs and benefits. Most felt that the administrative and logistical arrangements surrounding training courses and seminars sought the right balance between luxury and frugality. Many participating countries felt that the courses were useful and the information gained was being put to practical uses. The auditing courses were mentioned by several countries in this context.

All participating countries want the Centre to be funded after the completion of the current phase.

2.5 Views of Countries not Visited

The reviewers did not visit in person all of the participating countries in CARTAC because of the constraints of time and financial resources. Those countries that the reviewers were unable to visit were contacted by the reviewers by email and telephone.

All respondents indicated some familiarity with CARTAC’s services, although the degree of familiarity varied from one country to another. Some were familiar with the entire range of services. Others were less familiar and knew of the services that were offered in the area of their particular expertise and responsibility. It sometimes appeared that knowledge might have been spotty in a participating country because of an issue of internal government communication. Several respondents hinted that the pressure of day-to-day activities meant that they did not have time to get around to thinking about CARTAC. They suggested that CARTAC should make an annual visit to see what each country is doing. This would include a discussion of areas of weaknesses and a listing of priorities. This would help to focus the mind and would result in greater utilisation of services.

Several respondents had initiated requests for assistance. The requests were initially by telephone call to one of the long-term Advisors to discuss the issue. This was followed in a number of instances by a visit by the long-term Advisor to the country to discuss the matter in greater detail. After the details were fleshed out and the matter clarified a letter was sent by the Head of the relative agency or government department to the CARTAC Program Coordinator. In at least one instance the request for assistance was a result of a review of systems that was conducted by an international agency that made several recommendations. As a result, the country decided to request CARTAC to assist in implementing the recommendations. All of the respondents regarded the turn around time to be quite quick, usually within a few weeks or even a few days of making the initial telephone enquiry a concrete proposal regarding assistance was in place. The type of assistance the countries received varied. In some cases it related to assistance in writing of legislation or regulations and in other instances training for staff and in another instance a seminar to which regulators and private sector individuals were the recipients. All respondents that received assistance were pleased with the quality, including the training, which they noted is being put to good use.

Those who used CARTAC noted that as a result of the favourable experience, including the ease with which request can be made, they are likely to request more assistance in the future. One country that is taking advantage of courses and seminars to upgrade skills stated that a need has not yet arisen for direct assistance. Others are still working out how CARTAC can provide further assistance to them in the future. All indicated that it is their intention to use more of CARTAC’s assistance in the future.

In general respondents believe that the improvements that were made in their country as a result of CARTAC’s assistance are sustainable. However, with respect to training, most indicated that another round of training will be needed as the department or agency is in the process of hiring additional staff to bolster its capacity to carry out its functions. A plea was made to use local and regional experts where possible as this will lead to capacity building within the region.

Lack of knowledge regarding CARTAC is not the primary reason that some countries are not using CARTAC at all or not as much as others. All countries have sufficient knowledge of CARTAC that they could use its services if they wanted or needed to. A number of regional activities covering all aspects of CARTAC’s mandate have been organised and participants from all of the countries have been invited to these presentations. While all countries are generally aware of CARTAC, it is not clear in some instances that all of the government departments that could benefit from CARTAC’s services are aware of how CARTAC can help them. This is an internal government issue and relates to an issue of lack of communication across government departments. Communications among government officials in various departments was not as effective as it should have been and information that was useful about CARTAC’s activities was not passed on from one department to other departments.

The reasons for not participating are quite varied. Some initial reluctance to participate related to CARTAC’s newness and the jurisdictions were not aware of the range and quality of services and how to go about requesting such services. Part also related to caution regarding CARTAC’s association with the IMF. These concerns have been overcome and more requests are now coming from such countries.

Some countries did not immediately have the capacity to absorb some of the services on offer. The jurisdictions are working to rectify the situation and expect that requests for appropriate assistance will be forwarded to CARTAC at some time in the future. Others had more pressing problems that had to be attended to and were not able to plan their work effectively to integrate CARTAC’s assistance into their work plans. Some were in the process of formulating overall plans and did not want to get involved with CARTAC until their own plans were clear and it could be determined how best CARTAC could help while others believe that they are able at present to manage without CARTAC’s involvement. Some have had reform programs underway even before CARTAC came on the scene. They are continuing their programs and can see a need at a later stage for CARTAC’s involvement. Finally there are some countries that have greater capacity than others to resolve their own issues and it appears that those who have greater capacity have been slower than others in utilising CARTAC’s assistance.

2.6 Responsiveness and Appropriateness of Activities

The modus operandi of CARTAC is to provide resources to assist countries and regions to execute their action plans that cannot be provided by the host country or by multilateral or bilateral agencies in a timely manner. It is essential, therefore, that CARTAC, in order to be successful, must respond quickly with appropriate resources. This necessitates that there always be resources in reserve to respond to needs and that CARTAC not commit all of its resources to the long-term activities. There are other donor projects that are better suited to providing long-term inputs. CIDA’s ECEMP 3 is a good example of this approach. Many participating countries mentioned that they thought that it would be a mistake if CARTAC tied up all of its resources in longer-term activities if it meant that they could no longer undertake short term activities.

The mid-term reviewers verified these observations of the participating countries by a careful review of CARTAC’s central files and CARTAC long-term Advisors’ working files on training activities and seminars, professional attachments and short-term consultancies. The quality of the training inputs were verified by a review of the qualifications of the trainer, the quality of the training material, and the high grades given by the participants in the formal course evaluation. The final confirmation was the quality of the formal “Back to Office Report” which is prepared at the end of each training session by the trainer. A similar review of the two sets of files was also done for short-term experts. Again the conclusion is that the quality and timeliness of the input was appropriate based on the qualifications of the short-term consultant, the time lapsed between the formal request for assistance, the feed back from the participating country and the quality of the “Back to Office Report”.

The economic situation in the Caribbean has not remained static over the last two years. Prior to the creation of CARTAC many countries were under pressure due to the decline in price of many agricultural products and the downturn in tourism. This rapid decline in fortunes has caused crisis or near crisis in some countries and extreme caution in others. Perhaps the worst hit area is the countries of the OECS in general and Dominica in particular. CARTAC, which already had a large program planned for the OECS, responded by increasing the resources allocated to these countries. First, CARTAC increased resources going into Dominica, and second the Centre increased resources across the board in the other OECS countries to help prevent the situation in Dominica from spreading. The creation and strengthening of the SATAP units in individual OECS countries, in collaboration with ECCB, is another example of this responsiveness to changing needs in the region.

2.7 Project Costs, Impact and Results

Participating countries agree that the individual activities undertaken by CARTAC are the appropriate ones and that they are delivered in a timely fashion. The next question for the reviewers was whether these inputs are delivered with due regard to cost. A careful review of the files and interviews with the Program Coordinator and the long-term Advisors result in a positive answer. The salaries and other remuneration of the long-term advisors are commensurate with their qualifications. The salaries, level of effort and expenses of short term experts are commensurate with international and regional standards. Expenses associated with training courses are monitored carefully and must be kept within an approved budget. CARTAC travel expenses, are not extravagant, e.g. long-term Advisors do not stay in the most expensive hotels available. Furthermore, due regard is given to the cost of holding one course in several countries with the trainer moving or moving the participants from several countries to one central training centre. Based on this analysis the reviewers conclude that the right things are being done at an appropriate cost.

The inputs and outputs of CARTAC during the first 18 months of its existence have been significant. The number and cost of these interventions have been appropriate. Furthermore, the value of the outputs derived from these inputs has been positive: hundreds of people have received thousand of person-days of training; immediate increases in productivity in the work place have been reported by many people; and numerous strategies and detailed action plans have been co-developed with participating countries in all four areas of CARTAC activities. Not only are there the direct benefits from all of these activities but many secondary benefits were reported such as meeting people with the equivalent position in neighbouring countries, sharing problems, solutions and best practices, networking, and seconding staff to neighbouring countries to resolve specific issues.

All of these activities have addressed immediate problems or have been developed to prevent or detect other problems before they become unmanageable. As such they are all worthy from a micro cost benefit vantage point. CARTAC is conscious that its performance in the short run will be measured on the outputs derived from its individual inputs or activities but that ultimately its performance will be measured on its impact on a higher level of indicators. As a result, CARTAC is in the process of developing a number of indicators of change that will used, inter alia, to determine the long term results. CARTAC should be encouraged to continue the work of defining the Indicators of Change.

In summary, the activities of CARTAC are timely and appropriate, are delivered with due regard to cost and are effective in resolving both immediate and longer term issues. The outputs to date have justified the cost of inputs that have been used to achieve these outputs. The longer term and higher level results and impacts will take time to develop and will be a challenge to measure. CARTAC accepts this challenge and is developing a set of indicators that will measure change over time and which can be used to measure longer term results and impacts.

2.8 Summary Findings

  • 1. There was a clear need in the Caribbean region for increased technical assistance in the area of economic and financial management, the CARTAC project was well-designed to meet that need, and the CARTAC project has been well executed. CARTAC has played a significant and positive role in the region and is much appreciated by the participating countries and the donors.

  • 2. CARTAC has become well integrated within the region, fills a niche that cannot be filled by other projects, and performs an important role of helping coordinate technical assistance of other longer term donor projects in the region. CARTAC’s activities are of a high professional quality.

  • 3. Its success is in large part attributable to its flexibility and its quick response to requests for assistance. The lack of bureaucracy in making requests has also been a contributing factor. There are neither extensive forms to complete nor complex proposals to write to justify a request. A letter of request has been the usual form.

  • 4. CARTAC’s task has been facilitated because the region has been welcoming. Recipients have been uniformly satisfied with the quality of assistance. Interviewees with first hand knowledge of some of the training courses or advisory missions undertaken expressed their satisfaction with the speed and quality of assistance provided. The ability to share experiences with regional counterparts has been one of CARTAC’s more important contributions.

  • 5. Ultimately, any technical assistance program is only as successful as the people engaged in its operation. No amount of elaborate project identification, formulation, and monitoring employed by TA providing agencies can guarantee results if the staff, experts and counter-parties involved in its implementation are unsuitable or uncooperative. CARTAC has been fortunate in having good experts selected for its core staff, having the resources available to make use of the skills and talents that abound within the region and elsewhere, and having the resources available to make use of the region’s cadre of highly professional and committed civil servants.

  • 6. The approach taken by CARTAC in developing its program of interventions and activities has been both logical and methodical. The individual national and regional work programs thus developed are both coherent and comprehensive. The way that CARTAC does business is not intrusive and this contributes to its success and acceptance. CARTAC has struck the right balance between regional activities and individual country activities.

  • 7. CARTAC interventions appear to be most successful when they are fully integrated into a strategy and work-plan that is developed and owned by the participating country. No activity should be undertaken unless it is requested by an individual government or regional body. CARTAC has been and should continue to be primarily reactive rather than proactive so that all activities are owned by the participating country or the region. CARTAC should continue to concentrate on short-term projects. In doing so it will remain flexible and not get involved with activities that would tie up its resources in the long term and limit its flexibility.

  • 8. The combination of CARTAC’s planned, strategic long-term activities and its short-term one-off fire fighting activities is balanced and appropriate. CARTAC interventions have taken into full account the activities of the national government, bilateral donors, and multinational agencies. CARTAC becomes involved in a particular activity only if it the most appropriate agency to undertake the activity.

  • 9. CARTAC’s activities have been concentrated on OECS countries, Barbados and Guyana. Most other countries have participated to a lesser extent in CARTAC’s activities. A few countries such as the Dominican Republic, Montserrat, Haiti and Suriname, have not, with the exception of regional conferences, training sessions and seminars, participated at all, though it is understood that all four countries have done so since.

  • 10. The reasons that some countries have not used CARTAC as much as others include:

    • Poor communications between government departments as to the availability of CARTAC support;

    • A perception that support is primarily for English speaking countries;

    • An initial reluctance to participate because of CARTAC’s association with the IMF;

    • A perceived lack of ability to absorb technical assistance;

    • Pressing problems preventing the country from developing a coherent request for assistance;

    • A perceived sense of not quite being ready to make a formal request; and

    • Having a well developed reform program ongoing and no need to ask for assistance as they have the internal capacity to proceed without CARTAC’s assistance.

  • 11. CARTAC has been responsive to the rapidly changing needs of the region.

  • 12. CARTAC takes due regard to cost in all of its activities and is efficient in providing inputs. The value of the outputs derived from CARTAC’s activities justify the costs of the inputs used to obtain those outputs. It is too soon to measure results and impacts. CARTAC is developing Indicators of Change which should help any future evaluator measure results and impacts.

Chapter 3 Management of the Cartac Office

3.1 Introduction

The purpose of this chapter is to review the management and administration of the CARTAC office in Barbados. Topics covered include: (a) the management and administration of individual activities; (b) budgets and financial accounts; (c) filing systems and standard operating procedures; and (d) the system of reports.

3.2 General

The Government of Barbados has provided CARTAC with office accommodation in a new building. The layout of the office is pleasant and suits CARTAC’s needs.

The staff compliment in the CARTAC office consists of the Program Coordinator, five long-term Advisors, an economist, two administrative assistants, a financial assistant and a general assistant. The Program Coordinator, one administrative assistant and the general assistant are paid by the IMF. The economist is seconded from the CDB. The rest of the staff is paid out of CARTAC project funds. The reviewers spent a week in the CARTAC office and were able to observe that morale is high.

The administration of each project follows a similar pattern. Preliminary discussions mapping out the general parameters of the project are held with the participating country. Next, a senior level official in the participating country formally requests assistance in writing. This request starts the activity administration in CARTAC and results in the opening of a project file, the preparation of a work-plan and budget and the opening of a financial account and an attendant budget. The activity is executed, including the hiring of short-term consultants where necessary, arranging participants’ travel and accommodation etc and the costs are recorded. There is an end of project report which is included in the files. Contracts for short-term consultants usually consist of a one-page contract letter and the terms of reference. The files for training projects are more copious and often include copies of the training material and the participants’ evaluations of the activity. The Back to Office Report, prepared by the trainer, is a good record of the rationale for the training program, the approach taken in the course, the trainer’s evaluation of the effectiveness of the course and the trainer’s summary of the participant’s evaluation of the course.

The reviewers read a large number of the formal CARTAC files and found them in order. The reviewers also read a large number of the working files of the individual long-term Advisors to gain greater insights on individual activities. The reviewers found that the approach taken in developing individual activities to fit into a country strategy and work-plan, was logical in both its development and execution and to be generally of a high professional standard.

3.3 Budget and Accounts

The current budget of CARTAC, based on pledged funds minus the 3% and 10% retained by the UNDP and IMF respectively, is USD9,923K.

Table 5 shows both the dollar amounts and the percentage of total cost by major categories of expense. This table demonstrates that:

  • The five long-term Advisors, plus attendant travel costs, absorb some 42% of the budget;

  • Short-term Advisors absorb another 34% of which most, 24%, is used for Tax, Customs and Financial Services short-term Advisors;

  • Some 18% of the total budget is for training, seminars and professional attachments; and

  • Only 2.4% of project funds are used for administrative and other items, because the IMF provides considerable administrative support from Washington, finances most of the equipment and communication costs, and pays for two of the support staff in the CARTAC office.

Table 5

Allocation of Funds by Major Line Item

article image

Table 6 shows how the $9,923K has been allocated over the five calendar years that the current phase is expected to operate:

  • Some 90% of funds will be spent in the three substantive years of the current phase, 2002 to 2004 inclusive; and

  • The year 2003 will likely be the year that most funds are spent.

Table 6

Budget Expenditures by Year

article image

The reviewers looked at the system of accounts and found them clear and transparent. Some costs are paid out of Washington and some costs are paid out of CARTAC. This naturally results in some delays in reconciling activity accounts. While not a major issue, the reviewers recommend that the IMF and CARTAC review the situation with a view to ensuring that more current expenditure and commitment data is made available to CARTAC for Washington incurred disbursements.

3.4 Administration

Much of the administrative procedures in CARTAC revolve around the individual activities. The procedures are simple and straightforward. No activity can formally start without a written request from the host government. Typically an action plan and budget is prepared by the long-term Advisor and discussed with the Program Coordinator. Once approved, an account is opened by the financial assistant and a file is opened by the administrative assistant. If short-term Advisors are needed they are selected and then contracted using a simple one-page contract letter. Any missions to a country by a long-term Advisor require a standard “Back to Office Report” at the end of the mission which summarises the principal findings and conclusions of the mission and the next steps for follow up work. All expenses associated with the activity are recorded by the financial assistant. The procedures for activity management and administration are clear and work.

The reviewers also read the written Standard Operating Procedures document for other administrative tasks within the CARTAC office. They are clear, precise and appropriate for a small office.

3.5 Reports

CARTAC and its long-term Advisors prepare three sets of reports: (a) Back to Office Reports prepared at the end of a mission by a long-term Advisor or a short-term consultant; (b) monthly reports prepared by each long-term Advisor that are submitted to the backstop person or group in IMF HQ in Washington; and (c) six-monthly Work-Plans and Activity Reports for the Steering Committee.

The Back to Office Report prepared by the long-term Advisor or short-term consultant at the end of each mission provides a good summary of what was accomplished on the mission and an update of the action plan going forward. The reviewers read many of the files and can confirm that they provide a clear record of progress on an activity.

The monthly reports to Washington provide detailed information on the activities undertaken, raise technical issues and seek advice where needed. They also often contain appended material such as Back to Office Reports. The reviewers spoke to IMF staff in Washington about the reports and they expressed satisfaction with their format and content. The reviewers read several of these documents and found them professional and conclude that they serve the purpose that they were intended to serve.

CARTAC keeps an electronic copy of each Advisor’s Back to Office Report and monthly report to Washington. The reviewers asked for and received an electronic copy of all of these documents. They filled 17 floppy diskettes. There is no shortage of information available to the interested reader.

The third set of reports that are prepared by CARTAC are the Work-Plans and Activity Reports that are presented to the six-monthly Steering Committee meetings. These reports are a summary of all of the activities undertaken over the preceding six months and of activities that will be undertaken over the following six months. The reviewers read these documents twice. The first time in Canada prior to undertaking the field missions as a means of briefing themselves and the second time after completing all the field work. Needless to say, the reviewers got a lot more out the second reading because they knew what lay behind the summaries. And there lies the dilemma: the closer one is to the Centre the more one understands the context and coherence of the individual activities and the further one is away from the Centre the more difficult it is to see the coherence. This observation was made by a few people, not so much as a problem, but rather as a challenge. The reviewers believe that the answer is not to write more detailed Steering Committee reports. Rather, if there are individuals or organisations that would like more information on specific activities or groups of activities they might consider asking for some of the back up documents prepared by CARTAC where this does not intrude on the confidentiality concerns of the recipient, or for a brief oral explanation from the long-term Advisors.

3.6 Summary Findings

  • 1. The procedures for managing individual activities are simple, transparent and effective.

  • 2. The budget, provided that all pledged monies are made available to the Centre, is sufficient for the life of the three-year current phase of support for the Centre.

  • 3. The Standard Operating Procedures for office administration tasks are clear, precise and appropriate for a small office.

  • 4. The three sets of reports prepared in CARTAC suit the purposes for which they are prepared.

  • 5. The CARTAC office in Barbados is well-managed.

Chapter 4 Cartac Governance

4.1 Introduction

The purpose of this chapter is to review the governance of CARTAC. In particular, the reviewers have considered the role of the UNDP, IMF, Steering Committee and the Technical Panels.

4.2 Undp Role

The UNDP has performed a number of functions in setting up and managing the project:

  • Drafted and signed, along with IMF and CARICOM, the Project Document;

  • Drafted and signed a number of bilateral agreements with donors;

  • Received funds from donors and participating countries and forwarded them to the IMF as needed;

  • Recorded and allocated annual expenditures to the various donors and participating counties contributions;

  • Reported annual expenditures to donors and participating countries; and

  • Participated in the biannual meetings of the Steering Committee.

In addition to the above mentioned functions, the Project Document stated that “a Program Support Unit (PSU) for the Centre would be housed at the UNDP in Barbados, and would be responsible for the logistics and procurement of training venues, travel of experts and training participants, equipment procurement, and payment of appropriate allowances”. The Project Document also said that the program would be subject to an annual joint review by representatives of governments, UNDP, IMF, and cost-sharing partners.

In the event, the functions envisaged for the UNDP’s PSU were undertaken by staff at the CARTAC office in Barbados. This was a very sensible adjustment as it keeps administrative service providers in close proximity to the users of those administrative services. Secondly, the annual tri-partite reviews have been replaced by the 18 month cycle of plenary sessions of the Steering Committee, the last of which commissioned this mid-term review. Thus, once CARTAC was set up, the role of the UNDP is to accept, record and hold donor funds, forward funds to the IMF as required and allocate expenditures on an annual basis to the various donors. There are a number of issues associated with these roles.

It is standard UNDP operating procedure in projects such as this that the investment income on funds that are provided to the UNDP for later disbursement goes into a general project account for funding all UNDP projects. It does not go into the account that the money was originally intended for. This policy makes sense for small projects where money is provided in tranches. The reviewers question whether it makes sense in this particular case where CIDA provided USD5,000,000 up front for a 3 year project. A rough approximation of the average amount of Canadian donated funds in the project over the 3-year project life is USD2.5 million. Invested at 2% per annum, 3% per annum or 4% per annum would yield over the life of the project USD150,000, USD225,000 or USD300,000 respectively. The reviewers were told that the UNDP would reinvest interest thus earned into the specific project if so requested by the donor. The reviewers recommend that Canada consider making such a request.

A second issue relates to the annual allocation of expenditures to the donations from the donors and the participating countries. In the absence of other instructions, UNDP allocates expenditures on a pro rata basis to all balances that it is holding. This is a rational default position all other things being equal. But all things are not equal in this case. Canada put in its total contribution at the beginning of the project whereas other donors are putting in funds in two or more tranches. These donors have to have proof that their funds have been expended before they can be replenished. In these circumstances it makes sense to allocate all expenditures to this latter type of donor and little or none to Canada until all of the tranches have been paid into the project. An added advantage of this is that the Canadian money would be invested for a longer period and CARTAC would earn more money on it. The reviewers recommend that the UNDP accelerate the recording of expenditures against funds from donors who are donating in tranches. The reviewers also recommend that the UNDP allocate funds on a six-monthly basis until this issue is regularised.

A third issue is that there are significant delays, in some cases of several months, between when funds are received by UNDP and when they are recorded in the financial system. It is thus difficult, at any given moment in time, to get an accurate picture of the status of total contributions.

4.3 IMF as Executing Agency

The Project Document states that the Program Coordinator, a full time IMF employee, will provide leadership and direction in administering the project’s activities and maintaining effective relationships with the participating countries and donor and regional agencies. The Project Document also stated that the IMF’s technical assistance departments and Western Hemisphere Division (WHD) would provide backstopping to ensure quality control and adherence to international standards.

The IMF has responsibility for international standards in the technical areas covered by CARTAC. The IMF’s main responsibilities are to: (i) support the Program Coordinator and the long-term Advisors with advice, information, documentation and other technical materials; (ii) ensure that in the recruitment of staff and short-term experts the highest standards of expertise, experience and professionalism are maintained and; (iii) ensure that the Centre’s work programs are consistent with the project’s design and are technically sound. The IMF fulfils its responsibilities largely through correspondence between CARTAC and the IMF’s headquarters. IMF staff, on a monthly basis, monitor the work of each expert. The expert submits to the responsible IMF technical backstopper a monthly report for comment and guidance, as necessary. The Program Coordinator and the long-term Advisors send regular reports on their work plans and activities to the WHD and the technical assistance providing departments such as Fiscal Affairs, Monetary and Financial Systems, and Statistics. They also send ad hoc communications on specific matters as they arise.

Based on their review of the monthly reports and discussions with the long-term Advisors and their backstoppers in Washington the reviewers conclude that the system is working well. Washington allows the long-term Advisors a very large degree of autonomy while still monitoring them and providing support as needed. The TA department have exhibited a considerable degree of confidence in the work of the long-term Advisors and as a result have made relatively few comments on their work program and monthly report. HQ noted that the long-term Advisors have been responsive to the needs of the countries, have been addressing immediate issues and that this is generally supportive of the work of the area departments. There is no evidence that HQ is pushing its agenda on the long-term Advisors.

The IMF has done a good job from the technical point of view:

  • The IMF’s selection of the Program Coordinator, the long-term Advisors and short-term consultants was first rate;

  • The IMF, while monitoring on a monthly basis, has wisely decentralised the operations of the Program Coordinator and the long-term Advisors;

  • The Program Coordinator and the long-term Advisors have all established good personal relationships with their constituents and have exhibited an ability to see the needs of the participating countries;

  • The IMF’s selection of the Program Coordinator and long-term Advisors has been widely praised by participating countries, donors and regional organisations. The Program Coordinator and the long-term Advisors have earned the trust of all in the region and this reflects favourable on the Centre; and

  • It is difficult to imagine that any other organisation could have been a better technical Executing Agency.

Despite this technical competence, there have been certain potential tensions. The reviewers were told by many participating countries that there was initially concern that the IMF might attempt to introduce a proactive program through CARTAC. This concern was quickly put to rest by the actions, attitude and performance of both CARTAC and the IMF. CARTAC is perceived as a stand-alone organisation that exists in its own right and is not seen as an IMF project. If it were the reviewers do not think it would have been as effective as it has been. If it were ever seen to be an arm of the IMF in the future then the relationships between CARTAC and the participating countries could change drastically.

A second area of potential tension at the outset was the possibility of a misunderstanding regarding the role of the Steering Committee and the role of the IMF in setting CARTAC policies, programs and work plans. As seen above, this misunderstanding did not arise and everyone accepts the Steering Committee’s competence in this area.

A third possible area of tension revolved around extending the contract of long-term Advisors in the event that either the Steering Committee or the IMF questioned the need for their continued employment. Such a situation arose in early 2003 and its handling demonstrated that the IMF did not fully understand the role of the Steering Committee in such matters. At the February 2003 Steering Committee meeting, the IMF accepted that it had not been fully cognisant of the role of the Steering Committee, but stated that it was not the IMF’s intention to be heavy-handed or to avoid consulting with the region and the Steering Committee. On the contrary, the IMF said that it was fully committed to the principle of regional ownership and was anxious to consult as widely and as fully as possible. It is judgement of the reviewers that all players have learned an important lesson from this difficult situation and that a similar problem is unlikely to occur again.

4.4 Steering Committee

The Project Document, signed by representatives of UNDP, IMF and CARICOM stated that: (a) the Steering Committee, composed of officials of participating governments, regional institutions, and cost-sharing partners, will be responsible for setting the strategic direction of the Centre; (b) the Steering Committee will meet to review six-monthly work plans, ensuring that the program stays on track to achieve its goals and outputs; (c) the Steering Committee will delegate day-to-day program implementation responsibilities to the Program Coordinator; and (d) the Program Coordinator will serve as Secretary to the Committee.

The Steering Committee works well due to the skilled guidance and professionalism of the Chairperson. CARTAC produces semi-annual work-plans and semi-annual activity reports which are approved by the Steering Committee. These are well prepared and the Steering Committee members expressed their thanks and appreciation for the good job done by both the Chairperson and Secretary.

The reviewers do not see the necessity to change the modus operandi of the Steering Committee in any significant way. However, the reviewers would like the Steering Committee to give consideration to the following points.

One or two members mentioned that the format of the reporting of individual activities left them a little uneasy as they could not always see how the individual activities fit into “the big picture”. This is unfortunate as discussions with the long-term Advisors, a review of the files and a review of the Back to Office Reports and the long-term Advisor’s monthly reports to the IMF, clearly demonstrate the coherence of the individual activities. The reviewers are loath to recommend any increase in the length of the current reports. Rather, they would like to suggest that any individual or committee member who feels that they would like a further explanation on any point contact the Chair and/or the Program Coordinator prior to the meeting so that supplementary material could be produced on specific topics. This would not, of course, prevent any member from raising any further questions at the Steering Committee meeting itself which could then be addressed by the Program Coordinator or the long-term Advisors at the meeting.

A second area that may require some further consideration is the composition of the Steering Committee. The reviewers are fully cognizant of the thinking behind the current size and composition of the Steering Committee: The strong desire to limit the size of the Steering Committee and to have participating countries and regional bodies more heavily represented than bilateral and multilateral donors. The reviewers fully endorse the thinking behind this policy but recognise that it does not apply to the plenary sessions and that it results in both permanent members and groups of organisations being formed that are represented by just one of the members on an annual rotation. This latter has had the further irony of some groups being represented by bodies that had pledged funds but which had not yet actually contributed funds. The reviewers do not argue for making any changes to the composition of the Steering Committee for the remaining 18 months of the current phase. Rather, if there is a second phase of CARTAC, the reviewers suggest that the question of the number and representation of the Steering Committee be revisited. One possibility might be that, on the assumption that all current donors would want to continue funding this successful project, all those donors that provide some minimum amount of money, perhaps 10 or 15% of total funds, would have a permanent seat at the table.

One final area that the Steering Committee might want to consider is providing further guidance to CARTAC on setting priorities for CARTAC’s program. This issue was raised by several people that were interviewed. In discussing it with CARTAC, it is clear that CARTAC as a whole and each individual long-term Advisor employs a set of priorities in choosing activities. Some of these include subject matter, whether it is part of a bigger governmental or sub-regional program, who else is involved, cost of activity, amount of time involved etc. Perhaps now is the time to make the process more formal for greater transparency, improved allocation of resources if demand outstrips supply and protection for CARTAC. If this exercise is undertaken, it is important to remember that one of the great strengths of CARTAC is that it can respond quickly to emerging issues with high quality resources. Any prioritisation exercise must be flexible enough so that it does not constrain CARTAC from making rapid interventions when a critical need arises.

4.5 Technical Panels

The Steering Committee, in its meeting of February 28, 2002, created four Technical Panels to provide guidance to the long-term Advisors.

Each of the four Technical Panels consists of five to seven persons selected by the Steering Committee who are knowledgeable about the issues within the region. The purpose of the panels is to serve as “sounding boards” on technical matters for the CARTAC advisers and to add input to the work of advisors. It was agreed that each panel would decide on its own modus operandi, including the selection of a convener or chairperson. It was anticipated that, for the most part, the panels would operate in a “virtual” mode through the Internet.

Prior to the formation of CARTAC and the Technical Panels there already existed, in the core areas of CARTAC’s work, a number of regional professional and technical arrangements or mechanisms for exchanging information on recent developments, plans and prospects and, in some cases, for promoting regional cooperation in furthering improvements. Several of these involve donor agencies as well as participating countries.

It was clear from the outset that CARTAC would need to work closely with such networks and that it would need to ensure that, at the technical level, it took into account what had been tried before, the constraints imposed by lack of human and financial resources, constitutional or legislative impediments, or by what is politically and socially acceptable. It would also need such contacts to help tap into the pool of expertise that exists in the region in the specialised technical areas to be covered.

During the course of their in-country interviews the reviewers met several members of the Technical Panels and, based on these discussions and their discussions with the long-term Advisors, they derived the following findings.

Panellists were generally unaware of who were the other members of the panel and none of the technical panels has had a face-to-face meeting and communications have generally been by e-mail. Several members of technical panels indicated that they were uncertain of the role of the Technical Panel and their personal role. While they received material from time to time, and sometimes made comments, they were uncertain as to what was expected of them and how their comments were being factored into various plans. They did not receive the comments of other panellists and so did not have a feel for the relevance of their own comments. As a result there is a lack of synergy. Other members of the Technical Panels stated that they had never been asked for any comments or input.

The long-term Advisors stated that the Technical Panels had not worked out as planned and that the response from panellists varied: some responded rapidly whereas others did not respond at all. The long-term Advisors agreed that the technical panels were a good idea and said that it was very important that they receive the broad kind of advice envisaged from the Technical Panels. They did note, however, that other networks of practitioners are being formed in specific areas and these now provide guidance.

The reviewers conclude that the Technical Panels, while a good concept, have not performed the function that was originally envisaged for them. The reasons for this include:

  • The panels have never met as a group and are not really aware of their role;

  • Other networks have developed which have partially filled the need for the advice originally envisaged from the Technical Panel; and

  • Panellists are busy with their own daily jobs and, in small countries, there are lots of demands on key personnel.

The reviewers recommend that a further effort be made to improve the work of the Technical Panels. Each long-term Advisor should arrange a face-to-face meeting with the members of the panel to discuss the program, his or her expectations and obtain feedback from them.

4.6 Summary Findings

  • 1. There are a number of issues that the UNDP should review including the disposition of interest on donated funds, the allocation of expenses to various donors, and the speed of recording donations.

  • 2. The IMF has done a good job as the Executing Agency: It has hired the right people and, while monitoring their work, has not been heavy handed. Most possible tensions have been avoided and lessons have been learned from the one that was not.

  • 3. The Steering Committee is recognised by all players as the principal governance body of the Centre. It has been effective. The question of membership may need to be revisited in any follow-on phase of CARTAC.

  • 4. The Technical Panels have been less effective than originally envisaged. Further efforts are required for them to reach their full potential.

Chapter 5 Other Macro Issues

5.1 Introduction

The purpose of this chapter is to address some of the macro issues that have not been addressed elsewhere in the report. These include: (a) regional approach to technical assistance; (b) capacity building and sustainability; (c) the question of whether CARTAC should be reactive or proactive; and (d) regional support for CARTAC.

5.2 Regional Approach to Technical Assistance

The IMF has historically delivered technical assistance through: (a) short-term visits by one or more experts; and (b) by providing longer-term experts who reside in the country. Both the short-term and the long-term experts provide advice and hands-on support. In addition to providing these short-term and long-term experts, the IMF, out of HQ in Washington, DC, also provides technical and diagnostic reports, training courses and seminars. More recently, the IMF has started providing assistance through regional technical assistance centres. The first of these centres, the Pacific Financial Technical Assistance Centre (PFTAC), was established in August 1993. The experience of PFTAC helped to model the structure and modus operandi of CARTAC. More recently, two other regional technical assistance projects have been created; one in East Africa and the other in West Africa.

The desire for a regional approach to providing technical assistance in the Caribbean relates to the fact that the countries of the region have a number of similarities, small populations, small island states, and small and open economies, which are heavily dependent on one or two industries such as agriculture and tourism. Most of the economies are vulnerable to changes in external markets and severe shifts in terms of trade and are prone to natural disasters such as hurricanes. Individual donors have been hard-pressed to meet technical assistance requests, a significant part of which could not be provided in a timely manner. In several cases when assistance was provided, the approaches used produced less than optimum results and benefits derived were not sustainable. This diagnosis, plus the fact that there are potential economies of scale when dealing with a large number of similar economies, resulted in several donors expressing support for the creation of a regional technical assistance centre in the Caribbean. Regional programs have the potential to develop a stock of knowledge that can be used to improve the efficiency of the delivery of similar services across a variety of countries. A regional approach also offers the potential to develop a resource base of materials and expertise that can be supplied to interested parties.

Carefully designed attachments and regional seminars and workshops can be an effective form of training, by exposing participants to well-developed techniques and by assisting the development of networks of skilled individuals within the region. An important aspect of regionalisation is networking and the bringing together the heads of a variety of different organisations within the area, many for the first time. An important example of this has been the bringing together, on several separate occasions, the accountants general, permanent secretaries of finance, the directors of finance and the supervisors of insurance, for the first time, to discuss training and sharing of resources, knowledge and skills. This facilitates attachments, workshops and seminars and leads to mutual benefits for all participants. Many of the people the reviewers interviewed felt that the ability to share experiences with regional counterparts was one of the more subtle yet powerful positive outcomes of CARTAC.

Although CARTAC does not aggressively promote regional solutions to all issues, largely because they would often be inappropriate, activities such as arranging meetings and seminars of senior personnel implicitly fosters regional cooperation and solutions because people get together and get to know each other. CARTAC has implemented several projects that have wide regional interest and application including corporate governance, budget reform initiatives, VAT applications, and Smart Stream information technology solutions. All of these activities have resulted in significant networking benefits. CARTAC has also provided some assistance to the private sector by providing seminars on emerging issues of international and regional importance, including corporate governance and audits of financial institutions.

A regional approach presents the advantage of flexibility when applying resources according to changing country circumstances. It would be difficult to attract the high quality of technical assistance for a small country that would be available under a regional program. In a small country such expertise would likely be considerably underutilised and would be prohibitively expensive. Successes in a particular country in a region could be a model which could be a showcase for the whole region. Another major advantage of having people in the field is that it enables CARTAC to build a partnership with both the beneficiary countries and the donors that sit on the Steering Committee. It allows CARTAC to respond to countries’ technical assistance needs in a flexible and timely manner.

The increasing integration of the region in the global economy has created a number of challenges that must be addressed if the countries are to achieve faster and more sustainable growth. The work of CARICOM is important in this regard in its efforts to create a Caribbean Single Market and Economy. In as much as CARTAC exists to assist countries improve their capacity to adapt and adopt internationally agreed codes, standards and best practices of economic and financial management, its country and regional activities should support the convergence and integration objectives of the Caribbean Single Market and Economy (CSME). CARTAC has participated in the CARICOM Roundtable of Financial Services Regulators and was working with the regional regulators to harmonise and standardise the supervision of banks, insurance companies and securities.

The reviewers were told in discussions with the Office of Technical Assistance Management (OTM) of the IMF in Washington that CARTAC’s initiatives complement the technical assistance from headquarters. This helps the Fund to better tailor its overall technical assistance to the region. In summary, intermittent but consistent and ongoing inputs would be harder to deliver within the context of a series of national programs.

The greatest potential disadvantage of regional programs is that resources may be spread too thinly, marginalising the impact in any one country. This has not occurred in CARTAC so far but is an issue that should bear watching as the demand for CARTAC’s services increase. Attempting to push regional solutions on all countries can sometimes lead to difficulties and may lead to resentment. Perhaps suggesting a regional approach and allowing countries to decide what aspects of that regional approach is appropriate for it should be the preferred course. The assistance of CARTAC with regional harmonisation would be welcome.

CARTAC needs to continue to keep in mind that not all of the countries in the region are the same or at the same level of political and/or economic development. The OECS is a sub-region, whatever is done in one country, should wherever appropriate, be applied to all countries. This is particularly relevant and important to the integration of policies.

CARTAC has become so important in the area that it should be invited to make presentations to the Council on Finance and Planning (COFAP). Many of the Ministers of Finance who attend these sessions are also Heads of Government. The chairperson of the Steering Committee, who already reports to this body, could make the presentation. A similar presentation could be made to the Meeting of Officials, which is held preparatory to the COFAP meeting.

5.3 Capacity Building and Sustainability

The Project Document noted that “The economic strategy in the region puts a premium on improvements in economic and financial management, the quality of which is constrained in most cases by a shortage of technical and institutional capacity and weaknesses in the statistical base. The problem is exacerbated by the inability of most governments to retain skilled staff in the public service.” It is clear from this statement that the implicit assumption is that CARTAC’s activities will focus on building up technical and institutional capacity in the areas under its mandate. Both Mr Kohler of the IMF and Mr Owen Arthur, Chairman of CARICOM, in their speeches opening CARTAC, emphasised that capacity building was a major thrust of CARTAC.

The principal reason why capacity building is so important in the region is that there is often very little depth of skill in many of the organisations. Many of the jurisdictions have extremely limited resources. In too many instances a key individual has to undertake a variety of important tasks since there is no one else with the requisite knowledge and skills. All too often in such circumstances, opportunities for training and enhancement are limited as it is difficult to release such an individual from the job because there is no backup or the individual is too busy with operational issues to leave. The sustainability of organisational capabilities is plagued with problems associated with an extremely narrow human resource base. This problem is particularly acute in the OECS countries. Organisations in those countries have a very narrow base of skills and the absence or departure of one or two people can have a dramatic impact on operational capabilities.

The recognition of the human resource constraints in many of the countries requires an explicit acceptance of the fact that for many countries there will be an ongoing need for external technical inputs. This suggests that CARTAC will be a source of technical advice for some time to come. Intra-regional solutions may be found to some problems, and persons with training and experience in one country will be able to assist in resolving issues in another country. Some of this is already occurring. Some commentators suggested a greater use of local consultants. The argument being that the use of Caribbean consultants wherever possible will assist in broadening knowledge and experience in the region. These persons know the region and are able to bring a certain perspective based on their knowledge of the issues and culture. Moreover, in building up local expertise, the capacity, knowledge and skills in the region will be enhanced.

In this regard, the regional seminars of senior officials that have been organised by CARTAC and relating to public expenditure management, statistics, taxation and financial sector issues along with professional attachments are highly beneficial. All the countries in the region have taken significant steps to strengthen macroeconomic management. As economic and financial management becomes more sophisticated there will be need for greater capacity to carry out the new and improved tasks. So the demand for training will not likely diminish for some time.

The reviewers believe that the capacity building and utilisation exercise in the region would benefit from a region-wide database of courses and individuals who attend courses and are similarly qualified.

5.4 Modus Operandi of Cartac: Reactive or Proactive

There is a quiet but ongoing debate on whether CARTAC should reactive or proactive. The current modus operandi of CARTAC is that it supplies assistance in response to the receipt of an official request. This is correct and proper. In that regard, countries feel ownership of the initiatives and have an incentive to succeed. Countries in the region would likely react quite negatively if CARTAC developed and then pushed its own agenda. However, it would be quite wrong to assume that there is not a significant proactive component in all that CARTAC does. At the early stages of the development of a program in a particular country, CARTAC, in conjunction with the participating country undertook a diagnosis of the issues that needed to be tackled. CARTAC was perfectly free to provide guidance on what should and should not be tried at this stage and the subsequent program was often strongly influenced by CARTAC’s inputs.

At the February 2002 Steering Committee meeting the Program Coordinator asked the Committee’s views on the extent to which CARTAC should be proactive in supporting, or even initiating regional approaches to capacity building involving sharing of resources, or creation of “common services”. The Committee agreed that CARTAC should be proactive, particularly in helping conduct diagnostic analyses in the statistical and public expenditure management areas, as well as in actively “marketing” its services.

Regardless of the proactive component in a generally reactive approach, a question has been raised as to whether the request mechanism is likely to produce the most effective use of resources. Some feel that CARTAC should be more proactive in publicising itself or pressing countries that are not using its services to do so. Others have been more specific in suggesting that CARTAC should be proactive, by determining the key macro issues, developing a program for resolving them and then getting the participating countries to buy into the CARTAC program. In fact, the reviewers think that CARTAC has an element of all of these approaches with the balance struck being just about right. Were CARTAC to attempt to become more proactive than it currently is, it would need to be done with considerable finesse. There is considerable danger that it could change CARTAC’s image in the region if it appeared that CARTAC had become an enforcer of IMF goals in the region. CARTAC has built up a good reputation for cooperation and has gained acceptance by being seen as an agency that supports the needs of the individual countries and not as an agency that pushes the IMF regional agenda. In the process of solving a problem CARTAC may become aware through its experience of a larger need for structural or procedural reform. CARTAC should make a recommendation to the jurisdiction as to how to proceed further to avoid the problem or prevent it from becoming a bigger problem. When CARTAC comes across or solves a problem of reform they can pass on the solution to other projects.

CARTAC’s reactive stance has led to a concentration of resources in a few countries. This may be in recognition of the fact that some countries had needs that were more critical than others, that they had a plan and knew what they wanted and that CARTAC came along at the right time. Urgent issues have been attended to and over time there will be less concentration of resources as other jurisdictions become more aware of CARTAC’s assistance and are able to mobilise their own efforts.

5.5 Participating Country Support

Governments in the region have all been supportive of CARTAC’s activities. In those countries that have been the heaviest users of CARTAC’s services the governments are among the most enthusiastic supporters. But even those countries that are not yet very active users of CARTAC indicated that they recognise the benefits that CARTAC is providing, are very supportive of its activities, and are anxious for them to continue.

One indication of support and commitment is the financial or in-kind contributions that governments make to the Centre. Another relates to the reaction of governments to recommendations made by CARTAC. In a number of instances legislation and regulations have been put in place to ensure implementation of these recommendations.

In addition, despite rather limited resources, governments have cooperated with CARTAC by providing experts to assist in seminars, workshops and indeed hands-on assistance to other countries. They have also freed up senior personnel to attend courses, seminars and Steering Committee meetings. The number of staff that attend training activities is also a measure of support and commitment by governments, as is the cooperation with CARTAC in the establishment of important and successful regional networks such as found in Public Expenditure Management. Some governments have also assisted CARTAC’s training program by facilitating attachments to their own departments or agencies of staff from within the region.

5.6 Summary Findings

  • 1. Delivering technical assistance on a regional basis by having long-term Advisors who are based in the region has worked.

  • 2. CARTAC is always aware of the desire for a regional approach. Its working rule is that where an activity is best delivered on a regional basis, then that is how it will be delivered. Likewise, where an activity is best delivered on a national basis that is how it is done. Any attempt to apply a regional solution to a problem that was essentially national would likely be counter productive.

  • 3. Many of CARTAC’s training sessions and seminars have exposed participants to issues that occur in other jurisdictions. This has led to a greater acceptance to seeking regional solutions where they make sense. Regional secondments and attachments are a good example of how regional thinking is expanding.

  • 4. Lack of capacity and sustaining the capacity that does exist is a major problem in the Caribbean region. CARTAC is helping to increase capacity in individual countries and, by using local experts, increasing the technical capacity in the region. Many countries prefer regional consultants because they know the regions and engaging them increases regional capacity.

  • 5. Regardless of CARTAC’s good work, both the donors and the participating countries must accept the fact that good people in government are likely to be enticed into the private sector with more lucrative offers. As a result, capacity building is a long-term, on-going activity.

  • 6. CARTAC is reactive in the sense that it waits for participating countries to approach it for assistance for technical assistance. On the other hand it is quite proactive in helping devise strategies and action plans for governments with which it is working.

  • 7. If CARTAC became more proactive in the sense of developing its own agenda and then pushing that agenda, this would likely be resented by the participating country and would possibly prove to be counter-productive. CARTAC has struck the right balance between being reactive and proactive.

  • 8. Participating countries have been fully supportive of CARTAC.

Chapter 6 Summary Findings and Recommendations

6.1 Summary Findings

  • 1. There was a clear need in the Caribbean region for increased technical assistance in the area of economic and financial management, the CARTAC project was well-designed to meet that need, and the CARTAC project has been well executed. CARTAC has played a significant and positive role in the region and is much appreciated by the participating countries.

  • 2. CARTAC has become well integrated within the region, fills a niche that cannot be filled by other projects, and performs an important role of helping coordinate technical assistance of other donor projects in the region. CARTAC’s activities are of high professional quality. They are timely, appropriate and well executed.

  • 3. Its success is in large part attributable to its flexibility and its quick response to requests for assistance. The lack of bureaucracy in making requests has been a contributing factor. There are neither extensive forms to complete nor complex proposals to write to justify a request. A letter of request has been the usual form.

  • 4. CARTAC’s task has been facilitated because the region has been welcoming. Recipients have been uniformly satisfied with the quality of assistance. Interviewees with first hand knowledge of some of the training courses or advisory missions undertaken expressed their satisfaction with the speed and quality of assistance provided. The ability to share experiences with regional counterparts has been one of CARTAC’s more important contributions.

  • 5. Ultimately, any technical assistance program is only as successful as the people engaged in its operation. No amount of elaborate project identification, formulation, and monitoring employed by TA providing agencies can guarantee results if the staff, experts and counter-parties involved in its implementation are unsuitable or uncooperative. CARTAC has been fortunate in having good experts selected for its core staff, having the resources available to make use of the skills and talents that abound within the region and elsewhere, and having the resources available to make use of the region’s cadre of highly professional and committed civil servants.

  • 6. The approach taken by CARTAC in developing its program of interventions and activities has been both logical and methodical. The individual national and regional work programs thus developed are both coherent and comprehensive. The way that CARTAC does business is not intrusive and this contributes to its success and acceptance. CARTAC has struck the right balance between regional activities and individual country activities.

  • 7. CARTAC interventions appear to be most successful when they are fully integrated into a strategy and work-plan that is developed and owned by the participating country. No activity should be undertaken unless it is requested by an individual government or regional body. CARTAC has been and should continue to be primarily reactive rather than proactive so that all activities are owned by the participating country or the region. CARTAC should continue to concentrate on short-term projects. In doing so it will remain flexible and not get involved with activities that would tie up its resources in the long term and limit its flexibility.

  • 8. The combination of CARTAC’s planned, strategic long-term activities and its short-term one-off fire-fighting activities is balanced and appropriate. CARTAC interventions have taken into full account the activities of the national government, bilateral donors, and multinational agencies. CARTAC becomes involved in a particular activity only if it the most appropriate agency to undertake the activity.

  • 9. CARTAC’s activities have been concentrated on OECS countries, Barbados and Guyana. Most other countries have participated to a lesser extent in CARTAC’s activities. A few countries have not, with the exception of regional conferences, training sessions and seminars, participated at all, though it is understood that these countries have done so since.

  • 10. The reasons that some countries have not used CARTAC as much as others include:

    • Poor communications between government departments as to the availability of CARTAC support;

    • A perception that support is primarily for English speaking countries;

    • An initial reluctance to participate because of CARTAC’s association with the IMF;

    • A perceived lack of ability to absorb technical assistance;

    • Pressing problems preventing the country from developing a coherent request for assistance;

    • A perceived sense of not quite being ready to make a formal request; and

    • Having a well developed reform program ongoing and no need to ask for assistance as they have the internal capacity to proceed without CARTAC’s assistance.

  • 11. CARTAC has been responsive to the rapidly changing needs of the region.

  • 12. CARTAC takes due regard to cost in all of its activities and is efficient in providing inputs. The value of the outputs derived from CARTAC’s activities justify the costs of the inputs used to obtain those outputs. It is too soon to measure results and impacts. CARTAC is developing Indicators of Change which should help any future evaluator measure results and impacts.

  • 13. The procedures for managing individual activities are simple, transparent and effective.

  • 14. The budget, provided that all pledged monies are made available to the Centre, is sufficient for the current phase, though it is understood that very recently additional demands within the OECS sub-region will require some degree of “special purpose” funding.

  • 15. The Standard Operating Procedures for office administration tasks are clear, precise and appropriate for a small office.

  • 16. The three sets of reports prepared in CARTAC suit the purposes for which they are prepared.

  • 17. The CARTAC office in Barbados is well-managed.

  • 18. There are a number of issues that the UNDP should review including the disposition of interest on donated funds, the allocation of expenses to various donors, and the speed of recording donations.

  • 19. The IMF has done a good job as the Executing Agency: It has hired the right people and, while monitoring their work, has not been heavy handed. Most possible tensions have been avoided and lessons have been learned from the one that was not.

  • 20. The Steering Committee is recognised by all players as the principal governance body of the Centre. It has been effective. The question of membership may need to be revisited in any follow-on phase of CARTAC.

  • 21. The Technical Panels have been less effective than originally envisaged. Further efforts are required for them to reach their full potential.

  • 22. Delivering technical assistance on a regional basis by having long-term Advisors who are based in the region has worked.

  • 23. CARTAC is always aware of the desire for a regional approach. Its working rule is that where an activity is best delivered on a regional basis, then that is how it will be delivered. Likewise, where an activity is best delivered on a national basis that is how it is done. Any attempt to apply a regional solution to a problem that was essentially national would likely be counter productive.

  • 24. Many of CARTAC’s training sessions and seminars have exposed participants to issues that occur in other jurisdictions. This has led to a greater acceptance to seeking regional solutions where they make sense. Regional secondments and attachments are a good example of how regional thinking is expanding.

  • 25. Lack of capacity and sustaining the capacity that does exist is a major problem in the Caribbean region. CARTAC is helping to increase capacity in individual countries and, by using local experts, increasing the technical capacity in the region. Many countries prefer regional consultants because they know the regions and engaging them increases regional capacity.

  • 26. Regardless of CARTAC’s good work, both the donors and the participating countries must accept the fact that good people in government are likely to be enticed into the private sector with more lucrative offers. As a result, capacity building is a long-term, on-going activity.

  • 27. CARTAC is reactive in the sense that it waits for participating countries to approach it for assistance for technical assistance. On the other hand it is quite proactive in helping devise strategies and action plans for governments with which it is working.

  • 28. If CARTAC became more proactive in the sense of developing its own agenda and then pushing that agenda, this would likely be resented by the participating country and would likely prove to be counter-productive. CARTAC seems to have struck the right balance between being reactive and proactive.

6.2 Recommendations

  • 1. The CARTAC project was well designed, its execution, management and governance is good, it has struck a good balance between national and regional activities and a reactive and proactive stance, and its outputs are prodigious, professional and popular in the region. The Centre is working well, nothing major needs fixing and the reviewers recommend that the Centre stay the course for the next 18 months.

  • 2. CARTAC should continue to maintain a balance between regional and national activities. CARICOM is moving the region towards a single market economy and CARTAC’s efforts could assist the region in the pursuit of the larger issues. In that regard, CARTAC, within those areas for which it has a mandate, should make every effort to assist with regional harmonisation, particularly with respect to laws, codes and practices.

  • 3. CARTAC should be invited to make presentations to the CARICOM Council for Finance and Planning (COFAP). This is in recognition of the importance of CARTAC in the region. The Chairperson of the Steering Committee could make the presentation. A similar presentation could be made to the Meeting of Officials, which is held preparatory to the COFAP meeting.

  • 4. Where CARTAC produces reports as a result of its work in the region, such reports should be made widely available to relevant bodies and governments within the region. A wider dissemination of reports would mean that more people would benefit from CARTAC’s work.

  • 5. As agreed at the February 2003 Steering Committee, there should be greater transparency of employment for CARTAC posts. With respect to future replacement of long-term Advisors, the IMF should give reasons for the proposed change and give the Steering Committee at least three candidates from which to choose.

  • 6. CARTAC long-term Advisors should continue to concentrate on small to medium sized activities, which involve at most two to three months of technical assistance. Any activity that will require more of the long-term Advisors time should be subcontracted. It should continue to keep its resources flexible and not get involved with activities that will consume a disproportionate amount of resources.

  • 7. Donors should be encouraged to coordinate even more of their efforts through CARTAC. There is considerable merit in using CARTAC as a channel for special-purpose contributions for specific projects or programs. They must, of course, be fully funded, have sufficient funds to cover incremental overhead costs and be within CARTAC’s core area of expertise. This approach would increase effectiveness, help foster donor and participating country coordination, and keep down donor and recipient overheads.

  • 8. CARTAC should consider extending a greater effort in areas of general concern or interest to the region, including: (a) the regulation and review of pension arrangements; and (b) training designed to help supervisors deal with troubled banks or insurance companies.

  • 9. In order to avoid donor overlap, particularly in the areas for which CARTAC has a mandate, there should be a central list of all projects being undertaken by donors in the region. This would assist the persons charged with the implementation of projects to coordinate their efforts. There is some sentiment in the region for an organisation like the CDB or CARICOM to undertake such a task.

  • 10. The outputs from CARTAC’s activities in the first 18 months are impressive. In the longer run, the success of the Centre will be judged on outcomes, results and impact. The measurement of these longer-term outcomes and results will be a challenge. CARTAC should be encouraged to continue the work of defining the Indicators of Change as these could be an important yardstick in measuring longer-term results.

  • 11. Some CARTAC expenses for an activity are paid out of Washington and some are paid out of the CARTAC offices in Barbados. This occasionally results in delays in reconciling expenses for some activities. The reviewers recommend that the IMF and CARTAC review the situation with a view to speeding up information flow between Washington and CARTAC.

  • 12. Canada donated USD5 million to fund CARTAC in the very early days of the project. The interest on this money has not accrued to the CARTAC project but is being used for other UNDP projects. The reviewers recommend that Canada consider making a request to have the interest on its donated funds accrue to the CARTAC project.

  • 13. The reviewers recommend that all monies expended in 2003 be allocated to those donors that are donating in tranches before money is allocated to Canadian funds. The reviewers also recommend that the UNDP try to allocate funds on a six-monthly basis until this issue is regularised. Finally, there are significant delays, often of several months, between when funds are received by the UNDP and when they are recorded in the financial system. The reviewers recommend that the UNDP speed up this process as it is thus difficult, at any given moment in time, to get an accurate picture of the status of total contributions.

  • 14. The reviewers recommend that the size and make-up of the Steering Committee remain as is for the remaining months of the current phase. However, if there is a second phase of the project, the reviewers suggest that the question of the number and representation of the Steering Committee be revisited.

  • 15. CARTAC has an informal system of prioritisation of activities although the task has not been too difficult as the Centre has not yet had a significant resource constraint. Moving forward, the demand for resources may start to outstrip the supply of resources. Perhaps now is the time to make the process of prioritisation more formal for greater transparency, improved allocation of resources, and protection for CARTAC.

  • 16. The reviewers recommend that a further effort be made to improve the work of the Technical Panels. Each long-term Advisor should arrange a face-to-face meeting with the members of the panel to discuss the program, his or her expectations and obtain feedback from them.

  • 17. Local experts should be used by CARTAC wherever possible. Regional resources are often more effective simply because they know the region. Moreover, in using and building up local expertise, the capacity, knowledge and skills in the region will be enhanced. The reviewers believe that the capacity building and utilisation exercise in the region would benefit from a region-wide database of courses and individuals who attend courses.

  • 18. Based on the above analysis the reviewers conclude that this is a successful project. Capacity building is a long-term and on-going process. It will not be finished at the end of the current phase. The reviewers recommend, therefore, that donors fund a follow on phase and that all of the donors currently funding the Centre continue that funding in the follow-on phase.