Evaluation of the Pacific Financial Technical Assistance Centre

This report presents the analysis, findings, conclusions and recommendations of the evaluation of the Pacific Financial Technical Assistance Center (PFTAC) that was undertaken between April 26 and May 14, 2004.

Abstract

This report presents the analysis, findings, conclusions and recommendations of the evaluation of the Pacific Financial Technical Assistance Center (PFTAC) that was undertaken between April 26 and May 14, 2004.

Chapter 1 Introduction

1.1 Purpose of Report

This report presents the analysis, findings, conclusions and recommendations of the evaluation of the Pacific Financial Technical Assistance Centre (PFTAC) that was undertaken by James Bucknall, Percy Allan and Kolone Vaai between April 26 and May 14, 2004.

Chapter 1 briefly describes: (a) the operations and governance of PFTAC; (b) the objectives of the evaluation; (c) the issues that were addressed in the evaluation; (d) the methodology that was used; and (e) the evaluation schedule.

1.2 Background to Pftac

The Pacific Financial Technical Assistance Centre in Suva, Fiji was jointly established in 1993 by the United Nations Development Programme (UNDP) and the International Monetary Fund (IMF) as a regional office to implement the “Fiscal and Monetary Management Reform and Statistical Improvement Project” in 15 Pacific Island Countries (PICs).

The establishment of the Centre responded to the observation that economic growth and development in PICs were hampered by the lack of technical skills and institutional capacity to formulate and implement appropriate economic and financial policies to remove structural distortions, increase efficiency, and reduce macroeconomic imbalances. The Centre's aim has been, and continues to be, to build skills and institutional capacity in member countries for effective economic and financial management that can be sustained. Toward these objectives, the Centre provides advice and training in: (a) public financial management; (b) tax administration and policy; (c) banking regulation and supervision, and (d) macroeconomic and financial statistics through responsive, ad hoc help, pro-active reform assistance and personnel capacity building.

The overall guidance of PFTAC is provided by the Centre’s Tripartite Review Committee (TPRC) consisting of representatives of all participating Pacific Island Countries, the funding agencies, and the IMF. Other technical assistance (TA) providers and relevant regional institutions are invited as observers.

Until July 2002, PFTAC operated as a UNDP regional project with the IMF as executing agency. Because of the withdrawal of UNDP as a PFTAC-donor, the TPRC approved a new project document and decided that the IMF should seek to establish a PFTAC sub-account through which donor contributions would be managed. Financing for PFTAC has been provided by the Asian Development Bank (ADB), Australia (AusAID), Japan, the Pacific Island Forum Secretariat, IMF, New Zealand (NZaid), and UNDP. Preliminary discussions have also taken place with the Republic of Korea and the European Commission on the possibility of providing financial support to PFTAC.

The IMF manages the Centre’s operations, provides its Coordinator, and recruits and supervises its resident advisors through the IMF’s TA departments. PFTAC provides TA through four resident advisors and the Coordinator, supplemented by short-term experts, training and seminar activities, and professional attachments. The current three-year financing cycle ends in April 2005.

PFTAC was subject to an evaluation by an external team in early 1997. The evaluation assessed the contribution of the project in addressing the region’s problems and capacity constraints; commented on the merits of the design of the Centre; identified lessons; and suggested pointers for the future. A Strategic Review of TA in Economic and Financial Management in PICs was undertaken in mid 2001 to assess the nature and adequacy of current levels and forms of TA, including from PFTAC, to the region and the appropriateness of current arrangements for its programming and delivery.

1.3 Purpose of the Evaluation

At its May 2003 meeting in Suva, the TPRC endorsed the proposal to conduct an external evaluation of PFTAC in early 2004. The evaluation would have three main objectives:

  • Examine if there is a continued demand/need for the range of services provided by PFTAC to PICs and that PFTAC is the most effective way of providing such services;

  • Help the TPRC in establishing medium-term strategies for PFTAC’s continued work, including also the appropriateness/adequacy of PFTACs current objectives; and

  • Assist existing and potential funding agencies in determining their level of support for PFTAC beyond the current funding cycle.

The evaluation would cover the period since the last evaluation, 1997 to date although emphasis would be placed on the more recent years. The evaluation would assess the extent to which the objectives of PFTAC have been achieved and outputs produced, and would comment on the relevance of the TA and training provided, its effectiveness in addressing countries’ capacity constraints, as well as the efficiency by which the TA and training has been delivered.

1.4 Issues Addressed By the Evaluation

The contribution of PFTAC in addressing the region’s problems and capacity constraints in each of the four functional areas covered (public financial management, tax administration and policy, banking regulation and supervision, and macroeconomic and financial statistics) were evaluated. In particular, the evaluation covered:

1. The quality, timeliness and relevance of the inputs provided, activities undertaken, and outputs produced, including the appropriateness of the balance between hands-on assistance and strategic advice and the delivery modalities available to the Center;

2. The merits of the regional approach to providing TA in economic and financial management in the region, and the responsiveness of PFTAC in adapting to the region’s changing needs;

3. The use being made of PFTAC assistance, the impact it is having on improved economic and financial performance in the region, and the likely sustainability of results;

4. The extent to which PFTAC, as one of several TA providers in the region, is contributing effectively to:

  • Fostering greater government ownership of, and commitment to, economic and administrative reforms and capacity building;

  • Advancing regional cooperation and economic integration; and

  • Improving information sharing, cooperation and coordination among TA providers and donors, forging partnerships, and mobilising resources;

5. The reasonableness of the relationship between costs and the contribution of PFTAC to the region in the economic and financial management area, taking into account the difficulties inherent in judging costs and benefits of TA activities;

6. The role of the Tripartite Review Committee (TPRC), individual governments, donors, regional bodies, and the IMF in determining PFTAC’s priorities and direction, and guiding the PFTAC Coordinator;

7. The quality and timeliness of management, monitoring and backstopping at PFTAC itself and at IMF headquarters; and

8. The level of support and commitment of the regions’ governments to the work of PFTAC.

The Terms of Reference for the evaluation are presented as Appendix A.

1.5 Methodology

The methodology consisted of:

  1. Document Review;

  2. Interviews with the Chairperson of PFTAC, Coordinator of PFTAC and Resident Advisors, officials of participating countries, the IMF, donors and other interested organisations. See appendix B for a full list of people interviewed;

  3. Review of files in PFTAC for the case studies;

  4. Telephone Interviews from Fiji with a Representative Senior Official in some of the 10 PICs that could not be visited in person;

  5. Comparative analysis – benchmarking results against comparable organisations;

  6. Preparation and Delivery of a PowerPoint presentation of Principal Findings, Conclusions and Recommendations to representative of Chairperson of TPRC and Coordinator of PFTAC in Fiji; and

  7. Draft Report.

The evaluation of PFTAC was loosely based on the review methodology of the Council on the Cost and Quality of Government (CCQG) in Australia (www.ccqg.nsw.gov.au) which is designed for organisations that are funded by parties other than its clients. The performance review examined two parts of PFTAC: (a) Strategy – how well PFTAC manages its external environment by delivering relevant, effective and transparent services and (b) Operations – how well it manages its internal environment by using its resources economically, efficiently and prudently.

The following dimensions of PFTAC were examined:

Table 1.1

Strategic and Operational Review

article image

Some or all of the following best management practices were reviewed in both the strategic and operational areas.

Table 1.2

Strategic Review

article image
Table 1.3

Operational Review

article image

The evaluation team undertook focus group meetings or one-on-one interviews of PFTAC donors, PFTAC clients, other stakeholders and PFTAC management and staff to ascertain whether “best Practices” are applied in each dimension of PFTAC and whether they deliver the desired results. Some of the questions asked are presented in the following two tables.

Table 1.4

Questions Asked

article image

1.5.1 Document Review

The following types of documents were reviewed:

  1. The original UNDP project document, Fiscal and Monetary Reform and Statistical Improvement, 1993;

  2. The revised project document, Project Document for the period May 2002 – April 2005;

  3. Report of the Evaluation Team, 1997;

  4. Strategic Review of Technical Assistance in Economic and Financial management in Pacific Island Counties, 2001;

  5. Minutes of Selected TPRC Meetings;

  6. Selected Six-Month Progress Reports;

  7. Selected Three-Month Rolling Work Plans; and

  8. Selected PFTAC Files for Case Studies.

1.5.2 Interviews with Donors

There were two sets of interviews with donors:

1. Interview with aid organisations to determine:

  1. Other TA projects in PFTAC PICs;

  2. Issues of coordination with other TA providers;

  3. What has gone right with and should be continued with PFTAC;

  4. What might be changed in PFTAC in any subsequent phases; and

  5. Continued support for the project;

2. Where necessary, interviews with other government organisations. An example here was Stats NZ which is helping in a number of areas, especially with PC/Trade. Another was the Ministry of Finance in Australia

1.5.3 Meetings with PICs

There were two sets of interviews with PICs:

1. Interview with Finance and Central Bank to determine:

  1. Other TA projects in the two organisations;

  2. Issues of coordination with other TA providers;

  3. What has gone right with and should be continued with PFTAC;

  4. What might be changed in PFTAC in any subsequent phases; and

  5. Continued support for the project;

2. Interviews with other government organisations for the three case studies. An example here would be Customs in FSM which has introduced PC/Trade.

1.5.4 Case Studies

The objective was to take three major activities and research them in greater depth in order to develop findings and lessons learned. The methodology included a file review, interviews with the resident advisor in PFTAC and interviews with recipients. The three case studies, presented in Appendix C, are:

  • Association of Financial Supervisors of Pacific Countries;

  • PEM Course at USP; and

  • Advice on Customs Modernisation and Associated Computer Systems.

1.5.5 Telephone Interviews

In addition to the visits to the selected sample of PICs, the evaluation team also invited all of the PICs who are served by PFTAC for any comments and suggestions on the current range and quality of services provided by PFTAC in their respective countries. Although efforts were made to extend the period for receiving feedback from the countries as much as was practically possible, only one country had responded by the cut-off date of June 25, 2004. Feedback from the PNG was positive on the respect that PFTAC had built up in the region due to the quality of its advice and its willingness to help on a timely basis. The evaluation team notes that there will be a further opportunity for comment by all of the PICs when the final draft version of the evaluation report is made available to them through the TPRC.

1.6 Schedule

The development of the schedule of visits outside of Fiji was extremely difficult and went through a dozen or more iterations before a final schedule was settled upon. The challenge in developing the schedule resulted from the fact that (a) the full-team part of the mission was less than three weeks; (b) the team needed at least three days at both the beginning and the end of the mission in Fiji; (c) distances between PICs are huge and flights are not very frequent and rarely are direct thus making travel a challenge; (d) a strong desire that the team remain together during all three weeks of the mission, and (e) a need to be in each country for a sufficient length of time to do a professional job. As a result of these constraints, the best schedule that the evaluation team could come up with included Fiji, Australia, ADB, Vanuatu and Tonga as a team. To increase the geographic coverage, James Bucknall visited IMF HQ in Washington in February, NZAID and Stats NZ in March and April while in NZ on other business and visited FSM on his way back to Canada. Mr. Vaai conducted interviews in Samoa at the end of the three week period in Fiji. Mr. Allan interviewed the World Bank in Sydney.

Chapter 2 Pacific Island Countries and Region

2.1 Introduction

The purpose of Chapter 2 is to put the work of PFTAC into a broader context of the PICs and the Pacific region. It does this by describing: (a) which PICs are supported by PFTAC; (b) the immensity of the region and the challenges faced in moving from one country to another; (c) some basic statistics on each of the 15 countries served by PFTAC; (d) principal bilateral and multilateral donors to the region; and (e) some of the challenges faced by the PICs.

2.2 Pacific Island Countries Supported by PFTAC

The following Pacific Island Countries are supported PFTAC.

2.3 Pacific Islands Region by Size

The geographic area covered by PFTAC is enormous. It stretches from Palau in the East to Kiribati in the west and from the Marshall Islands in the north to Tonga in the south. The area covered is some 8,000 km from east to west and 4,000 km from north to south: some 32 million square km.

By contrast, the Caribbean Regional Technical Assistance Centre (CARTAC) serves 20 countries, the vast majority of which fall within an area some 2,000 km east to west and 1,000 km north to south for a total of 2 million square km. The physical area covered by PFTAC is some 16 times greater than the area covered by CARTAC.

However, simple distances do not adequately explain the challenges that PFTAC faces in traveling in the Pacific compared with CARTAC traveling in the Caribbean. Whereas the Caribbean region is reasonably well served by air with most islands connected by direct flights, this is not true in the Pacific where direct flights between islands are more rare. Most inter-island flights use hubs such as Auckland in New Zealand and Sydney and Brisbane in Australia. By way of concrete example, James Bucknall, in order to meet 3 donors and 4 recipient countries, made 29 flights.

The following table provides basic statistics on population, land area, GDP and foreign aid for the 15 PICs that are supported by PFTAC.

Table 2.2

Pacific Island Countries by Size

article image
Source: AusAID, Pacific Program Profiles 2003 – 04, page 91 plus additional data on PNG provided separately by AusAID.

The five largest PICs – PNG, Fiji, Solomon Islands, Vanuatu, and Samoa account for over 75% of population, territory, GNP and foreign aid in the region. PNG alone accounts for 70% of the population in the region.

2.4 Pacific Island Aid by Donor

The following table shows the principal bilateral and multilateral donors that support the PICs with international aid.

Table 2.3 above shows that

  • Bilateral donors provide more than 90% of the aid;

  • Australia, Japan and New Zealand provide almost 80% of foreign aid to the PICs; and

  • Less than 0.3% of the total aid going to the 15 PICs is used to finance PFTAC.

Table 2.3

Bilateral and Multilateral Aid to PICs

article image
Source: AusAID, Pacific Program Profiles 2003 – 04, page 93 plus additional data on PNG provided separately by AusAIDNote: Above figures exclude US ODA to former US Trust territories and French ODA to French territories.

This last point is worth repeating, “less than 0.3% of the total aid going to the 15 PICs is used to finance PFTAC”. Hence the cost effectiveness of PFTAC should be gauged against the value for money of the other 99.7% of aid to the region.

2.5 Challenges Faced by PICs

Pacific Island Countries face a number of challenges in their quest for economic development. Some of these are described below.

  • Many countries consist of several dozen islands scattered over relatively large geographical areas creating high intra-regional transportation cost;

  • Only one of the 15 countries has a population of over one million people and five states have populations of less than 20 thousand. Relatively small populations create high, and in some cases prohibitive, per capita costs for basic public infrastructure and services required to keep the country functioning;

  • Remoteness from the major trade markets and the related relatively high transportation costs limit potential for outward external growth strategies for most PICs;

  • Several countries are in a severe fiscal crisis with excessive budget deficits and a high public sector debt to GDP ratio;

  • Living standards are declining in some countries with the living standards in PNG, Solomon Islands and Nauru being lower now than they were at independence;

  • Urbanisation is undermining traditional cultures and rising township unemployment is contributing to social and political instability;

  • A shortage of skilled professionals is aggravated by an acute brain drain to Australia, New Zealand and USA;

  • Traditional land tenure systems are proving to be a disincentive to more productive farming practices;

  • Global warming may cause increased sea flooding, coastal erosion, coral bleaching and cyclones;

  • HIV/AIDS may infect up to 20% of PNG’s population within 10 years and may spread to other PICs;

  • Several countries will not be economically viable without foreign aid for the foreseeable future. Management of foreign aid resources from a wide range of donors exert relatively high demands on the thin pool of administrative skills in each country; and

  • Relatively limited access to private foreign capital due to the high investment risks arising from high vulnerability and susceptibility of the island economies.

2.6 Opportunities for PICs

  • The large sea area provides opportunities for commercial exploitation of the marine resources. In particular the development of fisheries and deep seabed minerals provide the greatest potential for commercial exploitation;

  • The geographical isolation of most island groups provides natural quarantine “buffers” for any spread of pests and diseases;

  • Remoteness of most island countries from the high security risk metropolitan areas provides potential opportunities for relatively low security risk tourist destinations in the Pacific;

  • Relatively close social and cultural linkages provide basis for regional cooperation. The “Pacific Way” of doing things in the region has been an effective basis for encouraging consensus decision making on regional issues. A relatively large number of regional organisations have been created to pursue the regional initiatives; and

  • Lessons learned from the least 20 years of experience in participating in regional collaboration provides a good basis for improved future regional initiatives.

Chapter 3 Description of PFTAC and Its Activities

3.1 Introduction

Chapter 3 provides a description of PFTAC and its main technical assistance activities. The chapter: (a) provides an overview of PFTAC; (b) describes PFTAC’s organisation and governance structure; (c) describes PFTAC’s principal donors and the size of their support; (d) describes PFTAC’s clients and other principal stakeholders; (e) presents PFTAC’s budget; (f) describes PFTAC’s activities by output and country; and (g) provides a comparison of PFTAC and CARTAC.

3.2 Overview of PFTAC

The Pacific Financial Technical Assistance Centre in Suva, Fiji was jointly established in 1993 by the United Nations Development Programme (UNDP) and the International Monetary Fund (IMF) as a regional office to implement the “Fiscal and Monetary Management Reform and Statistical Improvement Project” in 15 Pacific island countries (PICs). Until July 2002, PFTAC operated as a UNDP regional project with the IMF as executing agency. Because of the withdrawal of UNDP as a PFTAC-donor, the TPRC approved a new Project Document and decided that the IMF should seek to establish a PFTAC sub-account through which donor contributions would be managed.

The Centre's aim has been, and continues to be, to build skills and institutional capacity in member countries for effective economic and financial management that can be sustained. Toward these objectives, the Centre provides advice and training in public financial management, tax and customs policy and administration, banking regulation and supervision, and economic and financial statistics. The Centre’s mission is “to build skills and institutional capacity in member countries for effective economic and financial management that can be sustained at the national level”.

PFTAC’s assistance takes a number of forms, such as assistance with identifying capacity gaps, developing strategic approaches and action plans for medium-term reforms, high-level policy advice, and implementation of IMF supported standards and codes, technical advice and help, hands-on training, and organisation of seminars, workshops, and professional attachments. Assistance also covers ongoing, medium-term involvement as well as emergency fire-fighting help. Its advisors support member countries during missions and via e-mail and telephone communications, and act in many instances as sounding boards that officials use in their day-to-day operations.

3.3 Organisation and Governance

The core organisation consists of seven people: the project Coordinator, four resident advisors and two administrative assistants. The project Coordinator and the two administrative assistants are financed by the IMF and the four resident advisors are financed by Japan or through the sub-account financed by ADB, Australia, and NZ.

At any point in time there may be other people associated with PFTAC. During April/May, 2004 there was a consultant preparing a training program, a consulting counsel working on money laundering, and a technical aid person working with the Banking Regulation and Supervision resident advisor.

The overall guidance of PFTAC is provided by the Centre’s Tripartite Review Committee (TPRC) consisting of representatives of all participating Pacific Island Countries, the funding agencies, and the IMF. Other technical assistance (TA) providers and relevant regional institutions are invited as observers.

The IMF manages the Centre’s operations, provides its project Coordinator, and recruits and supervises its resident advisors through the IMF’s TA departments. PFTAC provides TA through four resident advisors and the project Coordinator, supplemented by short-term experts, training and seminar activities, and professional attachments.

3.4 Donors

Since 2002 the project has been an IMF administered project with financial support from the IMF, ADB, Australia, Japan and New Zealand1. The following table provides details of this financing for FY 2003 to FY 2005 inclusive.

Table 3.1

PFTAC Source of Funds, FY 2003-FY2005

article image

In addition, the Republic of Korea pledged financial support that is expected to be available starting in FY2005. The European Commission has also indicated its willingness to support PFTAC financially subject to the resolution of outstanding policy and legal issues.

The total sources of funds available to PFTAC for FY2003 – FY2005, excluding funds that may become available from other sources, is US$5.740 million. Of this amount, Japan supplies 33.7%, ADB supplies 22.6%, Australia 19.1%, IMF 16.8% and New Zealand 7.7%. If any one of these five donors were to stop funding PFTAC, the impact on PFTAC would be disruptive in the absence of replacement funding from other sources.

3.5 Clients

As mentioned earlier, 15 Pacific Island Countries are eligible to participate in the activities of PFTAC. These PICs are identified in Table 2.1 and repeated below in Table 3.2: four countries are from Melanesia, five from Micronesia and six from Polynesia.

Table 2.1

Participating Countries in PFTAC

article image
Table 3.2

Participating Countries in PFTAC

article image

3.6 Other Stakeholders

Not only does PFTAC work with the individual PICs but it also works with a large number of international and regional organisations. Some of the more important ones are shown in the following table.

International

  • Bank for International Settlements, especially the Stability Institute;

  • United Nations;

  • World Bank; and

  • OECD.

Regional

  • Association of Financial Supervisors of Pacific Countries (AFSPC);

  • Oceania Customs Organisation (OCO);

  • Pacific Islands Forum Secretariat (FORSEC);

  • Pacific Islands Tax Administrators Association (PITAA);

  • Secretariat of the Pacific Community (SPC); and

  • Forum Economic Ministers Meetings (FEMM);

3.7 PFTAC Budget

Table 3.3 shows PFTAC’s expenditures for FY2003 and FY2004 and budget for FY2005 in terms of PFTAC core expenditures, external professional support and governance. Of the total budget of US$5.757 million, core expenditures account for US$4.625 million or 80.4% of the total budget, external professional support accounts for US$0.983 million or 17.0% of the total budget and governance accounts for US$149 thousand or around 2.6% of the budget.

Table 3.3

PFTAC Expenditures and Budget

article image
Note: The difference in totals ($16,360) between the source of funds in Table 3.1 and the use of funds in Table 3.3 is the financing gap.

3.8 Pftac Activities By Output and Country, 1998-2003

Table 3.4 shows the number of advisory missions, participants in seminars, short-term consultancies and personnel attachments undertaken by PFTAC during the period 1998-2003.

Table 3.4

PFTAC Activities by Output and Country, 1998-2003

article image

Key: Output 1-Public Financial Management; Output 2-Tax Policy and Administration; Output 3-Banking Supervision; Output 4-Economic and Financial Statistics.

PFTAC undertakes four main types of activities: advisory missions; seminars; short-term technical consultancies and personnel attachments. The different utilisation could reflect one or more of the following: (a) different national institutional abilities to utilise PFTAC activities; (b) different levels of financial reforms; and (c) different priorities assigned to the areas covered by PFTAC. The magnitude of the outputs of these four activities are described in the following paragraphs.

3.8.1 Advisory Missions

PFTAC undertook 340 advisory missions during the period 1998-2003. Of these, 206 or 60% of the total were to Tonga, FSM, Vanuatu, Fiji, and the Marshall Islands (which account for 17.8% of the region’s population), 97 or 29% of the total were to Samoa, Solomon Island, Palau, Kiribati and PNG (82.1% of population), and 37 or 11% to Cook Islands, Niue, Nauru, Tuvalu and Tokelau (0.1% of population). The close working relations between PFTAC and the Finance Ministries of Tonga, FSM, Vanuatu, Fiji and the Marshall Islands reflect a high level of confidence by these countries.

The 340 missions were more or less evenly divided between the four core resident advisors over the six years under analysis although slightly more missions were undertaken by tax policy and banking advisors than by public financial management and statistics advisors.

3.8.2 Seminar Participation

252 people from the PICs participated in seminars put on by PFTAC over the six-year period under review. Of these, 146 or 58% came from Tonga, Vanuatu, Samoa, PNG and Fiji (which represents 89.6% of the region’s population). Significantly more people participated in seminars in the area of Tax Policy and Administration and Banking Supervision than participated in seminars in the other two core areas.

3.8.3 Short-Term Consultancies

Only 27 short-term consultancies were fielded during the six year review period. These were more or less evenly divided between the four core areas.

3.8.4 Personnel Attachments

There were 77 government officials from 10 PICs attached to public sector institutions in Australia, New Zealand and other PICs over the period under review. The purpose of the job attachments was to expose PIC officials to more advanced practices in their areas of work. The overwhelming majority of the attachments were in Tax Policy and Administration and Banking Supervision. This reflects the importance of “hands on experience” in these areas and the confidence of the island countries to which the attachments were made. PFTAC and StatNZ have recently agreed that the latter will accept two candidates each year that are nominated and paid for by PFTAC.

3.9 Comparison of PFTAC snd CARTAC

PFTAC and CARTAC are sister organisations and have many things in common: both have a project Coordinator, both have four resident advisors working in the same four areas, both have several donors, both work with island states, both have the IMF as their executing agency, and both have similar forms of governance. However, there are differences between the two organisations and these differences result in different challenges and different quantities of outputs. Some of the more important differences are outlined below.

3.9.1 Size of Region Serviced and Ease of Travel within the Region

The PICs supported by PFTAC are spread over an area 16 times greater than the Caribbean island states supported by CARTAC. Not only are the distances much greater in the Pacific than in the Caribbean, but the Caribbean is better served by airlines than is the Pacific in terms of frequency of flights and directness of routes.

3.9.2 Donor Financing and Methods of Operation

There are differences between the two centres in the area of financing and methods of operations. In the Caribbean, some donors have chosen to primarily channel their technical assistance efforts through CARTAC. By contrast, in the Pacific, the number of donors and providers are much more limited, and those active are themselves implementing a significant program of activities. This different mode of operation in the two regions contributes to a number of contrasts in both the budgets and in the roles played by the respective centres.

Table 3.5 shows that over the period 2003-05, CARTAC received more funds than PFTAC.

Table 3.5

Comparison of PFTAC and CARTAC Budgets

article image

The greater availability of funds available to CARTAC helps to explain a number of the differences between PFTAC and CARTAC.

  1. The budget for CARTAC, at US$9.923 million, is almost twice as big as the budget for PFTAC, at US$5.757 million, over a three year period;

  2. The budget for the project Coordinator, the resident advisors, regional travel, administrative costs, office costs and governance are roughly the same order of magnitude in the two organisations, although it should be noted that CARTAC has five resident advisors;

  3. There is seven times more money available in CARTAC for short-term consultants than there is in PFTAC; and

  4. There is four times more money in CARTAC for training, seminars and attachments than there is in PFTAC.

However, these differences do not necessarily imply that (i) PFTAC is under funded or (ii) that PFTAC cannot be as effective in its activities as CARTAC.

Whilst it is clear that PFTAC itself cannot devote as many resources to major capacity building efforts as CARTAC with its existing resource base, account must be taken of PFTAC’s ability to leverage its efforts in the field though cooperation with other bilateral and multilateral donors. Such cooperation is already evident in several areas. In the tax and customs area, for example, PFTAC has carried out assessments of PIC tax systems and administration and worked with other donors to supply the necessary follow-up expert advice. It will be important to continue efforts to ensure maximum cooperation with other agencies operating in the region.

At present, donors (including the IMF) and member countries seem content to maintain funding for PFTAC at broadly its present level though a case may be made in the next project document for some increase in funding to finance additional use of short term consultants, a tier of locally recruited experts and some additional regionally oriented training courses, workshops and personnel attachments.

3.9.3 Organisation

CARTAC is a slightly larger organisation than PFTAC in terms of the number of core staff. Both PFTAC and CARTAC have a project Coordinator and two administrative assistants. CARTAC has five resident advisors, one more than PFTAC. In addition to this, CARTAC also has a full-time economist on loan from the Caribbean Development Bank, a financial assistant because there is a bigger book-keeping work load in CARTAC than in PFTAC, a general assistant and a research assistant.

PFTAC has traditionally focussed on technical issues and there seems to be no support from either donors or member countries to significantly increase the “economic” content of its technical assistance by employing an economist. Where there is a need for such advice, it may best be provided by IMF HQ staff and the use of economists on the staff of the ADB, World Bank, AusAID or NZAid. However, PFTAC may wish to investigate the need for periodic employment of specialist advisors in specific fields of concern to the PICs as a means of complementing the economic research of IMF HQ economists e.g. in public debt compilation, measurement of taxation expenditures, construction of macroeconomic frameworks for PIC multi-year budgets and so on.

3.9.4 Quantity of Work

It is not possible to compare the number of advisory missions and the number of seminars as the data available to the evaluators are collected and presented differently by CARTAC and PFTAC. For future evaluations it would be helpful if the IMF could standardise output measures for each of the TACs it administers. Fortunately, the statistics for short-term consultancies and attachments are presented by PFTAC and CARTAC on the same basis and can be compared. The following two tables present these statistics.

Table 3.6 shows that CARTAC, in just 18 months, had three times the number of consultancies that PFTAC had in six years. This proportional difference was even greater in Tax and Customs and Financial Services. As explained in section 3.9.2, Finances, one reason that CARTAC is doing more in these two areas is that it has a larger budget for these activities than does PFTAC. Moreover, as explained above, in the PICs such TA is often supplied by bilateral and multilateral agencies.

Table 3.6

Comparison of Number of Consultancies in PFTAC and CARTAC

article image

Table 3.7 shows that the number of attachments in PFTAC is seven times greater than the number of attachments in CARTAC during the 18 month and six year periods respectively, i.e. PFTAC has done proportionally more attachments than CARTAC.

Table 3.7

Comparison of Number of Attachments in CARTAC and PFTAC

article image

3.10 PFTAC Versus PICTAC

From time to time PFTAC has been asked to provide assistance to other countries that are not on this list of 15 and on at least one occasion (East Timor), due to the very special circumstances, PFTAC agreed to provide some advisory support. It has been suggested that a name change to PIFTAC (Pacific Island Financial Technical Assistance Centre) would more clearly define the participants. The disadvantages of changing the name are the same for any re-branding exercise: initial costs and potential confusion. Furthermore, a name change would have no material impact on PFTAC’s profile or standard of service. Regardless, the evaluation team discussed the possibility of recommending a name change to either PIFTAC or PICTAC (Pacific Island Country Technical Assistance Centre). Both have certain advantages with the latter the additional two advantages that (a) its new nomenclature would be in line with its sister organisations, CARTAC, AFRITAC-East and AFRITAC-West and (b) by dropping the “F” for “Financial” the new name would implicitly be broadening the scope of PICTAC to include “economic” as well as “financial”.

As there was no strong support for a name change among either donors or member countries at the Tripartite Review Committee meeting in Suva in August 2004, the evaluation team decided not to recommend that PFTAC should be renamed PICTAC at this time. However, the team agreed to record that the discussion took place and suggests that the issue should be considered again at a future meeting of the TPRC to see whether there was renewed interest in changing the name.

Chapter 4 Performance Review of PFTAC

4.1 Introduction

The evaluation team members, either collectively or individually met or conducted phone interviews with a large number of officials and reviewed many documents. The findings from these interviews and documents are presented in the form of (a) performance diagnosis using strategic and operational questions and (b) a SWOT analysis where the findings of the performance analysis are used to document the main strengths and weaknesses within PFTAC and the principal opportunities and threats that PFTAC faces externally.

4.2 Review Findings

The performance review of PFTAC examined (a) strategy - how well PFTAC manages its external environment by delivering relevant, effective and transparent services and (b) operations - how well PFTAC manages its internal environment by using its resources economically, efficiently and prudently.

In undertaking this performance diagnosis, the evaluation team examined a number of dimensions of PFTAC. These are presented below in Table 4.1. The analysis is presented in the subsequent sections.

Table 4.1

Performance Diagnosis

article image

4.2.1 Strategic Diagnosis

A. Environmental Scan

PFTAC is well-respected and well-connected in the Pacific Island Countries and the greater Pacific area. It is aware of most of the changes that are taking place in the region, especially the changing PIC conditions and donors attitude to development in neighbouring countries. Much of the time PFTAC is able to incorporate these environmental changes into its daily operations. However, there is insufficient formal analysis and documentation of the results of the implications of these changing PIC conditions and donor attitudes to PFTAC and some PICs. A good example of partial Political, Economic, Environment, Social and Technological (PEEST) analysis of the PIC region was that written by Tom Wilson, former Public Financial Management Advisor, in a paper he presented shortly before he left PFTAC.2

B. Clients

All of the participating countries spoke highly of the need for and modus operandi of PFTAC. While this overall finding is valid, the evaluation team met a small number of PIC officials who, for a number of reasons that will be discussed later, had chosen not to use the services of PFTAC. From the point of view of the individual participating countries, PFTAC is a success. Many spoke of PFTAC’s openness to requests, the speediness of its response, the quality of its services and its key strategic support at crucial junctures. PFTAC has, in the view of the participating countries, filled an important role in the Pacific Island Countries.

It should be noted, however, that the evaluation team spoke with a few individuals who were less positive about PFTAC and some of its resident advisors over the last seven years than were the majority. Some of the concerns raised included a lack of sensitivity to the culture in the individual PIC, a perceived lack of understanding of the technical needs of the recipient country and its officials, and a tendency to write TORs that related more to what the TA could deliver or thought should be delivered rather than what the PIC felt was needed.

There are several explanations for these views including: insufficient cultural sensitivity, (although it must be noted that other people interviewed found the resident advisors culturally sensitive), a genuine disagreement on the nature of the problem and the appropriate response, and expectations on the part of PICs which are much greater than PFTAC has any hope of satisfying with current resources. The evaluators do not have sufficient information to provide a satisfactory analysis of the root cause of these concerns. But there is an issue that needs to be analysed further. Part of the solution would be to introduce formal satisfaction surveys. The evaluators note that the PICs endorse PFTAC in glowing terms at the TPRC meetings, however, a more formal and more probing survey of PIC satisfaction with PFTAC is required.

Participating countries agreed that PFTAC’s and other TA activities were more effective when the country had a mutually agreed gap analysis, a well developed reform strategy and an action plan that were owned by the country.

All of the participating countries that were consulted want the Centre to be funded after the completion of the current phase in 2005.

C. Stakeholders

The donors continue to support the concept of a regional Centre that provides technical assistance in the four areas of support provided by PFTAC. One person raised a question regarding the usefulness and impact of past PFTAC and all other donor TA in the PICs. All of the donors raised questions regarding impacts, outcomes and capacity building.

Australian, New Zealand and Japanese officials all said that their respective governments would be both willing and able to continue supporting PFTAC financially at or above current levels. Indeed Australia appeared willing to increase funding to PFTAC if PFTAC could expand its efforts in PNG, Solomon Islands and Nauru where Australia saw a heightened need to restore good governance. The Asian Development Bank indicated that it would like to continue supporting PFTAC in principal but that there was an internal debate on how much money would be available for TA in the PICs and the form that that TA would take. The EU is prepared to provide considerable financial support to PFTAC but this support is contingent on the IMF and the Commission resolving remaining policy and legal issues.

The measuring of results and capacity building are thorny issues which are challenging both PFTAC and CARTAC. On the one hand, it is quite understandable that the donors want to see measurable results so that these results can be presented to their relevant over-sight bodies. On the other, much of the ad hoc TA provided by PFTAC, while useful, does not really lend itself readily to outcome measurement. For example, a TA spending two weeks in a country helping that country to improve its annual budget is clearly time well spent. But at the end of it, one can say little more than that the budget was improved for that year and that the probability of the budget being better in future years is increased because of the capacity building component of the exercise. Whether there is the political will to balance the budget or not is a non-technical issue outside the control of PFTAC.

For the reform component of PFTAC’s activities, i.e. where PFTAC is part of a long-term, multi-donor initiative it may be easier to measure results. An excellent example here is the introduction of PC/Trade into FSM. This is a long-term, multi-donor effort that involves NZAid and Stats NZ in the development of PC/Trade, AusAID in providing a resident statistical advisor, part of whose duties include ongoing support for the introduction of PC/Trade, ADB in funding a resident economic advisor and PFTAC, which funded numerous activities, and is providing an entrepreneurial, managerial, technical, and mentoring role. While it would not be easy to show the results of each particular player in the introduction of PC/Trade, it should not be difficult to report on the overall results and outcomes of the project.

On the question of capacity building, there is little doubt that most interventions on the part of PFTAC result in some capacity building. However, the question remains as to how much long-term capacity building can be expected to be accomplished with the present level of resources available to PFTAC, especially when there is a high turnover of people in the PICs who have recently benefited from professional development. Capacity building, like education and training is an ongoing process. Schools and universities will be necessary for as long as children and young adults are there to be schooled and educated. Likewise, capacity building in all governments in the world is a continuous and ongoing need. All the time that the PIC governments are unable to build up this capacity by themselves, there will be a need for donor governments and regional institutions to provide support for this service from the outside.

The huge brain drain of trained people to Australia, New Zealand and elsewhere is seriously eroding the long-term effectiveness of capacity building in the PICs. Ceasing such capacity building would leave the PICs with an even bigger skill gap as professional emigrated.

Another issue raised by a couple of the people interviewed was the cost of the PFTAC TAs. One response to this concern might be that the salaries and benefits are necessary to attract appropriate people, that they are in line with what the IMF pays elsewhere such as CARTAC, and that they are in line with costs for resident TA provided by some other donors such as Australia in places such as the Solomon Islands. They are also consistent with the remuneration packages of other international agencies such as the World Bank.

Other regional stakeholders who were not PFTAC donors (e.g. World Bank, SPF, SPC, and FORSEC) were all positive about the concept and modus operandi of PFTAC and expressed a strong desire that PFTAC should continue its operations.

In terms of PFTAC’s efforts at consulting and briefing donors and other stakeholders the general view was that the present Coordinator went to considerable lengths to achieve two-way lines of communications. However, some stakeholders, as already mentioned, wanted more information on the success or otherwise of projects to which PFTAC contributed, not just reports on the inputs of PFTAC to these projects.

D. Governance

The following three paragraphs on roles and responsibilities regarding PFTAC are extracted from the project document that covers the period 2002-2005.

The IMF is the executing agency. In that capacity, the IMF is broadly responsible for the relevance and quality of advice and assistance from the Centre. In particular, the IMF is responsible for:

  • Assigning a Project Coordinator to the Centre from its staff;

  • Selecting from its panels of experts the Centre’s resident advisors and assist with the hiring of short-term experts for the Centre;

  • Providing backstopping services to the Centre’s advisors; and

  • Managing the cost-sharing contributions of donors and the financial operations of the Centre under the PFTAC sub-account.

The strategic direction and approach of the Centre will be guided by the Tripartite Review Committee (TPRC), which will consist of the representatives of the executing agency, the funding agencies and the participating countries. Meetings will take place every 12-18 months. The chairperson of the meeting will be the Governor of the Reserve Bank of Fiji. The Project Coordinator will serve as Secretary to the TPRC.

The day-to-day management of the Centre and its operations are the responsibility of the Project Coordinator. In particular, in cooperation with the Centre’s resident advisors, and guided by the decisions of the TPRC, the Project Coordinator is responsible for the work program of the Centre.”

The IMF fulfils its quality control responsibilities largely through correspondence between PFTAC and the IMF’s headquarters. IMF staff, on a periodic basis, monitors the work of each expert. The expert submits to the responsible IMF technical back-stopper a monthly or three-monthly report for comment and guidance. The project Coordinator and the resident advisors send regular reports on their work plans and activities to the regional and the technical assistance providing departments such as Fiscal Affairs, Monetary and Financial Systems, and Statistics. They also send ad hoc communications on specific matters as they arise.

Based on their review of the monthly or three-monthly reports and discussions with the resident advisors and their back-stoppers in Washington, DC, the evaluators conclude that the system is working well. Washington allows the resident advisors a large degree of autonomy while still monitoring them and providing support as needed. The TA departments have exhibited considerable confidence in the work of the resident advisors and as a result have made relatively few comments on their work program and monthly or three-monthly reports. The IMF HQ noted that the resident advisors have been responsive to the needs of the countries, have been addressing immediate issues and that this is generally supportive of the work of the area departments at the IMF.

The IMF has been an effective executing agency:

  • The IMF’s overall selection of the project Coordinator, the resident advisors and short-term consultants has been good;

  • The IMF, while monitoring on a monthly basis, has largely decentralised the operations of the project Coordinator and the resident advisors;

  • The project Coordinator and the resident advisors have by and large established good personal relationships with their constituents and have exhibited an ability to see the needs of the participating countries;

  • The IMF’s selection of the project Coordinator and resident advisors has been widely praised by participating countries, donors and regional organisations. The project Coordinator and the resident advisors have earned the trust of most in the region and this reflects favourable on the Centre; and

  • It is difficult to imagine that any other organisation could have been a better technical executing agency.

The TPRC has met three times during the review period under the skilled guidance of the Chairperson. The TPRC members expressed their thanks and appreciation for the good job done by both the Chairperson and Secretary.

While the evaluators found no evidence that this system of governance was not working well, the evaluation team felt some unease that a formal governance charter had not been produced and the team was unclear whether the actual role of the TPRC was that of a governing board or simply a sounding board. The team felt that this issue needed resolution since it is fundamental to knowing who ultimately sets the direction of PFTAC.3

E. Service Delivery

There is universal agreement that PFTAC provides a high standard of technical advice. This is due to the rigorous selection techniques used in choosing the resident advisors, the backstopping provided by the IMF in Washington, DC and by PFTAC, the high motivation on the part of the resident advisors, the responsiveness of PFTAC to requests and the general good relations between the resident advisors and their counterparts in the PICs. As mentioned, a small number of interlocutors mentioned issues such as cultural insensitivity, although others praised the advisors on their interpersonal skills.

The evaluation team does not have sufficient information to analyse how serious the issue of interpersonal skills really is, and much might be explained away as simply miscommunication, but the fact that a number of people raised the issue means that it has been an irritant in the past. However, this apparent irritant notwithstanding, the quality of the technical assistance is high and the work of PFTAC is highly regarded in the recipient PICs.

While the standard of technical assistance is high, a few people interviewed, donors rather than recipients, raised the question as to whether the technical assistance was leading to much capacity building. At the level of individual PFTAC interventions, the evaluation team found evidence of capacity building. However, at the overall regional level, the donor’s contribution, including PFTAC’s, to capacity building in the PICs is not clear due to the brain drain.

The serious brain drain resulting from the emigration of talented public servants in search of higher incomes is clearly eroding capacity building efforts. The only obvious solutions to this problem would be to raise the salaries of graduate public servants and/or outsource more government technical work to local private consultants, but these options may not be affordable or politically acceptable. Ceasing civil service capacity building, as a measure of despair, would of course simply aggravate the problem. There may be no choice but to support capacity building and accept that a high rate of attrition is unavoidable in a region where economic opportunity is limited.

F. Strategies and Reviews

PFTAC is providing financial and economic TA support in the following areas: (a) short-term ad hoc visits and Suva-based help desk advice to “fix” pressing problems, (b) repeat visits to identify local capacity gaps; design long-term reform TA initiatives and enlist support for their implementation; and (c) people capacity building through formal courses, workshops and on-the-job training and secondments to “best practice” institutions. Attempting to do all that in 15 countries with four resident advisors is a daunting task and one that is bound to lead to disappointments. The task is made all the more difficult as donors, PICs, IMF and PFTAC itself, all seem to place somewhat different weights on these three types of activity. It is not clear whether there is a common view among the stakeholders on: (a) whether priority should be on PFTAC providing short-term ad hoc TA or being a long-term reform agent, and if the latter what is the roll-out strategy, (b) whether priority should be on providing technical assistance or capacity building or (c) whether priority should be on “economics” or “financial” support. PFTAC needs to do some strategic thinking on these questions the results of which should be included in the next project document.

PFTAC could also better articulate its TA strategy, which appears to have three streams. It is suggested that PFTAC embrace a more staged approach to the reform and capacity building streams to ensure their success. For instance on the reform front PFTAC should make it clear to all clients that it will first do a ROSC review of a particular TA area, then seek local acceptance and endorsement of its findings before developing a reform agenda let alone an action plan for clients to consider and approve. These steps may be obvious to PFTAC, but nowhere in its literature is this process clearly explained. Also it is not clear that it is always diligently followed. Several clients commented on terms of reference that they had little role in drafting. This left an impression that they were not convinced of the merits of the project since they could not see the gaps in their policies and processes or understand their ramifications.

PFTAC, due to its limited resources, cannot undertake major reform initiatives in all 15 countries simultaneously. It is important, therefore, that PFTAC prioritise it inputs. Nothing will be gained in undertaking detailed ROSCs in 15 countries if PFTAC lacks the resources to follow up with viable reform packages.

For capacity building, PFTAC initiative in developing a PEM training course seems a concept applicable to each of its four TA areas. A proper grounding in the basic concepts and practices of fiscal, tax, banking and statistics disciplines should be a forerunner to other capacity building activities such as on-the-job training, regional workshops, overseas attachments and stints with PFTAC as assistant advisors. Having a deliberate program where local civil servants could progress up such a training and development ladder would give greater meaning to PFTAC’s capacity building commitment.

4.2.2 Operational Diagnosis

A. Culture and Leadership

The current project Coordinator is a strong leader who is inclusive, consultative and results oriented, a much sought after, but rare blend in senior managers. He is appreciated for his openness by the stakeholders, has been effective in hiring and motivating the staff, and has shown considerable entrepreneurship in obtaining funds for such activities as PC/Trade. This has allowed PFTAC to accomplish more than it would have been able to do with just the core budget. Morale appears to be high amongst the staff of PFTAC, the evaluators observed a strong willingness of staff to cooperate and be self motivated. However, the evaluators noted that no formal values statement exists for PFTAC even though professionalism, commitment, openness and responsiveness stood out as common character traits.

B. Communications

PFTAC maintains a good, open and transparent website and should be commended on the openness and transparency of their operations and the abundant material that is posted on its website.

PFTAC and its resident advisors prepare three sets of reports: (a) back-to-office reports prepared at the end of a mission by a resident advisor or a short-term consultant; (b) monthly or three-monthly reports prepared by each resident advisor that are submitted to the backstop person or group in IMF HQ in Washington; and (c) three-monthly rolling work-plans and six-monthly activity reports for stakeholders and the periodic meetings of the TPRC.

The back-to-office report prepared by the long-term advisor or short-term consultant at the end of each mission provides a good summary and a clear record of what was accomplished on the mission and an update of the action plan going forward.

The monthly or three-monthly reports to Washington, DC provide detailed information on the activities undertaken, raise technical issues and seek advice where needed. They also often contain appended material such as back-to-office reports. The evaluators spoke to IMF staff in Washington about the reports and the IMF expressed their complete satisfaction with their format and content.

The third set of reports that are prepared by PFTAC are the three-month rolling work-plans and six-monthly activity reports that are presented to the stakeholders and the TPRC. The stakeholders indicated that these reports provided them with a good record of PFTAC’s inputs, activities and outputs. However, some donors indicated that they would like PFTAC to describe in greater detail the impacts, or outcomes, or achievements of the projects receiving PFTAC inputs. It should be noted that the donors supporting CARTAC have made similar requests of CARTAC. For PFTAC to report its accomplishments, not just its efforts, would require closer monitoring of the progress and end results of PIC projects to which it may contribute only a small, though vital part. This could be time-consuming and distract from its ongoing TA activity. One compromise might be to do a quick stock take of the outcome of projects to which it has previously contributed each time a resident advisor visited a PIC. This could be documented in a simple project outcomes table that set out (a) the intended stages of the project, (b) the objectives of each stage, (c) the nature of PFTAC’s contribution and (d) the advisor’s general impression of the subsequent success of each stage and the project as a whole.

PFTAC’s recent introduction of oral briefing by resident advisors at the TPRC meetings is commendable. This provides all of the stakeholders a further opportunity to either submit questions to the resident advisors prior to the meeting or to ask questions of them during the meeting. The evaluators understand that not all resident advisors will speak at all TPRC meetings.

C. Organisation Structure

The core organisational structure of PFTAC consists of seven people: the project Coordinator, four resident advisors and two administrative assistants. The core structure is augmented from time to time by short-term consultants who provide technical assistance, training and run seminars. Each position in PFTAC has terms of reference. The major drawback of this structure is that the absence of technical support staff within PFTAC and the limited funds for contracting short term experts restricts the extent of the activities that PFTAC can undertake and the outputs and outcomes that PFTAC can deliver.

The limited number of full-time staff and the limited amount of funds for outsourcing also severely limits the amount of capacity building that can be undertaken. PFTAC with its four resident advisors, but no tier of locally recruited technical experts, is a top-heavy organisation. Most professional service firms leverage their output through employing a few highly paid partners who coordinate a team of aspiring lower paid technicians. PFTAC will need to gradually move towards this model if it is to cope with the growing demands on its services, yet hold down the costs of taking on more staff.4 Yet to do so it will need to obtain more funding or it could become a victim of its own success as demands upon it outstrip its capacity to deliver.

D. Human Resources

The resident advisors are technically very competent and understand that the most rigorous solution to a problem may not necessarily be the best in all circumstances. That being said, there remains the thorny issue of interpersonal skills required to fit in with the Pacific Island Countries. The evaluators are not in a position to make a firm judgement on the root cause of the few comments made by some interviewees but they are aware that the IMF is aware of the issue and that the IMF and the project Coordinator place considerable weight on the interpersonal skills when selecting candidates. Many of the comments referred to previous Coordinators and advisors rather than the present team. When asked about this issue the current Coordinator stressed that he had gone to particular pains to ensure that cultural adaptability, not just technical excellence, was given high priority in selecting the latest group of resident advisors. It is important that any future project Coordinator be cognizant of the importance of the issue and that he or she continue to put considerable weight on interpersonal skills when selecting future resident advisors.

E. Processes and Systems

There is strong backstopping and quality control by the IMF. Both the IMF and the resident advisors expressed satisfaction with the process involving monthly or three-monthly reports, telephone calls and email. There is a lean back office with good files, procedure manuals and paper records. The one glitch the evaluators observed was the lack of adequate back-up of computer records. A hard drive crashed while the evaluators were in Suva and, as there was no personal or corporate back-up of files, the hard-drive had to be sent to Washington to see if the data could be recovered. The IMF’s Technology and General Services Department has subsequently indicated that measures are in train to provide appropriate back up facilities to overseas offices.

F. Controls and Metrics

The budget is currently based on line items (e.g. inputs such as salaries, travel, rent), largely due to the limited quantity of funds available for short-term consultants and training. In the event that more funds are made available for these activities, it would be useful to also present the budget in terms of activities (i.e. outputs such as ad hoc advice, reform initiatives, and personnel capacity building) or, at least the four main areas upon which PFTAC concentrates (i.e. fiscal management, banking/financial institutions, economic statistics and tax/customs). Activity based costing would require the keeping of time sheets on TA projects (like professional service firms) and apportioning the costs of corporate overheads (i.e. Coordinator, two administrative assistants, IMF backup, office rents, etc) between the four output areas.

The evaluators also observed that PFTAC has not developed a formal risk management plan. Finally, as mentioned elsewhere, PFTAC has not developed project outcomes indicators.

The following two tables provide a summary of the strategic and the operational diagnoses.

Table 4.2

Summary of Strategic Diagnosis

article image
Table 4.3

Summary of Operational Diagnosis

article image

4.2.3 SWOT Analysis

A SWOT analysis uses the findings from the performance diagnosis to document the main Strengths and Weaknesses within an organisation and the Opportunities and Threats it faces externally. The following two tables present a summary of the more important internal and external features of PFTAC.

Table 4.4

SWOT Analysis – Internal Features

article image
Table 4.5

SWOT Analysis – External Features

article image

In both this chapter and the next the evaluation team draws a number of conclusions and makes a number of recommendations for changes that will require some time for the project Coordinator and the resident advisors to prepare. There will thus be a trade off between preparing this extra reporting material and working in the field. It is important that PFTAC not allow the additional reporting to take a significant amount of time from their true function of providing TA to the PICs. Diligence will be required by PFTAC to ensure that they get the balance right.

Chapter 5 Conclusions and Recommendations

5.1 Conclusions

5.1.1 General Overview

  1. The present structure of PFTAC consisting of an IMF employed Coordinator and four IMF engaged resident advisors was the most appropriate model given the severe budget constraint under which PFTAC has been operating over the last several years. If additional funds were made available to PFTAC, the present structure should be deepened to accommodate a tier of locally recruited experts and broadened by using more outside consultants so as to better leverage its output and reduce its unit costs of service delivery.

  2. PFTAC has been well executed. PFTAC has played a positive role in the region and is appreciated by the participating countries, regional bodies, donor agencies and other stakeholders.

  3. PFTAC is well integrated within the region and fills a niche that cannot be filled by other projects. PFTAC’s activities are of high professional quality and are timely, appropriate and well executed.

  4. Its appreciation is in large part attributable to its flexibility and its quick response to requests for assistance. The lack of bureaucracy in making requests has been a contributing factor. There are neither extensive forms to complete nor complex proposals to write to justify a request. A letter of request has been the usual form.

  5. PFTAC’s task has been facilitated because the region has been welcoming. Recipients have been almost uniformly satisfied with the quality of assistance. Most interviewees with first hand knowledge of some of the training courses or advisory missions undertaken expressed their satisfaction with the speed and quality of the assistance provided.

  6. Ultimately, any technical assistance program is only as successful as the people engaged in its operation. No amount of elaborate project identification, formulation, and monitoring employed by TA providing agencies can guarantee results if the staff, experts and counter-parties involved in its implementation are unsuitable or uncooperative. PFTAC has been fortunate in having good experts selected for its core staff. However, several interviewees mentioned that it was very important that the resident advisors have excellent interpersonal skills and that this requirement should be an important qualification when selecting the advisors. There appears to have some problems with resident advisors on this score under the previous Coordinator.

  7. Given the limited resources available to PFTAC, the approach taken by PFTAC in developing its program of interventions and activities appears to have been sensible, though not always well articulated. PFTAC interventions appear to be most successful when they are fully integrated into a gap (ROSC) analysis, reform strategy and work-plan that are developed in conjunction with the participating country so it feels a sense of ownership and control of the project. No activity should be undertaken unless it is requested by an individual government or regional body. PFTAC may prompt such requests by identifying FEM gaps through ROSC reviews that closely involve key stakeholders within FEM agencies. It is important that in the planning of work that not all of a resident advisor’s time is allocated to long-term initiatives. Time must be left free for undertaking short-term ad hoc functions. In this way, PFTAC can remain responsive to the changing needs for TA in the region.

  8. The resident advisors in PFTAC (unlike those in a professional services firm) do not keep a record of the time that they spend on ad hoc, reform and capacity building activities. It is thus not possible to provide a definitive judgement on whether their time allocation between these three activities is appropriate. That having been said, the evaluators saw nothing that would lead them to believe that the time allocation was inadequate. Nevertheless, recording the allocation of time between ad hoc, reform and capacity building efforts would give the Coordinator, TPRC, IMF and other interested observers a better idea of how the organisation apportions its scarce resources between competing demands. Some donors would like to see better results on capacity building but the lack of capacity building is largely the result of insufficient resources.

  9. During the period under evaluation PFTAC’s advisory missions have been the most concentrated on Tonga, Vanuatu, FSM, Republic of Marshall Islands, and Fiji and least concentrated on Tokelau, Tuvulu, Nauru, Niue, and Cook Islands.

  10. PFTAC appears to have been responsive to the rapidly changing needs of the region although it may not have responded to the major shift in Australia’s approach to aid that started some six months ago. Australia now supports a more direct approach to TA than it did previously. It is deploying senior officials from both The Treasury and the Department of Finance and Administration to upgrade the FEM of PNG and the Solomon Islands. PFTAC may be able to attract more funding from Australia if it targeted these PICs for extra TA.

  11. PFTAC takes due regard to cost in its activities and runs a lean administrative back-office (just two local administrators to serve the Coordinator and four resident advisors plus other technical advisors). Its four resident advisors are highly paid, but not relative to technical experts in other international organisations (e.g. World Bank or elsewhere in the IMF) or expert FEM officials seconded by Australia to PNG and the Solomon Islands. Nevertheless, the unit costs of PFTAC’s services will remain high as long as it does not employ locally recruited experts to do basic research thereby freeing up resident advisors to concentrate on higher value tasks.

  12. The donors feel that greater effort could be made to measure the impact of the larger PFTAC interventions. It is interesting to note that CARTAC is also grappling with the issue of measuring outcomes and performance indicators. CARTAC is, for example, trying to develop Indicators of Change, which should help to measure and report on results and impacts. This is an area that warrants greater cooperation between the various TACs around the world.

  13. The procedures for monitoring individual activities are simple, transparent and effective. However, it is not always immediately apparent how some individual activities, while useful in themselves, fit into the overall strategic approach of PFTAC interventions. PFTAC could better articulate its TA strategy by dividing it into three distinct streams (ad hoc, reform and capacity building) and highlighting the work flow process within each stream). Costing the contribution of PFTAC to each TA stream and to each component within a stream would enable a better evaluation of the balance achieved between different forms of TA.

  14. The budget, provided that all pledged monies are made available to the Centre, is just sufficient for the current phase at the current level of activity. Although it is recognized that PFTAC operates in a different environment, PFTAC is much less well-funded than CARTAC. Over 50 percent of CARTAC’s budget is available for short-term consultants, seminars, training programs, attachments etc. Less than 18 percent of PFTAC’s budget is available for these activities. As a result, CARTAC is able to directly lever the presence of its five resident advisors whereas PFTAC is not. However, PFTAC has sought to lever its resources by working closely with other multilateral and bilateral donors operating in the region in arranging seminars, training programs and short and long term consultants. As neither donors nor member countries appear to countenance a major increase in funding for PFTAC at this stage, this approach will need to be enhanced over the next project period.

  15. The team, in talking with officials in Canberra, came away with the distinct impression that (a) Australia would prefer to give an additional lump sum to PFTAC rather than have to handle a steady stream of one-off requests for funds; (b) Australia would look favourably on a request from PFTAC for additional core funding and (c) Australia would like to see a reasonable proportion of any additional funds it gives being used for TA in the three countries on which it is focusing, namely PNG, Solomon Islands and Nauru.

  16. The Standard Operating Procedures for office administration tasks seem appropriate for a small office. The PFTAC office in Suva is well-managed. The one exception to this conclusion is that data security such as in-house back-ups and off-premise storage of data needs to be reviewed and improved upon.

  17. PFTAC produces a number of periodic reports including back-to-office reports after each field mission, monthly or quarterly reports to IMF HQ, and quarterly work plans and six monthly progress reports for members of the TPRC. The recipients of all four of these reports indicated to the evaluation team that they were satisfied with both the frequency and content of these reports.

  18. The IMF has done a good job as the Executing Agency: It has hired the right people and, while monitoring their work, has not been heavy handed in directing the Centre’s activities. Indeed PFTAC seems to exercise a considerable amount of autonomy in its decisions on how best to serve PICs.

  19. The TPRC is recognised by all players as the de facto principal governance body of the Centre and has been effective as such. It should be noted, however, that we were unable to discover any formal terms of reference for the TPRC. There was some uncertainty on the part of the Chairman of TPRC as to the bounds of the TPRC’s authority – for example does it have control over staff appointments? The evaluation team concludes that the TPRC’s success as a governing body has largely been due to the fact that the constituents of the TPRC have chosen to use the TPRC as the means of expressing their views rather than putting pressure on the Centre in bilateral meetings. Nevertheless, a formal governance charter outlining the respective responsibilities, authorities and accountabilities of PFTAC, TPRC and IMF would help to avoid possible misunderstandings over roles, delegations and reporting lines in future.

  20. Delivering technical assistance on a regional basis by having resident advisors who are based in the region has worked and is considered by most in the region to be superior to the alternative of trying to provide all TA for the PICs out of Washington, DC, although some specialised assistance may still be better provided by HQ. It is doubtful that PFTAC would have the same acceptance and trust if it operated out of Sydney, Auckland, Jakarta, Singapore, Manila or Washington DC.

  21. PFTAC’s training sessions and seminars, while not as numerous as CARTAC’s, have exposed participants to issues that occur in other jurisdictions. This has led to a greater acceptance to seeking regional solutions where they make sense. Regional secondments and attachments are a good example of how regional thinking is expanding. PFTAC’s small countries’ fiscal management training course at the USP, if successful, could be a model for developing courses in the other TA areas since giving local officials a good grounding in a FEM discipline would prepare them better for on-the-job training, specialist workshops and overseas attachments.

  22. Lack of capacity and sustaining the capacity that does exist is a major problem in Pacific Island Countries. PFTAC is helping to increase capacity in individual countries through TA and, to the extent that they are using local experts, increasing the technical capacity in the region. Many countries prefer consultants from the region because they know the region and engaging them increases regional capacity. Both the donors and the participating countries must accept the fact that good people in government are likely to be enticed into the private sector and other countries with more lucrative offers. As a result, capacity building is a long-term, on-going activity that, because it prepares people for better opportunities, will have a high attrition rate. The proposal for PFTAC to hire a tier of locally recruited experts from experienced PIC officials could help build and sustain a small but important talent pool of indigenous FEM professionals within the region for ready accessing by PICs. Such locally recruited experts could be hired on a contractual basis for 3 - 6 months and, to give incentive to highly skilled experts, might be paid above existing civil service rates. As such they would help to further enhance the PIC ownership of PFTAC whilst, in the longer run, perhaps helping also to reduce the average unit cost of service delivery for PFTAC as a whole.

5.1.2 Appropriateness

A. Objective

1. PFTAC is seen as a necessary organisation because:

  • It provides “help-desk” and ad hoc “crisis” type FEM technical assistance which is in high demand by the PICs and is not readily available, especially at short notice, from other TA providers in the region; and

  • It undertakes FEM gap analyses of PICs, based on international standards that are modified to suit local conditions, and using the results assists the PICs in preparing long term reform programs and work plans that provide a useful guide to FEM TA needs that can be used by all TA providers in the region.

2. The original design of PFTAC has been used as a model to create CARTAC and two regional TA centres in Africa.

B. Regional Location

1. Having PFTAC located in the region, rather than solely relying on TA from the IMF in Washington, DC or from other locations, has many advantages:

  • Dedicated focus on PICS;

  • Greater opportunity to attract regional funding;

  • PFTAC is perceived as a regional body, not as an IMF outpost;

  • PFTAC is more accessible to clients than if it were located elsewhere;

  • By being located in the region, PFTAC has better intelligence on the region than if it were located elsewhere; and

  • By being located in the region, PFTAC is closer to the main donors which allows for greater cooperation on TA projects.

C. Range of Functions

1. The original four functional areas are still seen as appropriate by most clients and donors. However, current tasks in the following two areas should be expanded or modified:

  • Banking supervision function should be formally expanded to include regulation over insurance and superannuation, i.e. the function should be expanded from banking supervision to include non-bank financial institutions supervision. Consideration should also be given to PFTAC’s future involvement in Anti-Money Laundering (AML) and Combating of the Financing of Terrorism (CFT) activities; and

  • Some Ministries of Finance and Central Banks also said that they needed more technical assistance in economic management and monetary policy. The consensus is that this should be handled through HQ.

D. Objectives and Functions

The reference to “economic management” that is contained in PFTAC’s project document and mission statement is not reflected in PFTAC’s day to day operations.

  1. PFTAC’s banking regulation and supervision function does not formally acknowledge its coverage of insurance and provident funds.

  2. The regulation of offshore financial centres is not yet formally recognised in PFTAC’s functions.

  3. It is not clear whether the TPRC and PFTAC are committed to greater regional cooperation of the smaller PICs, for example in developing shared service centres that could have a critical mass of expertise to undertake specialist functions (e.g. general accounting, auditing, IT, purchasing, HR development, etc) at a higher standard and at a lower cost than if each PIC tries to do these tasks on its own.

5.1.3 Effectiveness

A. Ad Hoc Technical Assistance

1. With respect to the ad hoc, help desk advice, PFTAC is effective on all counts.

B. Longer Term Reform and Capacity Building

1. On FEM reform advice and capacity building, the quality and timeliness of PFTAC’s work is high. However, the volume of PFTAC’s output and the access to its services is limited by the lack of sufficient resources that is available to PFTAC. It is recommended that all future reform work be preceded by a gap analysis (ROSC) that results in a reform strategy and action plan developed in conjunction with the appropriate FEM agency or agencies in the host PIC.

5.1.4 Efficiency

A. Front Office

1. There is a persistent view that IMF staff cost more than other donors. While this may be true in some donor cases, the total costs of the resident advisors is comparable to that of expert officials seconded to PNG, Solomon Islands and Nauru from Australia and those technical advisors employed by the World Bank. However, the lack of an economist, a tier of locally recruited experts and more outside consultants means that PFTAC cannot lever its scarce senior expertise the way that professional service firms do or the way that CARTAC does. By deepening and broadening its expertise base, PFTAC could reduce the average cost of its technical advice as well as further indigenise its staffing profile thereby reinforcing its acceptance amongst PICs. Eventually, veteran technical support staff might branch out on their own as part-time consultants to PFTAC, thereby expanding the available local FEM advisory skill base further.

B. Back Office

1. Corporate overheads of 25% of total cost and 30% of total staff are reasonable given the small size of PFTAC5. These ratios would improve further if PFTAC had a resident economist, a small number of research assistants and additional funds for short-term TA and capacity building.

The ability to backstop resident advisors with IMF support allows economies of scale in accessing specialist technical advice.

5.2 Recommendations

5.2.1 Corporate Governance

1. The respective roles, responsibilities, authorities and accountabilities of TPRC and IMF with respect to PFTAC should be more clearly enunciated.

2. A formal governance charter defining the role, duties, delegations and reporting obligations of PFTAC in relation to IMF, TPRC and individual donors, clients and other stakeholders should be drafted by PFTAC and submitted to the IMF and TPRC for endorsement.

5.2.2 Strategic Planning

1. PFTAC should undertake, in conjunction with key stakeholders (including donors), a situation analysis (i.e. PEEST analysis and scenario planning) to identify challenges within its region that need to be addressed in its strategic plan and new project document.

2. PFTAC should adopt a more comprehensive strategic planning process and presentation format that covers:

  • Situation, financial and stakeholder analysis;

  • Vision, values and mission statements

  • Objectives, strategies, functions and action timetables; and

  • Minimum guarantees of service, risk management devices, and key performance indicators.

5.2.3 Risk Management

1. PFTAC should develop, possibly with the help of an outside consultant, a risk management plan that identifies and ranks key risks in terms of their probability and impact and comes up with strategies for mitigating or managing them.

2. This risk management plan should be incorporated within PFTAC’s comprehensive strategic plan and the new project document.

5.2.4 Budgeting

1. PFTAC should undertake activity costing of each of its functions (something that will become increasingly important if PFTAC obtains a significant increase in funds) to identify the breakdown in expenditure:

  • Between the four functions or programs of Public Financial Management, Tax and Customs Policy and Administration, Banking Regulation and Supervision and Economic and Financial Statistics; and

  • Between the three forms of technical assistance or outputs consisting of (a) short-term ad hoc advice often consisting of one-off assistance involving less than one month, (b) longer-term reform assistance often consisting of repeat visits and lasting many months or years, and (c) capacity building consisting of short-term and longer-term personnel training and development within each of the four functional areas.

2. Again if PFTAC is able to get significant incremental funds, PFTAC should submit its budget on a line item, program and output basis to the TPRC and report its annual financial results on this basis on its website.

5.2.5 Technical Assistance Framework

1. PFTAC should measure what shares of its TA goes on (a) reactive ad hoc advice, (b) pro-active reform assistance, and (c) capacity building such as extended courses, short-term workshops, overseas attachments and on-the-job training.

2. PFTAC should articulate a TA framework and strategy for helping the PICs adopt a relevant FEM reform vision and use gap analysis (e.g. ROSC) to devise jointly agreed strategies and action plans upon which future PFTAC and donor TA can be based. Such a framework should also accommodate short-term ad hoc advice and ongoing capacity building. This framework and strategy should also recognise that progress cannot be made in all 15 countries simultaneously. Rather one or a few pilots should be developed and then rolled out to other PICs as with PC/Trade.

uA01fig01

Proposed Technical Assistance Framework

Citation: Policy Papers 2004, 038; 10.5089/9781498330060.007.A001

5.2.6 Funding and Organisational Structure

1. In order to satisfy the demand for ad hoc advice, reform assistance and capacity building the organisational structure should be expanded as follows:

  • While it would not be useful to duplicate within PFTAC the economic research work of IMF HQ economists, PFTAC should examine the need for periodic employment of specialist advisors in specific fields of concern to the PICs as a means of complementing the economic research of IMF HQ economists, e.g. in public debt compilation, measurement of taxation expenditures, construction of macroeconomic frameworks for PIC multi-year budgets etc.

  • PFTAC should introduce a new tier of locally recruited experts to support the resident advisors. This should be piloted with one or two experts to start with and if successful further recruits could be sought if sufficient funds were available. The proposed local experts should be mature professionals who have research and policy advisory experience as employees of FEM agencies of PICs. Such local experts should be employed on 3 – 6 month contracts with renewal subject to satisfactory performance.

  • PFTAC should use more external consultants to provide additional TA, especially capacity building, in the PICs.

2. PFTAC should develop a detailed budget of the additional funds that would be required to cover these additional resources and this budget should be presented to the donors for additional funding.

5.2.7 Technical Assistance

1. PFTAC should expand the TOR of the resident advisor on banking regulation and supervision to cover also insurance and superannuation. Consideration should also be given to PFTAC’s appropriate future involvement in AML/CFT activities.

2. PFTAC should place greater emphasis on helping PICs develop the capacity to plan and roll out long-term reform strategies and help the PICs get additional funding from other regional donors.

5.2.8 Performance Reporting

1. PFTAC should maintain a registry of all PIC projects to which it contributes significant TA.

2. The following information should be furnished for each project:

  • Client, purpose, stages, timetable and expected outcomes;

  • PFTAC’s contribution in terms of tasks, time, costs and outputs; and

  • Project outcomes relative to original expectations.

3. Each project’s success should be rated on a scale of 1 to 5 on its completion or suspension.

4. The results of this performance reporting should be published on PFTAC’s website along with the main lessons learned from the experience.

5.2.9 Client Feedback

1. PFTAC should develop a minimum guarantee of services statement to reinforce its high reputation and to help manage and thereby contain the burgeoning expectations of both clients and donors

2. PFTAC should develop a formal feedback mechanism for capturing the extent of client satisfaction with its individual field services (i.e. significant ad hoc services, reform assistance and personnel capacity building). For instance at the end of each assignment the client could be asked to answer a simple questionnaire that invites a score out of five for (a) satisfaction with the project’s terms of reference, (b) the personal conduct of the advisor, and (c) the usefulness of the technical advice.

5.2.10 Information Exchange

1. PFTAC should exchange information with other TACs (CARTAC, AFRITAC-East, and AFRITAC-West) on processes, systems, structures, strategies and measurements that have proven useful for planning, delivering and reviewing technical assistance within the FEM area.

2. Coordinators of all TACs should meet annually in Washington, DC to discuss their mutual challenges, how they are tackling them and lessons to be learned from their experiences. This forum could also be used for exchanging each TAC’s strategic plan, work plans, progress reports, FEM application models (e.g. PC/Trade, PEM course, etc).

3. A copy of this Evaluation Report should be given to the Coordinators of each of the other TACs as it is the first example of an evaluation of a mature TAC.

5.2.11 Shared Service Centres

1. The FEMM or TPRC should commission PFTAC to undertake a review of the relative merits and potential cost savings of introducing shared regional service centres, especially for smaller states, in the public financial management areas such as accounting, reporting, budgeting, financial information systems and auditing.

2. Funding and partial staffing for such an exercise should be sought from the ADB and/or other donors.

5.2.12 PFTAC Name

While no change is recommended at this time, the IMF in conjunction with the TPRC should consider at a future review whether PFTAC’s name should be changed to PICTAC to conform with the nomenclature for other regional TACs, more accurately define the region to which it applies (PICs) and to facilitate expansion of its functions to cover economic, not just financial matters.

5.2.13 New Project Document

A new project document should be prepared for the next phase of PFTAC. This document should incorporate the results of many of the recommendations made above.

Evaluation of the Pacific Financial Technical Assistance Centre
Author: International Monetary Fund