Evaluation of the U.K. DFID-Financed Technical Assistance GDDS Project for Selected Anglophone African Countries (2001-06)
Author: International Monetary Fund
Publication Date:
21 Mar 2008
eISBN:
9781498334815
ISBN:
9781498334815
Language:
English
Keywords:
PP; TA mission; GDDS TA recommendation; TA delivery; GDDS coordinator; GDDS country; GDDS framework; GDDS norm; project country; TA project; priority plan; fund staff travel cost
Download PDF (1,022 KB)

This review of Phase I of the technical assistance (TA) General Data Dissemination System (GDDS) project (2001–2006) for 15 Anglophone African countries1—funded by the U.K. Department for International Development (DFID) and executed jointly by the Fund and the World Bank—(henceforth, referred to as the Anglophone African (AAf) project) focuses mostly on the components that were implemented by the Fund’s Statistics Department (STA). The review draws on various internal and periodic evaluations of project execution and reports by technical assistance providers. The GDDS—part of the Fund’s Data Standards Initiative—defined the framework for the AAf project. The main goal of the project was initially limited to assisting countries to become participants in the GDDS via preparatory workshops and development of metadata and plans for improvement. The goal was subsequently expanded to providing TA and promoting greater awareness and regional cooperation.

Abstract

This review of Phase I of the technical assistance (TA) General Data Dissemination System (GDDS) project (2001–2006) for 15 Anglophone African countries1—funded by the U.K. Department for International Development (DFID) and executed jointly by the Fund and the World Bank—(henceforth, referred to as the Anglophone African (AAf) project) focuses mostly on the components that were implemented by the Fund’s Statistics Department (STA). The review draws on various internal and periodic evaluations of project execution and reports by technical assistance providers. The GDDS—part of the Fund’s Data Standards Initiative—defined the framework for the AAf project. The main goal of the project was initially limited to assisting countries to become participants in the GDDS via preparatory workshops and development of metadata and plans for improvement. The goal was subsequently expanded to providing TA and promoting greater awareness and regional cooperation.

Executive Summary

This review of Phase I of the technical assistance (TA) General Data Dissemination System (GDDS) project (2001–2006) for 15 Anglophone African countries1 —funded by the U.K. Department for International Development (DFID) and executed jointly by the Fund and the World Bank—(henceforth, referred to as the Anglophone African (AAf) project) focuses mostly on the components that were implemented by the Fund’s Statistics Department (STA). The review draws on various internal and periodic evaluations of project execution and reports by technical assistance providers. The GDDS—part of the Fund’s Data Standards Initiative—defined the framework for the AAf project. The main goal of the project was initially limited to assisting countries to become participants in the GDDS via preparatory workshops and development of metadata and plans for improvement. The goal was subsequently expanded to providing TA and promoting greater awareness and regional cooperation. Thus, a good part of the project was process oriented.

This evaluation endeavors to provide an overview of the project without reciting all the component details. It indicates a broadly successful outcome of Phase I of the project, both for the recipient countries and the Fund. During the implementation of the project, 14 out of the 15 countries participating in the project became participants in the GDDS and used the framework to make progress in improving their statistical systems. An examination of country-by-country experiences reveals a number of concrete outputs achieved in the sectors targeted by the project. However, a number of problems have hampered reforms, reflecting the long-term nature of capacity building and the still-inadequate national priority and funding for statistical reforms.

The results to date suggest that the DFID-financed AAf project provided an opportunity to STA to strengthen delivery and coordination of TA to a group of countries trying to build statistical capacity. New challenges emerged via the requirements imposed by external funding, coordination with other Fund departments, the World Bank, and regional TA providers. The outcome broadly points to a good response by STA. DFID funding greatly increased STA’s leverage and reach for this kind of effort, enabling coordinated assistance to cover more topics and with more missions than would otherwise have been possible. During the course of the work, further progress has been made in integrating the project TA into the Fund’s broader objectives, notably, enhanced collaboration with other TA providers, the medium-term budget envelopes, and the assessment of outcomes. In this context, the AAf project has assisted STA to take an early lead in anticipating and implementing some of the subsequent Fund-wide recommendations in the provision of TA, notably those by the Fund’s Internal Evaluation Office and the joint Fund/World Bank Report on collaboration.

The project has also intensified STA’s support to international initiatives through the Marrakech Action Plan for Statistics and the Paris Declaration on Aid Effectiveness. In this context, greater efforts are being made to strengthen the links between the GDDS and the poverty reduction strategy approach in order to enhance national and donor support for statistical reforms. Such an outcome would underpin improvements in source data critical to the compilation of macroeconomic statistics.

Flexible budgetary arrangements within the project permitted the executing agencies to use their own administrative and accounting practices. Such arrangements made it easier for STA to incorporate the project activities into its own work program and procedures. This ensured that TA and its providers—mostly outside experts—were supervised by Fund staff to ensure quality control, while at the same time providing a basis for subsequent evaluation to assess the effectiveness of TA delivery. In this context, STA extended the coverage of the Technical Assistance Information Management System (TAIMS) to the project and information obtained from a limited number of project evaluation reports indicate that TA was broadly effective.

Funding by DFID enabled STA to augment its resources to substantially expand the TA resources available to support the TA program for Africa and especially the AAf project participating countries. At the same time, the parallel execution of the project resulted in efficiency gains mainly through lowering of “transactions costs” for recipient countries, by having a common regional advisor, attending joint workshops, which covered both Fund and World Bank issues, and use of a common GDDS framework.

DFID financing has now been extended into a second phase of the project for an expanded set of AAf countries. This Phase II incorporates important changes in approach toward a more modular, multicountry platform of TA delivery, reflecting lessons learned in Phase I of the project.

To date, while a number of participating countries have made concrete progress in their statistical description, output, quality, and dissemination, longer-term sustainability is not assured. Some progress is being made in efforts to secure priority in allocating medium-to long-term funding for statistical reforms within the poverty reduction strategy approach. Other efforts have been made through regional cooperation and to promote peer pressure while an attempt has been initiated to encourage some countries to graduate to the Special Data Dissemination Standard where sustained compilation and dissemination is monitored. It is encouraging, however, that DFID remains committed to continued funding, which remains essential to consolidating the gains achieved to date.

I. Introduction: Background and Relevance of the Project

1. This paper provides a summary review of Phase I of the General Data Dissemination System (GDDS) project (2001–06) for 15 Anglophone African countries, which was funded by the U.K. Department for International Development (DFID). As indicated by the name, the project is framed within an overarching Fund framework—the GDDS, as part of the Data Standard Initiative (DSI).2 It has two goals—to build national capacity through the GDDS to compile and disseminate metadata and the relevant data themselves and strengthen governance and decision-making in the private and public sectors through timely and transparent dissemination of high quality data as an ultimate objective. In this context, the technical assistance (TA) provided to countries is highly supportive of other key Fund surveillance and program objectives. The project was executed jointly by the Fund and the World Bank, but the present review focuses on components of the project that were implemented by the IMF Statistics Department (STA).

2. The review draws extensively on six-monthly and final reports prepared by STA and the World Bank, mainly for the benefit of providing work progress reports to DFID, and three joint project reviews (by DFID, the Fund, and the World Bank) undertaken in 2002, 2003, and 2007; the last of these took place several months after the project had closed. Additional information has been extracted from various TA reports generated by the missions and from interviews with country officials, as well as project evaluation reports (PERs) generated by STA on missions to Ghana, Kenya, and Sierra Leone—all visited by the joint missions during the 2007 evaluation. Furthermore, since DFID decided to extend the project under Phase II (2006–2009), this successor project was designed mostly during 2005/06, drawing extensively on the lessons of the Phase I experience. Indeed, the execution of the two phases of the project overlapped for a few months (May–December 2006), while the delayed final evaluation of Phase I also had to take into account developments that were taking place under Phase II. Accordingly, this review draws also on some aspects of the early stages of Phase II.

3. Under the DSI, the Fund introduced the Special Data Dissemination Standard (SDDS) and the GDDS. The former—intended largely for countries enjoying or seeking access to the international capital markets, with rigorous compilation, data-reach, and timeliness standards, and dissemination commitments for each participant—was approved by the Executive Board and established in 1996. In contrast, the GDDS—which was established in 1997—emphasizes the capacity building aspects, including plans for improving the statistical framework, and therefore is focused mainly on the preparation and dissemination of “metadata” (i.e., descriptions of how and from what sources statistics are compiled). It has also clear, but less challenging, goals—on the coverage of data, as well as periodicity and timeliness associated with the publication of data prepared by the national authorities. Countries process, at their own pace, the basic data dimensions for the GDDS system—core data categories and indicators—as indicated in Appendix Table 1. Participation in both the SDDS and the GDDS is voluntary, but there are expectations about performance and adherence for countries that choose to subscribe/participate in these systems. For the SDDS performance is mandatory and monitored. However, the GDDS is not strictly speaking a “standard” and is not monitored.3 To date, over the 12-year life of the DSI, the number of countries taking part in the SDDS has grown, now reaching 64, and those participating in the GDDS has also grown, now totaling 90.4

Appendix Table 1.

The Data Dimension of the GDDS: Core Data Categories and Indicators (IMF Only)

article image
Note: World Bank sociodemographic indicators omitted.

4. The value added from this overall review of the AAf project is that it provides a largely Fund perspective on the project, addressing several issues, namely (1) the progress towards building capacity through the GDDS to compile and disseminate data and how that may have affected decision-making; (2) how external financing was used to enhance TA delivery, collaboration with the World Bank and other TA providers, and promote regional collaboration; (3) how an externally financed project was integrated into STA’s other activities; (4) the mechanisms to review the project and assess its overall effectiveness; (5) lessons that were drawn from Phase I and how they have been adapted to the current TA program; and (6) what has been the experience with sustainability of capacity building efforts in the project? As noted above, the report relies on extensive available documentation of experience under Phase I of the AAf project. Thus, the following assessment and review sections do not venture into fine detail, nor do they focus intensively on experiences with individual countries.

5. The remainder of this paper covers the project and various aspects of its evaluation (Section II), its reviews (Section III), the integration of the project in STA’s overall TA program (Section IV), lessons that have been learned and incorporated in current TA programs, including Phase II (Section V), long-term issues, including sustainability, (Section VI), and the conclusions in Section VII.

II. Outcomes of the AAf Project

A. The Project

6. The immediate goal of the AAf project, especially Phase I, was to assist several African countries to join the GDDS. This goal reflected the need to prepare the necessary background—mostly advocacy and developmental steps—to help participating countries to fulfill the membership requirements, as a minimum. This would be followed by gradual shift in emphasis to the provision of TA to fill the reform needs identified in the plans for improvement.

7. The GDDS framework (for details, see IMF, STA, 1998) provided a suitable vehicle for the project. It offered an existing international initiative for DFID to support with financial resources and an opportunity to collaborate with the Fund and the World Bank through its extensive knowledge of and operations with the target countries. Moreover, the GDDS takes a comprehensive approach, which covers both macroeconomic/financial and socio-demographic statistics and emphasizes collaborative country and development partners’ efforts to facilitate the statistical development process. Thus, the GDDS framework’s emphasis on interagency statistical collaboration—focusing on the role of the country GDDS coordinator5—provided an opportunity to exploit the synergies of TA provided by the Bank and Fund to the various sectors of the national statistical system.

8. Box 1 illustrates the layout of the AAf project, including financing and administrative arrangements6, logistic, and project monitoring and evaluation as well as the roles of the participating entities.

The AAf Project, Phase I, November 2001–December 2006

Objectives: (1) Assist countries to join the GDDS via preparatory workshops and help with the development of metadata and plans for improvement; (2) provide TA to help countries implement plans for improvement in their statistics as set out in the metadata (phased in as countries became participants in the GDDS); and (3) foster cooperation among agencies at the national and regional levels through workshops (added while project was being executed).

Country participants: Fourteen (14) original countries comprising Botswana, Eritrea, Ethiopia, Kenya, Lesotho, Liberia, Malawi, Namibia, Nigeria, Sierra Leone, Sudan, Swaziland, Zambia, and Zimbabwe (not active because of arrears to the Fund). Ghana joined in 2004.

Monitoring and evaluation: Six-monthly progress reports, leading to 10 such reports over the life of the project; joint DFID/Fund/WB reviews as specified in the project document to ensure project objectives are on target and modified as necessary. Three mission reviews in 2002, 2003, and 2007, which visited some of the countries. Furthermore, all TA missions were subject to the regular Fund/STA monitoring and evaluation practices, including entries into the technical assistance information management system (TAIMS) and preparation of end-of-mission assessment reports (EMAR).

9. The IMF, through STA, focused on the overall GDDS and macroeconomic statistics issues and was responsible also for administrative aspects of the joint operations of the project, including for the regional advisor (a total combined cost of US$5.1 million). The World Bank was responsible for socio-demographic statistical aspect of the project. The project managers, based at headquarters, and the regional manager, based in Nairobi, were responsible for day-to-day operations of the project.

10. While DFID provided the bulk of the financing, the Fund and the Bank provided headquarters support and coordination of the TA extended to 14 (eventually, 15) participating Anglophone African (AAf) countries. Under the terms of Phase I, Fund staff travel costs were not covered by the project. Thus, bilateral TA over 2002–2006 in Fund-specialized areas was delivered entirely by outside experts. STA staff provided supervision of mission work (backstopping) to ensure quality control, and participated in certain workshops and regional events.

B. Outcomes—Participation in the GDDS

11. The GDDS framework sets specific goals as to statistics compilation and dissemination by national authorities, but the most visible requirement for participants has been the preparation, dissemination, updating of metadata concerning these compilations, and posting by the Fund on the Dissemination Standards Bulletin Board (DSBB). Since September 1996, the DSBB has been the central location to which SDDS and GDDS countries submit their metadata and other information with regard to their statistical compilations (e.g., plans for improvement, contact agencies, dissemination procedures). The GDDS norms do not require that participating countries disseminate their metadata in national publications or on official national websites, although certainly this would be encouraged, and is done by some countries.7

12. Figure 1 illustrates the progress by countries in different regions (by Fund area departments) and the AAf project countries in joining the GDDS. Africa (AFR) made, by far, the most progress in this regard during 2002-06, with a substantial contribution by the project participants. Fourteen of the 27 African countries that joined the GDDS between 2002 and 2006 were participants in the project. Within the project, AAf countries got off to a fast start in 2002–2003 with 12 of the 15 joining the GDDS after preparing their metadata and complying with other GDDS guidelines. For different reasons, two countries were unable to participate fully until 2005 (Ghana joined the project only in 2004, while Liberia suffered a prolonged civil war). Thus, both these countries have notably less experience, so far, with GDDS than the other participants

Figure 1.
Figure 1.

Regional and AAf Project Countries’ Representation in the GDDS and SDDS

(In percent of IMF member countries in each region)

Citation: Policy Papers 2008, 019; 10.5089/9781498334815.007.A001

Source: IMF Statistics Department.

13. Additional commitments of joining the GDDS include the nomination of a GDDS coordinator, who brings together the work of all statistical agencies for the preparation and update of metadata. In the project countries, these coordinators also served as contact persons for communication with project managers at the Fund and the Bank for the prioritization and scheduling of TA and other project matters. Some of the project countries also created interagency technical committees for the discussion of issues of common interest, and some of these committees continued to function as coordinating bodies for the prioritization of reforms and requests for TA.

C. Capacity-Building Outcomes

14. Project activities with the provision of TA in specific areas picked up momentum as more countries became participants in the GDDS. This transition was significantly facilitated by the preparation of plans for improvement, which constituted important outputs for countries early in the project and were an integral part of their metadata (Box 2). Figure 1 illustrates the sharp increase in GDDS participation of AAf countries after the launch of the project which was a key output of the project and a pre-requisite for other outcomes expected to be achieved within this framework. In the following paragraphs, the main inputs and activities that were used to achieve outputs are summarized, and then a number of project outputs are illustrated.

TA Prioritization and Method of Delivery in Phase I of the Project

Technical assistance was based on “Areas in Need of Improvement” that were identified in the GDDS metadata. Some countries also prepared project-specific “TA priority plans,” usually with assistance from the Regional Advisor. In a few cases, countries or agencies submitted ad hoc requests. Although the assistance was “custom-designed,” in each of the topical areas, requests for assistance turned out to be concentrated on a limited number of issues.

For example, in national accounts, much of the work was related to improving the use of available data from administrative sources. In government finance statistics, nearly all missions focused on issues of coverage and classification.

As to the method of delivery, for the Bank the usual pattern was that TA was provided in a single visit or one follow-up mission. For the Fund, the usual pattern was a series of three or four missions. The Fund started with four dedicated experts, one for each of its topical areas. The pool of experts was broadened to more than 30 individuals as the project progressed. The Bank usually left the choice of the expert to the country, guiding them with a short list of qualified candidates.

Fund TA was provided in the context of the practices of the Statistics Department, which provides for the professional supervision of field experts (backstopping) by the division in charge of the topical area of statistics, while Bank TA was undertaken under the supervision of the project manager.

The Fund used its customary form of assessment for each mission, which consists, inter alia, of a numerical rating by the expert on the extent to which each task was accomplished.

15. Table 1 summarizes the number of technical assistance and other missions to the countries during Phase I, broken down by fiscal year. There were 281 missions in total—of which 189 were Fund missions and 92 were Bank, and of which 233 were “topical TA missions” and the remaining 48 were workshops and regional conferences. In time profile, these missions were not particularly “front loaded,” in the sense that the number over 2005–2006 was comparable to those in 2003–2004. Thus it can be seen that the GDDS effort was sustained over the entire interval reflecting, to some extent, the buildup in momentum and “follow-up” on previous work that the predicable financing from DFID enabled during this period.8

Table 1.

The AAf Project: TA Missions and Other Project Activities, 2001–06

article image
Source: IMF Statistics Department.

16. Table 2 provides a decomposition of the 189 Fund missions by country of destination and mission topic, and Appendix Table 2 gives further perspective on the chronology of these activities. For the most part, participating AAf countries received between 10 and 20 missions each (with the notable outliers of Eritrea and Nigeria), well dispersed across substantive statistical sectors: external, fiscal, monetary/financial, national accounts, and prices, with some missions also focusing on broader topics of “GDDS and Planning.” Ghana and Liberia, although latecomers to, or inactive in, the early part of the project, enjoyed relatively full complements of TA missions in their tenure to date. The pattern of TA reflected in this table conveys some impression of areas of greatest statistical difficulty confronting these countries, with the external sector, fiscal, and national accounts among them accounting for about half of all missions.

Table 2.

The AAf Project: IMF TA Missions and Other Activities by Country, 2002–2006

article image
Source: IMF Statistics Department.

Botswana, Lesotho, and Swaziland.

Appendix Table 2.

TA Missions and Other Activities by Period, IMF, 2002–2006

article image
Source: IMF Statistics Department.

17. To complete the general contour, Appendix Table 3 shows the time profile of all TA missions, broken down by recipient countries. Here, too, the continued momentum in the last couple of years is apparent and, read closely, reflects some mission offsets between late joiners and initial members who received more of their TA in the first couple of years of the endeavor.

Appendix Table 3.

Activities of the AAf Project, Phase I, from Nov. 2001 to April 2006

article image
Source: IMF Statistics DepartmentNotes: Source of data-semi-annual progress reports, primarily annexes detailing all missions; p/m = persons month from budget data, Other include participation and presentations by Reg. Adv and WBTL in various regional events, Multi-country training events include sub-regional training workshops, joint training with other organizations, study tours, professional attachments.

18. The project results related to the second objective, namely, helping countries to build capacity through technical assistance, typically are specific to individual countries and topical areas in which TA is provided. The fact that there were 14 active countries, and four topical areas for the Fund, and eight sectors for the Bank (see Appendix Table 4), resulted in a large number of potential areas of assistance. These can be seen as “cells” in a two-dimensional (country-topic) matrix. Most countries requested assistance only in a limited number of topical areas. Box 3 provides an overview of project outputs in specific areas for individual countries, as verified by successive TA missions to participating countries. The outputs listed in this box are just a few of the concrete results that were achieved in certain countries and within specific sectors. This box omits mention of “more general” or less tangible steps forward, such as agreeing with national staff on steps to be taken sometime in the future, and developing and posting metadata on the DSBB. These reports suggest, not surprisingly, a certain gain in momentum in achieving results as the project progressed toward its conclusion in 2006, with specific advances in a larger number of countries and multiple steps forward in some of them.9 This is in line with the fact that many significant outcomes from statistical capacity building can only be achieved with sustained effort over a period of time, and are not in the realm of “quick fixes” that can be achieved with one or two missions.

Appendix Table 4.

TA Missions and Other Activities by Country, World Bank, 2002–2006

article image
Source: IMF, World Bank, and DFID, 2007a.

Selected Outputs Achieved in AAf Project Countries, 2002–06

article image
Source: Reports of TA experts after visits during the program.

A 2001 data ROSC mission had found Botswana’s monetary data to be incomplete.

A 2005 data ROSC mission to Kenya had recommended these data be compiled and published.

In line with recommendations from a data ROSC mission to Malawi conducted in 2004.

19. Evaluations of the AAf project by participant countries have tended to be positive, while at the same time acknowledging that more needs to be done to address continuing problems with data compilation and dissemination. Box 4, for instance, summarizes comments received by Fund staff at the launch workshop for Phase II of the DFID project, in September 2006 (and see also Box 7 and the Appendix, “Strengths and Weaknesses of the AAf Project, Phase I). The perceptions outlined in these comments are commensurate with those cited earlier from the provider’s perspective in various TA mission reports.

Interviews with AAf Project Participants at End of Phase I: Summary of Views

During the project workshop held in Cape Town, South Africa, in September 2006 (as Phase II was being launched), country delegations were surveyed on their views about the achievements and problems with Phase I.

In general, countries found that the AAf project had helped agencies and individual compilers in better understanding and identifying gaps in data series and methodologies. Most agencies indicated that data series had improved, and that they now have shorter time lags and better coverage. Some also reported greater response rate to questionnaires.

The ability to maintain current data quality is improving but still fragile in some areas. Asked whether agencies would be able to maintain the current quality of data, country representatives answered yes in several cases, but also stressed their intention to improve further (“not there yet”). The ability to maintain data at the current standard was perceived by several countries as still fragile due to such factors as staff not yet fully trained or staff turn-over, software problems, data not yet ready for publication, or outstanding gaps to close.

Countries are able to keep metadata up-to-date. Countries that have participated most actively in the AAf project are able to update or confirm their metadata on a regular basis, once a year. Some countries also observed that the awareness seminars helped in creating a demand for quality data, and, in some cases, provided a better understanding of roles and responsibilities among data-generating agencies.

The technical assistance received through expert missions was generally of high quality. Particularly useful were those missions that included general briefing or training for a larger audience or a good wrap-up meeting. In a few cases, a longer duration would have been helpful. The visits of the Regional Advisor and the World Bank team leader had helped create better understanding at the managerial levels and in coordinating ministries for improving data using GDDS as an organizing framework. The subregional seminars involving 4–6 countries on a specific topic were rated as very effective for training and mutual learning.

Summary Findings of the Third Review of the AAf Project, Phase I (2001–06)

The evaluation assessed DFID’s support for statistical capacity building through a combination of desk-based study and interview, and through country case studies in Ghana, Kenya, and Sierra Leone. The evaluation examined questions such as how well the project was managed and overseen, whether TA reached recipients in a timely manner, and what outcomes the project had delivered.

The evaluation team found that Phase I of the AAf project was well managed. There was some evidence of effective coordination between the WB and IMF, each playing to their comparative advantage. Other key findings were as follows:

  • The project is accessible and benefits from good communication between project staff and key stakeholders within recipient countries.

  • The GDDS regional advisor, based in Nairobi, is critical to both communication and coordination. As a result, countries have been able to benefit from timely access to high-quality expertise, drawing on the comprehensive pool of consultants available to the AAf project.

  • Most project-funded TA has been delivered very effectively. This evaluation did, however, find a few exceptional examples of incomplete TA (generally due to local constraints) or advice that was not entirely appropriate to the local context.

  • It is therefore clear that TA has been successfully delivered, and country capacity built, across a broad range of subjects—including development of metadata and plans for improvement, national accounts and prices, balance of payments, health and population data, and agriculture sector data.

  • GDDS awareness workshops have been particularly well received and have made a contribution to improved cross-agency communication.

  • It is harder to tell whether and how better data collection, analysis, and dissemination are contributing to planning and policymaking.

  • However, there is evidence that countries’ borrowing costs are reduced by 8 percent when signed up to the GDDS and up to 20 percent when the country meets the SDDS standard.1/

  • This and other ‘purpose level’ impacts will be explored in greater detail in the full GDDS Phase II evaluation planned, for late 2008 or early 2009.

Based on the findings outlined above, the evaluation team identified a number of issues and recommendations for consideration, which are summarized as follows:

  • Rearticulate the role of the GDDS regional advisor, and support national GDDS coordinators in redefining their role and reasserting the importance of their work (i.e., within the modular approach in Phase II).

  • Continue to consider how cross-government working, intercountry networking and sharing of knowledge will be sustained under the modular approach of GDDS Phase II.

  • Support countries to make annual updates to their metadata and to log the changes. Metadata should also be posted on national websites and linked to the IMF website. Encourage countries to document progress against their plans for improvement.

  • AAf project staff and partners should continue to give serious consideration to how best to strengthen local consultancy capacity.

  • Greater attention should be given to whether country plans are realistic, to the linkage between metadata, plans for improvement, and NSDSs, and to monitoring project outputs against this backdrop. This should include monitoring whether and how GDDS TA recommendations are being implemented.

  • The AAf project team should consider undertaking a risk assessment and mitigation planning exercise, covering both project risks and contextual risks. The aim of this exercise would be to increase the sustainability of project capacity-building efforts and, therefore, its contribution to purpose-level objectives.

Source: DFID, 2008.1/ See www.imf.org/external/pubs/ft/wp/2006/wp0678.pdf.

D. Results Related to the Enhancement of Cooperation

20. The first review of the project recommended that the project promote and, in a limited way, sponsor, the holding of “National GDDS Awareness Workshops.” Subsequently, such workshops were held in a majority of project countries and a few countries held second workshops the following year (see Box 5).

Experience with the National GDDS Awareness Workshops

Awareness workshops were usually one-day events which attracted between 50 and 200 participants. The agenda usually included presentations by the Regional Advisor on the role of the GDDS, presentations by agency representatives on improvements made as a result of TA received, and general presentations on the activities of statistics agencies. During the workshops, there usually was active discussion and, in most cases, some national press coverage.

Participant evaluations were overwhelmingly positive. It was pointed out that, in many cases, these workshops provided the first opportunity for participants to get an overview and better understanding of the activities of and linkages among data-producing agencies. The impact clearly went beyond the narrower objective of promoting knowledge of the GDDS. Typically, these workshops were seen as “eye openers” with regard to the functioning of the statistical system. These events were cost-effective, as the contribution by the project was limited to hiring a meeting hall, where necessary, the preparation of materials for participants, and some hospitality such as coffee and lunch, at an average expense of about $3,000–$4,000 per workshop.

21. The project also sponsored a number of other activities that were designed to foster cooperation at the national and regional levels (for instance see Box 6). These included country visits by the regional advisor, in which assistance was provided for the development of technical assistance priority plans at an interagency level. These commenced in early 2002 with a workshop in Botswana targeting the sensitization of senior officials10 in project countries, followed in December 2003, with another regional workshop in Namibia for national GDDS coordinators and/or their deputies. Subsequently, annual regional workshops provided the opportunity for senior staff from statistics agencies and GDDS coordinators to review project developments and engage in discussions of selected issues of common interest.

An Experience with a Regional Event

In response to the request by the Governor of the Central Bank of Lesotho, expressed also on behalf of Namibia and Swaziland (all three countries forming a currency arrangement with South Africa), a series of three seminars on the statistics of Private Capital Flows (PCFs) and the necessary survey methodology was launched in mid-2004. In addition to participants from the three countries, Macroeconomic and Financial Management Institute of Eastern and Southern Africa (MEFMI), which had been active in the region with related work, was invited to send an observer.

The first seminar prompted two observations: (1) The participants engaged in active exchanges of views, turning up experiences on the subject that were not fully known. They found these discussions stimulating and useful; (2) The observer sent by MEFMI was a senior official from the Bank of Uganda, who was in the process of introducing in his own country the procedures that were the subject of the seminar. This brought a dimension of reality to the discussions that could not have been achieved with lecturing alone.

However, the formation of human capacity, which usually is the target outcome of training activities, did not necessarily translate into formation of institutional capacity in the participating countries. Supplementary bilateral expert visits were required to help implement reforms on the ground.

Following up on an invitation by the observer from Uganda, the third and last seminar was held in Kampala and combined with a study tour, giving participants the opportunity to see a functioning system in a country in the region. In particular, participants came to appreciate better how the Bank of Uganda was interacting with survey respondents to build confidence and generate higher response rates. In the final evaluation, participants stated that they felt fully confident to introduce and run the system in their own country. The only remaining constraint was the availability of resources for running the survey.

22. A few study tours also were organized to permit country officials to see, first hand, how reforms in specific areas had been conducted in more advanced countries in the region. A number of regional, topically-oriented workshops were held; usually in cooperation and cost-sharing with regional agencies, including East AFRITAC.

23. As recommended by the second review of the project, further enhancement in cooperation was undertaken to work more effectively with other capacity-building efforts at the international and regional level. To this end, the regional advisor became more involved with efforts to promote the GDDS/National Strategy for the Development of Statistics (NSDS) by PARIS21, and he and some of the experts and Fund staff extended cooperation with East AFRITAC, the African Capacity-Building Foundation (ACBF), the MEFMI, the West African Institute for Financial and Economic Management in conducting regional workshops to the participating countries. The total number of such joint seminars came to 15 during 2003–06.

24. The experience with regional assistance, which brought together participants from a number of countries with common interests and a joint work program, was that these encounters generated stimulating discussions and more sharing of information. In several instances, countries found solutions to pending problems that were in tune with regional requirements and administrative capacities, but not necessarily known to the project experts. In some cases, the collaboration fostered south-to-south technical assistance that extended the reach of the project and further cemented the reform process.

25. Cooperation between the Fund and the World Bank also increased significantly during the project. Indeed, the two separate components of the project were merged in May 2004 in recognition of the trend but also paving the way for more fully integrated project activities. In this context, the project managers had to enhance coordination in planning and providing for the increased role of the regional advisor and also in preparing the joint six-monthly reports of the merged project. One major positive aspect of the project was the flexibility that allowed for TA missions to be better coordinated and for gaps to be filled as they became apparent.

III. The AAf Project Interim Reviews

26. An important feature of the AAf project has been the reviews, which were incorporated in the project documents to ensure adherence to project objectives while providing a basis for flexibility to adapt the project to changing circumstances. Thus, regular reviews have not only been an integral part of the project, but they have also constituted part of the multilayered approach to the monitoring and evaluation mechanism. Moreover, STA took advantage of the first two joint reviews to invite the Fund’s Office of Technical Assistance Management (OTM) staff to participate in the review missions to bring a broader perspective to the evaluation.11 Feedback from the various evaluation channels has been important to the adaptation of the project. Indeed, the impact of the first two joint reviews was significant in terms of changes introduced in the project and the eventual duration of the project itself. The third joint review, which was conducted by external experts with assistance from the executing agencies and DFID, started in early 2007. It provides critical insights in the ex post evaluation of the project, but its impact is only being felt during Phase II (2006–09) of the project.

A. First and Second Joint Reviews

27. Several issues of note were identified in the first review, including: (a) the need to raise awareness of government senior officials about a country’s statistical needs; (b) problems with project management and producing metadata to ensure country ownership; and (c) the need for countries to prioritize their TA requirements. The main recommendations of the first review were: (a) enhanced use of awareness workshops (regional and national) to promote the GDDS, with a more visible role of the regional advisor in this exercise; and (b) an extension of the project by six months.

28. Issues identified in the second review included: (a) the need for a sustainable system for maintaining metadata and for TA teams to routinely review these metadata; (b) efforts to encourage governments to take greater ownership of the terms of reference (TOR) and timing of TA missions; and (c) steps to better define the links between the DFID project and other statistical capacity-building efforts. The latter include efforts to monitor the MDGs, and initiatives by PARIS21, and several other agencies that had recently emerged in the region, notably, East AFRITAC, and the ACBF. The review recommended that the AAf project be extended for a further two years which, in total, carried it to April 2006 (with some spillover in funds and missions to about the end of 2006, as noted earlier).

B. Third Joint Review

29. In contrast to the first two reviews, the third review entailed a much more active role by DFID, which defined the broader scope of the review and engaged external experts, as part of its mission to headquarters, to interview Fund and Bank staff and in the preparation of the report. The scope included evaluation of the administrative arrangements and roles of the regional advisor and the headquarters’ based two project managers. It also included a joint mission to Ghana, Kenya, and Sierra Leone and preparation of country reports, as well as a main report. Advantage was taken of the country visits, for instance, to undertake more in-depth review and discussions with the authorities, as well as to conduct TA assessment using the PERs generated from the TA Information Management System (TAIMS, see below).

30. The main findings of the third review include comments on overall administration, where the report (see DFID, 2008) indicated that:

“In general, the financial management of the AAf project appears good overall. The IMF operated an efficient and comprehensive TA pool of international experts for GDDS Phase I. The GDDS regional advisor guided countries through the selection and use of consultants. Over time, countries were able to have greater input into TA TORs, which was welcomed—this improvement was suggested by the then DFID statistical advisor charged with liaising with the AAf project. Key informants for this evaluation indicated that during this period (around 2003–06), DFID was closely engaged in the AAf project operation and review, making a number of recommendations that were taken on board by the project and which resulted in countries using GDDS TA and workshops to better effect.”

A summary of the other findings and recommendations of the third review is provided in Box 7.

The country reviews

31. The evaluation of TA provided to Ghana, Kenya, and Sierra Leone drew on STA’s internal review process, especially the application of TAIMS and, in particular, the PERs (for details, see Box 8). The PERs also detail actual outputs of the project (usually comprising several TA missions) in comparison with the initial plans. Country authorities are asked to assess, on a numerical scale, achievements and sustainability of the project’s outputs. STA comments are usually added to the PER on receipt from the country.

Evaluations and the Technical Assistance Information Management System

Coinciding with the period of the AAf project, Fund evaluation of TA has been strengthened with the development of a comprehensive Technical Assistance Information Management System (TAIMS). The TAIMS was adopted formally by STA and other technical assistance departments in May 2005 to improve TA management and enhance monitoring and evaluation efforts and the sharing of information. TAIMS is a project management tool that covers all the main elements of the TA project cycle (scope of objectives and outputs, delivery, monitoring, and evaluation).

There are four reports in the TAIMS framework: a Project Framework Summary (PFS), Statement of Mission Tasks (SMT), End-of-Mission Assessment Report (EMAR), and Project Evaluation Report (PER). While the SMT and EMAR are prepared for each TA mission, the PFS and PER are only prepared for a country at the beginning and end of a project (i.e., a series of missions designed to achieve a set of related objectives over time). The PFS sets out the objectives and outputs to be achieved by the project, along with verifiable indicators, expected completion dates, and risks/assumptions. The PFS is adjusted and updated during the course of the project depending on the results of the SMTs and EMARs for individual missions. After the project has been completed, the PER is prepared with inputs from the staff/expert who delivered the TA, the backstopping staff in STA, and the recipient country officials, to provide a balanced feedback on the project. PERs are generally not prepared until a year after the completion of a project, in order to make a more informed judgment about the sustainability of results and final achievement of project objectives and outputs. The PFS and PER are particularly well suited to externally financed TA which is typically designed and implemented as a project, with beginning and ending dates and well defined objectives, outputs, and inputs.

32. The three countries received a total of 29 TA missions in specific macroeconomic topical areas, namely, the external sector, government finance, monetary and financial statistics, and the real sector (national accounts and prices). These missions constituted a total of eight projects comprising a range of inputs, ranging from a single mission up to a maximum of five missions to the relevant country. A total of five PERs have been prepared, as summarized in Table 3, to provide a basis for TA evaluation. However, it is noteworthy that in all cases where national data compiler’s surveys were undertaken, all such responses indicate that the areas with the least progress were coordination with donors and data dissemination, including to the Fund.

Table 3.

Summary of Project Evaluation Reports from Select Missions to Ghana, Kenya, and Sierra Leone, 2002–06

article image
Source: IMF Statistics Department.

Ratings: 4—fully achieved; 3—largely achieved; 2—partially achieved; 1—not achieved; NR—not rated.

Country data compilersȁ survey responses: OR—overall rating of the technical assistance; SA—strongly agree; A—agree; N—neutral.

33. The available PERs to date provide a very good basis (both quantitative and qualitative) for an overview of the outcome of TA projects with respect to the overall objectives, outputs (including priority and ratings), and, where available, the compilers’ views. The information sources provide a 360° assessment, as they contain inputs from the authorities (both at the senior officials’ and at the compilers’ level), staff, and experts. For each individual project, the information highlights, in summary form, areas of success or weaknesses which are critical to guiding more detailed assessment. The information can also be useful as a basis to compare different projects within TA in statistics or, for that matter, in the rest of Fund operations. Overall, it appears that the five projects were largely successful in meeting their objectives with generally favorable ratings for the outputs.

34. Although not randomly selected, the visits to Ghana, Kenya, and Sierra Leone nevertheless provide a broad and relevant experience from participating project countries. Kenya, an original participant, benefited from the posting of the regional advisor in Nairobi and substantial donor support.12 Such support and a strong local leadership role were instrumental in the broad (both macroeconomic and socio-demographic sectors) adoption and use of metadata, and especially plans for improvement in developing an NSDS (called the Strategic Implementation Master Plan in Kenya), of which the GDDS was an integral part.

35. Sierra Leone also was an original project participant, but suffered from a prolonged civil war and faced daunting resource problems, including sizable staff turnover. Nonetheless, the country managed to develop and utilize metadata effectively in some of the statistical agencies and has made good, but not yet complete, progress in preparing an NSDS. Ghana was a latecomer (in 2004) to the project and missed out on the regional GDDS awareness workshops, especially those in Botswana and Namibia. This outcome and interrupted leadership in a key agency were important factors in limiting the effective development and use of metadata, including plans for improvement to just one institution.

36. Data dissemination emerged as an important issue in all three countries. In Ghana, the adoption by the Ghana Statistics Service (GSS) of monthly press conferences to release the rebased consumer price index and the newly established producer price index was considered best practice.13 The practice not only raised the profile of the GSS but it also enhanced statistics in general and was welcomed by data users. In Kenya and Sierra Leone several statistical agencies/units expressed a lot of frustration with their inability to publish or post data, developed with TA from the project, on websites. Such data were thus not readily available to policy makers or the public, who were largely not even aware of their existence.

37. A common problem in all three countries was the still-inadequate resources—both skilled staff and budgetary funding—to support medium-to long-term reforms. TA delivery and absorption was quite effective as noted above; however, there were cases where some experts fell short of providing the planned TA. In this context, STA faced additional challenges and has responded by working with the authorities in addressing the underlying problems (Box 9).

STA’s Response in Addressing TA Problems in the AAf Project, Phase I

The third internal review mission of the AAf project in June 2007 reported a number of problems encountered by the authorities regarding some TA missions to the countries. Specifically, a TA mission to Kenya had not coordinated effectively with a previous mission to provide continuity in its work and advice to the country, while two unrelated missions to Ghana had each encountered problems with completing their tasks.

After completing an internal review, STA responded by (a) terminating the services of the three external experts involved; (b) hiring an expert to complete the TA report of the two relevant missions with joint recommendations to the Kenyan authorities; and (c) sending a follow-up mission to Ghana. That mission discussed and agreed with the authorities’ measures to address a number of the underlying problems to enable further TA missions to resolve the issues found by the June 2007 review mission. More specifically, the discussions focused on short-to-long-term actions to significantly enhance funding and coordination among the national statistical agencies. Such actions were aimed at improving access to source data essential for further progress in compiling and disseminating macroeconomic data. The mission met also with donors, who have since taken expedited action to enhance coordination among themselves and with local agencies, among other things, to strengthen support for national statistical capacity building.

C. Lessons from the Reviews

38. Broadly seen, the three project reviews suggest an evolution of the project from focusing primarily on promoting the GDDS and statistics in general, through the development of metadata, followed by TA and capacity-building support. Eventually, the project came to encompass even broader objectives of promoting regional cooperation, while providing an effective medium for collaboration between the domestic and regional statistical agencies. The project has adapted to address earlier problems and, in this regard, both the second and third reviews point to positive outcomes. A central theme in the recommendations by all three reviews was “how best to provide national ownership of the capacity-building efforts.” In this context, it appears that traditional TA—short-term missions ending typically with report writing—needed to be and was complemented with other methods (e.g., enhanced roles of the GDDS coordinators and the regional advisor to promote statistical awareness and national and regional coordination) to produce more effective results. Moreover, the GDDS provided a comprehensive strategic framework, which facilitated the prioritizing of reforms and TA needs. In two of the three countries, the GDDS provided also an important foundation for the preparation of the NSDS, reinforcing the close links between the two.

IV. Integration of the Aaf Project Into Sta Operations

39. As illustrated in Table 4, DFID, and a group of donors for the Africa Regional Technical Assistance Centers (AFRITACs),14 have provided external financing support to an increasingly important part of STA’s TA program in Africa. Compared with nothing in 2000 and 2001 (i.e., prior to this external funding), the AAf project-funded TA and AFRITAC-funded missions accounted for almost one third of total STA missions to Africa by 2003 and roughly three quarters of such missions by 2008. Virtually all of the increase in STA’s TA missions to Africa since 2002 (some 170 additional missions) was supported by the AAf project (67) and the AFRITACs (87), demonstrating that both the project and the AFRITACs’ funding have resulted in substantial “additionality” in TA resources. Moreover, the missions funded from these additional resources were fully integrated in STA day to day operations.

Table 4.

STA Technical Assistance Missions in Recent Years: Additionality Enabled by DFID and AFRITAC External Funding

(Calendar-year basis)

article image
Source: IMF Statistics Department.

Travel restrictions resulted in a low number of missions in 2002 as a consequence of the 9/11 attacks.

40. An important result of integrating the AAf project operations into STA’s TA activities was extensive documentation, including various mission briefs, back-to-office reports, TA reports, review reports, and other information. Moreover, STA was at the same time developing a fully inclusive evaluation tool of all its ongoing TA activities as discussed above. A positive outcome of this arrangement was the ready access to source information that has provided a basis for the review of this project and the evaluation of the TA provided to the participating countries.

41. There is, thus, little question that DFID external funding enabled the Fund to extend its reach and technical assistance beyond what its own resources would have supported. This additionality has provided for higher frequency and more extensive TA provided by STA to Africa. For example, rather than sending, at most, one mission per year to a country to provide an assessment and recommendations in a single sector, STA has been able to expand into other sectors, and also to follow up in a timely fashion to monitor and support implementation of prior recommendations, and provide hands-on assistance. The AAf project has also enabled other complementary activities to enrich and solidify the assistance, such as study tours to other countries in the region, regional workshops to facilitate knowledge sharing, specialized training courses, and statistical awareness programs to improve domestic coordination of activities which are often critical to the achievement of objectives. The fact that, by the end of the project, all countries but one were participating in the GDDS and working on their plans for improvement is a good indication of the extent of integration of the project into STA’s work program.

42. The AAf project was coordinated with the area departments, mostly in AFR and, for one country, in MCD. These departments were engaged in the preparatory work for, and the debriefing following, project-financed TA missions. In the briefing process, the area departments have often contributed their views on the scope and desirable focus of expert visits. In general, this cooperation has proven constructive and helpful for the experts, and has also frequently resulted in the TA better supporting AFR’s own programs in affected countries. In some cases, expert visits have been scheduled to overlap with area department missions when so requested; in other cases, overlaps have been avoided in order not to overstretch local counterpart resources.

43. The unique project structure, with its three components, has supported synergies and efficiency gains: The design of the Fund, WB, and joint component (regional advisor/regional workshops, national awareness workshops) has resulted in efficiency gains for the donor and a lowering of “transaction costs” for recipient countries having to deal with one regional advisor for Fund and WB TA, attending only one regional workshop to deal with two organizations and the methods and approach for both streams of TA were the same. Also, joint reviews were an advantage to participating countries, which had to receive only one mission for this purpose. This flexible, parallel execution of the various components of the project, using the GDDS as an organizing framework, was facilitated by the separation of the operating budgets for the Fund and the Bank. Under these arrangements, each of the executing agencies could adhere to its own administrative and accounting practices in fostering the common goals of the project.

V. Phase II of the Aaf Project

44. While Phase I of the project ended in 2006, it contributed substantially to shaping the successor project, and an assessment of the linkages between the two projects provides additional insights into the impact of the first project. In early 2006, following a regional meeting with countries participating in Phase I and discussions with the Fund and the World Bank, DFID agreed to finance a second phase. Phase II of the project extends over three years, 2006–2009, and with a somewhat different form, reflecting the experience and lessons learned during Phase I. Notably, there is more emphasis on multi-country collaboration and assistance, in workshops and meetings built around substantive “modules,” rather than primary emphasis on single-country TA. The thought behind this format revision is to encourage more common experience and collaboration among data compilers in regional countries, some of which are more advanced than others. This can have technical advantages in common understanding of statistical sources and procedures, and also provides mutual support among the community of compilers. In addition to the Phase I countries, several others have been added for this second phase, namely, The Gambia, Mauritius, Mozambique, Seychelles, South Africa, Tanzania, and Uganda.

Incorporating Phase I Lessons into Phase II of the Project

45. The lessons drawn from country coverage under Phase I indicate the advantages of a small group of countries with a common language, which facilitated administrative arrangements and TA delivery. The original choice of participants had nevertheless excluded certain other Anglophone African countries, which were already participating in the GDDS at the inception of the project. By implication, the Phase I project faced a more challenging task of assisting countries that required greater attention and TA in enhancing capacity building. The decision to expand coverage to 22 countries under Phase II reflected an understanding that the project could be expanded within existing arrangements to benefit more countries. A second lesson was that adding more countries could help expand the objectives of the project, particularly in pushing for greater possibility to graduate to the SDDS.

46. Based on the experience in Phase I, administrative arrangements under the successor project were left intact with regard to the project managers/advisor,15 but were significantly modified in respect of TA delivery. DFID consultations with STA revealed that the latter had faced unexpected and sizable pressure in backstopping experts under the project.16 The reasons for this development reflect the rapid growth in demand for TA, and therefore the required number of experts to be supervised. The rapid growth in TA also created a challenge of effectively matching experts and country authorities to ensure successful outcomes. The problem was exacerbated by the limited opportunity for staff visits to the affected countries/region, since the project did not pay for staff travel. To address these problems, Phase II introduced several innovations, which were endorsed by OTM and DFID: to (a) allow staff travel to be covered on a limited basis by the project in order to enhance field supervision and evaluation of TA, and (b) allow STA to contract for headquarters-based module managers (see below) under the project to ease some of the administrative burden (e.g., contract preparations) and backstopping responsibilities of STA divisions.

47. Among the major lessons about TA delivery drawn from Phase I of the project were that: (a) the regional approach was useful in promoting the sharing of experience and creating beneficial peer pressure; (b) the effectiveness of TA is enhanced if it is more focused in content and better targeted to small groups of countries with similar levels of capacity building needs; (c) getting countries to participate in setting specific TA objectives and in assessing the outcomes contributes to greater ownership and accountability; and (d) measures to alleviate administrative and backstopping burdens on STA divisions (e.g., through use of module managers) enhance ownership and accountability by TA providers. The resulting adaptation entailed the setting up of a limited number of modules—eight for the Fund and seven for the Bank (on socio-demographic topics)—run by module managers as the main approach to TA delivery. Moreover, by design, TA modules are launched and concluded with workshops for officials from countries participating in the relevant module to agree on and evaluate TA objectives, respectively.

48. The five topical TA modules in Phase II cover the traditional areas of STA activity, namely, monetary and financial (two parallel modules), balance of payments, government finance, and national accounts statistics. However, the modules focus on more narrowly defined issues within each of these areas. This tight focus was necessary to permit the setting of, and commitment to, realistic outcomes within the relatively short timeframe of Phase II. This process of narrowing down topical areas to more specific issues was carried out in a two-step consultative process with the participating countries, AFR, and MCD and resulted in broad agreement and an equitable distribution of the modules among the participating countries.

49. The functional TA modules were designed with broader objectives that link more directly with the evolving work program of the Fund, other international organizations, and donors. Thus, in line with objectives of members of the PARIS21 group, for some countries there is more emphasis on developing and implementing plans for systematic statistical improvement, and also stronger expectations about actual dissemination of the resulting economic and financial data. There are also greater efforts to promote more regional TA to deal with systemic issues on a multi-country basis.

50. The GDDS/Poverty Reduction Strategy Paper (PRSP) module of Phase II is contributing to efforts in mainstreaming statistics in the PRSP to enhance priority and funding for broader statistical reforms. This is important to provide for medium-to-long-term financial and other resources to generate the requisite source data for the compilation of macroeconomic statistics. It is also helping countries to create national summary data pages (NSDPs). Efforts to improve dissemination, including through improved and better coordinated national websites, will provide greater focus on outputs, i.e., data, better highlight the impact of TA inputs, while helping to institutionalize data processes and ensure sustainability.

51. The Southern African Customs Union (SACU) module is focusing on addressing the systemic issue of compiling customs data among the SACU members (Botswana, Lesotho, Namibia, South Africa, and Swaziland) on a regional basis, by working with the SACU Secretariat as well as member countries. The expectation is that the approach will lead to better sustainability with enforcement through the common organization and peer pressure to emulate best practice. Already this module has produced full documentation of current compilation practices by each country in SACU and a report recommending harmonization according to international best practices. Finally, the SDDS module aims to propel project countries to graduate to a self-sustaining standard that will ensure that countries produce and disseminate high-quality data.

52. In some important ways, the AAf project has provided STA with the experience to take an early lead in anticipating and implementing some of the subsequent Fund wide recommendations in the provision of TA. For instance, the 2005 review of TA programs by the Fund’s Internal Evaluation Office (IEO, see IMF, 2005) stressed (1) enhancing the strategic prioritization and resource allocation to increase the relevance of the TA program; (2) promoting the effectiveness of various modalities of TA delivery, including the country specificity of the diagnosis and proposed solutions, as well as domestic ownership of recommendations; (3) enhancing monitoring and evaluation activities so that lessons from past TA can be a guide to future allocation and design of TA; and (4) improving coordination with other TA providers. The Malan Report (IMF, 2007) advocated better coordination between the Fund, Bank and other providers in the delivery of capacity building and other services to low-income countries. The project has provided useful lessons in all these areas, which are relevant to other regions as well and where feasible have been shared with other TA providers.

VI. Longer-Term TA Issues

53. The evaluation of TA to low income countries must ultimately address longer-term strategic issues, including sustainability of reforms. In this regard, STA has enhanced medium-term TA evaluation via program evaluation for a country or a project for several countries, as in this report. Such an evaluation is conducted after three to five years, which covers the constituent projects executed during that period. STA is currently conducting such TA program evaluation in two selected area department regions every year on a staggered basis. The emerging value of this comprehensive review approach provides the means to introduce a more strategic approach to improvements in the TA program to promote greater effectiveness and has allowed STA to integrate its work into the broader Fund and international community evolving agenda in assisting developing countries.

54. Close collaboration with donors, particularly, DFID and the World Bank, and the various pilot modules in Phase II, have formed a good basis to contribute to various internationally agreed strategies for low-income countries. These include broader application of the GDDS framework to enhance national priority and funding in the poverty reduction strategy (PRS) approach, support for MAPS, and the Paris Declaration for Aid Effectiveness. (for details see Kibuka, 2007b, and World Bank and PARIS21, 2007).

55. The aim of most TA is for the reforms and advances achieved to become internalized and incorporated in routine ways of thinking and operational procedures. However, the sustainability of progress achieved through TA is never guaranteed. In the case of statistical capacity in the AAf countries benefiting from a large and sustained DFID-financed AAf project, this question cannot be answered definitively at this stage. Indeed, sustainability issues (ownership, effective preparation and use of metadata, plans for improvement, TA prioritization, and effective absorption and retention) have been central to all the reviews of the project. Given the long gestation period entailed in capacity building, the project experience so far is still relatively limited and, despite the infusion of resources, the amounts expended per country thus far are relatively small. This is especially the case in comparison with the potential benefits to a set of countries where it is acknowledged that economic data of various kinds are weak, and where it may be presumed that good policymaking is impeded by the lack of sound statistics on which to make judgments.

56. An emerging central issue in the sustainability of statistical capacity building is related to adequate awareness and, hence, priority to ensure availability of medium-term funding to maintain the reform process. Resources are especially critical to the chain system that determine how much a series of short-term TA missions can contribute to sustained improvements say in the capacity to produce and disseminate data. In the first instance, adequate skilled staff is essential to the transfer of knowledge from TA missions and its retention at workable levels requires maintenance of adequate staff strength, regular training for the upkeep and occasional update of skills. Retention of staff also requires competitive remuneration. Second, regular surveys/censuses and maintenance of administrative data systems are essential to provide source data (including appropriate weights) a prerequisite for compiling macroeconomic statistics. Equipment, whether for transport or IT systems (with appropriate office space and power supply) are not only essential but they also need regular maintenance and upgrades.

57. All the components of the statistical system require adequate resources from the national budget and donors to ensure its smooth functioning. Inadequate resources will contribute sooner or later to breakdowns that will compromise the data, increase time lags, and eventually give way to ad hoc substitute data. Under these circumstances, TA will have little to show for in terms of sustainable final outputs and would end up mostly as stop gap measures to support ad hoc procedures.17 For the Phase I project participating countries, early emphasis was put on advocacy via enhanced promotion of awareness regional and national workshops, with a more visible role of the project regional advisor. There is evidence of success in these areas, but there are also indications that beneficiary countries are not yet able to provide the funding needed to maintain the momentum of reforms.

58. There are also certain aspects of the AAf project in which participating countries seem to be slipping from adherence to some of the GDDS norms, such as regular updating and submission of metadata about their compilations of these statistics (See Appendix Table 5). Such slippage is a shortfall from the objectives of the project and, while not limited to the project countries, it should be monitored and redressed.

Appendix Table 5.

AAf Project Countries: Subscription Dates, ROSCs, and Metadata Updates

article image
Source: IMF Statistics Department.

59. There has been some evidence that the dissemination of data and their use as a basis to inform policy decisions (the final output of TA and capacity building) have not been explicitly emphasized. Phase I of the AAf project was process oriented, more so in the early stages, reflecting the requisite investment for introducing new initiatives (the project and the GDDS) but also because the GDDS itself was not fully focused on data outputs.18 As a result, evidence of availability of improved quality of data has been a haphazard process reflecting very much the preferences of national authorities as opposed to a key feature of the GDDS framework. As the AAf project has gradually adapted to more specific TA inputs, efforts have been intensified, especially in phase II to pilot work that directly promotes data dissemination. The GDDS/PRSP and SDDS modules of the project are providing TA to countries to develop a NSDP while strengthening the coordination of national agencies’ websites and the role of the GDDS coordinator in this regard. These changes facilitate a better link between TA inputs and the availability of data in the AAf project countries. Such developments should not only strengthen the GDDS framework, but would also bring it more in line with the SDDS and, thus, facilitate graduation to a higher data standard with built-in sustainability features (see IMF, 2008).

60. Another area of evident weakness is follow-through on indicated short- and medium-term objectives for improving national statistics, even though such plans are outlined on the DSBB with each country’s metadata. There has been, so far, no effective or systematic method for holding countries to their “commitments,” in terms of steady forward progress in the data compilation and dissemination area, and it appears that many have not turned these proposals into fact. This flaw, of course, is not confined to African GDDS countries. Phase II of the AAf project, however, is tightening up on the accountability for promises made, so there is some prospect of better performance over the 2006–2009 interval.

61. Certain other “indicators of progress” are available for the AAf project countries that should be mentioned as part of any evaluation of this effort. One is that, over the years of the project, some of the participant countries have attained sovereign bond ratings and thus, in concept, have gained access to international capital markets. This is illustrated in Appendix Table 6. At least one recent study has found evidence that participation in the GDDS has lowered borrowing costs for these countries, albeit by only a modest amount.19 To be sure, GDDS participation alone may not be the only factor in play here, as GDDS participation itself may be reflective of other conditions and policies that make some countries better candidates for international credit markets than others.

Appendix Table 6.

GDDS Countries with Sovereign Bond Ratings, by Rating Agency

(as of April 2007)*

article image
Source: IMF, 2008.Note: Counties in DFID Phase I Project in bold face.

VII. Conclusions

62. This review of the Phase I of the AAf project points to a broadly successful outcome for the recipient countries and for the Fund. All but one of the participating countries joined the GDDS and have achieved significant outputs in documenting their statistical systems in terms of metadata and preparing detailed plans for improvement. Many have made good progress during the project in following through with these plans for improvement and improving their statistical systems. However, reforms to date have been hampered by several factors, reflecting mainly the long-term nature of capacity building and the still-inadequate national priority and funding for statistical reforms.

63. For the Fund, DFID financial support has began to fill an important gap in the African regional efforts and overall budgetary situation, in promoting the GDDS framework through advocacy and capacity building, mostly through the provision of additional TA. Evaluations and reviews of the project have not only yielded encouraging evidence of improvements in TA effectiveness but have helped to reinforce the Fund’s internal TA procedures. The lessons learned and the experience with working closely with donors and regional institutions have also helped to strengthen the broader outreach essential to capacity building and have contributed importantly to the adaptation of STA’s TA program. STA has also benefited from the project by (1) initiating and expanding work on strategic issues in capacity building in statistics to support Fund-wide approaches to the PRSP and TA, and (2) promoting aspects of the GDDS that support international initiatives, such as MAPS, PARIS21, the Paris Declaration on Aid Effectiveness. development.

64. The unique structure of the AAf project, using the GDDS as an organizing framework, as well as the three components funded through separate operating budgets has permitted parallel execution, while allowing both the Fund and World Bank to follow their own administrative and accounting practices in fostering the common goals of the project.

65. Such flexible project arrangements made it easier for STA to incorporate the relevant elements of the TA project into its ongoing work program, which in turn was supported by the Fund’s DSI activities in Africa. At the operational level, STA integrated all aspects of capacity building, particularly the provision of TA into its operating procedures. This in turn, helped to ensure that TA and its providers—mostly outside experts—were backstopped by Fund staff for quality control purposes, while at the same time providing a basis for subsequent evaluation to assess the effectiveness of TA delivery. A limited sample of pilot PERs in three of the 15 countries demonstrate the usefulness of this framework in providing qualitative and quantitative information as a basis for a more comprehensive assessment of this effectiveness.

66. DFID’s funding for the GDDS project has provided additional resources to enable STA to widen its reach in providing TA and GDDS support to the 15 African countries beyond what its own resources would have supported. At the same time, the parallel execution of the project has resulted in efficiency gains for DFID and a lowering of “transactions costs” for recipient countries, by having one common regional advisor, attending joint workshops to deal with two organizations and same methods and approach defined within the same GDDS framework.

67. The experience of the Phase I project, informed by the joint reviews, has contributed substantively to expanding the breadth, depth, and effectiveness of STA’s TA program in Africa. Delivery of traditional TA has been complemented by advocacy (to promote greater statistical awareness) and regional collaboration, including with other development partners and regional organizations. The approach appears to have been effective, both in managing the project (thanks to the project managers and regional advisor), and also in terms of delivering “broader” TA, and in engendering regional awareness and peer pressure. Along the way, certain TA delivery issues emerged, providing additional challenges that have been addressed in a timely fashion.

68. STA has introduced several initiatives, most notably TA delivery in a modular form to address the needs of a group of countries while permitting improved focus of content and better targeting to countries with similar levels of capacity-building needs. The modular approach has not only helped to relieve the backstopping burden on STA staff, but is also a step toward increased ownership and accountability by TA recipients and providers. There has also been increased awareness of the constraints imposed by inadequate resources, which translate into severe deficiencies in source data critical to the compilation of macroeconomic statistics. In response, a pilot model has been designed to promote stronger linkages of the GDDS to the PRSP and the medium-term expenditure framework. Taking account of the reviews, DFID has agreed to finance Phase II of this initiative, and most of these recommendations (e.g., modular design and multi-country activities) have been incorporated in the design of the second phase.

69. The long-term issues emerging from the provision of TA, including those pertaining to sustainability, demonstrate the daunting nature of capacity building in developing countries. The project has had limited objectives and is aimed at a subset of the GDDS country population, and achieving even its limited objectives has been resource-intensive. There is, to be sure, still a long way to go to reach the ultimate objectives of disseminating high quality statistics to inform policy and market decisions. Phase II of the DFID project, with increased resources and country coverage, promises to extend the progress achieved under Phase I, and help consolidate this progress in participating countries.

Appendix I. Strengths and Weaknesses of the AAf Project, Phase I

Summary of Points Made by Country Delegations to the GDDS Workshop in Cape Town, South Africa (September 2006)

Part 1: Summary of interviews

1. During the launch workshop for GDDS Phase II, interviews were conducted with a random sample of the six countries in order to get their feedback on Phase I. In general, countries found that the GDDS project had helped agencies and individual compilers better understand and identify gaps in data series and methodologies. Most agencies indicated that data series had improved, and that they now have shorter time lags and better coverage. Some also reported a greater response rate to questionnaires.

2. The ability to maintain current data quality is improving but is still fragile in some areas. Asked whether agencies would be able to maintain the current quality of data, country representatives answered yes in several cases, but they also stressed their intention to improve further (“not there yet”). The ability to maintain data at the current standard was perceived by several countries as still fragile due to such factors as staff’s lack of training or staff turnover, software problems, data not yet ready for publication, or gaps to close.

3. Countries are able to keep metadata up-to-date. Countries that have participated most actively in the GDDS project (such as Botswana, Ghana, Swaziland) are able to update or confirm their metadata on a regular basis—once a year. Some countries also observed that the awareness seminars helped in creating a demand for quality data, and in some cases helped better understand roles and responsibilities among data-generating agencies.

4. The technical assistance received through expert missions was generally of high quality. Particularly useful were those missions that included general briefing or training for a larger audience or a good wrap-up meeting. In a few cases, a longer duration would have been helpful. The visits of the regional advisor and the World Bank team leader had helped create better understanding at the managerial levels and in coordinating ministries for improving data using GDDS as an organizing framework. The sub-regional seminars involving four-six countries on a specific topic were rated as very effective for training and mutual learning.

Part 2: Summary of discussions

Overview

  • TA missions received resulted in substantial progress in various areas of statistics: (improving timeliness, data compilation systems, additional data sources were identified, and improved methods of dissemination).

  • Countries set up GDDS technical committees but some had operational problems.

Cooperation among agencies within the region

  • Collaboration with other data-producing agencies was enhanced.

  • Regional cooperation in the form of workshops, such as:

    • Seminars on BOP (Private Capital Flows) were held in Lesotho and Swaziland.

    • Survey data documentation and management workshop was held in Namibia.

  • GDDS supported the south-south TA in trade statistics (two staff members from CSO Zambia undertook a TA Mission to CSO Botswana.

  • Cooperation with AFRITAC and STATCAP initiatives and integration with other donors.

  • Cooperation between Tanzania and Uganda to compile the East African database.

  • Encouraged cooperation between SADC member countries.

Feedback as to the effectiveness of TA

  • The feedbacks received from recipients of the technical assistances indicate that the TA missions were useful.

  • The missions helped to improve data sourcing and methodology.

  • The GDDS project played a greater role in having statistics as a component in the PRSPs.

  • The TA missions identified data gaps and areas for improvement.

Remaining needs

  • Countries stressed a need for continued assistance in various sectors.

  • Training and capacity-building needs were identified, i.e., more capacity building needed in training staff.

  • Develop in-house capacity within agencies involved.

  • TA needs to be delivered in a working language; the new approach should take this into consideration.

Problems highlighted

  • Non-response (cooperation) from other participating agencies.

  • Communication needs to be improved.

  • Operation of the technical committees could be strengthened.

References

  • Cady, John, and Anthony Pellechio, “2006, Sovereign Borrowing Cost and the IMF’s Data Standards Initiatives”; IMF Working Paper 06/78, (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund, 2002a, Botswana: Report on the Observance of Standards and Codes-Data Module, Response by the Authorities, and Detailed Assessments Using the Data Quality Assessment Framework, IMF Country Report No. 02/83 (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund, 2002b, Namibia: Report on the Observance of Standards and Codes-Data Module, Response by the Authorities, and Detailed Assessments Using the Data Quality Assessment Framework, IMF Country Report No. 02/202. (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund, 2005a, Zambia: Report on the Observance of Standards and Codes-Data Module, Response by the Authorities, and Detailed Assessments Using the Data Quality Assessment Framework, IMF Country report No.05/30. (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund, 2005b, Malawi: Report on the Observance of Standards and Codes-Data Module, Response by the Authorities, and Detailed Assessments Using the Data Quality Assessment Framework, IMF Country Report No. 05/60. (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund, 2005c, Kenya: Report on the Observance of Standards and Codes-Data Module, Response by the Authorities, and Detailed Assessments Using the Data Quality Assessment Framework, IMF Country Report No. 05/388 (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund, 2007a, Committee to Study Sustainable Long-Term Financing of the IMF, 2007, Final Report. Available via internet:www.imf.org/external/np/oth/2007/013107.pdf.

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund, 2007b, The General Data Dissemination System: Guide for Participants and Users, (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund, Internal Evaluation Office, 2005, “IEO Evaluation of the Technical Assistance Provided by the Fund”, Available via internet: www.imf.org/external/np/ieo/2005/ta/eng/013105.htm.

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund, Office of Technical Assistance Management, 2007, Implementing the Medium Term Strategy for Technical Assistance (October, 2007).

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund, Statistics Department, 1998, The General Data Dissemination System, (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund, 2003 and 2005, Fifth and Sixth Reviews of the Fund’s Data Standards Initiatives, Available via internet: ww.imf.org/external/np/sta/dsbb/2003/eng/review.htm; and www.imf.org/external/np/pp/eng/2005/070105s.pdf.

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund, 2008a, Assessing the General Data Dissemination System (GDDS) —What Has Been Accomplished After Ten Years, and Where Do We Go from Here? Available via internet: http://www.imf.org/external/pubs/ft/sdds/gdds-assess-08.pdf.

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund, 2008b, Implementation of the Medium-Term Strategy for Technical Assistance, Forthcoming, (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund and the World Bank, 2007, Report of the External Review Committee on IMF-World Bank Collaboration. Available via internet: www.imf.org/external/np/pp/eng/2007/022307.pdf.

    • Search Google Scholar
    • Export Citation

  • Kester, Anne Y., 2006, “IMF Data Standards Initiatives: A Consultative Approach to Enhancing Global Data Transparency”, IMF Working Paper 06/102, (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation

  • Kibuka, Robin D., 2007a, “Some Lessons from Implementing the GDDS Project, Phase I (2001-2006) for Anglophone African Countries,” A Presentation made to the Bamako Meeting, Mali, October 31.

    • Search Google Scholar
    • Export Citation

  • Kibuka, Robin D.,, 2007b, “Mainstreaming Statistics in the Poverty Reduction Strategy Approach to Provide for More Effective Technical Assistance: Some Experience at the IMF”, IMF Working Paper 07/255, (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation

  • Morrison, Thomas K., ed., 2005, Statistical Capacity Building: Case Studies and Lessons Learned, (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation

  • U.K. Department for International Development, 2008, “Evaluation of DFID Support to Phase One of the GDDS Project (2002-06) and to the World Bank Trust Fund for Statistical Capacity Building: Strategic and Management Overview” Forthcoming.

    • Search Google Scholar
    • Export Citation

  • World Bank and PARIS21, 2007, “Scaling Up Investment in National Statistical Systems,” (Washington).

1

Botswana, Eritrea, Ethiopia, Ghana, Kenya, Lesotho, Liberia, Malawi, Namibia, Nigeria, Sierra Leone, Sudan, Swaziland, Zambia, and Zimbabwe.

2

The DSI was one of the Fund’s responses to the 1990s’ economic and financial crises in emerging economies. It was informed by the important lesson from this episode that poor and/or delayed economic and financial data have serious and costly consequences for the economic well-being of individual countries and for the international financial system as a whole. A part of the initiative focused on “building statistical capacity” in countries that have tended to lack established statistical traditions, in other words, providing timely and reliable compilations of economic/financial data, and dissemination of such data to users and policymakers.

3

As described in the Sixth Review of the DSI, for instance, SDDS is a “monitored standard focusing on dissemination of data used by financial markets,” whereas GDDS is “a framework to guide countries to develop sound statistical systems. (For details, see IMF, STA, 2005.)

4

Over the years, six GDDS countries have “graduated” to the SDDS level; these are included in the mentioned SDDS total. Only one AFR country, South Africa, which was not part of this project, subscribes to the SDDS.

5

All countries participating in the GDDS have to nominate a GDDS coordinator, whose role is to work closely with all the national statistics agencies in updating the metadata and plans for improvement. The latter provides a basis for prioritizing TA needs and in the formulation of rolling plans for reforms.

6

The project agreement included a formal separation of the DFID financing into three separate budgets: a component for World Bank TA, a component for Fund TA, and a component for joint World Bank-Fund and regional activities, which, by agreement, was executed by the Fund. The financial arrangements with DFID were different for the Fund and the Bank, following established mechanisms for the two institutions. These arrangements facilitated the largely parallel execution of the project by the two agencies.

7

The process of joining the GDDS usually generates a number of specific outputs, which also occurred in the project countries. Preparation of a detailed description of statistical practices in the covered areas, often resulted in some 20 or more metadata tables, forming a document of usually more than 100 pages. In many of these countries, this was the first comprehensive documentation of the statistical system and procedures.

8

References in these tables to missions and activities subsequent to April, 2006, reflect DFID’s agreement to carry some unspent funds into months following the nominal end of the commitment period.

9

It is true, of course, that this “logging of progress” was made by TA experts paid by the project with, perhaps, some inclination to look on the bright side, but in general, elements of progress seem to be genuine, and all are relevant to public information and government policymaking. Whether such progress is being internalized in national commitments is a more difficult question that cannot be answered at this stage, so sustainability will remain an open issue for some time.

10

This high-level workshop was attended by the Director of STA; the Governor of the Bank of Botswana gave the keynote speech.

11

The first joint review of the project was undertaken in November/December 2002 and the final report was issued in January 2003. The review comprised a joint mission to Ethiopia and Namibia, but it also benefited from survey responses from all but one of the participating AAf countries. The second review took place in December 2003 and comprised a mission to Kenya, Swaziland, and Namibia, and held discussions with the Namibian authorities.

12

For instance, the World Bank has provided a STATCAP (a World Bank financing mechanism with a simplified process for project preparation supporting the implementation of a country’s NSDS) which, together with DFID, represents a total commitment of $30 million to support statistical reforms.

13

Both the rebased CPI and the launching of the PPI (in early 2007) were supported by the AAf project TA.

14

During the period 2001–06, the Fund opened two regional AFRITACs—East AFRITAC in Tanzania in 2002 and West AFRITAC in Mali in 2003.

15

As confirmed by the third and final review, the managers of the project were very effective and their role was left unchanged, even as the decision was made to expand the project.

16

The situation was aggravated by additional demands on backstopping by staff for experts in other regional TA centers, which were expanded with the addition of East AFRITAC in 2002, West AFRITAC in 2003, and METAC in 2005.

17

This is one of the reasons why long-term resident advisors may end up being ineffective while costing substantially more resources to sustain. Another reason is that long-term experts tend to be used to fill gaps in the establishment to ensure day to day work flow rather than strengthening capacity. Imposing conditions for the follow-up visits by peripatetic experts may keep the pressure on for continued progress and contribute to more effective TA.

18

As noted earlier, the GDDS focused on developing, updating, and posting metadata on the DSBB, an important step in identifying the statistical gaps and prioritizing TA needs to address such gaps. As such, metadata is an important intermediate output. However, the ex post evaluation of the effectiveness of TA and reforms needs to be measured against the ultimate benchmark i.e., the posting of data on the national website with easy access by policy makers and the public.

19

See, for instance, Cady and Pellechio, 2006.

Same Series

Other IMF Content

Other Publishers

Cover Policy Papers
Evaluation of the U.K. DFID-Financed Technical Assistance GDDS Project for Selected Anglophone African Countries (2001-06)
Author: International Monetary Fund
Volume/Issue:
Volume 2008: Issue 019
Publisher:
International Monetary Fund
ISBN:
9781498334815
ISSN:
2663-3493
Pages:
44
DOI:
https://doi.org/10.5089/9781498334815.007