Guatemala: Technical Assistance Report-Statistics Technical Assistance and Training Evaluation Mission

This paper discusses the key findings of the report on statistics technical assistance (TA) and training evaluation mission on Guatemala. It is observed that overall users trust macroeconomic statistics in Guatemala. Data-producing agencies viewed TA and training provided by Statistics Department, IMF during 1998–2015 as most effective to improving the quality of statistics. There is broad recognition of the Bank of Guatemala’s key role in producing and disseminating statistics. There was consensus that TA and training were effective and helped enhance data quality of all datasets, but to a modest extent Government Finance Statistics and Producer Price Index. The effectiveness of TA largely depends on capacity built in the main data-producing agencies.

Abstract

This paper discusses the key findings of the report on statistics technical assistance (TA) and training evaluation mission on Guatemala. It is observed that overall users trust macroeconomic statistics in Guatemala. Data-producing agencies viewed TA and training provided by Statistics Department, IMF during 1998–2015 as most effective to improving the quality of statistics. There is broad recognition of the Bank of Guatemala’s key role in producing and disseminating statistics. There was consensus that TA and training were effective and helped enhance data quality of all datasets, but to a modest extent Government Finance Statistics and Producer Price Index. The effectiveness of TA largely depends on capacity built in the main data-producing agencies.

I. Introduction

1. A TA Evaluation mission from the IMF Statistics Department (STA), with the support of the IMF Western Hemisphere Department (WHD), visited Guatemala City, Guatemala during May 27-June 2, 2015. The main purposes of the mission were to evaluate the effectiveness of STA’s technical assistance (TA) and training in the compilation and dissemination of macroeconomic statistics provided to Guatemala since 1998, and to discuss with the authorities future TA priorities. Specific objectives of the mission were to (1) elicit views of the authorities and other stakeholders on the effectiveness of TA and training provided by STA on macroeconomic statistics during FY 1998-20152, and (2) discuss how TA and training can be improved, especially with regard to the achievement of results. The lessons learned from the mission may be useful to further enhance the effectiveness of STA’s capacity building activities.

2. The evaluation covered TA and training provided to Guatemala by STA since 1998 in five areas of macroeconomic statistics—national accounts (NA), prices, government finance statistics (GFS), monetary and financial statistics (MFS), and external sector statistics (ESS).

3. The methodology used by the Evaluation mission comprised (i) reviewing TA and training activities with the view of assessing main outcomes and challenges faced in the implementation of recommendations; (ii) eliciting views of the authorities on the effectiveness of TA and training provided by STA—including discussing the sustainability of the main outcomes; (iii) eliciting users’ views on the usefulness of macroeconomic statistics and, where relevant, on STA’s capacity building activities to improve them; and (iv) assessing whether funding difficulties of the Technical Assistance Regional Center for Central America, Panama, and the Dominican Republic (CAPTAC-DR), that may likely lead to a reduction in STA’s TA to Guatemala, may have an impact in the areas of need.

4. To this end, the mission worked closely with officials from the main data-producing agencies in Guatemala—the Bank of Guatemala (BG), the National Statistics Institute (NSI), and the Ministry of Public Finance (MPF). As part of the mission’s preparation and to facilitate the discussion with the authorities and main data users, the mission collected responses through questionnaires distributed with the support of the BG prior to its visit to Guatemala. The mission also met with main users of Guatemala’s statistics, both from the public and private sector. A list of government officials and users that met with the mission is presented in Annex I.

5. Section II of the report provides background to the evaluation process and its scope. Section III outlines the findings and results achieved with the support of TA and training provided by STA to Guatemala in the areas of macroeconomic statistics covered by the mission. Section IV presents the views of the authorities and main users on TA and training based on the questionnaires and interviews. Section V presents Guatemala’ macroeconomic statistics under a regional perspective. Section VI reviews Guatemala’s data reported to the Fund and other international organizations. Section VII synthesizes lessons learned from the evaluation while Section VIII focuses on key conclusions and recommendations. Section IX presents the outcome of the discussions held with the authorities on TA and training priorities for the future.

II. Background

A. Evaluation of TA and Training

6. The evaluation of STA’s technical assistance and training is founded on the Organisation of Economic Co-operation and Development (OECD) Development Assistance Committee (DAC) Principles for Evaluation of Development Assistance.3 These principles state that:

“An evaluation is an assessment, as systemic and objective as possible, of an on-going or complete project program or policy, its design, implementation and results. The aim is to determine the relevance and fulfillment of objectives, developmental efficiency, effectiveness, impact, and sustainability. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision-making process of both recipients and donors.” (Paragraph 5)

“An important purpose of evaluation is to bring to the attention of policymakers constraints on developmental aid success resulting from policy shortcomings or rigidities both on the donor and recipient side, inadequate coordination, and the effectiveness of other practices such as procurement.” (Paragraph 9)

“Evaluation reporting should be clear, as free as possible of technical language and include the following elements: an executive summary; a profile of the activity evaluated; a description of the evaluation methods used; the main findings; lessons learned; conclusions and recommendations…” (Paragraph 39)

B. Background to the Evaluation of TA and Training to Guatemala

7. The Evaluation mission to Guatemala followed the recommendations in the IMF’s Independent Evaluation Office (IEO) report in 2005 for developing more systematic approaches to track progress on major TA activities and to identify factors behind successes and shortfalls.4 STA conducts evaluation exercises of its TA and training, including evaluation missions, to a few countries that receive relatively high TA volume from STA, aiming at systematically drawing out lessons learned from the evaluation with a view to enhancing TA and training.

8. To the extent possible, the Evaluation mission took into consideration the evolving work in the IMF to develop a RBM framework.5 While most TA under review was not formally formulated using this framework, the evaluation placed emphasis on the results that have been achieved with the TA and training support provided by STA to Guatemala during 1998-2015. Annex II identifies TA missions conducted by STA for each of the five macroeconomic datasets during FY2008-FY2015. Annex III presents the number of Guatemalan officials that participated in training provided by the IMF by training center and topical area.

9. The methodology used by the mission to evaluate the effectiveness of STA’s TA and training on macroeconomic statistics in Guatemala is presented in Box 1.

Guatemala: Methodology Used in the TA and Training Evaluation Mission

  • Identification of TA and training activities conducted by STA since 1998.

  • Review TA reports and topics of training provided.

  • Assess the main outcomes of TA missions and challenges in implementing recommendations.

  • Elicit the authorities’ views on the effectiveness and sustainability of TA and training provided by STA.

  • Discuss how TA and training can be improved, especially with regard to the achievement of results.

  • Identify the main challenges in the implementation of TA recommendations.

  • Elicit authorities’ views on the sustainability of the main outcomes resulting from STA’s TA and training.

  • Elicit users’ views on the usefulness of macroeconomic statistics and, where relevant, on STA’s capacity building activities to improve them.

  • Assess whether funding difficulties of CAPTAC-DR, which may lead to a reduction in STA’s TA envelope to Guatemala, would have an impact in the areas of need.

10. To get the views of the authorities and key users on the effectiveness of TA and future needs, two surveys were conducted mostly prior to the mission with BG’s valuable support.

  • 13 responses were received from a total of 25 questionnaires (52 percent response rate) to officials from the three main data-producing agencies (BG, NSI, and MPF).

  • 25 responses were obtained from a total of 42 questionnaires (60 percent response rate) addressed to main users from public and private sector entities residing in Guatemala.6

11. The mission also held individual meetings with representatives from 18 public and private sector institutions, including large commercial banks, supervisory agencies, business and industrial chambers, private universities, regional research centers, embassies, and local offices of international organizations.

12. The main comments made by TA recipients and key users of macroeconomic statistics are summarized in the Boxes 2, 3, and 4, and the summary statistics to both surveys are presented in Annexes IV and V, respectively.

III. TA and Training Provided by STA and Related Results

13. Most datasets covered by the Evaluation mission are compiled and disseminated by the BG (national accounts, MFS, and ESS). The MPF focuses on fiscal data and the NSI on price statistics, except XMPI that are also compiled by the BG.

14. Guatemala has received intense TA and training from STA on these macroeconomic datasets during 1998-2015.7 However, missions were unevenly distributed during these years. All countries in the region, but particularly Guatemala, have notoriously benefited from the launching of CAPTAC-DR in mid 2009. Since 2010, Guatemala’s TA has increased to an average of ten single-topic TA missions per year, mainly focused on national accounts, prices, and ESS. Similarly, participation of Guatemalan officials in IMF courses and other training activities has increased considerably to about 23 officials per year since 2010, mainly through courses delivered by CAPTAC-DR in Guatemala.

A. Multi-Topic Missions

15. During the period under review, STA conducted Multisector and data Report on the Observance of Standards and Codes (ROSC) 8 missions. Both missions provided recommendations to improve the compilation and dissemination of macroeconomic datasets.

The 1997 Multisector Mission

16. The July 1997 Multisector mission found that the capacity of the statistical system to generate the information required for the macroeconomic analysis was not entirely satisfactory. The quality of source data was not uniform and no harmonized data compilation methodologies were applied, which limited the usefulness of the information for the formulation, execution, and pursuit of the macroeconomic policy. In some cases, data coverage was insufficient or incomplete and in others, such as the censuses and surveys, very outdated.

17. Certain elements of the fiscal statistics, national accounts, and the balance of payments (BOP) were not sufficient for adequate surveillance and the pursuit of an economic program. The Multisector mission underscored that data source availability in some sectors had been severely affected by the international trade and financial liberalization that took place in Guatemala in the mid-1990s, and did not accurately represent the economic and financial activities.

18. The recommendations provided by the Multisector mission mainly focused on increasing data sources and improving the coverage of national accounts and, at to a lesser extent, ESS and the fiscal accounts. Updated information on the manufacturing sector, commerce, and services was very limited for the national accounts, and data quality of international trade in services, foreign investment (FDI), and other financial data was insufficient for BOP compilation.

The 2004 Data ROSC Module

19. The October 2004 Data ROSC provided an in-depth assessment of the quality of Guatemala’s macroeconomic statistics and prioritized recommendations to further improve their quality.9 The ROSC mission assessed the national accounts, CPI, GFS, MFS, and BOP statistics.

20. The mission also conducted an assessment of the three main data-producing agencies, and found that—although the legal framework for statistical activity was broadly adequate and institutions displayed a commitment to disseminate macroeconomic statistics—stricter implementation of the law and greater reach out efforts could help address reticence by nonfinancial entities to respond to data requests.

21. The mission noted that there was ample scope to improve the methodological basis and source data underpinning most datasets, as well as the mix and level of resources in a few cases. Following the mission, Guatemala completed preparations to become a participant of the General Data Dissemination System (GDDS) in December 2004.

B. Evaluation Mission’s Findings on Single-Topic Missions

22. This section presents the findings of the Evaluation mission by topical area based on the assessment of the implementation of the main recommendations from about 70 single-topic TA missions conducted during FY 1998-FY 2015. For each topical area, the section identifies the main results that were achieved with Fund support and the Evaluation mission’s findings. In addition, Table 1 shows the progress made by the authorities by comparing the coverage, periodicity, timeliness, and methodology at the time of the 1997 Multisector mission with the situation found by this Evaluation mission. Table 1 also summarizes milestones and areas for improvements in data quality during the period (last column).

Table 1.

Guatemala: Comparison of 1998 and 2015 for Periodicity, Timeliness, and Methodology of Macroeconomic Statistics

article image
article image

National Accounts

23. STA provided intense TA on national accounts to the BG with 28 missions since 1998. Two missions followed the 1997 Multisector mission and three missions the 2004 data ROSC. Since 2010, 23 missions were conducted under the CAPTAC-DR’s National Accounts Harmonization Project in Central America.

24. The Multi-topic missions underscored that the methodology of the national accounts was seriously outdated (SNA 1953) and GDP and expenditures were estimated with the 1958 base year. Source data were almost exclusively based on administrative records. Censuses were not conducted and the use of surveys was extremely limited. No adjustments were made to incorporate informal, illegal, and underground activities.

25. Most recommendations were implemented through the adoption of the 1993 SNA and the new base year project (2001), whose series were published in April 2007. STA’s TA supported since 2005 efforts toward a system of quarterly GDP, by production and expenditure approaches, based on international best practices. The system allowed the dissemination of estimates at current prices and volume measures consistent with annual data, which were first disseminated in September 2010.

26. CAPTAC-DR’s TA missions focused mostly on a broad range of issues for improving the quality of data sources and compilation methods for the annual accounts, quarterly GDP, the monthly indicator of economic activity (MIEA), and other related topics, including the implementation of the 2008 SNA. The most important recommendations provided by STA were addressed. Currently, the BG is working toward a new base year (2013), which in the case of Guatemala is the vehicle for completing the implementation of the 2008 SNA.

27. The latest population census was conducted in 2002, which may be adversely affecting the quality of national accounts. Although labor statistics are not part of STA’s capacity building, they shown some, but still insufficient, progress.

Prices

28. STA conducted 13 TA missions on price statistics to the NSI since 1998; initially to support the implementation of the recommendations of the Multisector mission to improve the CPI (weight updates, seasonality adjustments, and bridge tables to facilitate the classification matching). NSI received considerable TA from the Economic Commission for Latin America and the Caribbean (CEPAL, in Spanish) for updating the CPI base year. The Multisector mission recommended that a comprehensive producer price index (PPI) should replace the annual wholesale price index (WPI). The data 2004 ROSC mission found that the CPI methodology was brought in line with the 2004 CPI manual, and prices and weights broadly followed the 1993 SNA. Due to budgetary constraints, NSI does not compile regional CPIs. No PPI was produced by the authorities but a monthly WPI was compiled.

29. Since 2010, CAPTAC-DR provided TA to the BG to compile export and import price indices (XMPIs) and to the NSI towards initiating the compilation of PPI statistics.10

  • The six TA missions to the BG focused first on improving the unit value indices of exports and imports and then on developing methodology for the compilation of XMPIs based on price specifications rather than custom data. The Evaluation mission found that XMPIs were in the final stage of development.

  • Work with the NSI focused on the compilation of PPI statistics. Seven missions were conducted to support moving from a quarterly wholesale price index (discontinued now) to a monthly PPI. The PPI project was initially well supported by the NSI; however, lack of budgetary and human resources has brought uncertainty. The mission found progress, including the definition of samples, products and outlets, and the estimation of weights. Two PPI pilots had been carried out. The PPI project needs to be finalized soon and assurances for its viability and sustainability should be obtained through coordination among agencies, including the BG.

Government Finance Statistics

30. The data ROSC mission found that the methodology was consistent with the GFSM 1986 with no plan to migrate to the GFSM 2001 (which is still the case). STA fielded five TA missions to the MPF between 2009 and 2012, mainly in the context of the GFS Harmonization Project for Central America, and in coordination with the Council of Ministers of Finance of Central America, Panama, and Dominican Republic (COSEFIN). The GFS harmonization project had limited progress due to lack of political commitment in most participating countries as well as limited availability of data sources. At the time of the mission, CAPTAC-DR received a request from COSEFIN to consider redefining and initiating a new GFS regional harmonization.

31. Fiscal statistics disseminated by the MPF are limited to the budgetary central government and excludes the social security agency, other non-budgetary central government institutions, local governments, and public enterprises. Therefore, no data on consolidated fiscal statistics are available, even for central government operations. Although budgetary central government data are comprehensive and timely, source data for the rest of the public sector are scarce and have substantial lags. It is worth noting that the MPF reports quarterly government debt position data to the World Bank’s Public Sector Debt (PSD) database.11

32. The GFS missions designed and implemented a bridge table in line with GFSM2001 for MFP compilers to produce GFS, initially at the level of budgetary central government. Nevertheless, the MFP discontinued the use of the bridge table.

33. The Evaluation mission discussed with the authorities the creation of a fiscal statistics unit in the MPF as an initial step toward improving GFS compilation. Sustainability of results would require authorities’ buy-in and commitment at the highest levels of government.

Monetary and Financial Statistics

34. The data ROSC mission found that the methodology and scope of the central bank survey was in line with the MFSM 2000. Nevertheless, the other depository corporations (ODC) survey excluded deposit-taking institutions such as the offshore banking sector and credit and saving cooperatives. The classification/sectorization of commercial banks and ODC were deficient due to an outdated plan of accounts that did not allow for sufficient classification/sectorization (including the identification of nonresidents’ accounts).

35. STA provided seven TA missions to the BG between 2002 and 2008. These missions (except the first one) were conducted in the context of the successful MFS Harmonization Project for Central America, in close coordination with the Central American Monetary Council (CAMC). Guatemala was in the lead of this successful project. Significant improvements in coverage, classification, and sectorization were achieved, and data now include the offshore banks and, to a lesser extent, credit and saving cooperatives. The relative importance of both sectors has been gradually decreasing.

36. In addition, STA has provided remote TA to the BG, allowing the reporting of monthly financial soundness indicators (FSIs) to the Fund since January 2015. More residents/nonresident data disaggregation is required to further improve consistency between MFS and ESS, for which coordination with SIB is required.

External Sector Statistics

37. The data ROSC mission found that limited progress was made in the transition from BPM4 to BPM5. Departures from the criterion of residency affected estimates of private capital flows that included exchange transactions between residents. The main data sources were administrative records, including customs for trade data and the International Transactions Reporting System (ITRS) for other transactions. Virtually no data were collected through surveys, No international investment position (IIP) data were compiled.

38. STA provided 19 TA missions on a wide range of ESS, mainly including BOP, IIP, external debt, reserves template, and participation in IMF’s world-wide direct investment and portfolio investment surveys. Two missions took place in 2004-2006 and the rest were conducted through the ESS Harmonization Project in Central America, which will conclude in December 2015. The TA missions have assisted in the compilation and dissemination of:

  • Quarterly BOP and quarterly IIP statistics on a BPM6 basis;

  • Annual IIP statistics for the first time in 2010;

  • Inward and outward FDI data for Guatemala’s participation in the CDIS12;

  • Quarterly total external debt data reported to the World Bank’s QEDS database; and

  • Monthly reserves template data.

39. Further work is needed mainly to continue improving the coverage of the maquila sector13, domestic holdings of external assets, and the nonfinancial private sector.

40. STA assisted in Guatemala’s transition into BPM6.14 Some methodological differences with national accounts statistics need to be addressed by the authorities; e.g., within the ongoing work in the new GDP base year.

41. TA supported exploratory work to assess possible participation of Guatemala in the IMF’s Coordinated Portfolio Investment Survey (CPIS),15 reaching results that predictably confirmed that Guatemala does not hold large portfolio investment assets. On the positive side, work was useful to improve the recording of portfolio investment in the BOP and IIP.

C. Guatemala’s Participation in IMF Standards for Data Dissemination

42. Guatemala joined the GDDS in December 2004 with STA’s TA and training support. Most data categories meet the recommended coverage, periodicity, and/or timeliness. GDDS metadata are outdated (last update for most datasets took place in 2011), and no metadata are available for some categories including IIP.16

43. Despite significant improvements in the dissemination of macroeconomic statistics, the following data categories do not meet the requirements of the Special Data Dissemination Standard (SDDS) for coverage, periodicity, and/or timeliness:

  • Labor market indicators (employment, unemployment, and wages/earnings) are not estimated on a quarterly basis;

  • Revenue, expenditure, and financing data for social security agencies, local governments, and nonfinancial public enterprises are not disseminated;

  • Data on the insurance sector and savings cooperatives are not available weekly; and,

  • PPI is not yet compiled.

44. The authorities indicated that SDDS subscription would be a medium-term objective. The mission discussed the recent creation of the Enhanced GDDS (e-GDDS) to support countries’ efforts to move toward data dissemination to the benefit of market participants and the general public. The mission urged the authorities to post IIP metadata on the IMF website and suggested considering full participation in the e-GDDS once general government operations data become available.17

IV. Views on TA Effectiveness, Efficiency, and Impact and Sustainability

45. The Evaluation mission elicited views on the effectiveness, efficiency, impact, and sustainability from a wide range of state holders in Guatemala both through two surveys as well as during individual meetings (section II). The mission also elicited views from WHD staff working with Guatemala both before and after its visit to Guatemala.

A. Official Data-Producing Agencies in Guatemala

Technical Assistance

46. The authorities viewed TA provided by STA as most effective and, in some cases fundamental, in supporting their efforts to improving macroeconomic statistics. TA is highly regarded as a source of methodological knowledge and crucial in achieving future objectives. Areas covered by TA corresponded very well to their priorities, and the content and timeframe for implementing missions’ recommendations were sufficiently discussed. TA was useful and relevant for all datasets but GFS, where it was considered only somewhat useful.

47. The authorities considered that the quality of macroeconomic statistics in Guatemala is reasonably good and underscored that significant improvements were made during the period under review. The implementation of TA recommendations highly benefited the quality of NA, MFS, and ESS and, to a lesser extent, of GFS.

48. The authorities considered that the most relevant TA recommendations were implemented except for GFS. The limited implementation for GFS was attributed mostly to the lack of source data and managerial decisions. Although there might be some international funds for the completion of the first part of the PPI project, the authorities recognized that the likelihood of completing the PPI project is very limited. Some limitations in the availability of source data were also mentioned for MFS and ESS (see previous section).

49. The modes of TA delivery in Guatemala were suitable, relying mostly on short-term TA missions following the 1997 Multisector and the 2004 data ROSC missions. Until 2009, these missions were mainly delivered by peripatetic experts and, since 2010, through short-term TA missions conducted by CAPTAC-DR’s resident advisors supplemented by short-term expert missions. The authorities considered that CAPTAC-DR has played a significant role in the improvement of NA and ESS, and previously in MFS.

50. The authorities mentioned that the criteria to assess progress in capacity building activities should include: (a) brief reports on TA results; (b) dissemination of improved data; (c) better periodicity, timeliness, and/or coverage in data; (d) experts’ reports on methodological implementation; and (e) work plans with clear benchmark actions.

51. According to the authorities TA provided by the Fund was not particularly helpful to facilitating the coordination and interaction with other donors. Box 2 presents in detail comments provided by TA recipients, identifying those of a general nature from comments that are specific macroeconomic datasets.

Guatemala: Comments by recipients of TA on Macroeconomic Statistics

General Comments

  • The success of TA largely depends on the ownership of the beneficiary institution and on adequate coordination with the technical counterparts.

  • TA provided useful feedback on developments and allowed compilers to strengthen statistics.

  • Data dissemination has improved enormously during the last years, partially due to the progress in technology, but also due to its continuous support by IMF’s TA and training.

  • Following international guidelines is critical for macroeconomic statistics compilation.

  • No official data sharing protocols in place but institutions exchange data regularly and timely.

  • Data quality controls prior to dissemination are a good practice.

  • Guatemala should advance towards the subscription to the SDDS.

  • TA mission reports were adequate although their content and timeliness may be improved.

National Accounts

  • Support is needed in the compilation and validation of the results on specific national accounts topics such as work-in-progress, owner-occupied dwellings, etc.

  • Input-output tables should be prepared.

Prices

  • CPI is calculated at a national urban level; regional or rural calculations would be too costly.

  • NSI suffers from serious financial and human resources constraints.

  • PPI project is uncertain (no budget, only contractual staff).

Government Finance

  • Guatemala needs to compile and disseminate GFS according to current international standards; as well as implement the Manual on Fiscal Transparency.

  • Although largely incomplete and unfinished, the progress made in Guatemala on the regional GFS harmonization project was substantial until suspended.

Monetary and Finance

  • The role played by the IMF on the successful regional MFS harmonization project in Guatemala (and in the region) was essential.

  • It is necessary to improve periodicity and timeliness and increase the frequency of harmonized financial and monetary statistics and to develop timely flow of funds.

External Sector

  • Data should be compiled with the latest methodology.

  • Criteria to assess progress of ESS should include good data coverage and dissemination in line with current international standards and according to a preannounced calendar.

Source: Survey and meetings with recipients of TA in macroeconomic statistics in Guatemala.

Training

53. Considering that BG staff have been by far the largest recipient of training on macroeconomic statistics, the mission held a meeting with a selected group that had participated in training activities at the IMF HQ, the IMF Brazil training center, and/or at CAPTAC-DR’s HQ in Guatemala since 1998.

54. Training in macroeconomic statistics is highly regarded by the authorities and seen as very useful to facilitate sustainable improvements. Details are presented in Box 3.

Guatemala: Comments by recipients of Training on Macroeconomic Statistics

  • Training has been relevant and useful in broadening the knowledge of participants.

  • Transmitting the learning to other staff is an institutionalized practice in the BG.

  • Sponsoring institutions should take measures to minimize staff rotation after training, so the learning can be transferred before personal is moved.

  • Sharing experiences with other countries of the region facilitate the implementation of best practices.

  • Subsequent interaction with both other participants and lecturers has been very beneficial.

  • Training should be complemented with subsequent TA, since it will increase its effectiveness and continuity.

  • A more rigorous selection of participants could help ensuring that the level of knowledge of participants is comparable and thus make it more beneficial.

  • Hand-on training (workshops) is particularly welcome and preferred to regular courses.

  • Longer courses are preferred. When shorter versions are delivered, content is too compressed and it ends up being detrimental for the purposes of the course.

  • Providing background material prior to the course should be useful as a sort of knowledge leveling to participants.

  • Courses on inter-linkages between macroeconomic datasets would be welcome.

  • Assessment of individual course lecturers by participants would provide useful feedback.

  • No clear preference for training provided in Guatemala or abroad was expressed.

Source: Survey and group meeting with participants of course in macroeconomic statistics.

B. National Users of Statistics

55. There is a broad recognition of the BG’s key role in producing and disseminating statistics but also on the relative weakness of the NSI and to a lesser extent of the MPF. Overall, users trust macroeconomic statistics available in Guatemala and have observed significant improvements in the amount, detail, and access to data and related documentation. Several users considered that informal and illegal sectors play a large role but are not well covered (or not cover at all) in the official statistics. A number of users expressed strong interest for a flexible application for downloading data. Work on an interactive database freely available to users was being conducted by the BG at the time of the mission.

56. The mission found that TA and training activities provided by CAPTAC-DR are virtually invisible to most users in Guatemala. Few users that were familiar with STA’s capacity building activities considered that they played an important role in the improvements in the quality of statistics. They are of the view that TA facilitated the dissemination of better statistics through improved dissemination practices.

57. Advanced users of statistics in Guatemala seem concentrated in a small group. No broad knowledge about IMF’s TA; those aware consider it useful. Users underscored that a new population census to replace the 2002 census was of outmost importance.

Guatemala: Comments by Users of Macroeconomic Statistics

General Comments

  • The BG is broadly recognized as a reliable, serious, independent, and professional institution that produces very good quality statistics.

  • The BG is very accessible for statistical requests/questions.

  • BG’s outreach activities are highly regarded (including weekly breakfasts).

  • BG’s expectations survey is a well received product.

  • There is some lack of consistency between data produced by different institutions.

  • Appropriate metadata should always accompany new statistics developments.

  • Surveys’ results are not disseminated by the BG, but used internally only.

  • Customs data are under public scrutiny following recent scandal.

  • NSI plays a limited role in macroeconomic statistics.

  • More focus on employment and poverty indicators.

  • Labor statistics should improve coverage, frequency, and timeliness.

National Accounts

  • Quarterly national accounts are highly valued.

  • Timeliness of national accounts statistics should be improved.

  • Public availability of an official business register is appreciated.

  • The last base year was positively received but a new update is need.

  • Information on a regional and departmental level is essential.

  • More details of GDP by economic sector and specific analysis of the results are needed.

  • The monthly indicator of economic activity (IMAE) is too aggregate.

  • Compilation and dissemination of satellite accounts, especially tourism, are needed.

  • Input-output matrices should be publicly available.

Prices

  • Consensus about good quality of the CPI.

  • Methodological notes should be available to better understand the CPI compilation.

  • CPI should have regional and urban representation.

  • PPI should be produced.

Government Finance

  • The MPF should have an office dealing with statistics and fiscal deficit analysis.

  • Autonomous institutions are not covered in the fiscal data.

  • Broaden the coverage of fiscal data to include local governments.

  • Urgent need to implement the latest methodologies for the GFS compilation.

  • It is necessary to have consolidated data for the government sector as a whole.

  • Lacking a complete picture of the government finances may reflect that data are hidden for political reasons and creates serious doubts of fiscal sustainability.

Monetary and Financial

  • Improve the access and download ability of MFS data at all levels.

  • Improvements on banking system indicators, details on loans by currency, and availability of longer time series. Missing detail by economic activity.

  • More detail on interests rates by maturity, money and stock market, warranties, etc.

External Sector

  • Data on external trade statistics in volume terms should be available in detail, in a flexible and user-friendly format. Methodological notes should be available.

  • Increase the detail on the dissemination of the services in the BOP (call centers, tourism, IT related services, etc), and improve direct investment statistics

Timely dissemination of quarterly BOP would be much appreciated.

Source: Survey of users of macroeconomic statistics and users’ interviews in Guatemala.

C. Western Hemisphere Department

58. The 2014 Article IV Staff Report noted that data provision has some shortcomings, but is broadly adequate for surveillance. Unemployment is estimated only on an annual basis; revenue, expenditure, and financing statistics for social security agencies, local governments, and nonfinancial public enterprises are not reported; data for the offshore sector and savings cooperatives are only available quarterly, and with a lag of five weeks; and detail on private sector external debt by creditor and by maturity structure is not available.

59. Looking forward, WHD underscored the need for an updated GDP base year, (now considerably overdue), improved coverage of the informal economy, PPI statistics, and labor cost data. In addition, customs data raise some concerns. Lack of data on nonfinancial public sector hampers the ability to produce consolidated fiscal data beyond the central government. Price measures for real estate are not available but estimation efforts are in place.

V. Guatemala’s Macroeconomic Statistics in a Regional Perspective

60. Guatemala shares geographical and cultural similarities with other Central American countries and participates in regional arrangements and institutions that support the process of regional monetary and financial integration.18 The harmonization of macroeconomic statistics in Central America is highly regarded by the regional institutions and their state members. In this context, countries have received during similar spans of time TA and training that have been somehow homogeneous among them. Progress in the region has not been exclusive of Guatemala nor has the support from the IMF and other TA providers.

How Effective TA has been in Guatemala with Respect to Other Countries in the Region?

61. The mission obtained a first approximation to the effectiveness of TA and training in Guatemala relative to other countries in Central America using the following approach:

  • Review the assessments of the quality of macroeconomic statistics prepared by data ROSC missions conducted in Central American countries during 2001-2006; 19

  • Identify the amount of TA and training on macroeconomic statistics provided by STA to these countries during the period under review20; and

  • Elicit views of frequent users of macroeconomic statistics in the region on the quality of statistics in Guatemala with respect to other countries in the region during selected interviews with private sector banks, chamber of commerce and industry, universities, research centers, international organizations, and CAPTAC-DR’s resident advisors.

62. The assessments prepared by the IMF’s data ROSC missions during the past decade showed that the quality of Guatemala’s macroeconomic statistics ranked among the bottom two in the region, particularly in national accounts and BOP statistics.

63. In terms of capacity building provided by STA, Guatemala ranks among the two lowest TA recipient countries but among the top two countries receiving training in the region during 1998-2015.21 Naturally, when regional training takes place, host country’s participation is higher. A large part of regional training provided by CAPTAC-DR has taken place in Guatemala. When considering other courses in the region (e.g. Brazil and Mexico), Guatemalan representation falls to the lowest in the list.

64. Most viewed that progress was achieved in the region as a whole as well as in individual countries. Nevertheless, users considered that progress was uneven if assessed by individual topics and/or by countries. In users’ views, macroeconomic statistics in Guatemala rank among the top three in the region. In particularly, they underscored factors like accessibility, timelines, transparency, and detail. Some users underscored that this was not the case in the previous decade and significant progress has been made in the recent years.

65. The Evaluation mission considers that users’ views on the relative level of Guatemala’s statistics in the region are fair. Starting from a relatively poor level in the past decade, as consigned in reports both from STA and WHD, Guatemala made significant progress with TA and training provided by the IMF, mainly in national accounts, MFS, and ESS. Progress was reflected in several new datasets that are now available in a much more user-friendly manner, the implementation of new methodologies, the update of base year in national accounts and prices, and improvements in the inter-institutional data transferring.

66. The positive outcomes reflect the authorities’ strong commitment to improving macroeconomic statistics by fostering staff development, commensurate IT support, and survey programs on source data, supported by additional TA providers.

VI. Data Availability

A. Reporting to STA

67. The mission discussed with the authorities data reporting issues to STA. Overall, Guatemala reports data regularly. The mission stressed the importance of reporting preliminary data followed by revisions, rather than delaying until final data became available.

Improvements in macroeconomic statistics are reflected in data reported to STA. Periodicity of national accounts and data on exports and imports of goods and services increased from annual to quarterly. ESS also showed major improvements, including the production of quarterly IIP. Timeliness for macroeconomic data series improved significantly, except for gross national income and population (see Table 2).

Table 2.

Guatemala: Data in International Financial Statistics (IFS)

article image
Source: IFS, June 1998 and June 2015

Cross-sector Consistency

68. The mission reviewed STA’s cross-sector consistency table for Guatemala and found that data were broadly consistent with two exceptions. The first arose when comparing goods’ exports and imports in BOP data with Direction of Trade Statistics. The two datasets are based on different methodologies including different valuation bases. The second difference relates to foreign assets and liabilities for depository corporations in MFS and IIP. This reflects different recording treatment in direct and portfolio investment equity.

B. Other Data Dissemination

69. In addition to reporting for the IFS, Guatemala disseminates additional macroeconomic statistics for posting on other databases from by international organizations.

  • Coordinated Direct Investment Survey: Annual inward/outward data since 201022

  • Reserve template data: Monthly data since August 200823

  • Total external debt position data: Quarterly data since December 201124

  • Public sector debt position data: Quarterly data since December 201225

  • Financial soundness indicators: Monthly data since January 201026

VII. Lessons Learned from the Experience in Guatemala

70. A primary motive for this report is to help STA in assessing the effectiveness of its TA and training programs and to explore areas for improvement. To this end, the Evaluation mission summarizes below the main lessons learned from capacity building experiences in Guatemala during 1998-2015.

71. While lessons learned are based solely on Guatemala’s experience, the mission considers that they are likely to be pertinent to STA delivery of TA and training more generally, including issues faced by other developing economies and in other IMF Regional Training Centers (RTACs).

Quality of macroeconomic statistics and effectiveness of TA and training

  • Overall, users trust macroeconomic statistics and provide positive feedback on BG’s role in their production and dissemination. Macroeconomic statistics in Guatemala are adequate for surveillance, notwithstanding GFS are limited to the budgetary central government.

  • There was consensus that TA and training provided by STA was effective and contributed to improving the quality of macroeconomic statistics with different degrees of success. Data-producing agencies expressed high appreciation for TA and training delivered by STA.

  • TA and training has supported the adoption of internationally-accepted statistical methodologies that contributed to improve cross-country data comparability as well as inter-sectoral consistency in Guatemala.

  • TA and training has supported participation in IMF’s data standard initiatives. Guatemala is a GDDS participant since 2004. The authorities expressed strong interest in SDDS subscription as a medium-term objective. Guatemala may consider full participation in the e-GDDS initiative.

Guatemala in a Regional Perspective

  • Most users of macroeconomic statistics in Guatemala consider that these statistics have improved considerably and now rank among the top three in the region.

Delivery Mode

  • Flexibility in the modalities of TA and training delivery contributes to effectiveness.

  • TA and training in Guatemala have been conducted largely in the context of projects to harmonize macroeconomic statistics in Central America with active STA participation. The first harmonization project was in MFS, followed by GFS, and, since 2010, NA, prices, and ESS.

  • The 1997 Multisector and 2004 data ROSC missions provided useful foundations for action plans for improving macroeconomic data that were pursued through single-topic missions.

  • Close contact between TA providers and their national counterparts may enhance the effectiveness of TA by supporting authorities’ efforts in the implementation of recommendations. Guatemala benefited from having regular accessibility to resident advisors, given that CAPTAC-DR’s headquarters are located in Guatemala.

  • Remote TA could be considered to support the authorities’ work in implementing recommendations. A positive example was remote TA delivered by STA to Guatemala to compile and disseminate financial soundness indicators (FSI).

  • STA courses are highly regarded. Hands-on training and sharing experiences with other participants enhance the quality of training substantially by focusing on practical cases, coordinating activities, and learning from each other.

Institutional Capacity and Resource Availability

  • The effectiveness of TA depends to a large extent on the institutional capacity built in the main data-producing agencies. When domestic capacity is constrained, tasks become significantly more challenging. The BG demonstrated good track record in implementing TA recommendations.

  • The commitment and supportive attitude of senior management and staff in the data-producing agencies are decisive factors for TA success. Buy-in from senior management is instrumental to ensure resource availability needed for implementing TA recommendations and follow-up actions.

  • While resources are broadly sufficient for statistics produced by the BG, governance and resource issues were big constrains to improvements in GFS and PPI statistics under the scope of the MPF and the NSI, respectively.

  • Staff turnover in data-producing agencies may slow down and even deteriorate the progress achieved in macroeconomic statistics.

  • The viability and sustainability of the PPI project is uncertain. Insufficient staff and financial resources and frequent changes in leadership in NSI management may severely hamper the timely culmination of the PPI project and its sustainability.

Communication and Coordination

  • CAPTAC-DR does not have too much visibility with data users in Guatemala, neither in terms of TA and training provided nor in outcomes achieved. Familiar users underscored that capacity building mostly helped in improving data dissemination.

  • Close communication between TA providers and the authorities fosters more effective and relevant TA.

  • Area Departments may play an important role to enhance TA effectiveness by helping leverage the relevant data to policy-decision makers and key data users. Given their access to senior management of statistical agencies, they could foster the pace of implementation of TA’s recommendations and in turn support solutions to constraints faced by the agencies.

  • Inter-institutional coordination between data producing agencies has strengthened the capacity to produce and the sustainability in the production of certain specialized statistics. The BG and MPF hold weekly coordination meetings, and coordination with the NSI includes participation in regular statistical committees. Special institutional arrangements may also be of relevance.27

  • In general, the authorities recognized the need to further strengthen inter-institutional collaboration beyond existing mechanisms, and welcomed the Fund to bring the three main data-producing agencies together to reinforce these bonds.

  • TA missions may do more to deal with outreach and dissemination particularly for users outside the data-producing agencies.

  • Well-defined mechanisms are needed to avoid duplications, inconsistencies, and promote synergies among advice provided by different TA providers. TA providers may have different views on priorities, sequence, and/or approaches to address particular issues.

TA Reports

  • In general, the quality of TA reports was high. TA recipient agencies were able to provide comments on draft reports usually left by the missions at the end of their visits and discuss their recommendations and timeframes. Final reports were received within reasonable time in most cases, but serious delays were observed particularly in CAPTAC-DR’s national accounts and, to some extent, ESS missions.

  • The inclusion of benchmark actions in TA reports since 2012 proved to be of great importance in balancing the progress made and timely follow up.

VIII. Conclusions and Recommendations

72. The mission considers that Guatemala has notoriously benefited from STA’s TA and training during 1998-2015. TA and training activities were broadly effective in supporting the authorities’ efforts to improve the quality of macroeconomic statistics except GFS. Based on its findings and lessons learned, the mission presents below a number of recommendations to be considered in STA’s TA and training programs.

Effectiveness and Sustainability

  • When appropriate, base the design of a capacity building program on the diagnostic and recommendations of Multisector or data ROSC missions.

  • Take in consideration the absorption capacity when identifying TA objectives, timing, and recommendations as a major factor in effectiveness and sustainability.

  • Move toward the adoption of a formal RBM approach to TA delivery, which would facilitate preparation, follow-up, and evaluation of future TA and training efforts.

  • Foster authorities’ commitment in mission planning. Ownership is essential to ensure progress the implementing of the program and the sustainability of its results.

  • Promote the use of international statistical standards both for compilation and dissemination of data and metadata, including advance release calendars.

  • Address inter-sectoral data consistency issues more fully in TA and training activities. Single-topic missions could perform basic checking of cross-sectoral consistency.

Communication and Coordination

  • Maintain close liaison with senior management in data-producing agencies to enhance their understanding of TA objectives, recommendations, and follow-up actions.

  • Promote coordination among TA providers, including experts directly hired by the beneficiary institution, to ensure consistency in recommendations and priorities.

  • Foster outreach to increase visibility of capacity building efforts, especially when TA is delivered as part of a well-defined project.

  • Pursue adequate balance between TA field work and the preparation of final reports. The timeliness in the preparation and dissemination of final reports is crucial.

Modalities of Delivery

  • Explore nontraditional TA and training modalities to suit the evolving needs of the authorities, including by (video conferences, real time chats, etc).

  • Consider separate courses for introductory and advance participants. Advance participants benefit more from hands-on workshops and specialized seminars.

  • Conduct IMF courses that provide opportunities for Guatemala staff to discuss country-specific issues with Fund staff and other countries’ compilers. GFS

  • Consider the creation of a small unit of fiscal statistics in the MPF to facilitate GFS compilation. Consolidate progress made in bridging budget execution data to the GFSM2001 and gradually move toward the consolidated public sector.

  • Establish specialized contact points in STA and MPF to facilitate GFS work, including resuming bridging the national budget execution data to the GFSM 2001 format implemented by previous TA missions.

IX. TA and Training Needs for Guatemala in the Future

73. Despite notable improvements in most macroeconomic statistics in Guatemala, further work is needed in some specific datasets, including from STA’s capacity building program. Main TA needs are in GFS, national accounts, and prices, although the authorities would need to ensure sustainability of the results (see Box 5).

74. In addition, Guatemala should continue active participation in training activities on macroeconomic statistics conducted by STA in IMF regional training centers and/or jointly conducted with other relevant regional institutions (CAMC, CEMLA. CEPAL, etc).

75. At the time of the mission, CAPTAC-DR had no budget for GFS. Pressing TA needs were identified for the compilation of comprehensive GFS beyond the budgetary central government operations. Subject to funding, CAPTAC-DR’s involvement would require the authorities’ strong commitment to support GFS improvements.28

Guatemala: TA and Training Needs in the Future

National Accounts

  • The BG underscored the need to continue with CAPTAC-DR’s capacity building on national accounts to conclude the implementation of the new base year, and to adopt 2008 SNA.

GFS

  • The MPF expressed strong interest in receiving TA and training in GFS to (1) focus on data compilation using the GFSM 2001 and its update (GFSM 2014); (2) increase the availability of fiscal data beyond the central government, and (3) implement the consolidated fiscal accounts.

  • The BG supported the MPF views on the high-priority to be given to GFS.

  • There is an integrated system for accounting, treasury, and budget that may facilitate migration from administrative data to fiscal statistics; Nevertheless TA and training is required.

Prices

  • The NIS would like to progress in the compilation work for monthly PPI statistics.

  • The sustainability of the results of the PPI project is uncertain to say the least, unless commensurate and well-identify resources are made available to this endeavor.

MFS and ESS

  • STA support on MFS and ESS may be needed to make further progress.

  • Remote TA and participating in STA regular training courses were deemed appropriate.

Guatemala: Technical Assistance Report-Statistics Technical Assistance and Training Evaluation Mission
Author: International Monetary Fund. Statistics Dept.