Peru: Report on the Statistics Technical Assistance and Training Evaluation Mission (December 12 - 16, 2011)

This paper discusses the evaluation of technical assistance (TA) and training provided by the Statistical Department (STA) of the IMF to Peru during 1993–2011. The results showed that TA was reasonably effective, especially in the areas of government finance statistics, and responded well to the demands expressed by the Peruvian authorities. Many constraints to capacity were overcome through TA and training. The evaluation also found that the quality of TA reports, backstopping, and report management in STA were generally adequate.

Abstract

This paper discusses the evaluation of technical assistance (TA) and training provided by the Statistical Department (STA) of the IMF to Peru during 1993–2011. The results showed that TA was reasonably effective, especially in the areas of government finance statistics, and responded well to the demands expressed by the Peruvian authorities. Many constraints to capacity were overcome through TA and training. The evaluation also found that the quality of TA reports, backstopping, and report management in STA were generally adequate.

I. Introduction

1. As part of the STA TA program and in collaboration with the Western Hemisphere Department (WHD), a mission visited Lima, Peru during December 12–16, 2011. The main purpose of the mission and activities before and after the mission was to evaluate the results that have been achieved in Peru as a consequence of the TA and training in the compilation and dissemination of macroeconomic statistics during the past 20 years. Objectives of the mission were to: (1) elicit the views of the authorities and other stakeholders (Appendix I) on the effectiveness of TA and training provided by STA and discuss how it could be improved; and (2) discuss TA and training priorities for the period ahead. The mission also discussed outstanding issues related to data reporting to the Fund.

2. To address these tasks, the mission worked closely with officials from the main data compiling and disseminating agencies, including especially: the Central Reserve Bank of Peru (BCRP), National Institute of Statistics and Informatics (INEI), and the Ministry of Economy and Finance (MEF). The mission also met with officials from other government agencies, as well as users of Peru’s macroeconomic statistics and representatives from the donor community interested in the development of macroeconomic statistics in Peru. A list of government officials, users and donor representatives that met with the mission appears in Appendix I.

3. As part of the mission’s preparations and to facilitate these discussions with the authorities and private sector data users, the mission distributed a questionnaire and collected responses via the Resident Representative’s office in Lima, Peru, prior to the arrival of the mission (see Appendix V).

4. The mission would like to express its appreciation to the Peruvian authorities for their cooperation and assistance, as well as the excellent hospitality extended to the mission. The cooperation and dedication of the mission’s counterparts greatly facilitated the mission’s work.

5. Section II of the report provides background to the evaluation and its scope. Section III outlines the findings and discusses the results that have been achieved with the support of TA and training provided by STA to Peru in the four main areas of macroeconomic statistics. Section IV outlines the views of the authorities and other users on TA and training based on the questionnaire and interviews. Section V synthesizes lessons learned from the evaluation, while Section VI focuses on key conclusions and recommendations. Section VII briefly presents the outcome of the discussions held with the authorities on TA and training priorities for the future, and Section VIII reviews the outcome of discussions on data reporting to the Fund.

II. Background

A. Evaluation of STA TA and Training

6. The evaluation of the IMF Statistics Department’s technical assistance and training is increasingly founded on the Organization of Economic Co-operation and Development (OECD) Development Assistance Committee (DAC) Principles for Evaluation of Development Assistance.3 These principles state:

  • “An evaluation is an assessment, as systematic and objective as possible, of an on-going or completed project, program or policy, its design, implementation and results. The aim is to determine the relevance and fulfillment of objectives, developmental efficiency, effectiveness, impact and sustainability. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision-making process of both recipients and donors.” (paragraph 5)

  • “An important purpose of evaluation is to bring to the attention of policy-makers constraints on developmental aid success resulting from policy shortcomings or rigidities both on the donor and recipient side, inadequate co-ordination, and the effectiveness of other practices…”(paragraph 9).

7. “Evaluations should contain an executive summary, a profile of its purpose and scope (including the activities to be evaluated), a description of the methods used, the main findings, lessons learned, and conclusions and recommendations. Evaluations need to look at agency procedures and management issues.”

B. Background to the Evaluation of TA and Training to Peru

8. The TA evaluation mission to Peru follows on the 2005 Independent Evaluation Office (IEO) recommendations for developing more systematic approaches to track progress on major TA activities and to identify factors behind successes and shortfalls.4 The response of the IMF to the IEO recommendations pointed to the need to systematically draw out the lessons learned from the evaluation with a view to enhancing TA policy. These lessons should help provide a more informed background for future collaboration.

9. This TA evaluation is the first STA report that also takes into consideration the evolving work in the Fund to develop a Results Based Management framework.5 While none of the TA under review was formally formulated using this framework, the evaluation places emphasis on the results that have been achieved with the TA and training support of the Fund in macroeconomic statistics during the period under review (1993–2011). Appendix II lists the TA and related missions (a Multisector and a Data Report on the Observance of Standards and Codes (Data ROSC)) under review and Appendices III and IV describe training provided by the Fund to Peru on macroeconomic statistics.

10. To conduct the evaluation, the mission: (1) reviewed the amount, form, and topics of TA and training provided by STA since 1993 and reviewed the available TA documents prior to the mission (including briefing papers, TA reports and TA notes, and back-to-office reports); (2) summarized the findings of those missions and derived the results obtained by the Peruvian authorities with the support of this TA and training: (3) elicited the authorities’ views on the effectiveness of STA’s TA and training, and discussed the status of implementation of major recommendations made by STA, as well as the modes of TA and training delivery they consider most appropriate; (4) solicited views from national users of statistics, including interviews with TA donors and other stakeholders; and (5) consulted with WHD and used WHD documents (especially the Statistical Issues Appendix to the latest Article IV staff report, 2011) to collect WHD views on the usefulness and effectiveness of STA TA and training on Peruvian macroeconomic statistics.

11. To get the authorities’ and national users’ views on the effectiveness of TA, the mission provided each of the main data-compiling and disseminating agencies, and selected users, with a questionnaire on the usefulness of, and main lessons learned from, STA’s TA and training with a focus on results; the questionnaire also asked about future TA and training needs. The authorities provided completed questionnaires prior to the mission. The questionnaire and a summary of the responses appear in Appendix V.

12. This evaluation report summarizes findings, lessons, and recommendations regarding the TA and training with respect to: (1) relevance and effectiveness (including alignment with government priorities); (2) efficiency (including coordination with donors); and (3) impact and sustainability (including constraints on absorptive capacity). While relative rankings, and thus quantitative analysis of these characteristics were not possible to assign, qualitative assessments were undertaken, based on the views of stakeholders gathered through the process described above. In addition, the evaluation includes an assessment of the quality of the TA reports, backstopping, and report management.

13. The timeframe for conducting an evaluation, in principle, is not wide. A TA evaluation cannot be thoroughly accomplished before a project has been completed. At the same time, time, evaluation of a TA project years after it has been completed suffers from the passage of time, in part because key officials and other stakeholders may have changed jobs or shifted their focus. On the other hand, progress with the development of macroeconomic statistics can be a long-term undertaking and the perspective of time can help review the progress with results that have been achieved over long periods. Given this trade-off, the mission chose to review a long span of TA and training to evaluate the achievement of results, but the mission also recognizes that direct interaction with key players that may have been involved directly in the process may no longer be possible. Thus, the reader is cautioned that some critical information for the evaluation of some of the TA projects may not have been available to incorporate in this evaluation.

III. TA and Training Provided by STA and Related Results

14. Peru has benefited from significant STA TA and training. Since 1993, STA delivered a multisector mission in 1993 and several single-topic missions to assist the Peruvian authorities improve their statistics. A Data ROSC was conducted in 2003, which is not technically a TA mission, but contained diagnostic information as well as recommendations to improve many aspects of data quality. More specifically, Peru benefited from nine national accounts, two consumer price index, four government finance, two monetary and financial, and one balance of payments statistics missions. The 1993 multisector mission and the 2003 Data ROSC served as benchmarks and diagnostic missions, from which stemmed many of the single-topic missions, which provided follow-up support. In addition, over 100 officials have attended IMF training, at headquarters, in regional training centers, and in Peru. Notably, none of the TA has been delivered using long-term statistical advisors and all of the TA was provided directly from IMF financial resources, without recourse to other donors; however, training delivered through regional centers relied on support from partners that maintain those centers.

15. The mission also found that the quality of the TA reports, backstopping and report management were generally adequate and has improved over time. Most of the documents related to the TA provided by STA were found without difficulty (a list of TA missions was readily available from the TAIMS system that tracks IMF TA). However, in some instances, particularly for missions in the early 1990s, documents were either difficult to find or were not found. In one or two instances, a final TA report evidently was not prepared. In other instances a single report was prepared to cover multiple single-topic visits. In general, the quality of the TA reports has been high, in part due to thorough backstopping and review processes in STA. While these review processes may have been expensive, the final product benefited from such time and resource investments.

16. This section outlines the findings of the 1993 multisector statistics mission, the 2003 Data ROSC mission, and provides this TA evaluation mission’s findings based on an assessment of the implementation of the main recommendations from the set of single-topic missions that followed these broader diagnostic missions. The section then summarizes the main results that were achieved with Fund support and the mission’s findings. As noted above, Appendix II lists the TA missions delivered by STA during 1993–2011, while Appendices III and IV show the details of training provided.

A. The 1993 Multisector Mission

17. In general, the 1993 multisector mission identified several areas to strengthen inter-agency coordination regarding the compilation and dissemination of macroeconomic statistics. It recommended the establishment of a permanent high-level commission involving the BCRP, the INEI and the MEF to coordinate those efforts, including work on resolving data discrepancies to improve inter-sectoral consistency across related data sets, work to strengthen the legal foundation for data collection and statistical reporting, and work to assess the adequacy of human resources dedicated to these statistics. The mission also observed scope to further exploit the use of electronic technologies and improve the automation of data compilation.

18. Concerning the national accounts (NA) and price statistics, the multisector mission found a possible over estimation of Gross Domestic Product (GDP) in current prices due to an underestimate of the consumer price index (CPI) during 1979–90, especially during years of hyperinflation; encouraged the compilation of an urban CPI extending the Lima CPI to other cities; recognized the lack of adequate source data, which hampered NA compilation; and found a particular need to update the NA base year from 1979.

19. Concerning government finance statistics (GFS) the multisector mission found the following shortcomings: incomplete institutional coverage and inadequate sectoring of the statistics, fragmented data sources, weak reporting by government units to the MEF, poor accounting and administrative procedures, and many exceptions to budget regulations. The absence of a unit at the MEF responsible for compiling and disseminating GFS was identified as a significant limitation.

20. Concerning monetary and financial statistics (MFS) the multisector mission found the following shortcomings: incorrect methodological treatment of certain items, inadequate sectoring of the nonfinancial public sector and inconsistency with the sectoring for GFS, a complex and rigid system to compile the statistics, the need for more detailed breakdowns for the BCRP, and for the Superintendency of Banking Insurance (SBS) to apply legal sanctions to institutions that do not report in a timely fashion.

21. Concerning the external sector (BOP) statistics, the multisector mission observed that the measurement of international reserves needed to distinguish between transactions and other economic flows; and recognized that the authorities were taking actions to reduce the size of revisions to the current account, as well as to improve data collection on services and private external debt. On the latter, the mission recommended that new surveys also cover financial services, interest payments, private external assets, and foreign direct investment. It further recommended that additional human resources be dedicated to improve the number of survey responses. In addition, the mission called for the registration in the balance of payments of transactions by financial institutions with nonresidents, even if they did not affect international reserves. Public external debt data were of high quality, but transactions needed to be sectorized by type of debtor. At the same time the mission observed that private external debt (as well as domestic debt statistics) needed strengthening.

B. The 2003 Data ROSC

22. The 2003 Data ROSC report provided an in-depth assessment of the quality of the national accounts, consumer price index, wholesale price index, government finance, monetary, and balance of payments statistics, as well as an assessment of the data compiling agencies, the BCRP and the INEI. The mission noted that there was room to improve the scope, classification, sectorization, and basis for recording in all data sets. The mission also found that Peru had a broadly adequate legal and institutional framework to support quality official statistics, and that the institutions demonstrated professionalism and provided guidelines on ethical conduct to their staff. However, the report also observed that coordination mechanisms among the producers of official statistics were weak and led to some duplication of efforts. In general, it also noted low response rates to surveys, particularly for NA source data (see below). It confirmed that the data needs of users of official statistics were not monitored regularly (except for the CPI). On the other hand, all of these datasets were accessible by the public. However, in this context, the mission recommended that data on NA and prices be released simultaneously to all interested parties, and the presentation and dissemination of metadata be improved.

23. While the 2003 Data ROSC exercise found that, to a large extent, Peru’s macroeconomic statistics and statistical database were adequate for effective surveillance, important shortcomings in national accounts and prices statistics had the potential to detract from the accurate and timely analysis of economic and financial developments and the formulation of appropriate policies.

24. Concerning the national accounts these important shortcomings included weaknesses in: data sources (including low response rates to data collection surveys), as well as statistical techniques and assessment and validation of source and intermediate data. In addition, the mission found inconsistencies within the NA data set. The mission recommended the implementation of a new benchmark and base year for the NA, and the adoption of the methodology used to compile annual NA to compile quarterly NA and to reconcile these quarterly and annual data. It recommended further implementation of the System of National Accounts 1993 (1993 SNA) methodology (the complete set of institutional accounts) and of double deflation techniques, to the extent possible. It also recommended some additional surveys, improved survey design and execution, as well as techniques to raise response rates.

25. Concerning price statistics, the mission found that the CPI suffered important shortcomings in data sources. The mission recommended to reintroduce the weight for imputed rent for owner-occupied housing into the Lima index; an update to the classification system; and a fixed calendar for weight revisions. On the wholesale price index (WPI) the Data ROSC observed that internationally accepted statistical standards were not followed. The mission recommended that coverage of the WPI be expanded to include mining, oil and gas extraction, electricity and water, and public transportation and communication. It also suggested that improved data be sought from the National Tax Superintendency (SUNAT), and that a fixed calendar of weights be established. The mission observed that a producer price index (PPI) was not being compiled and recommended that a subset of WPI related data be used to pilot a PPI with a view to replacing the WPI at some time in the future.

26. Concerning GFS the mission found that the overall quality of the statistics was good, but certain shortcomings needed to be addressed: the responsibility for compiling these statistics was not clearly specified; GFS produced by the BCRP and the MEF were not reconciled on a regular basis; the tables disseminated did not provide the full recommended breakdown of revenues, expenses, financing, and debt; feedback from users was not sought on a regular basis; available metadata was limited; and there was no migration plan to adopt the Government Finance Statistics Manual 2001 (GFSM 2001) methodology. Thus, the mission recommended that: a single agency be assigned the responsibility for compiling and disseminating GFS; BCRP and MEF/Accountant General’s Office data be reconciled; more detailed data be published for all levels of government; debt tables by subsectors of the public sector be published by type of sector of debt holder and instrument; and the authorities develop a migration path to the GFSM 2001.

27. Concerning MFS the mission found that the overall quality of the statistics was good, but certain shortcomings needed to be addressed: the lack of formal procedures for consultation between the BCRP and the SBS led to duplication of effort in the validation of other depository corporations (ODC) data; the scope of the depository corporations survey (DCS) was limited; the classification of financial instruments and sectoring of institutional units did not always follow international guidelines; some financial instruments were valued at historical costs; accrued interest was not classified within the corresponding instrument; there were no formal procedures for consulting users; and metadata needed to be regularly updated. Thus, the mission recommended that: all depository corporations be included in the DCS; the sectoral breakdown of depository corporations and financial surveys conform to the Monetary and Financial Statistics Manual 2000 (MFSM 2000) methodology; coordination between the BCRP and the SBS be improved to avoid duplication; and the authorities develop a plan for the full adoption of the MFSM 2000.

28. Concerning BOP statistics the mission found that the overall quality of the statistics was good, but there were important shortcomings in data sources. The mission recommended that the authorities: improve the scope of data sources for compiling financial flows of individual resident; compile the Data Template on International Reserves and Foreign Currency Liquidity; and compile data on public and private external debt on an accrual basis.

C. TA Evaluation Mission Findings on Single-Topic Missions

29. On NA statistics, STA provided nine short term missions since 1993, but progress has been slow at times, in part due to Management and staff turn-over, and the time needed to develop source data as well as to implement changes to the massive amount of data being processed. Nevertheless, the most important TA recommendations have been incorporated as part of large projects to change the base year. Following the 1993 multisector mission, the work on the 1994 base year was published in 2000, and following the 2003 Data ROSC exercise the results of the most recent effort to update the base year to 2007 are scheduled to be published in early 2012.6 In both cases the improvements in the NA were based in large part on STA recommendations. In all cases, and consistent with TA findings and recommendations, improvements to the NA depend to a great extent on the availability and quality of source data.

30. On the CPI, STA provided two TA missions (in 1994 and 2011), as well as recommendations as part of the 2003 Data ROSC mission. Many of the TA recommendations were implemented (e.g. updating the classifications), although some recommendations (such as incorporation of imputed rent for owner-occupied dwellings) have not been pursued. A national CPI covering 26 urban areas, including Lima, was scheduled for dissemination in February 2012, beginning with data from January.7 In addition, the authorities have initiated work on a PPI (which may need TA support in the future), while work on previous recommendations made through the multisector and Data ROSC missions to expand the coverage of the wholesale price index has not been undertaken (partially due to a lack of source data).

31. On the GFS, STA has fielded two TA missions to the BCRP in 1993 and 1994, and two missions to the MEF in 2006 and 2008. The recommendations of the earlier missions to the BCRP have been largely implemented and resulted in good quality annual and sub-annual fiscal statistics using the Government Finance Statistics Manual 1986 (GFSM 1986). The recommendations of the missions to the MEF have been partially implemented, including the introduction of new budgetary classifications in 2009 that are better aligned with the classifications of the GFSM 2001. However, in August 2009 the MEF discontinued the dissemination of monthly fiscal statistics; although they are still compiled, they are only for internal MEF use. The BCRP disseminates annual and sub-annual GFS in close coordination with the MEF based on the same source data. At present there is no decision to develop a migration plan to adopt the GFSM 2001 methodology.

32. On MFS, STA has fielded two TA missions to the BCRP in 2007 and 2008. These two missions were not viewed by the authorities as necessary to address compilation or dissemination issues of concern to Peruvian users per se, but were perceived to be related more to the adoption by Peru of STA’s standardized reporting forms (SRFs) for reporting monetary and financial data to the IMF. To the extent methodological recommendations were provided, these have been largely implemented, and the monetary and financial statistics currently compiled and disseminated are of good quality. Peru now regularly reports MFS to the Fund using SRFs covering the central bank survey.

33. On the BOP statistics, TA in 1997 assisted the authorities to improve the proxy methods used to estimate remittances; however, over time the authorities developed stronger tools (including surveys) to further enhance the quality of statistics on remittances. The authorities stated that training courses and remote TA support for specific methodological issues have been the best way to fulfill their needs on external sector statistics.

34. With regard to training, the mission found that the authorities made good use of the staff who participated in various courses on statistics organized by the IMF. Key staff in the BCRP and the INEI that work directly on the compilation and dissemination of macroeconomic statistics have benefitted from this training. In particular, following years of training, the BCRP counts on a significant cadre of staff that are well versed in GFS, MFS, and the BOP statistics. However, some trained staff in those institutions are no longer working in those statistical areas, in part due to competing priorities that led to staff reallocations. In the case of the MEF, for example, there are no staff currently working on the compilation of GFS.

D. Results

35. During 1993-2012 the Peruvian authorities have made progress to strengthen their macroeconomic statistics (Table 1). Most progress was achieved with regard to the application of more recent internationally accepted statistical methodologies. However, the authorities also began to compile and disseminate important new sets of external sector data, such as international investment position data, and the reserve assets and foreign currency liquidity data in the IMF’s Reserves Template. Other notable improvements during the period have included the dissemination of large amounts of data via electronic means rather than exclusively by hard copy, and the subscription of Peru to the SDDS. Some of these efforts were supported through the provision of IMF TA (directly and remotely), and training.

Table 1.

Peru: Comparison of Macroeconomic Statistics 1993 versus 2012

article image
Source: Central Reserve Bank of Peru, Nota Semanal (Weekly Report), No. 2, January 1993 and No. 23, June 2012.

36. In summary, the main results that have been achieved in recent years, with the support of the Fund’s TA and training, include the following:

  • New base years for the national accounts. During 1993–2000, NA statistics were updated to a 1994 base year, and more recently work has been on-going to update the accounts to a 2007 base year. The latter set of national accounts statistics strengthened the compilation of financial services and has followed the guidelines of the 1993 SNA as well as some elements of the 2008 SNA. The 2007 base year NA statistics are scheduled for dissemination as soon as the revision of manufacturing statistics is completed;

  • Compilation of a new national CPI with updated classifications and monthly data beginning in January 2012, scheduled for dissemination in February 2012;

  • Provision of quarterly and annual GFS data, as well as an institutional classification table, for general government to the Fund using the GFSM 2001 framework by the BCRP;

  • Compilation of MFS following the MFSM 2000 guidelines, and initiation of the reporting to STA using the Fund’s SRFs for the central bank, ODCs, and the other financial corporations survey;

  • Strengthened data sources and compilation methods for remittances in the balance of payments;

  • Taken together, application of the 1993 SNA, the GFSM 2001, the MFSM 2000, and the Balance of Payments Manual, fifth edition have strengthened inter-sectoral consistency and the international comparability of Peruvian macroeconomic statistics;

  • The TA also helped foster inter-institutional collaboration, especially across data producing agencies, as well as between those agencies and the international community. The improved coordination and collaboration among data producing agencies has been facilitated through the establishment of technical working groups (“comisiones”) and formal agreements (“convenios de colaboración”).

  • Peru subscribed to the SDDS in 1996. Subscription involved the development and dissemination of Advance Release Calendars (http://dsbb.imf.org/Pages/SDDS/ARCCtyCtgList.aspx?ctycode=PER) and dissemination of data according to that calendar on Peru’s National Summary Data Page (http://www.bcrp.gob.pe/docs/Estadisticas/NEDD/Data.htm). While these achievements could not be directly attributed to TA, TA did contribute to underlying data sets required under the SDDS.

E. Summary of Findings

37. In short, the TA has been relevant, in most cases, responding to demand expressed by the Peruvian authorities. However, as explained above, in the case of MFS TA, the support was less relevant to immediate domestic needs. Nevertheless, the interventions remained focused on STA’s comparative advantages and core expertise.

38. In addition, STA TA was reasonably effective, especially in the areas of GFS and MFS. For the NA and prices, longer gestation periods were needed to implement the recommendations, and were at times dependent on the development of adequate source data, and at other times, on sufficient institutional support, including the agency’s top management.

39. The TA interventions were efficient. In large part this was due to the focus on STA’s core areas of technical expertise, allowing donors to channel their attention to other, complimentary areas, such as data source development, information technology, or other areas of statistics, such as social development indicators.

40. The TA had strong and positive impacts in most cases, as demonstrated by the results obtained by the authorities with this support. Many constraints to capacity were overcome through TA and training. Examples of results included the implementation of changes in national accounts base years; compilation of a national CPI; implementation of updated methodologies for GFS and MFS; and stronger inter-institutional collaboration. These impacts also demonstrate the long-term nature of the interventions, as well as the sustainability of those impacts.

IV. Views on Ta Effectiveness, Efficiency, and Impact and Sustainability

A. Official Data-Producing Agencies in Peru

41. The discussions with the main data-producing agencies, as well as the responses to the questionnaires sent prior to the mission, indicated that most of STA’s TA has been effective across statistical areas. All three main data-producing agencies (BCRP, INEI, and MEF) indicated that the TA: (1) was in accordance with the authorities’ needs, though to a lesser extent in the area of MFS; (2) provided effective hands-on training in the areas being addressed; and (3) contributed to the dissemination of more and better quality statistics. The data-producing agencies were particularly appreciative of the international perspective and knowledge of international standards and methods that TA and training brought to share with the Peruvian authorities. They also valued the opportunities that TA missions provided to develop and enhance inter-agency communication and cooperation, which is a particular challenge in the case of Peru. Moreover, these agencies were grateful for the feedback that TA missions encouraged, both inside and outside each agency.

42. On modalities for delivering the TA, the compiling and disseminating agencies diverged in their preference for TA. The INEI, which produces real sector statistics, preferred and continues to prefer short-term TA and training. The BCRP, which compiles and disseminates all the other macroeconomic statistics for Peru, considered that during the period under review it had essentially exhausted its needs for short-term TA. In the foreseeable future it suggested that preference should be given to direct consultations with Fund staff via remote means (TA through e-mail) and to training courses. The MEF has received short-term TA on GFS in the past and may explore such support again in the future. In either case, the MEF plans to continue to benefit from IMF training courses on GFS.

B. National Users of Statistics

43. The responses to the questionnaires distributed by the mission showed that users have observed a clear improvement in the quantity and quality of data compiled and disseminated over the period covered by this review, especially during the most recent 5 years, partly as a reflection of the TA provided by STA. In general, the questionnaires highlighted the importance and effectiveness of: (1) TA to facilitate coordination among national agencies and interactions with donors; and (2) training of government officials to improve the quality of statistics (Appendix V).

44. These perspectives were confirmed by the mission during interviews with users. The interviewees further expressed their: (1) concern about the limited availability of detailed data and data quality, (though mainly for specific sectors: manufacturing, agricultural labor); (2) desire for greater transparency on INEI’s statistics; but also (3) particular satisfaction with the BCRP’s website and the availability of time series. Interviews with users also exposed a need for further outreach by official compilers to the user community to educate them about the data, including data innovations, characteristics and breakdowns, as well as accessibility.

C. WHD

45. WHD indicated to the mission that STA’s TA proved useful for improving Peruvian macroeconomic statistics and that data provision to the Fund for surveillance purposes has improved in terms of quality and availability. The current WHD team has noted that over recent years adequate fiscal, monetary and external sector data have been provided on a regular and timely basis. Although WHD noted in the December 2011 Staff Report that Peru’s data provision to the Fund is broadly adequate for surveillance, it also indicated that there is scope for improvement. Areas identified included: (i) better coordination among the official statistics-producing agencies; (ii) implementation a new benchmark and base year for GDP; (iii) expansion of the coverage of the wholesale price index (WPI) to include mining, oil and gas extraction, electricity and water, public transportation, communication; (iv) finalization of the migration to the standardized report forms for monetary data; and (v) expansion of the scope for compiling financial flows of individual residents.

46. In this regard, the mission observed the need for further strengthening collaboration among data-producing agencies, although progress has been achieved over time through technical working groups and formal agreements. In addition, as noted above, the recent national accounts work has been designed to address the introduction of new benchmark and base year for GDP. At the time of the mission dissemination of these data was expected soon (early 2012). However, more work on the financial flows of residents has been recognized as an important area for future national accounts development. Source data has limited progress to expand the WPI; nevertheless, work to develop a more comprehensive PPI may make more sense and be in line with international standards. Finally, work on compiling and reporting of the SRFs for MFS is making progress. Form 1SR (central bank) was recently reported to STA, and form 2SR (ODCs) was targeted to be submitted possibly as early as the first half of 2012.

V. Lessons Synthesized from the Experience in Peru

47. A primary motive for this report is to help the Fund’s Statistics Department improve its TA and training delivery. With this in mind, the mission synthesized the following lessons from the TA and training experiences with Peru during 1993–2011.

A. Overall TA and Training to Peru was Effective during 1993–2011

48. The mission found that, in general, there was consensus among data compilers and users that the IMF TA and training interventions contributed to improved macroeconomic statistics in Peru over the course of the last 20 years (section IV). The TA led to increased quantity and improved quality of macroeconomic statistics, including especially increased transparency about concepts and methods, as well as improved dissemination of the statistics (a list of results is shown in section III.B. paragraph 35 of this report). The model of having diagnostic or assessment missions, such as a Multi-sector mission or a Data ROSC (which assesses various aspects of data quality) provided a useful starting point to develop a work program that was pursued through a series of single-topic TA missions and supported by training activities, so that macroeconomic statistics could improve over time.

49. The TA was effective in Peru, in part because of the strong capacity built in the country (paragraphs 34 and 39). The quality of the counterparts allowed the TA providers to pass on information and exchange ideas effectively and efficiently. When domestic capacity was constrained, these tasks became more challenging.

50. The effectiveness of TA was reasonably consistent across all statistical areas, although progress in the national accounts frequently took several years and results depended in large part on the availability of source data and the steady leadership in the agencies involved (paragraph 29). When either of these factors was lacking, the results that could have been achieved through TA were more limited. Addressing these obstacles to achieve better or faster results rests mainly in the hands of the authorities, at times in partnership with the donor community. Nevertheless, the TA did not set prior conditions for the provision of follow-up TA based on progress regarding these challenges. On the other hand, TA missions appeared sufficiently spread over time to allow for progress implementing recommendations between visits, including on strengthening data sources. Making certain critical prerequisites are met prior to follow-up TA has potential merit for future interventions.

B. TA Helps Enhance Data Quality

51. As shown in the case of Peru, TA helps enhance data quality. In particular, the adoption of internationally accepted statistical methodologies contributes to improved international data comparability, as well as inter-sectoral consistency across the national data set (paragraph 35, sixth bullet). The TA mission also provides an opportunity for the Fund to provide the authorities with an assessment of the work that has been done to improve the statistics. Using the Fund’s judgment provides the authorities with the feedback they need to confirm prior work, enhances credibility of the statistics with users, and helps to provide direction for additional improvements.

C. Management and Staff Turn-over Can Present Obstacles to Effective TA

52. Management and staff turnover in the key compiling agencies limits (and can even reverse) progress with macroeconomic statistics. While TA and training can attempt to institutionalize improvements (through standardized: (1) compilation methods following internationally accepted statistical standards; (2) reports; and (3) comprehensive and documented metadata), more could be done to focus on management’s understanding and buy-in of the TA recommendations and follow-up actions. Training a critical mass of compilers would also contribute to mitigating these risks. (See paragraphs 29, 31, and 34.)

53. For example, one way to institutionally strengthen the INEI would be to consider enhancing its governance and management structure through the establishment of an Executive Board to oversee the agency’s strategic direction and long-term planning. Frequent or political changes in Directors have adversely affected the performance of the INEI, and thereby hindered improvement in real sector statistics. The consequences of these changes, along with less than competitive remuneration and benefits to attract and retain well qualified and trained staff, and insufficient financial resources needed to conduct source data censuses and surveys on a programmed and regular basis, has meant that the quality of those statistics could only be improved slowly over time.

D. Communications Issues Help Determine the Effectiveness and Relevance of TA

54. Several aspects of communication issues can help determine the effectiveness of TA. These issues pertain to communication between the Fund (both the area department and STA) and the authorities. Specifically, the experience of Peru underscores the importance of clarifying TA objectives in advance. Also, communication within and across statistics producing agencies can be fomented with effective TA. The TA mission itself provides opportunities for such communication, as well as catalytic impulses to institutionalize these important communication channels.

Closer Communication between the Fund and the Authorities Would Foment More Effective TA

55. The experience in Peru highlights the value of close collaboration with area departments on the key TA recommendations and statistical results as essential to support stronger evidence-based policy consultations. When this is present, results can be achieved efficiently, as with the recent adoption of a GFSM 2001 presentation in fiscal data or compilation of a national CPI (paragraphs 30 and 31). However, when communication falters, as in the MFS TA mission objectives, clarity on objectives can be lost (paragraph 32).

Communicating Clarity on TA Objectives is Essential

56. Another general lesson from the experience in Peru concerns the setting of clear objectives for the TA. In Peru, TA sometimes was not purely demand driven. In the area of MFS, the need and usefulness to report MFS to the Fund through SRFs may not have been fully understood by the authorities prior to the missions (paragraph 32). Improved outreach prior to and after TA missions where STA is pursuing international objectives could give the authorities a better understanding of the missions. For example, the MFS TA missions could have conveyed to the authorities ahead of time the value to the Fund’s area department for timely and comprehensive MFS, which facilitates inter-sectoral consistency, financial programming and policy discussions. In general, STA may need to educate TA recipients well in advance about new developments in methodology or dissemination standards as part of the support that could be provided by the TA. Ideally, recipients would be fully informed and should welcome these aspects of the TA, prior to a mission.

Effective TA Can Facilitate Communication Within and Across Agencies

57. TA can be catalytic to allow users’ (and even compilers’) views to statistical managers and resource decision-makers to help set priorities and to understand their needs (paragraph 40). Views can be expressed to TA providers in an independent and non-threatening atmosphere, which allows the frank collection and express of these opinions. At the same time, TA providers normally have sufficient access to decision-makers to provide this feedback, place the feedback into a broader context (including international comparisons and benchmarks) and give the decision makers a window into the perspectives of their subordinates and user community.

58. The mission found that TA also provides opportunities for strengthening inter-agency cooperation. The case of Peru demonstrated these impacts, which were institutionalized through inter-agency working groups and formal agreements (paragraph 35, seventh bullet). While the willingness to exploit these opportunities may vary from one context to the next, collaboration and interaction at the time of the TA mission usually increases. The issue is whether the authorities choose to pursue more institutionalized forms to make such interactions more frequent and regular, as in Peru.

E. Flexibility in the Modalities of TA and Training Delivery Contributes to Efficiency

59. On modalities, the application of a mix of TA and training interventions was applied appropriately in Peru, depending on the needs of the authorities (paragraph 41). Multisector and Data ROSC missions were followed by short-term TA missions, which provided the bulk of assistance. However, these were also complemented by training of officials through headquarters, regional and local courses. Remote TA also has been efficient; mainly in areas where sufficient technical capacity already existed in the country and where the TA was directed to specific issues identified by the authorities.

60. Looking ahead, STA’s modalities to deliver TA may need to be adjusted depending on the statistical area involved and the evolving needs of the authorities (see also Section VII). Improvements in communications technology and increased capacity of member countries to absorb TA, means that it is likely that remote TA will play an increasing role in the future in TA delivery. Close consultations between the authorities and Fund staff would facilitate STA’s ability to respond appropriately to future TA and training needs. Moreover, there may be scope to enhance TA delivery through remote means (video conferences, real time chats, etc.). While not a perfect substitute for a field visit, which allows for more extensive face-to-face exchanges and tends to focus attention (and eliminate or minimize office distractions), remote interactions can provide substantial pre-mission exchanges of information and can be useful to help support follow-up actions.

F. Thus, There Would Be Benefits from a Formal Results-Based Management Approach

61. In general, the application of a full results-based management (RBM) framework would provide benefits for TA delivery and evaluation (paragraph 9). An RBM framework contains objectives, outcomes, indicators, milestones, and risk assessments and assumptions that clearly identify targeted results, with specific, measurable, attainable, relevant, and time-bounded (SMART) characteristics.8 While the information available through briefing papers, TA reports and notes, back-to-office reports, and the administrative system contain useful information, the current framework does not explicitly contain all the elements of a full RBM framework in a consistent manner.

62. Despite the lack of availability of SMART verifiable indicators and milestones, the mission was able to determine, in a broad sense, how successful the TA had been and what results had been achieved. Importantly, the mission was also able to gather opinions of relevant stakeholders on their perceptions of the relative success or otherwise of the TA and training.

63. A stronger TA framework that focused on results directly may have facilitated the tracking of achievements. It also would have clearly identified the assumptions and risks that could potentially delay or derail the attainment of those targeted results. While such a framework would not overcome such constraints, it would allow for explicit recognition of those risks and possible identify risk mitigation efforts that could accompany the TA.9

RBM Benefits for TA Delivery

64. Some of the benefits of the full IMF RBM framework on the delivery of TA include the following.

  • Clear ex-ante identification of objectives (with associated SMART verifiable indicators, assumptions, risks and risk mitigation efforts);

  • Focus on results through the identification of (up to four) key outcomes, along with associated SMART verifiable indicators (with baselines) that can help measure progress toward achieving those results and SMART milestones that represent mid-way measurement points regarding those indicators, (again along with assumptions, risks and risk mitigation efforts). Ideally all of this would be documented in a Logical Framework (see Appendix VI for the IMF log frame) and linked closely with briefing papers, Terms of Reference, TA reports or TA notes, back-to-office reports and other periodic documents related to the TA project.

  • Linking these elements to resource inputs and outputs of goods and services delivered by the TA intervention; and

  • Providing a mechanism to consistently pursue TA recommendations over time by identifying and documenting progress against those SMART milestones and verifiable indicators.

RBM Benefits for TA Evaluation

65. The evaluation of TA in Peru would have benefited if the Fund had adopted a results-based approach. A stronger evaluation could have been made if RBM-styled SMART verifiable indicators were associated with baselines, which could be compared following the implementation of the technical assistance. For TA evaluations, full implementation of an RBM framework ex-ante would assist evaluators to summarize results and help focus on the results that are based on clear objectives, as well as measuring progress through clear indicators with baselines against which progress could be objectively measured.10 The absence of such a full framework, which will continue in the short run, means that evaluators must rely on their own ex-post compilation of objectives, outcomes, verifiable indicators and baseline measurements, milestones, risks and assumptions. This makes the evaluation inherently more subjective.

VI. Conclusions and Recommendations

66. This section of the report attempts to summarize the main recommendations that could be distilled from the evaluation of TA and training to Peru during 1993–2011. Most of the recommendations pertain to STA delivery of TA, although in a few instances, the report highlights some recommendations that might be useful specifically for future TA support from STA specifically in the case of Peru. The main recommendations are as follows.

67. On effectiveness, TA and training should:

  • Whenever practical first conduct a diagnostic mission and then develop a work program to provide TA and training support to meet the needs identified;

  • Promote the institutionalization of standardized: (1) compilation methods following internationally accepted statistical standards; (2) reports; and (3) comprehensive and documented metadata;

  • Set SMART prior conditions (milestones/benchmarks) before providing follow-up TA, but remain flexible to country needs and circumstances to respond appropriately;

  • Provide fair and frank feedback on statistical improvements to guide further progress and lend credibility of the statistics to users;

  • Serve as an independent conduit to communicate users’ (and compilers’) views to statistical managers and resource decision–makers to help set priorities and to understand their needs; and

  • Provide opportunities for the recipient to strengthen inter-agency cooperation.

68. On relevance, TA and training should:

  • Liaise closely with the management of data-producing agencies to enhance their understanding and promote their buy-in of the TA recommendations and follow-up actions;

  • Clarify and agree on TA objectives well in advance to make sure they are either demand-driven or at least well understood and supported by the authorities (as part of STA awareness campaigns about new developments in statistical methodology or dissemination standards).

69. On efficiency, TA and training should: emphasize open communication with others, including the donor community, to promote the most efficient delivery of TA and training. In addition, the modality of delivery matters. STA should continue to provide TA and training using a variety of modalities in a flexible manner to suit the recipient’s needs.

70. On impact and sustainability, TA and training should:

  • Aim to build strong capacity in the recipient country, including a critical mass of compilers, to facilitate future TA absorption capacity and promote the sustainability of the results achieved from previous interventions;

  • Be founded on a Result-Based Management framework, in part because it would strengthen TA delivery through the formal and structured identification of objectives, outcomes, SMART verifiable indicators, milestones and related assumptions and risks, with a central focus on results. In addition an RBM framework requires documenting this information over time, which would also strengthen subsequent evaluations.

VII. Possible TA and Training for Peru in the Future

71. The Peruvian authorities expressed interest in receiving TA and training. For real sector statistics, the INEI would like to explore short-term TA support regarding:

  • Presentation of the new 2007 base year of the national accounts;

  • Compilation of quarterly national accounts using the 2007 base year;

  • Compilation of the financial accounts for all sectors using the 2007 base year;

  • Compilation of the monthly manufacturing production index using a 2007 base year; and

  • Compilation of an industrial manufacturing producers’ price index.

In addition, several line ministries (inter alia the Ministries of Agriculture and Production) expressed interest to attend courses taught by the Fund on national accounts statistics, to train staff that supply source data for national accounts compilation. For GFS, the MEF intends to explore options for receiving TA and training on GFS, especially to focus on the compilation of data using the GFSM 2001. The mission recommended that, in the meantime, they consult with the BCRP on work done to date to convert national budget execution data to the GFSM 2001 format. For MFS and external sector statistics (as well as for GFS), the BCRP will rely on regular training courses and remote TA. No short-term TA mission requests are anticipated.

72. In general, the authorities recognized the need to further strengthen inter-institutional collaboration beyond existing mechanisms, and welcomed the Fund’s intervention to bring the three main macroeconomic data producing agencies together to reinforce these bonds. They also recognized the need to promote among users a better understanding of what statistics are available, and how users can avail themselves of these data. While Fund TA can serve as a catalyst for such efforts, no amount of TA or training could substitute for direct action by these three agencies.

VIII. Data Reporting to STA

73. The mission also discussed with the authorities the issues regarding data reporting to STA, focusing mainly on regular reporting via the Integrated Correspondence System. Data issues and actions were discussed concerning annual and high frequency GFS, the BOP and the IIP, exchange rates and international liquidity, and direction of trade data. Contact information was updated or confirmed. The mission stressed the advisability of reporting preliminary data followed by revisions, instead of delaying reporting until final data became available. The authorities agreed to address all of the issues raised and were already in contact with STA information management staff to begin to resolve any outstanding issues before the mission left the field.

Peru: Report on the Statistics Technical Assistance and Training Evaluation Mission (December 12 - 16, 2011)
Author: International Monetary Fund