Bhutan
Technical Assistance Evaluation Report (March 4-8, 2013)

This paper discusses Bhutan’s Technical Assistance Evaluation report. Participation in the statistical projects financed by the Japanese government in national accounts, prices, government finance, and balance-of-payments statistics is having a major effect on Bhutan’s statistical development. The statistical compilers were well informed about the donor as the source of funding. The method of combining regional workshops and multiple missions was considered to be particularly suitable. Cross-sectoral data consistency and coordination are increasingly recognized as important. Computerized accounting systems for the budgetary central government have been introduced, namely the Multi-Year Rolling Budget and the Public Expenditure Management System.

Abstract

This paper discusses Bhutan’s Technical Assistance Evaluation report. Participation in the statistical projects financed by the Japanese government in national accounts, prices, government finance, and balance-of-payments statistics is having a major effect on Bhutan’s statistical development. The statistical compilers were well informed about the donor as the source of funding. The method of combining regional workshops and multiple missions was considered to be particularly suitable. Cross-sectoral data consistency and coordination are increasingly recognized as important. Computerized accounting systems for the budgetary central government have been introduced, namely the Multi-Year Rolling Budget and the Public Expenditure Management System.

I. Introduction

1. As part of the IMF Statistics Department (STA) technical assistance (TA) program and in collaboration with the IMF Asia and Pacific Department (APD), a mission visited Thimphu, Bhutan, during March 4–8, 2013. The purpose of the mission was to evaluate the results that have been achieved in Bhutan as a consequence of the TA and training in the compilation and dissemination of macroeconomic statistics during the past 12 years. The main tasks were to: (i) examine the outcomes of previous TA missions; (ii) elicit the authorities’ views on the effectiveness of TA and training provided by STA and how it could be improved; (iii) identify the main challenges in the implementation of TA recommendations; (iv) ascertain the scope for enhancing the role of the authorities in the coordination of the TA delivered by the international community; (v) meet with users of macroeconomic statistics, as well as with representatives of TA donors; and (vi) discuss priorities for the future. The mission also discussed outstanding issues related to data reporting to the Fund.

2. To address these tasks, the mission worked closely with officials from the main data compiling and disseminating agencies, particularly the Research and Statistics Department of the Royal Monetary Authority (RMA), the National Accounts and Price Division of the National Statistical Bureau (NSB), and the Department of Public Accounts of the Ministry of Finance (MoF). To gain input from compilers and data users, the mission distributed a questionnaire and collected responses (see Appendices IV and V).

3. Section II of the report provides background to the evaluation process. Section III outlines the results that have been achieved with the support of TA and training provided by STA to Bhutan in macroeconomic statistics covered by the mission. Section IV outlines the views of the authorities and other users on TA and training based on the questionnaires and interviews. Section V synthesizes lessons learned from the evaluation, while Section VI focuses on key conclusions and recommendations. Finally, Section VII reviews data reporting to the Fund.

II. Background

A. Evaluation of STA TA and Training

4. The evaluation of STA’s TA and training takes into account the Organization for Economic Cooperation and Development’s Development Assistance Committee Principles for Evaluation of Development Assistance.1 These principles state:

  • • An evaluation is an assessment, as systematic and objective as possible, of an ongoing or completed project, program or policy, its design, implementation and results. The aim is to determine the relevance and fulfillment of objectives, developmental efficiency, effectiveness, impact and sustainability. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision-making process of both recipients and donors. (Paragraph 5).

  • • An important purpose of evaluation is to bring to the attention of policymakers constraints on developmental aid success resulting from policy shortcomings or rigidities both on the donor and recipient side, inadequate coordination, and the effectiveness of other practices… (Paragraph 9)

  • • Evaluations … need to look at agency procedures and management issues. (Paragraph 29)

B. Background to the Evaluation of TA and Training to Bhutan

5. The TA evaluation mission to Bhutan follows on the 2005 Independent Evaluation Office (IEO) recommendations for developing more systematic approaches to track progress on major TA activities and to identify factors behind successes and shortfalls.2 The response of the IMF to the IEO recommendations pointed to the need to systematically draw out the lessons learned from the evaluation with a view to enhancing TA policy. These lessons should help provide a more informed background for future collaboration. The Fund is moving in this direction to strengthen its evaluation. This report seeks to focus on outcomes and to apply results-based management techniques as far as possible.

6. To conduct the evaluation, the mission: (i) reviewed the amount, form, and topics of TA and training provided by STA since 20003 and reviewed the available TA documents prior to the mission (including briefing papers, TA reports, and back-to-office reports); (ii) summarized the findings of those missions and observed the results achieved by the Bhutanese authorities with the support of this TA and training; (iii) elicited the authorities’ views on the effectiveness of STA TA and training, and discussed the status of implementation of major recommendations made by STA, as well as the modes of TA and training delivery they consider most appropriate; (iv) elicited views from national users of statistics, including interviews with TA donors and other stakeholders; and (v) consulted with APD and used APD documents (especially the Statistical Issues Appendix to the latest Article IV staff report, 2011) to collect APD views on the usefulness and effectiveness of STA TA and training for the Bhutanese macroeconomic statistics system.

7. The mission sent questionnaires to the main data-compiling and using agencies. The questionnaires sought views the usefulness of, and main lessons learned from, STA TA and training, as well as future TA and training needs.

8. The timeframe for conducting evaluation presents difficulties. A TA evaluation cannot be thoroughly accomplished before a project has been completed. At the same time, evaluation of a TA project years after it has been completed suffers from the passage of time, in part because key officials may have changed jobs. On the other hand, progress with the development of macroeconomic statistics can be a long-term undertaking and the perspective of time can help review the progress achieved. In the light of these considerations, the mission chose to review a long span of TA and training including very recent missions.

III. TA and Training Provided by STA and Related Results

9. Bhutan has benefited from a wide range of STA TA and training. The 2004 multisector mission served as a benchmark and diagnostic mission. Prior to 2004, there had been no recent STA TA. Subsequently, a series of single-topic missions were held, specifically one on General Data Dissemination System (GDDS), two on national accounts, four on consumer and producer price indexes, one on monetary and financial statistics (MFS), and three on external sector statistics. Beginning in 2011 and 2012, national accounts, prices, government finance statistics (GFS), and external sector statistics have been part of three-year regional projects under the Japan Administered Account for Selected IMF Activities (JSA). These projects have resulted in a substantial increase in TA delivery. No Report on Standards and Codes mission has been conducted for Bhutanese statistics. In addition to TA, Bhutanese officials have had 74 cases of attendance at STA training since 2000, including Headquarters, regional training centers, and other regional hosts. Appendix II lists the TA missions delivered by STA during 2004–2012, while Appendix III shows the details of training provided.

10. This section outlines the findings of the TA evaluation mission by topical area and provides findings based on an assessment of the implementation of the main recommendations from the set of single-topic missions that followed the 2004 multisector mission. For each topical area, the section summarizes the main results that were achieved with Fund support and the evaluation mission’s findings. In addition, Table 1 shows the progress in Bhutanese statistics by comparing the periodicity, timeliness, and methodology of the main areas of statistics at the time of the 2004 multisector mission and at the time of the evaluation mission. Table 1 also summarizes other improvements in data quality during the period (right column). In addition, Section VII Table 2 shows how these improvements were reflected in reporting to STA publications.

Table 1.

Bhutan: Comparison of 2004 and 2013 for Periodicity, Timeliness, and Methodology of Macroeconomic Statistics

article image
article image

partly complete;

imminent;

internal use only.

Table 2.

Bhutan: Data in International Financial Statistics

article image

A. National Accounts

11. National accounts are compiled by the NSB. The 2004 multisector mission found that national accounts benchmarks, input-output ratios, coverage adjustment ratios, and base year (1980) were all outdated. The national accounts were also still based on the 1968 System of National Accounts (SNA). At that time, the Asian Development Bank (ADB) was providing TA in national accounts for both data sources and compilation methods. The mission made recommendations in the context of the ADB project for continuing improvement of methods, updating the base year to 2000, moving to the 1993 SNA, and improving staff training, which were dealt with by the ADB.

12. There was no follow-up by STA until Bhutan joined the JSA national accounts project in 2011. Since the ADB was already providing TA in 2004, the lack of IMF follow-up at that time was appropriate as it avoided duplication. The NSB implemented the main recommendations under the ADB project. Additional staff members were employed, from three staff in 2004 to seven at the time of the evaluation mission, making improvement of methods and frequency feasible.

13. Under the JSA project for national accounts, missions were conducted in June–July 2011 and July 2012. The main tasks of these missions were devising sources and methods for a system for calculating quarterly gross domestic product (GDP) by kind of economic activity at constant prices. In addition, recommendations were made for longer-term improvements in the annual national accounts, including an updated base year, incorporating household expenditure survey data, revising estimates for the nonobserved economy, and implementation of the 2008 SNA.

14. The main results that have been achieved in recent years, with the support of the Fund’s TA and training, include the following:

  • • For quarterly national accounts, the mission identified potential data sources, as well as provided training in benchmarking and other compilation techniques.

  • • NSB subsequently developed experimental estimates, including back data for several years. The estimates have not yet been circulated outside NSB. The estimates exposed some shortcomings in the annual estimates and need to be reviewed by the scheduled follow-up mission. The estimates cover only 75 percent of the economy, with potential indicators proposed by the mission for the remaining industries that seemed plausible, but subsequent investigations showed they were not feasible without new surveys.

15. Additional missions have been requested to review the existing quarterly estimates and develop methods to fill the remaining gaps in indicators for quarterly GDP, as well as to support the improvements in the annual national accounts. (The mission was later arranged for April 2013.) The authorities also wish to update the base year for the national accounts (currently 2000) taking into account the results of the Bhutan Living Standards Survey 2012.

16. NSB staff members have had 11 instances of attendance in eight STA national accounts courses since 2000. Seven of these were general national accounts courses, two for quarterly national accounts, and two attended the launch workshop for the JSA project. (Some staff attended more than one course and some courses were attended by more than one staff member).

B. Consumer Prices

17. Price indexes are compiled by the NSB. The 2004 multisector mission found that the Consumer Price Index (CPI) had outdated weights and was published only twice a year, although there was a partial index on a quarterly basis that covered only Thimphu food prices. After that mission, with ADB support, the weights were updated to 2003 based on the Living Standards Survey. There was no further TA from STA until 2011.

18. Bhutan was included in the JSA project for price statistics. Missions were held in April–May 2011, April–May 2012, and February 2013. They recommended improvements in the frequency of the CPI, first to quarterly, and then to monthly. They also recommended updating the base period, expanding item coverage, and improving collection procedures.

19. The main results that have been achieved in recent years, with the support of the Fund’s TA and training, include the following:

  • • The CPI is now published quarterly, and will be published monthly starting with May 2013 data.

  • • The range of goods and services included in the index has been expanded.

  • • The weights have been revised to 2012 based on the results of the Bhutan Living Standards Survey.

  • • Responding to requests from users, the revised CPI was developed to show imported and domestic items separately.

20. With the World Bank providing hardware and an application, and STA providing technical advice, an innovative and highly efficient system was developed. Prices recorded all over the country by mobile device are sent instantaneously to the NSB in Thimphu. This system was so successful that it is now being applied in other countries in the region.

21. The recommendations of the 2004 mission report were found to have had little effect due to the lack of follow-up by STA. However, the series of JSA-funded missions have been highly effective because they provided continuing support for different stages and addressed new issues that arose during the implementation process.

22. NSB staff members have had seven instances of attendance in five STA prices courses since 2000. Five of these instances were for general courses, and two were for the launch workshop for the prices component of the JSA project.

C. Producer Prices

23. There was no Producer Price Index (PPI) at the time of the 2004 multisector mission. The mission recommended that a PPI be developed, but there was no follow-up by STA and no further action was taken by NSB.

24. Missions by STA were conducted under the JSA project in conjunction with CPI missions, as listed above, starting in 2011. The mission recommended procedures for weights, collection procedures, and item selection for the PPI.

25. The main results that have been achieved in recent years, with the support of the Fund’s TA and training, include the following:

  • • A monthly PPI has been introduced. It is published three months at a time, once each quarter. The data were first published for the second quarter of 2012, with back data for several years. The indexes cover logging, mining, manufacturing, electricity, water, transport, and communications.

  • • A construction materials price index is being developed.

26. NSB plans further extension of the range of producer price indicators. In addition to the construction materials price index, already under development, the NSB also proposes to develop price indexes for construction output, exports, and imports. The NSB has requested further TA under the JSA project to support these efforts.

D. Government Finance Statistics

27. The 2004 multisector mission found that there was no official assignment of responsibility for GFS, although the MoF Department of Public Accounts had undertaken the task. It also found that monthly or quarterly data were not available, data did not cover extrabudgetary trust funds or public corporations, and data were not published on the Internet.

28. Recommendations made by the mission to address the problems were largely not implemented. However, quarterly data are now produced and made available internally. None of the staff at the time of the 2004 mission is currently involved in GFS compilation and current staff members are not aware of the mission report. Nevertheless, Bhutan has continued as a consistent reporter of annual data in Government Finance Statistics Manual 2001 (GFSM 2001) format for inclusion in the GFS Yearbook.

29. Computerized accounting systems for the budgetary central government have been introduced, namely the Multi-Year Rolling Budget and the Public Expenditure Management System. The ADB and World Bank provided TA in these tasks. While the 2007 Public Finance Act did not deal with GFS compilation, it is considered by the MoF staff to provide a strong foundation for producing statistics, and draft regulations under the Act now being prepared will authorize MoF to adopt International Public Sector Accounting Standards and GFSM 2001.

30. The MoF has joined the JSA project for GFS. Three staff attended the launch seminar held in Bangkok in 2012, where a draft work plan was developed. Three nominations were made for the follow-up seminar in April 2013. While the MoF was keen to be involved in the project, there was disappointment expressed that there had been no follow-up missions conducted at the time of the evaluation mission.

31. There have been 11 attendees at STA GFS courses since 2000, but only three of those are still involved in GFS compilation. The courses were considered by the MoF to be highly educative and useful.

E. Monetary and Financial Statistics

32. MFS are compiled by the RMA. The 2004 multisector mission found that the institutional coverage of monetary statistics is complete. The format of monetary statistics is fundamentally consistent with the structure and the detail of the Monetary and Financial Statistics Manual (MFSM) sectoral surveys. However, there are a number of issues concerning the source data, specifically related to the use of the residence principle, valuation, and accrual accounting, for which the mission made recommendations.

33. The RMA’s Monthly Statistical Bulletin in 2004 was only publishing comprehensive data on the broad money survey. The mission recommended that the RMA initiate a monthly dissemination of monetary accounts for the RMA and other depository corporations (ODCs).

34. Bhutan has been compiling and submitting monetary data based on the Standardized Report Forms (SRFs) to STA since 2008. During the past several cycles of the International Financial Statistics (IFS), STA identified a number of data problems in the monetary data submitted by the RMA.

35. A MFS mission took place in February 2011. The mission found the following issues based on discussions with RMA staff: (i) for the central bank balance sheet, RMA staff was compiling the IMF report form (SRF-1SR) manually by entering the data from the RMA balance sheet to the SRF; (ii) for the ODCs aggregated balance sheet, semi-automation had been implemented; however, due to the introduction of several new categories in the balance sheet of ODCs, vertical-check errors were found; and (iii) misclassifications of financial instruments and institutional units in the SRFs. The mission together with RMA staff corrected the misclassified financial instruments and institutional units.

36. The 2011 mission also provided hands-on-training on Financial Soundness Indicator (FSI) methodology and assisted the authorities with introducing the FSI templates (data and metadata templates). Based on the availability of data, the mission assisted RMA staff to compile 11 core FSIs and five encouraged FSIs for ODCs, as well as one FSI for Real Estate Markets (residential real estate loans to total loans). The mission assisted the RMA staff with the successful compilation of FSIs from 2009 Q1 to 2011 Q3, which were published on the IMF’s FSI website in 2012.

37. In summary, during the period 2004–2012, the Bhutanese authorities have made noticeable progress towards strengthening their monetary statistics. Much of the progress was in conforming to internationally accepted statistical methodologies, such as the MFSM. In addition to improvements with regard to the technical aspects of compilation, there has been good progress in data dissemination practices and in the increased volume of data released.

38. The main results that have been achieved in recent years, with the support of the Fund’s TA and training, include the following:

  • • Compilation of MFS following MFSM, and initiation of the reporting to STA using the Fund’s SRFs for the central bank and ODCs surveys;

  • • Compilation of FSIs following the FSI guide and initiation of reporting to STA since 2012; and

  • • Dissemination of MFS increased through additional publications including the Weekly Monitor (for RMA management only), the RMA Monthly Statistical Bulletin which is disseminated on the RMA website and contains detailed monetary and financial data, and the annual Monetary Policy Statement, first published in 2011.

F. External Sector Statistics

39. Balance of payments and other external sector statistics are compiled by the RMA. These statements are compiled separately for India as a partner and for the rest of the world, reflecting the particular importance of India for the Bhutanese economy. Since 2002, efforts had been made to compile data following the Balance of Payments Manual, fifth edition (BPM5). Despite these efforts, the balance of payments statistics suffered from large and widely fluctuating errors and omissions, particularly those with India, reflecting mainly coverage and classification problems. The 2004 multisector mission found that the conceptual framework for the compilation of balance of payments statistics needed improvements in several areas. The main areas included (i) following a more rigorous approach in the implementation of the BPM5 concept of residence; (ii) showing separately capital account transactions (capital transfers) from current account; (iii) valuing imports on a free on board basis and showing freight and insurance services separately; and (iv) improving definitions and classifications of balance of payments transactions. The major areas where coverage problems exist and data sources needed to be generated included the following: (i) informal unrecorded imports of goods from India; (ii) imports of various types of business services; and (iii) foreign construction companies involved in long-term construction projects.

40. In addition, the 2004 multisector mission found that the external debt statistics were comprehensive and covered all external government borrowings. However, dissemination of external debt statistics needed improvement. The mission recommended that the MoF and RMA cooperate to publish external debt data on a quarterly basis (outstanding debt, disbursements, and debt service) and the forward-looking debt service schedule.

41. A follow-up mission took place in 2005. The mission devised a strategy to compile and publish balance of payments statistics on a quarterly basis, to develop a business survey including survey questionnaires, to introduce new data models, and to revise some other data models. The mission also established the ground work to collect and publish international investment position (IIP) statistics. The mission visited all major data suppliers to assess the quality of data sources and to discuss the new business survey. Apart from conducting interviews in Thimphu, the mission also traveled to the distant cities of Tala, Phuentsholing, and Paro.

42. The 2005 mission developed a strategy and timetable, with a plan to introduce quarterly balance of payments business surveys (end-April 2005); complete interim revisions to balance of payments statistics based on the mission’s recommendations (June 2005); publish annual 2004–05 statistics; prepare quarterly balance of payments and IIP statistics (February 2006) with the aim of publishing quarterly data with a lag of four to six months; conduct a balance of payments benchmark survey (2006); and publish a balance of payments concepts, sources, and methods publication (March 2007). However, the compilation of quarterly balance of payments and IIP statistics by February 2006 can be viewed today as too optimistic, as quarterly data are still not available.

43. A second follow-up mission took place in 2009. The mission found that renewed effort should be made to improve and expand the coverage of transactions with nonresidents that have been partially covered in the past, such as services (in particular travel, freight, telecommunications, and government services not included elsewhere, among others), workers’ remittances, and direct investment. The mission recommended compiling the IIP with available data on a trial basis (which had not been compiled because of the lack of resources.) The mission also recommended increasing the number of balance of payments compilers from two to at least three.

44. In 2010, for the first time, Bhutan reported balance of payments data suitable for dissemination purposes in the Fund’s IFS and in the Balance of Payments Statistics Yearbook.

45. The last follow-up mission was conducted in 2011. Special emphasis was placed in addressing RMA’s concern regarding net errors and omissions in the balance of payments, which in recent years had been considerably high. In addition, the mission prepared Bhutan’s IIP statistics for June 2009 and June 2010, and also reviewed with the authorities the procedures for compiling the data template on international reserves and foreign currency liquidity. The mission identified and corrected some measurement problems, which significantly lowered the net errors and omissions. A JSA-funded mission was scheduled to be held in May 2013.

46. In summary, impressive results have been achieved in the production of external sector statistics that meet international standards. The authorities are now compiling and disseminating a significantly improved balance of payments statement, in which the level of net errors and omissions came down to reasonable levels. In addition, important new sets of external sector data, such as IIP, international reserves and foreign currency liquidity data, and external debt statistics were compiled and are in the process of being disseminated. Other notable improvements during the period included the increased use of electronic data dissemination and improved timeliness as well as periodicity of data.

47. The main results that have been achieved in recent years, with the support of the Fund’s TA and training, include the following:

  • • Strengthened data sources and compilation methods in the balance of payments, with very significant impact on the detail and accuracy of the disseminated data;

  • • Compilation of the annual IIP following BPM5 and reporting to STA since 2011;

  • • Participation in the IMF’s Coordinated Direct Investment Survey from 2011; and

  • • Assistance in fostering interagency collaboration, especially across data-reporting and data-producing agencies. Visits to these agencies during the TA missions were instrumental in strengthening these interactions and improving the provision of timely source data.

General Data Dissemination System (GDDS)

48. The 2004 multisector mission explained the objectives and operations of the GDDS and assisted the authorities to develop draft metadata. Further action was stalled awaiting commitment at the political level to participate in the GDDS. In February 2010, a follow-up mission was conducted and produced revised metadata, which were posted on the GDDS website when Bhutan began its participation in the GDDS shortly afterward. The GDDS coordinator is in the RMA.

49. The main results that have been achieved in recent years, with the support of the Fund’s TA and training, include the following:

  • • Bhutan began participation in the GDDS in 2010.

50. However, progress in macroeconomic statistics methodology and frequency, as reported elsewhere in this report, means that the metadata are outdated. As a result, the GDDS coordinator would like to conduct outreach to the affected agencies in conjunction with STA to update the metadata and plans for improvement to take into account progress and current priorities.

Intersectoral linkages

51. The 2004 multisector mission had a section dealing with cross-sectoral consistency and single-topic missions have addressed cross-sectoral consistency issues to varying degrees. In February 2013, STA’s cross-sectoral data consistency group contacted the RMA on large inconsistencies between monetary statistics and the IIP. The RMA is currently identifying reasons for these inconsistencies and ways of addressing them.

52. One RMA staff member attended the STA macroeconomic linkages course at the Singapore Training Institute. The RMA stated this had been a very useful course, and that they wished more staff, including from other agencies, could attend. The RMA suggested that TA could be delivered by a resident or regional multisector advisor, who could provide assistance in specific topics as well as in strengthening understanding and analysis of intersectoral linkages.

IV. Views on TA Effectiveness, Efficiency, Impact, and Sustainability

A. Official Data-Producing Agencies in Bhutan

53. The discussions with the main data-producing agencies and the responses to the questionnaires4 indicated that STA’s TA has been effective in varying degrees. The 2004 multisector mission was not immediately followed up by TA in GFS, national accounts, or prices, which meant it was less effective. The effectiveness of TA delivery was strongest for projects with a series of missions, such as has occurred recently for national accounts, prices, and external sector statistics.

54. Overall, the recommendations from the TA missions conducted in external sector statistics, national accounts, prices, and MFS have been implemented to a large extent. A lack of source data and limited resources were the key factors explaining delays in implementing some recommendations.

55. Two main data-producing agencies (RMA and NSB) indicated that the TA from STA was highly successful because it: (i) addressed issues that were highly relevant for the country and the objectives of the TA were developed, in most cases, with close consultation with the authorities of the counterpart agency, which facilitated adequate absorption of the TA provided; (ii) provided practical hands-on assistance with actual compilation and data collection and training; (iii) contributed to the increase in the scope and quality of statistics; and (iv) increased awareness on the need for good data dissemination practices and contributed towards the implementation of these dissemination practices.

56. The NSB and RMA were appreciative of the sequencing and timing of TA, which permitted gradual but tangible progress between different missions, while the MoF is only beginning to receive TA through a regional project.

57. Regarding resources, the authorities noted that TA recommendations were effective in setting priorities to ensure resource availability and consistency with the agency’s overall strategy. Even for recommendations where limited staff resources and source data delayed implementation, TA still provided the important benefit of enhancing the authorities’ understanding of international standards and setting of longer term objectives.

58. On modalities for delivering TA, all compilers except those for prices agreed in preferring training to TA. However, short-term TA missions were also considered to be highly effective. No agency received longer-term missions or resident advisors. National accounts compilers expressed their preference for longer missions, in particular for the initial stage of new projects, followed by shorter missions for subsequent implementation.

59. The authorities highlighted the importance of continuity of TA through the use of the same staff and experts and ongoing communication. The NSB has also received assistance from ADB and the World Bank, and consider this continuity as the Fund’s greatest strength compared to other TA providers.

60. The RMA and NSB both appreciated the availability Fund staff and experts to address follow-up questions through e-mails. They considered this contact to be an important ingredient for successful TA.

B. National Users of Statistics

61. The mission held discussions with the RMA Governor and the Secretary of the Ministry of Finance. In both cases, there was a high-level commitment to improving statistics and recognition of the progress in timeliness and availability of statistics that had already been made. They both agreed that the TA and training had assisted in economic decision making and related statistics directly to dealing with issues such as the level of economic activity and the causes of the current rupee shortfall.

62. The mission also circulated a questionnaire5 to users and held a meeting (see listing in Appendix I). The responses to the questionnaires showed that users have observed a clear improvement in the quantity and quality of data compiled and disseminated over the period covered by this review, especially during the most recent eight years, partly as a reflection of the TA provided by STA. However, interviews with users exposed a need for further outreach by official compilers to the user community to educate them about the data, including data innovations, characteristics, and available detail, as well as to improve accessibility.6

63. On the overall usefulness of TA missions, most users reported that TA was very or mostly useful. TA on CPI and PPI were slightly less appreciated, with 30 percent of users indicating that TA was somewhat useful only. (The subsequent discussion suggested that this surprising result was due to inadequate outreach.)

64. Prior to Fund involvement on TA programs, i.e., before 2004, GFS and balance of payments statistics were considered of good quality, while national accounts, prices, and MFS were mostly rated as of moderate quality. The assessment of the data quality improvement since 2004 placed all macroeconomic statistics as of good quality, but none of them reaching the highest level of very good quality, indicating that there is room for improvement in all statistical areas.

65. On the assessment of the relevance of STA’s TA, most users were not aware of the TA work. Out of the less than half that were aware, relevance was spread over evenly from very relevant to somewhat relevant. This indicates that communication with users regarding TA projects could be improved and that users are interested in improving statistical outputs rather than the process of how improvements were achieved.

66. Most users agreed or strongly agreed that TA provided resulted in data quality improvements that facilitated economic and policy analysis, formulation, and monitoring, including enhanced dissemination practices. However, some users noted that not all statistics are disseminated in user-friendly formats. In addition, users are not always aware of the details and historical series available in the different datasets.

67. The areas of statistics in which users believed improvement is required are: (i) more frequent data, in particular quarterly GDP; (ii) improvement of balance of payments and price statistics; (iii) improvement of communication of statistics, coordination with donors, and users’ understanding of statistics; (iv) improvement of metadata, such as footnotes; and (v) disaggregation of gross capital formation, in particular for construction and public investment.

C. Asia and Pacific Department of IMF

68. The most recent staff report on the Fund’s Article IV Consultation was published in June 2011 (IMF Country Report No. 11/123). It reported that “data provision has some shortcomings, but is broadly adequate for surveillance. Most affected areas are: national accounts, balance of payments, and fiscal data.”

69. The specific shortcomings identified in the staff report for national accounts were that:

  • The accuracy and reliability of the data are affected by inadequate source data. Key shortcomings include heavy reliance on production data collected by line ministries, which often lack quality control, long lags in providing estimates, large revisions of historical data and, in certain cases, ad hoc estimation procedures. Lack of reliable data on expenditure components hampers estimation of national savings and domestic investment. The key factors contributing to the data deficiency are the shortage of qualified personnel and facilities. The NSB is also constrained by the absence of a Statistics Act.

70. The specific concerns for balance of payments statistics were:

  • Despite recent improvements, external statistics continue to be affected by shortcomings. These include: (i) limited coverage of services’ transactions in the current account, (ii) compilation of trade data on a quarterly basis, while other external transactions are compiled on a fiscal year basis. Consequently, errors and omissions in the balance of payments presentations are large and preclude a more accurate assessment of external flows.

71. The specific problems identified for fiscal data were that:

  • Despite recent improvements in the quality of GFS, the fiscal data are subject to frequent and substantial revisions, particularly in the expenditure area. The compilation and dissemination of budget execution data and GFS are very limited. Only annual budget execution data are compiled and disseminated, but with a long lag. No GFS are disseminated nationally. Sub-annual data on budget execution are not available. There are inconsistencies between the fiscal and monetary data with regard to bank financing, stemming from differences in the definition of government.

72. The following specific issues were identified for monetary statistics:

  • While monetary statistics are generally consistent with the Fund’s guidelines, there is room for improvement in a number of areas, specifically: (i) the valuation of financial assets is based on purchase price while market or market-price equivalents would be preferable; and (ii) to avoid misclassifications, a list of government units and nonfinancial public enterprises should be prepared and shared with the reporting financial institutions.

73. The next Article IV visit will occur later in 2013. APD indicated to the mission that STA’s TA proved useful for improving Bhutanese macroeconomic statistics and that data provision to the Fund for surveillance purposes has improved in terms of quality and availability. They also identified priorities for future work to include addressing cross-sectoral consistency and developing data dissemination formats (e.g., Excel files rather than pdf documents).

V. Lessons Synthesized from the Experience in Bhutan

74. A primary motive for this report is to help STA improve its TA and training delivery. With this in mind, the mission synthesized the following lessons from the TA and training experiences with Bhutan during 2004–2013.

A. Technical Assistance Missions

Effectiveness

75. The implementation of TA recommendations was high in areas other than GFS. Recommendations were implemented to a large extent but taking into account priorities and resources available. They were considered to have clear objectives and be well organized. The results-based management approach of setting of objectives and defining tasks, and assessing outcomes has helped achieve this.

Timing

76. Missions as part of a project that included follow-up missions were regarded as more successful than one-off missions. Accordingly, the authorities suggested that TA should be followed up, but spread sufficiently well to give time to implement recommendations and make progress. However, TA missions in Bhutan have not always been evenly spread over time. A significant gap occurred between the multisector mission and the provision of TA in MFS, national accounts, and prices. In balance of payments, a follow-up mission took place a year after the multisector mission, but a gap of four years to the next TA mission likely impacted on the speed of implementing key recommendations. The GFS regional workshop was not followed up by a TA mission in the subsequent 24months. The authorities consider that the length of missions could be more flexible, with advice that four weeks would be suitable for starting detailed technical projects like national accounts, while follow-up missions may need only one week in some cases.

Preconditions for success

77. An essential foundation for the successful missions was that they had clear objectives and the authorities were committed to them, including having adequate staff resources, so that developmental work could be done while continuing existing operational tasks. Also essential was that there was a high degree of continuity on the side of both the TA receivers and those conducting the missions. These factors were lacking in GFS, although the completion of public financial management projects and the new JSA project by STA have the potential to improve this situation.

78. The quality of the counterparts allowed the TA providers to pass on information and exchange ideas effectively and efficiently. In addition, the high quality of expertise and advice of Fund mission staff and experts was well-recognized and advice was described in words such as high quality, hands-on, and practical. In several cases, the authorities mentioned that very good personal relationships had been developed.

79. IMF missions were praised because of the availability of the staff or expert for ongoing contact after the mission. This practice was contrasted favorably with World Bank and ADB, where outsourcing meant reduced contact with the main agency and no follow-up after the end of the project.

80. The presence of an IMF staff or expert was instrumental for enhancing coordination and seeking collaboration from data providers, particularly on balance of payments statistics.

81. Hands-on work during missions made it possible to automate compilation procedures, such as for the calculation of sectoral balance sheets and surveys in MFS, and implement new frameworks, such as the IIP.

82. Missions would be more effective if they conducted outreach or included advice to compilers on improving dissemination practices and communication with users. For example, one issue raised by users that could be better communicated was that the NSB publications did not provide information on the causes of revisions. Another issue was that formats were not user-friendly (e.g., short time series, pdf files). The user feedback revealed a range of concerns about data quality.

Reports

83. The agencies confirmed that they were able to comment on draft reports during the missions. Recommendations and timeframes were discussed and agreed after suitable consultation with the authorities for all missions. Final reports were received within a reasonable time in all cases.

84. Reports were received by the appropriate officials. Awareness of reports was generally high, although because of staff turnover, MoF was no longer aware of the 2004 multisector report dealing with GFS. Based on that experience, MoF urged that new projects provide copies of previous reports, so as they build on the previous work, rather than just repeat it, and to assist in being realistic about obstacles to future progress.

85. The length and content of reports was considered appropriate. Even in cases where recommendations were not yet achieved or were not achieved until much later, the RMA considered them beneficial as a guiding light for longer-term thinking. However, in national accounts, a few proposals had turned out to be impractical. To deal with this situation, more detailed technical notes or appendices would have been appreciated, or training could have focused on how to develop new alternatives.

86. “Benchmark actions” are identified in STA TA reports. These actions are used to help in evaluating progress in the area of price statistics and in considering any future request for follow-up TA and are based on high priority recommendations in the report. The agencies understood the role of benchmark actions.

B. Training

Effectiveness

87. STA training is highly appreciated and considered to be highly relevant to the work of the agencies. It plays a key role in staff development in the NSB and RMA, with the Fund considered as the key provider of technical training in macroeconomic statistics. The quality of STA staff and teaching methods was praised. Workshop and case study sessions were particularly praised for usefulness and developing personal interaction. Other than for GFS, a high proportion of course participants remained in the fields covered. The Fund’s Financial Programming and Policy course was also favorably mentioned because it was considered to support wider understanding by statistical staff. In overall rankings, most compilers ranked training as more useful, while some others ranked TA missions more useful, but in verbal comments, both were regarded as essential and mutually supportive.

88. The regional workshops under JSA projects were appreciated both for training and interaction with compilers from other countries in the region.

89. The application of a mix of TA and training was applied appropriately in Bhutan, with authorities generally making course nominations that reflected current or imminent work. The only exception was for FSIs, where the first attendance was in 2005, but compilation did not begin until several years later.

Access

90. All agencies are aware of the availability of and application procedures for relevant Fund training at Headquarters, Singapore Training Institute, and India Training Institute.

Different types of training

91. The authorities have a good appreciation of the different characteristics of training options. In general, Headquarters courses are preferred because of greater access to a range of Fund staff including the relevant STA country economists, which had lead to fruitful continuing contact. The regional contacts developed on Singapore Training Institute courses were also recognized as valuable, though personal contact with Fund staff and other participants were considered to be weaker, particularly with two-week courses being too short to generate such strong connections. A suggestion was made that small countries would be a suitable specialized target audience for more focused courses because the full Asian group included many large countries, which faced different issues from small countries.

92. Compilers praised the initiative of offering basic and advanced courses on MFS, and suggested that this could be followed in other sectors. Since advanced courses are not offered in external sector statistics, the mission recommended that the RMA apply for courses in the subcomponents of the balance of payments such as reserve assets, external debt statistics, and remittances.

GDDS

93. All agencies were involved in the preparation of the metadata and are appreciative of the Fund’s missions that assisted them in that process. The plans for improvement were largely consistent with subsequent work. However, they had not been used as a guide for TA and there has been no GDDS activity since then to update metadata to account for the recent developments, nor to use it as a tool for interagency communication.

94. The authorities suggested a short workshop to be conducted with a Fund staff member, in conjunction with another mission, to update metadata and revise the plans for improvement section to reflect current TA priorities.

95. To ensure awareness of TA missions among all the agencies, it was suggested that the Fund copy all mission documents to the GDDS coordinator.

Macroeconomic linkages and cross-sectoral data consistency

96. The agencies support STA’s increasing work to address consistency of macroeconomic statistics and agree that the Bhutanese statistical system can be improved by addressing these issues. The RMA proposed that a long-term multisector advisor would be considered as a TA delivery mode for addressing specific topics in several agencies as well as intersectoral linkages and interagency coordination. The authorities also see the benefit of addressing these issues in single-topic missions and covering them to a greater degree in training.

97. The authorities, in particular the balance of payments and national accounts compilers, are very interested in attending STA training courses on linkages in the system of macroeconomic accounts. As noted above under GDDS, suggestions were made to involve the GDDS coordinator in a greater role to increase interagency communication in Bhutan.

Donor coordination

98. In the case of NSB, the World Bank and ADB have also been active TA providers. NSB coordinated donors and put them in touch with each other, and was pleased with liaison by STA staff with these other donors. Contact among donors and between donors and NSB has been effective. For example, for price statistics, the World Bank provided hardware and an application for prices to be reported instantaneously by GPS-linked mobile device, while the Fund provided technical advice on the design of price collection procedures. However, the MoF was not clear about delineation between TA on GFS and public financial management assistance provided by the IMF Fiscal Affairs Department. The 2011 Economic and Social Commission for Asia and the Pacific report on the capacity of the Bhutan statistical system is publicly available, so it is helpful for all TA providers.

Link with fund surveillance

99. RMA and NSB statistical staff considered that they had good communication with APD. The TA projects in macroeconomic statistics were seen by the authorities as having been designed to be supportive of APD needs for surveillance.

Donor recognition

100. The beneficiary agencies of JSA projects had been made fully aware by the mission staff and in all documentation that the Japanese government was the source of funding. The agencies were highly appreciative.

VI. Conclusions and Recommendations

101. This section of the report attempts to summarize the main recommendations arising from the evaluation of TA and training to Bhutan during 2004–2012. While the recommendations are based solely on Bhutan’s experience, the mission considers that they are likely to be pertinent to STA delivery of TA and training more generally, particularly to the issues faced by small, developing countries and the particular advantages of JSA projects.

102. With the reasonable record of success in most areas and positive feedback from the authorities, the mission concludes that TA and training for Bhutan have been well-conducted and only minor suggestions for tweaking has been identified. Bhutan is likely to have a strong need for continuing assistance from STA, as many shortcomings remain and internal development capacity is limited, while sustainability is at continuing risk because of small numbers of statistical staff and staff turnover.

103. Mission scheduling and recommendations should take into account absorption capacity as the major factor in effectiveness and sustainability. In particular, with statistical agencies in Bhutan having limited staff, goals should be based on the limited resources available and not be over-ambitious. The success record and supportive attitudes of the Bhutanese statistical staff and senior management have been essential, so these attitudes should be looked for and taken into account in mission planning.

104. TA programs should be planned on the basis that a series of missions to follow through implementation is needed in complex cases, such as for substantial redevelopments or creation of new data systems, for example the new price indexes and quarterly balance of payments and national accounts, or developments of new data collections. Because of the effectiveness of a series of visits, TA efforts would benefit from having multi-year projects even though it would mean concentrating TA in fewer countries. Donor-funded projects, such as the JSA, have had a strong focus on follow-up visits, and are in line with this approach. The Fund’s Regional Allocation Plan procedures may tend to focus less on follow-up, to the extent that Fund area departments may recommend different countries each year to share around missions between countries. As well, area department staff tends to focus on identification of data shortcomings, rather than assessing whether the preconditions exist for successful TA and the best delivery styles.

105. Missions should provide more advice on dissemination and outreach. The consultation with users showed that international organizations and private sector users were not fully aware of progress related to STA TA projects and had received insufficient contact from the statistical compilers. Users gave useful input on improving dissemination formats, but also showed that they would benefit from greater explanation of the nature of the statistics and the improvements being undertaken. The mission concluded that many negative responses could have been avoided by better communication on the part of statisticians. Furthermore, compilers would have gained insights from listening to user viewpoints. Accordingly, Fund TA would be more effective if missions conducted outreach themselves or provided advice on outreach to be undertaken by the statistical compilers. In addition, missions should encourage the use of user-friendly formats for disseminating statistics, such as spreadsheets instead of pdf files.

106. Missions should provide copies of previous mission reports to compilers when the gap between missions is so long that counterpart statisticians were not involved in the previous mission.

107. Intersectoral data consistency issues should be incorporated more fully in STA’s data work, courses, and TA. Single-topic missions could do checking of cross-sectoral consistency, along the lines of checks done by the STA working group on the topic, as well as discuss interagency cooperation. Intersectoral coordination could be supported by STA by notifying the GDDS coordinator of all Fund TA and by having a short workshop to update the metadata. It would also support interagency coordination to agree to the GDDS coordinator’s suggestion that a Fund staff member conduct a short outreach seminar for all the agencies involved to inform new staff about the GDDS, update the metadata, and promote interagency coordination. Such a seminar could be undertaken as an add-on to another mission.

108. Fund training is so highly regarded that substantial changes are not recommended. A basis for success was the nomination of appropriate candidates and the integration in staff development planning, so this should be recommended and sought after in other countries. With the likely reduction in TA after the conclusion of the JSA projects, Fund courses will provide the best opportunities for Bhutanese staff to have access to Fund staff and other compilers to discuss country-specific issues. The longer courses, particularly in Headquarters, are recommended as being more suitable for achieving this objective, so should be maintained or expanded. Longer courses in regional training centers would also be beneficial. For additional courses, separate courses for introductory and advanced participants, and for small and large countries would be desirable.

VII. Data Reporting to STA

109. The mission and authorities discussed data reporting issues. The improvements in the timeliness, frequency, and availability of data identified in this report have largely been incorporated in data reported to STA for inclusion in IFS. The improvement in external sector statistics is particularly notable. However, the new PPI has not yet been reported, and available data on GDP by expenditure at current prices are no longer reported. (The NSB will try to restore reporting). Direction of trade statistics are also available in the RMA’s Selected Economic Indicators, but are not reported to STA, so that partner data are used to estimate Bhutan’s trade in the IMF Direction of Trade database. The mission updated and confirmed contact information for each dataset.

110. Table 2 compares data reporting to STA in January 2000 with December 2012. While MFS data have improved in frequency, and external sector data have become available, the table shows that reporting for national accounts has deteriorated.