A. Evaluation of STA TA and Training

6. The evaluation of STA’s technical assistance and training is increasingly founded on the Organization of Economic Co-operation and Development (OECD) Development Assistance Committee (DAC) Principles for Evaluation of Development Assistance.² These principles state:

• “An evaluation is an assessment, as systematic and objective as possible, of an on-going or completed project, program or policy, its design, implementation and results. The aim is to determine the relevance and fulfillment of objectives, developmental efficiency, effectiveness, impact, and sustainability. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision-making process of both recipients and donors.” (Paragraph 5)

• “An important purpose of evaluation is to bring to the attention of policymakers constraints on developmental aid success resulting from policy shortcomings or rigidities both on the donor and recipient side, inadequate co-ordination, and the effectiveness of other practices…” (Paragraph 9)

• “Evaluations should contain an executive summary, a profile of its purpose and scope (including the activities to be evaluated), a description of the methods used, the main findings, lessons learned, and conclusions and recommendations. Evaluations need to look at agency procedures and management issues.”

B. Background to the Evaluation of TA and Training to FYRM

7. The TA evaluation mission to FYRM followed the recommendations in the IMF’s Independent Evaluation Office (IEO) report in 2005 for developing more systematic approaches to track progress on major TA activities and to identify factors behind successes and shortfalls.³ STA conducts evaluation exercise of its TA and training, including evaluation missions to a few countries that receive relatively high TA volume from STA. The evaluation exercise therefore aimed at systematically drawing out lessons learned from the evaluation with a view to enhancing TA and training. These lessons are expected to provide a more informed background for future collaboration.

8. To the extent possible, the TA evaluation took into consideration the evolving work in the IMF to develop a RBM framework.⁴ While none of the TA under review was formally formulated using this framework, the evaluation placed emphasis on the results that have been achieved with the TA and training support of the IMF in macroeconomic statistics during the period under review (2000–2012). Table 1 (in the next section) lists the TA and related missions under review. Appendices II and III describe training provided by the IMF to FYRM on macroeconomic statistics.

^{Table 1.}

List of Technical Assistance Missions to FYRM Conducted by the IMF Statistics Department between 2000 and 2012

^{Table 1.}

List of Technical Assistance Missions to FYRM Conducted by the IMF Statistics Department between 2000 and 2012

Year	Topic
2000	Balance of Payment Statistics
2002	Balance of Payment Statistics
2004	National Accounts Statistics
2004	National Accounts Statistics
2004	Data Module ROSC
2005	Balance of Payment Statistics
2005	Money and Banking Statistics
2005	National Accounts Statistics
2006	Balance of Payment Statistics
2006	Government Finance Statistics
2007	Government Finance Statistics, regional project
2007	National Accounts Statistics
2007	Export/Import Price Index
2007	Training: Government Finance Statistics
2008	Government Finance Statistics, regional project
2008	National Accounts Statistics
2008	National Accounts Statistics
2008	National Accounts Statistics
2008	National Accounts Statistics
2008	Balance of Payment Statistics
2009	Government Finance Statistics, regional project
2009	National Accounts Statistics
2010	SDDS Assessment
2011	Quarterly National Accounts
2012	Quarterly National Accounts, regional project

View Table

^{Table 1.}

List of Technical Assistance Missions to FYRM Conducted by the IMF Statistics Department between 2000 and 2012

Year	Topic
2000	Balance of Payment Statistics
2002	Balance of Payment Statistics
2004	National Accounts Statistics
2004	National Accounts Statistics
2004	Data Module ROSC
2005	Balance of Payment Statistics
2005	Money and Banking Statistics
2005	National Accounts Statistics
2006	Balance of Payment Statistics
2006	Government Finance Statistics
2007	Government Finance Statistics, regional project
2007	National Accounts Statistics
2007	Export/Import Price Index
2007	Training: Government Finance Statistics
2008	Government Finance Statistics, regional project
2008	National Accounts Statistics
2008	National Accounts Statistics
2008	National Accounts Statistics
2008	National Accounts Statistics
2008	Balance of Payment Statistics
2009	Government Finance Statistics, regional project
2009	National Accounts Statistics
2010	SDDS Assessment
2011	Quarterly National Accounts
2012	Quarterly National Accounts, regional project

9. To conduct the evaluation, the mission: (i) reviewed the TA-related Fund documents and TA reports as well as training provided by STA since 2000; (ii) summarized the findings of those TA missions and derived the results obtained by the Macedonian authorities with the support of these TA and training: (iii) elicited the authorities’ views on the effectiveness of STA’s TA and training, and discussed the status of implementation of major recommendations made by STA, as well as the modes of TA and training delivery they considered most appropriate; (iv) solicited views from national users of statistics; and (v) consulted with EUR and used EUR documents (especially the Statistical Issues Appendix to the latest Article IV staff report, 2011) to collect EUR views on the usefulness and effectiveness of STA TA and training on macroeconomic statistics to FYRM.

10. To get the authorities’ and national key data users’ views on the effectiveness of TA, the mission provided each of the main data-compiling and disseminating agencies, and selected data users, with questionnaires on the usefulness of, and main lessons learned from, STA’s TA and training with a focus on results. Two sets of questionnaires were used—one for key TA beneficiaries and one for key data users. The questionnaires also asked about future TA and training needs. The questionnaires were sent to (i) 11 TA recipients of whom 8 responded, representing 73 percent response rate, and (ii) 60 key data users of which 18 responded, representing 30 percent response rate. The summary of the results of the surveys was made prior to the mission so as to facilitate deeper evaluation during the mission. The questionnaires and a summary of the responses appear in Appendices IV–VIII.

11. This evaluation report summarizes the findings, lessons, and recommendations regarding the TA and training with respect to: (i) relevance and effectiveness (including alignment with government priorities); (ii) efficiency (including coordination across government agencies and with donors); and (iii) impact and sustainability (including constraints on absorptive capacity). By nature, the assessments relied heavily on qualitative analysis, based on the views of stakeholders gathered through the process described above and to the extent possible substantiated by tangible evidence. In addition, the evaluation includes an assessment of the quality of the TA reports, backstopping, and management report.

12. It is important to keep in mind that TA for statistical capacity building normally has a long time lag in producing results hence the benefit of TA might not be totally captured within the review period. However to the extent feasible, the mission attempted to capture as comprehensively as possible, the contributions of the STA TA and training program to the improvement of macroeconomic statistics in FYRM. More importantly, the mission focused on the view of the authorities regarding the effectiveness of STA’s TA and training program vis-à-vis their objectives as well as on what further improvements could be made.

A. The 2004 Data Module ROSC

17. The 2004 Data Module ROSC was conducted against the GDDS benchmark. The mission provided an in-depth assessment of the quality of the FYRM’s macroeconomic statistics using the IMF’s Data Quality Assessment Framework (DQAF). The data assessment covered the national accounts, consumer price index, producer price index, government finance, monetary and financial, and balance of payments statistics. The mission also conducted an institutional assessment of the main macroeconomic statistics compiling agencies: the NBRM, the SSO, and the MOF. More importantly, the ROSC mission provided prioritized recommendations.

18. The ROSC mission found that although the SSO and the NBRM had a favorable legal and institutional environment that supported the compilation of statistics, the MOF had been only able to produce limited coverage of government finance statistics according to the international standard. At the time of the mission, the institutional arrangement of MOF allowed only the production of budgetary central government data on a timely basis but not the consolidated data for the general government in accordance with the framework of the GFSM 2001. All three institutions were found to be characterized by a high degree of professionalism and observance of ethical standards by their staff. The mission found that resources for the compilation of national accounts statistics were broadly sufficient, while the NBRM also allocated adequate resources for the compilation of the monetary and financial, and balance of payments statistics. However, it indicated that not enough resources were available for the compilation of GFS and price indices. There were procedures in place for the SSO and the NBRM for monitoring the relevance of the data produced but no processes for consultations with users in the case of fiscal data.

19. For national accounts statistics, the lack of a statistical register and low response rate to surveys made the latter source data not suitable for compilation of national accounts. Hence, national accounts was mostly compiled based on data from financial statements of enterprises available from administrative databases. On technical issues, deficiencies were observed in deflation methods for the national accounts and the index formula for price indices (CPI and PPI). More specifically, statistical techniques for the compilation of volume measures of value added in some activities and for measuring taxes on products were found not adequate and no inventory valuation adjustments were made for the compilation of GDP by production and by expenditure. A methodological problem was found with regard to the application of the index formula that could potentially cause a bias in the price indices. The treatment of missing prices could also be improved. The mission recommended improving the response rate to surveys by invoking and implementing the tools given by State Statistics Act (1997). The mission also recommended that the authorities complete the statistical register, improve procedures for the compilation of volume measures, implement inventory valuation adjustments, increase human resources for the compilation of price indices, revise the formula for CPI and PPI, include exports in PPI, and improve procedures for collecting the data from the household budget survey.

20. Data Module ROSC also noted many areas for improvement in GFS to meet international standards, namely GFSM 2001. It found that MOF compiled fiscal statistics following national concepts and definitions, broadly similar to GFSM 1986, but there was no migration plan to GFSM 2001. Coverage was limited to budgetary central government despite some availability of source data. Domestic debt data for central government were not disseminated. The basis of recoding was on a cash basis rather than the recommended accrual basis. Improvements were needed to better align the national chart of accounts with GFSM 2001 and an automated procedure for the assessment of source data was required. Data seemed to be consistent within the fiscal data set but was not always the case with monetary statistics.

21. The mission found that the quality of the MFS was good and in most cases consistent with the framework for compiling monetary statistics as recommended in the Monetary and Financial Statistics Manual 2000 (MFSM 2000). However, some improvements were needed with regard to: extending the scope of the depository corporations survey to include savings houses, better classification of some financial instruments such as shares and other equity and insurance technical reserves, following the correct basis for recording of non-tradable long-term securities issued by the Bank Rehabilitation Agency and reviewing the processes for assessing source data to ensure their effectiveness in identifying data inconsistencies and misclassifications. The mission recommended an expansion of the institutional coverage of monetary statistics to include savings houses and classification improvement of financial assets.

22. For balance of payments statistics, the mission found that the overall quality of the statistics was good, except for some shortcomings. Important shortcomings were connected with scope in the case of reinvested earnings, basis for recording of interest which was recorded on a due-for-payment basis instead of accrual basis, and source data for private current transfers and unrecorded trade. The mission recommended close collaboration with the SSO to use surveys for better estimation of private transfers and incorporation of the results of supplementary surveys covering known data gaps, such as reinvested earnings, travel, and trade credit.

B. TA Evaluation Mission Findings on Single-Topic Missions

23. STA has provided intensive TA on national accounts statistics with 11 missions. Following the 2004 Data Module ROSC mission, a TA project focusing on the development of quarterly national accounts was initiated in May of the same year. Two follow-up missions on national accounts were carried out afterwards in October 2004 and August 2005 to assist in the implementation of the project. A new system of quarterly compilation of GDP by production, based on international best practices was established providing both estimates at current prices and estimates of volume measures consistent with the annual data. The system which is now operational was not implemented until late 2008 when the first quarterly estimates based on the expenditure approach also became available. Further recommendations were provided on development of quarterly GDP by expenditure and reconciliation between the quarterly estimates by production and expenditure. To have good benchmarks for quarterly data, STA also provided recommendations to improve the annual accounts. However, these recommendations were implemented at a rather slow pace. These included improving the methodology for the calculation of volume measures of annual GDP and improving the accuracy of annual GDP current price data.

24. A mission on price statistics was carried out in December 2007. Its main purposes were to assist the authorities to improve the unit value indices of exports and imports, and to develop the methodology for establishing price indices of exports and imports based on price specifications. The mission also reviewed the status of implementation of previous recommendations on the CPI and PPI and found that they had not been implemented (the procedures for imputing missing prices for seasonal products was not introduced, no data had been collected for the imputed rents of owner-occupied dwellings, and exports had not been included in the scope of the PPI). However, it found that important projects were ongoing for the updating of both indices with new weight base and reference periods. Moreover, the SSO was engaged in developing the harmonized CPI in accordance with EU regulations.

25. Following the 2004 Data Module ROSC, STA provided four GFS TA missions to assist the authorities to improve the fiscal statistics. The TA was primarily aimed at ensuring that the GFS meet the GFSM 2001 standards which could also advance compilation efforts as per the 1995 European System of Accounts (ESA 95). STA selected FYRM as one of the pilot countries to present the fiscal statistics in the GFSM 2001 format in the context of IMF Surveillance work and in EUR Staff Reports, following the IMF Board’s directive. For this purpose, the 2006 GFS mission reviewed the fiscal source data and MOF Treasury’s data compilation practices against the GFSM 1986 framework with a view to developing an automatic bridge table linking the data into GFSM 2001 framework. The mission found some data gaps, especially related to source data for the development of the balance sheet. Follow-up missions attempted to develop a bridge table to the extent possible and provided hands-on training. By the last GFS mission in 2009, the GFSM 2001 format data were produced for 2005–2007. MOF reported GFS in the GFSM 2001 format to STA for the first time in 2009. Unfortunately the two staff who were counterparts of the GFS missions both left the MOF. Very limited staff in the Treasury’s Statistics Division (two people) combined with other work priorities explained the limited implementation of the GFS missions’ recommendations. The MOF no longer reports data to STA. Nevertheless as compared with the situation in 2000, the MOF has disseminated more fiscal data (broadly consistent with GFSM1986), higher frequency data (monthly data for central government operations), and started to disseminate general government operations data on an annual basis. Also, since last year, central government debt data (both stocks of external and domestic debt) have been disseminated on the MOF’s website. In connection with the subscription to the SDDS, tangible improvements have been achieved in GFS both in terms of methodological and dissemination aspects.

26. On MFS, STA fielded one TA mission to the NBRM in 2005. The mission was instrumental in assisting the NBRM to commence compilation of the full set of sectoral balance sheets and surveys of the depository corporations. The mission also provided assistance on addressing classification issues of more disaggregated loans, treatment of treasury bills and NBRM reserves used for guarantees, data revision practices, and consistency of monetary data with other macroeconomic statistics. The mission also assisted the authorities to implement the Standardized Report Forms (SRFs) for reporting monetary data to STA. All of the recommendations of the mission have been implemented although with some time lag. The MFS currently compiled and disseminated are of good quality and follow the recommendations in the MFSM 2000.

27. For BOP statistics, STA conducted four TA missions during 2000 and 2006. The missions helped improve the international transactions reporting system (ITRS) especially in its redesign to include data validation and consistency checks as it was the main data source for the compilation of external sector statistics. Subsequent TA missions helped with the expansion of data sources to include direct reporting in form of surveys of direct investment, trade credit, travel, and current transfers. These missions reviewed the possibility of using partner country data in the case of trade in part to address the shortcomings of the ITRS system and also took into account new developments in the economy. The TA also promoted better collaboration among different data collection agencies for related external sector statistics (NBRM, MOF, and SSO) to ensure consistency between the different macroeconomic statistics and avoid duplication of efforts. Assistance on reconciliation between flow data (including transactions of the BOP) and stock data for items reported in the international investment positions statement was provided. The authorities made very good use of TA in terms of the progress made in implementing the recommendations of the TA missions. Implementation of a few remaining recommendations is still on-going particularly in respect of current transfers and travel estimates.

28. With regard to training, the FYRM authorities indicated that they had benefited significantly from training offered at the IMF headquarters as well as training programs offered at the regional training center in Vienna and in the country. They received training in all macroeconomic statistics. The mission found that the authorities had made sufficient effort to secure training for key staff directly involved in the compilation and dissemination of macroeconomic statistics. It was noted that training of the key staff had yielded positive results in regard to improvements in the quality of statistics compiled. However, not unique to FYRM, in some instances trained staff moved to different areas and were replaced by new staff some of whom were yet to benefit from the different training programs provided by the IMF. Both theoretical and hands-on training were also delivered during the TA missions, especially following introduction of new data collection and compilations procedures. However, further training to cover new developments in compilation was highlighted by the authorities as a key requirement especially in view of the introduction of more recent guidelines and manuals for the compilation of macroeconomic statistics. Overall, the authorities indicated that training offered by the IMF was very beneficial especially owing to its practicality during the workshop sessions, the expertise of the trainers, and the opportunity accorded to participants to network and share experiences.

C. Results

29. During the period 2000–2012, the Macedonian authorities have made noticeable progress towards strengthening their macroeconomic statistics (Table 2). Much of the progress was in terms of conforming to internationally accepted statistical methodologies for data compilation of macroeconomic statistics. In addition, there has been good progress in data dissemination practices, sensitization of users about revisions of data, as well as an increase in the volume of data releases. Furthermore, the authorities have made progress on the institutional and legal structure to enhance data compilation.

^{Table 2.}

Comparison of Macroeconomic Statistics 2000 and 2012¹

¹Notation “-” means ‘not available”.

^{Table 2.}

Comparison of Macroeconomic Statistics 2000 and 2012¹

Statistics	Coverage		Periodicity		Timeliness		Methodology
	2000	2012	2000	2012	2000	2012	2000	2012
National accounts	National	National	A	Q	15M	80D	1993 System of National Accounts/ European System of Accounts 1995 (1995 ESA).	Added some elements of the 2008 System of National Accounts/European System of Accounts 2010
Consumer price index	National	National	M	M	3D	3D	Consumer Price Indices: An ILO Manual 1989	Added some elements of the most updated methodology following EU regulations
Government finance statistics	Central Government General Government	Central Government General Government	M	M A	Q	Q 2Q	A Manual on Government Finance Statistics 1986 (GFSM 1986)	Progress has been made toward the GFSM 2001 framework. Some GFS data were reported to STA.
Monetary and financial statistics	Banking sector	Banking sector	M	M	1M	21D	A (Draft) Guide to Money and Banking Statistics in IFS 1984	Monetary and Financial Statistics Manual 2000
Balance of payments	National	National	M	M	75D	2M	Broadly follow Balance of Payments Statistics Manual, fifth edition (1993)	More adherence to Balance of Payments Statistics Manual, fifth edition(1993)
International investment position	−	National	−	A	−	1Q	−	Balance of Payments Statistics Manual, fifth edition (1993)
Reserves template	National	National	M	M	45D	1M	Balance of Payments Statistics Manual, fifth edition (1993)	International Reserves and Foreign Currency Liquidity: Guidelines for a Data Template (2001)
External debt	Public Sector	National	Q	Q	25D	1Q	National classification	External Debt Statistics Guide 2003

¹Notation “-” means ‘not available”.

View Table

^{Table 2.}

Comparison of Macroeconomic Statistics 2000 and 2012¹

Statistics	Coverage		Periodicity		Timeliness		Methodology
	2000	2012	2000	2012	2000	2012	2000	2012
National accounts	National	National	A	Q	15M	80D	1993 System of National Accounts/ European System of Accounts 1995 (1995 ESA).	Added some elements of the 2008 System of National Accounts/European System of Accounts 2010
Consumer price index	National	National	M	M	3D	3D	Consumer Price Indices: An ILO Manual 1989	Added some elements of the most updated methodology following EU regulations
Government finance statistics	Central Government General Government	Central Government General Government	M	M A	Q	Q 2Q	A Manual on Government Finance Statistics 1986 (GFSM 1986)	Progress has been made toward the GFSM 2001 framework. Some GFS data were reported to STA.
Monetary and financial statistics	Banking sector	Banking sector	M	M	1M	21D	A (Draft) Guide to Money and Banking Statistics in IFS 1984	Monetary and Financial Statistics Manual 2000
Balance of payments	National	National	M	M	75D	2M	Broadly follow Balance of Payments Statistics Manual, fifth edition (1993)	More adherence to Balance of Payments Statistics Manual, fifth edition(1993)
International investment position	−	National	−	A	−	1Q	−	Balance of Payments Statistics Manual, fifth edition (1993)
Reserves template	National	National	M	M	45D	1M	Balance of Payments Statistics Manual, fifth edition (1993)	International Reserves and Foreign Currency Liquidity: Guidelines for a Data Template (2001)
External debt	Public Sector	National	Q	Q	25D	1Q	National classification	External Debt Statistics Guide 2003

¹Notation “-” means ‘not available”.

30. In the area of national accounts statistics where most TA has been provided during the period 2004–2012, the SSO has made significant progress in improving the scope and quality of the national accounts. A tangible measure of this progress is the subscription to the SDDS. The support of the STA TA had a significant role in the development of a quarterly compilation system of GDP by production at current prices and in volume terms. STA TA has also assisted in improving the methodology for the compilation of the annual accounts, particularly the GDP estimates based on the production and the expenditure approaches. Revised annual series of volume measures of GDP have been compiled. However, further improvements are planned, particularly with regards to the estimates of changes in inventories. Also, the estimates of household consumption expenditure still need to be implemented.

31. As recommended by the 2007 price statistics mission, the unit value of exports and imports are now compiled monthly using the Standard International Trade Classification (SITC). The indices have also been compiled using the international standard—NACE classification (Nomenclature des Activités Économiques dans la Communauté Européenne). Price-specific indices of exports and imports have not yet been introduced owing to the lack of staff resources, although work is planned to be initiated with the support of a European Union project (IPA 2009). Improvements to the CPI have been introduced in accordance with international standards and EU regulations concerning the treatment of seasonal products and the use of the geometric mean for the calculation of the elementary indices.

32. The series of GFS missions developed a bridge table to automatically recast the fiscal data to the GFSM 2001 framework and the data were produced for 2005–2007. Further, the coverage of fiscal statistics was extended to general government operations. MOF increased dissemination of fiscal data significantly, including the new data series for annual general government operations and stock of public debt. To meet the international standard for dissemination of statistics, the MOF has disseminated an advance release calendar since 2011. GFS training and TA to the MOF Statistics staff helped their understanding of the production of fiscal data in the ESA95 framework as well. However, the lack of staff resources has limited the sustainability of the TA in GFS.

33. Impressive results have been achieved in the production of external sector statistics that meet international standards. The authorities are now compiling and disseminating important new sets of external sector data, such as international investment position (IIP) data, reserve assets and foreign currency liquidity data, and external debt statistics. The compilation of reserves data meeting the IMF’s “International Reserves and Foreign Currency Liquidity: Guidelines for a Data Template” (2001) was notably useful for policy decision makers. Other notable improvements during the period included the increased use of electronic data dissemination and improved timeliness as well as periodicity of data.

34. In summary, the main results that have been achieved in recent years, with the support of STA TA and training, include the following:

• FYRM graduated from being a GDDS participant to a subscriber of the SDDS in November 2011;

• Adoption of improved methodology for the compilation of the volume measure of GDP by production and expenditure;

• Development of a quarterly compilation system for GDP by production at current prices and in volume terms;

• Development of monthly unit values of exports and imports compiled using international standards (the SITC classification and NACE classification);

• Development of a bridge table to recast fiscal data into the GFSM 2001 framework, but some important data gaps remain to complete GFS balance sheets.

• Compilation of MFS based on the MFSM 2000, and reporting to STA using the IMF’s SRFs for the central bank and other depository corporations (ODC);

• Collection of information for the compilation of the OFC survey to expand the coverage of monetary and financial statistics;

• Improved adherence to Balance of Payments Statistics Manual, fifth edition, for the compilation of balance of payments and IIP;

• Improved dissemination of quarterly BOP and IIP statistics, and strengthened data sources for the compilation of trade credits, foreign direct investment, loans and workers’ remittances; and

• Improved inter-institutional collaboration across data producing agencies which has been facilitated through discussions of methodology among responsible institutions and formalization of data sharing and better sequencing of surveys. As a subscriber to the SDDS, statistical agencies collaborate closely to fulfill the data reporting requirements. Nevertheless, there is room for further collaboration to increase cross sectoral data consistency.

D. Summary of Findings

35. STA TA and training has been relevant, in most cases, responding to demands expressed by the Macedonian authorities. The authorities confirmed that STA TA was reasonably effective especially in national accounts, BOP and MFS. For national accounts and prices, implementation of recommendations was delayed in some instances due to insufficient resources and difficulties faced in establishing adequate source data. Lack of staff resource has been the major reason for the delay in implementing the TA recommendations, especially for GFS.

36. The TA interventions were efficient in large part due to STA’s focus on the provision of assistance in the core areas of its technical expertise and STA’s selection of appropriate experts. The authorities commended the efforts made by STA in the selection and use of experts with a good understanding of the local and regional setting and systems which facilitated the recommendation of appropriate tasks for improvement. Further, the continuation of the same TA experts has led to more efficiency in the delivery of TA. The provision of complimentary training to staff involved directly in the compilation of statistics enhanced the implementation of TA’s recommendations and sustainability. Also, the authorities indicated that they found TA reports to be sufficiently clear and one agency noted that the quality of reports had improved over time. This reflects positively on the quality of STA back stopping practices, particularly in striving to produce clearly written TA reports.

37. The TA had a strong and positive impact in most cases, as demonstrated by the results obtained by the authorities. Many constraints to capacity were overcome through additional recruitment and training of staff, in some cases. Examples of results included the implementation of changes in national accounts to increase periodicity to quarterly, expansion of the monetary and financial statistics to include savings houses, compilation and dissemination of the IIP, updated methodologies for GFS, BOP and MFS; and stronger inter-institutional collaboration such as an establishment of statistical committee with members from various agencies to ensure common strategy.

A. Official Data-Producing Agencies in FYRM

38. The discussions with the main data-producing agencies, as well as the responses to the questionnaires sent prior to the mission, indicated that STA’s TA has generally been effective. The authorities noted the key role played by the Data Module ROSC mission in terms of setting the agenda for key improvements to be made to the various statistics. In particular, the NBRM indicated that its reorganization of the statistics department benefited significantly from the concurrent undertaking of the Data Module ROSC mission. Overall, the recommendations from the TA missions conducted in BOP, national accounts and MFS had to a large extent been implemented. The authorities highlighted the lack of sufficient resources as key limitations faced in the implementation of TA recommendations, especially for GFS. Further, availability of source data was highlighted as the key factor explaining delays in implementing some recommendations. They also noted that the evaluation of TA effectiveness was useful because it allowed them to express their views regarding the STA TA and to reflect on how best to enhance their use of TA.

39. The three main data-producing agencies (NBRM, SSO and MOF) indicated that the TA: (i) addressed issues that were highly relevant for the country and the objectives of the TA were developed, in most cases, with close consultation with the authorities of the counterpart agency which facilitated adequate absorption of the TA provided; (ii) provided practical hands-on assistance with actual data collection; (iii) contributed to the increase in the scope and quality of statistics; and (iv) increased awareness on the need for good data dissemination practices and contributed towards the implementation of recommended data dissemination practices.

40. SSO and NBRM were appreciative of the sequencing and timing of TA which permitted gradual but tangible progress between different missions while other agencies held the view that some improvement to the sequencing could be done. Of particular note was the importance of the Data Module ROSC mission which was considered very timely and provided a comprehensive assessment of the status of statistical compilation and dissemination. The authorities found the ROSC report useful and they generated their work plans following the ROSC’s recommendations. The NBRM adopted the DQAF approach for their own regular assessment of the statistics compiled and related procedures. They also valued the opportunities that TA missions provided to foster inter-agency communication and cooperation, which is a particular challenge in the case of the FYRM.

41. As regards to resources, authorities noted that TA recommendations should be more consistent with available staff resources and should provide prioritized recommendations accordingly. One agency indicated that a rather general recommendation to increase staff resources so as to allow for more statistical improvement could create an unrealistic expectation from their staff despite the reality of the budget constraint. Instead, TA could be more helpful by identifying clearly the priorities under a realistic scenario for staff resources. In this connection, in some cases, STA TA could also involve and, if possible, get a clear agreement with the agency’s top management in setting the priorities of recommendations to ensure of the availability of resources and the consistency of recommendations with the agency’s overall strategy.

42. Even in the cases where limited staff resources and source data delayed the implementation of TA’s recommendations, TA still provided the important benefit of enhancing the authorities’ understanding of international standards. Despite some delay in the GFS improvement, the authorities indicated the usefulness of GFS TA to enhance their understanding of ESA95 compliance. In particular, the collaboration with the authorities when determining the TA and training objectives had been instrumental in avoiding overlaps with other TA and training providers.

43. On modalities for delivering the TA, the compiling and disseminating agencies diverged in their preference possibly reflecting different stages of statistical development. The SSO preferred and continues to prefer short-term TA and training. The NBRM staff responsible for external sector statistics appreciated earlier TA in the form of short-term missions but expressed preference for future TA to be delivered remotely and the need for more training. With other higher priorities and limited staff resources, the MOF preferred training and indicated that IMF TA should be collaborated with other national agencies (like SSO) and international agencies (like Eurostat) in light of the ongoing work towards achieving ESA95 compilation standards. Such collaboration should ensure consistent recommendations and right sequences of TA.

44. The data compiling agencies emphasized the important role played by experts after conducting the TA missions with regard to addressing follow-up questions through emails as an important ingredient for successful TA. Authorities highlighted the importance of continuity of TA through remote means after the missions. They also highlighted the importance of clearly written reports.

B. National Users of Statistics

45. The responses to the questionnaires distributed by the mission and discussions with data users showed that users had observed significant improvements in the quantity and quality across all macroeconomic statistics during the last ten years. However, the users highlighted GFS as the area that required the most improvement. In general, the responses to the questionnaires highlighted the importance and effectiveness of TA and its relevance in influencing the improvements in statistics observed during the period. These perspectives were confirmed by the mission during interviews with users. Some interviewees further expressed their concerns about: (i) the limited availability of detailed data of good quality on employment, real estate prices, confidence indicators, disposable income, import and export price indices and living standards ⁵; (ii) implementation of dissemination policy regarding equal access, with one response highlighting an incidence that a cabinet minter quote some SSO’s statistics before they were publically released⁶; (iii) timeliness and comprehensiveness of GFS statistics; and (iv) accuracy of estimation of private transfers. Interviews with users also exposed a need for improvements to the SSO’s web content and for an increase in the level of detail disseminated for GFS. Also, some key data users indicated that TA should pay more attention to agency collaboration, especially among source data agencies and MOF.

C. Eur

46. This section on the views of the EUR-FYRM team reflects the information in the latest Staff Report of 2011 for FYRM and the discussion with the EUR team members. The 2011 Staff Report indicates that data provision has some shortcomings, but is broadly adequate for surveillance and monitoring purposes. EUR recognized the usefulness of STA TA and training and that there has been tangible progress in data availability and data quality during the last few years. It noted tangible contributions from TA on the quality improvements in balance of payments and national accounts statistics. Regarding national accounts statistics, data now included the quarterly GDP series by production and expenditure both at current and constant prices. The consistency of annual and quarterly national accounts data has improved. In this connection, EUR recognized that continuing and regular engagement (preferably by the same TA person/team) has paid off over time.

47. EUR also indicated that the focus of TA should be extended beyond methodology to other aspects of data quality such as dissemination format and improvement of source data. High priorities should be in the areas of national accounts statistics, fiscal data (based on the FYRM’s budget law on coverage and classification) and GFS as well as labor market statistics. It was noted that national accounts statistics, at times, had large revisions that could benefit from clearer explanation of the revisions. While EUR noted improvement in the data dissemination practices partly the result of FYRM’s subscription to SDDS and more downloadable data, further improvement could be made with TA support. The SSO’s publication could benefit from the inclusion of data status (preliminary, revised, or final data) and more comprehensive metadata. The reconciliation of GDP based on expenditure and production approaches also requires further improvement.

48. TA should give more focus on source data, especially fiscal data, so better GFS could be produced. Data coverage of government below-the-line financing from NBRM was inadequate, as a result of different coverage of accounts between the MOF and the NBRM.