Annex 5. Do Staff Follow the Operational Guidelines on Data Provision for Fund Surveillance?
- International Monetary Fund. Independent Evaluation Office
- Published Date:
- April 2016
As part of the 2012 Review of Data Provision to the Fund (IMF, 2012b), the IMF Statistics Department (STA) and Strategy, Policy, and Review Department (SPR) jointly reviewed a sample of 50 staff reports for Article IV consultations discussed by the Board between January 1, 2011 and March 31, 2012 to determine “whether the 2008 guidance note on data provision has been implemented” and “the extent to which these procedures have been effective in strengthening surveillance.” This evaluation uses the same sample of countries (Table A5.1) to replicate the review for the period between January 1, 2014 and February 18, 2015, to examine now their compliance with the 2013 guidance note on data provision (IMF, 2013a).1 The analysis compares the reports on a number of dimensions such as the application of the A, B, C rating; identification of data sources in the tables of the staff report; and the inclusion of information on metadata provided by countries in the “Data Standards and Quality” section of each report’s Statistical Issues Appendix (SIA).
A, B, C rating. Compared to the 2012 review, the A, B, C classifications in the sample group were slightly higher overall. More than half of the 48 countries reviewed were rated B,2 while only four were rated C. Since the 2012 review, the ratings improved for five countries (two from C to B and three from B to A), decreased for two countries from B to C, and remained the same for the rest. One country, previously classified as B, had no SIA. For those countries whose ratings improved, no explanation was given for two, clear descriptions of the improvements were given for two, and the data discussion for one (whose rating moved from B to A) suggested data were of such poor quality that a C rating might have been merited. All staff reports for the C category appropriately included a discussion of data issues in the main body of the staff report. For the A and B countries—where the guidance allows more discretion, encouraging a discussion in the report “whenever considered relevant for surveillance”—the discussion of data issues varied greatly, with no discussion for about a third of the B-rated countries (including a number of fragile states) yet significant discussions for about a third of the A-rated countries. Overall, these results suggest little increase in candor and, given the variety of results, lack of significance of the A, B, C ratings for identifying data deficiencies for surveillance.
|Central African Rep.||Brunei Darussalam||Albania||Afghanistan||Belize|
|Congo, Rep. of||Fiji||Germany||Libya||Mexico|
|Equatorial Guinea||Marshall Islands||Hungary||Qatar||Paraguay|
|Gambia, The||Papua New Guinea||Macedonia||Turkmenistan||St. Lucia|
|Guinea||Singapore||Norway||United Arab Emirates||St.Vincent|
Selected Economic Indicators tables. None of the data tables in the staff reports provided sources of data at the level of detail recommended in the 2013 operational guidance note. The 2013 guidance note specifies3 that the “[t]ables and charts reporting statistical data included in the staff report should provide the source of the data, explicitly distinguishing among official statistics, other sources of data, and staff estimates, particularly if data from different sources are presented in the same table/figure.” The example of the table in the guidance note calls for the footnotes to “document the data sources for each data category, structural breaks in data, and the reasons for using staff estimates instead of official data.” This was the least observed dimension in the sample; most often, the source of data was simply described as “authorities and IMF staff estimates.”
Metadata provided to the Dissemination Standards Bulletin Board. Eight reports in the sample did not note when metadata provided by the countries were out of date. Staff is expected to provide information on metadata for SDDS Plus adherents, SDDS subscribers, and GDDS participants in the SIA section on “Data Standards and Quality.” This review looked for any discussion on metadata in the SIA for countries whose metadata had not been updated for more than five years. Eight reports, or more than 15 percent of the sample, contained no mention of outdated metadata.
Overall, this evaluation’s review suggests that, by and large, the latest operational guidelines on data provision for Fund surveillance have had little impact on the staff’s treatment of data issues.