Annex 2. Summary of Background Papers and Document
- International Monetary Fund. Independent Evaluation Office
- Published Date:
- April 2016
BP/16/01. The Rules of the Game: Data-Related Mandate, Obligations, and Practices at the IMF
This paper describes the evolution of and current set of obligations and practices for data provision by IMF members and for data collection and dissemination by the Fund. For member countries, the legal framework stipulates the guiding principles, the minimum set of data to be provided, and the procedures to be followed in case of misreporting. Most of the economic data the Fund collects—in the context of surveillance and for other operations—are provided by countries voluntarily, on the basis of trust and mutual benefit. For the Fund, very few legal obligations exist concerning data. Nonetheless, the Fund contributes to the production and dissemination of good quality data by members, and has mechanisms in place to monitor the quality of the data collected. At the same time, it is subject to a comprehensive transparency policy applicable to its own documents and the data they include.
BP/16/02. Progress Through Crises: The Evolution of the IMF’s Statistical Arsenal
Deficiencies in the provision or interpretation of statistical information have been identified as among the contributing factors in several of the major economic crises of recent times. While not a main cause of any particular crisis, these deficiencies acquired enough prominence to trigger formal efforts to correct them, including at the IMF. Thus, the Latin American debt crises of the early 1980s prompted a sharp increase in the Fund’s preoccupation with statistical issues, in particular with the coverage and timeliness of debt statistics. The Mexican crisis in 1994 revealed the importance of timely provision of key information—on international reserves and the central bank’s balance sheet in this case—to both the IMF and financial markets. This led to the establishment of the SDDS and GDDS by which countries voluntarily subscribe to disseminate an agreed set of data (and associated metadata). Deficiencies in the quality and integrity of data—again centered on reserves and external borrowing—were in part behind the Asian crisis of 1997 and led to additional prescribed components of the SDDS, the inclusion of a data module in the ROSC process, and the development of a Data Quality Assessment Framework. At the same time, the perceived urgency of strengthening the capability for early detection of crises led to the establishment of the very dataintensive FSAP and Vulnerability Exercise. Finally, the recent global financial crisis gave renewed impetus to efforts to strengthen the IMF’s statistical arsenal, with the Fund participating actively in the G20 Data Gaps Initiative and expanding anew the scope of the SDDS through the creation of the SDDS Plus, a higher tier aimed at systemically important countries.
BP/16/03. Old Acquaintances: Past Views on Data Problems in the IMF
Problems related to data have been almost a constant throughout the history of the Fund. Whether exogenous (i.e., due to deficiencies in the data provided by third parties or generated by emerging data needs) or endogenous (derived from flawed institutional practices), data issues have been identified and documented on numerous occasions. Likewise, the impact of these problems on the Fund’s performance in delivering on its mandate has been long known, yet despite repeated attempts to address some of these concerns, pervasive problems persist. This paper reviews the most prominent data issues in recent years (2007–15), as reflected in both IMF documents and previous IEO evaluations. While these documents focused on different topics, data problems were, at times, explicitly recognized as affecting findings or recommendations.
BP/16/04. Inadequate Statistics and Faulty Analysis
The IMF’s economic and financial analysis and the quality of its policy advice and economic programs are predicated on the availability of timely, accurate data. By and large, the process of data provision to the Fund works well: within the capabilities of their national statistical systems, countries provide a vast amount of information that is in most cases reliable and available within a reasonable period of time. Nevertheless, there have been instances where data inadequacies have led to a wrong assessment of a country’s situation and hence to incomplete or inappropriate policy recommendations. Based on bad data, staff may have provided a more positive assessment of a given economic situation than warranted—misleading both the country’s population and the international community—or may have given policy recommendations that unnecessarily postponed needed adjustments. Instances of data that subsequently prove to be wrong or incomplete are probably quite frequent, but usually of little consequence and therefore go unreported. However, this paper discusses several cases where staff documented that their analysis had been adversely affected by faulty data. Most of these cases involved the fiscal deficit and its financing, and the level and liquidity of the central bank’s international reserves.
BP/16/05. On the Effect of IMF Data Standards Initiatives: Do They Affect Foreign Direct Investment, Exchange Rate Volatility, and Sovereign Borrowing Costs?
The IMF’s Data Standards Initiatives—in particular, the SDDS and GDDS—are designed to help countries improve their data dissemination practices and, in the process, increase transparency about the macroeconomic and financial situation of participating countries, reducing noise-to-signal ratios for investors. IMF research suggests that subscription to these initiatives can have significant positive effects on selected international financial variables, including foreign direct investment inflows, exchange rate volatility, and sovereign bond spreads or yields. This paper evaluates the robustness of these findings using both the same raw dataset used by the IMF authors and an updated dataset that incorporates revisions, additional countries, and more recent periods. In both cases, the data were adjusted for potential problems that may have been previously overlooked–nonseasonally adjusted quarterly data and measurement errors. The original econometric models, as well as models with different specifications that controlled for additional factors and/ or estimated with different methods, were applied to both datasets. The results indicate that the IMF findings are, in general, not robust. They were often based on potentially problematic transformations of the data that, when removed or corrected, substantially changed the original conclusions. Nor do the results seem robust to changes in the sample. In some instances, this may reflect insufficient consideration of the effect of factors other than IMF data initiatives–such as global developments that may affect all countries, or time dependency. One conclusion–that participating in the SDDS helps reduce exchange rate volatility–may reflect a misinterpretation of the original results. Although the favorable impact of the SDDS on sovereign borrowing costs failed to stand up to some of the robustness checks, it appears to be relatively more immune to tests based on “cleaned” data and alternative econometric specifications.
BP/16/06. Data and Statistics at the IMF: Quality Assurances for Low-Income Countries
How does the IMF deal with the challenge of obtaining timely, high-quality data for its operational purposes? This paper examines the different ways the IMF performs quality assurances on macroeconomic statistics for internal and external use. It focuses on how the IMF handles data and metadata on countries that are classified as low income because these countries tend to face the greatest resource constraints in producing and disseminating the high-quality macroeconomic statistics and metadata needed to fully support the IMF’s surveillance and financial programs. The paper takes up two issues that have been highlighted in previous IMF reviews on statistics. The first is whether reputational risks derive from the IMF’s dissemination of data that may be of questionable quality, given that data users often cannot distinguish IMF data from official country statistics. The second is whether the IMF incurs a further reputational risk when the data it reports in its various databases and reports are not consistent.
BD/16/01. How Well Is the IMF Doing on Data? Evidence from Surveys
This background document presents the evidence gathered by the IEO for the evaluation from surveys of three groups of stakeholders: (i) IMF staff, (ii) external users of data that are published by the IMF, and (iii) providers of country data to the Fund (mainly country authorities). External users hold IMF-provided data in high regard, but there is a widespread misconception that the Fund monitors and endorses the quality of the data it disseminates. Data providers are generally satisfied with the reporting process, although there is a significant lack of familiarity with the Fund’s data-related procedures, especially in the area of data quality monitoring. Nearly all data providers assess the Fund’s technical assistance and training in the statistical domain very positively. According to IMF staff, source-data issues continue to adversely affect the conduct of the Fund’s core operations (surveillance and lending), and current quality-monitoring systems are questionable. While there is considerable interest in centralized provision of statistical services, STA’s work is largely unknown and far from meeting the expectations of other departments. The positive potential of recent internal data management initiatives—a move to structured databases, implementation of a common surveillance database and economic registry, and new governance structure—is recognized by some IMF staff, but largely unknown to the majority (as of February-March 2015, when the survey was conducted, albeit almost four years after the launching of the initiatives). Overall, IMF staff are reasonably satisfied with the data available for their work, although they highlighted gaps in some areas, most notably for balance sheet analysis and on macro-financial linkages.