Data problems are well known …
103. In general, the Fund has been able to rely on a large amount of data of sufficiently acceptable quality. Nonetheless, this evaluation finds—as have other reports in the past—that data deficiencies still affect the Fund’s strategic operations (Figure 14). In particular, inadequate data and data practices have implied that the Fund has been, at times, not fully equipped to play its critical role of helping to secure global macro-financial stability.96
104. These data deficiencies stem from diverse factors. Some have their origin at the very source of the data: member countries. Many of them lack the necessary technical capacity or resources to produce the timely, good quality data essential for economic analysis; others are reluctant to share certain data with the Fund; and all prefer to use the methodology that best suits their own domestic situation, posing difficulties for data comparability. In addition, there will always be data gaps. At times, the data are not produced—by countries or markets—and, in some other instances, the necessary conceptual framework for the “required” data is not even developed. That said, the amount and quality of data available to the Fund have markedly improved over time, in part due to the Fund’s own capacity-building activities.
105. Within the Fund, effective flows of data have been hampered by internal institutional constraints. In general terms, data management in the Fund has lacked coordination and relied on weak structures, resulting in a proliferation of databases and making data sharing cumbersome. Moreover, incentives for staff to pay due attention to data are largely absent. At the same time, STA is disconnected from the rest of the Fund and focused largely on external activities. Finally, the systems in place to identify and address faulty or inadequate data do not work properly.
106. In its role as data disseminator, STA adds only marginal value by re-disseminating “official” data that are, for the most part, already in the public domain and easily available given technological advances. Besides, the Fund risks its credibility and reputation due to comparability and consistency issues in the data it disseminates. Relatedly, an open data policy has become best practice in academia and comparable institutions, while the Fund has lagged significantly behind.
… but a number of closely interrelated factors have prevented the success of past initiatives.
107. The problems with data in the IMF have long been recognized, and solutions to address them have accordingly been set in motion. Though some noteworthy progress has been made, many of the obstacles to reform have yet to be tackled, owing to a long history of a piecemeal approach to addressing data issues, compounded by institutional inertia, lack of incentives, organizational rigidities, and long-standing work practices.
108. First and foremost, there is no corporate strategy for economic data in the Fund. Departments, and sometimes even divisions and country teams, have developed their data practices to suit their own needs, largely in isolation from the rest of the institution. Data are still largely viewed as a consumption good (“owned” by the economists that use the data), rather than as a strategic capital asset for the Fund as a whole. For a knowledge-based institution such as the IMF, this is a critical distinction. The lack of a centralized vision has led to duplication of both data and data systems, driving up costs and contributing to reputational risk.
109. An effective data strategy would, as a starting point, need clear and sustained commitment from Management in implementing a vision of how information can strengthen the Fund’s ability to effectively fulfill its ever more challenging mandate. This would be much more than a process-oriented approach focused on data management.
110. A data strategy would thus entail a much broader array of issues, such as (among others): (i) a clear definition (and prioritization) of the scope of the data the IMF needs; (ii) more regular reviews of the minimum set of data required for surveillance; (iii) a discussion of the IMF’s stance vis-à-vis member countries’ statistical systems (e.g., should it press for strengthening national statistics offices? should it play a stronger watchdog role on provision and quality issues? should data quality shortfalls be flagged more forcefully in Fund documents?); and (iv) an institutional view of how the IMF can stay at the forefront of statistical developments (e.g., the future use of big data;97 nowcasting to detect macroeconomic turning points, the growth of unstructured datasets, new technological innovations for delivering data from external sources).
111. Thus, a data strategy would be much more than a data management strategy and the associated information technology and budget issues, although these constitute important components. The data management structure recently put in place has spurred important progress, improving the accessibility and sharing of data. However, these are not ends in themselves; they are merely a means to create operational value. Moreover, these efforts to strengthen data management are still of a fragmented, short-term nature, with major changes being put in place before seeing how they fit into a long-term strategy. This progress faces the risk of not being sustained (as with the many previous attempts listed in Annex 7), if a Fund-wide change does not take place (Box 10).
112. The long-entrenched divisions between STA and other departments constitute another fundamental problem. STA has become largely isolated from other departments and its outputs detached from the Fund’s main operations. This has deprived the Fund of a true service-providing department of statistics such as those that peer international organizations enjoy, and this despite the clear appetite within the staff for this kind of centralized service.98
113. Lack of staff incentives and accountability constitutes another obstacle for good data management. Fund economists want ever more data to do their analyses, yet data management is seen as a low-visibility task without reward. Much of the work has therefore been devolved to research assistants, who typically are on short-term contracts with little opportunity to go on missions to countries. Yet data literacy hinges crucially on both experience and the ability to engage in discussions with country authorities on data issues.
Pitfalls in Building a Data Governance Framework
Statistical Analysis System Institute, a leader in data analytics and management, notes a few of the reasons why data governance fails (see below, where the italicized parenthetical additions translate these into IMF specifics):
The culture doesn’t support centralized decisionmaking (data-related decision making in the Fund is—in sharp contrast with the general culture of the organization—extremely decentralized; for example, the oversight of data management and STA falls under different Deputy Managing Directors).
Organization structures are fragmented, with numerous coordination points needed (each IMF department manages its own data).
Business executives (economists) and managers consider data to be an “IT issue” (many of the past IMF papers on data management were from a largely IT perspective).
Data governance is viewed as an academic exercise.
Business units (area and functional departments) and “technical units” (STA and TGS)1 do not work together.
114. Inadequate incentives have also led to lack of candor in assessments of data adequacy for surveillance. This lack of candor stems from several factors, including insufficient attention to data quality, concerns about undermining the relationship with authorities (including fear of “speaking truth to power,” particularly for advanced or systemically important countries),99 and concerns as to whether surveillance even makes sense if data are termed “inadequate.” Yet candid assessments could induce country authorities to undertake the effort to strengthen the quality and availability of data.
115. In seeming contrast to economists’ apparent lack of interest in data work, the institution as such may be placing too much emphasis on data alone as the solution to understanding economic and financial developments. Thus, more data are always seen as better. This considers only one side of the equation—the demand side—while ignoring the supply side and the costs imposed on staff and on data providers in member countries. Data gaps will unavoidably always exist, not least because of the rapidly evolving global economic landscape. Their existence (and the recognition that statistics, by their very nature, are always retrospective and often produced with considerable delay) underscores the dangers of overreliance on either data (or the associated analytical tools) and the importance of judgment and experience in detecting emerging risks. As John Tukey, a renowned statistician, perceptively noted, an approximate answer to the right question can be more powerful than an exact answer to the wrong question (Tukey, 1962).
116. The improvement of both the quality and comparability of data ultimately depends on the capacity and willingness of member countries, as the Fund has neither the capacity to systematically monitor data quality nor the leverage to push more forcefully for the adoption of statistical standards. Thus, the resulting discrepancies among the Fund’s different outputs may be unavoidable at present but they highlight the importance for the Fund—especially given the heightened relative weight of multilateral surveillance today—to help and encourage countries to strengthen their statistical apparatus and adopt international standards for all the data they report (not just for data reported to STA). Within the limited role of the Fund in this area, in the short term, the gaps in metadata—clearly explaining the sources and attributes of the different datasets—need to be filled, while, with a long-term perspective, the Fund’s capacity-building activities (which are highly appreciated) should continue to contribute to strengthening countries’ statistical systems.
117. Finally, an environment of fiscal austerity, in both the Fund and member countries, has put any focus on data activities on the back burner—in direct contrast to the fact that an increasingly complex, interlinked global economy should place a premium on data issues.
