7. Access by the Public, Integrity of Data, and Quality of Data
- International Monetary Fund. Statistics Dept.
- Published Date:
- December 2013
7.1 This chapter elaborates on the SDDS dimensions of access by the public, integrity, and quality of data. It begins with an overview, followed by discussions on each of the dimensions.
7.2 As noted in Chapter 1, in addition to calling for the dissemination of comprehensive, timely, and reliable information on key economic sectors of the economy, the SDDS requires that ready and equal access to data be provided to the public, that compilers exercise objectivity and professionalism to ensure the integrity of the data, and that compilation methods and sources be disclosed to allow users to judge the quality of data (see also Box 1.1 in Chapter 1). This chapter elaborates on these SDDS dimensions.
Access by the Public
7.3 To promote ready and equal access, the SDDS prescribes (1) the dissemination of ARCs by statistical agencies and (2) the simultaneous release of data to all interested parties. ARCs underscore sound management and transparency of statistical compilation. Data users, including market participants, favor ARCs because such information facilitates the planning and organizing of their analysis and related activities. In a number of countries, ARCs are mandated by statistical laws, policy directives, and administrative guidelines.
7.4 Simultaneous release does not refer to access by government ministries and agencies; such pre-release access is governed by the integrity dimension of the SDDS. Simultaneous release refers to access to all parties on equal terms, with release referring to the first availability of the data to the public. Formats of dissemination of the data include electronic as well as traditional means. Subscribing countries are encouraged to release data in electronic formats to provide simultaneous access to all users. The dissemination of the ARCs and simultaneous release of data are further discussed in Chapters 8–10 under operational aspects of the SDDS.
Integrity of Data
7.5 The integrity dimension of the SDDS calls for transparency of the data agency’s practices and procedures, including administrative ones as well as those for data revisions and changes in methodology, a key factor in statistical practices. The four prescribed practices are (1) dissemination of terms and conditions under which official statistics are produced, (2) identification of internal government access to data before release to the public, (3) identification of ministerial commentary on the occasion of statistical release, and (4) provision of information about revision and advance notice of major changes in methodology.
Dissemination of Terms and Conditions
7.6 The terms and conditions are those under which official statistics are produced, including those relating to the confidentiality of individually identifiable information. This prescribed practice is embodied in the “Fundamental Principles of Official Statistics” adopted in 1994 by the United Nations.1 It is instrumental in fostering confidence in the objectivity and professionalism of official statistics.
7.7 The terms and conditions under which statistical agencies operate may take various forms, including statistics law(s), charters, and codes of conduct. These terms and conditions may refer to matters such as the relationship of the statistical unit to a larger department or ministry. For example, they may distinguish between budgetary and personnel matters and between matters on statistical methodology and decisions about publications of statistics. The terms and conditions may also refer to a requirement that the statistical unit publish data it has collected as a safeguard against possible pressure to keep some findings from the public, the qualifications and terms of reference of the chief statistician/ director, and review of programs by an independent group of experts. All of these may be viewed as safeguards of the professionalism and objectivity of the producing agency.2
7.8 Another key part of the terms and conditions under which statistical agencies operate refers to the procedures and processes related to the confidentiality of individual responses from persons, businesses, and organizations to official inquiries. These procedures and processes may cover topics ranging from computer security to restrictions on interagency disclosure.
7.9 Dissemination of the information about the terms and conditions may take a variety of forms. For example, the annual report of a statistical agency may include a section on steps taken to ensure the confidentiality of individual responses, and the section may refer to the legal obligations of the agency’s employees with respect to confidentiality and security of computer systems and agency buildings. Key publications and Internet sites may describe the basis for an agency’s data collection and dissemination activities and its practices. Several examples are given below:
- [Name of agency] collects, processes, and publishes information on [country’s] demographic, social, and economic aspects. In addition, it produces data on the country’s physical environment and natural resources.
- [Name of agency] was created by [name of presidential decree or legislation], and its sphere of duties is set forth in the [name of legislation; e.g., law on Statistical and Geographical Information], which also provides it with the technical and administrative autonomy needed to carry out its functions.
- [Name of country] has at its disposal many data about people and companies. These individual data are never released to other parties or government agencies. [Name of agency] does not publish or disclose data combinations from which individual data can be derived.
7.10 Survey forms and transmittal letters may carry statements or quote relevant legislation or codes referring to confidentiality of response and assurances that the response will be used only for statistical purposes and will not be divulged to others, including regulatory and tax agencies.
7.11 For posting on the DSBB, subscribing countries are to summarize the terms and conditions for making data available to the public. The terms may vary across data categories, especially in countries with decentralized statistical systems.
7.12 The dissemination of the terms and conditions, as just described, is applicable to official statistics and their producing agencies. Subscribing countries would be considered in observance of the SDDS if a private organization producing any data covered in the SDDS did not provide such identification. For posting on the DSBB, the subscribing country could note “not applicable” in the metadata for such data.
7.13 Countries, however, are encouraged to foster openness about the conditions under which key data are prepared and released. Countries may provide relevant information in the metadata about the terms and conditions under which data are produced by private entities with an appropriate note about the source of the information.
Identification of Internal Government Access to Data Before Release to the Public
7.14 This practice prescribes the listing of persons or officials holding designated positions within the government, but outside the agency producing the data, who have prerelease access to the data and the reporting of the schedule according to which they receive access. The prescribed practice is intended to provide for any necessary prerelease access within government that the government deems appropriate, while giving full transparency to those practices. In this regard, country practices differ. Some countries maintain strict embargoes on the availability of data even within the government prior to their release, while in others such procedures would be viewed as unduly restrictive and detrimental to fast and effective government reaction. Thus, rather than prescribing a specific and uniform set of practices, the SDDS places emphasis on the means by which the necessary degree of transparency can be assured.
7.15 Such identification of prerelease access—that is, statements of “who knows what and when”—may take a variety of forms. The identification may, for example, be included in brief notices to the public, or in annual reports of the producer of statistics, or in both. A brief notice could be along the following lines:
- Data from [name of agency] are available to all users simultaneously with limited exceptions. The exceptions are on a need-to-know basis; these persons receive advance copies of news releases (or related materials) no earlier than 48 hours before the release date and time. The exceptions are: the Head of government, Governor of the central bank, Minister of Finance….
- In addition, other ministers, policy advisers, and, on a need-to-know basis, a limited number of ministerial and central bank staff may be briefed on a confidential basis [specify] hour(s) in advance on the day of release.
7.16 This example highlights that the element is intended to prevent undue—potentially political—influence or undue policy advocacy. This approach is not meant to result in a list of all statisticians and other employees of the producing agency, who of necessity often see data at early stages or in fragments. For posting on the DSBB, subscribing members are to present a summary statement or identify the location of a more detailed, updated list (for example, the press offices of the data producer).
7.17 The identification of internal government access as just described is intended for official statistics and their producing agencies and does not apply to data produced by private institutions. Subscribing members would be considered in observance of the SDDS if a private organization producing any data covered in the SDDS did not provide such identification. For posting on the DSBB, the subscribing member would note “not applicable” in the metadata for data produced by a private institution.
Identification of Ministerial Commentary on the Occasion of Statistical Releases
7.18 Ministerial commentary is not necessarily expected to maintain the same degree of objectivity or freedom from political judgment as would be expected from a producer of official statistics. Therefore, the practice under the SDDS is to identify such commentary so that its source will be transparent to the public.
7.19 The identification of ministerial commentary on the occasion of statistical release may take several forms. One common form is for the minister (or other policy or political official) to issue a statement that is clearly separate from the statistical release—the ministerial statement under the minister’s letterhead and that of the statistical agency under its letterhead or logo. Alternatively, the statistical agency’s material may be presented separately in a release that contains both ministerial commentary and data. The agency’s material may include data, explanatory text (for example, of an unusual event affecting the data), objective analysis, citations of data sources, and the producer’s logos or other insignia.
7.20 Subscribing members are to describe the procedures applicable for each data category for the DSBB. Procedures may vary across data categories, especially in countries with decentralized statistical systems.
7.21 The identification of ministerial commentary as just described is intended for official statistics and their producing agencies. Subscribing countries would be considered in observance of the SDDS if a private organization producing any data covered in the SDDS did not provide such identification. For posting on the DSBB, the subscribing country could note “not applicable” in the metadata for the specific data category/component produced by the private organization.
Revision and Advance Notice of Major Changes in Methodology
7.22 The SDDS emphasizes practices related to revisions that enhance the transparency of the producing agency’s statistical operations. The practices described are intended to give producers of official statistics several ways of providing information about revisions and changes in methodology and thus to strike a balance between user concerns about revisions and the resource cost to producers of providing the information.
7.23 Relevant information about data revisions may include statements about the policy followed and data about the size of past revisions. Revisions are made to incorporate improvements or other changes dealing with any of the several aspects of statistical methodology: analytical frameworks; concepts, definitions, and classifications; accounting conventions; source data; and estimating methods (compilation procedures). Revision policies may refer to any or all of these, as appropriate. For instance, there may be a policy about how often new concepts, definitions, and classifications are incorporated—for example, only at five-year intervals.
7.24 For the incorporation of additional source data, the policy may set out the period over which new sources are introduced—for example, a policy of revisions on a continuing basis (revising monthly data until an annual, more comprehensive survey becomes available) or a policy of making no routine revisions. Data on the size of past revisions may refer to estimates such as dispersion and bias. Such information may be summarized in news releases and may be presented more fully in comprehensive reports or separate periodic evaluations.
7.25 Advance notice of changes in methodology should deal with major changes in methodology as defined above—for example, change in base year, major expansions of sample size, the introduction of alternative data sources, the introduction of new weighting schemes for indices, the introduction or change in the methods of seasonal adjustment, and reclassifications of transactions or industries. The notices may take a variety of forms, ranging from short statements in news releases to presentations and papers in public forums. Members are encouraged, as well, to provide easy access to information about revisions after they are released (for example, telephone, fax, or Internet access to a person able to answer questions about revisions).
7.26 For the DSBB, subscribing countries are to describe the policy, size of past revisions (at least for major aggregates), or other relevant information about revisions (such as the sources of revision and/ or components that show the largest revisions) for each data category and the procedures for providing advance notice of major changes in methodology. The descriptions could, of course, note that no revisions are routinely made and/or that no major changes in methodology have been made.
Quality of Disseminated Data
7.27 Data quality can be difficult to define. One reason is that accuracy and timeliness can be viewed as tradeoffs. For statistics used in tracking short-term developments, timeliness commands a premium, whereas for statistics needed to understand or demonstrate detailed structures or interrelationships, other quality aspects of the data may need to receive more emphasis. Another reason is that few quantitative indicators are available for some series (for example, national accounts series). Under these circumstances, sample size and response rate, which are often used as quantitative indicators of quality for survey-based data, are not applicable.
7.28 To allow users to assess the quality of data, the SDDS calls for (1) the dissemination of documentation on methodology and sources used in preparing statistics, (2) the provision of statistical cross-checks and assurances of reasonableness, and (3) the dissemination of deviations from internationally accepted methodologies in the metadata.
Documentation on Methodology and Sources
7.29 The availability of documentation on methodology and sources underlying statistics is key to users’ awareness of the strengths and weaknesses of the data. The prescribed documentation may take several forms, including summary notes accompanying release of the data, separate publications, and papers available on request from the producers. Subscribing members are encouraged to include and highlight statements about important features of quality. These may indicate errors to which the data are subject, sources of noncomparability over time, measures of coverage for census data or sample error for survey data.3
7.30 For posting on the DSBB, subscribing members are to provide information identifying the documentation and the means of gaining access to it. Subscribers are also to provide summary documentation of methodology on the DSBB, including statements of divergences from internationally accepted statistical methodologies (see below).
7.31 To support and encourage users’ verification of data, the SDDS prescribes the dissemination of component detail, especially for aggregate series, statistical frameworks that support statistical crosschecks and provide assurance of reasonableness, and comparisons and reconciliations with related data.
7.32 Component detail is to be at a level that does not conflict with other desirable characteristics, such as statistical reliability and the confidentiality of individually identifiable information. (Component detail, although used in this context to facilitate cross-checks, is also useful in its own right for in-depth analysis and research.) With a sufficient amount of published detail below the aggregate (or indicator) level, especially in combination with documentation, a user can test the reasonableness of the data. Such detail allows a knowledgeable user to compare rates of change for components of time series, calculate deviations from past trends for components of time series, and calculate percentage composition.
7.33 Statistical frameworks include accounting identities and statistical relationships. Dissemination of statistics within a framework, such as monetary aggregates within the DCS, lets the user of a component that serves as an indicator test the consistency of the indicator with other elements of the framework and with the framework as a whole.
7.34 Comparisons and reconciliations may take different forms. They include those that cut across frameworks, such as exports and imports as part of the national accounts and as part of the balance of payments. They may also include those that reconcile or compare data, such as those on employment that may be prepared from different sets of source data within the same country. In posting the data on the DSBB, subscribing countries are to briefly describe the component detail, the framework, and the comparisons and reconciliations.
Deviations from Internationally Accepted Statistical Methodologies
7.35 To assist users of the data disseminated under the SDDS in assessing their quality, the SDDS also prescribes the dissemination of deviations from internationally accepted statistical methodologies in the metadata.4 Deviations should be specified in the relevant indicators of the metadata (i.e., information describing methodology) posted on the DSBB. In instances where a subscriber has not provided clear metadata on deviations from internationally accepted statistical methodologies, the SDDS nonobservance procedures would apply (see paragraph 10.6).
7.36 SDDS subscribers are also encouraged to undertake and publish a data quality assessment, using recognized data quality assessment tools, such as the Fund’s Data Module of the Report on the Observance of Standards and Codes that uses the Data Quality Assessment Framework, or the Eurostat or European Central Bank data quality monitoring frameworks. Reassessments should take place at no more than seven- to ten-year intervals. Assessments (and reassessments) could be conducted by Fund staff, or alternatively, a subscriber could request another subscriber or external agency to conduct a peer review.