Front Matter Page
IMF POLICY PAPER
USE OF THIRD-PARTY INDICATORS IN FUND REPORTS
November 2017
IMF staff regularly produces papers proposing new IMF policies, exploring options for reform, or reviewing existing IMF policies and operations. The following documents have been released and are included in this package:
A Press Release summarizing the views of the Executive Board as expressed during its November 10, 2017 consideration of the staff report.
The Staff Report, prepared by IMF staff and completed on October 12, 2017 for the Executive Board’s consideration on November 10, 2017.
The IMF’s transparency policy allows for the deletion of market-sensitive information and premature disclosure of the authorities’ policy intentions in published staff reports and other documents.
Electronic copies of IMF Policy Papers are available to the public from http://www.imf.org/external/pp/ppindex.aspx
International Monetary Fund
Washington, D.C.
Front Matter Page
USE OF THIRD-PARTY INDICATORS IN FUND REPORTS
October 12, 2017
Executive Summary
Fund staff use indicators developed by other organizations as input into analysis in surveillance and, to a lesser extent, in program work. While the Fund has been able to rely on data and statistics provided by member countries and compiled internally, continued efforts to foster global economic and financial stability require staff to work with indicators drawn from numerous third-party compilers. These indicators of varied qualities are used to measure concepts such as business environment, competitiveness, and quality of governance.
It is anticipated that staff will continue to draw on other institutions’ expertise and estimates. This practice is consistent with the Executive Board’s guidance in areas where internal expertise is lacking or limited. It also puts a premium on staff’s understanding of the third-party indicators (TPIs) used to add analytical value, avoid flawed conclusions and presentation, and support traction with the membership.
This paper outlines a framework to promote best practice with respect to use of TPIs in Fund reports. The framework will apply to all documents that are subject to the Fund’s Transparency Policy. Staff are encouraged to follow similar guidelines for other Fund documents. It draws on lessons from the current practice in the Fund and other selected international organizations (IOs), and insights from the application of an adapted data quality assessment framework (DQAF) to a subset of TPIs commonly used by Fund staff. Common good practices across IOs include the emphasis on staff judgment, review, and consultation with stakeholders.
The proposed framework has three elements: (i) a principles-based Guidance Note (forthcoming); (ii) an “Indicators Digest” that compiles selected indicators’ characteristics to inform staff’s judgment, based on the adapted DQAF; and (iii) the review process to ensure best practice in Fund reports.
The forthcoming Guidance Note will provide specific proposals for strengthened staff guidance, based on the framework above. It will emphasize (i) transparency when selecting indicators and interpreting results; (ii) robustness, to promote indicators as one of many inputs and use of multiple indicators and sources to measure similar concepts; and (iii) reflection of stakeholders’ views. In applying these principles, staff would consider a number of factors, including the assurances of integrity, methodological soundness, accuracy, reliability, and accessibility of TPIs, and compiler’s characteristics (e.g., is it an international organization with a broad membership).
Approved By
Martin Mühleisen and J. R. Rosales
Prepared by the Strategy, Policy, and Review Department and Statistics Department, in consultation with other departments.
Contents
INTRODUCTION
CURRENT PRACTICES IN USING THIRD-PARTY INDICATORS
A. Current Practices in the Fund
B. Practices in Other International Organizations
C. Lessons Learned
PRINCIPLES FOR BEST PRACTICE FOR THE FUND
A. Principles for Best Practice
B. Other Specific Guidance
THE THIRD-PARTY INDICATORS DIGEST
A. Purpose, Scope, and Coverage
B. The Methodology
EXAMPLES OF GOOD PRACTICES
CONCLUSIONS
ISSUES FOR DISCUSSION
References
BOXES
1. Use of TPIs in Selected OECD Country Economic Surveys: 2014–17
2. Example of Good Practices in Using Third Party Indicators in Surveillance 2014–16
FIGURES
1. Current TPI Practices at the Fund: Results of Text Mining Exercise
2. Other Specific Guidance
3. Adapted Data Quality Assessment Framework
TABLE
1. Indicators Digest: Summary of Indicators
ANNEXES
I. The Continuum of Data Types Underpinning TPIs
II. Third-Party Indicators Text Mining Exercise
Front Matter Page
Press Release No. 17/456
FOR IMMEDIATE RELEASE
November 22, 2017
International Monetary Fund
Washington, D.C. 20431 USA
