The post-COVID survey had more options for selection, including more timely and frequent analysis and engagement, consideration of unconventional policies, and use of digital communication tools.
For more details on constructing the sentiment index, please refer to the corresponding IMF working paper.
The database we compiled includes about 2600 Article IV staff reports for all member countries from 2000 to 2018. The sample also includes reports that are combined Article IV consultation and program review.
In order to do so, we first tapped into the IMF’s enterprise knowledge known as the Enterprise Business Vocabulary (EBV). However, to be able to identify exact concepts and phrases as they appear in Staff Reports, we trained a word2vec model on IMF documents.
We assign a value of +1 to agree, -1 to disagree and 0 to partially agree.
Please refer to Devlin, Chang, Lee, and Toutanova (2019) for more details on BERT.
While all the results presented are from Article IV SRs, we applied our trained model to estimate sentiments in Buff statements, which are another expression of authorities’ views. Model accuracy increases from 81 to 85 percent, and our results generally hold.
For the Fund-supported program years, we use the combined Article IV and program review/request documents to gauge sentiments captured by the surveillance cycle during and after Fund-supported programs.
We do not compare countries that received CD and FSAPs vs. those who have not, as there are very few countries that have not received those services during the sample period. If instead we compare for every year average sentiments in those who have and those who haven’t received CD and FSAPs separately, we find that sentiments are indeed higher in those who have.
The variables we use here are from the Database of Political Institutions (DPI) 2017 published by the Inter-American Development Bank. Specifically, we use data on (i) political systems (presidency, assembly-elected president, parliamentary), (ii) election cycles (number of years until the end of the term), and (iii) whether the party of executive control all relevant houses.
All the results listed are statistically significant. Please refer to the corresponding WP for details.
The database was developed by the Development Issues Unit of SPR and Marcio De La Cruz Gomez.
The total number of policy recommendations analyzed is 2,449. Information on policy recommendations comes from the policy discussions section and the appendix on the implementation of main recommendations in Article IV staff reports. Information on the implementation status comes from the policy discussions section, staff appraisals, Executive Directors’ Statements to the Executive Board in Article IV staff reports.
The ‘N/A’ category corresponds to the recommendations that were not followed up in subsequent consultations or their implementation status was not recorded in the main text or the appendix on the implementation of main recommendations in Article IV staff reports.
Factors in AEs such as greater financial sector development, better financial data collection, and larger number of FSAPs could explain the richer financial sector advice in AEs.
The sample includes Canada, New Zealand, Republic of Korea, and Singapore. Other AEs (France, Spain, Kingdom of the Netherlands, Cyprus, Slovak Republic) are in currency unions and there is no individual monetary policy advice.
IEO, 2019, “IMF Advice on Unconventional Monetary Policies, Evaluation Report 2019.”
The analysis in this section was conducted by the Communications department (COM), in consultation with SPR.
For instance, the April 2017 WEO Chapter 3 on hollowing out of the middle class, and the October 2017 FM chapter on inequality.
We identified the number of downloads and number of media mentions per flagship per year from 2014 – 2018. Download numbers were provided by Omniture, and included downloads of all LOE (language other than English) versions. Media mentions were provided by TrendKite, and the search construct was IMF keywords AND Flagship keywords across all seven official languages of the Fund. We divided the download numbers and media mentions of flagships by the number of chapters produced each year to account for different quantities of content. Additionally, we identified the number of media mentions for Article IV staff reports per year from 2014 – 2018. We did not divide by the number of chapters per published year to account for differing quantities of content. The search construct was IMF keywords AND staff report keywords, conducted in all seven official languages of the Fund.
The Bank of Canada’s comparison of speeches by the IMF, BIS, and other major central banks suggests greater readability of Fund public speeches compared to the others (https://www.bankofcanada.ca/2018/06/staff-analytical-note-2018-20/).
To gain insight on readability of IMF products, we used the Flesch-Kincaid Grade Level score to examine the average readability of the WEO (chapter 1) and associated messaging products. The “Flesch–Kincaid Grade Level Formula” indicates the years of education generally required to understand the text, with a score of 12 indicating that the text is expected to be understood by a U.S. high school graduate.
The tables and boxes of the WEO Ch 1 were not included in analysis. BoE estimate sourced from speech given by Andrew Haldane in March 2017, “A little more conversation, A little less action,” which inspired this analysis. See speech, here: https://www.bankofengland.co.uk/-/media/boe/files/speech/2017/a-little-more-conversation-a-little-less-action.pdf. The Economist and Wall Street Journal scores were secured using an online readability tool www.webfx.com/tools.
The date range for the flagships was 30-days following the publication date of the main chapter. Download number includes all published chapters, including associated translations and data files.
The Lending Tracker and Special Note Series launched in April.
Apart from being cases of good traction at the time periods considered, the sample chosen is meant to be representative of all income groups and regions. In addition, the sample includes commodity exporters, a factor shown to affect traction, as well as G20 countries.
As a caveat, these case studies are not necessarily illustrations of member countries with high traction of Fund surveillance, but where it was strong from the country teams’ perspective during the specific period mentioned. However, it should be noted that our sentiment index picks up an increase in sentiments for many of those countries around the time period considered.
Recent reforms implemented in the Human Resource Department (HRD) would help in monitoring and improving incentives for responding to member needs going forward. For instance, HRD tracks and reports periodically to Management the average mission chief tenure; has initiated a new mobility framework with emphasis on service in assignments on low income and fragile countries, and introduced a knowledge-sharing tool to support mission chiefs in meeting the program needs of member countries.