Background papers to the "Strategy for IMF Engagement on Social Spending"

Abstract

Background papers to the "Strategy for IMF Engagement on Social Spending"

Background Paper V. Trends and Patterns in Fund Engagement on Social Spending: A Text Mining Analysis1

This note shows how the discussion of social spending issues has evolved in IMF surveillance and program staff reports since the late 1970s. Discussions of social spending and inequality issues steadily increased until 1999 and stabilized at this higher level until 2008. The dip in 2007–09 coincided with the global financial crisis and the introduction of the 2007 Surveillance Decision. The recovery in the frequency of social spending terms after the late 2000s coincides with the increase in research by IMF staff on social spending issues and its link with inclusive growth. However, there is a wide variation in the intensity of the discussion of social spending issues for both surveillance and program staff reports as well as for various types of program reports. These wide variations in the discussion of social spending is consistent with calls for a more systematic approach to IMF’s engagement on social spending.

A. Introduction

1. This note shows how the discussion of social spending issues has evolved in IMF surveillance and program staff reports since the late 1970s. Based on the intensity of use of social spending terms, social spending issues received increasing attention in the 1980s and 1990s, and have featured prominently in IMF staff reports ever since, with the exception of a 3-year dip during 2007–09 coinciding with the onset of the global financial crisis and the introduction of the 2007 Surveillance Decision. This overall trend is consistent across all types of documents and measures used. A similar trend was observed for in-depth analysis of poverty and distributional issues.

B. Description of the Database Used for Text Mining

2. A database comprising 8,998 IMF staff reports over the period 1979–2018 was compiled for this exercise.2 The main source is the IMF’s Institutional Repository, which includes all documents submitted to the IMF Executive Board. Document information (“metadata”) such as country name, type of reports, and years was first sourced from the Knowledge Exchange website and then completed and corrected through text analysis techniques. The resulting classification was cross-validated with information from the Monitoring of Fund Arrangements (MONA) database for the 1992–2018 period, and SEC Board Calendar Management System for the 2001–2018 period. The database includes 3,836 Article IV staff reports, 1,897 program review staff reports, 1,023 combined Article IV and program review staff reports, and 2,242 Selected Issues Papers (SIPs). In the subsequent analysis, surveillance documents refer only to Article IV staff reports, whereas program documents include both program review staff reports and combined Article IV and program review staff reports. SIPs, which typically cover specific topics, have been included in a separate group. The database covers all staff reports classified as “Public,” “Official Use,” and “Confidential.” A set of 118 “Strictly Confidential” documents was not included in the database.3 Programs are classified according to the official classification of the IMF lending facilities. Facilities financed from the General Resources Account (GRA) are the Extended Fund Facility (EFF), the Precautionary Credit Line (PCL), the Precautionary and Liquidity Line (PLL), the Flexible Credit Line (FCL), the Stand-By Arrangement (SBA), and the Rapid Financing Instrument (RFI).4 For the purposes of this study, staff reports classified as Extended Arrangements (EA) and Use of Fund Resource (UFR) are also labeled as GRA-supported programs. The Policy Coordination Instrument (PCI)—a non-financial arrangement—was also grouped with GRA facilities. Facilities financed through the Poverty Reduction and Growth Trust (PRGT) are the Extended Credit Facility (ECF), the Rapid Credit Facility (RCF), the Standby Credit Facility (SCF).5 For the purposes of this study, LIC facilities available prior to the PRGT—the Exogenous Shocks Facility (ESF), the Structural Adjustment Facility (SAF), Enhanced Structural Adjustment Facility (ESAF), and Poverty Reduction and Growth Facility (PRGF)—are also labeled as PRGT-supported programs. The Policy Support Instrument (PSI)—a non-financial arrangement for low-income member countries—was grouped with PRGT facilities. Also included in PRGT were programs financed under a combination of PRGT and GRA accounts. Ex-Post Assessment (EPA), Staff Monitored Program (SMP), and Post-Program Monitoring were included in either PRGT or GRA depending on the underlying program or the country’s eligibility for PRGT facilities.

3. The number of IMF documents peaked during the global financial crisis, underscoring a marked increase in the number of program staff reports that began in 2004 and began tapering in 2013 (Figure 1). The number of program staff reports increased as a consequence of the global financial crisis but has declined since 2013. The number of Article IV staff reports has remained broadly constant over time. SIPs were introduced in the second half of the 1990s, with their number remaining broadly stable after 2004. Article IV staff reports covered mostly middle-income countries (50 percent) followed by advanced economies (28 percent) and low-income countries (22 percent). The distribution is largely a reflection of composition of member countries across income groups. Program staff reports covered slightly more facilities financed through GRA (58 percent). SIPs had broadly the same distribution as surveillance staff reports (48 percent middle-income, 32 percent advanced economies, and 20 percent low-income).

Figure 1.
Figure 1.

Overview of Fund Documents by Type

(June 1978–June 2018)

Citation: Policy Papers 2019, 017; 10.5089/9781498318907.007.A005

Source: IMF Institutional Repository and IMF staff calculations.Note: Joint is for program reports covering Article IV as well.

C. Methodology

4. The intensity of the discussion of social spending in documents is measured by the term frequency. The term frequency is defined as the number of occurrences in a document of specific terms, normalized by the number of words in that document to account for possible variations of document length over time (and scaled by a factor 10,000 for presentational purposes).6,7 To allow comparisons between countries with a different number of staff reports per year,8 the term frequency is averaged across all staff reports in a given year for each country. The list of social spending-related terms was pre-defined by IMF subject matter experts who examined a representative number of staff reports.

5. To ensure that both general and specific discussions of social spending issues were captured by the exercise, both general and specific terms were included. Overall social spending frequency is the sum of the frequencies of general and specific social spending terms (Box 1). General terms capture broad social spending concepts. Specific terms were grouped into categories such as pension, health, education, etc. The list of terms includes only root words; for instance, a search for “pension” would capture pension, pensions, pensioner.

D. Results

6. The discussion of social spending issues increased over time. The frequency of discussions of social spending issues in IMF documents rose steadily for decades and peaked in 1999 when the Poverty Reduction Growth Facility (PRGF) was introduced. Since then, discussions of social spending have featured prominently in IMF documents, although the streamlining that followed the overhaul of the IMF’s lending and conditionality frameworks during 2009–2011, in tandem with the global financial crisis, appears to have temporarily crowded out attention on social spending issues (Figure 2). The subsequent recovery of social spending issues in all staff reports coincides with renewed IMF research on social spending issues and its link with inclusive growth, and suggests a certain resilience in engagement on social spending issues. Today, engagement on social spending issues remains substantial. On average, Fund documents had about 20 (normalized) occurrences of social spending related terms in 2018,9 although with variation across countries.10 The discussion of social spending issues in surveillance and program staff reports followed a broadly similar pattern. However, particularly after 2000, the normalized frequency count for surveillance staff reports is somewhat higher than for program staff reports.

Figure 2.
Figure 2.

Normalized Frequency Count of Overall Social Spending Concepts

(June 1978-June 2018)

Citation: Policy Papers 2019, 017; 10.5089/9781498318907.007.A005

Sources: IMF Institutional Repository and IMF staff calculations.

7. In both surveillance and program documents, there is wide variation in the intensity of the discussion of social spending issues, and some countries show large repeated references to social spending issues (“outliers”) (Figure 3, panel A). In surveillance documents, the number of staff reports without any discussion of social spending issues has fallen from an average of 33 documents in 1979 to zero after 2012. After 1993, there are only a few instances of reports where such social spending issues are not dealt with (Table 1). For some countries, surveillance documents repeatedly paid relatively high attention to social spending issues (Austria, Belgium, Finland, Netherlands, Norway and Luxembourg).11 For program documents, the number of staff reports without any discussion of social spending issues fell to zero after 1999. In recent years, the discussion of social spending issues in PRGT and GRA program cases diverged, with PRGT programs falling slightly. Social spending issues received substantial attention on a repeated basis in Argentina, Bolivia, Kazakhstan, Malawi and Peru.

Figure 3.
Figure 3.

Normalized Frequency Count-Breakdown by Surveillance and Program and Outliers

(June 1978-June 2018)

Citation: Policy Papers 2019, 017; 10.5089/9781498318907.007.A005

Sources: IMF Institutional Repository and IMF staff calculations.
Table 1.

Normalized Frequency Count-Repeated Outliers

article image

Social Spending Related Terms Used in the Text Mining Analysis

  • Overall social spending concepts is the sum of the general social spending concepts and specific social spending concepts.

  • General social spending concepts include the following terms: social spending, social expenditure, social policy, social protection, social program, safety net, social transfer, social assistance, social benefit, social package.

  • Specific social spending concepts includes the following terms:

    • Pension: pension, retirement, retiree, old-age benefit

    • Health: health, health expenditure, health spending

    • Education: education, education expenditure, education spending

    • Income support: income support, guaranteed minimum income, meal program, food stamp, ration card, voucher

    • Energy subsidy: energy subsidy

    • Other subsidies: food subsidy, agriculture subsidy, consumer subsidy, price subsidy, fertilizer subsidy

    • Other benefits: disability benefit, maternity benefit, child benefit, child allowance, unemployment benefit

  • Distribution analysis concepts includes the following terms: inequality, distribution, redistribution, distributional, poverty, vulnerable, Gini, income decile, income quantile, regressive, progressive.

8. The frequency of discussion of social spending issues varies across income groups and regions (Figure 4). For surveillance documents, social spending issues are discussed more in high-income countries. Differences are less pronounced between income groups for program staff reports. For surveillance documents, social spending issues are discussed more in reports for European countries followed by countries from the Western Hemisphere. For program documents, the differences among regions are again less pronounced.

Figure 4.
Figure 4.

Normalized Frequency Count of Overall Social Spending Concepts

Citation: Policy Papers 2019, 017; 10.5089/9781498318907.007.A005

Sources: IMF Institutional Repository and IMF staff calculations.

9. The specific social spending topics that have received most attention are related to pensions, health, and education (Figure 5). There are, however, variations among country groups, with pension-related issues figuring most prominently in advanced economies, and health and education issues more prevelant in low-income countries. Energy subsidies, social assistance, income support, and other subsidies are less discussed, except for Middle-Eastern countries where subsidy issues were raised more regularly.

Figure 5.
Figure 5.

Normalized Frequency Count of Overall Social Spending Concepts by Topic

Citation: Policy Papers 2019, 017; 10.5089/9781498318907.007.A005

Sources: IMF Institutional Repository and IMF staff calculations.

10. Discussion of distributional issues follows a broadly similar pattern over time as the discussion of social spending issues. Discussion of distributional issues increased sharply in program staff reports between 1999 and 2005, declining thereafter with a dip in 2009 (Figure 6). Overall, the discussion of distributional analysis is correlated with the discussion of social spending issues and is more prominent in program staff reports than in surveillance staff reports. Within program documents, distributional issues are more discussed in PRGT programs than in GRA programs. Unlike the discussion of distributional issues in program staff reports, the discussion of distributional issues in surveillance documents has continued to increase in recent years, with a similar pattern shown in SIPs, and many repeated outliers. Positive outliers tend to be staff reports for low and middle-income countries for both surveillance and program staff reports.

Figure 6.
Figure 6.

Normalized Frequency Count of Distribution Analysis Concepts

(June1978-June2018)

Citation: Policy Papers 2019, 017; 10.5089/9781498318907.007.A005

Sources: IMF Institutional Repository and IMF staff calculations.

References

2

See Guillaume et al. (forthcoming).

3

A set of 187 program and surveillance documents with no specific country focus such as reviews of facilities was not included in the analysis.

4

The GRA arrangements comprise a variety of lending programs available to all Fund members with different disbursement schedules and maturities depending on the balance of payment needs of the member.

5

The Poverty Reduction Growth Trust (PRGT) Arrangements are lending programs providing concessional financing support to low-income countries.

6

The term frequency approach was preferred to the topic modeling approach. Topic modeling is an unsupervised approach to identify keywords and common patterns representing topics throughout document collections using hierarchical probabilistic models. In topic modeling, there are no benchmarks to measure against and no direct method to adjust parameters. Moreover, the number of possible topics is limited, whereas staff reports cover a very broad range of topics over time.

7

Sensitivity analysis shows that the results in the trend of social issues discussed in IMF programs are broadly uncorrelated to the length of IMF documents. The length of the staff reports dropped from 2,015 words on average in 1982 to 974 words in 1984, and from 1,950 words on average in 2005 to 1,224 words in 2007. It is, therefore, important to account for these changes in the length of staff reports. The “normalized average frequency count per document” used as indicator in this analysis has a flatter shape and a less pronounced dip between 2007 and 2009 than the unnormalized indicator (“average frequency count per document”). However, overall, both indicators have broadly similar trends.

8

There is only one Article IV staff report at most per year whereas there would be several program review staff reports (two or more depending on the frequency of the review cycle). To ensure comparability, the frequency count is the average of the normalized term frequency for all the staff reports for a country in a year.

9

Equivalent to about 40 unnormalized occurrences per report.

10

In addition to the main text of the staff report, program staff reports include often the letter of intent, the memorandum of economic and financial policies, the technical memorandum of understanding and the data reporting requirements.

11

The discussion of social spending issues in SIPs follows a similar pattern to Article IV documents.

A Strategy for IMF Engagement on Social Spending: Background Papers
Author: International Monetary Fund. Fiscal Affairs Dept., International Monetary Fund. Strategy, Policy, &, and Review Department
  • View in gallery

    Overview of Fund Documents by Type

    (June 1978–June 2018)

  • View in gallery

    Normalized Frequency Count of Overall Social Spending Concepts

    (June 1978-June 2018)

  • View in gallery

    Normalized Frequency Count-Breakdown by Surveillance and Program and Outliers

    (June 1978-June 2018)

  • View in gallery

    Normalized Frequency Count of Overall Social Spending Concepts

  • View in gallery

    Normalized Frequency Count of Overall Social Spending Concepts by Topic

  • View in gallery

    Normalized Frequency Count of Distribution Analysis Concepts

    (June1978-June2018)