Chapter

ANNEX 2 Quantitative Analysis

Author(s):
International Monetary Fund. Independent Evaluation Office
Published Date:
April 2007
Share
  • ShareShare
Show Summary Details

This annex extends and deepens the empirical analysis presented in Chapter 2, the sections on “Accommodation of Aid” and “Analysis of Aid” of the main report. It discusses some developments over time and provides additional evidence on differences between strong and weak performers; between PRGF- and ESAF-supported programs in SSA (SSA PRGF and SSA ESAF); and between programs in SSA and other regions (non-SSA PRGF and non-SSA ESAF). The first section presents the findings. The second section discusses the underlying data and methodology.

Findings

This section is structured as follows. It first discusses trends in program aid forecasts. Next, it examines trends in program design with regard to current account and fiscal deficits. The section concludes with an analysis of the relationship between programmed aid and the programmed current account and fiscal deficit.

PRGF aid forecasts

Aid projections in SSA PRGFs for the initial program year were slightly optimistic (see the section “Forecasting Aid Inflows” of the main text). In SSA actual aid levels including debt relief have fallen short of aid predictions for the initial program year (t0) (see panel B of Figure A2.1).1 A driver of this could be overoptimism regarding the timing of debt relief, since aid in t0 net of debt relief and related actions does not seem to be systematically overestimated. Aid in the initial program year to non-SSA countries is also not overestimated.

However, the IMF underpredicted medium-term aid inflows in SSA PRGFs (see the section “Forecasting Aid Inflows” of the main text). This holds true for aid with or without debt relief (see panel B of Figure A2.1).

Figure A2.1.Programmed and Actual Aid Levels in Sub-Saharan Africa Programs

(In percent of GDP)

Source: IEO staff estimates based on IMF, MONA database.

Note: Observations: PRGF 26 and ESAF 23; filters: initial error <1, maximum error 20 percent of GDP.

ESAFs and PRGFs consistently anticipated medium-term tapering of aid flows (panels A and B of Figure A2.1). However, during the PRGF period, average actual aid flows (or updated projections) to SSA program countries in the three years following the program year turned out to be considerably higher than projected.2 This was not the case for aid including debt relief during ESAFs.

Medium-term underprediction of aid was not observed outside of SSA. Figure A2.2 shows that in other regions, average outcomes followed PRGF projections for the outer years more closely.3

Figure A2.2.Programmed and Actual Aid Levels in Sub-Saharan Africa and Other Regions1

(In percent of GDP)

Source: IEO staff estimates based on IMF, MONA database.

Note: Observations: SSA PRGF 26, non-SSA PRGF 9, SSA ESAF 23, and non-SSA ESAF 15; filters: initial error <1, maximum error 20 percent of GDP.

1 Aid includes debt relief, rescheduling, and arrears.

IMF program design appears to have caught up with the increased persistence of aid flows to SSA by reducing programmed tapering in recent years. Figure A2.3 displays annual average differences between aid levels that were projected for the program year and for two years later. It shows the development over time for the overall average and for two groups of countries distinguished by macroeconomic performance.4 As illustrated, programmed aid tapering in the medium term has decreased over time, especially in programs of countries with good macroeconomic performance.

Figure A2.3.Trends in Programming the Persistence of Aid Flows to Sub-Saharan Africa

(In percentage point of GDP difference)

Programmed Tapering of Aid Flows in SSA (t + 2 - t0)

Source: IEO staff estimates based on IMF, MONA database.

Note: 186 observations spread out over the entire period; filter: aid >0 percent.

Program design

This section discusses programmed current account and fiscal deficits. The discussion of the latter is further disaggregated into programmed public spending and domestic revenue generation. For each of these dimensions, developments over time are discussed as well as how outcomes relate to program design.

Current account

The IMF became more accommodative regarding current account adjustments. Figure A2.4 shows the average programmed change in the nonaid current account during the program year in SSA. While under early ESAFs, sharp adjustments of the current account deficit in the program year were common, average adjustments in PRGFs have hovered around zero.

Figure A2.4.Programmed Current Account Adjustments in Sub-Saharan Africa

(In percentage point of GDP difference)

Nonaid Current Account Deficit: Difference Between t0 and t - 1

Source: IEO staff estimates based on IMF, MONA database.

Note: 305 observations spread out over the entire period; filter: abs(ΔCA[t0– t–1]) <20 percent.

The typical SSA PRGF envisaged a medium-term consolidation of the nonaid current account deficit. However, Figure A2.5 shows that, in the medium term, realized current account deficits were greater on average than programmed—financed by the higher-than-expected aid inflows discussed above.

Figure A2.5.Average Current Account Deficits in Sub-Saharan Africa PRGF Programs1

(In percent of GDP)

Source: IEO staff estimates based on IMF, MONA database.

Note: 34 observations; filters: initial error <2 percent, maximum error <20 percent.

1 Before grants and interest payments.

Fiscal balance

SSA PRGFs—in contrast to ESAFs—allowed for increases in expenditures during the program year. Figure A2.6 shows that the programmed difference in expenditures5 between the program year and the year before has shifted upward from tightening in the earlier ESAF years to accommodation of increased expenditures during PRGF.

Figure A2.6.Programmed Expenditure Adjustments in Sub-Saharan Africa

(In percentage point of GDP difference)

Source: IEO staff estimates based on IMF, MONA database.

Note: 306 observations spread out over the entire period.

But like ESAFs, SSA PRGFs envisaged medium-term consolidation of expenditures (see Figure A2.7). However, outcomes show that this consolidation did not materialize, as expenditures increased instead. This difference between programmed and actual medium-term expenditures was again financed by higher-than-expected aid levels.

Programmed domestic revenue mobilization in SSA PRGFs was both more ambitious and more successful than in ESAFs (see panel B in figure A2.7). The average PRGF-supported program in SSA envisaged a 2 percent increase of revenues as a share of GDP over the course of four years. ESAFs, by contrast, programmed domestic revenues to move largely in line with GDP. In terms of outcomes, PRGF programs generated more revenue than ESAFs.

Figure A2.7.Programmed and Actual Expenditures and Revenues in Sub-Saharan Africa ESAFs and PRGFs

(In percent of GDP)

Source: IEO staff estimates based on IMF, MONA database.

Note: 27 observations for ESAFs and 28 observations for PRGFs; filter: initial error < 1 percent of GDP.

On average, SSA PRGFs envisaged slight increases during the program year in the primary fiscal deficit before grants. As shown in Figure A2.8, some fiscal expansion during the program year was allowed in PRGFs but such expansion was uncommon during the ESAF period.

Figure A2.8.Programmed Fiscal Adjustments in Sub-Saharan Africa

(In percentage point of GDP difference)

Nonaid Fiscal Deficit: Difference Between t0 and t - 1

Source: IEO staff estimates based on IMF, MONA database.

Note: 296 observations spread out over entire period; filter: abs(ΔFD[ t0 – t – 1]) < 20 percent.

Programmed use of aid increases

This section looks at the correlation between projected aid increases and the changes in the nonaid current account deficit and the nonaid fiscal deficit. It first introduces the underlying conceptual framework and then discusses the findings.

Conceptual framework: Estimation of “spend” and “absorb” ratios

This section is based on the analytical framework of “spending and absorption of aid increases” suggested by Berg and others6 and discussed in Box 2.1 of the main report. The analysis starts from the premise that from a balance of payments perspective assuming zero nonaid net capital flows, aid increases can either be used to (1) widen the current account deficit before grants and interests (in IMF parlance, this is labeled “absorption” of aid); or (2) increase real reserves. From a fiscal perspective assuming zero nonaid external financing, the additional resources that come in the form of increased aid can either be used to (1) widen the primary fiscal deficit before grants (in IMF parlance, this is labeled “spending” of aid); or (2) substitute for net domestic financing.

The difference between aid spent and aid absorbed determined the envisaged private sector response to aid increases. If the fiscal deficit moves in line with the current account deficit, the increased fiscal demand is balanced by increased net imports. If the widening of the fiscal deficit exceeds that of the current account deficit and the aggregate supply is fixed, this leads to crowding out of the private sector. Correspondingly, a fiscal deficit that widens less than the current account deficit enables crowding-in or, if there is excess domestic demand, allows for disinflation by closing the gap between aggregate demand and aggregate supply.

A series of regressions produced estimates of programmed average spending and absorption of aid increases in SSA PRGFs. To illustrate the methodology, Figure A2.9 plots programmed current account and fiscal adjustments7 against anticipated aid increases. The observations are drawn from PRGF program requests and reviews between 1999 and 2005 for all SSA countries. Regressions on this data with suppressed constants produced slope estimates of the current account and fiscal responses to increases in aid (see the table in Figure A2.9).8 These are the basis for absorption and spending estimates for incremental aid, respectively, used throughout this report.9 Estimates of the proportion of incremental aid used to reduce domestic debt are derived from the difference between spending and absorption, and estimated reserve accumulation from the difference between absorption and the aid increase.

Figure A2.9.Derivation of Estimates for Spending and Absorption of Unanticipated Aid in Sub-Saharan Africa PRGFs

Note: p value in parentheses; * **, and *** significant at 10 percent, 5 percent, and 1 percent levels, respectively; filter: 0 < Δ(Aid[t0 – t – 1] < 10 percent.

Source: IEO staff estimate based on IMF, MONA database.

Findings

As reported in the section on “Current account adjustment” of the main text, initial levels of reserves were a key driver of differences in programmed absorption of aid increases in SSA PRGFs. Figure A2.10 shows that countries with reserve levels below 2.5 months of imports were programmed to use almost all of the anticipated aid increases (95 percent on average10) to raise reserve levels. Programs for countries with higher initial reserve stocks, on the other hand, allowed for the full use of the additional resources to increase net imports. The coefficients of the first regression reported in the table in Figure A2.10 show that this difference stems from differences in both adjustments that were independent of changes in aid and from a steeper response to aid increases for countries with high reserves. This adds up to a significant difference between the two groups of countries depending on reserves level in the constrained model.

Figure A2.10.Spending and Absorption in Sub-Saharan Africa PRGFs: Importance of Initial Conditions

Note: Thresholds are reserves of 2.5 months of imports and inflation of 5 percent; p values in parentheses; *, **, and *** significant at 10 percent, 5 percent, and 1 percent levels, respectively; filter: 0 < ΔAid[t0 – t –1] < 10 percent.

Source: IEO staff estimates based on IMF, MONA database.

As reported in Chapter 2, section on “Fiscal adjustment,” initial inflation levels were key determinants of SSA PRGF program approaches to the spending of incremental aid. Figure A2.10 illustrates that observed inflation before the start of a program influenced the average programmed spending for countries with high reserves. Countries with inflation levels below 5 percent11 were allowed to spend almost all the anticipated aid increases (79 percent on average12), while countries with higher inflation were programmed to use nearly all incremental aid (85 percent on average13) to reduce domestic financing, instead.

The criteria typically used to identify “mature stabilizers” had a significant impact on programmed spending of aid increases but not on their absorption. Programs in countries with initial conditions of single digit inflation, positive growth, and domestic financing below 1 percent of GDP envisage spending increases of about half of anticipated aid increases, on average. This compares with almost no spending of incremental aid increases (17 percent on average14) in countries that do not meet these conditions (see Figure A2.11).

Figure A2.11.Spending and Absorption in Sub-Saharan Africa: The “Mature Stabilizer” Performance Criteria1

Note: p values in parentheses; *, **, and *** significant at 10 percent, 5 percent, and 1 percent levels, respectively; filter: 0 < ΔAid[t0 – t – 1] < 10 percent.

Source: IEO staff estimates based on IMF, MONA database.

1 Following the Fund’s definition of “mature stabilizers,” good performance was defined as inflation below 10 percent, positive growth, and domestic financing below 1 percent before the program.

There were little differences between average programmed spending and absorption of aid increases in ESAFs and PRGFs in SSA. The negative constant and the positive coefficient of the PRGF dummy in the table in Figure A2.12 suggests that there has been a shift from sharp fiscal adjustments independent from anticipated aid inflows in ESAFs toward less such adjustment in PRGFs. However, differences in the programmed response to aid increases mitigate this difference leading to only slight differences between ESAFs and PRGFs in the average programmed spending (Figure A2.12).15 Meanwhile, there is no significant difference in aid absorption between ESAFs and PRGFs.

Figure A2.12.Spending and Absorption in Sub-Saharan Africa: ESAF Versus PRGF

Note: p values in parentheses; *, **, and *** significant at 10 percent, 5 percent, and 1 percent levels, respectively; filter: 0 < ΔAid[t0 – t –1] < 10 percent.

Source: IEO staff estimates based on IMF, MONA database.

The average non-SSA program used a smaller proportion of aid increases to reduce domestic debt and/or for disinflation than its SSA counterpart. Tighter average aid-independent adjustments in the current account and fiscal deficits in non-SSA PRGFs are outweighed by greater responsiveness to aid increases.16 As illustrated in Figure A2.13, these differences lead to slightly more average spending and less average absorption in non-SSA countries than in SSA.

Figure A2.13.Spending and Absorption in PRGFs: Sub-Saharan Africa Versus Non-Sub-Saharan Africa

Note: p values in parentheses; *, **, and *** significant at 10 percent, 5 percent, and 1 percent levels, respectively; filter: 0 < ΔAid[t0 – t– 1] < 10 percent.

Source: IEO staff estimates based on IMF, MONA database.

Programmed responses to anticipated aid reductions depended on initial reserve levels and were asymmetric. Countries with very high initial levels of reserves are, on average, allowed to finance the aid reductions to avoid fiscal adjustments, mainly through the depletion of reserves (Table A2.1). Those with very low initial levels of reserves, by contrast, have to fully bear anticipated reductions in aid, in the form of full fiscal and current account adjustments. The programmed fiscal response to aid reductions does not depend on inflation levels.

Table A2.1.Regressions. PRGFs: Sub-Saharan Africa Versus Non-Sub-Saharan Africa
Aid IncreasesAid Reductions
Current

account

deficit
Fiscal

deficit
Current

account

deficit
Fiscal

deficit
Delta aid0.3820.5310.9281.045
(0.549)(0.038)**(0.003)***(0.000)***
Interaction with1

Reserve levels
0.150.005–0.073–0.125
(0.201)(0.921)(0.099)*(0.000)***
Initial inflation–0.018–0.017–0.008–0.011
(0.354)(0.024)**(0.572)(0.276)
Observations93939393
Source: IEO staff estimates based on IMF, MONA database.Note: p values in parentheses; *, **, and *** significant at 10 percent, 5 percent, and 1 percent levels, respectively; filter: 0 < ΔAid[t0 – t– 1] < 10 percent.

Almost 80 percent of SSA PRGFs limited the possibility of authorities to fully spend unanticipated windfalls in aid or fully finance unanticipated shortfalls. As seen in Figure A2.14, this stance is similar to the Fund’s position toward unanticipated changes in aid in other regions.

Figure A2.14.Treatment of Unanticipated Aid Inflows in Sub-Saharan Africa PRGFs

(In percent of programs)

Sources: IMF staff reports; and IEO staff estimates based on IMF, MONA database.

Note: Figure depicts percentages of different adjusters as observed in the initial request.

Methodology and Data

This section discusses data definitions and methodology used to derive the results presented in the evaluation report. MONA, the principal data source used in the analysis, had a break in series in 2001. All programs starting before the break in 2001 plus the 2002 PRGF for the Democratic Republic of the Congo are captured in MONA I. MONA II, which has a different set of macroeconomic variables, captures all other programs.

The proxy variable for net aid inflows was derived from balance of payments data by adding changes in liabilities to official creditors (disbursements – amortization) to official current transfers and capital transfers, deducting external interest payments and—where applicable—adding the programmed financing gap and external arrears (Table A2.2).17 The variables “projected new rescheduling” and “other balance of payments support” were often not available in the database. In these cases, the value was assumed to be either zero or captured by some other variables. Hence, they were omitted in the calculations.

Table A2.2.Proxies for “Net Aid” Per GDP
MONA IMONA II
(Official transfers, net

+ other balance of payments support

– scheduled net interest payments

+ official borrowing from multilateral and bilateral lenders (excluding Fund)

– scheduled principal payments (excluding Fund)

+ rescheduling contracted before program

+ projected new rescheduling

+ increases in external payments arrears

+ programmed financing gap)

/ GDP

(Official current transfers, net

– interest payments

+ capital account balance

+ [changes in] liabilities to official creditors

– [of which:] credit and loans from IMF (excludes reserve position in the Fund)

+ arrears, net change (in the financial account) (+ increase)

+ programmed financing gap)

/ GDP

The nonaid fiscal deficit was derived from the difference between expenditures excluding interest payments and domestic revenue (Table A2.3). One challenge in constructing the variable was the very poor data on grants in MONA II. When no data on grants were reported, the balance of payments entry for official current transfers and capital transfers (as captured by the reported capital account balance) was used as a proxy for grants.

Table A2.3.Proxies for “Nonaid Fiscal Deficit” Per GDP
MONA IMONA II
– Fiscal balance including grants (percent of GDP)

+ grants (percent of GDP)

– interest payments (percent of GDP)
(Total expenditures and net lending

– total revenues and grants

+grants

–interest payments)

/ GDP

The nonaid current account deficit was derived by netting official current transfers and interest payments from the current account balance (Table A2.4). Of all variables, this had the best data availability in MONA.

Table A2.4.Proxies for “Nonaid Current Account Deficit” Per GDP
MONA IMONA II
(– Current account, excluding official transfers

– net interest payments)

/ GDP
(– Balance on current account

+ official current transfers (net)

/ GDP

The dummy variable “good macroeconomic performance” was defined to be 1 if in the year prior to the program (t – 1) inflation was below 10 percent, real growth was greater than zero, and domestic financing was 1 percent of GDP or less. All three variables were derived from the MONA database in order to capture estimates about the past year’s performance at the time of program design. While the first two variables were readily available, domestic financing was estimated by deducting the external financing proxy (net aid) from the nonaid fiscal deficit.

The program year (t0) was defined as that calendar year with the greatest overlap with the actual program year from the program request or subsequent reviews. For example, if the actual program year with conditionality that was agreed upon in a review lasted from November 2003 to October 2004, 2004 was defined as the program year for that review.

MONA is not subjected to the same level of scrutiny as published databases of the Fund. It has a number of errors and inconsistencies. The most obvious errors—such as mixed-up currency denomination—were corrected manually. Others, such as discrepancies between reported actuals and obviously erroneous zeros, led to omission of the observation. To the extent possible, some omitted variables were reconstructed from other observations. Remaining errors were assumed to be unsystematic and, thus, without significant influence on the findings except for a reduction of their robustness.

To ensure consistency, programs and outcomes were compared within the MONA database. Since variable definitions in MONA I were very different from those in the World Economic Outlook or other databases, programmed values from MONA were not compared with estimates derived from other databases. Instead, data from the program request were compared to data from the latest available review for the respective years. This approach minimized errors stemming from differences in data definitions.

The differences between actuals (or updates) and projections for the program year (t0) in SSA PRGFs are significantly smaller than zero at the 5 percent confidence level, even after correcting for optimistic growth forecast errors.

All the differences between actuals (or updates) and projections for the three years following the program year (t + 1, t + 2, t + 3) are significantly greater than zero at the 5 percent confidence level even after correcting for optimistic growth forecast errors.

Forecast errors are not significantly different from zero in non-SSA PRGFs. For t + 1 and t + 2, there is a significant positive difference in forecast errors between SSA and non-SSA PRGFs at the 5 percent confidence level even after correcting for optimistic growth forecast errors.

“Good performance” defined as initial conditions of inflation below 10 percent, positive growth, and domestic financing below 1 percent of GDP.

Expenditures exclude interest payments.

Programmed changes between one year before the program and the program year.

Obviously, both deficits are influenced by many more factors than the deliberately few variables used in this regression, which serve the purpose of shedding some light on correlations with expected changes in aid rather than testing whether the underlying model is complete. The dashed lines in the graph show the linear estimates for the unconstrained model, while solid lines show these estimates for the constrained model, which by suppressing the constant, forces all changes in the two deficits to be linked to aid increases. The results of the regressions reported in the tables show that the constant was positive but not significantly different from zero. Hence, its suppression had only a limited upward bias on the reported estimates. At the same time, the suppression of the constant helps avoid an underestimation of spending ratios that would arise from measurement errors in the unconstrained model.

All figures presented show only differences found to be significant at least at the 10 percent level in the constrained and/or unconstrained regressions. Significance in one of the two tests sufficed for depiction.

Not significantly different from 100 percent.

Five percent was identified by the data as the threshold, which generates the most significant difference between programs with low and high initial inflation. Without controlling for other initial conditions like domestic financing or growth, the highest inflation threshold with significant differences was found to be 7 percent. Once we control for those other conditions, even higher thresholds like 10 percent generate significant differences. However, even then only for programs with initial inflation below 7 percent, the estimated average spending ratio is not significantly different from 100 percent.

Not significantly different from 100 percent.

The coefficient on the fiscal deficit of 15 percent is not significantly different from zero percent.

Significantly different from zero.

Only the difference in the aid-independent adjustment of the fiscal deficit remains significant when controlling for the inflation threshold of 5 percent.

Most regional differences can be explained by differences in compliance with the reserve and inflation thresholds identified above. Only the difference in the aid-independent adjustment of the fiscal deficit remains significant when controlling for these dummies.

To check whether there is a systematic bias of the estimated spending ratios given that the proxy for aid inflows was derived from balance of payments data and the fiscal deficit was derived from fiscal data, fiscal aid data compiled from the case studies was compared with the balance of payments derived proxy from MONA. Both were highly correlated and without a significant bias.

    Other Resources Citing This Publication