III. Review of Experience
- Tubagus Feridhanusetyawan, Alun Thomas, Tessa Van der Willigen, Uma Ramakrishnan, S. Reichold, Juan Zalduendo, and James Walsh
- Published Date:
- September 2005
22. Since little time has elapsed since the introduction of the guidelines, any review of experience must be treated with caution. The present chapter describes this experience. It does not attempt to ascertain whether changes are statistically significant, and indeed, given the small sample sizes (generally some 20 arrangements a year, divided roughly equally between GRA and PRGF), any attempts to establish significance would likely fail. To be sure, changes in the characteristics of members that seek Fund support or in their circumstances could influence some of the developments described below. But the hope was that the guidelines would make a visible difference, and this review seeks pointers as to whether this difference has yet materialized.
A. Parsimony and Criticality
Coverage of conditions
23. Streamlining was expected to focus conditionality on areas critical to the goals of the Fund-supported program. Substantial changes are evident in this area.
24. Major shifts have occurred in the coverage of structural conditionality, suggesting a greater focus on criticality (Figure 1). Consistent with the notion that conditions in noncore areas are less likely to be critical to the goals of Fund-supported programs, conditionality in both GRA- and PRGF-supported programs has shifted out of noncore areas, toward areas where the Fund shares expertise with the World Bank in the former case, and toward core areas in the latter case. Between 1995–97 and 2001–03, the share of structural conditions in noncore areas fell by 17–18 percentage points in both GRA-and PRGF-supported programs. Structural conditionality has also shifted out of measures related to economic flexibility and private sector efficiency, in arrangements under the GRA toward debt and financial vulnerabilities (which now account for almost 40 percent of all conditions), and in PRGF arrangements toward economic management (which now represents almost 45 percent of all conditions).15
Figure 1.Institutional and Economic Classification of Structural Conditionality
25. Conditionality is now also focused on fewer areas. In both GRA- and PRGF-supported programs conditionality is now less scattered among fourteen different sectoral categories than it was pre-guidelines. Between 1998–2000 and 2001–03, in GRA-supported programs, the average coverage fell by about three quarters of a sector to a little under six sectors, and in PRGF-supported programs by almost two sectors to a little over six sectors. A similar trend is evident among supply-side categories. This greater focusing seems consistent with a greater emphasis on those few key areas where action is critical to the success of the program.
26. In addition, greater focus is evidenced by a stronger relation of structural conditions to the economic situation at the start of the program in recent years.16 The number of conditions in economic management declines with stronger pre-program fiscal performance, and the number of conditions in private sector efficiency drops with stronger pre-program growth performance. These links have appeared or strengthened in the post-guidelines period.17 Similarly, conditions aimed at addressing vulnerabilities are more numerous where initial external debt levels are high—a link that was not present in pre-guidelines programs and is consistent with a sharper focus on priorities, but which may also capture the growing understanding of the importance of such vulnerabilities.
Numbers of conditions
27. The guidelines were also expected, broadly speaking, to lead to reduced numbers of structural conditions. Of course, the number of conditions is a crude metric. First, it weights equally conditions of quite different import. Moreover, a focus on numbers takes no account of the possible tension between parsimony and criticality (when problems are complex and require a multi-pronged approach), nor of that between parsimony and clarity (most noticeable when vaguely specified program review clauses are replaced by a number of precisely defined conditions). Indeed, in the 2000–02 conditionality review, Directors stressed that the number of conditions is only one, imperfect, indicator of the extent of progress with streamlining.
28. Overall, numbers of conditions in Fund-supported programs have been broadly stable since the introduction of the guidelines (Figure 2).18 This development, however, masks differences between PRGF- and GRA-supported programs.
Figure 2.Structural Conditions in Fund-Supported Programs
Source: MONA database and staff estimates (for 2004).
29. In PRGF-supported programs, the number of structural conditions is on the decline. From 17–18 in the late 1990s, the average number of structural conditions per program year has fallen to 13–14 (Figure 2). The reduction in the number of prior actions has been especially marked, but numbers of structural benchmarks are also down somewhat from the late 1990s (Figure 3).19 An examination of the scope of individual structural conditions in the PRGF cases with the largest numerical declines suggests that the decline in conditions is genuine, and not the result of a broadening of the scope of conditions.20
Figure 3.Types of Structural Conditions in Fund-Supported Programs
Source: MONA database and staff estimates (for 2004).
30. In GRA-supported programs, the number of structural conditions has shown no decline, and has even recently increased. In 2001–02, the average number of conditions a year stayed around its late 1990s level of some 16–20 conditions a year, but in 2003–04 this number rose to 21–23. Sample sizes, however, are small—about ten arrangements a year, and only six in 2004—and subject to large compositional changes. In more homogeneous subgroups of GRA-supported programs, a small decline in numbers of conditions is evident among the same members pre- and post-guidelines, but a rise in arrangements that were treated as precautionary on approval (Figure 4). One factor that is driving up the number of conditions in GRA-supported programs is the disappearance, in line with the guidelines, of vaguely-specified program review clauses in favor of specific conditions (see Section III.B): abstracting from programs in which structural measures were monitored entirely through review clauses, the average number of conditions in 1995–98 would have been some two conditions higher. The number of prior actions and structural benchmarks in GRA-supported programs has been variable but broadly flat, and the number of structural PCs was sharply higher in 2003–04 than in previous years.21
Figure 4.Total Structural Conditionality in GRA-Supported Arrangements
Source: MONA database and staff estimates.
31. The growing understanding of the importance of vulnerabilities—especially in the financial sector—has clearly played a large role in conditionality in GRA-supported programs.22 Between 1998–2000 and 2001–03, conditions in the area of debt and financial sector vulnerabilities increased by one and a half conditions a year per program, even as the average number of conditions across programs remained unchanged. The increase in vulnerability-related conditionality has been across the board, affecting programs with both relatively high and relatively low emphasis on vulnerability. Moreover, programs with above-average vulnerability-related conditionality were also the sole source of increase in the number of conditions outside the vulnerability category, and most of this increase was related to economic management, much of it serving most likely to help rein in public debt.
32. Also striking is the influence of weak track records on the number of conditions in GRA-supported programs.23 GRA-supported programs with members with relatively weak track records have recently shown a tendency to higher numbers of conditions (Figure 5). Such members have also recently become more prominently represented among members using GRA resources, with their share more than doubling from 25–30 percent in 2001–02 to almost 65 percent in 2003-04.
Figure 5.Number of Structural Conditions in GRA-Supported Arrangements with Members with Relatively Weak Track Records
Source: MONA database and staff calculations.
33. In both GRA- and PRGF-supported programs, prior actions are used as a screening device. Prior actions have continued to be used more in countries with relatively weak track records (Figure 6)—a link that is corroborated by empirical analysis.24
Figure 6.Prior Actions in Countries with Relatively Weak Track Records
Source: MONA database and staff calculations.
Relations between Fund and World Bank conditionality
34. “Aggregate” Fund-Bank conditionality appears to be on the decline (Figure 7, panel 1). As noted in Section II, developments in “aggregate” conditionality do not speak to whether the Fund is implementing properly its own guidelines, but aggregate conditionality is important to borrowers and to aid donors. In a set of 30 members with overlapping Fund arrangements and Bank policy-based operations both pre- and post-guidelines, “aggregate” conditionality from these two kinds of operations has declined—in GRA cases, because Bank conditions have become fewer, and in PRGF cases, on account of reductions in numbers of conditions at both institutions.25
Figure 7.Bank and Fund Structural Conditionality
35. Consistent with the guidelines, and unlike in the past, the Fund now shows little tendency to make its conditionality depend on the Bank’s (Figure 7, panel 2). In the late 1990s, the Fund tended to set more conditions in the Bank’s core areas when the Bank did not have a concurrent operation—seemingly “taking over” relevant, but not critical, conditions when the Bank was not present. This tendency has almost disappeared in the post-guidelines period, consistent with a focus by the Fund on measures critical to the success of the program it supports.26
36. As the Fund has focused its conditionality, the Bank does not seem to have increased its conditionality in the areas from which the Fund has withdrawn (Figure 7, panel 3). Social sectors, public enterprise reforms, and business environment—sectors with reduced Fund conditionality—have not generally been areas of increased Bank conditionality, and indeed, for members using Fund resources under the GRA, these have been areas where the Bank has also reduced its conditions. This result should, however, be interpreted with caution given the sample size (12 GRA- and 18 PRGF-supported programs). Moreover, these comparisons do not take into account the many other means through which Fund and Bank support members’ reform efforts, including through technical assistance and the policy dialogue more generally.
A look at individual conditions and justifications
37. At the most disaggregated level, two questions can be asked of the formulation of conditions. First, to what extent are conditions applied to reforms that are not likely to have a critical impact on program goals? And second, are conditions specified at a level of detail that may be perceived as micromanagement (whether their link to a critical reform area is clear or not)?
38. A review of arrangements approved in 2001–03 suggests that relatively few conditions are now set in what seem, a priori, “odd” areas—although there still exist conditions (typically structural benchmarks) whose criticality is neither obvious nor convincingly explained. Examples of conditions whose impact on program goals appears minor and for which the staff report does not provide a clear explanation of criticality (although the problem to be addressed by the measure is typically discussed) include: formulating an action plan to establish a Constitutional Council (Cameroon, SB, 2002), reducing employment in the education sector by 5 percent (Tajikistan, SB, 2004), or announcing criteria to increase borrowing limits for state and local governments that propose qualifying revenue-yielding projects of a self-sustaining nature in water and sanitation, with a view to modernizing the sector (Brazil, SB, 2003). Cases also exist where a condition is set on preparation of a measure, but no follow-up on effectiveness of the measure was found, raising questions why the initial step was considered critical and the subsequent ones were not—for instance, submission to Parliament of a tobacco law and a lottery law (Serbia and Montenegro, SB, 2002).27
39. At the same time, tension remains between clear specification of conditions and micromanagement. It remains relatively easy to find examples of conditions that are specified in considerable detail. Examples include providing resources in the budget to increase the number of auditors by 400 (Turkey, SB, 2002), sending letters of redundancy to 340 health sector employees (Macedonia, SB, 2003), and issuing guidelines requiring double signatures in operating school bank accounts, including of a member of the community-based school management committee (Sierra Leone, PC, 2004).28 Further examples can be found in the many conditions relating to the introduction or upgrading of ASYCUDA (Automated System for Customs Data) software, often only in certain offices (Albania, Armenia, Bosnia, Gambia, Madagascar, Uganda). An example of highly detailed specification of conditions in the financial sector—the source of many conditions in GRA-supported programs—is given in Selected Issues, Box 2 (Brazil, 2002). Section IV.A will return to the level of detail at which conditions are set.
40. The rationale for structural conditionality is generally explained in staff reports, although room for improvement remains. About three quarters of staff reports for new arrangements include a separate box with explicit rationales for conditions in specific areas, the status of structural conditionality from earlier arrangements, and the areas of reform covered in World Bank operations.29 These boxes are particularly helpful in providing explanations of the criticality of conditions in noncore areas, as required under the guidelines. Most of the remaining staff reports address these issues elsewhere, but boxes seem especially useful as a presentational device to ensure that the relevant questions are dealt with prominently and in an integrated manner.
41. Major progress has been made in clarity on program-related conditions since the introduction of the new guidelines. All quantitative and structural conditions are now almost always included in a separate table in the letter of intent or staff report, as suggested in the operational guidance note (¶9). Moreover, letters of intent now almost never feature detailed policy matrices covering the authorities’ broader agenda—a practice discouraged in the guidelines (¶10).
42. Recent practice has also largely conformed to the guidelines’ stipulations on the scope and frequency of program reviews:
- The backward-looking component of reviews is generally slated to consider the implementation of explicit program-related conditions; only rarely is the scope of a review defined to include ill-specified “progress in [a given sector].”
- As allowed for in the guidelines, flexibility to introduce new conditions appears to be used primarily to respond to changes in circumstances—if the latter are understood to include situations in which it is determined, during the course of the program, that the program’s intermediate objectives are unlikely to be met unless specific actions or conditions are formulated (e.g., revenue targets without tax policy changes, or state-owned enterprise reform without specific plans and conditions). The introduction of entirely new conditionality during the course of the program, without reference to changed circumstances, is rare.30
- The scheduled frequency of reviews in PRGF arrangements is semiannual, with few exceptions. Countries experiencing or emerging from capital account crises, understandably given their rapidly changing situations, mostly have scheduled quarterly reviews. For other countries, average review frequency has varied widely, with no tendency to decline since the guidelines went into effect. In 2003–04 arrangements for noncrisis cases typically had semiannual reviews, the exceptions being Ecuador, Gabon, Paraguay, and Romania, which had quarterly reviews (and all of which also had above-average numbers of structural conditions and PCs).
43. Broadly consistent with the findings of an earlier survey of country authorities and staff, case studies of ten recent Fund-supported programs suggest that the staff has made serious efforts to follow good processes—although this does not mean there is no room for improvement. The case studies—of Brazil, Bulgaria, Burkina Faso, Ecuador, Guinea, Guyana, Georgia, Romania, Sri Lanka, and Tanzania—are presented in Selected Issues, Chapter VII. In all cases, the staff has attempted to incorporate almost all key ingredients of an effective process suggested in the guidelines and their supporting documents. This generally positive assessment is consistent with the results of a survey carried out as part of the recent review of IMF-World Bank collaboration.31 Majorities of both country authorities and staff in these surveys gave broadly positive assessments of country ownership and of the Fund’s flexibility in program design, although the staff was generally more positive than the authorities (with positive responses of the order of 90 percent, compared with 65-75 percent on the authorities’ part) (Figure 8). While some perception gap may be inevitable, the results of the staffs efforts clearly still leave room for improvement in the authorities’ eyes. Section IV.D will return to the question of further scope for improvement in process.
Figure 8.Ownership and Policy Space
Source: Surveys conducted for the review of Bank-Fund collaboration, September 2003.
D. Program Implementation
44. Of course, the ultimate objective of the new guidelines was to help improve economic outcomes—but their effectiveness in this regard cannot yet be determined. Because Fund resources are provided to members to permit a sustainable adjustment without destruction of prosperity and must be repaid, outcomes cannot be judged only relative to immediate program targets, but need to encompass issues of medium-term sustainability and growth. The question of program outcomes must therefore be left to a subsequent review.
45. By contrast, program implementation can be observed, and it too was expected to improve. First, with the emphasis on ownership and implementation capacity, policy programs were expected to be implemented more consistently. Moreover, with conditions limited to measures critical to the success of the program, it was expected to become a less frequent occurrence that programs would be judged to remain on track even as conditions were not met. Observed program implementation—the degree to which programs remain on track, and waiver rates—was expected to improve on both counts.
46. In line with expectations, a substantial decline occurred in programs with permanent interruptions (Figure 9, top panel). Cases in which programs went off track and were not brought back on track prior to expiration have declined significantly among GRA-supported programs. An improvement is also visible among PRGF arrangements, at least relative to 1998–2000, although the recent sample includes only seven expired arrangements. The share of programs that went irretrievably off track even before the first review has also declined, although for GRA-supported programs not below the level of 1995–97. The decline in permanent program interruptions suggests that Fund resources provided early in the arrangement are placed at less risk and, more generally, that members’ adjustment efforts have been sustained.
Figure 9.Permanent and Temporary Interruptions and Delays, 1995–2003
47. Developments in program interruptions and review delays have not been as positive (Figure 9, bottom panel). The frequency of interruptions of at least six months has been stable or rising, and review delays remain broadly at their late 1990s levels of 80–100 days.
48. The implementation of conditionality has shown a minor improvement in PRGF arrangements, but no improvement in GRA-supported programs (Figure 10). In the latter, waivers for structural PCs and (implicitly) structural benchmarks have actually risen, and the overall measured implementation of structural conditionality in on-track programs has been stable.32 In PRGF-supported programs, the trend in implementation has been somewhat better, with overall implementation rising (though from a low in the immediate pre-guidelines period), mostly on account of improved observance of SBs.33 In PRGF arrangements too the waiver rate for structural PCs has failed to decline as had been expected. Waiver rates for quantitative PCs (at 10–20 percent) remain much lower than those for structural PCs (at 40–50 percent), and show little evidence of a trend in either GRA- or PRGF-supported programs.
Figure 10.Implementation of Conditionality, 1995–2003
49. Expectations had been that waivers for structural conditions would become less common. In the 2000–02 conditionality review, Directors expressed concern over the high and rising incidence of waivers, and “believed that waivers should become less frequent as conditionality is focused on measures that are critical to program objectives, and ownership is strengthened.”34 They were concerned not only that “waivers [were] often an indication of poor program implementation and/or a lack of realism in program design,” but also that the Fund needed “to adhere more closely to the existing policy that, in cases of significant policy slippages, waivers should be granted only if appropriate corrective action has been taken to achieve the objectives of the program.” Recent developments raise the question whether these concerns remain valid.
50. Despite the high or rising waiver rate, the Fund does not appear to be taking PCs less seriously than in the past; in fact, there is evidence of the opposite, as the share of conditions that are implemented has risen. The incidence of waivers granted for conditions that never get implemented, even as the arrangement continues—and that can thus be identified in hindsight as conditions the arrangement could have done without—is relatively low (Figure 11). Moreover, this incidence (termed the “lapsed rate”) appears to have declined in both GRA- and PRGF-supported programs, with a somewhat greater decline in the latter.35
Figure 11.Composition of Structural PC Waiver Rates: Delayed Measures, Alternative Measures, and Lapsed Rates in Fund-Supported Programs
51. Many measures subject to structural PCs that are waived are implemented with a delay, often and increasingly as prior actions for the granting of the required waivers. As shown in Figure 11, delays in implementation are both the most important and the fastest growing reason for the high waiver rate, and rough estimates suggest that most of this growth may be accounted for by missed conditions that become prior actions for waivers (see Selected Issues, Chapter I). Alternative (or slimmed down) actions account for a smaller share of waivers.
52. All in all, the picture that emerges suggests a focus on criticality in formulating structural conditionality and monitoring its implementation, but also persistence of difficulties in formulating the precise timing and modalities of measures. Consistent with the guidelines’ emphasis on criticality, the Fund seems to be making serious efforts to formulate as program-related conditions only measures on which it is willing to insist, and this insistence may be contributing to the absence of a decline in program interruptions and review delays. At the same time, many structural conditions are still waived when the actions are taken on a different timeframe or with different modalities. Difficulties in precise formulation of conditions may be especially important in noncore areas, where implementation is weakest.36