Chapter

IV. Assessment

Author(s):
Tubagus Feridhanusetyawan, Alun Thomas, Tessa Van der Willigen, Uma Ramakrishnan, S. Reichold, Juan Zalduendo, and James Walsh
Published Date:
September 2005
Share
  • ShareShare
Show Summary Details

53. Thus, insofar as conclusions can be drawn from brief experience, the guidelines appear to have brought important changes. Conditions have become more focused and more concentrated in the Fund’s core areas, although they have not become significantly fewer in number; and they are specified more clearly. The eventual implementation of conditions also seems to have improved, albeit often with delayed timing or in modified form. Importantly, more programs are successfully implemented, with fewer suffering permanent interruptions. And the staff appears to be making serious efforts to use effective processes in the course of program negotiation.

54. Four questions nonetheless emerge. First, has streamlining been sufficient? Second, should the sharp shifts in the sectoral composition of conditionality be viewed entirely positively, or are there risks? Third, how should the high structural PC waiver rate be viewed? And fourth, has the Fund found the proper balance between national ownership and selectivity? This section will consider each of these questions in turn.

A. The Extent of Streamlining

55. While the coverage of conditionality has been considerably streamlined, there has been, overall, no parallel decline in numbers of conditions. Section IV.D will examine the extent to which numbers of conditions and PCs are affected by a desire to apply extensive conditionality where ownership is in doubt. Overall, however, given the uncertainties and judgment surrounding the concept of criticality, it is extremely difficult to identify individual cases where too many conditions may have been specified. Reasonable rationales have typically been given in program documents, and indeed the Executive Board has generally expressed few reservations about these (see ¶38, and Selected Issues, Boxes 3 and 6, which describe the Board’s support for the number and range of structural conditions in the recent arrangements for Bolivia, the Dominican Republic, Ecuador, Paraguay, and Romania). The present section therefore asks whether the tools of conditionality are used, or program goals specified, in ways that may compromise parsimony.

Use of structural benchmarks

56. Numbers of benchmarks have not significantly declined. In assessing this development, it is important to distinguish between the breadth of coverage of reforms monitored by benchmarks and the level of detail at which these reforms are monitored.

57. The guidelines stipulate that structural benchmarks, like PCs, can only be used for critical measures, but it seems likely that they are still used, in some cases, to monitor “less critical” reforms (along the lines of the possible examples cited in ¶38). There is considerable anecdotal evidence that the guidelines’ statement that the “non-implementation [of SBs] would not, by itself, warrant an interruption of purchases or disbursements under an arrangement” is often misread to imply that the reform at issue need not be critical. The high prevalence of SBs that are not part of clusters of conditions (some of which are also objectively definable, raising the question why they were not set as structural PCs) points in the same direction.37

58. The use of structural benchmarks in this way is consistent with the persistence, to some degree, of the “relevance” model of conditionality. According to this model—that of the interim guidance note of 2000—there is a role for conditionality on measures that are macro-relevant, rather than critical.38 This model avoids the considerable difficulties involved in a criticality test for measures that will bear fruit only some way into the future. Such measures are “critical” in the sense that their implementation is viewed as necessary to achieve longer-term objectives, but it may be difficult to establish that they must be taken during the period of the Fund-supported program. The guidelines take the view that measures that are not unambiguously critical to the achievement of program goals should not be part of conditionality, but could be part of the policy dialogue more generally—although the relevance model would argue that a focus on program-related conditions tends to overshadow the broader policy dialogue.

59. It is also striking that benchmarks often continue to be detailed. The guidelines specify that SBs should be “clear markers in the assessment of progress” in implementation, and in the 2000–02 conditionality review, the Executive Board noted that “[s]tructural benchmarks were extensively used to map out steps in the implementation of particular structural policies,” and “saw a need for structural benchmarks to be used more sparingly, by limiting each benchmark to an important and representative step toward a policy outcome[, which] would also help avoid the impression of micromanagement.”39 As noted above (¶39), however, some arrangements still include SBs on very narrowly circumscribed actions.

60. Detailed specification of benchmarks is not necessarily to be resisted. Whether there is a tradeoff between detail and the guidelines’ principle of parsimony is a difficult question. Detail tends to increase the number of structural conditions—which to some extent illustrates the difficulties involved in using numbers as a metric. Members may nonetheless consider that detailed Fund involvement constitutes micromanagement and thus represents a failure to limit conditionality to the “minimum necessary” (as required by parsimony, see ¶13). At the same time, detailed specification certainly supports another key principle underlying the guidelines, that of clarity. It can provide the authorities with welcome transparency as to how the Fund will assess progress at the time of a program review; indeed, the guidelines themselves call for the program elements to be taken into account for the completion of a review to be specified as fully and transparently as possible in the arrangement, thus encouraging the setting of numerous specific SBs rather than a single, more vaguely specified condition or review clause. Discussion of detailed steps also helps give the Fund confidence that specific action will be taken (although it does not necessarily follow that conditionality needs to be specified at the same level of detail).

61. All in all, the establishment of structural benchmarks in noncritical areas would be more obviously a contradiction of the guidelines than highly detailed specification—but the latter is likely to be a more important issue in practice. Although conditionality has become more focused, nonetheless SBs should not be applied to noncritical reforms. To counteract the tension within the guidelines whereby nonobservance of structural PCs, but not SBs, interrupts purchases or disbursements under the arrangement, it could be clarified that the difference between these conditions is a difference in the nature of the steps to be monitored (i.e., small steps in a larger process), not in the reform itself. At the same time, the level of detail at which SBs are specified probably contributes more significantly to the still relatively high numbers of conditions. It is less obvious that detailed specifications should be avoided, but it might be clarified that SBs should be applied to “important and representative steps.”

Program goals and strategies

62. Another important question is whether parsimony has been compromised by overly broad program goals or inadequately specified strategies. Since measures critical to the achievement of program goals must be included in conditionality, vague program goals risk wide-ranging conditionality, defeating the guidelines’ drive toward parsimony and focus. And identification of strategies can only help to focus conditions on measures critical to program success.

63. An examination of staff reports suggests that overly broad inclusion of goals is not an important factor in the formulation of conditionality.40 The majority of goals presented in staff reports are within the scope of the guidelines—external stability, macroeconomic stability (which is a means to external stability), and growth in the case of the PRGF.41 Many arrangements in the GRA—as many as in the PRGF—cite growth as a goal, seemingly independently of its implications for sustainability; and some cite poverty reduction. But in both these cases these goals do not appear to impact the formulation of conditionality; instead, they seem to function as a statement of broad support by the Fund for the authorities’ objectives.

64. By contrast, some room for improvement exists at least in the presentation, and possibly also in the formulation, of strategies. Deficiencies in the presentation in staff reports do not necessarily reflect deficiencies in the staffs thinking, but they do suggest—as has previous been argued by the Independent Evaluation Office (IEO)—that some program documents should be clearer in presenting the “story line” or “big picture” that underlies the program, and thus that the Board is not always given the best possible information on which to base its decisions.42 Several such deficiencies can be identified:43

  • Although two thirds of requests for arrangements emphasize a few key strategies—priority areas for action, such as the financial sector, public expenditure management, or governance—one third do not cite explicit strategies.
  • Moreover, even when strategies are cited, program documents do not generally explain why they were selected. The case of Lesotho (2001 PRGF, see Selected Issues, Box 1) can be considered best practice in this regard. Of course, in some cases the cause of external imbalance may be clear, and no explicit discussion of strategies is necessary. But in many cases, particularly in the PRGF, the problems are complex and explanations of the choice of strategies would provide insights.
  • Although the correlation between strategies and conditionality is generally good, some strategies do not translate into significant conditionality, and programs quite commonly have extensive conditionality in areas not cited in their strategies. In either case, one may wonder whether it is the conditionality or the strategy that is better thought out, and hence whether conditionality could have been better tailored or strategy better explained.

B. The Risks of Streamlining

65. As was recognized at the time of the 2000–02 conditionality review, streamlining brings a risk that reforms that could make a substantive difference to a country’s prospects will receive insufficient attention. While it is too early to pass judgment on the guidelines’ impact, some pointers are worth noting.

66. Potentially worrisome is the sharp shift in conditionality away from growth- and efficiency-related reforms. As noted above, both PRGF arrangements (despite having growth as a program goal) and GRA-supported arrangements (despite the links between growth and external sustainability) now place much less emphasis on these reforms than they did in the 1990s.44 Indeed, econometric tests do not suggest that GRA-supported programs place more emphasis on growth-related measures when external debt is high.45 Relatedly, programs that cite pro-growth reforms as a strategy do not generally include more conditionality in these areas than do other programs.46

67. It is not possible to discriminate, at this early stage, between benign and less benign interpretations of these developments. As noted above, the decline in supply-side conditionality is consistent with the guidelines’ emphasis on criticality, and, as long as country ownership is present, does not necessarily indicate that less action is being taken. The decline may also reflect changes in the challenges faced by members who turn to the Fund, or the weak state of knowledge as to measures to sustain growth once “low-hanging fruit” have been picked: for instance, while there is broad agreement that institutions are crucial for growth, the measures that countries could take to strengthen their institutional framework are less clear, especially once country-specific circumstances are taken into account. An additional explanation could be that the improved growth performance of low-income countries in recent years has resulted in a sense of lesser urgency for additional growth-related reforms. But the new guidelines may also have led to lesser emphasis on reform agendas that require a longer period to mature, or on politically difficult measures. The relative lack of conditionality in these areas could potentially compromise the attainment of medium-term objectives (a point on which judgment must await the availability of outcomes data), signaling “reform gaps” with longer-term consequences. While at this early stage it is not possible to conclude whether the guidelines are having untoward consequences, risks exist that will need to be kept under review.

68. The risk that some areas will not receive sufficient attention in reform plans is highlighted by trends in “aggregate” Bank-Fund conditionality, but this too could be an entirely benign development. As noted in Section III.A, the World Bank does not seem to have increased its conditionality in the areas from which the Fund has withdrawn. It was a concern of Directors during the 2000–02 conditionality review that “strengthened collaboration [with the Bank would be] needed to ensure that important measures are adequately covered as the Fund applies conditionality more sparingly outside its core areas.”47 Of course, with the growing recognition that it is ownership rather than conditionality that ultimately drives forward reform agendas, it is not obvious that gaps in Bank-Fund conditionality (nor indeed in aggregate conditionality once other agents, such as regional development banks and bilateral donors, are taken into account) need to be viewed with concern. Much more worrisome would be gaps in analytical work, policy advice, or technical assistance. In these areas the recent review of Bank-Fund collaboration provides comfort that coordination is improving, although it also stressed that there was need for sustained commitment and steady implementation.48

C. Waivers of Structural Performance Criteria

69. As noted above, the guidelines have not, as had been expected, led (so far) to a decline in the waiver rate. What has declined is the rate at which measures fail to be implemented at all. At the same time, many waivers continue to be required in cases where measures are either significantly delayed or modified. This flexibility is consistent with the waiver policy and can be interpreted as ex post provision of policy space. Indeed, in the surveys conducted for the review of Bank-Fund collaboration, 83 percent of authorities agreed (or strongly agreed) that the Fund’s waiver policy was flexible in taking into account the adoption of corrective measures. The case studies of processes (Selected Issues, Chapter VII) also suggest that policy space has often been provided in successfully implemented programs not just up front, but also during the course of implementation.

70. The provision of ex post policy space is not inconsistent with the guidelines’ emphasis on criticality. While, according to the staff statement (¶7), “a judgment that a condition is of critical importance…means that if it were not implemented, it is expected that the goals will not be achieved,” this should not be read to suggest that, if delayed or alternative actions are substituted for those initially slated, the latter were not truly “critical.” In accordance with the principle of ownership, the Fund’s conditions are drawn directly from the member’s program, as elements that the Fund considers critical to the achievement of program goals. It may indeed be the case that there is little scope for delay or modification of these elements, but a condition may also be more demanding than the minimum that would be required to reach program goals, leaving some room for maneuver, and hence for waivers. The guidelines’ supporting documents could clarify this point, to avoid an overly narrow reading of the definition of criticality.

71. This said, there are disadvantages to setting demanding conditions and then providing policy space ex post via waivers:

  • First and foremost, the lower the likelihood that conditions will be met, the greater will be the role of the Fund’s discretion in granting or withholding waivers, and hence the less the assurance to the member that it will be able to maintain timely access to Fund resources.
  • Moreover, high waiver rates risk encouraging overambitious program targets in the belief that waivers would be granted in case of nonobservance (even in cases where there is actually little scope for delay or modification), and thereby also risk undermining the ability of Fund arrangements to signal members’ commitment to sound policies.49

72. The prevalence of waivers could be reduced by setting structural conditions in less demanding ways. In some cases, the authorities may prefer conditions to be set following an ambitious timetable or modalities—because it will help them focus their efforts, overcome vested interests, or signal their commitment—and the Fund should follow the authorities’ preferences, as long as it is satisfied that the condition will in all likelihood be met. However, if the authorities do not find such conditions helpful, the staff could explore with them the scope for conditions that permit more room for maneuver in meeting program goals. The following paragraphs discuss several possibilities in this regard.

73. As noted above, delays in implementation are by far the most common reason for waivers, suggesting that timetables for conditions are often overambitious. Indeed, structural measures can be such that what is critical is that they be taken at all—not so much that they be taken in a given quarter, or sometimes even a given year. To be sure, a tension exists between realism and ambition, and the authorities themselves may have a preference for ambitious conditions, if they consider these helpful in driving their agenda forward. However, in the surveys conducted for the review of Bank-Fund collaboration, 40 percent of authorities disagreed (or strongly disagreed) that the timetable for implementation of structural measures in Fund-supported programs was realistic—suggesting that there are many cases where the staff presses for faster implementation, and the authorities reluctantly agree. Moreover, the appropriate balance between realism and ambition requires a thorough assessment of both technical and political implementation capacity. While the case studies in Selected Issues (Chapter VII) suggest that the staff already seeks to be careful in taking implementation capacity properly into account, and while technical assistance often helps in this regard, the staff clearly faces serious constraints—in terms of skills, time, and access to information—in forming a judgment on capacity. Because the staff must rely to an important extent on the authorities’ assessment of capacity, it is especially important that it not press for overambitious timetables.

74. One possibility that has recently been used in the face of timing uncertainties is to specify that a condition will be set, leaving its precise timing to be set at a subsequent review. In Turkey, it was foreshadowed at the time of the seventh review under the Stand-By Arrangement in April 2004 that presentation of a social security reform to Parliament and adoption of a new banking act would be set as PCs at the time of the eighth review.50 In cases of particular uncertainty, and experimentally, the Fund could make further use of this approach. It would be crucial that the measures in question are ultimately established as actual conditions, to avoid their potential misuse to signal commitments that do not materialize in reality.

75. As a solution—in a few cases—to the difficulty of determining the “critical” timing of a measure, there may also be some scope to make use of floating tranches. The possibility of floating tranches—“purchase(s) or disbursement(s) … made available whenever the measure is implemented”—was introduced in the 2002 guidelines (¶13), although it was recognized that the potential for using them would remain limited.51 Floating tranches have not yet been employed, perhaps in part because the staff has not been clear on the circumstances in which they should be considered. The internal logic of floating tranches requires that two criteria be met: the measure must strengthen the external position over the medium term, so as to warrant the release of additional Fund resources; but it must also create an additional balance of payments need in the short term—otherwise, the program will be either underfinanced without the measure, or overfinanced with it. (Indeed, overfinancing would suggest that the Fund is “buying” implementation of the measure, creating a difficult tension with the principle of ownership.) These restrictive criteria suggest that floating tranches would be appropriate only for a small set of measures in Fund-supported programs, probably concentrated in the areas of trade liberalization and civil service reform. Clarifying these criteria would be conducive to consideration of floating tranches in those cases where they are met.52

76. Additional room may also exist to provide policy space within conditions. The guidelines explicitly provide for outcomes-based conditionality (including in the structural area), as a way of providing the authorities with policy space within conditions, although as with floating tranches it was recognized that the scope for moving in this direction was likely to be limited. The outcomes sought by structural change are often not sufficiently within the member’s control, nor observable with a short enough lag, to be stipulated as conditions.53 However, on the spectrum from specific actions to outcomes, the potential for structural conditions providing policy space is likely to be concentrated in the intermediate range. Although definitional problems make it difficult to gauge whether conditions providing policy space are used more than previously, such conditions have been used in a number of cases in recent years. Examples include revoking a bank’s license if it is not sold by a specified date (Turkey, SB, 2002); revision of the value-added tax (VAT) threshold to reduce the number of VAT taxpayers substantially, and revision of the threshold for the presumptive tax accordingly (Tanzania, SB, 2003); and conditions preserving the authorities’ ability to choose between expenditure and tax measures to achieve a certain budget target (e.g., Bolivia (2003), Colombia (2003)).54 With conditions of this type, there may be a risk that they would be met in unacceptable or suboptimal ways. But if greater care were taken to distinguish between the risk of unacceptable action (which must be avoided) and the risk of merely suboptimal action (which the Fund should accept in the name of national ownership), it might be possible to formulate more conditions providing policy space. Indeed, the scope for setting conditions on outcomes should itself be explored further.

D. Ownership and Selectivity

77. While on the whole the evidence—particularly the decline in permanent program interruptions—is consistent with improved national ownership since the advent of the guidelines, some cases of very poor implementation remain. The question thus arises whether the Fund could do better at reaching agreement on modified but still acceptable policy programs that would garner greater national ownership—or, failing this, at exercising selectivity. Only by supporting policy programs that are consistent with achievement of external sustainability and that are implemented does the Fund fulfill the purposes of its financial assistance and safeguard its resources.

78. Process is important in securing ownership, but the relevant case studies prepared for this review confirm that good processes do not guarantee ownership, and indeed that some aspects of “good processes” are idiosyncratic.55 There are several examples where no missing process element can obviously be identified, yet implementation fell short.56 And there are cases where the process deviated from the one envisaged in the guidelines, yet good implementation followed. The provision of policy space, for instance, is not a sine qua non for good implementation: sometimes agreement is reached readily (as in Burkina Faso), and sometimes the authorities eventually come to agree largely with a firm position taken by staff (as in Romania). Similarly, there are cases of successful implementation where large numbers of conditions were used in the presence of divided or otherwise less than perfect ownership (as in Bulgaria and Romania).

79. Further improvements in process are of course desirable, and the case studies point to two areas where these might particularly be sought:

  • The authorities generally do not write the first draft of the LOI, but it is not clear that this should be a source of significant concern. In only a very few cases (one among the case studies) have the authorities expressed any interest in drafting the LOI, although this does not at all mean that they have not been active in designing the program reflected in that first draft or indeed in providing portions of the draft (through the intermediary of existing official documents or specially produced strategy notes), or in contributing to substantive revisions of the draft. It does not seem likely that the authorities will in many cases wish to take on the task of drafting the LOI, but it is clear that the task of document production is less important than the central work of program design.
  • There is little evidence that the staff advises the authorities on a range of available policy options and implementation plans during the process of program development. The staff statement emphasizes that staff should stand ready to do so, particularly in cases where administrative capacity is weak. The process of program design, however, tends to be driven more by an interplay between the staff and the authorities’ initial views, with the staff exploring the room to accommodate the authorities’ preferences rather than proactively developing policy options. Rarely does the staff seem to take the view that the authorities’ administrative capacity is too weak for this interplay to ensure that the range of appropriate policy options is covered. It may be worth encouraging staff to be more proactive in developing a range of policy options, even when the authorities themselves have put forward clearly formulated plans.

80. The case studies point strongly to the great difficulty of gauging ownership, and hint at the possibility that the staff sometimes unduly discounts issues of divided ownership. It is difficult for the staff to come to a definite conclusion that ownership is lacking at the outset of a program (or that ownership may be present now, but liable to change), and difficult also to raise ownership issues with the authorities, given the need to build and maintain a relationship of trust and confidence. Judgments on ownership become especially difficult when some responsible officials are committed to the program, but others are not. While the case studies include examples where calculated risks paid off (notably with regard to parliamentary support in Brazil, Bulgaria, and Tanzania), there are also cases where differences in the degree of commitment or even involvement by government officials, including in key implementing line ministries, became a source of weak implementation.

81. Faced with the difficulty of reliably gauging ownership, the Fund continues to employ prior actions as a screening device, but it is important not to interpret implementation of prior actions as an unambiguous sign of commitment. As noted in Section III.A, arrangements with members with weak track records generally feature more prior actions than others. But while previous analysis already suggested that program implementation (subsequent to the prior actions) for members with large numbers of prior actions was no better than the Fundwide average, updated research suggests that such implementation is actually worse than the Fundwide average.57 Prior actions may still be successful in a limited sense: they may bring programs up to a minimum implementation standard, albeit at the lower end of Fundwide experience. They may also serve to strengthen ownership of the actions in question.58 But prior actions are not a panacea, and due attention must thus be paid to the risk of nonimplementation, or, by extension, of purely nominal or temporary implementation of subsequent conditionality, which will not permit sustained progress toward program goals.

82. There also appears to be a tendency, in cases of weak track records, to raise the number of structural conditions, performance criteria, and program reviews. This tendency is clearly evident in the GRA (see Section III.A), where the number of conditions in weak track record cases is one factor that has compromised parsimony in recent years. PCs are frequently favored in these cases, and indeed account for the bulk of the recent increase in conditions in the GRA. Quarterly program reviews also remain another frequently used instrument in these cases, compromising clarity to some extent.

83. Recent developments in conditionality—particularly in the GRA—thus raise the question whether the Fund has been sufficiently selective in approving arrangements, or whether it has attempted to substitute conditionality for ownership.59 A clear answer to this question must await the availability of outcomes data, but there is certainly a case for renewed scrutiny of proposed Fund-supported programs on this score, to ensure “that the member’s program will be carried out” (guidelines, ¶5) and thus to provide adequate safeguards for Fund resources.

84. While it will inevitably remain difficult to screen out cases with weak ownership, more use could perhaps be made of staff-monitored programs. During the 2000–02 conditionality review, Directors already considered that “greater selectivity, including some period of successful implementation before committing Fund financing, would, in some cases, be the preferred course of action to address instances of past poor performance.”60 Staff-monitored programs (SMPs) appear to have fulfilled their screening function well, on average leading to implementation of subsequent Fund-supported programs at least on a par with implementation elsewhere.61 Of course, SMPs are not appropriate instruments in the presence of acute balance of payments needs; and prolonged use of SMPs, without financial support, risks resulting in adjustment fatigue. Nonetheless, while prior actions can sometimes be met in a “hit and run” way, sustained implementation of sound policies over the period of a SMP makes demands on administrative and political capacity that are comparable to those of an upper credit tranche Fund-endorsed program.

    Other Resources Citing This Publication