Surveillance

An important common message from all three reports is the need for greater candor to make surveillance more effective. Each evaluation identifies specific instances where surveillance was less candid than it could have been and where greater candor would have been desirable. This issue is not new; it has often been discussed in the IMF, and these discussions have produced agreement in general terms on the need for greater candor. However, it is not easy to create an incentive structure which encourages candor in practice, especially if transparency is also important.

If surveillance is viewed solely as an instrument for enabling the IMF to act as a confidential adviser, candor does not present serious problems because transparency is not a major issue. However, once surveillance is expected to perform the function of generating peer pressure through the Executive Board, or the broader function of informing markets, transparency becomes essential and this can discourage candor. The authorities are likely to be concerned that highlighting vulnerabilities which are at best probabilistic could precipitate adverse market reactions which would not have occurred otherwise; insistence on candor in such situations is likely to strain relations with the IMF. The Brazil case study shows that the desire to retain influence in the advisory role can lead to moderation of concerns on policy, which can reduce the effectiveness of surveillance. There is no simple solution to this problem, but if the broader role of surveillance is indeed to be strengthened, as has been increasingly emphasized in recent years, it is necessary to find ways of raising the standards of both candor and transparency.

The linkage between surveillance and program design is another important issue surfacing in all three reports. The prolonged use evaluation indicates that programs have tended to crowd out surveillance activities, thus reducing the potential contribution of surveillance to improving program design, while the fiscal adjustment evaluation suggests that surveillance has drawn too few lessons from past failures. The capital account crises evaluation suggests that forcing wide-ranging reforms during a crisis, when these have not been adequately discussed earlier in the course of surveillance, can weaken ownership. The common message is that surveillance can help in future program design if it provides a frank assessment of critical weaknesses which need to be addressed and encourages the authorities to develop a road map of reforms to address these weaknesses. This would increase ownership of policy changes which may be included in future programs.

The evaluations show that one reason why it is difficult to ensure that conditionality in IMF-supported programs is consistent with longer-term objectives is the mismatch between the time frame of IMF-supported programs and the much longer time frame needed for structural reforms. For example, reforming the tax system often calls for structural and administrative reforms which take considerable time to implement, and it may not be possible to devise appropriate prior actions or performance criteria which would ensure genuine progress towards this longer-term objective within the short time frame of IMF arrangements. In fact, the need to achieve quick fiscal improvements may even generate adverse incentives, focusing attention on actions that are more easily quantified and monitored but are less important for longer-term sustainability. The fiscal adjustment report provides examples of such actions: raising tax rates rather than broadening the tax base or, on the expenditure side, imposing wage cuts rather than undertaking deeper civil service reform.

The mismatch problem could be overcome if the structural and institutional components of IMF-supported programs were drawn from a longer-term strategic framework, which is owned by the country. In principle, the PRSP process provides a mechanism for low-income countries which could provide such a framework and the ongoing PRSP/PRGF evaluation will throw light on how far this is achieved in practice. It is not clear what mechanism, if any, can serve this purpose for other countries. The fiscal adjustment evaluation, in the context of protecting critical social programs, suggests that surveillance could be used to invite the authorities to spell out their reform objectives and the IMF (and where appropriate the World Bank) could assist the authorities in working out reform programs if requested to do so. However, this approach could raise apprehensions that it would make surveillance potentially more intrusive. The idea of using the surveillance dialogue to achieve an understanding on the appropriate framework for longer-term structural reforms is clearly appealing. How to do it in a manner which keeps the country in the driver’s seat to ensure ownership poses a major challenge.

Program Design and Uncertainty

All three evaluations highlight the difficulty in dealing with uncertainty in program design. Uncertainty is a familiar problem for all types of policymaking but, arguably, it presents greater difficulties in the design of macroeconomic programs where outcomes depend upon the interaction of several factors, many of which are subject to large degrees of uncertainty. For example, the state of investor expectations that influences the level of private investment, or the state of market confidence that affects the availability of external financing, are difficult to quantify, and investors’ response to different types of policies is even more difficult to predict. Yet programs depend critically upon assumptions about how these factors will behave in response to exogenous developments and policy changes.

Since uncertainty is inescapable, programs have to be based on probabilistic assessments which may turn out to be wrong, but it is important to avoid excessively optimistic assumptions. However, all three evaluations suggest a systematic bias toward overoptimism with some undesirable consequences. For one, it leads to unrealistic conditionality—for example, overoptimistic fiscal targets are adopted based on overoptimistic revenue forecasts arising from overoptimism about growth and about the short-term impact of tax reforms. In certain circumstances, it can also lead to inappropriate program design, as when overoptimism about the pace of recovery of private investment encourages excessive fiscal tightening to make room for private investment which does not materialize as quickly as expected. Although program targets are adjusted in practice, in the light of developments, this is clearly not a substitute for making more realistic assumptions in the first place. Underperformance in growth and frequent revisions of targets can undermine the credibility of IMF-supported programs.

Realism is desirable but it may not be enough, since even realistic forecasts may not materialize. It is therefore important to ensure that the risks to programs are explicitly considered and there is sufficient flexibility in program design to deal with unexpected developments. All three evaluations undertaken thus far suggest that programs typically do not have well-defined contingency plans worked out in advance to take account of uncertainty. This can be especially problematic in capital account crises, where countries can experience a sudden shift from a “good” to a “bad” equilibrium, in which case what is needed is not merely a marginal recalibration of policies, but possibly a radical alteration in program design.

Explicit contingency planning to deal with such situations would help achieve flexibility, but there may be resistance to this approach if it is felt that contingency plans may become widely known, which could lead markets to overreact to possible risks. In certain circumstances, this could trigger destabilizing behavior and actually undermine the probability of success of the original program. The problem is particularly acute in capital account crises, where the impact on confidence in the early days of the program is often critical and there is an understandable desire to downplay the possibility of unfavorable but hypothetical outcomes.

Even if explicit contingency planning is not undertaken, it should be possible to identify the critical assumptions (including assumptions about policy responses) on which the program depends, and to monitor events closely so that program redesign is triggered as early as possible if needed. This calls for greater transparency about the underlying assumptions and rationale of program design. The fiscal adjustment evaluation shows that in many cases the underlying rationale is not made sufficiently clear in program documents. This is clearly an area where practice can be greatly improved, and it should be the focus of subsequent evaluations.

Conditionality and Ownership

All three evaluation reports suggest that the specific structure of conditionality is much less important than the underlying domestic political commitment to core policy adjustments. This is exemplified by the experience in Pakistan, where conditionality extended to introducing taxation of agricultural income, or in the Philippines in connection with reforms on tax administration. The fiscal adjustment evaluation refers to instances where lack of commitment to reduce widespread evasion forces programs to rely on increasing tax rates in existing “easier to collect” taxes. Another example is the experience of Indonesia where complex conditionality structures, which were ostensibly accepted publicly by the authorities, failed to impress markets because political commitment was perceived to be lacking.

If strong domestic ownership is essential for success, it follows that there will be circumstances in which countries may not be ready to undertake all the adjustment and reform that may be needed, which raises the issue of how the IMF should respond in such cases. The prolonged use evaluation argues that the IMF must be willing to consider “second-best” adjustment programs which are not ideal, but nevertheless meet a minimum standard of acceptability. How to establish the minimum standards which would enable the IMF to respect considerations of ownership, while ensuring a sufficient degree of sustainability and also preserving incentives for stronger adjustment, is a major challenge.

A corollary of this approach is that where minimum standards are not met, the IMF must be willing to hold back from financing. As the prolonged use evaluation points out, this approach can lead to interruption of other flows and this is sometimes a factor which encourages the IMF to continue lending. The trade-offs clearly involve very difficult choices and it is difficult to envisage rules that can be mechanically applied. However, it should be easier to make judgments about sustainability if a clearer picture of reform road maps were developed through surveillance.

Selection Bias

Finally, it is important to recognize that because the IEO does not evaluate all programs or surveillance cases, IEO evaluations are necessarily selective and some selection bias is therefore inevitable. Cases chosen for evaluation are likely to be those where programs ran into difficulties, if only because these are precisely the cases which attract the greatest attention and public comment. Ex post evaluation of such cases is obviously necessary for accountability and can contribute to learning. However, this does imply that IEO evaluations are less likely to focus on successful programs even though there is surely much to learn from success stories.

This bias is less likely to arise in the case of thematic studies, where the IEO evaluation covers a much larger range of experience. This is evident from the fiscal adjustment evaluation, which illustrates the wide variety of experience under IMF-supported programs. This suggests that the IEO should ensure a reasonable balance between country and thematic studies in its work program.

As stated at the beginning of this chapter, these reflections emerge from a limited set of studies and it would be premature to present them as firm conclusions. They are presented only as common themes that have surfaced from more than one evaluation and which will need continuing attention in the future.