Chapter 3 Precrisis Surveillance

Kevin Barnes, Ali Mansoor, Benjamin Cohen, and Shinji Takagi
Published Date:
September 2003
  • ShareShare
Show Summary Details

In this chapter, we present our assessment of IMF surveillance in the precrisis period in the countries covered in this evaluation, focusing on two aspects: how informative was surveillance about the risks that each country faced, and how much impact did it have on the authorities’ policies.

The Diagnosis Role of Surveillance

Predicting a crisis accurately is inherently difficult, especially in circumstances where there are possibilities of multiple equilibria. Surveillance should therefore be evaluated not in terms of its ability to predict the crisis, but rather in terms of effectiveness in identifying the vulnerabilities that could lead to a crisis. Judging from this perspective, our evaluation indicates that the IMF staff was, in varying degrees, aware of most of the vulnerabilities in all three cases. Surveillance was particularly effective when the vulnerabilities were of macroeconomic nature, reflecting the fact that the focus of IMF surveillance during the precrisis period was on macroeconomic issues. The extent of the problems in some cases, however, was seriously underestimated and the surveillance reports failed to link perceived vulnerabilities to an accurate assessment of the risk and the likely dynamics of a crisis.

In Indonesia, staff reports in the period before the crisis noted that the weakness of the banking sector and the buildup of external debt had increased the country’s vulnerability to external shocks. But the true extent of problems in the banking sector, and the degree to which financial system weaknesses had contributed to the poor quality of private investment, were not fully appreciated. While the growth of total external debt was noted, the magnitude of short-term debt and the associated vulnerability were not adequately recognized. The IMF also did not focus attention sufficiently clearly on the increasingly rampant corruption and cronyism that characterized the Indonesian economy. Admittedly, this phenomenon was difficult to document using the usual sources on which surveillance reports rely, but it was a subject of growing concern in academic writing and in the press, as documented in the Indonesia country annex. Downplaying of these issues may have reflected the prevailing approach to governance issues at the time, but it clearly led to an inadequate appreciation of underlying vulnerability.

In Korea, while many of the vulnerabilities that would later contribute to the crisis were identified, the overall assessment turned out to be excessively optimistic. In large part, this was due to the poor quality of the data provided by the authorities on bank loan quality, reserves, and external debt. However, the data that existed, such as those available from the Bank for International Settlements (BIS), were also not adequately utilized.1 At the same time, the surveillance team (in common with most observers in the public and private sectors at the time) was overly sanguine in its interpretation of the data. In particular, there was insufficient appreciation of the risks introduced by Korea’s financial liberalization strategy, which encouraged the buildup of short-term external borrowing by weak, poorly regulated financial institutions. Some internal staff communications raised concerns over the level of short-term external debt. The maturity structure of external debt was an issue raised in discussions with the authorities, but efforts to clarify these concerns, for example by pressing the authorities more forcefully for the appropriate data, do not seem to have been pursued until the crisis had already broken out.

In contrast with Indonesia and Korea, surveillance for Brazil was essentially accurate in assessing most of the elements of the eventual crisis. From as early as 1995, the staff had recognized the vulnerability of the crawling peg to a shift in market sentiment. The staff was critical of the loose fiscal stance and consequent excessive burden on monetary policy, while acknowledging the political obstacles to tightening fiscal policy. Over time, the staff increasingly downplayed the degree to which the real was overvalued relative to historical levels, but continued to advocate accelerating the rate of downward crawl. Until 1998, however, relatively little attention was paid to capital account issues.

The following shortcomings were found to be common to surveillance exercises in two or all three of the countries studied:

  • In Indonesia and Brazil, staff reports for Article IV consultations were often insufficiently candid about potential vulnerabilities, which were raised in a more pointed manner in internal documents and the internal review process—reflecting a tendency to give the authorities the “benefit of the doubt” on issues where assessments of risk were inevitably of probabilistic nature. Internal incentives, which were generally not seen to reward candor if it led to contentious relations with the authorities, contributed to this tendency (see below and also Chapter 5).
  • In Indonesia and Brazil, surveillance reports were not sufficiently frank in bringing to the attention of the Executive Board political factors that might influence the ability of the authorities to implement agreed policy measures. In the case of Indonesia, this reflected a general hesitancy at that time by the Board to delve deeply into governance issues.2
  • In all three cases, crucial data, particularly on the size and composition of external debt and on the health of the financial sector, were not available or could not be relied on. In some cases, this was because key information was withheld or not collected by the authorities. In other cases, available data were not adequately utilized.
  • In Indonesia and Korea, not enough attention was paid to the underlying fragility of the financial sector and the likely impact on capital flows. While some in the IMF expressed concerns in these areas, particularly in internal reviews and through multilateral surveillance exercises (mainly, World Economic Outlook and International Capital Markets reports), these concerns were not fully incorporated into the assessments contained in staff reports for Article IV consultations.
  • In Indonesia and Korea, balance sheet risks, including those arising from currency and maturity mismatches, were not sufficiently explored. This shortcoming was corrected to some extent in Brazil, as the staff correctly analyzed the balance sheet effects of possible devaluation.
  • In all three cases, but particularly in Korea, the possibility that a shock elsewhere in the international financial system could be transmitted to the country in question through global portfolio shifts or changes in risk tolerance (as opposed to more conventional channels such as trade links) was recognized, but surveillance failed to explore the consequences for the specific country being analyzed if such transmission were to occur.
  • In Korea and Indonesia, the IMF drew too much comfort from analyses indicating that the exchange rate was not overvalued or was only moderately so. The possibility of multiple equilibria, that is, the possibility that a change in market sentiment could cause a sharp depreciation even without a major initial overvaluation was not investigated. In Brazil, the IMF did identify significant overvaluation but moderated its own assessment over time.
  • In all three cases, there was not generally enough engagement with the private sector, either regarding its analysis of country conditions or regarding factors influencing their global portfolio allocations and appetite for risk. (In this respect, the dialogue with the private sector in the case of Brazil seems to have been greater than in the Asian cases.) Since country-level dialogue was necessarily concentrated on a small group of senior economic officials, the staff did not always recognize the broader range of views prevalent among current and potential policymakers which would condition policy choices.
  • In all three cases, more effort was put into estimating the likelihood of shocks occurring than into exploring the consequences if a shock were to occur. This reflected an understandable desire on the part of staff members to present management and the Executive Board with a “bottom line” risk assessment as an output of the surveillance process. Yet, once a crisis had begun, the staff’s previous characterization of a crisis as “likely” or “unlikely” in a given country under given circumstances was not of much use to decision makers at the IMF or its shareholder governments. While the surveillance reports produced for the three cases studied here contained elements of a stress test–oriented analysis, and did lead to efforts to improve data collection on areas of potential vulnerability, there were also many topics about which the staff found itselfill-prepared once the crisis had begun, both analytically and in terms of the availability of crucial information.

The Impact of Surveillance

Even where vulnerabilities were identified, the IMF’s surveillance in the period leading up to the crisis tended to have little practical influence on critical policies and was generally not successful in promoting remedial action to address these vulnerabilities. This should not be interpreted necessarily as a shortcoming. As previous internal and external reviews have noted, IMF surveillance is only one influence on economic policies in member countries, and generally not the predominant one.3 While it is too much to expect IMF surveillance to achieve more than it is capable to do, evidence from the three case studies is nevertheless useful in pointing out several factors that contributed to the limited impact of surveillance.

First, surveillance suffered from a reluctance to state candidly difficult or embarrassing facts and views, for fear that this would alarm the markets or generate conflict with national authorities. As documented in the country annexes, the evaluation team has identified a number of occasions when important concerns were raised in internal documents or during the internal review process, but these issues were not adequately reflected or were discussed only in an oblique manner in the documents later prepared for the Executive Board (e.g., concerns raised by the Research Department on banking sector problems in Korea, or identification by MAE of serious governance problems in the Indonesian banking sector). Interviews with staff members suggest that there was a perception that frank, critical assessments, in situations where information was inevitably partial and required an element of judgment, would not receive backing from management or the Board should the authorities object strongly.4 Even if members of the staff or the Board knew of and discussed these issues off-the-record, the fact that these discussions were not contained in written reports hindered effective diagnosis and decision making and made it difficult to transfer country-based knowledge among staff members.

Second, in some cases country authorities were not receptive to the IMF’s policy advice, typically reflecting domestic political constraints (e.g., deregulation in Indonesia). When an issue of highly sensitive nature was involved, such as exchange rate policy in Brazil, there were honest differences of view.

Third, the impact of IMF advice was necessarily limited when no program was involved. This meant that the IMF’s influence was particularly limited by the general strength of capital flows to emerging markets in the period preceding the crisis. The IMF’s views did not figure strongly until the crises were at hand.

Fourth, information weaknesses affected not only the quality of surveillance, but also its impact. As a 1999 review of surveillance by an IMF-commissioned group of outside experts (Crow and others, 1999, henceforth “the Crow Report”) noted, the absence of hard numerical evidence on financial sector weaknesses, reserves, and external debt limited the staff’s ability to make a forceful case to the authorities about the vulnerabilities in Korea. The same also applied to Indonesia, particularly in the area of banking data.

The Role of Transparency

In practice, few of the IMF’s assessments during the precrisis period entered the public domain, apart from generally muted references in multilateral surveillance reports such as the World Economic Outlook and International Capital Markets reports. One reason is that the IMF was wary of the risk of precipitating a crisis through too public a discussion of vulnerabilities. Furthermore, there is a potential conflict between the IMF’s role as “confidential advisor” to the authorities and its role as an information provider and “watchdog” for the international financial community, if its assessments are published.

Although it is not possible to test the proposition rigorously, the evaluation team is of the view that the IMF’s influence would have been strengthened if staff reports for Article IV consultations had been published, so as to influence the public policy debate and promote better risk assessment by private investors and lenders.5 The vulnerabilities that brought about all three crises were widely recognized, if generally underappreciated, in the public and private sectors, so an open discussion would not have come as much surprise to the markets. Instead, the fact that the IMF did not publicize its concerns may have contributed to the market’s tendency toward excessive optimism. Regarding the IMF’s role as a confidential advisor, in practice, in none of the three country cases—except perhaps Brazil in late 1997 and in 1998—was the IMF effective in this area in its surveillance (as opposed to program negotiation) role. Thus, by not publishing its assessments, the IMF had the worst of both worlds. In some cases, the sensitivity of the authorities to the public dissemination of IMF staff views also diminished the staff’s incentives or ability to undertake analytical work, further reducing the impact of surveillance on policy. While it is difficult to generalize from the three cases examined here, the evidence suggests that the benefits of making the IMF’s views public outweigh the costs.

Since the crises, each of the three countries has agreed to the publication of Public Information Notices (PINs)6 and background Selected Issues papers following their Article IV consultations, as well as LOIs and supporting documents when IMF-supported programs have been operative. Nevertheless, up to 2002, none of the three countries covered in this study had agreed to the publication of staff reports, a step that remains voluntary under the IMF’s transparency policy.7 While the publication of PINs represents considerable progress in putting IMF surveillance assessments in the public domain, these notices typically remain somewhat anodyne. Without the publication of staff reports, the full argumentation and nuanced judgments of IMF surveillance are not available to the public.

Recent Initiatives and Further Steps to Strengthen Surveillance

Previous internal and external reviews of the role of surveillance in crisis cases have highlighted many of the same issues discussed above. In particular, a review of surveillance in Mexico before the 1994–95 crisis, which was discussed in the 1995 IMF Annual Report,8 stressed the need for improved data collection; more constructive dialogue with national authorities, including more candid assessment of potential risks; greater frankness at the Board level in assessing member policies; and more attention to financial sector issues. Following the Asian crisis,9 in 1999, the Crow Report recommended, among other things, an increased emphasis on the domestic financial sector, the capital account, and global market conditions; improvements in cross-departmental information exchange; and a focus on identifying vulnerabilities.

The IMF has moved to address many of these concerns in the last several years.

  • Procedures have been put in place to alert management to, and promote greater cross-departmental discussion of, prospects faced by countries identified as particularly vulnerable. In this connection, analytical work has been done on the design and use of various types of early warning systems, although it has not yielded an operationally robust tool for surveillance purposes. Nevertheless, the findings of this work have sharpened the diagnostic capacity of the IMF in the context of surveillance, such as financial soundness indicators, external vulnerability indicators, and, more recently, debt sustainability analyses.
  • The IMF has strengthened its analysis of country-level financial sector issues, most notably through the Financial Sector Assessment Program (FSAP) in collaboration with the World Bank.
  • Reports on the Observance of Standards and Codes (ROSCs) are regularly prepared, and generally published. These reports examine national authorities’ adherence to internationally accepted standards and codes in a number of areas, including especially financial supervision, corporate governance, and data dissemination.
  • The International Capital Markets Department (ICM) was formed, and efforts have been made to recruit staff with financial market experience, in order to give a more prominent role to the analysis of global financial market conditions and of the capital account.
  • A Capital Markets Consultative Group has been established to provide a formal channel for consultations with the private sector, though these discussions currently do not cover conditions in specific countries. According to staff members interviewed by the evaluation team, informal contacts with private sector analysts have also become more common and accepted in the past five years.
  • Quarterly vulnerability assessment experiences were initiated in May 2001 to provide an operational framework for assessing crisis vulnerabilities in emerging market countries, by integrating bilateral and multilateral surveillance as well as market intelligence and IMF-wide country knowledge.
  • Revised guidelines for surveillance were issued in September 2002. Among other things, the new guidelines emphasize the importance of candid discussions of exchange rate issues, comprehensive assessments of crisis vulnerabilities, and measures to alleviate the vulnerabilities that are identified. The guidelines also mandate fuller discussions of the capital account, governance issues, data deficiencies, and the authorities’ responsiveness to previous consultations.

These are valuable steps. However, the current evaluation suggests that the following additional steps would enhance further the role of surveillance in crisis prevention:

  • Surveillance should be oriented toward looking for points of vulnerability, and developing and analyzing stress test scenarios, rather than toward simply trying to predict the future. A full discussion of the real and financial consequences of a menu of possible shocks—such as a worsening of the global macroeconomic environment, a terms of trade shock, a large domestic bankruptcy, or a financial crisis in a neighboring country—would clarify the risks ahead, and would be a useful input to later decision making. If and when one of the identified shocks occurs, the groundwork will have been laid for a more informed exploration of options on the part of IMF management and the Board, as well as the country authorities. A full discussion of scenarios can also help to expose gaps in information and analysis that staff would then attempt to close in advance of a potential crisis. Some IMF surveillance exercises have already begun to use such an approach, for example debt sustainability analyses and stress-testing undertaken in a number of FSAP exercises.10
  • IMF surveillance should identify those structural policies that are most critical to crisis prevention and mitigation and present an assessment in Article IV consultations of the quality of the dialogue with the authorities in these areas, including progress made over time. In many countries, there is an extensive outstanding reform agenda but relatively little effort is made until a crisis occurs to assign priorities to specific reform measures. While continuing to encourage policies that contribute to long-term growth, which may range over a wide area, IMF surveillance should put special emphasis on those policies that would reduce the likelihood and seriousness of a crisis. The revised surveillance guidelines suggest that policy discussions should focus on such issues if “crisis vulnerabilities are non-negligible.” However, it can be argued that such crisis-prevention measures should have a high priority in surveillance of all countries with significant access to international financial markets, since, as the country cases studied here indicate, the seriousness of potential vulnerabilities often do not become apparent until a crisis is imminent.
  • Analysis of balance sheet positions and mismatches has become increasingly common in surveillance reports, but this is not yet done in a systematic or standard fashion. The staff, in Allen and others (2002), has analyzed the role of balance sheet effects in financial crises, and outlined the different mismatches that are most relevant. This could serve as a guide for more systematic analysis of these issues in surveillance reports. More explicit guidelines should be established for the kinds of mismatches that should be examined at the levels of the public, private, and external sectors. This, in turn, would guide the development of statistical reporting systems in support of surveillance and improvements in the timeliness of statistics.
  • Procedures should be introduced to ensure that staff assessments are as candid as possible. To the extent that the staff avoids controversial statements out of fear of a negative response, either directly from national authorities or at the Board level, the Executive Board must play a key role in changing the environment in which surveillance assessments are generated and received. This may mean improving the incentives to produce candid surveillance reports (see Chapter 5). A sharper delineation of the issues surveillance is expected to cover in this area (see above) will also help to promote candor.

While these efforts will undoubtedly reduce the probability of surveillance failing to recognize the risks of a crisis that materializes, the same efforts may also increase the probability of surveillance exaggerating the risks of a crisis that does not materialize. It is important that, with these efforts, surveillance remains realistic in assessing the likelihood of a crisis.


1While coverage was imperfect, both residency-based and nationality-based data on loans extended by banks based in major countries were available from the BIS. On the borrowing side, the data were classified according to the country of residence and therefore excluded, in the case of Korea, the liabilities of Korean overseas affiliates. Some of this information, however, was available from the U.K. and U.S. national sources.
2As discussed in the section “Structural Conditionality” in Chapter 4, the Executive Board adopted a revised approach to governance issues in mid-1997.
3See, for example, “Biennial Review of the Implementation of the Fund’s Surveillance over Members’ Exchange Rate Policies and of the 1997 Surveillance Decision,” SM/97/53, February 1997.
4The existence of perverse internal incentives was also noted in the IEO’s evaluation of prolonged use of IMF resources (IEO, 2002).
5Under current policy, the IMF encourages the publication of staff reports for Article IV consultations, but the ultimate decision on publication is left to the authorities.
6Publication of PINs began in May 1997.
7Beginning with the 2002 Article IV consultation, however, Korea agreed to the publication of staff reports. Some 60 percent of staff reports for Article IV consultations have been published in recent years.
8The underlying confidential report “Mexico—Report on Fund Surveillance, 1993–94,” EBS/95/48, was prepared in March 1995, and is generally referred to within the IMF as the Whittome Report after its author.
9At the height of the Asian crisis in March 1998, there was a preliminary internal review of surveillance in countries affected by the crisis, including Thailand, Indonesia, and Korea (“Review of Members’ Policies in the Context of Surveillance—Lessons for Surveillance from the Asian Crisis,” EBS/98/44). This review identified five key lessons, namely, the importance of timely available data, the need to extend focus beyond core macroeconomic issues, the need to pay attention to policy interdependence across countries, the importance of policy transparency, and the benefits of supportive peer pressure.
10A framework for assessing external and fiscal sustainability is suggested in “Assessing Sustainability,” SM/02/166, May 2002.

    Other Resources Citing This Publication