I. Methodological Framework for Assessing the Effectiveness of Surveillance¹

1. An important contribution of the 2008 TSR is the establishment of a strengthened methodological framework for assessing the effectiveness of Fund surveillance (EFS). The 2008 TSR is the first opportunity to establish, implement, and fine-tune the methodological framework for EFS assessment discussed by the Board in February 2007. This background paper presents the framework and reports on its implementation in the 2008 TSR.

The EFS assessment framework

2. The framework is based on a working definition of EFS and a set of seven key diagnostic questions that reflect prior work on the priority quality attributes of surveillance (Appendix). The EFS working definition is that surveillance is effective when it provides, to each member country and to the international community, the information and policy advice best-suited to help preserve global financial stability and the external stability of each member country. Whether the information is used and advice followed is critical for ultimate effectiveness, but largely beyond the Fund’s control. The diagnostic questions, which derive from staffs earlier work, endorsed by the Board in February 2007, emphasize substance, quality, and outcomes over process, quantities, and outputs. They reflect the understanding that priority quality dimensions of effective surveillance are relevance, appropriateness, practicality, candor, and evenhandedness. The questions (which are supported and clarified by more detailed sub-questions) are:

(a) Is surveillance focused on the issues that are most critical to its mandate?

(b) Is the analytical content of surveillance of sufficiently high quality to add value reflecting the Fund’s unique expertise?

(c) Is surveillance candid?

(d) Is surveillance consistent and evenhanded?

(e) Is surveillance effectively communicated to its key audiences?

(f) Is surveillance having an impact?

(g) Is surveillance cost-effective?

3. The framework also entails an agreed set of sources of information and techniques to answer the diagnostic questions, allowing for triangulation. The three categories of information deemed most relevant for EFS assessment are output quality, outcomes, and impact. In principle, TSRs would collect information on all three, but this one focuses on the quality of outputs and on value-added, which is a form of impact. In the future, information on outcomes and impact will also be collected, drawing, inter alia, on surveillance agendas. The agreed techniques and their use in the 2008 TSR are covered in detail in the next section. A key feature of the approach to information and techniques is that it supports triangulation. That is, when possible, findings about particular aspects of EFS are drawn from different sources of information or different audiences, using a variety of methods, and compared for consistency. For example, different internal and external audiences were asked similar questions about the comprehensibility of staff reports.

4. The framework provides the basis for assessing the quality of surveillance over time and across countries and for assessing progress against time-bound surveillance priorities. Findings under the main diagnostic questions establish performance benchmarks for future TSRs. Alongside such permanent components, the framework also contains specific provisions for assessing performance against the more time-bound priorities outlined in the three-year Statements of Surveillance Priorities (SSP). Collected information is analyzed not only in aggregate, for the membership as a whole, but also separately for key country groupings (e.g., advanced/emerging/developing; area departments’ coverage; floaters and peggers etc.). This permits the identification of systematic differences in treatment, which can then be examined against the background of underlying economic realities to look for suggestions of bias or lack of evenhandedness.

Assessment techniques and their application in the 2008 TSR

5. The EFS assessment framework mainly features the following techniques. For all of the techniques covered, the reader is referred to the Statistical Appendix for additional and more detailed information on how they were implemented in the 2008 TSR, as well as the results of their application.

Standardized qualitative assessment of a representative sample of staff reports

6. A representative sample of Article IV reports were internally reviewed in order to collect standardized qualitative information across the main diagnostic questions.

The review was conducted by PDR economists in a division not involved in the review process of individual country surveillance reports. The universe from which the sample was selected was defined as the 88 Article IV staff reports discussed by the Board in the period July 1, 2007 to February 28, 2008. A sample of 50 reports/countries was reviewed, rather than the universe, to allow for more qualitative and in-depth review of staff reports, compared to earlier surveillance reviews. Detailed questionnaires corresponding to the main diagnostic questions were developed by the team to collect standardized qualitative information on the reports.

7. Several precautions were taken to maximize consistency across reviewers. Pairs of economists from the review team “test-drove” the questionnaire for two countries per pair. That is, pairs of economists reviewed a given set of two countries against the full questionnaire and then met to compare their responses and exchange information about problems encountered and the solutions adopted. This measure helped develop common understandings across the reviewers on how to interpret the questions and implement the questionnaire uniformly for staff reports reflecting a range of country and operational circumstances; this in turn helped ensure consistency of assessment across reviewers. Furthermore, when the internal review was completed, the team member responsible for a given part of the questionnaire checked the results across the 50-country sample for accuracy and consistency. Countries were divided up among reviewers in a way that ensured that each reviewer reviewed a group of countries that was diverse in terms of area departments, income level, and Fund program status.

8. PDR research assistants (RAs) reviewed the reports to provide the perspective of “an average college-educated reader.” RAs who were relatively new to the Fund reviewed the executive summaries and staff appraisal of staff reports and answered several overarching questions on the readability of reports and the clarity of key conclusions provided by the report in specific areas. These questions and the RAs’ responses provided checks for questions in thematic parts of the questionnaire on candor and evenhandedness, focus, and exchange rates.

Case studies

9. Case studies are useful for drilling down, i.e., for undertaking thorough and comprehensive analysis of developments where country, temporal, or other specific circumstances need to be taken into account, as well as for topics that require gathering information beyond that available in staff reports; additional sources considered in case studies could include additional internal documents (e.g., briefing papers) and external ones (e.g., media coverage of Article IV consultations, relevant/concurrent analyses produced by other international institutions, credit rating agencies, consultancies, or research institutes), and interviews. Five case studies were conducted in the 2008 TSR (see Box 1).

Stakeholder surveys

10. Surveys are particularly helpful to refine diagnostics, understand the drivers of certain phenomena, and cast light on desirable remedies to problems identified separately. Key groups are: country authorities (to gauge their views of surveillance of their country); Executive Board members (with emphasis on their views as proxy for “the international community” on surveillance of countries other than the one(s) they represent); Article IV mission chiefs (as “producers”); and the public, including financial market participants, opinion-makers (media, think tanks), and civil society organizations (CSO). Both anonymous surveys and structured interviews by staff or, where more appropriate, an independent consultant, are expected to be used in the EFS assessment framework over time. For the 2008 TSR, stakeholder consultations were carried out through surveys and, in some cases, interviews with all of the above groups as detailed in the Section III.

11. Problems encountered. Response rates for the surveys of think tanks and CSOs were so low (320 think tanks contacted yielded a 3 percent response rate; and 1000 CSOs contacted a 0.3 percent response rate) that feedback from these sectors was not incorporated, and in the future other means of obtaining feedback from these audiences will need to be explored. Ex post, scope for improving the parallelism of questions on similar topics in surveys of different audiences was recognized (for example, in the question on the overall quality of analysis, the breakdown of different areas of analysis was not identical across surveys). A more general problem is that, ex post, the sample of respondents to a survey or participants in interviews may not be fully representative of the original population targeted.

Given that participation is voluntary, little can be done about this, beyond making the results available by relevant subsets within a given targeted stakeholder group, when possible. In the case of the survey of Executive Directors, the breakdown of respondents across country categories in not known, as to preserve their anonymity, it was felt necessary to refrain from asking respondents questions related to the country(ies) they represent. Thus while the overall response rate (58 percent) was satisfactory, it is an open question whether the responses really capture the diversity of views of the full Board.

Studies by independent consultants

12. Using independent consultants to investigate particular questions, themes, and/or audiences provides a different governance structure to assessments conducted by staff. Such consultants may conduct a survey, review documents, or use other sources of information, sometimes in combination with their own background or expertise. They provide views and analysis that have the potential advantages of (i) eliciting more candid feedback from both internal and external persons who are interviewed or surveyed and (ii) providing independent, external views on surveillance. For the 2008 TSR, an independent consultant was hired to conduct one of the case studies ((v) in Box 1), and another to conduct structured interviews with country authorities. Both of their reports are provided as part of the TSR documentation.

Indicators

13. Output indicators—numerical indices summarizing information on outputs and output quality—can be useful, but should be drawn on sparingly and with caution, given their weaknesses. For the 2008 TSR, a proximity indicator that measures the sharpness of the focus of SIPs, working papers, and surveillance agendas on issues most likely to be relevant to external stability was developed as part of the analytical underpinnings of surveillance case study. Also, an indicator on the extent, quality, and impact of cross country analysis in SIPs was developed as part of the thematic study of cross country analysis and spillovers.

Reporting in Article IV staff reports of past surveillance advice uptake

14. The requirement for staff reports to report on follow up to past surveillance advice earns mixed reviews as part of the EFS assessment framework. Since the 2002 review of surveillance, staff reports have been required to include a brief assessment of the authorities’ response to the key policy challenges identified in previous consultations—a requirement instituted specifically with an eye on providing information on the effectiveness of past surveillance. The staff paper for the February 2007 Board seminar noted pitfalls of this measure, notably that the results are not detailed enough to allow conclusions to be drawn on the effectiveness of surveillance on a country-specific basis, let alone to be aggregated to form a membership-wide assessment. Also, it may be inappropriate and distortionary to assess the quality and impact of staffs past advice in the same forum where the staff’s current assessment and policy advice is presented. The measure also suffers from the pitfalls of self-assessment, since the assessment of uptake of advice is done by the same department and staff that developed the advice. The 2004 surveillance review found that the reporting in many cases was “cursory, partial, and pro-forma.” The present analysis finds that only 60 percent of staff reports include this “required” information, and in only about one-half of these cases is the information reasonably substantial, i.e., one-half page in length or more.

15. Further experience has clarified that this reporting serves two distinct functions. For some consultations, it can provide useful background information to Executive Directors; but it was instituted as a universal requirement in Article IV reports not for this purpose but as a means of EFS assessment. The first function is clearly valuable, while as the preceding paragraph indicates, the second is not well served. Staff therefore proposes that it be understood that the sole purpose of the reporting is the first function. Therefore staff could use judgment in deciding whether the information is useful background but would not be required to present the information as a form of EFS assessment. These sections of staff reports do not say anything about whether the authorities’ actions were influenced by surveillance, and there is some risk of selective reporting when key areas of advice are not identified ex ante. In future TSRs, staff will be able to draw on the three-year surveillance agendas to assess actual outcomes against those intended ex ante.

Assessing performance against the SSP

16. As part of the TSR, the performance against priorities established in the SSP will be assessed. It is expected that this assessment will draw on the assessment tools outlined in the previous section. The SSP provides that at the time of each TSR, the Managing Director will report on progress in attaining the SSP priorities, management’s and staffs contributions, and the factors that impeded progress. The specific questions set out in the box accompanying the SSP (see Annex in the overview paper) will provide useful guidance to develop specific performance benchmarks, and it is expected that assessment of the SSP priorities will draw on the various tools outlined in the section on assessment techniques above.

Basic Framework for Assessing the Effectiveness of Surveillance

Object of the assessment. Surveillance is effective when it provides, to each member country and to the international community, the information and policy advice best-suited to help preserve global financial stability and the external stability of each member country. Whether the information is used and the advice followed is critical for ultimate effectiveness, but largely beyond the Fund’s control. What the Fund is responsible for in making surveillance effective is the focus of its efforts, the richness of its analysis and advice, and the quality of its communication. It is these that enable the Fund to maximize, within its mandate, its relevance to its members. The actual impact of surveillance will also depend on the Fund’s legitimacy—a dimension largely related to governance, but for which evenhandedness in surveillance is also important.

Key questions. This broad conceptual framework can be translated into a list of overarching questions that should constitute the backbone of future TSRs. Relative emphasis on the different aspects could vary over time as circumstances require. The key questions—which conceptually apply to all forms of surveillance: bilateral, regional or global—are listed below (focus areas for the 2008 TSR are italicized).

(a) Is surveillance focused on the issues that are most critical to its mandate?

• ❖ Is surveillance (and underlying analytical work) focused on global and single country domestic and external stability, and their interdependence?

• ❖ Is analysis focused on areas where risks are greatest?

• ❖ Is surveillance timely in its analysis of issues?

(b) Is the analytical content of surveillance of sufficiently high quality to add value reflecting the Fund’s unique expertise?

Analysis of risks and real-financial linkages

• ❖ Does surveillance provide adequate health checks, including sharp analysis of exchange rate issues ?

• ❖ Does surveillance adequately integrate the analysis of economic and financial risks?

• ❖ Is surveillance good at diagnosing risks and anticipating adverse developments?

Multilateral perspective

• ❖ Does surveillance help members understand the international context and implications of each member’s policies (both inward and, where relevant, outward spillovers)?

• ❖ Does it make enough use of the Fund’s cross-country knowledge?

Quality of Policy Advice

• ❖ Are surveillance recommendations appropriate and well-tailored to the economic situation?

• ❖ Do surveillance recommendations have enough operational acumen? Or are they sometimes impractical given country circumstances, or too generic to be useful?

(c) Is surveillance candid? ²

• In assessing domestic and external stability, does surveillance too often wear rose-colored spectacles (e.g., assume problems away)? Does it tend to focus too much on central scenarios at the expense of risks?

• ❖ Does surveillance tend to couch the description of problems (e.g., vulnerabilities, exchange rate misalignment) or recommendations in unclear or overly hedged language?

(d) Is surveillance consistent and evenhanded?

• ❖ Is the quality of surveillance (as defined in a, b and c above) consistent across countries and country groups?

• ❖ Are surveillance policy recommendations consistent across countries (i.e., similar for countries in similar conditions)?

• ❖ Is the degree of candor in assessing risks and recommending policy adjustments consistent across countries?

(e) Is surveillance effectively communicated to its key audiences?

• ❖ Are the key vehicles available (mission concluding statements, staff reports, summings up, published bundles) effective and efficient in conveying the messages of surveillance to their key audiences—country authorities, the Board, the broader international community, and the public? In particular, are staff reports and bundles (in their current format) well-suited to communication with the Board, the international community, and the public?

• ❖ Are other forms of communications and outreach used sufficiently and effectively?

• ❖ Is surveillance communicated in a sufficiently timely fashion to its various audiences?

• ❖ Separately from the vehicle, are surveillance messages presented in a convincing, user-friendly manner (e.g., well argued, prioritized)?

(f) Is surveillance having an impact?

• ❖ Is surveillance significantly informing policy decision making? the public debate? markets?

• ❖ Are the policy recommendations of surveillance followed?

(g) Is surveillance cost-effective?

• ❖ Do the modalities of surveillance allow it to deliver the best value at the lowest cost?

II. Report on InterviewsWith Country Authorities³

1. Interviews were conducted with senior officials from 19 countries.⁴ According to the WEO classification, 7 countries belong to the group of advanced economies and 12 to the group of emerging market and developing countries.⁵ The interviews were based on a brief questionnaire distributed to interviewees before the meetings and concentrated primarily on the key questions related to implementation of surveillance to be examined in the 2008 TSR, namely: appropriateness of focus, given the mandate of surveillance and the circumstances of the member; analytical value-added; and the effectiveness of current modalities of bilateral surveillance, especially the way in which its messages are communicated to the authorities and other audiences in the country.

2. Almost all of those interviewed said IMF surveillance added significant value, through several channels: (i) as an integrated macroeconomic assessment from a global perspective; (ii) as a test against the authorities’ own judgments; (iii) as a transparent source of standardized information and, for some countries, (iv) as a source of specific policy advice. On the latter, most interviewees from advanced economies said surveillance generally did not say anything they had not already heard about policies, but it provided a useful comprehensive synthesis. This was especially appreciated by interviewees from smaller advanced economies. Many interviewees from emerging markets noted that the uniqueness of IMF surveillance as an external assessment was less than before, which highlighted the importance of adopting surveillance outputs so as to maintain IMF relevance. The messages from interviewees across different groups of countries were broadly similar, except where noted below.

Appropriateness of focus

3. Many regarded the 2007 Decision as a welcome legal framing of what IMF surveillance had already been doing in recent years. Therefore, in judging its implementation, they did not expect to see radical changes in approach. In any event, it was still too early to judge the full extent of any changes on those countries that had not yet completed a full cycle under the new framework.

4. Almost all those interviewed supported the new focus, but stressed that it had to be tailored to specific country circumstances. In particular, the new framework should not become another “check list.” For example, several of those interviewed had the impression that some surveillance reports were overemphasizing the discussion of regional spillovers or deviations of exchange rates from “equilibrium” even when the evidentiary basis was limited and other issues were more important.

5. Many of those interviewed thought that the coverage of exchange rate issues still lacked uniformity. For example, there was wide disparity in the coverage of CGER analysis across surveillance reports with different advanced and emerging market economies. Many emphasized the need for new operational guidelines on surveillance to ensure greater uniformity. Some thought the treatment of exchange rate issues in the bilateral surveillance reports for member countries of currency unions as integrated as those of the EU was especially problematic, since they did not think the current account balance of individual member countries of the union contained useful information about the exchange rate.

6. Interviewees thought the FSAP approach had added major value and had helped to deepen surveillance of financial sector issues, but the follow-up could be improved further. The integration with surveillance can be patchy and needs to move further from a follow-up of static assessments against standards to a more dynamic analysis of key risks and risk management. A number of interviewees said the insufficient technical expertise on financial sector issues of regular Article IV missions had reduced the value of some follow-up discussions on the financial sector.

7. There was broad support for deeper surveillance coverage of real-financial sector linkages. Many interviewees recognized that this involved significant analytical challenges, but that the Fund should take the lead in analyzing such linkages from a global perspective. For example, more attention should be given to approaches that treated endogenously the consequences of various shocks for credit expansion and growth. One interviewee noted that the FSAP Handbook already set out possible analytical approaches and another suggested that greater efforts could be made to incorporate World Bank analysis of the real sector into surveillance exercises.

8. Many interviewees stressed the need for realism about the Fund’s role in predicting crises. Financial markets are highly dynamic and the IMF neither could nor should be expected to detect all potential crises, especially if domestic supervisors failed to do so. The Fund role should be to highlight the most significant vulnerabilities as it sees them, with the focus on global inter-linkages. In this respect, some interviewees thought that the Fund had been slow to analyze the potential ramifications of the U.S. mortgage-financing crisis.

9. While recent surveillance reports were moving in the right direction, many interviewees would like to see even more emphasis on distilling relevant experience from other member countries. Several interviewees mentioned specific examples where greater emphasis on analyzing how other countries had addressed certain policy issues would have been useful (e.g., dealing with the macro challenges of terms of trade appreciation in advanced economy natural resource exporters; detailed policy responses to capital inflows in emerging markets). They noted, for example, that more could be done in bilateral surveillance to draw out country-specific policy lessons both from multilateral surveillance analysis and from bilateral surveillance of other countries facing similar policy issues. (See the next section for specific examples.)

10. A number of those interviewed thought that bilateral surveillance still gave too much attention to general structural reform issues, where the IMF often had little new to add. Several mentioned pro forma and rather superficial discussions of trade policy as an example. Fund surveillance should be even more selective and go deeper with topics chosen according to country circumstances and linked more dynamically to future macro challenges. Some interviewees noted that the Executive Board had to contribute to this selectivity, by refraining from new mandates outside the core competencies and not criticizing staff when greater selectivity means some issues are not covered.

11. While there was broad support for a greater emphasis on regional linkages where these were substantial, the relevance of a more “regional” approach to country surveillance depended on each country’s circumstances. Interviewees from countries where regional trade and financial linkages were not large said greater regional surveillance risked distracting from more important activities. Some from relatively smaller countries, including among the advanced economies, said a shift to regional surveillance reports could risk crowding out some issues of importance to them. In the words of one interviewee, earlier attempts to “streamline” surveillance, with distinctions between systemic and non-systemic countries, had not been well thought through, since the integrated assessment of IMF surveillance was often of greater value added to the latter countries. In countries where regional linkages were substantial, interviewees saw greater regional surveillance as a useful supplement to, rather than a replacement of, bilateral surveillance. In such cases, the main value added would come from an exploration of economic and financial spillovers within the region rather than from comprehensive regional surveys.

12. In countries with IMF-supported programs, a broader surveillance dialogue is still crowded out by the continued focus on detailed program implementation during Article IV missions. Interviewees from developing countries who commented on this issue said a great surveillance focus on the forward-looking macro challenges of enhancing growth was needed, rather than detailed reporting on program implementation. This would require greater selectivity, with some interviewees welcoming what they saw as a recent reduction in surveillance attention given to detailed social expenditure and poverty issues.

Analytical value-added

13. Many of those interviewed thought the exchange rate analysis (specifically, the CGER exercise) was not yet sufficiently robust to justify the weight put upon it in many recent surveillance reports. In particular, a good conceptual definition of ‘external stability’ was still lacking to make the framework operational. While most supported the Fund’s continued efforts to develop its analytical tools in this area, the language used in reporting results needed to be very cautious. Greater uniformity of treatment across countries was also needed. Several interviewees also said the Fund was often too quick to translate an analysis of price competitiveness into exchange rate policy advice, whereas many other policies also influenced cost structures. In contrast, some others stressed that, while the underlying analysis was inevitably imperfect, the Fund as an institution was still not sufficiently forceful in expressing its conclusions when there was evidence of major misalignments.

14. Many interviewees identified the Fund’s fiscal analysis as a particular source of value-added. Longer-term fiscal sustainability exercises that took account of ‘hidden’ fiscal costs in the broader public sector had often been helpful in adding to the transparency of the public policy debate, even in a number of cases where the underlying facts were already well-know to the authorities.

15. Treatment of financial sector issues in surveillance has greatly improved in recent years, in large part because of the FSAP, but further improvements in the analytical toolkits used in country work were needed if the Fund were not to fall behind the curve. Some interviewees said that the follow-up of the FSAP process in subsequent missions had not had the same ‘level’ of dialogue as on more general macro policies; in some cases, there as too much of a ‘checklist’ approach to following up on minor details rather than considering risks and risk management in a holistic manner. One interviewee suggested that, while it was unrealistic to expect the Fund to predict specific crises, it could do more to explore in advance possible spillover routes of financial shocks and the influence of different institutional settings. Another noted that, while the FSAP Handbook already had a useful chapter on approaches to treating endogenously credit expansion and economic growth, most FSAPs relied on simpler stress tests; consequently, the ‘next generation’ of FSAPs and surveillance needed to upgrade the analytical tools applied.

16. Many said they would like to see more evidence-based policy advice, adapting global analysis and cross-country experience to country-specific circumstances. One positive example cited of such analysis was the recent study of the impact of high oil prices, which had used the Fund’s comparative advantage to combine a global assessment with regional and country specifics. One issue mentioned where a number of interviewees from emerging market and developing countries would like to see more concrete, operational advice was on the interlinked issues of capital market reform, financial sector liberalization, and foreign exchange markets.

17. The timing of IMF analytical inputs influenced their effectiveness. For example, a number of those interviewed said the Fund’s analysis of the fallout from the sub-prime crisis, while comprehensive, had been less timely, and therefore less helpful to them in providing a quick assessment of the potential consequences for their own country than reports by others, including investment banks. A number of those interviewed said they relied more on reports produced by a few major central banks for information on developments in key advanced and emerging market economies, because of their greater frequency. Moreover, the timing of IMF analysis needed to determined as part of a broader strategy to maximize the input to the domestic policy debate, not supply-driven by the interests and capabilities of the Article IV team. Several interviewees quoted examples where IMF analysis of an issue had been of high quality, but had come too late to influence the domestic policy debate. However, many said that pre-mission consultations on what were likely to be the most critical policy issues for surveillance had improved the relevance of the dialogue and of thematic background reports in Selected Issues papers.

18. While acknowledging recent improvements, many of those interviewed said bilateral surveillance needed to take greater account of country-specific institutional elements. In addition to political economy considerations (e.g., advice to implement across the-board reductions in subsidies was often not realistic), a number quoted examples from the financial sector where they thought that greater familiarity with the institutional setting would have improved the quality of the policy advice.

19. A number of interviewees from emerging market and developing countries said the Fund approach to analyzing macroeconomic issues in their countries—built around the financial programming framework—was behind the times and needed updating.

More sophisticated and ambitious approaches, especially to analyzing real-financial sector linkages, were potentially available (see earlier discussion).

Modalities of bilateral surveillance

20. Interviewees commented on two aspects of surveillance modalities: (i) the way in which IMF surveillance messages were communicated (i.e., the frequency, speed, and product design of bilateral surveillance outputs and the quality of communications), and (ii) factors affecting the efficiency of surveillance.

21. No strong demand for a radical change in the types of surveillance output emerged from the interviews, but many thought the time lags needed to be shortened.

Many said that the key value added—in terms of influence on policymakers—came at the time of the concluding statement of the Article IV missions and any associated briefings of high officials. Production of staff reports and the subsequent Board discussion several months later generally had limited additional impact. Many thought the time between the end of the staff visit to the country and the subsequent Board discussion and release of the staff report was too long. Some also suggested that less time and resources could be devoted to preparing the formal staff reports, since the mission’s concluding statement already contained the key messages. One interviewee suggested that resources could be shifted to producing shorter, but more frequent and hence more timely, policy-oriented notes on key groups of countries.

22. In contrast, many interviewees argued strongly that the way in which surveillance messages were communicated in the various surveillance outputs needed substantial further improvement. Many said that, while the drafting of Article IV reports had improved in recent years, more work was needed to produce shorter, sharper, and more focused reports. Several gave specific examples where they had revamped the language of staff reports in order to extract salient messages to brief senior policy officials or parliamentarians in a form that would have more influence on the domestic policy debate. One interviewee said the informal approach of many of his colleagues was to “only read the Boxes,” since they tended to contain most that was original in a staff report.

23. The language of the Board Summing Up was behind the times, in terms of modern approaches to policy communications, according to a number of those interviewed. Several said that the language used was too coded and cryptic and therefore less accessible outside a narrow circle --”as if the Fund is talking to itself,” in the words of one interviewee. One Finance Minister said the Summing Up was an important document for his country, since he used it to convey the IMF messages to his Cabinet colleagues; however, he had to ask his staff to “translate” it into plain English before it could serve this purpose. Several of those interviewed suggested the Fund could learn more from recent advances in the way some central banks conveyed their policy messages (e.g., in the context of inflation-targeting frameworks). The conventions used to convey the number of Directors expressing a particular view (‘some,’ ‘a number of,’ ‘a few,’ etc.) were judged especially opaque by many interviewees, who said such language detracted from the clarity of the surveillance message.

24. A variety of views were expressed on the Fund’s transparency initiatives. Most of those interviewed favored publication of country surveillance reports, but a number argued that the emphasis on making public the Fund’s advice, except on those issues that were highly market sensitive, had gone too far and risked undermining the Fund’s role as a confidential advisor in their countries. This group thought the appropriate degree of transparency should be decided according to country-specific circumstances. In any event, many said that more scope should be given for the authorities to present their views when they disagreed with the Article IV mission’s concluding statement, since the opportunity to express such views at the time of the Board discussion could, in their view, come too late.

25. While many stressed that surveillance was a core activity of the Fund and needed adequate resources, several suggestions for improving the efficiency of surveillance operations were made during the interviews, including (i) greater use of modern communications prior to the Article IV mission to reduce the large amount of time devoted to technical questions; (ii) more use of videoconferencing with relevant HQ experts so that scarce IMF expertise (e.g., on the financial sector) is used efficiently during surveillance; (iii) devoting more in-house research resources to developing analytical toolkit modules for country teams (e.g., for analyzing real-financial sector linkages); (iv) developing more systematically a “knowledge data bank” on different country experiences with a select group of policy issues that were likely to arise frequently in coming years (possibilities mentioned included responses to capital inflows, operational intricacies of exchange markets, and comprehensive fiscal balance sheets); and (v) further efforts to reduce staff turnover on Article IV missions, although many recognized recent progress in this area.

Country Interviews Drawn Upon for this Report

Australia	France
Germany	Hungary
Switzerland	Iceland
Japan	Latvia
United States	Russia
China	Serbia
India
Indonesia
Lebanon
Saudi Arabia
Cameroon
Morocco
Bolivia

View Table

Australia	France
Germany	Hungary
Switzerland	Iceland
Japan	Latvia
United States	Russia
China	Serbia
India
Indonesia
Lebanon
Saudi Arabia
Cameroon
Morocco
Bolivia

III. Supporting Information for the Subprime Case Study

1. This section provides more detailed, but not exhaustive, factual supporting information on staff analysis produced in the period leading up to August 2007, organized by country case. References to paragraph numbers in the main text are to Supplement 1, Chapter VII.

IV. Statistical Appendix¹²

A. Sample Selection for Review of Staff Reports

The following selection process resulted in the 50-country sample of staff reports:

1. The sample universe includes 88 Article IV consultation staff reports discussed by the Board between July 1, 2007 (after the 2007 Decision began to be implemented) and

February 28, 2008.

2. The membership was stratified by income group (advanced, emerging, and developing) and by region (5 regions according to area department), and percentages of the membership within each strata calculated. The sample universe was similarly stratified, and a target number of countries was specified (based on a total of 50 staff reports) to match the percentage of the membership within each strata. The actual country sample was then randomly drawn from within each strata to match the target number as closely as possible, within the constraints of the available universe (Tables 1 and 2).

• Advanced: advanced economies according to the WEO classification.

• Emerging markets: Includes countries listed in Emerging Market Bond Index (EMBI) Global, or the Morgan Stanley Capital International (MSCI) EM Free or countries that have accessed syndicated loans every year between 2002-2006.¹³ Excludes countries classified as Advanced in the WEO (Cyprus, Slovenia).

• Developing: countries that fall into none of the above categories.¹⁴

3. The sample includes five program countries and three streamlined Article IV consultations.

Table 1.

List of 50 Countries for the Review of Staff Reports

Notes 1. Bolded countries are part of the MCM case study countries (see Appendix in FSS paper for details). 2. (*) Denotes program countries. 3. Italics denote members of currency unions or exchange arrangement with no separate legal tender. 4. (S) denotes streamlined consultations.

Table 1.

List of 50 Countries for the Review of Staff Reports

AFR			APD			EUR
ADV EMG	DEV/LIC	ADV	EMG	DEV/LIC	ADV	EMG	DEV/LIC
Angola	Botswana	Korea	India	Bhutan	Finland	Bulgaria	Montenegro
South Africa	Burkina Faso(*)		Vietnam	PNG	France	Hungary	Bosnia & H.
	CAR(*)			Cambodia	Germany	Russia
	Congo, DR			Lao PDR	Iceland	Serbia
	Guinea(*)			Myanmar	Ireland
	Lesotho				Israel
	Namibia
	Swaziland
	Zambia
	Guinea-Bissau
MCD			WHD
ADV EMG	DEV/LIC	ADV	EMG	DEV/LIC
Lebanon(*)	Afghanistan(*)	United States	Chile (S)	Bolivia
Morocco (S)	Sudan(*)		Colombia	Dominica
Egypt	Syria		Uruguay(*)	Barbados (S)
Pakistan	Yemen		Mexico	Haiti(*)
			Ecuador

Notes 1. Bolded countries are part of the MCM case study countries (see Appendix in FSS paper for details). 2. (*) Denotes program countries. 3. Italics denote members of currency unions or exchange arrangement with no separate legal tender. 4. (S) denotes streamlined consultations.

View Table

Table 1.

List of 50 Countries for the Review of Staff Reports

AFR			APD			EUR
ADV EMG	DEV/LIC	ADV	EMG	DEV/LIC	ADV	EMG	DEV/LIC
Angola	Botswana	Korea	India	Bhutan	Finland	Bulgaria	Montenegro
South Africa	Burkina Faso(*)		Vietnam	PNG	France	Hungary	Bosnia & H.
	CAR(*)			Cambodia	Germany	Russia
	Congo, DR			Lao PDR	Iceland	Serbia
	Guinea(*)			Myanmar	Ireland
	Lesotho				Israel
	Namibia
	Swaziland
	Zambia
	Guinea-Bissau
MCD			WHD
ADV EMG	DEV/LIC	ADV	EMG	DEV/LIC
Lebanon(*)	Afghanistan(*)	United States	Chile (S)	Bolivia
Morocco (S)	Sudan(*)		Colombia	Dominica
Egypt	Syria		Uruguay(*)	Barbados (S)
Pakistan	Yemen		Mexico	Haiti(*)
			Ecuador

Notes 1. Bolded countries are part of the MCM case study countries (see Appendix in FSS paper for details). 2. (*) Denotes program countries. 3. Italics denote members of currency unions or exchange arrangement with no separate legal tender. 4. (S) denotes streamlined consultations.

Table 2.

Sample and Membership Strata by Region and Income Level

Table 2.

Sample and Membership Strata by Region and Income Level

AFR			APD			EUR			MCD			WHD			Total
	ADV	EMG	DEV/ LIC	ADV	EMG	DEV/ LIC	ADV	EMG	DEV/ LIC	ADV	EMG	DEV/ LIC	ADV	EMG	DEV/ LIC
In percent
Membership (benchmark)	0	3	21	3	4	11	11	9	3	0	9	8	1	9	8	100
Sample of 50	0	4	20	2	4	10	12	8	4	0	8	8	2	10	8	100
In numbers
Membership (benchmark)	0	6	38	5	8	20	21	16	6	0	16	15	2	17	15	185
Sample of 50	0	2	10	1	2	5	6	4	2	0	4	4	1	5	4	50

View Table

Table 2.

Sample and Membership Strata by Region and Income Level

AFR			APD			EUR			MCD			WHD			Total
	ADV	EMG	DEV/ LIC	ADV	EMG	DEV/ LIC	ADV	EMG	DEV/ LIC	ADV	EMG	DEV/ LIC	ADV	EMG	DEV/ LIC
In percent
Membership (benchmark)	0	3	21	3	4	11	11	9	3	0	9	8	1	9	8	100
Sample of 50	0	4	20	2	4	10	12	8	4	0	8	8	2	10	8	100
In numbers
Membership (benchmark)	0	6	38	5	8	20	21	16	6	0	16	15	2	17	15	185
Sample of 50	0	2	10	1	2	5	6	4	2	0	4	4	1	5	4	50

B. Summary of Staff Report Reviews by Thematic Background

Table 3. Summary of Staff Report Review - FocusCircle

View Full Size

• Download Figure
• Download figure as PowerPoint slide

Table 4.

Summary of Staff Report Review - Health Check

Table 4.

Summary of Staff Report Review - Health Check

Questions	Total	ADV	EMG	DEV	AFR	APD	EUR	MCD	WHD	Program	Non-Program
	(in percentages of applicable sample)
Effectiveness of Surveillance
1. Does the staff report include a self-contained comprehensive
statement assessing the effectiveness of surveillance in the country?
Yes, please specify below how extensive	60	75	59	56	42	63	58	100	50	88	55
1-2 sentences.	4	0	6	4	0	13	0	13	0	0	5
One paragraph <= 10 lines (approximately 100-150 words.)	24	25	24	24	25	25	17	38	20	38	21
More extensive coverage: > 10 lines, or several paragraphs.	4	0	0	8	8	0	0	13	0	13	2
Half page box or table.	16	13	24	12	8	0	25	25	20	38	12
Full page box or table.	12	38	6	8	0	25	17	13	10	0	14
No	40	25	41	44	58	38	42	0	50	13	45
If no, please skip to question 5
2. What previous policy recommendations are followed up on?
(check if mentioned in SR being reviewed)
Fiscal policy changes.	67	67	50	79	60	80	100	75	0	57	70
Monetary policy changes.	40	33	30	50	40	60	43	38	20	57	35
Exchange rate policy changes (includes intervention.)	13	17	20	7	0	20	14	25	0	0	17
Structural reform in “core” areas (fiscal, monetary, financial sector
frameworks.)	67	83	60	64	60	80	71	50	80	57	70
Other structural reforms (e.g. labor markets, product markets, trade.)	43	83	50	21	60	60	71	13	20	14	52
Other, please specify.	33	0	40	43	20	20	29	38	40	29	35
3. What assessment of effectiveness of surveillance is provided?
Not applicable.
Advice mostly followed.	32	50	22	31	20	0	43	38	40	43	29
Advice partly followed.	57	50	67	54	40	100	43	50	40	43	62
Advice mostly not followed.	11	0	0	23	20	0	14	13	0	14	10
Other information (including e.g. whether advice coincided with
authorities’ views from the outset.)	7	0	22	0	20	0	0	0	20	0	10
4. What reasons were provided for not following IMF advice?
Not applicable.
Broadly agree with advice, but want to proceed at a slower pace (e.g.
because of political issues.)	40	67	43	30	0	60	75	20	25	0	53
Agree in general, but needed TA or other assistance to help implement,
so not yet implemented.	20	0	0	40	0	40	25	0	25	20	20
Disagree with advice.	20	67	14	10	0	20	25	20	25	0	27
No reason(s) provided.	35	0	14	60	50	40	0	40	25	40	33
Other information.	35	0	43	40	50	0	50	20	50	60	13
Recent Economic Developments and Policies
5. Is the coverage of recent economic developments and policies
sufficiently informative?
Yes	74	88	71	72	67	75	92	38	90	75	74
No, because:	18	0	18	24	33	13	8	38	0	13	19
There is no clear story line.	12	0	12	16	25	13	8	13	0	13	12
The pieces of information appeared inconsistent, and the story line did
not address these.	2	0	0	4	0	0	0	13	0	0	2
There was a lack of data/information in certain areas.	6	0	6	8	17	0	0	13	0	0	7
Other explanations	8	13	12	4	0	13	0	25	10	13	7
Short-Term Economic Outlook
6. Is the short-term economic outlook, including risks to this outlook,
presented in a substantiated way?
Yes	58	88	59	48	50	63	75	38	60	25	64
No, because:	30	13	29	36	33	25	25	38	30	63	24
There is no discussion of the risks to the outlook.	8	0	12	8	0	13	0	13	20	25	5
It is not clear why these risks were highlighted.	4	13	6	0	0	0	8	0	10	0	5
It is not clear how serious are the risks posed to the economy.	16	0	12	24	25	13	17	25	0	38	12
Other explanations , please specify	12	0	12	16	17	13	0	25	10	13	12
Assessment of External Stability and Coverage of Macro Policy
Areas
7. Is there a clear (implicit or explicit) assessment of present and
prospective external stability?
Not applicable (for members of currency unions)
Yes	78	80	82	75	82	75	78	75	80	86	77
No, because:	22	20	18	25	18	25	22	25	20	14	23
The discussion does not allow the reader to make a clear judgment on
the threats to external stability faced by the country.	13	20	18	8	18	0	22	13	10	0	15
The assessment misses some issues that may impact on external
stability.	7	0	6	8	0	13	11	0	10	14	5
Other explanations (including inadequate data or knowledge.)	7	0	0	13	9	13	0	13	0	0	8
8. Is there an adequate coverage of all core macroeconomic policy
areas (i.e., exchange rate, monetary and fiscal policies), including
identification of key policy challenges?
Yes	76	88	71	76	58	88	83	63	90	88	74
No, because:	24	13	29	24	42	13	17	38	10	13	26
One or more of the core areas are not adequately covered.	16	13	24	12	17	13	17	25	10	0	19
All core areas are adequately covered but not well integrated.	6	0	6	8	17	0	0	13	0	13	5
Other explanations (including inadequate data or knowledge.)	2	0	0	4	8	0	0	0	0	0	2
Fiscal Policies
9. Are recommended medium-term and short-term fiscal policy
objectives clearly articulated? Are the envisaged short-term fiscal
measures put in a clear medium-term fiscal road map?
Yes	76	88	71	76	58	75	92	63	90	63	79
No, because:	24	13	29	24	42	25	8	38	10	38	21
The discussion of medium-term fiscal objectives is inadequate.	10	0	24	4	8	25	0	13	10	0	12
The discussion of short-term fiscal objectives is inadequate.	8	0	12	8	17	13	0	13	0	25	5
The envisaged short-term fiscal measures are not put in a clear medium-
term fiscal road map.	6	13	0	8	8	0	8	13	0	25	2
Other explanations .	4	0	0	8	17	0	0	0	0	0	5
10. If fiscal adjustment is recommended, does the report provide
sufficient rationale for the magnitude, pace and composition of
proposed fiscal adjustment, along with a discussion of political
feasibility of such adjustment?
No adjustment is recommended
Yes	58	88	58	45	44	29	64	71	83	83	53
No, because:	33	13	33	40	44	57	27	14	17	17	35
There is no convincing justification of the recommended magnitude of
(med term) fiscal adjustment.	10	0	17	10	22	29	0	0	0	17	9
There is no convincing justification of the recommended pace of fiscal
adjustment.	15	13	8	20	22	43	9	0	0	17	15
There is no convincing justification of the recommended composition of
fiscal adjustment.	5	0	8	5	11	0	0	0	17	17	3
There is limited or no discussions of possible measures to achieve the
desired fiscal adjustment in the short term (current and next fiscal year.)	0	0	0	0	0	0	0	0	0	0	0
The discussion of political feasibility of proposed measures seems
inadequate.	15	0	8	25	22	14	18	14	0	17	15
Other explanations	10	0	8	15	11	14	9	14	0	0	12
Monetary Policy
11. Does the report provide an adequate assessment of the monetary policy stance?
Not applicable (currency unions, currency boards)
Yes	81	100	93	65	71	75	100	75	88	83	80
No, because:	19	0	7	35	29	25	0	25	13	17	20
The discussion of the monetary policy stance is insufficient.	17	0	7	29	29	13	0	25	13	17	17
The staffs assessment of the monetary policy stance is missing or
unclear.	3	0	0	6	0	13	0	0	0	0	3
Other explanations.	3	0	0	0	0	13	0	0	0	0	3
12. If monetary policy adjustment is recommended, does the staff provide a convincing justification for the proposed changes in the monetary policy stance, and possible downside risks?
No adjustment is recommended or not applicable
Yes	68	67	78	60	50	50	75	60	100	80	65
No, because:	5	0	0	10	25	0	0	0	0	20	0
There is no convincing justification of the recommended monetary
policy adjustment.	5	0	0	10	25	0	0	0	0	20	0
There is no discussion of possible downside risks (e.g., possible balance-
sheet effects, impact on the financial sector.)	0	0	0	0	0	0	0	0	0	0	0
Other explanations	27	33	22	30	25	50	25	40	0	0	35
Exchange rate questions revised and moved
Financial Sector Issues
13. If this country had an FSAP or FSAP update with this Article IV consultation, does the report provide an adequate coverage of its findings?
No FSAP or FSAP update in this period	88	100	82	88	83	100	83	88	90	88	88
Yes	10	0	12	12	17	0	8	13	10	13	10
No, because:	0	0	0	0	0	0	0	0	0	0	0
There is no discussion of the FSAP findings in the report.	0	0	0	0	0	0	0	0	0	0	0
FSAP findings are discussed, but not integrated with the analysis of the
macro policy areas (i.e., relevance not clear).	0	0	0	0	0	0	0	0	0	0	0
FSAP findings are discussed, but policy implications are not clearly
drawn.	0	0	0	0	0	0	0	0	0	0	0
Other explanations	2	0	6	0	0	0	8	0	0	0	2
Memo item: number of reports in the subsample	50	8	17	25	12	8	12	8	10	8	42

View Table

Table 4.

Summary of Staff Report Review - Health Check

Questions	Total	ADV	EMG	DEV	AFR	APD	EUR	MCD	WHD	Program	Non-Program
	(in percentages of applicable sample)
Effectiveness of Surveillance
1. Does the staff report include a self-contained comprehensive
statement assessing the effectiveness of surveillance in the country?
Yes, please specify below how extensive	60	75	59	56	42	63	58	100	50	88	55
1-2 sentences.	4	0	6	4	0	13	0	13	0	0	5
One paragraph <= 10 lines (approximately 100-150 words.)	24	25	24	24	25	25	17	38	20	38	21
More extensive coverage: > 10 lines, or several paragraphs.	4	0	0	8	8	0	0	13	0	13	2
Half page box or table.	16	13	24	12	8	0	25	25	20	38	12
Full page box or table.	12	38	6	8	0	25	17	13	10	0	14
No	40	25	41	44	58	38	42	0	50	13	45
If no, please skip to question 5
2. What previous policy recommendations are followed up on?
(check if mentioned in SR being reviewed)
Fiscal policy changes.	67	67	50	79	60	80	100	75	0	57	70
Monetary policy changes.	40	33	30	50	40	60	43	38	20	57	35
Exchange rate policy changes (includes intervention.)	13	17	20	7	0	20	14	25	0	0	17
Structural reform in “core” areas (fiscal, monetary, financial sector
frameworks.)	67	83	60	64	60	80	71	50	80	57	70
Other structural reforms (e.g. labor markets, product markets, trade.)	43	83	50	21	60	60	71	13	20	14	52
Other, please specify.	33	0	40	43	20	20	29	38	40	29	35
3. What assessment of effectiveness of surveillance is provided?
Not applicable.
Advice mostly followed.	32	50	22	31	20	0	43	38	40	43	29
Advice partly followed.	57	50	67	54	40	100	43	50	40	43	62
Advice mostly not followed.	11	0	0	23	20	0	14	13	0	14	10
Other information (including e.g. whether advice coincided with
authorities’ views from the outset.)	7	0	22	0	20	0	0	0	20	0	10
4. What reasons were provided for not following IMF advice?
Not applicable.
Broadly agree with advice, but want to proceed at a slower pace (e.g.
because of political issues.)	40	67	43	30	0	60	75	20	25	0	53
Agree in general, but needed TA or other assistance to help implement,
so not yet implemented.	20	0	0	40	0	40	25	0	25	20	20
Disagree with advice.	20	67	14	10	0	20	25	20	25	0	27
No reason(s) provided.	35	0	14	60	50	40	0	40	25	40	33
Other information.	35	0	43	40	50	0	50	20	50	60	13
Recent Economic Developments and Policies
5. Is the coverage of recent economic developments and policies
sufficiently informative?
Yes	74	88	71	72	67	75	92	38	90	75	74
No, because:	18	0	18	24	33	13	8	38	0	13	19
There is no clear story line.	12	0	12	16	25	13	8	13	0	13	12
The pieces of information appeared inconsistent, and the story line did
not address these.	2	0	0	4	0	0	0	13	0	0	2
There was a lack of data/information in certain areas.	6	0	6	8	17	0	0	13	0	0	7
Other explanations	8	13	12	4	0	13	0	25	10	13	7
Short-Term Economic Outlook
6. Is the short-term economic outlook, including risks to this outlook,
presented in a substantiated way?
Yes	58	88	59	48	50	63	75	38	60	25	64
No, because:	30	13	29	36	33	25	25	38	30	63	24
There is no discussion of the risks to the outlook.	8	0	12	8	0	13	0	13	20	25	5
It is not clear why these risks were highlighted.	4	13	6	0	0	0	8	0	10	0	5
It is not clear how serious are the risks posed to the economy.	16	0	12	24	25	13	17	25	0	38	12
Other explanations , please specify	12	0	12	16	17	13	0	25	10	13	12
Assessment of External Stability and Coverage of Macro Policy
Areas
7. Is there a clear (implicit or explicit) assessment of present and
prospective external stability?
Not applicable (for members of currency unions)
Yes	78	80	82	75	82	75	78	75	80	86	77
No, because:	22	20	18	25	18	25	22	25	20	14	23
The discussion does not allow the reader to make a clear judgment on
the threats to external stability faced by the country.	13	20	18	8	18	0	22	13	10	0	15
The assessment misses some issues that may impact on external
stability.	7	0	6	8	0	13	11	0	10	14	5
Other explanations (including inadequate data or knowledge.)	7	0	0	13	9	13	0	13	0	0	8
8. Is there an adequate coverage of all core macroeconomic policy
areas (i.e., exchange rate, monetary and fiscal policies), including
identification of key policy challenges?
Yes	76	88	71	76	58	88	83	63	90	88	74
No, because:	24	13	29	24	42	13	17	38	10	13	26
One or more of the core areas are not adequately covered.	16	13	24	12	17	13	17	25	10	0	19
All core areas are adequately covered but not well integrated.	6	0	6	8	17	0	0	13	0	13	5
Other explanations (including inadequate data or knowledge.)	2	0	0	4	8	0	0	0	0	0	2
Fiscal Policies
9. Are recommended medium-term and short-term fiscal policy
objectives clearly articulated? Are the envisaged short-term fiscal
measures put in a clear medium-term fiscal road map?
Yes	76	88	71	76	58	75	92	63	90	63	79
No, because:	24	13	29	24	42	25	8	38	10	38	21
The discussion of medium-term fiscal objectives is inadequate.	10	0	24	4	8	25	0	13	10	0	12
The discussion of short-term fiscal objectives is inadequate.	8	0	12	8	17	13	0	13	0	25	5
The envisaged short-term fiscal measures are not put in a clear medium-
term fiscal road map.	6	13	0	8	8	0	8	13	0	25	2
Other explanations .	4	0	0	8	17	0	0	0	0	0	5
10. If fiscal adjustment is recommended, does the report provide
sufficient rationale for the magnitude, pace and composition of
proposed fiscal adjustment, along with a discussion of political
feasibility of such adjustment?
No adjustment is recommended
Yes	58	88	58	45	44	29	64	71	83	83	53
No, because:	33	13	33	40	44	57	27	14	17	17	35
There is no convincing justification of the recommended magnitude of
(med term) fiscal adjustment.	10	0	17	10	22	29	0	0	0	17	9
There is no convincing justification of the recommended pace of fiscal
adjustment.	15	13	8	20	22	43	9	0	0	17	15
There is no convincing justification of the recommended composition of
fiscal adjustment.	5	0	8	5	11	0	0	0	17	17	3
There is limited or no discussions of possible measures to achieve the
desired fiscal adjustment in the short term (current and next fiscal year.)	0	0	0	0	0	0	0	0	0	0	0
The discussion of political feasibility of proposed measures seems
inadequate.	15	0	8	25	22	14	18	14	0	17	15
Other explanations	10	0	8	15	11	14	9	14	0	0	12
Monetary Policy
11. Does the report provide an adequate assessment of the monetary policy stance?
Not applicable (currency unions, currency boards)
Yes	81	100	93	65	71	75	100	75	88	83	80
No, because:	19	0	7	35	29	25	0	25	13	17	20
The discussion of the monetary policy stance is insufficient.	17	0	7	29	29	13	0	25	13	17	17
The staffs assessment of the monetary policy stance is missing or
unclear.	3	0	0	6	0	13	0	0	0	0	3
Other explanations.	3	0	0	0	0	13	0	0	0	0	3
12. If monetary policy adjustment is recommended, does the staff provide a convincing justification for the proposed changes in the monetary policy stance, and possible downside risks?
No adjustment is recommended or not applicable
Yes	68	67	78	60	50	50	75	60	100	80	65
No, because:	5	0	0	10	25	0	0	0	0	20	0
There is no convincing justification of the recommended monetary
policy adjustment.	5	0	0	10	25	0	0	0	0	20	0
There is no discussion of possible downside risks (e.g., possible balance-
sheet effects, impact on the financial sector.)	0	0	0	0	0	0	0	0	0	0	0
Other explanations	27	33	22	30	25	50	25	40	0	0	35
Exchange rate questions revised and moved
Financial Sector Issues
13. If this country had an FSAP or FSAP update with this Article IV consultation, does the report provide an adequate coverage of its findings?
No FSAP or FSAP update in this period	88	100	82	88	83	100	83	88	90	88	88
Yes	10	0	12	12	17	0	8	13	10	13	10
No, because:	0	0	0	0	0	0	0	0	0	0	0
There is no discussion of the FSAP findings in the report.	0	0	0	0	0	0	0	0	0	0	0
FSAP findings are discussed, but not integrated with the analysis of the
macro policy areas (i.e., relevance not clear).	0	0	0	0	0	0	0	0	0	0	0
FSAP findings are discussed, but policy implications are not clearly
drawn.	0	0	0	0	0	0	0	0	0	0	0
Other explanations	2	0	6	0	0	0	8	0	0	0	2
Memo item: number of reports in the subsample	50	8	17	25	12	8	12	8	10	8	42

Table 5.

Summary of Staff Report Review - Financial Sector Surveillance

^1/four countries — Angola, Afghanistan, CAR and Congo — for which questions Q3 and Q4 were considered NOT applicable by reviewers are excluded.

Table 5.

Summary of Staff Report Review - Financial Sector Surveillance

Questions	Total	ADV	EMG	DEV	AFR	APD	EUR	MCD	WHD	Program	Non-Program
				(in percentages of applicable sample)
(1) Does the report provide a clear assessment of FS vulnerabilities? And if so, is it clear from the report what is the level of FS vulnerabilities?
- Yes, FS vulnerabilities are MEDIUM or HIGH (M/H).	26	25	29	24	8	25	25	50	30	50	21
- Yes, FS vulnerabilities are LOW.	32	50	29	28	33	38	33	13	40	0	38
NO, because	42	25	41	48	58	38	42	38	30	50	40
- there is no assessment of FS vulnerabilities and its absence is not justified;	12	0	6	20	33	0	0	13	10	38	7
- insufficient quantitative analysis (none/few of the tools listed in (2) are used);	12	25	12	8	0	0	25	25	10	0	14
- the assessment is too hedged (there is no clear bottom line);	10	0	12	12	8	13	17	13	0	0	12
- insufficient qualitative analysis (none/few of areas listed in (2) are covered);	8	25	6	4	0	0	17	25	0	0	10
- there is no assessment of FS vulnerabilities, because FS is too small to be macro relevant;	4	0	6	4	17	0	0	0	0	13	2
- analysis of FS vulnerabilities is incomplete because relevant data/information is not
available;	4	0	6	4	13	0	0	10	0	5
- other	4	0	6	4	8	13	0	0	0	13	2
(2) What tools/information are used to assess FS vulnerabilities? - indicate which apply
Quantitative analysis:
- financial soundness indicators (FSIs);	80	100	82	72	58	75	100	63	100	50	86
- market indicators;	34	63	53	12	0	50	33	25	70	25	36
- balance-sheet analysis (BSA or CCA);	14	25	29	0	0	0	17	13	40	25	12
- stress tests;	16	38	24	4	8	25	33	0	10	13	17
- other	6	13	12	0	0	0	8	0	20	13	5
Qualitative analysis:
- analysis of weaknesses in the FS structure or infrastructure (e.g., legal framework,
governance, payments);	22	38	29	12	8	25	17	25	40	25	21
- analysis of weaknesses in FS supervision and regulation;	54	63	47	56	25	50	67	50	80	38	57
- other	0	0	0	0	0	0	0	0	0	0	0
Information sources/references:
- references to FSAP findings;	22	13	29	20	17	0	25	13	50	13	24
- references to SIP and/or previous staff analysis;	42	63	41	36	33	75	50	13	40	25	45
- discussions with market participants and/or credit rating agencies;	4	25	0	0	0	0	8	0	10	0	5
- other	2	13	0	0	0	0	0	0	10	0	2
(3) Does the report provide a clear analysis of the risk factors that could trigger FS instability, including the analysis of risk transmission channels? 1/
Yes	26	50	31	14	22	13	25	14	50	33	25
NO, because	74	50	69	86	78	88	75	86	50	67	75
- the key risks to the FS stability are not clearly identified;	48	13	44	64	56	38	33	86	40	67	45
- insufficient analysis of risk transmission channels between the domestic NFS and FS;	28	25	19	36	33	38	25	29	20	17	30
- insufficient analysis of risk transmission channels between the external sector and FS;	26	25	31	23	22	25	33	57	0	33	25
- insufficient analysis of risk transmission from domestic policies to the FS;	22	25	19	27	22	13	42	43	0	33	23
- available data/information is NOT sufficient for accurate diagnostics;	6	0	0	14	11	13	0	0	10	17	5
(4) Does the discussion of FS issues permit conclusions as to whether the FS is a potential source (or an amplifier) of macroeconomic or external instability? 1/
-Yes, FS instability would have significant macroeconomic or external stability implications.	20	38	13	18	11	25	17	14	30	33	18
-Yes, FS instability would have little/no macroeconomic or external stability implications.	7	13	0	9	11	13	0	14	0	17	5
-Yes, FS vulnerabilities are LOW and FS is not expected to amplify problems originating in
other sectors.	2	0	6	0	0	13	0	0	0	0	3
				(in percentages of applicable sample)
NO, because	72	50	81	73	78	50	83	71	70	50	75
- insufficient analysis of risk transmission from the FS to the rest of the economy;	52	38	56	55	67	25	58	43	60	33	55
- insufficient analysis of the impact of the FS on the conduct of monetary, exchange rate, or
fiscal policy;	30	25	31	32	33	13	33	57	20	17	33
- macroeconomic stability implications are NOT clearly drawn;	30	38	38	23	11	13	50	14	50	33	30
- insufficient analysis of the role of the FS in amplifying or muting problems originating in
other sectors;	20	38	19	14	22	0	33	0	30	17	20
- external stability implications are NOT clearly drawn;	15	38	19	5	11	0	33	14	10	17	15
- available data/information is NOT sufficient for accurate diagnostics;	13	0	6	23	22	25	0	14	10	33	10
(5) Does the report identify concerns with regard to FS development? And if so, how are they motivated?
Yes	46	38	41	52	58	75	33	38	30	63	43
- Yes, FS underdevelopment is an impediment for growth	26	0	12	44	50	25	0	38	20	50	21
- Yes, FS underdevelopment poses macroeconomic or external stability problems	14	38	18	4	8	13	33	0	10	13	14
- Yes, both of the above	6	0	12	4						0	7
NO	54	63	59	48	42	25	67	63	70	38	57
- the FS development problems are not clearly described;	24	0	29	28	25	13	17	25	40	25	24
- the FS development issues are not discussed or are not relevant	20	13	18	12	8	25	25	25	20	13	21
- the report does not clearly explain how FS development problems might affect future
growth and/or macroeconomic stability;	14	13	24	8	17	0	17	25	10	0	17
- specific FS development problems are mentioned, but their causes and consequences are
not discussed;	10	13	18	4	0	0	8	38	10	0	12
(6) Does the report provide clearly-justified FS policy recommendations?
Yes	60	63	53	64	42	100	58	38	70	38	64
- Yes, the report provides recommendations on how to address FS vulnerabilities.	30	50	24	28	25	38	33	13	40	13	33
- Yes, the report provides recommendations on how to address concerns with regard to FS
development.	10	13	12	8	8	13	8	0	20	0	12
- Yes, both of the above	20	0	18	28	8	50	17	25	10	25	19
NO	40	38	53	64	58	0	42	63	30	63	36
- NO recommendations are made.	16	13	24	28	50	0	17	0	0	38	12
- the report makes general statements about the importance of “prudential supervision”,
“vigilance”, etc. or statements of general support of the authorities’ policies and policy plans.	14	0	12	8	0	0	17	38	20	25	12
- recommendations are made, but not clearly justified.	10	25	18	28	0	0	17	25	10	0	12
- the report simply re-states the FSAP recommendations.	4	0	47	36	0	0	0	13	10	0	5
Memo item: number of reports in the subsample	50	8	17	25	12	8	12	8	10	8	42
1/ four countries - Angola, Afghanistan, CAR and Congo - for which questions Q3 and Q4 were	considered NOT applicable by reviewers are						excluded.

^1/four countries — Angola, Afghanistan, CAR and Congo — for which questions Q3 and Q4 were considered NOT applicable by reviewers are excluded.

View Table

Table 5.

Summary of Staff Report Review - Financial Sector Surveillance

Questions	Total	ADV	EMG	DEV	AFR	APD	EUR	MCD	WHD	Program	Non-Program
				(in percentages of applicable sample)
(1) Does the report provide a clear assessment of FS vulnerabilities? And if so, is it clear from the report what is the level of FS vulnerabilities?
- Yes, FS vulnerabilities are MEDIUM or HIGH (M/H).	26	25	29	24	8	25	25	50	30	50	21
- Yes, FS vulnerabilities are LOW.	32	50	29	28	33	38	33	13	40	0	38
NO, because	42	25	41	48	58	38	42	38	30	50	40
- there is no assessment of FS vulnerabilities and its absence is not justified;	12	0	6	20	33	0	0	13	10	38	7
- insufficient quantitative analysis (none/few of the tools listed in (2) are used);	12	25	12	8	0	0	25	25	10	0	14
- the assessment is too hedged (there is no clear bottom line);	10	0	12	12	8	13	17	13	0	0	12
- insufficient qualitative analysis (none/few of areas listed in (2) are covered);	8	25	6	4	0	0	17	25	0	0	10
- there is no assessment of FS vulnerabilities, because FS is too small to be macro relevant;	4	0	6	4	17	0	0	0	0	13	2
- analysis of FS vulnerabilities is incomplete because relevant data/information is not
available;	4	0	6	4	13	0	0	10	0	5
- other	4	0	6	4	8	13	0	0	0	13	2
(2) What tools/information are used to assess FS vulnerabilities? - indicate which apply
Quantitative analysis:
- financial soundness indicators (FSIs);	80	100	82	72	58	75	100	63	100	50	86
- market indicators;	34	63	53	12	0	50	33	25	70	25	36
- balance-sheet analysis (BSA or CCA);	14	25	29	0	0	0	17	13	40	25	12
- stress tests;	16	38	24	4	8	25	33	0	10	13	17
- other	6	13	12	0	0	0	8	0	20	13	5
Qualitative analysis:
- analysis of weaknesses in the FS structure or infrastructure (e.g., legal framework,
governance, payments);	22	38	29	12	8	25	17	25	40	25	21
- analysis of weaknesses in FS supervision and regulation;	54	63	47	56	25	50	67	50	80	38	57
- other	0	0	0	0	0	0	0	0	0	0	0
Information sources/references:
- references to FSAP findings;	22	13	29	20	17	0	25	13	50	13	24
- references to SIP and/or previous staff analysis;	42	63	41	36	33	75	50	13	40	25	45
- discussions with market participants and/or credit rating agencies;	4	25	0	0	0	0	8	0	10	0	5
- other	2	13	0	0	0	0	0	0	10	0	2
(3) Does the report provide a clear analysis of the risk factors that could trigger FS instability, including the analysis of risk transmission channels? 1/
Yes	26	50	31	14	22	13	25	14	50	33	25
NO, because	74	50	69	86	78	88	75	86	50	67	75
- the key risks to the FS stability are not clearly identified;	48	13	44	64	56	38	33	86	40	67	45
- insufficient analysis of risk transmission channels between the domestic NFS and FS;	28	25	19	36	33	38	25	29	20	17	30
- insufficient analysis of risk transmission channels between the external sector and FS;	26	25	31	23	22	25	33	57	0	33	25
- insufficient analysis of risk transmission from domestic policies to the FS;	22	25	19	27	22	13	42	43	0	33	23
- available data/information is NOT sufficient for accurate diagnostics;	6	0	0	14	11	13	0	0	10	17	5
(4) Does the discussion of FS issues permit conclusions as to whether the FS is a potential source (or an amplifier) of macroeconomic or external instability? 1/
-Yes, FS instability would have significant macroeconomic or external stability implications.	20	38	13	18	11	25	17	14	30	33	18
-Yes, FS instability would have little/no macroeconomic or external stability implications.	7	13	0	9	11	13	0	14	0	17	5
-Yes, FS vulnerabilities are LOW and FS is not expected to amplify problems originating in
other sectors.	2	0	6	0	0	13	0	0	0	0	3
				(in percentages of applicable sample)
NO, because	72	50	81	73	78	50	83	71	70	50	75
- insufficient analysis of risk transmission from the FS to the rest of the economy;	52	38	56	55	67	25	58	43	60	33	55
- insufficient analysis of the impact of the FS on the conduct of monetary, exchange rate, or
fiscal policy;	30	25	31	32	33	13	33	57	20	17	33
- macroeconomic stability implications are NOT clearly drawn;	30	38	38	23	11	13	50	14	50	33	30
- insufficient analysis of the role of the FS in amplifying or muting problems originating in
other sectors;	20	38	19	14	22	0	33	0	30	17	20
- external stability implications are NOT clearly drawn;	15	38	19	5	11	0	33	14	10	17	15
- available data/information is NOT sufficient for accurate diagnostics;	13	0	6	23	22	25	0	14	10	33	10
(5) Does the report identify concerns with regard to FS development? And if so, how are they motivated?
Yes	46	38	41	52	58	75	33	38	30	63	43
- Yes, FS underdevelopment is an impediment for growth	26	0	12	44	50	25	0	38	20	50	21
- Yes, FS underdevelopment poses macroeconomic or external stability problems	14	38	18	4	8	13	33	0	10	13	14
- Yes, both of the above	6	0	12	4						0	7
NO	54	63	59	48	42	25	67	63	70	38	57
- the FS development problems are not clearly described;	24	0	29	28	25	13	17	25	40	25	24
- the FS development issues are not discussed or are not relevant	20	13	18	12	8	25	25	25	20	13	21
- the report does not clearly explain how FS development problems might affect future
growth and/or macroeconomic stability;	14	13	24	8	17	0	17	25	10	0	17
- specific FS development problems are mentioned, but their causes and consequences are
not discussed;	10	13	18	4	0	0	8	38	10	0	12
(6) Does the report provide clearly-justified FS policy recommendations?
Yes	60	63	53	64	42	100	58	38	70	38	64
- Yes, the report provides recommendations on how to address FS vulnerabilities.	30	50	24	28	25	38	33	13	40	13	33
- Yes, the report provides recommendations on how to address concerns with regard to FS
development.	10	13	12	8	8	13	8	0	20	0	12
- Yes, both of the above	20	0	18	28	8	50	17	25	10	25	19
NO	40	38	53	64	58	0	42	63	30	63	36
- NO recommendations are made.	16	13	24	28	50	0	17	0	0	38	12
- the report makes general statements about the importance of “prudential supervision”,
“vigilance”, etc. or statements of general support of the authorities’ policies and policy plans.	14	0	12	8	0	0	17	38	20	25	12
- recommendations are made, but not clearly justified.	10	25	18	28	0	0	17	25	10	0	12
- the report simply re-states the FSAP recommendations.	4	0	47	36	0	0	0	13	10	0	5
Memo item: number of reports in the subsample	50	8	17	25	12	8	12	8	10	8	42
1/ four countries - Angola, Afghanistan, CAR and Congo - for which questions Q3 and Q4 were	considered NOT applicable by reviewers are						excluded.

^1/four countries — Angola, Afghanistan, CAR and Congo — for which questions Q3 and Q4 were considered NOT applicable by reviewers are excluded.

Table 6.

Summary of Staff Report Review - Exchange Rate

^1/“Fixed” defined as arrangements with no separate legal tender, currency boards, other conventional pegs and pegs within horizontal bands (Annual Report on Exchange Arrangements and Exchange Restrictions (AREAER), 2007).

^2/“Flexible” defined as crawling pegs (Botswana), managed floats and independent floats (Annual Report on Exchange Arrangements and Exchange Restrictions (AREAER), 2007).

^3/The sample contains 17 cases with CA deficit over 5 percent of GDP and 9 cases with a CA surplus over 5 percent of GDP.

Table 6.

Summary of Staff Report Review - Exchange Rate

Question	Not Applicable	Total	ADV	EMG	DEV	AFR	APD	EUR	MCD	WHD	Fixed ER reg.1/	Flex. ER reg. 2/	Issued before 09/01/07	Issued after 09/01/07	CA<5 or CA-5 3/	3CA > 5 and CA<-5	program	Non-Program
	(in number)			(in percentages of applicable sample)
Q1. If staff report provides an explicit label for the de facto regime, please state what it is:
1.1 No separate legal tender		4	0	0	6	4	0	8	0	10			0	0	4	4	0	5
1.2 Currency board		6	0	6	8	0	0	17	0	10			8	10	12	0	13	5
1.3 Other conventional peg arrangements		32	0	24	48	58	25	0	63	20			32	5	38	25	38	31
1.4 Peg within horizontal band		4	0	12	0	0	13	8	0	0			8	38	0	8	0	5
1.5 Crawling peg		4	0	0	8	8	0	0	13	0			4	0	4	4	0	5
1.6 Crawling band		0	0	0	0	0	0	0	0	0			0	5	0	0	0	0
1.7 Managed float with no predetermined path for the exchange rate		28	13	35	28	17	63	17	25	30			20	0	19	38	25	29
1.8 Independently floating		22	88	18	4	17	0	50	0	30			28	43	23	21	25	21
Q2. Does the report provide a clear description of “de facto” exchange rate regime and exchange rate policies?
Not applicable because the regime and policies are well-known and therefore a	5
description is redundant (e.g., the US, UK, Euro Area members).
Yes		96	100	94	96	100	100	88	100	89	96	95	91	100	100	91	100	95
No, because:		4	064	0	0	13	0	11	4	5	9	0	09	0	5
There is too limited a discussion, or no discussion, of the “de facto” exchange
rate regime.		0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
(For regimes other than hard pegs or pure floats) There is too limited a
discussion, or no discussion, of exchange rate policies.		4	064	0	0	13	0	11	4	5	9	0	09	0	5
Other explanations (specify).		0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Q3. Does the report provide a reasoned appraisal of the “de facto” exchange rate regime?
Not applicable	9
Yes		71	67	94	52	56	88	50	75	88	68	73	85	57	69	76	75	70
No, because:		29	33	64	8	44	13	50	25	13	32	27	15	43	31	24	25	30
The report does not discuss the appropriateness of the regime at all.		10	0	6	14	11	0	25	13	0	21	0	10	10	1	55	0	11
The report merely mentions that “the regime has served the country well” or
similar wording.		12	33	0	19	22	13	13	13	0	5	18	0	24	8	14	25	11
Other explanations (specify).		7	0	0	14	11	0	13	0	13	5	9	5	10	8	5	0	8
Q4. For countries with discretionary exchange rate policies, does the report provide a reasoned appraisal of exchange rate policies?
Not applicable	26
Yes		88	100	91	85	100	100	67	60	100	60	95	86	91	100	79	75	90
No, because		12	0 9 15	0	0	33	40	0	40	5	14	9	0 21	25	10
The report does not describe exchange rate policies clearly.		4	090	0	0	33	0	0	20	0	7	0	07	0	5
The report describes exchange rate policies but does not give a view on their
appropriateness.		0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
The depth of appraisal is shallow.		8	0	0	15	0	0	0	40	0	20	5	7	9	0	14	25	5
Q5. Does the report provide a clear assessment of the consistency of the level of the exchange rate with its fundamental determinants?
Yes	92	88	94	92	92	100	92	88	90	87	96	88	96	88	96	100	90
No, because:	8	13	6	8	8	0	8	13	10	13	4	12	4	12	4	0	10
Exchange rate level/competitiveness is not explicitly discussed.	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Exchange rate level/competitiveness is discussed but there is no clear bottom line.	6	13	0	8	8	0	8	0	10	9	4	8	4	12	0	0	7
Other explanations (specify).	2	0	6	0	0	0	0	13	0	4	0	4	0	0	4	0	2
Q6. Please specify all techniques utilized for making an assessment
Basic indicators (REER, market shares, export growth...)	82	63	94	80	83	75	75	100	80	87	78	84	80	81	83	88	81
Discussion of CA sustainability or DSA	42	50	47	36	50	63	25	50	30	48	37	32	52	35	50	38	43
PPP approaches	12	0	18	12	25	13	8	0	10	17	7	8	16	8	17	0	14
Quoting an estimate from CGER	26	75	41	0	8	25	42	13	40	4	44	28	24	19	33	25	26
Variants of CGER external sustainability approach	30	63	29	20	42	0	67	25	0	30	30	16	44	38	21	25	31
Econometric methods:	50	63	59	40	42	25	67	50	60	57	44	36	64	46	54	38	52
Variants of CGER macroeconomic balance approach	32	63	47	12	17	25	67	38	10	35	30	16	48	31	33	25	33
Variants of CGER equilibrium real exchange rate approach	40	63	41	32	33	13	50	38	60	39	41	36	44	38	42	25	43
Other approaches (specify):	32	38	24	36	50	38	33	13	20	26	37	24	40	38	25	67	64
Q7. Does the report provide robust analysis to underpin the assessment of the exchange rate level?
Yes	56	75	59	48	50	38	75	38	70	52	59	44	68	65	46	50	57
No, because:	44	25	41	52	50	63	25	63	30	48	41	56	32	35	54	50	43
(i) Assessment in done in a superficial way without any analysis.	6	0	0	12	8	25	0	0	0	0	11	12	0	8	4	0	7
(ii) Sophistication/depth of analysis is inadequate given available data.	16	0	18	20	8	38	8	25	10	22	11	28	4	19	13	25	14
(iii) The report does not disclose critical features of methodologies and/or key
assumptions, or presents analysis in a “black box” way.	22	13	35	16	25	13	25	25	20	22	22	20	24	8	38	25	19
(iv) Analysis/conclusions appears to be either not taking into account all important
factors or internally inconsistent (at odds with other facts presented in the report).	6	13	0	8	8	0	0	13	10	9	4	8	4	4	8	0	10
Memo item: number of reports in the subsample	50	8	17	25	12	8	12	8	10	23	27	25	25	26	24	8	42

^1/“Fixed” defined as arrangements with no separate legal tender, currency boards, other conventional pegs and pegs within horizontal bands (Annual Report on Exchange Arrangements and Exchange Restrictions (AREAER), 2007).

^2/“Flexible” defined as crawling pegs (Botswana), managed floats and independent floats (Annual Report on Exchange Arrangements and Exchange Restrictions (AREAER), 2007).

^3/The sample contains 17 cases with CA deficit over 5 percent of GDP and 9 cases with a CA surplus over 5 percent of GDP.

View Table

Table 6.

Summary of Staff Report Review - Exchange Rate

Question	Not Applicable	Total	ADV	EMG	DEV	AFR	APD	EUR	MCD	WHD	Fixed ER reg.1/	Flex. ER reg. 2/	Issued before 09/01/07	Issued after 09/01/07	CA<5 or CA-5 3/	3CA > 5 and CA<-5	program	Non-Program
	(in number)			(in percentages of applicable sample)
Q1. If staff report provides an explicit label for the de facto regime, please state what it is:
1.1 No separate legal tender		4	0	0	6	4	0	8	0	10			0	0	4	4	0	5
1.2 Currency board		6	0	6	8	0	0	17	0	10			8	10	12	0	13	5
1.3 Other conventional peg arrangements		32	0	24	48	58	25	0	63	20			32	5	38	25	38	31
1.4 Peg within horizontal band		4	0	12	0	0	13	8	0	0			8	38	0	8	0	5
1.5 Crawling peg		4	0	0	8	8	0	0	13	0			4	0	4	4	0	5
1.6 Crawling band		0	0	0	0	0	0	0	0	0			0	5	0	0	0	0
1.7 Managed float with no predetermined path for the exchange rate		28	13	35	28	17	63	17	25	30			20	0	19	38	25	29
1.8 Independently floating		22	88	18	4	17	0	50	0	30			28	43	23	21	25	21
Q2. Does the report provide a clear description of “de facto” exchange rate regime and exchange rate policies?
Not applicable because the regime and policies are well-known and therefore a	5
description is redundant (e.g., the US, UK, Euro Area members).
Yes		96	100	94	96	100	100	88	100	89	96	95	91	100	100	91	100	95
No, because:		4	064	0	0	13	0	11	4	5	9	0	09	0	5
There is too limited a discussion, or no discussion, of the “de facto” exchange
rate regime.		0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
(For regimes other than hard pegs or pure floats) There is too limited a
discussion, or no discussion, of exchange rate policies.		4	064	0	0	13	0	11	4	5	9	0	09	0	5
Other explanations (specify).		0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Q3. Does the report provide a reasoned appraisal of the “de facto” exchange rate regime?
Not applicable	9
Yes		71	67	94	52	56	88	50	75	88	68	73	85	57	69	76	75	70
No, because:		29	33	64	8	44	13	50	25	13	32	27	15	43	31	24	25	30
The report does not discuss the appropriateness of the regime at all.		10	0	6	14	11	0	25	13	0	21	0	10	10	1	55	0	11
The report merely mentions that “the regime has served the country well” or
similar wording.		12	33	0	19	22	13	13	13	0	5	18	0	24	8	14	25	11
Other explanations (specify).		7	0	0	14	11	0	13	0	13	5	9	5	10	8	5	0	8
Q4. For countries with discretionary exchange rate policies, does the report provide a reasoned appraisal of exchange rate policies?
Not applicable	26
Yes		88	100	91	85	100	100	67	60	100	60	95	86	91	100	79	75	90
No, because		12	0 9 15	0	0	33	40	0	40	5	14	9	0 21	25	10
The report does not describe exchange rate policies clearly.		4	090	0	0	33	0	0	20	0	7	0	07	0	5
The report describes exchange rate policies but does not give a view on their
appropriateness.		0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
The depth of appraisal is shallow.		8	0	0	15	0	0	0	40	0	20	5	7	9	0	14	25	5
Q5. Does the report provide a clear assessment of the consistency of the level of the exchange rate with its fundamental determinants?
Yes	92	88	94	92	92	100	92	88	90	87	96	88	96	88	96	100	90
No, because:	8	13	6	8	8	0	8	13	10	13	4	12	4	12	4	0	10
Exchange rate level/competitiveness is not explicitly discussed.	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Exchange rate level/competitiveness is discussed but there is no clear bottom line.	6	13	0	8	8	0	8	0	10	9	4	8	4	12	0	0	7
Other explanations (specify).	2	0	6	0	0	0	0	13	0	4	0	4	0	0	4	0	2
Q6. Please specify all techniques utilized for making an assessment
Basic indicators (REER, market shares, export growth...)	82	63	94	80	83	75	75	100	80	87	78	84	80	81	83	88	81
Discussion of CA sustainability or DSA	42	50	47	36	50	63	25	50	30	48	37	32	52	35	50	38	43
PPP approaches	12	0	18	12	25	13	8	0	10	17	7	8	16	8	17	0	14
Quoting an estimate from CGER	26	75	41	0	8	25	42	13	40	4	44	28	24	19	33	25	26
Variants of CGER external sustainability approach	30	63	29	20	42	0	67	25	0	30	30	16	44	38	21	25	31
Econometric methods:	50	63	59	40	42	25	67	50	60	57	44	36	64	46	54	38	52
Variants of CGER macroeconomic balance approach	32	63	47	12	17	25	67	38	10	35	30	16	48	31	33	25	33
Variants of CGER equilibrium real exchange rate approach	40	63	41	32	33	13	50	38	60	39	41	36	44	38	42	25	43
Other approaches (specify):	32	38	24	36	50	38	33	13	20	26	37	24	40	38	25	67	64
Q7. Does the report provide robust analysis to underpin the assessment of the exchange rate level?
Yes	56	75	59	48	50	38	75	38	70	52	59	44	68	65	46	50	57
No, because:	44	25	41	52	50	63	25	63	30	48	41	56	32	35	54	50	43
(i) Assessment in done in a superficial way without any analysis.	6	0	0	12	8	25	0	0	0	0	11	12	0	8	4	0	7
(ii) Sophistication/depth of analysis is inadequate given available data.	16	0	18	20	8	38	8	25	10	22	11	28	4	19	13	25	14
(iii) The report does not disclose critical features of methodologies and/or key
assumptions, or presents analysis in a “black box” way.	22	13	35	16	25	13	25	25	20	22	22	20	24	8	38	25	19
(iv) Analysis/conclusions appears to be either not taking into account all important
factors or internally inconsistent (at odds with other facts presented in the report).	6	13	0	8	8	0	0	13	10	9	4	8	4	4	8	0	10
Memo item: number of reports in the subsample	50	8	17	25	12	8	12	8	10	23	27	25	25	26	24	8	42

^1/“Fixed” defined as arrangements with no separate legal tender, currency boards, other conventional pegs and pegs within horizontal bands (Annual Report on Exchange Arrangements and Exchange Restrictions (AREAER), 2007).

^2/“Flexible” defined as crawling pegs (Botswana), managed floats and independent floats (Annual Report on Exchange Arrangements and Exchange Restrictions (AREAER), 2007).

^3/The sample contains 17 cases with CA deficit over 5 percent of GDP and 9 cases with a CA surplus over 5 percent of GDP.

Table 7.

Summary of Staff Report Review - Spillovers and Cross Country Analysis

^1/The results are in percentages of all answers given to the question which may be less or more than the 50 sample size.

Table 7.

Summary of Staff Report Review - Spillovers and Cross Country Analysis

Questions	Total			ADV		EMG		DEV		AFR		APD		EUR		MCD		WHD		Program		Non-Program
							(in percentages of applicable sample)
1. Outward side: is there a discussion of the impact of this country’s outlook/ policies on other economies?
Yes	10			25		12		4	17		0		8		0		20		13		10
Passing reference (acknowledging effects and risks in general terms)	4			064					8		0		0		0		10		13		2
More than passing reference, with some analysis	4			13		6		0	80800										05
Extensive substantive analysis (e.g. possible scenarios and transmission
mechanisms to show size of problem and why it needs attention)	2			13		0		0	0		0		0		0		10		0		2
Other, please specify	0			0		0		0		0		0		0		0		0		0
No discussion	90			75		88		96	83		100		92		100		80		88		90
2. Inward: Where relevant, does the staff report refer to the following trends in its analysis?
Backward-looking trends (NA, Y, N)	NA	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N
(i) commodity price movements	(8)	63	37	57	43	63	38	65	35	75	25	71	29	36	64	86	14	56	44	57	43	6436
(ii) overall global economic environment	(18)	29	71	63	38	20	80	22 78	33	67	14	86	33	67	0	100	50	50	17	83	31 69
(iii) overall global financial environment	(38)	52	48	88	13	47	53	17 83	40	60	0	100	67	33	75	25	43	57	50	50	52	48
(iv) regional trends/developments in neighboring economies	(20)	50	50	83	17	38	62	48	52	44	56	50	50	64	36	33	67	50	50	43	57	52	48
(v) other (please specify)	(67)	14	86	50	50	0	100	0	100	0	100	0	100	33	67	0	0	0	0	0	100	17	83
Forward-looking trends (NA, Y, N)	NA	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N
(i) commodity price movements	(8)	61	39	63	38	67	33	5743	67	33	71	29	50	50	33	67	78	22	43	57	6436
(ii) overall global economic environment	(10)	38	62	75	25	31	69	29	71	40	60	43	57	42	58	14	86	44	56	17	83	41	59
(iii) overall global financial environment	(36)	56	44	88	13	65	35	0	100	20	80	33	67	75	25	60	40	57	43	50	50	57	43
(iv) regional trends/developments in neighboring economies	(18)	41	59	67	33	36	64	38	62	33	67	38	63	27	73	33	67	86	14	29	71	44	56
(v) other (please specify)	(67)	14	86	0	100	0	100	20	80	0	100	0	100	0	100	0	0	100	0	50	50	0	100
3. Inward: Does the staff report refer to relevant risks identified in the last WEO/GFSR in its analysis? (NA, Y, N)	NA	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N
(i) commodity price movements	(20)	55	45	71	29	60	40	44	56	50	50	86	14	45	55	20	80	67	33	0	100	61	39
(ii) overall global economic environment	(22)	38	62	86	14	38	63	19	81	33	67	43	57	45	55	0	100	50	50	0	100	43	57
(iii) overall global financial environment	(30)	49	51	100	0	60	40	8	92	25	75	17	83	67	33	67	33	67	33	0	100	55	45
(iv) regional trends/developments in neighboring economies	(36)	25	75	40	60	27	73	19	81	25	75	29	71	20	80	0	100	40	60	0	100	29	71
(v) other (please specify)	(59)	14	86	0	0	0	100	20	80	0	100	0	100	0	100	0	0	100	0	0	100	17	83
4. If there are references to WEO/GFSR-identified risks, is coverage adequate (in quantity and substance) compared to what would seem warranted? 1/
Yes	61			75		62		50	71		17		70		50		86		0		63
No, because	39			25		38		50	29		83		30		50		14		100		38
Insufficient /generic/more shallow than warranted	33			25		38		33	14		83		30		50		14		0		34
Not in line with WEO/GFSR messages	0			0	0	0	0	0	0	0	0		0	0
Other, please specify	6			0		0		17	14		0		0		0		0		100		3
			(in percentages of applicable sample)
Cross-country analysis
5. For all sample: How extensive is the use of cross-country analysis in the staff report?
No cross-country analysis used		10		13		12		8		0		0		17		25		10		25		7
A few instances (1-3)		46		0		35		68		83		50		0		75		30		63		43
Many instances (4-5)		24		50		24		16		17		25		42		0		30		0		29
Extensive use of cross-country analysis		20		38		29		8		0		25		42		0		30		13		21
6. If cross country analysis is used in more than one instance,
please note the characteristics of the comparator sample.
Not applicable
Comparator sample is consistent across instances of use		23		14		14		32		50		14		10		50		0		50		18
Comparator sample is broadly consistent across instances of use		65		71		64		63		50		57		90		50		67		50		68
Comparator sample varies widely across instances of use		13		14		21		5		0		29		0		0		33		0		15
7. If cross country analysis is present in the staff report, what type of analysis is it? (check all that apply, and please indicate how often each type is used with: 0) None, 1) A little, 2) Some. 3) Many questions addressed and 4) Question(s) addressed				Weighted average
Not applicable
Comparison of indicators over a peer group		2.0		2.1		2.4		1.7		1.6		2.1		2.2		1.5		2.3		1.5		2.1
Narrative illustrating experience of one or several other countries		0.6		1.0		0.2		0.8		1.0		1.0		0.4		0.0		1.0		0.0		0.6
Econometric analysis seeking to prove appropriateness of policy		0.5
recommendation				1.0		0.6		0.0		0.0		0.0		1.0		0.0		0.5		1.0		0.5
Other (please specify)		0.0		0.0		0.0		0.0		0.0		0.0		0.0		0.0		0.0		0.0		0.2
8. If cross country analysis in the staff report summarizes the findings in selected issues papers, what type of analysis is it? 1/
Not applicable
Comparison of indicators over a peer group		50		71		17		56		33		60		83		0		25		0		52
Narrative illustrating experience of one or several other countries		9		0		17		11		17		20		0		0		0		0		10
Econometric analysis seeking to prove appropriateness of policy
recommendation		23		14		50		11		17		20		17		0		50		100		19
Cannot tell from the staff report		9		0		17		11		17		0		0		100		0		0		10
Other (please specify)		9		14		0		11		17		0		0		0		25		0		10
9. If the country is facing challenges addressed by recent analysis in the WEO , GFSR, or relevant REO, are these findings reported or otherwise leveraged? 1/
None of the challenges is particularly relevant for the country
Some challenges seem relevant, but no references are made
(specify topics)		82		63		78		100		75		86		80		100		75		100		80
Some references are made (specify topics)		18		38		22		0		25		14		20		0		25		0		20
Extensive references are made (specify topics)		0	000			00000					00
Other (please specify)		0	000			00000					00
Memo item: number of reports in the subsample		50		8		17		25		12		8		12		8		10		8		42

^1/The results are in percentages of all answers given to the question which may be less or more than the 50 sample size.

View Table

Table 7.

Summary of Staff Report Review - Spillovers and Cross Country Analysis

Questions	Total			ADV		EMG		DEV		AFR		APD		EUR		MCD		WHD		Program		Non-Program
							(in percentages of applicable sample)
1. Outward side: is there a discussion of the impact of this country’s outlook/ policies on other economies?
Yes	10			25		12		4	17		0		8		0		20		13		10
Passing reference (acknowledging effects and risks in general terms)	4			064					8		0		0		0		10		13		2
More than passing reference, with some analysis	4			13		6		0	80800										05
Extensive substantive analysis (e.g. possible scenarios and transmission
mechanisms to show size of problem and why it needs attention)	2			13		0		0	0		0		0		0		10		0		2
Other, please specify	0			0		0		0		0		0		0		0		0		0
No discussion	90			75		88		96	83		100		92		100		80		88		90
2. Inward: Where relevant, does the staff report refer to the following trends in its analysis?
Backward-looking trends (NA, Y, N)	NA	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N
(i) commodity price movements	(8)	63	37	57	43	63	38	65	35	75	25	71	29	36	64	86	14	56	44	57	43	6436
(ii) overall global economic environment	(18)	29	71	63	38	20	80	22 78	33	67	14	86	33	67	0	100	50	50	17	83	31 69
(iii) overall global financial environment	(38)	52	48	88	13	47	53	17 83	40	60	0	100	67	33	75	25	43	57	50	50	52	48
(iv) regional trends/developments in neighboring economies	(20)	50	50	83	17	38	62	48	52	44	56	50	50	64	36	33	67	50	50	43	57	52	48
(v) other (please specify)	(67)	14	86	50	50	0	100	0	100	0	100	0	100	33	67	0	0	0	0	0	100	17	83
Forward-looking trends (NA, Y, N)	NA	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N
(i) commodity price movements	(8)	61	39	63	38	67	33	5743	67	33	71	29	50	50	33	67	78	22	43	57	6436
(ii) overall global economic environment	(10)	38	62	75	25	31	69	29	71	40	60	43	57	42	58	14	86	44	56	17	83	41	59
(iii) overall global financial environment	(36)	56	44	88	13	65	35	0	100	20	80	33	67	75	25	60	40	57	43	50	50	57	43
(iv) regional trends/developments in neighboring economies	(18)	41	59	67	33	36	64	38	62	33	67	38	63	27	73	33	67	86	14	29	71	44	56
(v) other (please specify)	(67)	14	86	0	100	0	100	20	80	0	100	0	100	0	100	0	0	100	0	50	50	0	100
3. Inward: Does the staff report refer to relevant risks identified in the last WEO/GFSR in its analysis? (NA, Y, N)	NA	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N
(i) commodity price movements	(20)	55	45	71	29	60	40	44	56	50	50	86	14	45	55	20	80	67	33	0	100	61	39
(ii) overall global economic environment	(22)	38	62	86	14	38	63	19	81	33	67	43	57	45	55	0	100	50	50	0	100	43	57
(iii) overall global financial environment	(30)	49	51	100	0	60	40	8	92	25	75	17	83	67	33	67	33	67	33	0	100	55	45
(iv) regional trends/developments in neighboring economies	(36)	25	75	40	60	27	73	19	81	25	75	29	71	20	80	0	100	40	60	0	100	29	71
(v) other (please specify)	(59)	14	86	0	0	0	100	20	80	0	100	0	100	0	100	0	0	100	0	0	100	17	83
4. If there are references to WEO/GFSR-identified risks, is coverage adequate (in quantity and substance) compared to what would seem warranted? 1/
Yes	61			75		62		50	71		17		70		50		86		0		63
No, because	39			25		38		50	29		83		30		50		14		100		38
Insufficient /generic/more shallow than warranted	33			25		38		33	14		83		30		50		14		0		34
Not in line with WEO/GFSR messages	0			0	0	0	0	0	0	0	0		0	0
Other, please specify	6			0		0		17	14		0		0		0		0		100		3
			(in percentages of applicable sample)
Cross-country analysis
5. For all sample: How extensive is the use of cross-country analysis in the staff report?
No cross-country analysis used		10		13		12		8		0		0		17		25		10		25		7
A few instances (1-3)		46		0		35		68		83		50		0		75		30		63		43
Many instances (4-5)		24		50		24		16		17		25		42		0		30		0		29
Extensive use of cross-country analysis		20		38		29		8		0		25		42		0		30		13		21
6. If cross country analysis is used in more than one instance,
please note the characteristics of the comparator sample.
Not applicable
Comparator sample is consistent across instances of use		23		14		14		32		50		14		10		50		0		50		18
Comparator sample is broadly consistent across instances of use		65		71		64		63		50		57		90		50		67		50		68
Comparator sample varies widely across instances of use		13		14		21		5		0		29		0		0		33		0		15
7. If cross country analysis is present in the staff report, what type of analysis is it? (check all that apply, and please indicate how often each type is used with: 0) None, 1) A little, 2) Some. 3) Many questions addressed and 4) Question(s) addressed				Weighted average
Not applicable
Comparison of indicators over a peer group		2.0		2.1		2.4		1.7		1.6		2.1		2.2		1.5		2.3		1.5		2.1
Narrative illustrating experience of one or several other countries		0.6		1.0		0.2		0.8		1.0		1.0		0.4		0.0		1.0		0.0		0.6
Econometric analysis seeking to prove appropriateness of policy		0.5
recommendation				1.0		0.6		0.0		0.0		0.0		1.0		0.0		0.5		1.0		0.5
Other (please specify)		0.0		0.0		0.0		0.0		0.0		0.0		0.0		0.0		0.0		0.0		0.2
8. If cross country analysis in the staff report summarizes the findings in selected issues papers, what type of analysis is it? 1/
Not applicable
Comparison of indicators over a peer group		50		71		17		56		33		60		83		0		25		0		52
Narrative illustrating experience of one or several other countries		9		0		17		11		17		20		0		0		0		0		10
Econometric analysis seeking to prove appropriateness of policy
recommendation		23		14		50		11		17		20		17		0		50		100		19
Cannot tell from the staff report		9		0		17		11		17		0		0		100		0		0		10
Other (please specify)		9		14		0		11		17		0		0		0		25		0		10
9. If the country is facing challenges addressed by recent analysis in the WEO , GFSR, or relevant REO, are these findings reported or otherwise leveraged? 1/
None of the challenges is particularly relevant for the country
Some challenges seem relevant, but no references are made
(specify topics)		82		63		78		100		75		86		80		100		75		100		80
Some references are made (specify topics)		18		38		22		0		25		14		20		0		25		0		20
Extensive references are made (specify topics)		0	000			00000					00
Other (please specify)		0	000			00000					00
Memo item: number of reports in the subsample		50		8		17		25		12		8		12		8		10		8		42

^1/The results are in percentages of all answers given to the question which may be less or more than the 50 sample size.

Table 8.

Summary of Staff Report Review - Candor

Table 8.

Summary of Staff Report Review - Candor

Questions	Total	ADV	EMG	DEV	AFR	APD	EUR	MCD	WHD	Program	Non-Program
		(in percentages of applicable sample)
Q1. Baseline MT/LT Scenario: Is it possible to identify that (one of) the MT/LT scenario(s) provided is the baseline scenario (defined as the ‘most probable’ scenario)?
(1) Yes	74	88	71	72	75	88	75	63	70	50	79
(2) No	26	13	29	28	25	13	25	38	30	50	21
If the answer to Q1 is (1) YES,
Q1-1. Is the baseline scenario convincing?
(a) Yes, the baseline MT/LT scenario is convincing.	86	100	75	89	67	100	89	80	100	100	85
(b) No, the baseline is not convincing.	14	0	25	11	33	0	11	20	0	0	15
If the answer to Q1-1 is (b) NO, is it because (Please check all that apply)
(i) External assumptions (e.g. exchange rates, CPI of G7 countries, oil and
commodity prices, debt relief, etc.) do not appear realistic and the SR does not
provide clear justification for them. (Please explain: )	0	000			00000					0	0
(ii) Policy Implementation assumptions do not appear realistic and the SR does
not provide clear justification for them.(Please explain: )	60	0	67	50	67	0	0	100	0	0	60
(iii) Underlying parameters (e.g. the elasticity of variable x on y) used in the
baseline MT/LT scenario do not appear realistic and the SR does not provide clear
justification for them. (Please explain: )	20	0	33	0	0	0	100	0	0	0	20
(iv) There are internal inconsistencies (e.g. inconsistencies among MT/LT
scenario, the public and the external DSA).(Please explain: )	0	000			00000					0	0
(v) Others (Please explain: )	20	0	0	50	33	0	0	0	0	0	20
If the answer to Q1 is (2) NO, then is it because
(a) No MT/LT scenario is provided.	8	0	20	0	0	0	33	0	0	0	11
(b) A MT/LT scenario is provided but there is no indication in the SR that this is the baseline
scenario defined as above.	77	100	60	86	67	100	67	100	67	100	67
(c) SR provides two or more scenarios with convincing explanation that the probability of
outcome is equal among all scenarios.	0	000			00000					0	0
(d) SR provides two or more scenarios but provides no indication of probability of any of the
scenarios.	15	0	20	14	33	0	0	0	33	0	22
(e) Others (Please explain: )	0	000			00000					0	0
Q2. Risks to the baseline MT/LT scenario: Does the SR contain sufficient information on the risks surrounding the baseline MT/LT scenario (e.g. risks of the baseline scenario not occurring or occurring differently than expected)?
If the answer to Q1 is (1) YES (i.e. the baseline MT/LT scenario clearly identifiable), please choose
among (1)-(3).
(1) Yes, the risks to the baseline MT/LT scenario are sufficiently covered in the SR.	49	57	33	56	67	57	44	20	43	50	48
(2) There is some coverage of the risks to baseline MT/LT scenario in the SR, but more discussion
is needed. (Please explain: )	41	43	42	39	22	43	44	80	29	25	42
(3) No, there is no coverage of risks to the baseline MT/LT scenario in the SR.	11	0	25	6	11	0	11	0	29	25	9
If the answer to Q1 is (2) NO (i.e. the baseline MT/LT scenario is not clearly identifiable), please
choose between (4)-(6).
(4) No. There is no discussion of risks to the baseline MT/LT scenario in the SR.	23	0	40	14	33	0	33	0	33	25	22
(5) There is some discussion of risks but there is no clearly-identified baseline MT/LT scenario to
link them to.	46	100	20	57	33	0	33	100	33	75	33
(6) Others (Please explain: )	31	0	40	29	33	100	33	0	33	0	44
Q3. Consistency: Is the overall verbal description of the baseline (and alternative, if applicable)
MT/LT scenario consistent with the numerical presentation in the SR?
(1) Yes, numerical and verbal presentation are consistent.	78	63	82	80	75	100	67	88	70	63	81
(2) The numerical and verbal presentations are consistent for some areas, not for others. (Please explain
in comment )	2	004			80000					0	2
(3) No. Verbal presentation does not appear to be a fair representation of numerical analysis (e.g.
excessive hedging). (Please explain in comment)	4	13	0	4	0	0	0	13	10	0	5
(4) Not applicable.	16	25	18	12	17	0	33	0	20	38	12
(a) There exist numerical representation of the baseline MT/LT scenario in the SR (e.g. tables,
charts), but no verbal discussion of this in the text of the SR.	63	100	33	67	50	0	50	0	100	67	60
(b) Others (Please explain in comment )	38	0	67	33	50	0	50	0	0	33	40
Q4. Incorporation of the risks: Are the risks to baseline MT/LT scenario incorporated in the bottom
line message (e.g. staff appraisal) of the SR?
(1) Yes, the risks to baseline MT/LT scenario are appropriately incorporated in the bottom-line message
(e.g. staff appraisal) of the SR.	54	75	53	48	50	63	58	38	60	25	60
(2) Only some of the risks to baseline MT/LT scenario appear to be appropriately incorporated in the
bottom-line message (e.g. staff appraisal) of the SR. (Please explain: )	18	25	12	20	0	25	17	50	10	13	19
(3) No risks appear to be incorporated in the bottom-line message of the SR.	28	0	35	32	50	13	25	13	30	63	21
(a) There is no coverage of risks to the baseline to MT/LT scenario in the SR, and they are not
incorporated in the bottom-line message (e.g. staff appraisal) of the SR.	57	0	83	38	50	0	67	0	100	60	56
(b) There is coverage of risks to baseline MT/LT scenario in the SR, but they are not incorporated
in the bottom-line message (e.g. staff appraisal) of the SR.	43	0	17	63	50	100	33	100	0	40	44
Memo item: number of reports in the subsample	50	8	17	25	12	8	12	8	10	8	42

View Table

Table 8.

Summary of Staff Report Review - Candor

Questions	Total	ADV	EMG	DEV	AFR	APD	EUR	MCD	WHD	Program	Non-Program
		(in percentages of applicable sample)
Q1. Baseline MT/LT Scenario: Is it possible to identify that (one of) the MT/LT scenario(s) provided is the baseline scenario (defined as the ‘most probable’ scenario)?
(1) Yes	74	88	71	72	75	88	75	63	70	50	79
(2) No	26	13	29	28	25	13	25	38	30	50	21
If the answer to Q1 is (1) YES,
Q1-1. Is the baseline scenario convincing?
(a) Yes, the baseline MT/LT scenario is convincing.	86	100	75	89	67	100	89	80	100	100	85
(b) No, the baseline is not convincing.	14	0	25	11	33	0	11	20	0	0	15
If the answer to Q1-1 is (b) NO, is it because (Please check all that apply)
(i) External assumptions (e.g. exchange rates, CPI of G7 countries, oil and
commodity prices, debt relief, etc.) do not appear realistic and the SR does not
provide clear justification for them. (Please explain: )	0	000			00000					0	0
(ii) Policy Implementation assumptions do not appear realistic and the SR does
not provide clear justification for them.(Please explain: )	60	0	67	50	67	0	0	100	0	0	60
(iii) Underlying parameters (e.g. the elasticity of variable x on y) used in the
baseline MT/LT scenario do not appear realistic and the SR does not provide clear
justification for them. (Please explain: )	20	0	33	0	0	0	100	0	0	0	20
(iv) There are internal inconsistencies (e.g. inconsistencies among MT/LT
scenario, the public and the external DSA).(Please explain: )	0	000			00000					0	0
(v) Others (Please explain: )	20	0	0	50	33	0	0	0	0	0	20
If the answer to Q1 is (2) NO, then is it because
(a) No MT/LT scenario is provided.	8	0	20	0	0	0	33	0	0	0	11
(b) A MT/LT scenario is provided but there is no indication in the SR that this is the baseline
scenario defined as above.	77	100	60	86	67	100	67	100	67	100	67
(c) SR provides two or more scenarios with convincing explanation that the probability of
outcome is equal among all scenarios.	0	000			00000					0	0
(d) SR provides two or more scenarios but provides no indication of probability of any of the
scenarios.	15	0	20	14	33	0	0	0	33	0	22
(e) Others (Please explain: )	0	000			00000					0	0
Q2. Risks to the baseline MT/LT scenario: Does the SR contain sufficient information on the risks surrounding the baseline MT/LT scenario (e.g. risks of the baseline scenario not occurring or occurring differently than expected)?
If the answer to Q1 is (1) YES (i.e. the baseline MT/LT scenario clearly identifiable), please choose
among (1)-(3).
(1) Yes, the risks to the baseline MT/LT scenario are sufficiently covered in the SR.	49	57	33	56	67	57	44	20	43	50	48
(2) There is some coverage of the risks to baseline MT/LT scenario in the SR, but more discussion
is needed. (Please explain: )	41	43	42	39	22	43	44	80	29	25	42
(3) No, there is no coverage of risks to the baseline MT/LT scenario in the SR.	11	0	25	6	11	0	11	0	29	25	9
If the answer to Q1 is (2) NO (i.e. the baseline MT/LT scenario is not clearly identifiable), please
choose between (4)-(6).
(4) No. There is no discussion of risks to the baseline MT/LT scenario in the SR.	23	0	40	14	33	0	33	0	33	25	22
(5) There is some discussion of risks but there is no clearly-identified baseline MT/LT scenario to
link them to.	46	100	20	57	33	0	33	100	33	75	33
(6) Others (Please explain: )	31	0	40	29	33	100	33	0	33	0	44
Q3. Consistency: Is the overall verbal description of the baseline (and alternative, if applicable)
MT/LT scenario consistent with the numerical presentation in the SR?
(1) Yes, numerical and verbal presentation are consistent.	78	63	82	80	75	100	67	88	70	63	81
(2) The numerical and verbal presentations are consistent for some areas, not for others. (Please explain
in comment )	2	004			80000					0	2
(3) No. Verbal presentation does not appear to be a fair representation of numerical analysis (e.g.
excessive hedging). (Please explain in comment)	4	13	0	4	0	0	0	13	10	0	5
(4) Not applicable.	16	25	18	12	17	0	33	0	20	38	12
(a) There exist numerical representation of the baseline MT/LT scenario in the SR (e.g. tables,
charts), but no verbal discussion of this in the text of the SR.	63	100	33	67	50	0	50	0	100	67	60
(b) Others (Please explain in comment )	38	0	67	33	50	0	50	0	0	33	40
Q4. Incorporation of the risks: Are the risks to baseline MT/LT scenario incorporated in the bottom
line message (e.g. staff appraisal) of the SR?
(1) Yes, the risks to baseline MT/LT scenario are appropriately incorporated in the bottom-line message
(e.g. staff appraisal) of the SR.	54	75	53	48	50	63	58	38	60	25	60
(2) Only some of the risks to baseline MT/LT scenario appear to be appropriately incorporated in the
bottom-line message (e.g. staff appraisal) of the SR. (Please explain: )	18	25	12	20	0	25	17	50	10	13	19
(3) No risks appear to be incorporated in the bottom-line message of the SR.	28	0	35	32	50	13	25	13	30	63	21
(a) There is no coverage of risks to the baseline to MT/LT scenario in the SR, and they are not
incorporated in the bottom-line message (e.g. staff appraisal) of the SR.	57	0	83	38	50	0	67	0	100	60	56
(b) There is coverage of risks to baseline MT/LT scenario in the SR, but they are not incorporated
in the bottom-line message (e.g. staff appraisal) of the SR.	43	0	17	63	50	100	33	100	0	40	44
Memo item: number of reports in the subsample	50	8	17	25	12	8	12	8	10	8	42

Table 9.

Summary of Staff Report Review - Communication

Table 9.

Summary of Staff Report Review - Communication

Questions	Total		ADV		EMG		DEV		AFR		APD		EUR		MCD		WHD		Program		Non-Program
1. Please assess the text according to the following communication factors (Y/N)	(in percentages of applicable sample)
	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N
(i) Text makes frequent use of (unexplained) jargon, uncommon words, and/or
unclear acronyms.	36	64	38	63	35	65	36	64	17	83	75	25	42	58	38	63	20	80	13	88	40	60
(ii) Text makes frequent use of pro forma language or cliches that weakened the
significance of a statement or conclusion.	36	64	0	100	29	71	52	48	50	50	50	50	8	92	63	38	20	80	25	75	38	62
(iii) Text makes frequent use of qualifying or guarded words or phrases that made it
hard to discern a bottom-line on a topic.	18	82	38	63	18	82	12	88	17	83	0	100	25	75	13	88	30	70	13	88	19	81
(iv) Text makes frequent use of overly long sentences .	30	70	25	75	41	59	24	76	8	92	38	63	25	75	50	50	40	60	25	75	31	69
(v) Text is poorly organized.	14	86	25	75	18	82	8	92	8	92	25	75	0	100	25	75	20	80	13	88	14	86
(vi) Other ( please use this space to write in here any other factor that significantly
impacted the communication quality of the text ).	58	42	63	38	59	41	56	44	42	58	63	38	75	25	50	50	60	40	50	50	60	40
2.a Trying to put aside any information you have about this country other than what is provided in the staff report, the single most appropriate classification for this country among the following possibilities is:
(i) A country that is likely to be highly vulnerable to an economic
(external/domestic) crisis.	6		0		0		12		25		0		0		0		0		13		5
(ii) A country facing significant challenges or risks (other than persistent low
income issues, if these apply), but not highly vulnerable to an economic
(external/domestic) crisis.	30		25		24		36		25		25		42		50		10		63		24
(iii) Broadly alright in terms of domestic stability and external vulnerability
(possibly despite persistent low income issues.)	28		13		41		24		33		38		25		25		20		13		31
(iv) Doing well (despite persistent low income issues.)	36		63		35		28		17		38		33		25		70		13		40
2.b Please name the top risks to external stability in this country, as conveyed by the text (no more than three, can be less than three, and “none” or “not clear” are also possible answers.)
The top risks in this country are: (Name up to three, or choose (1) = “no main
risks” or (2) = “not clear”)
	No		No		No		No		No		No		No		No		No		No		No
	main		main		main		main		main		main		main		main		main		main		main
	risk		risk		risk		risk		risk		risk		risk		risk		risk		risk		risk
	12		25		12		8		17		0		17		13		10		13		12
	Not		Not		Not		Not		Not		Not		Not		Not		Not		Not		Not
	clear		clear		clear		clear		clear		clear		clear		clear		clear		clear		clear
	34		38		37		32		17				42		38		40
Risk 1	58		50		53		64		67		63		50		50		60		63		57
Risk 2	42		38		41		44		33		63		33		50		40		38		43
Risk 3	38		38		41		36		33		50		25		50		40		38		38

View Table

Table 9.

Summary of Staff Report Review - Communication

Questions	Total		ADV		EMG		DEV		AFR		APD		EUR		MCD		WHD		Program		Non-Program
1. Please assess the text according to the following communication factors (Y/N)	(in percentages of applicable sample)
	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N	Y	N
(i) Text makes frequent use of (unexplained) jargon, uncommon words, and/or
unclear acronyms.	36	64	38	63	35	65	36	64	17	83	75	25	42	58	38	63	20	80	13	88	40	60
(ii) Text makes frequent use of pro forma language or cliches that weakened the
significance of a statement or conclusion.	36	64	0	100	29	71	52	48	50	50	50	50	8	92	63	38	20	80	25	75	38	62
(iii) Text makes frequent use of qualifying or guarded words or phrases that made it
hard to discern a bottom-line on a topic.	18	82	38	63	18	82	12	88	17	83	0	100	25	75	13	88	30	70	13	88	19	81
(iv) Text makes frequent use of overly long sentences .	30	70	25	75	41	59	24	76	8	92	38	63	25	75	50	50	40	60	25	75	31	69
(v) Text is poorly organized.	14	86	25	75	18	82	8	92	8	92	25	75	0	100	25	75	20	80	13	88	14	86
(vi) Other ( please use this space to write in here any other factor that significantly
impacted the communication quality of the text ).	58	42	63	38	59	41	56	44	42	58	63	38	75	25	50	50	60	40	50	50	60	40
2.a Trying to put aside any information you have about this country other than what is provided in the staff report, the single most appropriate classification for this country among the following possibilities is:
(i) A country that is likely to be highly vulnerable to an economic
(external/domestic) crisis.	6		0		0		12		25		0		0		0		0		13		5
(ii) A country facing significant challenges or risks (other than persistent low
income issues, if these apply), but not highly vulnerable to an economic
(external/domestic) crisis.	30		25		24		36		25		25		42		50		10		63		24
(iii) Broadly alright in terms of domestic stability and external vulnerability
(possibly despite persistent low income issues.)	28		13		41		24		33		38		25		25		20		13		31
(iv) Doing well (despite persistent low income issues.)	36		63		35		28		17		38		33		25		70		13		40
2.b Please name the top risks to external stability in this country, as conveyed by the text (no more than three, can be less than three, and “none” or “not clear” are also possible answers.)
The top risks in this country are: (Name up to three, or choose (1) = “no main
risks” or (2) = “not clear”)
	No		No		No		No		No		No		No		No		No		No		No
	main		main		main		main		main		main		main		main		main		main		main
	risk		risk		risk		risk		risk		risk		risk		risk		risk		risk		risk
	12		25		12		8		17		0		17		13		10		13		12
	Not		Not		Not		Not		Not		Not		Not		Not		Not		Not		Not
	clear		clear		clear		clear		clear		clear		clear		clear		clear		clear		clear
	34		38		37		32		17				42		38		40
Risk 1	58		50		53		64		67		63		50		50		60		63		57
Risk 2	42		38		41		44		33		63		33		50		40		38		43
Risk 3	38		38		41		36		33		50		25		50		40		38		38

Table 9.

Summary of Staff Report Review - Communication

Table 9.

Summary of Staff Report Review - Communication

Questions	Total	ADV	EMG	DEV	AFR	APD	EUR	MCD	WHD	Program	Non-Program
2.c The staff’s bottom-line view about the exchange rate issues (i.e., level and policy) in this country is clear (4 = strongly agree, 3 = weakly agree; 2 = don’t agree; 1 = not available)		(in percen tages of applicable sample) Weighted average
	3.3	2.8	3.1	3.6	3.4	3.6	2.8	3.4	3.4	3.6	3.2
1. Not available	6	25	6	0	0	0	17	0	10	0	7
2. Don’t agree	20	13	29	16	25	13	25	25	10	13	21
3. Weakly agree	12	25	12	8	8	13	17	13	10	13	12
4. Strongly agree	62	38	53	76	67	75	42	63	70	75	60
2.d The staff’s bottom-line view on financial sector vulnerabilities in this country is clear (4 = strongly agree, 3 = weakly agree; 2 = don’t agree; 1 = not available)		Weighted average
	3.0	4.0	3.0	2.6	2.5	3.3	3.6	3.0	2.5	1.9	3.2
1. Not available	20	0	12	32	42	13	8	0	30	50	14
2. Don’t agree	16	0	29	12	8	13	8	38	20	25	14
3. Weakly agree	12	0	6	20	8	13	0	25	20	13	12
4. Strongly agree	52	100	53	36	42	63	83	38	30	13	60
3. Among the staff’s policy advice/recommendations, does the text distinguish clearly a relatively small set of issues that require utmost priority ( in terms of their importance for stability, not temporal priority)?
(i) Yes, clearly stated and explained.	28	38	12	36	42	25	25	13	30	38	26
(ii) Priorities are presented but not sufficiently explained/defended.	12	13	6	16	25	0	17	0	10	25	10
(iii) Prioritization is not clear, including because there are too many or a “laundry list”
presentation.	60	50	82	48	33	75	58	88	60	38	64
4. Assuming you read this report in order to get the IMF’s specific and conclusive views
on this country, to make an economic or financial decision about the country, how would
you rate the degree to which the report met your needs:
(i) High	34	25	35	36	33	25	25	50	40	38	33
(ii) Medium	48	63	47	44	42	63	67	38	30	38	50
(iii) Low	18	13	18	20	25	13	8	13	30	25	17
5. On balance, do you think that the main conclusions of this text would be comprehensible to the average college-educated adult?
Yes	90	100	76	96	100	88	92	75	90	88	90
No	10	0	24	4	0	13	8	25	10	13	10
Memo item: number of reports in the subsample	50	8	17	25	12	8	12	8	10	842

View Table

Table 9.

Summary of Staff Report Review - Communication

Questions	Total	ADV	EMG	DEV	AFR	APD	EUR	MCD	WHD	Program	Non-Program
2.c The staff’s bottom-line view about the exchange rate issues (i.e., level and policy) in this country is clear (4 = strongly agree, 3 = weakly agree; 2 = don’t agree; 1 = not available)		(in percen tages of applicable sample) Weighted average
	3.3	2.8	3.1	3.6	3.4	3.6	2.8	3.4	3.4	3.6	3.2
1. Not available	6	25	6	0	0	0	17	0	10	0	7
2. Don’t agree	20	13	29	16	25	13	25	25	10	13	21
3. Weakly agree	12	25	12	8	8	13	17	13	10	13	12
4. Strongly agree	62	38	53	76	67	75	42	63	70	75	60
2.d The staff’s bottom-line view on financial sector vulnerabilities in this country is clear (4 = strongly agree, 3 = weakly agree; 2 = don’t agree; 1 = not available)		Weighted average
	3.0	4.0	3.0	2.6	2.5	3.3	3.6	3.0	2.5	1.9	3.2
1. Not available	20	0	12	32	42	13	8	0	30	50	14
2. Don’t agree	16	0	29	12	8	13	8	38	20	25	14
3. Weakly agree	12	0	6	20	8	13	0	25	20	13	12
4. Strongly agree	52	100	53	36	42	63	83	38	30	13	60
3. Among the staff’s policy advice/recommendations, does the text distinguish clearly a relatively small set of issues that require utmost priority ( in terms of their importance for stability, not temporal priority)?
(i) Yes, clearly stated and explained.	28	38	12	36	42	25	25	13	30	38	26
(ii) Priorities are presented but not sufficiently explained/defended.	12	13	6	16	25	0	17	0	10	25	10
(iii) Prioritization is not clear, including because there are too many or a “laundry list”
presentation.	60	50	82	48	33	75	58	88	60	38	64
4. Assuming you read this report in order to get the IMF’s specific and conclusive views
on this country, to make an economic or financial decision about the country, how would
you rate the degree to which the report met your needs:
(i) High	34	25	35	36	33	25	25	50	40	38	33
(ii) Medium	48	63	47	44	42	63	67	38	30	38	50
(iii) Low	18	13	18	20	25	13	8	13	30	25	17
5. On balance, do you think that the main conclusions of this text would be comprehensible to the average college-educated adult?
Yes	90	100	76	96	100	88	92	75	90	88	90
No	10	0	24	4	0	13	8	25	10	13	10
Memo item: number of reports in the subsample	50	8	17	25	12	8	12	8	10	842

Table 9-1.

Summary of Staff Report Review - Communication from Health Check

Table 9-1.

Summary of Staff Report Review - Communication from Health Check

Questions	Total	ADV	EMG	DEV	AFR	APD	EUR	MCD	WHD	Program	Non-Program
				(in percentages of applicable sample)
Communication
14. Was the bottom-line conclusion in the staff report on,
respectively, the exchange rate and the financial sector and the
impact on each on external stability, in agreement with and
supported by the analysis in the report? (answer Y (yes) or N (no);	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
exchange rate:	70	38	82	72	67	88	50	75	80	100		64
financial sector:	64	100	71	48	42	50	75	63	90	50		67
	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA
exchange rate:	10	38	0	8	8	0	33	0	0	0		12
financial sector:	2	0	0	4	8	0	0	0	0	13		0
	N	N	N	N	N	N	N	N	N	N	N
exchange rate:	20	25	18	20	25	13	17	25	20	0		24
financial sector:	34	0	29	48	50	50	25	38	10	38		33
15. Did the length/word count of the report impact how you would
rate its communication?
It felt too long, given the chosen focus/coverage.	22	13	24	24	33	13	25	13	20	38		19
It felt word-count constrained, given the chosen focus/coverage.	12	25	24	0	0	25	8	25	10	0		14
Length as related to the chosen focus and coverage seemed about right.	66	63	53	76	67	63	67	63	70	63		67
16. How does this staff report strike you overall in terms of its
communication quality?
Overall, an example of very good communication quality.	10	13	6	12	0	13	17	0	20	0		12
Overall, an example of good communication quality.	44	75	47	32	25	25	50	50	70	38		45
Overall, an example of poor communication quality.	22	13	18	28	33	38	8	38	0	38		19
Communication quality is neutral or average.	24	0	29	28	42	25	25	13	10	25		24
Memo item: number of reports in the subsample	50	8	17	25	12	8	12	8	10	8		42

View Table

Table 9-1.

Summary of Staff Report Review - Communication from Health Check

Questions	Total	ADV	EMG	DEV	AFR	APD	EUR	MCD	WHD	Program	Non-Program
				(in percentages of applicable sample)
Communication
14. Was the bottom-line conclusion in the staff report on,
respectively, the exchange rate and the financial sector and the
impact on each on external stability, in agreement with and
supported by the analysis in the report? (answer Y (yes) or N (no);	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
exchange rate:	70	38	82	72	67	88	50	75	80	100		64
financial sector:	64	100	71	48	42	50	75	63	90	50		67
	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA
exchange rate:	10	38	0	8	8	0	33	0	0	0		12
financial sector:	2	0	0	4	8	0	0	0	0	13		0
	N	N	N	N	N	N	N	N	N	N	N
exchange rate:	20	25	18	20	25	13	17	25	20	0		24
financial sector:	34	0	29	48	50	50	25	38	10	38		33
15. Did the length/word count of the report impact how you would
rate its communication?
It felt too long, given the chosen focus/coverage.	22	13	24	24	33	13	25	13	20	38		19
It felt word-count constrained, given the chosen focus/coverage.	12	25	24	0	0	25	8	25	10	0		14
Length as related to the chosen focus and coverage seemed about right.	66	63	53	76	67	63	67	63	70	63		67
16. How does this staff report strike you overall in terms of its
communication quality?
Overall, an example of very good communication quality.	10	13	6	12	0	13	17	0	20	0		12
Overall, an example of good communication quality.	44	75	47	32	25	25	50	50	70	38		45
Overall, an example of poor communication quality.	22	13	18	28	33	38	8	38	0	38		19
Communication quality is neutral or average.	24	0	29	28	42	25	25	13	10	25		24
Memo item: number of reports in the subsample	50	8	17	25	12	8	12	8	10	8		42

Table 10.

Summary of Staff Report Review - Evenhandedness

1/ Q3 in Candor is derived from Q2c in the Communication section, answered by non-economist readers.

Table 10.

Summary of Staff Report Review - Evenhandedness

	Does regional difference matter in Surveillance? AFR/APD/EUR/MCD/WHD				Does income matter in surveillance? Does exchange rate regime matter in ADV/EMG/DEV surveillance? Peg/Crawl/Float							Does size matter in surveillance? <$1000bn/ $999-$100bn/$99-$10bn/<$9bn
(in percentages of applicable sample)
Focus	AFR	APD	EUR	MCD	WHD	ADV	EMG	DEV	Peg	Crawl	Float	>$1000bn &999-$100 $99-$10bn			<$9bn
Q: Does the report ask (implicitly or explicitly) one or a few well identified questions? (Y/N)	58	75	100	88	100	100	94	72	83	73	100	100	55	78	88
Health Check
Q1. Is the coverage of recent economic	67	75	92	38	90	88	71	72	74	73	75	71	70	78	70
developments and policies sufficiently
informative?
Q2. Is the short-term economic	50	63	75	38	60	88	59	48	61	40	75	86	64	43	67
outlook, including risks to this outlook,
presented in a substantiated way?
Q3. Is there a clear (implicit or	82	75	78	75	80	80	82	75	73	75	75	100	60	83	75
explicit) assessment of present and
prospective external stability?
Q4. Is there an adequate coverage of	58	88	83	63	90	88	71	76	70	80	83	100	45	78	89
all core macroeconomic policy areas
(i.e., exchange rate, monetary and
fiscal policies), including identification
of key policy challenges?
AVERAGE	64	75	82	53	80	86	71	68	69	67	77	89	60	71	75
Financial Sector Sustainability
Q1. Does the report provide a clear	42	63	58	63	70	75	59	52	43	60	83	86	45	57	56
assessment of FS vulnerabilities? And
if so, is it clear from the report what is
the level of FS vulnerabilities?
Q2. Does the report provide clearly-	42	100	58	38	70	63	53	64	52	73	58	71	36	61	78
justified FS policy recommendations?
AVERAGE	42	81	58	50	70	69	56	58	48	67	71	79	41	59	67
Spill-over and Cross-Country
Q1. Where relevant, does the staff	37	29	38	7	47	69	26	25	27	25	65	64	33	23	33
report refer to the global economic
environment trends in its backward and
forward analysis?
Q2.How extensive is the use of cross-	17	50	83	0	60	88	53	24	35	33	75	86	64	26	38
country analysis in the staff report? (4-
5 instances or more)
Average	27	39	60	4	54	78	39	25	31	29	70	75	48	25	35
Candor
Q1. Risks to the baseline MT/LT	67	57	44	20	43	57	33	56	50	40	56	57	18	26	75
scenario: Does the SR contain
sufficient information on the risks
surrounding the baseline MT/LT
scenario (e.g. risks of the baseline
scenario not occurring or occurring
differently than expected)?
Q2. Incorporation of the risks: Are the	50	63	58	38	60	75	53	48	57	33	75	71	36	57	63
risks to baseline MT/LT scenario
incorporated in the bottom line
message (e.g. staff appraisal) of the
SR?
Q3. Would you agree that the staffs	47	54	33	45	50	33	39	53	42	57	36	33	33	53	54
bottom-line view about the exchange
rate issues (i.e., level and policy) in
this country is clear? 1/
AVERAGE	54	58	45	34	51	55	42	52	49	44	55	54	29	45	64
Overall quality	49	66	69	46	71	78	60	55	56	56	75	79	47	55	66

1/ Q3 in Candor is derived from Q2c in the Communication section, answered by non-economist readers.

View Table

Table 10.

Summary of Staff Report Review - Evenhandedness

	Does regional difference matter in Surveillance? AFR/APD/EUR/MCD/WHD				Does income matter in surveillance? Does exchange rate regime matter in ADV/EMG/DEV surveillance? Peg/Crawl/Float							Does size matter in surveillance? <$1000bn/ $999-$100bn/$99-$10bn/<$9bn
(in percentages of applicable sample)
Focus	AFR	APD	EUR	MCD	WHD	ADV	EMG	DEV	Peg	Crawl	Float	>$1000bn &999-$100 $99-$10bn			<$9bn
Q: Does the report ask (implicitly or explicitly) one or a few well identified questions? (Y/N)	58	75	100	88	100	100	94	72	83	73	100	100	55	78	88
Health Check
Q1. Is the coverage of recent economic	67	75	92	38	90	88	71	72	74	73	75	71	70	78	70
developments and policies sufficiently
informative?
Q2. Is the short-term economic	50	63	75	38	60	88	59	48	61	40	75	86	64	43	67
outlook, including risks to this outlook,
presented in a substantiated way?
Q3. Is there a clear (implicit or	82	75	78	75	80	80	82	75	73	75	75	100	60	83	75
explicit) assessment of present and
prospective external stability?
Q4. Is there an adequate coverage of	58	88	83	63	90	88	71	76	70	80	83	100	45	78	89
all core macroeconomic policy areas
(i.e., exchange rate, monetary and
fiscal policies), including identification
of key policy challenges?
AVERAGE	64	75	82	53	80	86	71	68	69	67	77	89	60	71	75
Financial Sector Sustainability
Q1. Does the report provide a clear	42	63	58	63	70	75	59	52	43	60	83	86	45	57	56
assessment of FS vulnerabilities? And
if so, is it clear from the report what is
the level of FS vulnerabilities?
Q2. Does the report provide clearly-	42	100	58	38	70	63	53	64	52	73	58	71	36	61	78
justified FS policy recommendations?
AVERAGE	42	81	58	50	70	69	56	58	48	67	71	79	41	59	67
Spill-over and Cross-Country
Q1. Where relevant, does the staff	37	29	38	7	47	69	26	25	27	25	65	64	33	23	33
report refer to the global economic
environment trends in its backward and
forward analysis?
Q2.How extensive is the use of cross-	17	50	83	0	60	88	53	24	35	33	75	86	64	26	38
country analysis in the staff report? (4-
5 instances or more)
Average	27	39	60	4	54	78	39	25	31	29	70	75	48	25	35
Candor
Q1. Risks to the baseline MT/LT	67	57	44	20	43	57	33	56	50	40	56	57	18	26	75
scenario: Does the SR contain
sufficient information on the risks
surrounding the baseline MT/LT
scenario (e.g. risks of the baseline
scenario not occurring or occurring
differently than expected)?
Q2. Incorporation of the risks: Are the	50	63	58	38	60	75	53	48	57	33	75	71	36	57	63
risks to baseline MT/LT scenario
incorporated in the bottom line
message (e.g. staff appraisal) of the
SR?
Q3. Would you agree that the staffs	47	54	33	45	50	33	39	53	42	57	36	33	33	53	54
bottom-line view about the exchange
rate issues (i.e., level and policy) in
this country is clear? 1/
AVERAGE	54	58	45	34	51	55	42	52	49	44	55	54	29	45	64
Overall quality	49	66	69	46	71	78	60	55	56	56	75	79	47	55	66

1/ Q3 in Candor is derived from Q2c in the Communication section, answered by non-economist readers.