Chapter 7. Recommendations
- International Monetary Fund. Independent Evaluation Office
- Published Date:
- April 2014
134. Detailed answers to the evaluation questions have already been given at the end of each of the preceding four chapters. Here we assemble the principal findings that call for some corrective action and the corresponding recommendations and relate them to the broader issues of learning, best practice, and communication/transparency.
135. Learning is the area where the evaluation found the most room for improvement. First, while the impact of commissioned studies has been positive, it can be enhanced with the introduction of a more structured process for implementing and disseminating their recommendations. Second, although past forecast performance can be a valuable source of learning about structural changes in the economy, IMF economists do not exploit this opportunity as frequently and systematically as they should. Third, experience matters for better forecasts, especially when these are heavily based on judgment, but is not always shared effectively between successive desk economists. Finally, in-house training related to forecasting would be more valuable if it were more directly related to the economists’ daily work.
Recommendation 1: The IMF should maintain its practice of commissioning external evaluations of IMF forecasts by recognized experts in order to help ensure that forecasts are of high quality and that the process follows best practices.
In commissioning studies the following elements should be considered:
(i) Commissioned studies should be undertaken on a regular schedule.
(ii) Recommendations in the studies should be clearly spelled out.
(iii) The IMF should respond formally to the recommendations. These responses should be made public.
(iv) The findings of the studies should be presented formally to the Executive Board.
(v) An assessment of whether accepted recommendations have been implemented should be included in the subsequent commissioned study.
(vi) Future studies should include an assessment of the forecasting process itself.
Recommendation 2: The IMF should enhance processes and incentives for learning from past forecast performance.
In doing so, the following elements should be kept in mind:
(i) The IMF should consider approaches to develop a more systematic review of forecast errors in the forecasting process. Guidance to staff should make clear that forecasting is an important activity to be taken seriously by mission chiefs as well as desk economists.
(ii) Processes should be implemented to ensure that crucial country information is preserved when staff members move to new country assignments.
(iii) The IMF Institute for Capacity Development should review its course offerings in the area of forecasting to ensure that they include courses relevant for the day-to-day needs of country desk economists.
B. Best Practice
136. The appropriate choice of forecast method will depend at least partly on data availability and the structural characteristics of the economy in question, but sufficient guidance for desk economists on the choice of methods is lacking.
137. While for short-term WEO-round forecasts the IMF has a well-functioning process to promote global consistency, such guidance for medium-term forecasts is less developed.
Recommendation 3: The IMF should extend guidance to desk economists about how best to incorporate advances in forecasting methodologies for both short- and medium-term forecasts.
In doing so:
(i) Attention should focus on how forecasting methods should be adapted to economies with different structural features and data availability.
(ii) As a complement to what it has done for short-term forecasts, the IMF should consider developing a process to ensure the consistency of the medium-term growth projections of major regions and economies based on fundamental structural drivers identified in the literature on the determinants of economic growth. This view should be disseminated to desk economists as a guide to their own country-specific analysis.
(iii) The IMF should monitor the consistency of medium-term forecasts across the institution as it does for the short-term outlook.
138. The IMF-wide forecasting process and the methods used by country desk economists to produce forecasts have been criticized by some country officials as lacking transparency, and this has led to questions about the accuracy and analytical basis of IMF forecasts. For lack of comprehensive documentation of the forecasting process, the evaluation team had to spend considerable effort combining information from various sources to determine the exact nature of the process both at the level of the country desk economist and at the level of the coordination of forecasts within departments and at the IMF as a whole.
139. The databases that contain WEO forecasts are not as user-friendly as they could be, and it is difficult for individuals outside the IMF to access vintages of forecasts other than the most recent. Likewise, certain elements of the MONA database containing information about IMF-supported programs are difficult to access.
Recommendation 4: The IMF should prepare a general description of the WEO forecasting process intended for authorities in member countries and other users of these forecasts. The description should be posted on the publicly accessible part of the IMF website, and it should be reviewed and revised as needed.
In preparing the description the IMF should consider:
(i) Providing a broad understanding of how the top-down and bottom-up components of the forecasts are gathered and combined, and how coordination is achieved within and across area departments.
(ii) Offering information about the assumptions and types of methods used by the IMF to produce country forecasts; while the description should remain general and not country specific, it should provide enough information for readers to understand the IMF’s approach to forecasting.
Recommendation 5: Data related to forecasts and outturns that already exist internally should be made available to the public.
(i) The part of the WEO database that is open to the public should include current and all past forecasts as well as the corresponding realized outcomes.
(ii) The MONA database available externally should include all past forecasts linked to each program review.