IMF Forecasts: Process, Quality, and Country Perspectives

The evaluation report on IMF Forecasts: Process, Quality, and Country Perspectives was released on March 18, 2014. This evaluation assessed the IMF forecasting process and the quality of the resulting forecasts, and surveyed member country authorities for their views on the quality and usefulness of the forecasts published by the IMF.

The evaluation found that the processes and methods used to generate short-term forecasts for Article IV consultations and the World Economic Outlook were well structured and, in general, appropriately tailored to country-specific characteristics. Country officials had confidence in the integrity of IMF forecasts and placed high value on the IMF’s analyses of scenarios and risks for the world economy.

In terms of forecast quality, the evaluation concluded that the accuracy of IMF forecasts was comparable to that of private sector forecasts. There were no significant biases in general except during certain episodes. Specifically, the evaluation found a tendency for significant over-predictions of GDP growth in the World Economic Outlook during regional or global recessions, as well as during crises in individual countries. It also found that short-term forecasts of GDP growth and inflation made in the context of IMF-supported programs tended to be optimistic in high-profile cases characterized by exceptional access to IMF resources and that at the first program review forecast biases were typically reduced or reversed.

The evaluation identified a number of areas for improvement. The IMF should: promote a culture of learning from past forecast performance, including by introducing a more structured process for implementing and disseminating the recommendations of commissioned studies of forecast performance; provide appropriate guidance to desk economists on best practices in forecasting for the short and medium term; and enhance transparency by describing to the public the top-down and bottom-up approaches employed in the World Economic Outlook forecasting process and by making historical forecasts more easily accessible.

In discussing the evaluation on February 27, 2014, the Executive Board welcomed the IEO’s broadly positive findings about the quality of IMF staff forecasts and generally supported all of the IEO’s recommendations. Directors agreed that additional efforts are desirable to enhance learning from forecast errors and independently commissioned studies, to improve transparency in IMF forecasting, and to ensure that best practices and latest methodologies are followed.

Recurring Issues from a Decade of Evaluation: Lessons for the IMF

The evaluation report on Recurring Issues from a Decade of Evaluation was submitted to the Evaluation Committee on April 30, 2014. This evaluation was initiated to address the concern, raised in the second external evaluation of the IEO and endorsed by the Executive Board, that the broader lessons of IEO evaluations tend to be diluted by the current follow-up process. Accordingly, the evaluation identified generic and substantive issues affecting IMF performance from the IEO’s first 20 evaluations and assessed where they currently stand.

The evaluation focused on recurring issues in five areas: Executive Board guidance and oversight; organizational silos; attention to risks and uncertainty; country and institutional context; and evenhandedness. It found that although the Board and Management have taken actions to address the five sets of issues, challenges remain for each and are likely to persist. To varying degrees, these challenges emanate from the IMF’s character as a multilateral institution with multiple objectives and a complex governance structure. Despite their difficulty, efforts to address these issues are important for enhancing the IMF’s effectiveness and credibility.

In keeping with the suggestion of the 2013 external evaluation report, this evaluation did not propose specific recommendations on how to address the five sets of issues. Rather, it put forward for Board consideration a framework of reviewing and monitoring recurring issues that could be useful in establishing incentives for progress, strengthening the Board’s oversight, and providing learning opportunities for the IMF.

This evaluation will be discussed by the Executive Board in early FY2015.

IMF Technical Assistance: Revisiting the 2005 IEO Evaluation

The report on IMF Technical Assistance: Revisiting the 2005 IEO Evaluation was released on April 1, 2014. The report provided an update on key findings and recommendations of its 2005 evaluation of IMF technical assistance. The 2005 evaluation had found that IMF technical assistance was too driven by institutional priorities and not sufficiently linked to country needs and implementation capacity. While IMF technical assistance had generally been effective in improving the technical capabilities of recipient agencies, there was significant variability in the extent to which these agencies were able to follow through and have an impact. The evaluation made recommendations to improve the IMF’s internal process for allocating technical assistance resources among member countries, to enhance the delivery of technical assistance to recipients, and to monitor the impact of technical assistance.

Since the 2005 evaluation, there has been an upsurge in IMF technical assistance activities, with significant financing from outside donors; and the donors’ requirements have coincided with many of the evaluation’s recommendations. The review found that the IMF has given greater emphasis to demand considerations during the process of allocating resources for technical assistance. This has improved country ownership and coordination with donors. The review also found that IMF functional departments have adopted various quality-control and self-assessment practices for technical assistance. And the IMF has intensified its efforts to implement institution-wide results-based management of technical assistance.

Against this background of progress, the report highlighted three issues of continuing importance, namely: giving sufficient attention to country needs and implementation capacity in the allocation of resources for technical assistance activities; ensuring that the IMF’s quality-assurance processes are able to cope with the significant expansion of technical assistance activities; and improving the monitoring and evaluation of IMF technical assistance.

The report is the third in an IEO series that revisits past evaluations. The first two reports in this series, which revisited the 2002 evaluation of Prolonged Use of IMF Resources and the 2004 evaluation of Fiscal Adjustment in IMF-Supported Programs, were included in last year’s Annual Report. All three reports are available on the IEO website’s new dedicated page featuring evaluation updates.