Review of Ex Post Assessments and Issues Relating to the Policy on Longer-Term Program Engagement
Author:
International Monetary Fund
Search for other papers by International Monetary Fund in
Current site
Google Scholar Close
Publication Date:
20 Mar 2006
eISBN:
9781498332903
Language:
English
Keywords:
PP; EPA report; EPA team; country authorities; EPA staff report; Moldova EPA; a number of EPAS; EPA team leaders; LTPE definition; staff EPA; EPA unit
Download PDF (1.1 MB)

This review assesses the experience gained to date in carrying out the longer-term program engagement (LTPE) policy and preparing the ex post assessments (EPAs). It discusses the main characteristics of LTPE members and the findings of the EPAs, analyzes the role of the EPAs in the Fund’s due diligence, and provides recommendations for improving their effectiveness.

Abstract

This review assesses the experience gained to date in carrying out the longer-term program engagement (LTPE) policy and preparing the ex post assessments (EPAs). It discusses the main characteristics of LTPE members and the findings of the EPAs, analyzes the role of the EPAs in the Fund’s due diligence, and provides recommendations for improving their effectiveness.

I. Introduction

1. The Fund’s policies on longer-term program engagement (LTPE) were established in early 2003, in response to the report of the Independent Evaluation Office (IEo) on prolonged use of Fund resources. Following the report, Management convened a Task Force to consider the IEO’s recommendations.1 At the discussion of the Task Force’s conclusions, the Board endorsed specific measures to help guard against problems that may arise from prolonged use.2 These measures included new procedures to strengthen the Fund’s “due diligence” for countries with LTPE through systematic ex post assessments and strategic forward planning.3 At the same time, the Board stressed the importance of continued efforts to improve the prospects for successful implementation of all Fund-supported programs, including through measures to strengthen Fund surveillance and improve program design and conditionality.

2. To promote the effective implementation of LTPE policies, the Board called for a systematic follow-up. To this end, the present paper draws on experience gained to date in carrying out the policy and the preparation of EPA reports. It is organized as follows: Chapter II provides background discussion on key policy issues and requirements for EPAs, and summarizes the main characteristics of LTPE members as well as the findings of the 32 EPAs produced through end-August 2005. Chapter III analyzes the experience with EPAs to date, their role in the LTPE policies, and the resource costs involved in preparing them. Chapter IV discusses the EPA regime as well as definitional and procedural issues that have emerged in the application of the LTPE policies, and makes recommendations for improving the effectiveness of EPAs. Chapter V outlines issues for discussion.

II. Background

A. The Problem of Longer-Term Program Engagement

3. The Fund’s policies reflect a judgment that, in some cases, LTPE may be a sign of inadequate progress in dealing with members’ key economic problems and could have significant adverse implications for the credibility and effectiveness of Fund-supported programs. In particular,

  • LTPE might be the result of inadequate design or implementation of programs, and a reluctance of the Fund to promote a more ambitious adjustment and reform agenda as a condition for its continued support.4 In this context, LTPE signals a failure to help members attain external viability so that further Fund financial support is no longer needed—a key objective of Fund-supported programs.

  • LTPE might interfere with some members’ institutional development. While the impact of Fund advice and support on development of technical economic management skills tends to be broadly positive, there is a concern that LTPE may in some cases have an adverse effect on policy formulation, preventing the member from “standing on its own feet.” 5 6

  • It may also have an adverse impact on the Fund’s general resources and liquidity, and on the availability of limited PRGF resources to support low-income members.

4. At the same time, LTPE may be appropriate under certain circumstances. A member’s sustained engagement with the Fund through a succession of financial arrangements may be appropriate and desirable to address deep-seated economic and structural problems that by their nature require many years to resolve.7 LTPE may also reflect members’ demand for Fund-supported programs to provide a seal of approval about the quality of their policies where this is important to help secure support from donors and creditors.

B. operational Aspects of the Policy

5. The policy is based on an explicit definition of the concept of LTPE. For upper credit tranche stand-by or extended arrangements or a mix of GRA and PRGF or ESAF resources, a member is defined as having LTPE when it has spent at least seven of the last ten years under such arrangements. Arrangements that are cancelled are counted up to the time of cancellation. Members with access to concessional Fund resources are defined as having LTPE when they have had two or more multi-year arrangements under the PRGF or ESAF, including those that went off track or were cancelled.8

6. Precautionary arrangements are counted in the definition of LTPE. During the discussion of the Task Force’s conclusions, it was recognized that precautionary arrangements do not normally involve drawings of Fund resources, and may be used to provide an effective signaling device to facilitate members’ transition from sustained reliance on Fund financing. However, it was noted that precautionary arrangements are not formally distinct from other stand-by arrangements; the member retains the right to make purchases, and sometimes does so, and such arrangements reduce the Fund’s forward commitment capacity. More recently, it was decided that time spent under the Policy Support Instrument (PSI) would not count towards determining LTPE, and Directors agreed to revisit this issue at the time of the present review.9

7. For members with LTPE, EPAs were established to assess the overall strategy with the member, focusing on identifying lessons for future Fund involvement. EPAs are expected to provide “an analysis of the economic problems facing the country, a critical and frank review of progress during the period of Fund-supported programs, and a forward-looking assessment that takes into account lessons learned and presents a strategy for future Fund engagement.”10 Such an assessment should distinguish those cases in which LTPE has been and remains beneficial from those in which it largely reflects inadequate progress in achieving program objectives. Procedural requirements for EPAs are designed to encourage a fresh perspective and credibility, and to ensure that they are completed prior to discussions on any successor arrangement (Box 1).

Procedural Requirements for EPAs1

EPA reports are prepared by an inter-departmental staff team. They should include representatives from the area department and one each from PDR and at least one other functional department.2 The team leader, nominated by the area department, should, as best practice, not be the current mission chief, unless he/she is carrying out his/her first assignment on that country.

EPAs are prepared ahead of discussions for a successor arrangement. They should preferably be prepared (in draft form) prior to the briefing paper for an Article IV consultation when the existing arrangement is substantially complete. In those cases where this is not possible, a draft EPA should be prepared prior to the briefing paper for the final review of the arrangement or for a combined Article IV-UFR mission. In exceptional cases when this proves not feasible, a stand-alone Board meeting can be held to discuss the EPA, as was done for Georgia and Uruguay. In any case, the Board is expected to consider the EPA before the discussions for a new arrangement are completed. This requirement is essential given the need for EPAs to inform prospective Fund engagement. If no successor arrangement is contemplated, an EPA should be undertaken in the context of the first post-program Article IV consultation.

While draft EPA reports are to be discussed with the authorities, their analysis and conclusions should not be subject to negotiation. Only factual corrections and limited revisions should be incorporated in a revised report. The authorities’ views are expected to be noted in the staff report accompanying the EPA.

The publication of EPAs is voluntary but presumed. A PIN is issued after the Board meeting with the consent of the authorities. If the authorities do not consent to the publication of a PIN, a brief factual statement is issued to inform the public that the discussion took place.

1 Staff guidance on EPAs is set out in “Operational Guidance for Assessments of Countries with Longer-Term Program Engagement,” August 20, 2003. 2 The team should solicit the views of the Fund resident representative and the World Bank. It could also draw on outside experts, in cases in which this is considered useful.

8. In addition, LTPE policies placed emphasis on the need to devise better programs so that the objective of limiting inappropriate prolonged use can be achieved. Consistent with this objective, a number of recent efforts have been undertaken aiming to strengthen surveillance, program design, and conditionality.11

C. Members with LTPE

9. Since the framework was adopted, 57 member countries have been identified as having LTPE (Table 1). More than 80 percent of these members are low-income countries (LICs) where entrenched structural weaknesses exacerbated at times by emergence from episodes of armed conflict, require a long-term development effort. Among LICs, LTPE has been the norm rather than the exception, as nearly all members that have borrowed concessional resources currently meet the LTPE definition. HIPC countries make up almost half of LTPE cases and constitute the majority of LICs with LTPE (59 percent).12 13

Table 1.

Members with Longer-Term Program Engagement, 2003-2005 1/

article image

Members for which an EPA has been issued and discussed, as of end-August 2005.

Members for which an EPA was issued after August 2005 and through February 2006.

All members meeting the LTPE definition from the time the policy was adopted to August 2005.

Macedonia FYR meets the LTPE definition due to its two arrangements supported by ESAF/PRGF resources in 1997-2000 and 2000-2001, respectively. Macedonia FYR has since graduated from use of concessional resources. The current SBA turned precautionary after a first drawing upon approval.

No formal EPA has been prepared for Latvia; the 2004 Article IV staff report included a short discussion on the effectiveness of Fund recommendations during the 1992-2002 period (the last SBA expired in December 2002).

Bolivia has a stand-by arrangement.

Moldova has had only one PRGF but meets the LTPE definition due to its stand-by and extended arrangements.

10. LTPE has also been common among transition economies, where institution building and structural reforms have played an important role, along with macroeconomic stabilization (Figure 1). Two members that met the definition at end-2003 are no longer considered as having LTPE (Kazakhstan and Latvia), reflecting a trend toward less reliance on Fund-supported programs in this group. The share of transition economies among countries with LTPE is expected to decline further, as several EU-accession countries graduate from the use of Fund resources.

Figure 1.
Figure 1.

Members with LTPE and Members with Completed EPAs

Citation: Policy Papers 2006, 008; 10.5089/9781498332903.007.A001

1/ Includes two countries that no longer meet the definition of LTPE.

11. The number of non-transition, middle income countries with LTPE has remained small. Only four such cases have had LTPE since end-2003. Three of these members had at least one precautionary arrangement during the past ten years.

D. Key Findings of the EPAs

12. The present study covers the 32 EPAs completed by end-August 2005.14 These covered a broad range of programs, especially among LICs, and their composition is broadly representative of LTPE members (Figure 1).

13. In the EPAs, LTPE is attributed to the members’ initial conditions and uneven program implementation, and to a lesser extent to inadequate program design.15 In particular:

  • Members with LTPE were seen as facing multiple macroeconomic and structural challenges. On the macroeconomic side, inflation, fiscal consolidation, debt sustainability (both public and external), and current account imbalances were the most commonly reported immediate challenges. Growth and poverty reduction objectives were, as expected, prevalent among PRGF users. On the structural side, public enterprise reform, financial sector issues, and governance problems were commonly reported. Pension reform and labor market issues were important for GRA users, while public expenditure management and civil service reform were more common among PRGF users.

  • Policy design was generally considered consistent with the identified challenges. Design problems with macroeconomic policies under Fund-supported programs were identified in relatively few cases.16 EPAs, however, were more critical of the design of structural reforms. Several EPAs found problems with structural conditionality, and in about one quarter of all cases, problems persisted during the LTPE period. In some cases the issue was the focus of structural conditionality, in others the quantity. For PRGF users, EPAs more often found excessive structural conditionality. In a few cases stronger conditionality was considered to be needed.17 Concomitantly, progress in addressing macroeconomic problems was generally seen as better than in tackling structural challenges.

  • Program performance was seen as being plagued by implementation problems.18 Slippages were frequently noted in meeting quantitative fiscal targets and structural reforms which required legislative approval or that had significant distributional implications, particularly pension, civil service, and labor market reforms. Lack of ownership was the most frequent reason for implementation problems, while lack of technical expertise (in PRGF countries), coordination problems with the World Bank (mostly in PRGF countries), and unfavorable external shocks also played a role.

  • Continued Fund program engagement was typically considered useful. In about 75 percent of cases, EPAs’ call for continued Fund involvement was justified by reference to an unfinished structural reform agenda and the view that Fund input and advice could strengthen the reform effort in this area. The role of signaling was also noted in about two-thirds of EPAs. Balance of payments needs were of lesser importance. In two cases, a new Fund-supported program was not recommended; in both cases, the authorities had already expressed the intention not to seek another arrangement (Kazakhstan and Vietnam).19

  • EPAs did not report a negative impact of continued Fund-supported programs on institutional development and policy formulation, a key concern raised in the IEO report. In fact, several EPAs found a positive impact on members’ strength of domestic institutions and technical capacity.

14. The EPAs confirmed many of the trends described in earlier reviews of Fund program design and conditionality. While on some issues individual EPAs came to different conclusions and the coverage of some important aspects of program design was uneven, in general their findings were consistent with those in the policy reviews.20 Box 2 summarizes the main results of these reviews and compares them to the findings of the EPAs.

EPA Findings Compared with Lessons from the Reviews of Conditionality and Program Design

2005 Conditionality Review—some key messages

  • Streamlining of structural conditions to cover critical measures has improved since 2001. While the number of conditions in programs has not declined, sectoral coverage has become more focused on core areas of Fund expertise, consistent with the revised guidelines on conditionality in 2002.

  • Implementation has improved, in line with a greater focus on ownership, but there remains room for further improvement.

  • Going forward, the review recommended: (i) more selectivity in cases of poor ownership, instead of compensating with more conditionality; and (ii) fostering ownership by providing more policy space—for example by accepting more policy options and allowing more realistic timetables.

Findings of EPAs on conditionality:

  • In most cases, Fund-supported programs focused on macro-critical issues and aligned conditionality accordingly. However, some EPAs pointed out shortcomings in the design of structural conditionality.

  • Views on streamlining of conditionality were mixed. Most EPAs did not find excessive conditionality and several EPAs saw evidence of streamlining in the breadth of sectoral coverage. However, in about one fifth of the cases, EPAs saw too much conditionality as a problem and advocated streamlining. In a few other cases, EPAs concluded that stronger conditionality, often in the form of prior actions, would have been appropriate.

  • Most countries had at least some episodes of poor implementation which EPAs usually attributed to a lack of ownership. It is not clear to what extent greater focus on ownership would have affected implementation, as EPAs generally do not differentiate between periods before and after the changes in the conditionality guidelines.

  • EPAs advocated more selectivity in terms of program engagement where ownership was poor, but did not discuss whether providing more policy space would have improved ownership. While some EPAs agree on the need for more realistic timetables, a number of EPAs criticized the Fund for having been too accommodative of domestic policy constraints and implementation slippages, and others supported decisions to delay reviews when ownership was poor. In this respect, the EPAs do not seem to share the recommendation that programs foster greater ownership by providing more policy space.

2004 Review of Program Design—some key results:

  • External adjustment was broadly appropriate in GRA countries, but not sufficient to ensure debt sustainability in PRGF countries without debt relief. However, in GRA countries the adjustment was sometimes too abrupt.

  • Macroeconomic projections tended to be overly optimistic. This applies especially to GDP growth and inflation projections for PRGF countries and over longer horizons. Short-term projections tended to be more accurate. Often related to lower growth, projections of debt ratios were frequently overoptimistic in PRGF countries.

  • Macroeconomic and structural policies were generally aligned to the program purpose, but the following patterns emerged: (i) inflation was usually higher than programmed, primarily because of monetary overruns; (ii) successful disinflation did not depend on the exchange rate regime, but on fiscal adjustment; and (iii) fiscal adjustment generally fell short of programmed levels.

Findings of EPAs:

  • EPAs only rarely discussed the appropriate balance of financing and external adjustment. Debt sustainability issues were generally taken into account in Fund-supported programs. In the case of HIPC countries, this involved relying on debt relief. In a few cases programs failed to prevent a worsening of debt problems largely due to an overestimation of the capacity to repay debt and higher-than-programmed external borrowing.1

  • While many EPAs reported errors in program projections, it is difficult to identify any systematic biases. Some cases reported overly optimistic and others overly pessimistic projections of key macroeconomic indicators.

  • Many EPAs confirmed the results of the review of program design regarding macroeconomic and structural policies. Successful disinflation was usually accompanied by successful fiscal adjustment as well as monetary restraint. The exchange rate regime did not appear to play a systematic role in that respect. Fiscal slippages were common. However, somewhat in contrast to the program design review, higher than programmed inflation was generally not identified as a problem.

1/ For example in Bolivia, Kyrgyz Republic, and Moldova.

III. Assessment of EPAs

15. This section provides an assessment of the EPA reports. It draws on internal evaluations of the 32 EPAs, focusing on whether the EPAs addressed the right questions and provided analysis and recommendations that are internally consistent and convincing. The evaluations did not directly examine whether the reports were “right” (for which separate ex post assessments would have been needed) but used surveys of EPA team leaders, mission chiefs, and member country authorities to inform judgments regarding the independence, usefulness, and effectiveness of the EPAs. (Box 3 discusses the methodology for this review and Annexes II-IV provide detailed information on the questionnaire and surveys used.) The purpose of this exercise is not to rate individual reports or single out individual issues, but to derive lessons from the EPA experience, both in terms of procedures and output, and, thereby, identify best practice for future EPAs.

Methodology for Assessing EPAs

To assess experience with EPAs as well as the LTPE policy, all the EPA reports issued through end-August 2005 were reviewed and evaluated. In addition, surveys were conducted of the EPA team leaders, the mission chiefs (where different from the EPA team leaders), and the authorities of the respective members:

The evaluations focused on a few areas:

  • Looking backward, were the focus and content of the reports appropriate to (i) explain the reasons for LTPE and the economic challenges members faced, and (ii) assess the policy design and program conditionality and provide a frank discussion of progress made under Fund-supported programs, including an examination of what worked and why program objectives were or were not achieved?

  • Looking forward, did the reports articulate a clear strategy for future Fund engagement consistent with the lessons learned from experience, and have the EPA findings materially affected the design of successor programs?

Questionnaire: To provide for a consistent evaluation of the 32 reports and standardization of ratings, the staff developed a detailed questionnaire containing 24 questions on different aspects of the quality of the report’s discussion. For each of these questions, the EPAs were rated on a scale of 1=limited and weak discussion to 3=very good analysis, with 0 assigned for issues not discussed. The questionnaire also included 37 questions on the findings of the reports (discussed in section II above) and auxiliary questions on the authorities’ and the Executive Board’s reaction to the reports.

Survey of EPA team leaders (72 percent response rate): A web-based survey administered by TGS was sent to staff who led the EPA teams. Questions covered the adequacy of inputs (time and staff), the freshness of perspective, findings of the EPA, and the impact of the EPA on subsequent relations.

Survey of mission chiefs (70 percent response rate): A web-based survey administered by TGS was sent to staff who led Article IV and program negotiation missions immediately after the EPA reports were presented to the Board. The survey was not sent to mission chiefs in the 11 cases where they were also the EPA team leader. Questions in the survey related to the value added of the EPA, the interaction between the country and EPA teams, and the impact on subsequent relations with the authorities.

Country Authorities (34 percent response rate): A list of questions was sent to all 32 country authorities through Executive Directors’ offices. Questions covered procedural issues, the analysis presented in the EPA, and the usefulness and impact of the exercise.

A. Experience with EPAs

16. The EPAs’ chief advantage is that they enable the Fund and the authorities to step back from the more immediate demands (and confines) of program operations to assess the effectiveness of the relationship. While Article IV reports are also expected to provide a fresh perspective, the EPAs provide an institutional mechanism for a thorough analysis of economic challenges and distillation of lessons from a member’s evolving program relationship with the Fund in a broader and more long-term context. Having examined the effectiveness of Fund-supported programs, the EPAs are well placed to suggest a strategy for the Fund looking ahead, including, where possible, for the member’s lasting exit from Fund financing. Unlike Article IV consultations, EPAs are expected to be timed to provide input to successor arrangements, where such arrangements are requested.

17. The internal review of the EPAs indicates that they have proved useful in some cases (Table 2).

Table 2.

Summary Assessment of EPAs

article image

  • Overall, EPAs have succeeded in explaining the macroeconomic and structural issues that confronted members over the review period, though the depth of coverage varied across reports (Figure 2). Specific topics, such as medium-term fiscal reform and sustainability, financial sector reform, and civil service and public enterprise reform were well covered where relevant. Some reports were very effective in focusing on specific issues that played an important role in economic performance (Box 4).

    Figure 2.
    Figure 2.
    Figure 2.

    Analytical Content of Discussion

    (in percent of total EPA reports)

    Citation: Policy Papers 2006, 008; 10.5089/9781498332903.007.A001

    Source: Staff Analysis of 32 EPAs prepared through end-August 2005.Sources: Staff Analysis of 32 EPAs prepared through end-August 2005, and survey of mission chiefs.

    Best Practices

    Some presentations have been particularly effective in meeting the objectives of EPAs. Examples of best practice in the analysis of events, distillation of lessons, and discussion of future Fund engagement include:

    • A thorough analysis of policy challenges and various aspects of Fund engagement, including the HIPC Initiative, were included in the EPA for Bolivia. The report examined key elements of economic performance, including the unraveling of the fiscal situation, and the government’s hydrocarbon strategy, and the role of the Fund in these areas. It also candidly discussed the Fund’s vulnerability to yielding to political pressure and consequent accommodation of program slippages.

    • Key issues relating to macroeconomic performance were carefully analyzed in the EPA for Malawi, including issues such as growth, food security, and governance. The report also discussed program flexibility, and argued convincingly for its recommended course in Fund relations (use of a Staff Monitored Program before a resumption of financial engagement). The Moldova EPA took a critical view of the content and sequencing of reforms in past Fund-supported programs, and suggested how reforms might have been sequenced differently.

    • The areas of success and failure during the period of Fund program engagement were laid out clearly in the EPA for Vietnam. The report is concise and frank and did not shy away from pointing out the limitations of the Fund’s influence, and its dependence on broad support from donors, and in particular from the World Bank.

    • The strategic aspects of Fund program engagement were distilled effectively in the EPA for Bulgaria, which pinned down the elements of program design that differentiated successful from unsuccessful programs. The report also integrated a discussion of the World Bank’s engagement, and the interaction between the Fund and the Bank.

    • Lessons learned from Fund engagement are clearly laid out in the EPA for The Gambia which sets priorities for the future in a direct and concise way. The report shows that recommendations for future Fund engagement, even when they are not innovative (one more program, and then reevaluation) can be well-argued. Lessons learned were also clearly laid out in the EPA for Uruguay.

  • The discussion of program design and performance under Fund-supported programs was of high quality in several EPAs. Nearly all reports provided a detailed assessment of the appropriateness of structural conditionality and its focus. Discussions of policy targets and outturns of key macroeconomic variables featured prominently and were well documented.

  • All EPAs presented a medium-term policy program and a coherent plan for continued Fund engagement. The reports laid out the policies required to address remaining economic and structural challenges, and identified elements of policy programs that could be supported by the Fund.

  • Over time, as experience was gained, EPAs generally improved. A number of reports were effective in describing complicated developments in the reform process, and explained well progress and delays compared to the corresponding surveillance or program reports.

18. In some areas, however, this review considers that the EPA reports were less effective. In particular:

  • In a number of cases, the analytical content could have been stronger. A focus on key issues was more effective than a chronological description of developments or an attempt to cover all topics. In general, there remains scope for greater selectivity in coverage and at the same time more rigorous treatment of issues of critical importance for the achievement of policy objectives, including identification of vulnerabilities and risks to the programs, and a clear assessment of policy dilemmas confronting country authorities and Fund staff. New insights, compared to existing Fund documents, were somewhat limited in several reports, a point also noted by the majority of mission chiefs in the survey responses. Of course, the lack of new insights should not be necessarily interpreted as that EPAs are not useful. If a country’s strategy has proved successful, changes in program design may not be necessary.21

  • In many cases, critical aspects of program design were covered to a limited degree. In particular, the appropriateness of quantitative conditionality to support program objectives was not widely assessed, and in most EPAs an explicit discussion of the appropriateness of the mix between adjustment and financing as well as access levels was missing (though coverage of the issue was somewhat more frequent in GRA than in PRGF cases).22

  • The reasons that members had LTPE with the Fund and the rationale for Fund engagement could also be better documented. While these were well discussed in EPAs covering GRA programs, the discussion was limited in PRGF and HIPC Initiative cases, and in one third of EPAs a discussion was missing. The implicit assumption appears to have been that the Fund has an ongoing role in helping low-income members to address deep-seated macroeconomic, structural and institutional challenges through Fund-supported arrangements. EPAs did not question the basic infrastructure that generates a large number of members with LTPE.

  • A relatively small number of EPA reports assessed the appropriateness of the ongoing policy strategy. While some reports, in particular for transition economies, examined the Fund’s overall approach and soundness of its advice for addressing members’ challenges, others covered such issues only to a limited degree. The apparent lack of attention to these issues may reflect the Fund’s role in LICs being more focused on macroeconomic management and less on adjustment to a specific balance of payments shock.

19. Overall, country authorities, with few exceptions, thought that EPAs described past policies and programs accurately, drew the right conclusions, and provided valuable lessons. Although the low response rate limits strong inferences, those that responded considered that they understood sufficiently well the purpose and process of the EPA and that they had sufficient opportunity to express their views. They regarded EPAs as a valuable tool to assess their longer-term program involvement with the Fund. In some cases, they expressed differences of opinion on the staffs interpretation of specific policies or events, or considered that the issues the authorities judged important had received insufficient attention in the EPA.23

20. In the same vein, in the summings up for Board meetings covering EPAs, Executive Directors welcomed the opportunity to review the Fund’s involvement in individual country cases and generally agreed with the EPA findings. Directors considered that the EPAs provided useful lessons and welcomed the exchange of views between staff and the authorities on members’ long term program relationship with the Fund. They took the opportunity to comment on various aspects of program design, and in particular on conditionality and ownership, where they emphasized the importance of political commitment to reforms. Directors generally supported the forward-looking strategies and reform priorities advocated in EPAs, and stressed that these should be incorporated in successor arrangements.

B. The Role of EPAs

21. The assessment of the EPAs brought to the fore a number of broader issues regarding the role that EPAs should play in the Fund’s due diligence, and their impact on the Fund’s relations with its members.

  • Although the reports can raise useful questions, draw analytical and strategic lessons for policy design, and allow for some reflection on the Fund’s longer-term relation with a member, they are unlikely to launch a fundamental challenge to the current strategy and relationship with a particular country because they remain a product of staff and management.24 While in many cases a fundamental challenge to country strategies and policies may not be needed as these are indeed appropriate, in some cases a reevaluation and redirection of Fund policies may be called for. In this context, EPA team leaders expressed the concern that the current process of producing EPAs may discourage criticism of policies pursued (Figure 3). In particular, some EPA team leaders felt that there was resistance from area department front offices and PDR to critical assessments of program design and Fund involvement. Several also considered that there was, at least implicitly, a need to align the message of the EPA with that of the corresponding Article IV (or program review) report. It may be that in the most controversial cases—where the Fund’s involvement is perhaps most in need of reexamination—the independence of the EPA team is most important.

    Figure 3.
    Figure 3.

    Independence

    (In percent of total EPA reports)

    Citation: Policy Papers 2006, 008; 10.5089/9781498332903.007.A001

  • The development of clear exit strategies from Fund program involvement is a challenge for EPA reports. This may reflect the premise, particularly for LICs, that Fund-supported programs can play a useful role by providing an anchor for macroeconomic policies and helping to advance structural reforms.25 The recent creation of the PSI, a voluntary and demand-driven instrument, could facilitate the definition of a strategy for LICs for exiting from use of Fund resources when there is no need for financial engagement, but rather a need for signaling.

  • While EPAs provide a useful framework for stepping back and evaluating a program relationship, it is not clear how directly this affects future program design. In this regard, mission chiefs and EPA team leaders had somewhat diverging views (Figure 4). While most team leaders thought that EPAs had no impact on subsequent programs, two-thirds of mission chiefs of countries for which EPAs had been prepared found that the findings of EPAs had an important impact on subsequent Fund engagement. It should be recognized that it is difficult to gauge the impact of EPAs on successor arrangements and attribute specific measures to the EPAs as it is unclear to what degree subsequent programs would have been different in the absence of the EPAs.26

    Figure 4.
    Figure 4.

    Impact of EPAs

    (In percent of total EPA reports)

    Citation: Policy Papers 2006, 008; 10.5089/9781498332903.007.A001

    Source: Survey of EPA leaders and mission chiefs.

C. Resource Costs

22. The costs of preparing EPAs have been higher than expected (Table 3). It was originally estimated that additional resources of about US$1 million per year in FY 2004 and 2005 would be needed, with roughly three-fourths of this required for PRGF-eligible countries.27 Information provided by EPA team leaders and time logged in the Fund’s Time Reporting System (TRS) suggest that on average the EPA team spent about 6 staff months preparing the EPA report.28 Including estimated mission costs for the EPA leaders (where they traveled to LTPE countries), this is equivalent to about US$1.5 million in 2004 and US$2.4 million in the first eight months of 2005. As these estimates do not include costs related to the review process (for which there is no information available), the total staff costs are higher.

Table 3:

Aggregate Costs of EPAs 1/

article image

For FY 2005, TRS data were integrated with estimates provided by EPA-team leaders.

For 2004, the figures are based on information provided by EPA-team leaders as the Budget Reporting System did not allow identification of time spent on EPAs.

Travel costs were added in those cases where EPA-team leaders other than the mission chief traveled to countries separately from an Article IV or program negotiation mission.

Estimates of staff years were translated into dollar amounts by applying standard annual staff costs. Standard cost estimates for staff and travel were provided by the Office of Budget and Planning.

23. Aggregate costs are expected to remain high in 2006-07. Under current policies, 19 EPAs are due and are expected to be completed before negotiations for a new program begin or, for members without a successor arrangement, at the time of the first post-program Article IV consultation. Looking further ahead, new EPAs will be required for members that have already had one, and are close to completing their current Fund-supported programs, as under the current rules a new EPA is required at the end of each new arrangement for members that continue to meet the LTPE definition. Depending on the number and length of successor arrangements as well as the number of PRGF arrangements that would be succeeded by PSI, there could be 10-11 EPAs per year in the medium-term.29

IV. Recommendations

24. There is reason to consider changes to the policy on EPAs. EPAs are unlike any other instrument at the Fund’s disposal, and provide an opportunity for careful review and discussion of strategy for future program engagement, involving the Executive Board, during a prolonged period of Fund-supported programs. However, the quality and effectiveness of EPAs have been uneven, and the resource costs involved are considerable and higher than expected.

A. The EPA Regime

25. Against this background, three broad approaches regarding EPAs could be considered:

26. One approach would be to seek to strengthen the EPA process to allow this instrument to realize more of its potential. A key way to achieve this could be to increase the independence of EPA teams. To the extent that pressure from outside the EPA team and subordination of EPAs to the current country strategy is, or is perceived to be, an issue, this could be tackled by increasing EPA chiefs’ distance from departments and country authorities. Different options could be considered:

  • EPAs could be prepared by experts outside the Fund staff, either by the IEO or external consultants. This would promise a higher degree of independence. It might also bring the additional benefit of enabling the EPAs to review issues related to Fund’s governance, including decision-making by management and the Board, a subject that is difficult for the staff to tackle.

  • Alternatively, an EPA unit within the Fund staff could prepare all reports. This unit could have a dedicated team to lead EPAs with qualified staff that have broad country experience and understanding of Fund policies. Having a separate unit would increase accountability, provide for greater quality control, and allow clear identification of the budgetary costs of EPAs.

  • While either of these options could result in more independent and candid reports, a number of issues would need to be considered. As regards the IEO, such a proposal would fundamentally alter its scope and nature and refocus its work program. The IEO is enjoined by its terms of reference from interfering with the Fund’s operational activities, including programs, and hence cannot address fully a key objective of the EPAs, namely to present a strategy for future Fund engagement. It is not clear that such a strategic redirection of the IEO away from broader policy and cross-country issues is desirable, or that this is an area where the IEO has a comparative advantage (see Box 5). Use of external consultants would also suffer from several drawbacks. It may be difficult to find teams with the necessary expertise and understanding of the Fund’s operations and policies. Fund staff would need to be closely involved in briefing external authors in preparing a response, and later in devising a forward-looking strategy taking the external recommendations into account. In addition, confidentiality issues would need to be addressed. Establishing an EPA unit would avoid some of the above shortcomings but would not address fully the issue of independence as the unit would not be outside of staff and management. All of these proposals would be likely to increase costs considerably, as the number of EPAs to be undertaken during 2006-07 is still large.

    The IEO’s Evaluation and Staff EPA of Jordan’s LTPE

    On November 21, 2005, the Executive Board discussed “Evaluation of IMF Support to Jordan, 1989-2004” prepared by the IEO and the EPA report for Jordan (PIN No. 06/02).

    Directors expressed broad agreement with the assessments of both reports. They generally found that they covered a wide range of similar issues and considered that the findings of the studies were consistent with each other. In addition to its findings specific to Jordan, the IEO report also drew a number of broader lessons for the Fund, with which staff and management largely concurred, although both the IEO and staff cautioned about drawing general lessons from a single country case study.

    Some Directors questioned whether the concurrent preparation of two reports with significant overlap on Jordan’s LTPE had been necessary and thought the analysis in these reports could have been more differentiated. The Director of the IEO noted that, now that EPAs are well established, individual country evaluations may not be the best use of its limited resources, with possible exception of exceptional access cases. In future, evaluation of the EPA policy itself, as opposed to assessing LTPE in individual country cases, may be a more appropriate use of the IEO’s resources, with a better focus on issues where more general lessons can be drawn.

27. A more radical approach would be to terminate the requirement for EPAs. By end-February 2006, EPAs have been prepared for about 75 percent of the LTPE members. Reports have not generally found pervasive problems of inadequate program design and conditionality, which were the primary concerns of the IEO report. Moreover, the Fund’s engagement with LICs, which constitute the majority of LTPE members, is expected to be long-term and a frequent re-examination of the strategy is costly and subject to diminishing returns. Although these arguments are not without merit, in the staffs view there is value in stepping back and reevaluating periodically the Fund’s strategy in longer-term program relationships, even when these appear well justified. For LICs in particular, the achievement of durable growth and external sustainability remain key macroeoconomic challenges, and domestic policies have not always allowed these objectives to be achieved without recurring debt relief efforts by the international community.

28. A third approach, which the staff recommends on balance, is to try to sustain the main substance of the policy, while allocating resources where the value of EPAs is highest. While the requirement to conduct EPAs for all LTPE members would be maintained, the regime for EPAs would be flexible. Area departments would choose whether to prepare a separate report, or to merge the EPA with Article IV or program review reports depending on circumstances. This approach would allow resources to be used more effectively, focusing them on cases where a reassessment of policies is most in need.*

B. Definition of LTPE and Procedural changes

29. Assuming that the Board endorses continuation of EPAs, the staff recommends a number of definitional and procedural changes to improve the focus and effectiveness of the reports. There has been considerable discussion about whether the definition identifies the right members as having an LTPE. There is a concern that the LTPE label confers a stigma, and this discourages in particular members seeking precautionary arrangements. Questions have also been raised about the more demanding test requiring an EPA for PRGF than GRA users. Finally, the content of the reports could be improved and the EPAs could be more focused and concise documents.

Definitional issues

30. The staff recommends that time spent under a PSI and non-drawing precautionary arrangements be excluded from the definition of LTPE. At the time of the Board discussion on the new EPA policy, the main arguments in favor of counting precautionary arrangements towards LTPE were that under those arrangements members retained the right to make purchases (and had done so on occasion) putting the Fund’s resources and reputation at stake, and that precautionary arrangements reduce the Fund’s forward commitment capacity. However, most precautionary arrangements do not result in purchases.30 Most members with precautionary arrangements use them as a means to signal their commitment to a strong policy framework and a closer relationship with the Fund than is normal under Article IV surveillance. In addition, being defined as LTPE may discourage members seeking precautionary arrangements because it may reduce the positive signal conveyed by the relationship with the Fund. The staff proposes that precautionary arrangements be treated as outside the LTPE definition when no drawings are made. For members that ultimately draw upon their precautionary arrangement, the entire duration of the arrangement would be included in calculating LTPE. The proposed treatment of precautionary arrangements would parallel the exclusion of programs under the PSI (where signaling is a key purpose) from the definition of LTPE.

31. The current definition for LTPE is more demanding for PRGF users.31 While two three-year arrangements (SBAs or Extended Arrangements) would not qualify a GRA user as having LTPE, they would do so for a PRGF user. In addition, while the definition allows GRA users to graduate from LTPE, this is not the case for users of the Fund’s concessional resources. Two PRGF arrangements will qualify a member as having LTPE irrespective of the time elapsed after the first arrangement, and there is no mechanism for the graduation from LTPE once that definition has been met once. Furthermore, whereas only the months prior to cancellation are counted toward the definition of LTPE for GRA users, in the case of PRGF users, cancelled PRGF arrangements are counted toward the definition of LTPE.

32. To ensure even treatment, the staff proposes applying a single LTPE definition to all members. Members will qualify as having LTPE if they have received GRA or PRGF/ESAF resources or any blend of the two for seven out of the last ten years.32 Only time in program engagement prior to cancellation would be counted toward the LTPE definition.33

Content of reports

33. Selectivity in the coverage of issues should be the norm for separate EPA reports. While the operational guidelines directed EPA teams to address a broad set of issues, there is scope for the EPAs to be more focused and concise documents with greater analytical content. In this regard, teams should be encouraged to be more selective in choosing the key themes to be addressed in the report and put more emphasis on program design and the policy priorities for successor programs. To better enable the teams to achieve this, the EPAs should be able to benefit from views outside the Fund. On an experimental basis, teams could consider hiring an external consultant to bring a different perspective to the assessments and to further open EPAs to outside input and scrutiny. In addition, EPA team leaders should be able, if needed, to travel to the country for which the EPA is prepared, and consult with donors, outside experts, and country authorities, before the report is finalized, enhancing the interaction with these groups.34 Although this proposal would add to resource costs, these costs are expected to be more than counterbalanced by other cost-saving measures proposed in this section.35

Frequency of EPAs

34. The staff proposes that successive EPAs be separated by at least five years for members that continue to meet the LTPE definition. The aim should be to allow sufficient lapse of time between two EPAs to allow for the next assessment to provide a fresh perspective. Under current policy, EPAs would be required even annually for members with a series of one-year stand-by arrangements, or at three-yearly intervals for PRGF users. Such frequent reports are less likely to add major insights. The new rule would set a minimum interval. Also, as under existing guidance on the timing of EPAs, these should be prepared when an existing arrangement is substantially complete, and before negotiations begin on a successor arrangement.36 For countries that graduate from Fund-supported programs within this 5-year period, an EPA would not be required.

35. Any changes to the LTPE definition and the timing of EPAs should not affect EPAs that are under way. As a transitional measure, EPA teams that have done substantial work should proceed to finalize the reports and present them to the Board.

Other procedural issues

36. Area departments would have the flexibility to merge the EPA into Article IV or program review staff reports. Area departments would inform management of the reasons for choosing to prepare it as a separate report or as part of other reports. In the latter case, and since there is some overlap in the content of these reports, merging them would help to streamline the production and reduce the cost of EPA reports.*

37. The current procedures on the timing and Board consideration of EPAs are appropriate. As under the existing guidelines, the EPA should be discussed together with the Article IV consultation when the current arrangement is substantially complete or where this is not feasible with the last program review. If no successor program is contemplated, the EPA should be discussed with the first post-program Article IV consultation. This applies both to cases where the EPA is merged with the staff report for the relevant Board meeting or it is a separate document. In exceptional cases, a stand-alone EPA discussion could be considered.

38. The authorities’ reaction to the EPA should be included in the staff report, as under the existing guidelines. The staff report on the Article IV consultation or program review should continue to present accurately and fairly the authorities’ views on the EPA. However, where a stand-alone discussion of the EPA is envisaged, the authorities’ response should be included in the EPA report (preferably as an Annex). In addition, the authorities can provide a statement regarding the EPA staff report and Executive Board Assessment, which will be published together with all related documents.

39. Following the discussion with the authorities of the draft EPA (in cases where separate EPA reports are prepared), factual corrections and revisions could be made before the report is circulated to the Board. As under the current guidelines, the EPA should not be subject to negotiations with the authorities and, as noted above, the authorities’ views should be separately reported to the Board. However, where warranted, factual corrections and revisions regarding the content of EPAs could be made, subject to the usual review process. The experience thus far does not indicate undue influence or pressures from country authorities on staff to modify the message of EPAs. Any changes to the draft report would need to be flagged to the area department and PDR and be approved by management. After the report has been issued to the Board, and before publication of the EPA, any changes should adhere to the existing policy on corrections and deletions.

C. Resource Implications of the Proposals

40. Under present policies, 10-11 EPAs are expected to be carried out on an annual basis, requiring 5-6 staff years.37 This assumes that the number of LTPE members stabilizes to around 35-40, of which 30-35 are PRGF-eligible members. This figure takes into account an estimate of 2-6 PSI arrangements with LIC members that otherwise would qualify as LTPE members.

41. The staff’s proposed approach sets forth recommendations in a number of areas to make more efficient use of EPAs. Simple calculations indicate that the implementation of the key recommendations could result in savings on the order of 3% -4% staff years. Specifically:

  • The proposal to exclude time spent under non-drawing precautionary arrangements from the LTPE definition is assumed to result in 4 members not qualifying as LTPE over the medium-term, implying savings of about % staff year.

  • The recommendation to reduce the frequency of EPAs, separating successive EPAs by at least five years for members that continue to meet the LTPE definition is projected to lower the number of EPAs by about 35-40 percent each year (savings of about 2 staff years).

  • Unifying the LTPE definition is not expected to result in material savings. This is because this proposal would affect the timing that PRGF-eligible members qualify as LTPE but would not change the number of LTPE members at the steady state.

  • Allowing flexibility to merge EPAs with Article IV or last program review reports could result in considerable further savings. The unit cost of EPAs is expected to decline in view of the streamlined focus of the assessments and the synergies from allowing EPAs to be carried out by the mission team. Estimating the precise amount is difficult, but this proposal could cut 1% staff years annually, or about half of the remaining costs if one were to assume that the majority of the EPAs with LICs would be merged with Article IV or last program review reports.

  • In view of the low number, possibly as low as three separate EPAs, expected to be carried out each year, the cost of the proposal to allow an additional mission trip of the EPA-team leader (US$10,000 for the EPA team leaders that decide to do so) is expected to be relatively small.

V. Issues for Discussion

42. Directors may wish to discuss the following issues:

On policy:

  • Directors may wish to express views about the value added of the EPA reports produced to date and their contribution to the Fund’s ongoing relations with LTPE members.

  • Directors may wish to comment on the three options regarding the EPA regime. Do Directors agree with the staff’s proposal to continue conducting EPAs, notwithstanding their higher-than-anticipated cost, but to revise the current framework to support more selective and effective reports?

  • Do Directors agree that the objectives of the policy would best be achieved by allowing more flexibility to reallocate resources to the EPAs where their value is highest? Do Directors agree with merging the EPAs with the relevant Board documents (preferably the Article IV report) when Fund-supported programs have been successful in meeting their objectives?

  • Do Directors consider that the focus on countries with 7-out-of-10 years of Fund-supported programs is appropriate? Do Directors agree with the proposal to eliminate differential treatment of PRGF and GRA users and the unification of the definition of LTPE?

  • Do Directors agree that time spent under PSI will not count towards LTPE? Do Directors also agree to exclude precautionary arrangements that remain undrawn from the definition of LTPE?

  • Do Directors agree that a minimum interval of five years between preparation of successive EPAs should be introduced?

On operational issues (as an input for management’s revised guidance to staff):

  • Do Directors agree that the value of the reports could be enhanced by better selectivity and focus on a few critical issues while streamlining the size of the reports?

  • Do Directors consider that, budget constraints permitting, staff should expand efforts to reach out and consult with donors, outside experts, and country authorities, including if needed through hiring an external consultant and an additional trip of the EPA-team leader to the country before the report is largely finalized?

  • Do Directors consider appropriate the current guidelines whereby an EPA is discussed together with the Article IV report when an arrangement is substantially completed or with the last program review?

EPA Findings on LTPE Members

43. This annex summarizes key findings and lessons learned from the EPA reports. Beyond the question of how well EPAs have achieved their aims, the substantive findings of the EPAs are an important source of lessons for future Fund strategy towards members with LTPE. This annex identifies consistent themes emerging from the EPAs, in particular on the policy challenges faced by LTPE members, policy design and implementation, and program outcomes. It is important to keep in mind that the focus and coverage vary substantially across EPAs, and not all issues are addressed in every EPA.

Policy Challenges

44. All countries faced multiple policy challenges, both structural and macroeconomic, and progress needed to be made on many different fronts. On average, the EPAs identified challenges in about five separate macroeconomic and eight structural areas. There was no significant difference in the breadth of challenges faced by GRA and PRGF users. Among GRA users, challenges faced by non-transition and transition economies showed an equally broad dispersion across multiple policy areas.

45. Macroeconomic policy challenges cover a broad range, with some key issues mentioned in almost every EPA (Figure A1). Inflation, fiscal consolidation, debt sustainability (both public and external), and current account imbalances were among the most commonly reported challenges. Differences between GRA and PRGF users remained small. Balance sheet vulnerabilities and capital account issues were less frequently noted, and as expected were largely concentrated among GRA users.

Figure A1.
Figure A1.

Key Macroeconomic and Structural Challenges

(In percent of total respondents)

Citation: Policy Papers 2006, 008; 10.5089/9781498332903.007.A001

Source: Staff Analysis of 32 EPAs prepared through end-August 2005.

46. Structural challenges showed more variation across EPAs, but some areas were nevertheless very common. Public enterprise reform, financial sector issues, governance problems and the achievement or maintenance of high sustainable growth were common concerns. While pension reform and labor market rigidities were mentioned in many EPAs of GRA users, these issues were almost completely absent from the EPAs of PRGF users. On the other hand, public expenditure management and civil service reform were more common among PRGF users.

Program Design

47. The policy design was generally consistent with the identified challenges. Programs almost always included fiscal adjustment, some reserve buildup, monetary tightening, and, in a significant number of cases, a depreciation of the exchange rate. Problems in fiscal policy design were reported in about a third of the cases, where EPAs criticized either the size, composition, or timing of fiscal adjustment. Weaknesses in the design of either monetary or exchange rate policy were less frequently reported (in about one fifth of the cases), and related to weak control over credit growth, premature abolition of direct instruments of monetary control, insufficient adjustment or flexibility of the real exchange rate, or inadequate reserves build-up.

48. Problems with macroeconomic projections were noted in less than a third of the cases.38 Unrealistic projections were reported with equal frequency for PRGF and GRA users. In some cases, worsening debt sustainability had been masked by overoptimistic GDP growth projections.

49. Less than half of the reviewed reports covered the adjustment-financing mix (though the treatment was somewhat more extensive in GRA than in PRGF users), while even fewer reports discussed the appropriateness of access decisions.39 Where this issue was discussed, the mix between adjustment and financing, and access levels were considered broadly appropriate. The apparent lack of attention to these issues is consistent with the composition of LTPE cases: cases where the Fund’s role consisted primarily of short-term balance of payments financing remained the exception. In most cases the focus was on achieving and maintaining macroeconomic stability and advancing structural reforms.

50. EPAs were generally more critical about the design and focus of structural reform programs. Problems were identified more often in GRA users, particularly fiscal structural and public enterprise reforms. Banking sector reform was generally deemed to have been appropriately designed. Almost half of the EPAs found conditionality initially inappropriate, and in about half of these cases, the problems persisted during the LTPE period. In some cases the issue was the focus of structural conditionality, in others the quantity—both too much and too little. For PRGF users, EPAs more often found excessive conditionality.

Program Implementation

51. Uneven program implementation was a major problem among many countries with LTPE. Very good and very poor implementation were the exceptions, and the majority of the countries had mixed performance. In only a few countries policies were carried out and reviews were completed as envisaged while in about one third of the EPAs (more than half of the programs involved for these countries) reviews were left uncompleted. In most cases, periods of relatively good and poor implementation alternated.

52. Implementation problems, where they occurred, arose with equal frequency in quantitative and structural conditionality. With respect to quantitative conditionality, slippages were common in meeting fiscal targets. Implementation problems with structural reforms were more frequent in areas that required legislative approval or that had significant distributional implications, particularly pension and civil service reform, and labor market reform. Financial sector and trade reforms were better implemented. The similarity in meeting quantitative and structural conditionality may indicate that conditionality is often interrelated and structural conditions are critical for meeting quantitative targets. It also suggests that the lack of ownership commonly associated with poor implementation usually affects both structural and quantitative areas.

53. EPAs stress the pivotal importance of ownership for program implementation. Strong ownership was found to be the most important reasons for successful policy implementation (Figure A2).40 As a corollary, when policies were not well implemented, almost all EPAs identified a lack of ownership as a reason. The lack of technical expertise (in PRGF countries), coordination problems with the World Bank (mostly in PRGF countries), and unfavorable external shocks were also noted as playing an important role in cases of weak program implementation.

Figure A2.
Figure A2.

Reasons for Successful and Unsuccessful Policies

(In percent of total respondents)

Citation: Policy Papers 2006, 008; 10.5089/9781498332903.007.A001

Source: Staff Analysis of 32 EPAs prepared through end-August 2005.

Outcomes

54. Progress in addressing the identified policy challenges remained mixed. Half of the EPAs found that macroeconomic problems have been substantially reduced. Good progress was found in reducing inflation, correcting exchange rate misalignments, and eliminating trade restrictions. Structural challenges, on the other hand, were judged as substantially reduced in only a few EPAs, while most EPAs found them somewhat reduced. Addressing problems related to pension obligations, labor market rigidities, governance, civil service and the wage bill has proven to be much more elusive and more than half of EPAs found that these problems have not been significantly reduced.

55. Progress has been better in addressing macroeconomic problems than structural reforms. In part, this is likely to reflect the less complex nature of these problems and the higher degree of discretion that the government has in implementing macroeconomic policies. Half of the EPAs found that macroeconomic problems have been substantially reduced. Structural challenges, on the other hand, were mostly found to be only somewhat reduced, with only a few reports indicating a substantial reduction. Significant differences also exist across specific reform areas. Good progress has usually been achieved in reducing inflation, exchange rate policies, and eliminating trade restrictions. Reducing problems related to pension obligations, labor market rigidities, governance, civil service and the wage bill, on the other hand, has proven much more elusive. More than half of EPAs found that these problems have not been much reduced, despite LTPE.

56. These results suggest that “classic” macroeconomic stabilization is not the primary concern in LTPE. Stabilization has often been successful, and generally does not require LTPE. However, progress in difficult structural reforms has been much slower. As a result, most cases of LTPE are countries that face deep-seated structural problems, such as LICs and transition economies. While it is important to try to improve on the often disappointing record in particular through better program implementation, it is equally important that programs include realistic time frames.

Questionnaire used by Staff in Evaluating EPAs41

Part I: EPA Requirements

For questions in Sections IA-IE, except where otherwise indicated, the answer options are: not discussed, not well, average, or very well discussed.

I. A. Reasons for LTPE

How well have the reasons for LTPE been provided and developed?

Has the completion point under HIPC Initiative been reached? (yes/no)

I. B. Macroeconomic and structural challenges

How well are the macroeconomic challenges explained?

How well are the structural challenges outlined?

How well are the macroeconomic projections examined?

I. C. Assessment of program involvement

How well is program design discussed?

How well is the adjustment/financing mix discussed?

How well is the adequacy of program design to address risks discussed?

How well is the Fund’s response to problems in program implementation discussed?

I. D. What worked, what did not

How well are problems in program implementation discussed?

How well is the appropriateness of conditionality discussed?

How well are the reasons for successful policies evaluated?

How well are the reasons for unsuccessful policies explained?

How well are the reasons for successful conditionality explained?

How well are the reasons for unsuccessful conditionality explained?

I. E. Rationale and medium-term strategy for continued Fund program involvement

How well is the rationale for continued Fund program involvement discussed?

How well are particular requirements for continued Fund program involvement outlined?

How well is the exit strategy developed?

How well are required medium-term policies outlined?

I.F. Overall impression and reaction

Answer options (except where otherwise indicated): substantial shortcomings, broadly adequate, or very good.

Overall what is your view of the quality of this report?

What is your view of the quality of: covering reasons for LTPE, challenges, what worked, and future involvement?

Analytical content?

Distilling lessons learned?

Presentation?

I.G. Organization, lessons, Board reaction

Is the report organized: Along thematic lines? Chronologically? Concentration on a few core problems? (answer options: yes/no)

What are the main lessons provided? (open question)

What was the reaction of the Board to the report? from Summing Up following Board meeting (not mentioned, low quality highlighted, average, high quality highlighted; plus open question).

Part II: Content of EPAs

II. A. Reasons for LTPE

Answer options: not discussed, no problem, some problems or major problems.

article image

II. B. Macroeconomic and structural challenges

Answer options: not discussed, challenges not much reduced, somewhat reduced, or not an issue anymore; also a yes/no question asked whether the problem had been present in the first half of the LTPE period.

article image

II. C. Assessment of program involvement

For each of the policies, the following questions were asked: Were the policies part of the program? (yes/no); were they part of conditionality? (yes/no); were there problems in program design? (answer options: not discussed, no/few problems, some problems, major problems); were there implementation problems? (answer options: not discussed, no/few problems, some problems, major problems).

article image

II.D. Assessment of program involvement

Answer options (except where otherwise indicated): not discussed, yes, no

Have the macroeconomic problems been substantially reduced? (not discussed, little reduced, somewhat reduced, substantially reduced)

Have the structural problems been substantially reduced? (not discussed, little reduced, somewhat reduced, substantially reduced)

Were the macroeconomic projections realistic?

Has the mix between adjustment and financing been appropriate?

Has the structural conditionality been broadly appropriate from the beginning?

Did conditionality focus on the right areas?

Was the conditionality appropriately set?

Has quantitative conditionality been broadly met?

Has the structural conditionality been broadly met?

Have all reviews in all programs been completed?

II.E. What worked, what did not and program appropriateness

Answer options (except where otherwise indicated): yes, no

article image

Was access appropriate? (not discussed, yes, no)

Was the type of Fund facility used appropriate? (not discussed, yes, no)

Was the program too accommodative of slippages? (not discussed, too little accommodation, broadly adequate, too accommodative)

II. F. Conditionality and the Guidelines

Answer options: yes, no

Do EPAs say whether programs appropriately identified (or missed) the crucial structural issues the country needed to address? Were structural conditions aligned to the objectives of the programs?

Do EPAs assess whether structural conditionality adhered to the macro-criticality principle? Were the numbers of conditions excessive? And if so, were the programs covering too many areas or setting conditions at too high levels of detail? Do EPAs identify a change in the number of conditions after the 2002 Conditionality Guidelines?

Do we have EPAs that consider that less streamlined conditionality would have led to better outcomes?

Do EPAs say whether programs provided appropriate policy space, including realistic timetables, to improve ownership and implementation?

Do EPAs identify a tendency to substitute conditionality, especially prior actions, for ownership? Are we selective enough in cases of poor ownership to say no to continuation of an existing program or to a new program?

II.G. Program design

What do the EPAs say about the appropriateness of programmed external adjustment? Was it in line with debt sustainability considerations?

How accurate were program projections on growth, inflation, current account, and debt/GDP ratios?

What were the determining factors in cases where successful disinflation took place (at least a 20 percentage point decline)? What was the exchange rate regime?

II.H. Rationale and medium-term strategy for continued Fund program involvement

Answer options (except where otherwise indicated): yes, no

Does the report recommend continued Fund program involvement? (not discussed, yes, no) What is the rationale for continued Fund program involvement?

article image

What is required for continued Fund program involvement?

article image

II. I. Authorities’ reaction (from accompanying staff report)

Answer options: not indicated; considerable disagreement, major challenges to staff analysis; broad agreement; very friendly reception

What was the authorities’ overall reaction?

Specific reactions and views?

Answer options: yes, no

article image

Survey to Country Authorities

Process

1. Do you consider that the purpose and the process of the EPA had been adequately explained, in particular the rationale for the EPA and its implications for the country’s future relations with the Fund?

2. Was the difference between the roles of the EPA team leader and the regular area department mission sufficiently clear?

3. Did you have sufficient opportunity to express your views during the discussion with the EPA team leader?

4. Were those views adequately represented, either in the EPA or in the staff report?

5. If a successor program was sought, did the need to prepare an EPA disrupt or delay the negotiations on the successor program?

6. Was the EPA published? If not, why?

EPA Analysis

7. Was the description of past programs and policies broadly accurate?

8. Do you consider that the EPA drew the right conclusions from the analysis?

Impact of the EPA on Future Policies and Programs

9. Do you feel that valuable lessons were learned from the EPA?

10. Did the EPA provide significant additional insights beyond what had already been raised by Fund staff in the context of Article IV or program discussions?

11. If there was a successor program, were the recommendations of the EPA included in the successor program?

12. If there was no successor program, did the EPA affect policies?

13. Overall, how do you view the requirement to have an EPA following long-term program engagement?

Comments and Suggestions

14. Please provide any observations or suggestions that you may have regarding the process and the usefulness of the EPA, including on how to improve it.

Survey to EPA Team Leaders42

Timing and Resources

1. Were you the mission chief at the time you were appointed team leader of the EPA? If yes, how long had you been mission chief prior to being appointed EPA team leader?

2. How much time elapsed from your appointment as team leader until the team was staffed and the main work started?

3. How much time elapsed from the beginning of the main work until the EPA was issued to the Board?

4. What is your estimate of the time that you spent on preparing this EPA ?

5. What is your estimate of total staff years spent by the rest of your team on preparing this EPA?

6. In your view, if there were not enough time and resources devoted to the EPA, what were the reasons? Please specify.

7. From your experience, what would you consider the 2-3 main elements for achieving the most efficient use of time and resources?

8. In hindsight, how could the resource cost of your EPA have been most effectively reduced without compromising the quality?: (a) less extensive review process, (b) narrower focus of EPAs, (c) include more team members who are or had been on the mission team, (d) smaller EPA team, (e) eliminate visit to the country, (f) other, please specify.

Breadth of Consultation and Inputs for EPAs

9. In preparing the report, how useful were discussions with: (a) current mission chief, (b) previous mission chief, (c) area dept front office, (d) other Fund departments, (e) current authorities, (f) previous authorities, (g) private sector in the country, (h) civil society in the country, (i) donors, (j) World Bank and other Multilateral Development Banks, (k) capital markets, (l) other, please specify.

10. How useful was the discussion of the final EPA with the current authorities?

11. How useful was the discussion of the final EPA with the previous authorities?

12. Do you think it would have been useful to have discussions with the authorities before finalizing the report?

Reasons for LTPE

13. What types of arrangements were covered by the EPA?

14. Did you find that LTPE was the result of: (a) deep seated problems, (b) inappropriate program design, (c) weak program implementation, (d) exogenous shocks, (e) program for signaling, (f) other, please specify.

15. Were program objectives well articulated in program documents?

16. Have original program objectives been achieved?

17. What was the impact of the LTPE on building technical capacity?

18. What was the impact of the LTPE on the strength of domestic institutions?

19. What was the impact of the LTPE on domestic policy ownership?

20. To what extent did collaboration with the WB help achieve program objectives?

21. If collaboration problems with the WB existed, please explain (text)?

22. Rationale for continued Fund program involvement?: (a) further reforms are needed due to deep seated problems, (b) further reforms are needed due to recent shocks, (c) because previous program not fully implemented, (d) for signaling purposes, (e) to maintain macro stability, (f) other reasons please specify.

23. Rationale against continued Fund involvement? (a) no since past program goals have been achieved, (b) no, poor program implementation and track record needed, (c) no, past Fund programs negatively affected institutional capacity or policy ownership. (d) other reasons, please specify.

Freshness of Perspective

24. At the time of the EPA were you in the same area department?

25. Was the team composition conducive to taking a fresh look?

26. Would you have preferred (more/just right/less) economists from the current mission team?

27. To what extent did the following try to change the key message: (a) current mission chief, (b) previous mission chief, (c) area department front office, (d) PDR, (e) other departments, (f) management, (g) authorities, (h) Executive Directors.

28. If so, did you alter the message?

Impact of EPAs on Design of Successor Programs

29. Does the current process allow for independent assessment?

30. Does the current process discourage criticism of policies pursued?

31. Did you perceive pressure to align the EPA message with other documents? If yes, which ones?

32. What was the timing of EPAs relative to negotiations of the successor program?

33. How consistent was the successor program with the main recommendations of the EPA?

34. What were the most important recommendations included in the new program?

35. What were the most important recommendations not included in the new program?

36. How different would the program have been if the EPA had not been carried out?

General Comments

37. Would the impact have been greater if the start of negotiations had been delayed to after the EPA was finalized?

38. In what area, if any, was there a large difference between the successor program and the EPA?

39. How would you characterize the Board discussion of the EPA?

40. Please provide any other comments you may have on the EPA exercise.

Survey to Mission Chiefs43

General

1. Were you the EPA team leader?

2. Did you negotiate the program following the EPA?

3. What types of arrangements were covered by the EPA?

4. Was the EPA team leader from the area department of the EPA country?

Value Added

5. Did the EPA bring a fresh perspective to the understanding of the Fund’s long-term program engagement with the country?

6. How useful were the recommendations in shaping the Fund’s future relationship with the country?

7. If the recommendations were not useful, please explain.

8. Did you agree with the key recommendations?

9. What was the value added of the EPA with respect to policy recommendations compared to the previous Article IV consultation?

10. In your view, were the time and resources devoted to the EPA appropriate relative to the value added of the report?

11. Do you think the current process: (a) allows for an independent and candid assessment? (b) discourages criticism of policies pursued? (c) provides strong incentives for criticism of policies pursued?

12. Please provide other comments on value added of the EPA, including suggestions for improvements.

Consultation

13. In your view, did the EPA team consult sufficiently with the mission team?

14. How well were the views of the mission team reflected in the EPA?

15. Please provide other comments on consultation during the EPA, including suggestions for improving the consultation process.

Authorities

16. Were the authorities clear about the rationale for the EPA and its implications for them in their future relations with the Fund?

17. Was there any confusion on the part of the authorities with respect to the roles of the EPA team leader and the mission chief?

18. How important did the authorities consider the lessons of the EPA for their engagement with the Fund?

19. Please provide other comments on the authorities and the EPA including suggestions for improving the interaction with the authorities in future EPAs.

Subsequent Fund Engagement

20. How important were the findings of the EPA in shaping subsequent Fund engagement?

21. What was the timing of the EPA relative to the negotiations of a successor program, if any?

22. Did the timing of the EPA allow for its main results to be used as an input to negotiations on a new program?

23. Did you perceive that there were pressures to conform to the recommendations of the EPA, even in cases where the mission team may have had an alternative view?

24. How consistent was the successor program with the main recommendations of the EPA?

25. What were the most important recommendations of the EPA that were included in the new program?

26. What were the most important recommendations of the EPA that were not included in the new program and why?

27. Please provide other comments on the EPA and subsequent Fund engagement with the country, including suggestions for improvement.

Country Selection / Other

28. Please provide any general comments or recommendations regarding the choice of countries for which EPAs should be done, and the timing and frequency of the exercise.

29. Please provide any other general comments you may have.

1

“Independent Evaluation Office Report: Evaluation of Prolonged Use of IMF Resources,” September 25, 2002 (“IEO report”), and “Conclusions of the Task Force on Prolonged Use of Fund Resources,” February 4, 2003. (“Task Force report”).

2

The terminology of longer-term program engagement, which includes prolonged access to Fund resources in the context of precautionary arrangements, was introduced in the guidance note on implementing the expost assessments (see “Operational Guidance for Assessments of Countries with Longer-Term Program Engagement,” August 20, 2003).

3

“Public Information Notice: EVIF Concludes Discussion on Prolonged Use of Fund Resources,” April 09, 2003.

4

The IEO observed that the Fund’s institutional culture, including political influences on the Fund’s decision making, could have contributed to insufficient attention to political feasibility of programs, and at times to support of programs that had low probability of success.

5

Directors, however, recognized that LTPE may be beneficial in transition economies and emerging markets countries with institution-building issues (“Report on the Incidence of a Longer-Term Program Engagement,” February 25, 2004).

6

The IEO report also noted that the Fund’s expanded role and prolonged engagement with low income countries could blur the division of labor with the World Bank.

7

These problems have been particularly prevalent in LICs and transition economies. Recently, the Board reiterated its support of the Fund’s role in low-income countries. See “Public Information Notice: IMF Executive Board Reviews the Role of the Fund in Low-Income Member Countries,” September 30, 2004.

8

Countries switching between GRA and PRGF or ESAF resources are considered as having LTPE if either of these criteria is met. For example, a country which has undergone two ESAF/PRGFs followed by a SB A would be considered as having LTPE even if it has not yet spent seven of the past ten years in Fund Arrangements. See “Report on the Incidence of a Longer-Term Program Engagement,” February 25, 2004.

9

See “Public Information Notice: IMF Executive Board Discusses Policy Support and Signaling in Low-Income Countries,” October 14, 2005.

10

See “Public Information Notice: IMF Concludes Discussion on Prolonged Use of Fund Resources,” April 09, 2003.

11

Recent Board papers and decisions include: “Review of the 2002 Conditionality Guidelines”, March 3, 2005; “Review of the 2002 Conditi onality Guidelines - Selected Issues,” March 4, 2005; “Public Information Notice : IMF Executive Board Discusses Review of the Conditi onality Guidelines,” April 15, 2002; and “The Design of IMF-Supported Programs; Atish Ghosh et. al; Part I: Overview; IMF Occasional Paper 241; September 20, 2005; “Review of PRGF Program Design — Overview,” August 8, 2005.

12

All but two of the 29 members that have reached the HIPC decision point are considered as having LTPE; and one more member already has a second PRGF arrangement in place.

13

As expected from the large share of LICs, the regional distribution is heavily tilted towards countries in Sub-Saharan Africa.45 percent of LTPE cases are members in the AFR region. The other regions (Europe, Middle-East and Central Asia, Asia, and Western Hempisphere) each have between 12 and 16 percent of the cases.

14

Four additional EPAs were completed during September-December 2005 (Jordan, Mongolia, Pakistan, and Ukraine). These were not included here because of the time demands of the survey process.

15

Annex I provides a detailed discussion of the characteristics of members with LTPE.

16

Including in the EPAs on Bulgaria, Ethiopia, Honduras, and Kyrgyz Republic.

17

Including in the cases of Guinea-Bissau, FYR of Macedonia, and Peru.

18

Mixed performance was common for the countries under review with periods of relatively good and poor implementation. In only a few countries were policies carried out and reviews completed as envisaged. In about one third of the countries reviews were left uncompleted.

19

Most EPAs suggested key elements that should be considered in subsequent arrangements, including increased ownership (called for in half of the EPAs), and a change in emphasis of conditionality under a new arrangement. One quarter of EPAs indicated that no change from the current strategy was required.

20

See footnote 11 for references to these reviews.

21

EPA team leaders indicateci that conflicting assignments by mission team members assigned to the EPA, as well as the inability to visit the country before the report was essentially completed were a problem compromising the quality of the reports. On the latter, the prohibition of discussion with authorities prior to sending the report to management had been introduced to shield the staff from external pressure. However, pressure from the authorities does not appear to have been an issue.

22

Discussion of the appropriateness of the type of Fund facility remained also relatively infrequent and—when discussed—the existing facility was almost always deemed appropriate. The sole exception was a case where staff felt there was a premature move to a PRGF arrangement, instead of EPCA, for a post-conflict country.

23

In most cases, country authorities considered that their comments and views were fairly and adequately represented in staff documents. In a few cases, they felt that the discussion of their views was incomplete and in one case, a member expressed its preference for the formal response to be included in the EPA report rather than the accompanying staff report. Such a response is not directly excluded under the guidelines.

24

This point was also raised by EPA-team leaders, a number of whom suggested that the EPAs be assigned to the IEO.

25

EPAs typically argue for future Fund program involvement until structural reforms, especially in core areas of Fund expertise, are sufficiently advanced. In some cases, EU accession was seen as a framework that could replace Fund programs in anchoring macroeconomic policies.

26

In fact, policies under successor arrangements—where such arrangements have been approved—have been mostly in line with the requirements for continued Fund support laid out in EPAs.

27

Task Force report.

28

There was considerable variation in time spent, ranging from 0.2 to 1.1 staff years.

29

See paragraphs 40-41 for further discussion on resource costs.

30

During 1980-2005 period, drawings were eventually made in 17 percent of arrangements that were precautionary on approval.

*

This staff proposal was not endorsed by the Executive Board. While being mindful of the need to contain the cost of the overall EPA process, most Directors considered that EPAs should be a stand-alone document, prepared by an interdepartmental team led by a mission chief from a department other than the home area department.

31

In its report, the IEO defined prolonged users as countries having spent at least 7 years out of a 10-year period under Fund-supported programs under the GRA and under the concessional facilities. The IEO report selected a seven-year threshold “so that any country with just two three-year arrangements (extended arrangements or ESAF/PRGF) in a decade would be classified as ‘temporary’ user, not as a prolonged user,” although it indicated that “the definition could use different criteria for general and concessional resources” (IEO Report). The Task Force introduced a separate definition for users of concessional facilities, although it recognized that the Fund should be cautious in adding new requirements for these members in view of the expectation that they will be users of Fund resources over a sustained period. In practice the LTPE definition proved more demanding for PRGF-eligible members.

32

The uniform definition for both GRA and PRGF users would also be consistent with the standard used in the IEO report.

33

Applying this new definition would remove 16 members from the list of countries having LTPE.

34

Revised operational guidelines to staff will be issued later in 2006 to reflect the Board discussion.

35

The additional cost of this proposal is estimated at US$10,000 per visit.

36

During this interval, existing instruments will ensure the timely provision of periodic assessments of the effectiveness of Fund-supported programs.

*

This staff proposal was not endorsed by the Executive Board. While being mindful of the need to contain the cost of the overall EPA process, most Directors considered that EPAs should be a stand-alone document, prepared by an interdepartmental team led by a mission chief from a department other than the home area department.

37

Resource estimates after the initial bulge related to the first round of EPAs for current LTPE members is completed.

38

Ghosh et al (2005) and Occasional Paper No. 241 analyze projections underlying Fund-supported programs and find significant biases in medium-term projections, while short-term projections are more accurate.

39

Discussion of the appropriateness of the type of Fund facility remained also relatively infrequent and—when discussed—it was almost always deemed appropriate. The sole exception was a case where staff felt there was a premature move to a PRGF, instead of an EPCA, for a post-conflict country.

40

Nonetheless, even successful policy implementation may not eliminate the need for LTPE, as many economic problems are deep-seated and require a long-term effort to resolve them.

41

PDR staff evaluated the 32 reports completed by end-August 2005 using the questions in this Annex. In most cases, scales were established (some examples of options are provided in italics) to allow for aggregation of responses. Text boxes for comments were also available.

42

For most questions, scales were established to allow for aggregation of responses.

43

For most questions, scales were established to allow for aggregation of responses.

Same Series

Other IMF Content

Other Publishers

  • Collapse
  • Expand
Cover Policy Papers
Review of Ex Post Assessments and Issues Relating to the Policy on Longer-Term Program Engagement
Author:
International Monetary Fund
Volume/Issue:
Volume 2006: Issue 008
Publisher:
International Monetary Fund
ISBN:
9781498332903
ISSN:
2663-3493
Pages:
52
DOI:
https://doi.org/10.5089/9781498332903.007
  • Figure 1.
    View in gallery
    Figure 1.

    Members with LTPE and Members with Completed EPAs

  • Figure 2. Figure 2.
    View in gallery
    Figure 2.

    Analytical Content of Discussion

    (in percent of total EPA reports)

  • Figure 3.
    View in gallery
    Figure 3.

    Independence

    (In percent of total EPA reports)

  • Figure 4.
    View in gallery
    Figure 4.

    Impact of EPAs

    (In percent of total EPA reports)

  • Figure A1.
    View in gallery
    Figure A1.

    Key Macroeconomic and Structural Challenges

    (In percent of total respondents)

  • Figure A2.
    View in gallery
    Figure A2.

    Reasons for Successful and Unsuccessful Policies

    (In percent of total respondents)

Figure 1.

Members with LTPE and Members with Completed EPAs
Figure 2.

Analytical Content of Discussion

(in percent of total EPA reports)
Figure 2.

Analytical Content of Discussion

(in percent of total EPA reports)
Figure 3.

Independence

(In percent of total EPA reports)
Figure 4.

Impact of EPAs

(In percent of total EPA reports)
Figure A1.

Key Macroeconomic and Structural Challenges

(In percent of total respondents)
Figure A2.

Reasons for Successful and Unsuccessful Policies

(In percent of total respondents)