5 Planning Sustainable Levels of Public Investment
Author:
International Monetary Fund. Fiscal Affairs Dept.
Search for other papers by International Monetary Fund. Fiscal Affairs Dept. in
Current site
Google Scholar
Close

Abstract

This handbook is aimed at anyone who is involved in a Public Investment Management Assessment (PIMA) or who has a practical interest in public investment management. It is intended to be useful for country authorities, IMF staff, staff of other financial institutions and development organizations, and anyone who is interested in exploring different aspects of public investment management to understand how country systems are designed and how they work in practice.

Efficient public investment requires robust planning processes. Investment planning institutions must ensure that public investment is fiscally sustainable and effectively coordinated over time, across sectors, and across levels of government; that project proposals are based on stringent analysis; and that all possible financing and delivery modes are covered by the planning process.

Institution 1: Fiscal Targets and Rules

Does the government have fiscal institutions to support fiscal sustainability and to facilitate medium-term planning for public investment?

The purpose of Institution 1 is to gauge the presence of mechanisms that smooth total public investment spending across the economic cycle and promote long-term fiscal sustainability. Excessive volatility in investment spending undermines the efficiency of public investment. The assessment focuses on the existence of fiscal policies; it does not require stating a preference for any specific fiscal policy or the share of spending that public investment should occupy.

The three dimensions under this institution start with high-level objectives and become progressively more operational:

  • The first dimension asks whether there are long-term fiscal targets or limits to promote long-term debt sustainability. These targets or limits generally focus on an end point, without laying out the annual fiscal steps to get there.

  • The second dimension asks about fiscal rules that set limits to fiscal aggregates to achieve sustain-ability objectives. Fiscal rules help determine short-term annual fiscal aggregates, which should be consistent with the projections of the medium-term fiscal framework (MTFF).

  • The third dimension assesses whether there is an MTFF to align fiscal policy and budget preparation.

Dimension 1.a: Is there a target or limit for government to ensure debt sustainability?

Questionnaire

article image

Definition of Key Terms

article image

Institutional Design

The purpose of this dimension is to ascertain the existence of fiscal targets or limits to ensure debt sustainability. Several different targets or limits might contribute to ensuring debt sustainability. Possible targets include public debt/GDP, change in public debt/GDP, Net debt-creating flows/GDP, and overall deficit, excluding net interest payments/ GDP (that is, primary deficit).

  • A low score on this dimension indicates that there are no specific targets or limits to ensure debt sustainability. This may be because fiscal policy documents contain no targets or limits at all. Alternatively, there could be some targets or limits in fiscal policy documents, but these have no formal status or are changed frequently and do not contribute to long-term sustainability. For instance, if a limit only appears in a technical appendix to the budget and changes each year, it is not an actual constraint on medium- to long-term policies.

  • A medium score implies that there is at least one target or limit in place for central government. These would have a clear formal status to reflect the government’s commitment to debt sustain-ability and will typically be stable over time. However, the targets or limits do not need to constitute a legally binding fiscal rule (this will be discussed under Dimension 1.b).

  • For a high score, at least one target or limit should cover all or most of the general government. GFSM 2014 defines general government as comprising central government plus subnational governments (SNGs), and social security funds and not-for-profit institutions controlled by them. There are examples of countries that have encountered major fiscal crises caused by unsustainable borrowing practices by SNGs. Ideally, targets or limits to ensure debt sustainability should apply to all SNGs. This will usually be the case for targets and limits imposed by the central government on SNGs. In some countries, targets and limits may also be established separately by SNG legislative organs. To qualify for a high score on Dimension 1.a, most of the general government expenditure should be covered by at least one target or limit adopted by the central government and by SNGs at the level below central government.

Important Documents

article image

Effectiveness

The effectiveness of a target or limit to achieve debt sustainability must be judged by the actual developments in the debt position and debt-related risks. The detailed requirements will depend on the specific nature of the target or limit. A debt sustainability analysis will provide support for this assessment. It is also commonly discussed in IMF Article IV reports.

  • Low effectiveness implies that the target or limit has not contributed to an improved debt position. The mechanism is ineffective if, after three years, there is no significant change in the trend line that existed before imposition of the target or limit and the debt level remains outside the range considered sustainable. If the debt level has moved further from the target since it was established, the target is also ineffective.

  • Medium effectiveness implies that the debt target has contributed to moving debt closer to the target, but it is still not within the target range. For this score, the debt target should have contributed to closing at least half the gap between the initial debt level and the target. Alternatively, if the debt level fluctuates around the target, the target should have been met at least once the past three years.

  • High effectiveness means that the debt level is within the target or limit. The assessment is further strengthened if data for the past three years show that the debt levels have been within the target or limit the whole period, or if there has been a clear movement toward meeting the target. If the debt is outside the target or limit most years, but meets it in the most recent year, the reasons for this should be identified. Box 5.1 gives an example of how Ireland’s debt rule and other fiscal rules have been effective in reducing the debt over the past few years.

Useful Data Series

article image

Ireland Debt Rule and Other Fiscal Rules

Ireland’s fiscal policy is guided by the European Union framework, which includes a range of fiscal rules regarding the fiscal deficit, the structural deficit, spending growth, and debt levels. The rules are codified in the 2012 Fiscal Responsibility Act and include

  • A debt rule limiting debt to 60 percent of GDP, or if debt exceeds 60 percent of GDP, an annual pace of reduction of no less than 1/20th of the difference between the actual debt ratio and the 60 percent of GDP limit.

  • A budget balance rule, requiring a general government budget balance or surplus, which is tighter than the Maastricht limit of a deficit of 3 percent of GDP.

  • A medium-term objective of achieving a structural budget deficit of no greater than 0.5 percent of GDP or, if the structural deficit exceeds 0.5 percent of GDP, an annual reduction is required, the size of which depends on the cyclical position of the economy (0.6 percent of GDP for 2016 in Ireland).

  • An expenditure benchmark that limits the annual growth in general government primary expenditure to potential GDP growth, as assessed over a 10-year period (defined as the past 5 years, the current year, and a projection for the next 4 years).

The fiscal rules have contributed to a significant reduction in the level of public debt in Ireland in recent years. Debt-to-GDP was reduced from close to 120 percent in 2012 to slightly below the 60 percent thresh old in 2019 (Figure 5.1.1).

Figure 5.1.1.
Figure 5.1.1.

General Government Debt and Compliance With Debt Rule in Ireland

Sources: Government of Ireland 2019; Ireland PIMA 2017.

Dimension 1.b: Is fiscal policy guided by one or more permanent fiscal rules?

Questionnaire

article image

Definitions of Key Terms

article image

Institutional Design

This dimension assesses whether there are specific and permanent fiscal rules to guide fiscal policies. A fiscal rule is numerical and can be applied to different fiscal aggregates. Common examples of fiscal rules are the debt rule, budget balance rule (including overall balance, cyclically adjusted and structural, and over the business cycle), expenditure rule, and revenue rule. The choice of rules is generally based on the specific country circumstances rather than theoretical considerations. There is overlap between this dimension and Dimension 1.a, in cases when the fiscal rule is a debt rule.

Debt rules and budget balance rules are most common internationally. Table 5.1 gives an overview of different types of fiscal rules currently operational in different countries and the legal basis for these rules.

  • A low score on Dimension 1.b implies that there are no permanent fiscal rules. This could be because rules are completely lacking. It could also be that rules are formally in place, but that these have changed frequently and cannot be seen as permanent. Numerical rules differ from “procedural rules” that set standards on public financial management related to budget monitoring, reporting, and correction mechanisms. Procedural rules include budget timetables and deadlines as well as setting rules and enforcing expenditure ceilings at the ministry level. Only numerical rules qualify as fiscal rules for the purpose of this institution.

  • A medium score indicates that there is at least one fiscal rule for the central government. This rule is numerical, and it has a clear formal and legal basis in law, regulation, or international treaty. A policy statement in the budget documents, without a clear legal basis, does not constitute a fully fledged fiscal rule.1 However, if it has been consistently applied over several years, a policy-based rule may still be very effective.

  • A high score requires that there also is a comparable fiscal rule for a major part of general government, for instance, SNG. To be effective tools at the general government level, fiscal rules should cover all or most of the general government. Ideally, fiscal rules should apply to the whole general government sector, as is the case in the European Union (EU). To qualify for a high score on Dimension 1.b, most of the general government expenditure (75 percent or more) should be covered by fiscal rules adopted by the central government and SNGs at the level below central government. This is equivalent to the requirement for a high score on design under Dimension 1.a.

Table 5.1.

Fiscal Rules, by Legal Basis: International Practices

(Number of countries)

article image
Source: IMF Fiscal Rules Dataset 2017.

Important Documents

article image

Effectiveness

The effectiveness of fiscal rules depends on how consistently they are applied at the fiscal planning and budgeting stage and whether budget outturns are in line with the rules. The rules should be reflected in medium-term budget estimates and in annual approved budgets and the outturns will be documented in the annual fiscal execution reports. If fiscal rules are not observed during the budget process, they are not effective. In addition, if the rules are applied at the budgeting stage but fiscal forecasts are consistently optimistic, the fiscal rule is not effective. A fiscal rule without any correction mechanism will not be effective when projections fail to materialize. Because budget outturns are subject to significant uncertainty, the effectiveness thresholds for budget outturns should be somewhat higher than for budget allocations.

In most countries, the main fiscal rule can be a debt rule, a budget balance rule, or an expenditure rule. As operational rules, the assessment under this dimension should focus on the balance or the expenditure rule. The discussion of thresholds for effectiveness of fiscal rules will therefore focus on budget balance and expenditure rules.

  • Low effectiveness implies that the fiscal aggregates in the budget and budget outturns deviate from the limits established by the fiscal rule, without explicit justification by escape clauses. If the approved budget balance deviates from the fiscal rule or the final budget outturn deviates significantly, then effectiveness is low.

  • For medium effectiveness, the budget outturn deviates to some extent. Medium effectiveness may be attributable to escape clauses that are broad and undermine the credibility of the fiscal rule. Some fiscal rules give governments extensive powers to deviate from the fiscal rule when they deem this to be necessary, even in the absence of external shocks or emergency situations. Frequent application of escape clauses over several years may be a sign that the fiscal rule is not effectively disciplining fiscal policies. Mission teams must assess whether adjustments to or suspensions of fiscal rules in exceptional circumstances, such as the 2020/21 COVID-19 pandemic, have undermined the effectiveness of the rules.

  • High effectiveness indicates that fiscal rules are stringently followed when budgets are prepared and that budget outturns are in line with the rules. This will generally require that the rule includes a correction mechanism to capture deviations from the projections and adjust policies accordingly. Escape clauses may be used in special cases but should not undermine the credibility of the fiscal rule. In case of past deviations being corrected the effectiveness may be high. Box 5.2 illustrates how Bulgaria’s fiscal rule framework has helped ensure fiscal sustainability during 2002–17.

Useful Data Series

article image

Public Finance Act in Bulgaria

In Bulgaria, the Public Finance Act and the annual state budget define a clear set of fiscal rules and provide a good foundation for fiscal planning and sustainability (Table 5.2.1). The Public Finance Act complies with relevant European Union regulations and presents tighter constraints on some indicators.

Table 5.2.1.

Fiscal Rules in Bulgaria

article image
Sources: Authors’ analysis of the Bulgarian Public Finance Act.

The fiscal rules have been effective in guiding fiscal policies, and the rules are largely adhered to. As Figure 5.2.1 shows, from 2002 to 2017, deviations were limited to the 2009 financial crisis and a 2014 banking crisis.

Figure 5.2.1.
Figure 5.2.1.

Adherence to Fiscal Rules in Bulgaria

(Percent of GDP)

Sources: Bulgaria state budget and Public Finance Act; Eurostat; and IMF staff assessment.

Dimension 1.c: Is there a medium-term fiscal framework to align budget preparation with fiscal policy?

Questionnaire

article image

Definitions of Key Terms

article image

Institutional Design

This dimension assesses whether there is an MTFF to guide the preparation of the central government budget. The strength of the MTFF depends on its specificity and whether it represents a limit to the annual budget. This is confirmed most clearly in the legal framework addressing the MTFF and in the annual budget instructions. It may also be confirmed that a constraint was intended by the cabinet when approving the MTFF.

  • A low score indicates that there is no MTFF prepared and approved prior to budget preparation. In some countries there is no MTFF at all. In other countries an MTFF is presented in budget documents, but it is not approved before budget preparation. This MTFF is a result of the budget process rather than a framework to guide budget preparation.

  • A medium score indicates that an aggregate MTFF is prepared but does not differentiate current and capital spending or identify the fiscal space for new investment projects. An MTFF of this type does discipline the overall budget process but has limited direct impact on the level and composition of capital spending.

  • For a high score, the MTFF also indicates the allocation to capital versus current spending and how the capital spending envelope should be distributed between ongoing and new projects. This MTFF will provide clear guidance on the development of the capital budget.

Important Documents

article image

Effectiveness

If the MTFF is effective, the deficit and the capital budget allocation in the approved annual budget should be close to the approved MTFF. In many countries, macroeconomic conditions change quickly, and updated forecasts are often made late in the budget preparation process. Therefore, revenue and expenditure estimates may change compared with the MTFF estimates. However, fiscal policy relating to deficits as a percentage of GDP should not change significantly based on updated forecasts, unless there is a fiscal shock.

  • Low effectiveness indicates that the MTFF has little impact on the approved budget. In some countries, the budget process largely ignores the framework established by the MTFF. In other cases, the MTFF is the formal starting point for the budget deliberations, but the final budget is quite different. If the final capital budget is significantly higher or lower than the capital allocation in the MTFF, then effectiveness could be rated as low.

  • Medium effectiveness implies that the MTFF to some extent constrains the approved budget. If the final capital budget is somewhat higher or lower than the capital allocation in the MTFF, effectiveness could be rated as medium.

  • High effectiveness indicates that the budget document is consistent with the allocations that were indicated in the MTFF, in particular for capital spending. If the final capital budget is largely in line with the capital allocation in the MTFF, effectiveness could be rated as high. There may be adjustments related to new developments in the time period between the MTFF and the budget. If these are clearly explained and documented in the budget documents, a somewhat higher deviation might be consistent with high effectiveness. Box 5.3 shows how MTFF capital spending estimates in Estonia discipline subsequent investment spending.

Useful Data Series

article image

Medium-Term Fiscal Framework in Estonia

In Estonia, the annually prepared medium-term State Budget Strategy provides a medium-term fiscal framework (MTFF) that specifies planned current and capital spending. Capital spending is allocated by ministries, by main funding source, and by major programs and projects. The Budget Strategy describes decisions regarding ongoing and new investment projects, but there is no clear specification of budget allocations to existing and new capital projects in the published State Budget Strategy. This is specified in the underlying, detailed medium-term estimates provided by the ministries to the Ministry of Finance (Table 5.3.1).

Table 5.3.1.

Medium-Term Fiscal Framework Capital Spending Ceilings in Estonia, 2015–21

(Millions of euros)

article image
Sources: Estonia Ministry of Finance State Budget Strategies, 2015–18.

Institution 2: National and Sectoral Planning

Are investment allocation decisions based on sectoral and intersectoral strategies?

Public investment should be guided by strategies that set out the goals and objectives to be achieved through public investment spending and plans for how to realize these. The strategies should set out the direction and high-level ambition or aspirations for future public investment, informed by current gaps and trends (for example, population, technology, environmental) that would shape future infrastructure needs and demands. The plans should explain how public investment goals and objectives will be achieved through a broad portfolio of projects that complement each and prioritize individual major projects. These goals and objectives are nonfinancial in nature, but plans should be subject to broad constraints in terms of the economic viability of addressing the underlying infrastructure needs. Plans should not be expected to go into much detail about major projects and will of ten not include any information on specific smaller projects. Public investment spending contributes to the capital stock and is often driven by gaps in this stock, and thus plans should make some reference to the existing nonfinancial fixed assets.

Each dimension of this institution addresses a key aspect of the planning phase:

  • The first dimension captures whether national and sectoral public investment strategies and plans are prepared and how comprehensive they are. National and sectoral goals are achieved through the contribution of all projects once they are completed.

  • The second dimension highlights the importance of costing of public investment plans. The total cost of a plan should reflect the financing constraints—if there are no constraints, there are no priorities. Costing major projects conveys greater confidence that fiscal constraints applied to the plan, and the number of major projects identified in the plan, are realistic.

  • The third dimension reinforces the concept of nonfinancial project benefits—the value of an individual project, and its contribution to overall goals, can best be assessed through its contribution to outputs and outcomes.

Dimension 2.a: Does the government prepare national and sectoral strategies for public investment?

Questionnaire

article image

Definitions of Key Terms

article image
article image

Institutional Design

The purpose of this dimension is to clarify the existence and the contents of national strategies and plans, in particular, how they cover public investment. National strategies and plans provide the long-term foundation for public investments. The terms strategies and plans have different meanings, as defined previously. However, in practice these terms are often used interchangeably. When assessing national and sectoral planning documents, it is important to verify whether they include both strategies and plans or are limited to one of these aspects.

The strategies and plans can cover both current and capital spending. A strategy or plan that includes public investment but is not limited to public investment meets the definition of a public investment strategy or plan. Institution 7, Budget Comprehensiveness and Unity, gives higher scores when recurrent and capital budget preparation are closely coordinated. Plans are not different.

  • A low score indicates that plans are not prepared or that they provide limited coverage of future investment projects. Plans and strategies may be high level and focus more on broad policy directions than on specific projects. These plans do not provide concrete guidance on specific future investment projects, although they may be used to justify such projects. Some plans and strategies may mention a few projects that are under preparation but do not provide a concrete description of these projects or an overview of planned investments in the sector.

  • A medium score indicates that either national or sectoral plans are published and identify the planned major budget-funded projects in a sector. The plan and strategy need not address all possible future projects or be as detailed as the budget. To qualify for a medium score on Dimension 2.a, the plan and strategy should identify some of the 3–5 most important projects in most main infrastructure sectors. The definition of main sectors, and the government’s role in each sector, may vary between countries, and needs to be determined in each country. In many countries, main public infrastructure sectors will include transport (roads, railways, airports, and seaports), electricity, telecommunications, and water supply.

  • A high score indicates that most major projects are identified in published national and sectoral plans, and these plans should cover all financing sources, including external sources, public-private partnerships (PPPs), and public corporations (PCs). For a high score, the plan should identify most of the 3–5 most important projects in each main sector.

Important Documents

article image

Effectiveness

The effectiveness of the capital project planning process depends on the realism of the plans and the correspondence between the major project priorities found in the plans and the projects that are included in subsequent budgets. If projects turn out to be very different from those envisaged in the plan, or if the annual budget includes major projects that have not been covered by the planning process, planning is not very effective. A fragmented planning framework, with many partly overlapping but inconsistent planning documents, may also undermine effectiveness. The effectiveness assessment should be based on comparing the projects that have been included in the budgets for the years covered by a strategy.2

  • Low effectiveness implies that the plans have little impact on which projects are approved in the budget and implemented. This might be because the initial plans fail to identify projects, as discussed under institutional design. It might also be the case that plans do identify major projects, but these are largely ignored when budgets are prepared. If budgets for relevant years include few of the investment projects in national or sectoral plans, then effectiveness is low. If few of the projects approved in the budget have been identified in national or sectoral plans, then effectiveness is also low. In addition, if it is not possible to assess the consistency between the plans and the subsequent budgets, then effectiveness will be low.

  • Medium effectiveness indicates that many projects are identified in plans, but there may be significant changes in the projects that are selected for implementation. For medium effectiveness, some projects that are described in national or sectoral plans are subsequently approved for budget funding. If some projects approved in the budget have been identified in national or sectoral plans, then effectiveness is also medium.

  • High effectiveness indicates that there is close correspondence between the national and sectoral plans and subsequent budgets. Most major projects that are approved for implementation have been identified in relevant plans, and the project scope and design is consistent with the initial plans. For high effectiveness, most projects that are described in national or sectoral plans are approved for budget funding. If most projects approved in the budget have been identified in national or sectoral plans, effectiveness is also high. Box 5.4 describes the national planning process in Botswana, which facilitates high correspondence between planned and realized investment projects.

Plans are less effective if they have a narrower coverage than the capital budget. For example, if the budget includes externally funded projects but plans do not, then the plans are not effective. They are more effective if they have a wider coverage than the budget.

National and sectoral plans should be consistent with each other. Sectoral plans may be more detailed than the national plan. If there are significant differences between the sector portion of the national plan and separate sectoral plans in terms of coverage, priorities, or major projects, then, planning is not effective.

Botswana National Development Plan

Botswana has several strong planning institutions—as reflected in the National Development Plan (NDP). The plan is in its 11th iteration and is valid for six years. NDP 11 is based on the government’s 2036 strategic vision, includes six-year estimates per ministry and per project, and provides the basis for the total medium-term and annual budgets. Sector and subsector strategies are integrated within the NDP and are linked to broader policy goals (thematic policy areas). The NDP indicates the status of funding for each sector, program, and project, categorized as Committed Projects, Programmed Investment, Appraised Projects, and In Preparation. Table 5.4.1 summarizes the NDP allocations for one ministry.

Table 5.4.1.

NDP Allocations for the Ministry of Defence, Justice, and Security

article image
Source: Botswana National Development Plan 2017. Note: ICT = information and communications technology; MDJS = Ministry of Defence, Justice, and Security; NDP 11 = 11th National Development Plan; o/w = of which. The inclusion of multiyear projections in the NDP provides an effective platform for investment funding and implementation. The NDP framework includes project codes, names, and their total estimated costs, which provides a tangible bridge to the annual and multiyear budget and execution processes.

Useful Data Series

article image

Dimension 2.b: Are the government’s national and sectoral strategies or plans for public investment costed?

Questionnaire

article image

Definitions of Key Terms

article image

Institutional Design

This dimension assesses whether public investment plans are costed and subject to financial constraints. If the plans are to guide capital project priorities, they must be concrete and contain initial cost estimates for the major projects or groups of projects identified in the plans.

  • A low score indicates that there is no concrete cost information in strategies and plans. The documents may mention specific projects, but in the absence of any costing it is not possible to make any judgment about the likelihood that the projects will be implemented. The failure to provide cost estimates and indicate resource availability is a common weakness in many strategies and plans.

  • A medium score requires that the plans provide aggregate estimates for groups of projects. However, the plan does not indicate resource availability for these project groups.

  • For a high score, the plans should provide cost estimates for individual major investment projects and indicate the overall financial constraints of the investment, and the cost estimates should be consistent with these financial constraints. The financial constraint should be based on top-down considerations of future fiscal space. It is different from a bottom-up assessment of sectoral spending needs. The plans should provide a comparison between the overall spending needs and financial constraints and discuss strategies for closing any financing gaps.

Important Documents

article image

Effectiveness

The effectiveness of this dimension should be assessed by comparing the broad cost estimates in the plans to the final cost estimates in the subsequent budgets. Underestimation of project costs at the planning stage is a common weakness in many public investment management systems. This may be because of weak capacity and inadequate methodologies. Strategic misrepresentation is also common—low initial cost estimates are provided to reduce the risk that a project is rejected as too costly compared with the expected benefits.

  • Low effectiveness implies that there are significant differences between initial plan estimates and budgeted amounts. If cost estimates in the plan are systematically lower or higher than in the budget, the planning process is not effective. If broad cost estimates in plans are significantly higher than budgeted capital expenditure for the same period as the plan, effectiveness would be assessed as low. This assessment should be based on a sample of major projects, covering the most important public investment sectors. If it is not possible to compare cost estimates, the effectiveness of this dimension will be low.

  • Medium effectiveness implies that the differences between initial estimates in the plan and the budgets are more moderate. If broad cost estimates in plans are somewhat higher than budgeted capital expenditure for the same period as the plan, effectiveness should be considered to be medium.

  • High effectiveness implies that initial estimates are generally accurate. If broad cost estimates in plans are broadly in line with budgeted capital expenditure for the same period as the strategy, effectiveness should be considered as high. Box 5.5 describes the Irish National Development Plan 2018–2027, in which there is close correspondence between cost estimates in the plan and the outturns, and between the fiscal framework for the plan and subsequent budgets.

Useful Data Series

article image

Strategic Investment Planning in Ireland

Ireland prepared a National Development Plan (NDP) for 1989–1994 as the basis for a request for European Union financial support. The second National Development Plan, for 1994–2000, was largely a strategic investment plan. New plans have been prepared at regular intervals and the focus has shifted from European Union financing to national investment priorities. The current plan covers the period 2018–2027 (Table 5.5.1).

Table 5.5.1.

Strategic Investment Priorities in Ireland’s 2018–2027 National Development Plan

article image

The NDP 2018–2027 is managed by the Department of Finance. The plan is fully costed, and fully coordinated with the budget process. The NDP provides financing indications that are consistent with long-term fiscal projections, and these will be updated and revised during medium-term and annual budget considerations. Capital investment allocations are provided for a five-year period and will be rolled over annually. The NDP 2018–2027 combines direct investment by the Exchequer of €91 billion and state-owned sector investment of around €25 billion. This will increase public investment from about 3 percent to about 4 percent of gross national income during the period.

The NDP includes 10 strategic investment priorities that are aligned with the 10 strategic outcomes in the National Planning Framework and identifies 43 major investment projects or programs. There is a substantial contingency allocation. Annual progress reports show that the NDP has been effective in guiding public investment in Ireland.

Source: Government of Ireland 2018, 2020.

Dimension 2.c: Do sector strategies include measurable targets for the outputs and outcomes of investment projects?

Questionnaire

article image

Definition of Key Terms

article image

Institutional Design

The purpose of this dimension is to assess whether sector strategies identify measurable targets for outputs and outcomes of public investment. The sector strategy could be defined as part of a national strategy or in a separate sectoral strategy document. A strategy should identify multiple outputs and describe how they work in concert to achieve specific outcomes. Outcome targets should be aligned with the goals or objectives of the strategy. At least one measurable target must be provided for each major output and for each outcome in the strategy. The assessment must also cover the validity and reliability of the performance indicators.

  • A low score indicates that there are no specific output or outcome targets for investment in the sector strategy. There may be targets related to broad policy initiatives, but if these are not linked to public investment, then it is not possible to ascertain the results of the investments.

  • A medium score indicates that the sector strategies include targets for outputs of the investment projects. Outputs are generally easier to measure than outcomes and therefore easier to include in planning documents. It is reasonable to measure outputs annually.

  • A high score indicates that strategies provide targets for both outputs and outcomes. Outputs are the means for achieving an outcome. Knowledge of outputs is necessary to understand the realism of outcomes and to monitor the progress in achieving them. Inclusion of only outcome measures satisfies neither a medium nor a high score. Outcomes are more difficult to measure than outputs because there are other influences than government outputs on the problem or condition targeted by the government output. Because of this complication, outcomes are sometimes measured through surveys and infrequently, for instance, once every few years.

Important Documents

article image

Effectiveness

The effectiveness of measurable targets depends on whether and how they are used:

  • First, measurable outputs and targets can be used explicitly to analyze or justify proposed current or capital budget allocations, either by line ministries or by the Ministry of Finance (MoF).

  • Second, measurable outputs and targets can be used by a line ministry to manage a program. This must be demonstrated by monitoring reports prepared by line ministries or through a central monitoring system.

  • Third, outcomes can be measured through spending reviews, performance audits, evaluation reports, or other specialized studies. Program outcomes can be measured on a periodic (that is, not annual) or on a sample basis.

The share of major projects in which measurable targets are used for decision making, project management, and evaluation purposes is a key indicator for effectiveness:

  • Low effectiveness means that there is little evidence that output and outcome data are actively used. This is done in few major projects.

  • Medium effectiveness means that there is frequent but. There is documentation that performance data are used in some major projects.

  • High effectiveness indicates that there is systematic and extensive use of output and outcome information. This is actively used and documented in most major projects. Box 5.6 illustrates how outcome and output targets from Malaysia’s national planning framework are actively used in line ministries’ investment projec t proposals, and in prioritizing among these.

Useful Data Series

article image

National Planning in Malaysia

The Malaysian government publishes national strategies for public investment under its five-year development plan series, with the current plan, the 11th Malaysia plan, covering 2015–20 (Figure 5.6.1). Each sector produces its own strategies for public investment under various sectoral master plans and blueprints, albeit with a different time coverage. The 11th Malaysia plan includes high-level outcome targets and identifies so me major investments. The sectoral strategies include measurable targets for outputs and outcomes, and when ministries submit their budget requests, details on the targets and outcomes for each project are required.

Figure 5.6.1.
Figure 5.6.1.

Outcome and Output Targets for Infrastructure in the 11th Malaysia Plan

Sources: 11th Malaysia plan, annual budget documents.Note: GNI = gross national income; LGK/KV = Greater Kuala Lumpur/Klang Valley; RGT-2: re-gasification terminal.

National and sectoral plans have been effective in guiding the strategic selection of investment projects. Sectoral strategies are published and their formulation is broadly guided by the national strategy plan. On the basis of national and sectoral strategies, ministries prepared a master list of top priority projects for 2016–20, which was approved by the cabinet.

Institution 3: Coordination between Entities

Is there effective coordination of the investment plans of central and other government entities?

Institution 3 addresses coordination between plans at different government levels, that is, coordination between entities that have the right to independently allocate resources to public investment. Coordination in this context also includes how one entity—central government—provides financing to other entities.

In addition to the public investment plans themselves, the dimensions of this institution focus on fiscal relations between entities involved in the public investment:

  • First, how is capital spending by different government levels coordinated? SNGs are often as important, and sometimes more so, than central government in public investment planning, financing, and implementation. Coordination in this context means to ensure that priorities are consistent and individual projects complementary between the central government and SNGs.

  • Second, how does the central government provide funds to SNGs that may be used for public investment? The more predictable this flow of funds, the more realistic are funding constraints used in public investment plans. The focus here is on funds for which SNGs have discretion—in other words, SNGs independently decide the projects financed by these funds.

  • Third, how does the central government monitor contingent liabilities (CLs) related to other parts of the public sec tor? Central government is of ten called upon to bail out other entities, including SNGs and PCs, in the event of serious problems arising from projects. Central government has an interest in monitoring and minimizing such CLs.

Dimension 3.a: Is capital spending by SNGs coordinated with the central government?

Questionnaire

article image

Definitions of Key Terms

article image

Institutional Design

The intent of this dimension is to determine if there is a system to ensure that major projects are planned and selected by each level of government with awareness of planning and selection decisions at other government levels. The need for coordination and the direction of this coordination is dependent on assignment of responsibilities and on the funding arrangements for different capital projects. We can define five main funding arrangements for projects involving the central government and SNGs (Table 5.2). In principle, coordination with central government occurs automatically in the last three funding arrangements, so the need for formal discussions should relate mainly to the first two funding arrangements. However, this automatic coordination does not always occur in practice, so the assessment also needs to cover these forms of funding arrangements.

Table 5.2.

Funding Arrangements for Investment Projects Involving Subnational Governments

article image
Source: IMF staff.

The focus of this assessment will usually be on the level below central government, but in some countries, it may also be relevant to capture the next level of SNGs. If lower levels of SNGs are responsible for more than 25 percent of SNG public investment, the PIMA should also comment on practices at this level. The scoring should still be based on the first level below central government.

  • A low score on Dimension 3.a implies that there is no institutional requirement for systematic sharing and coordination of spending plans. The institutional requirement can be expressed through legislation, regulations, or intergovernmental agreements. The central government does not know which investments are being planned at the SNG level, and there is no dialog to ensure that SNG investment plans are consistent with central government investment plans. For instance, local roads might be built without knowledge of future developments in the national road network.

  • A medium score implies that there is an institutional requirement that central government and SNG projects are presented in a consolidated format. Central government receives the capital spending plans of SNGs and ensures that these are published alongside central government investments. Ideally, SNG and central government projec t s would be commingled and sorted by common project type and location. The legal requirement for sharing and presentation of SNG investment plans will need to be anchored in central government legislation, but there may be additional provisions in SNG legislation. If the provision is only in SNG legislation, this is not sufficient for a medium score.

  • A high score indicates that the coordination includes a system of formal discussions between the central government and SNGs. For the design assessment, formality means that the system of discussions, or system of coordination, is defined in legislation or in written procedures, such as budget instructions or ministry regulations.

Important Documents

article image

Effectiveness

The effectiveness assessment should focus on whether the arrangements that are put in place actually lead to effective coordination of public investments between the central government and the SNG. Is there a flow of information in this system that can be used for project planning and selection by each entity? Effective coordination should have impacts on the design and selection of specific investment projects. Identification of such impacts will support the assessment of effectiveness.

  • Low effectiveness indicates that the level and impact of coordination is low. This may be attributable to the absence of formal coordination mechanisms, which would also result in a low score on institutional design. Alternatively, there may be formal arrangements in place, but these are not working in practice. Central government may receive SNG investment plans, but these are not reviewed and the information in them is not used for coordination purposes. In some countries, discussions between the central government and SNGs occur after budget decisions have been made and serve as a vehicle for information rather than coordination. If effectiveness is low, there are few examples of SNG capital projects submitted to and effectively coordinated with the central government.

  • Medium effectiveness indicates that the formal coordination mechanisms are operational but do not cover all SNG public investment decisions. If there are few concrete examples of the impact of this coordination, it is only moderately effective. For medium effectiveness, some SNG capital projects are submitted to and effectively coordinated with the central government.

  • High effectiveness implies that information about investment plans is shared and forms the basis for active coordination, and that the impact of this is clearly documented in plans and budgets. In this case, most SNG capital investments are coordinated with central government. Some countries have annual formal consultations with SNGs early in the budget process and use these to decide investment priorities. If discussions do not have a clear formal basis, discussions must have occurred with regularity for the past three years to be equivalent to formal discussions.

Useful Data Series

article image

Many countries have comprehensive institutional arrangements for consultation between the central government and the SNGs on public investment. Box 5.7 provides an example of the institutional arrangements for Chile. Chile has not been subject to a PIMA, so there is no assessment of the effectiveness of these arrangements from a PIMA perspective. Box 5.8 describes similar mechanisms in Indonesia, where the PIMA indicated that these mechanisms were effective.

Subnational Government Investments in Chile

Decree N. 3876 of 2000 stipulates that is the responsibility of the Ministry of Interior and Public Security, through its Sub-secretariat for Regional and Administrative Development (SUBDERE), to coordinate regional public investment. This duty is the responsibility of the Co-ordination of Public Expenditure Unit (Coordinación Gasto Público, CORGAPU) with the support of intendentes (central government appointees in the regions), with the latter responsible for coordination with public services and ministries within their regions.

The intendente, together with the Regional Council, Ministry Regional Secretariats (SEREMIs), and of the public services in the region, develop the Anteproyecto Regional de Inversión (ARI), including a financial estimation of the projects to be undertaken, to accomplish their institutional objectives. The ARI needs to be sent to the Unit of Coordination of Public Expenditure via the online Chi leI ndica platform. Any discrepancies between the priorities of the intendente and the regional authorities of the sectoral ministries must be resolved in the evaluation phase of the ARI or in the budgetary discussions carried out in the national budget office (DIPRES).

The preparation of the ARI is strongly guided at the central government level. National ministries and services give their regional representatives specific guidelines as to which policies, programs, and institutional goals should be considered for the regional ARI. In parallel, to design the ARI, intendentes have to consider the nonbinding Regional Development Strategy (ERD), the presidential commitments, the Special Development Plans for Extreme Zones, and Community Development Plans. However, the official memorandum that provides instructions for the preparation of the ARI and the Public Programme for Regional Investment (Programa Público de Inversiones Regionales, PROPIR) specifically mentions that the intendente may consult the mayor when appropriate.

Once the ARI is approved at the central government level and by the DIPRES, national ministries and services inform regional representatives of the details of investments and programmes to be considered in the PROPIR. This information is also available on the ChileIndica online platform. This platform has to be updated regularly by regional governments, because it is the instrument used by the central government to monitor execution of investments. However, the information of this platform is not publicly available, which represents a significant restriction on the possibilities for monitoring by citizens and ensuring accountability.

Source: OECD 2017b.

Subnational Government Investments in Indonesia

Subnational governments (SNGs) engage in public investment in social and economic infrastructure. Following adoption in 2001 of the fiscal decentralization policy, SNGs became responsible for developing their own medium-term plans (RPJMDs) and an annual work plan (RKPD). These plans are not included in national plans or sectoral plans at the national level. The role of SNGs in providing infrastructure is substantial. In 2018, SNGs’ aggregate budget for public investment was approximately 130 percent of the central government budget for public investment.

Mechanisms exist to annually coordinate investment spending of SNGs with national priorities. This occurs through several channels and phases in which each SNG participates:

  • The development planning forum (Musrenbang), which consists of a series of meetings in each region between dinas (local governments’ working units) and Bappenas (the regional development agency) leading to a regional work plan, and meetings at the national level between regions, line ministries and Bappenas to coordinate national work priorities and strategies. These meetings are held between February and April.

  • Technical coordination meetings (Rakontek) held with line ministries normally twice yearly. Rakontek is also a series of meetings between line ministries and regions normally held to discuss the physical project plan under DAK (specific allocation fund) transfer.

  • The Ministry of Home Affairs reviews SNG overall budget plans (APBD) to ensure that the overall spending is aligned with regional and national priority programs at the end of the national budget process.

SNG annual plans are published on the websites of the Ministry of Home Affairs (under DG Regional Development), the Ministry of Finance (under DG Fiscal Balance), and the National Public Procurement Agency (LKPP).

Source: Indonesia PIMA 2017.

Dimension 3.b: Does the central government have a transparent, rule-based system for making capital transfers to subnational governments and for providing timely information on such transfers?

Questionnaire

article image

Definitions of Key Terms

article image

Institutional Design

This dimension assesses whether there are rules in place to ensure predictable funding of SNG capital investments. A rules-based system is a capital transfer plan whereby both the transfer pools (the amount of funds to be distributed) and the distribution formula are set in rules. In the absence of such rules it is very difficult to ensure that funding decisions are transparent and consistent across SNGs and from year to year, or even within the year. The transfers need not be reserved for capital spending purposes. There may be a general transfer plan that can be used for both current and capital spending.

A rules-based system for capital transfers should increase the predictability of these transfers. If the rule is used only to define the transfer pool, but the distribution of this amount among SNGs can be changed from year to year without reference to a rule, it does not give predictability to SNGs. Such a design does not constitute a rules-based system for the purpose of this dimension.

  • A low score indicates that there is no legal or regulatory framework that establishes a transparent and rules-based capital transfer system. This does not mean that there are no transfers to SNGs. However, such transfers may be ad hoc or may be a result of annual considerations, without a specific allocation formula to estimate the transfers. Sometimes, formulas or similar methodologies are assumed to be in place, but the details of these are not known outside the central government. If the methodology is not available and not transparent, the score on design is low.

  • A medium score implies that there is a transparent and rules-based transfer system, but that the transfer amounts are not known to SNGs well ahead of the budget year. If transfers from the central government are known less than six months before the fiscal year, it is difficult to fully reflect these in the SNGs’ own budget process.

  • A high score implies that the transfers are based on a transparent and rules-based transfer system and communicated at least six months before the budget year. This should give SNGs ample time to plan for the use of these funds in their internal budget processes.

To establish predictability, the rule for calculation amounts and notification dates must be based in regulation, law, or constitution. If the rule and dates are based on government policy, annual budget instructions, or a minister’s order, it can easily be changed. Notifcation can be done in published documents, for instance, a budget document, and in direct communications to the SNGs. Allocations that are included in internal central government documents, but not published or conveyed to SNGs, cannot be counted as notification in this regard.

Important Documents

article image

Effectiveness

The effectiveness of this dimension depends on whether and when SNGs are able to realistically predict the amounts and the timing of the financial resources they will have available for public investment. As noted under Institution 2, accurate financial constraints support effective planning. If total annual amounts transferred to SNGs during the fiscal year differ from the announced amounts, the rules are not effective, unless this is related to project implementation delays or similar reasons. If the financing constraint used in planning the SNG capital budget is incorrect, project prioritization may be inappropriate.

  • Effectiveness is low if there is no mechanism for predictable transfers or if actual transfers deviate substantially from the amounts determined by the rules-based system. If the assessment of institutional design concludes that there is no transparent, rules-based capital transfer system, it is difficult to envisage that these transfers can be predictable. Even if a transfer system is in place, it does not necessarily deliver the intended results. If aggregate actual capital transfers deviate significantly from amounts notified to SNGs, effectiveness is low.

  • Effectiveness is medium if aggregate actual capital transfers deviate somewhat from amounts notified to SNGs or actual notification is done less than six months before the fiscal year. Even if there are moderate deviations and delays, the transfer system still provides some degree of predictability to SNGs.

  • Effectiveness is high if both the amounts received and the timing is in line with expectations. This implies that the aggregate transfers are broadly in line with amounts notified to SNGs and actual notification is done at least six months before the fiscal year. High effectiveness will generally coincide with a high score on institutional design for the same dimension. Box 5.9 describes coordination of investments and transfer of funds to SNGs in Mali. The transfer mechanism in Mali was assessed to be highly effective in the 2018 PIMA, although the role of SNGs in public investment in Mali generally is limited.

For the system to be effective, SNGs should have access to promised funds when needed to finance public investment spending. There may be timing issues for actual transfers. SNGs often note that actual transfers do not occur early in the fiscal year. However, capital transfers need not be immediate cash transfers equal to the annual amount promised to them. Central government may have a system of monthly or quarterly allocations that could apply to capital transfers as well as to line ministries. Such mechanisms should be structured so they do not create obstacles for predictable and rational implementation of capital projects in SNGs. If unpredictable in-year transfers undermine the system, effectiveness could be assessed as medium.

Coordination of Investments and Transfer of Funds to SNGs in Mali

In Mali, SNG investment plans are defined through a coordinated process with the central government. The economic, social, and cultural development program (PDSEC) provides the reference for investment projects in territorial communities. This program has been implemented since 2015 through state-region contracts, concluded for five years. There are information and coordination mechanisms at different levels of government: a national steering committee for technical support to local authorities (NOCs); regional orientation, coordination, and monitoring of development actions (CROCSAD); local orientation, coordination, and monitoring of development actions (CLOCSAD); municipal orientation committees for coordination of development actions (CCOSAD).

The amount of transfers allocated to SNG investment expenditure is generally known at least six months before the start of the financial year. Since 2007, Mali has had a National Support Fund for Territorial Communities (FNACT), funded mainly by development partners and in part by budget allocations and state subsidies. This fund is administered by the National Agency for the Investment of Territorial Communities (ANICT). For the allocation of investment grants, the ANICT relies on an equalization formula that combines two series of criteria: (1) situation criteria (the needs of communities with regard to their territorial context), and (2) performance criteria (good tax administration for local authorities). Scoping letters sent by the ANICT communicate each year to the communities the transfers provided for in the central budget. In 2016, this letter was sent on June 9, more than six months before the next fiscal year.

In 2016, SNG capital expenditure reaches 27.4 billion CFAF, or 14 percent of the total expenditure of the territorial communities. Although this share has more than doubled in three years, it remains limited. The state budget constitutes the main source of funding for communities, while the communities’ own resources are weak. Since 2007, the revenue from local authorities’ own resources (in particular non-tax resources) has remained low. In addition, the state itself executes a significant part of the expenditure made at the local level through transfers to the deconcentrated services but also through projects executed by central government agencies, in particular multiregional projects. PDSEC contracts have been signed with five regions and two are in the process of being signed (Kidal and district of Bamako). All of the funding allocated to contracts was included in the 2017 finance law.

Source: Mali PIMA 2018.

Different fiscal years at central and local levels may impact effectiveness. The questionnaire assumes that SNG and central government fiscal years are the same. This is true in most countries, but not all. Effectiveness, with regard to date of notification, should be assessed relative to the start of the SNG fiscal year.

Useful Data Series

article image

Dimension 3.c: Are contingent liabilities arising from capital projects of subnational governments, public corporations, and public-private partnerships reported to the central government?

Questionnaire

article image

Definitions of Key Terms

article image

Institutional Design

This dimension is intended to assess the extent to which the MoF and the legislature are aware of CLs arising from major public investment projects, regardless of who finances or is responsible for the project. Public investment may entail significant CLs, and monitoring and reporting of these CLs is essential for fiscal risk management. These liabilities are defined during the project planning and implementation period, when it is decided who will be responsible for implementing a project and how it will be procured and financed.

  • A low score indicates that there is no legal requirement for systematic reporting of CLs from capital projects undertaken by SNGs, PCs, and PPPs. Partial information on such liabilities may be available on an ad hoc basis, but often this only happens after problems occur and CLs materialize.

  • A medium score indicates that there are legal requirements in place to ensure that these CLs are systematically reported. Reporting should include all four of the following attributes:

    • Coverage: CLs must be reported by at least two of the three categories of projects listed: SNG, PCs, and PPPs.

    • Frequency: reporting must occur at least once annually.

    • Mode: reporting can be made through financial statements, regularly scheduled and mandatory specialized reports, or regularly scheduled audits (if they are conducted annually and published within four months of the end of the reporting period).

    • Audience: the report must be readily available to the MoF, and they must be notified of the availability of the report.

  • A high score indicates that there are also legal requirements that these CLs also are comprehensively disclosed in budget documents. The total known size of CLs should be reported for each of the three categories (that is, SNGs, PCs, and PPPs). It is not required that presentation in the budget is made at the same level of detail as in the reports that are received. CL information need not be presented in a specific type of central government budget document. A variety of documents, with different purposes, makes up the budget document. These include the budget law, policy statements, analyses, reports, and other information. This criterion is satisfied if CLs are presented in any of these documents, including only for information.

Important Documents

article image

Effectiveness

The assessment of effectiveness should include to what extent reporting on and disclosure of CLs take place in practice. This assessment should be based on review of practices and budget documents for at least the past three years. Reporting on and disclosure of CLs are mutually reinforcing—the disclosure assists in the verification of the data and systematic consideration of fiscal risks. Because of this, the effectiveness assessment should reflect the combination of the two requirements. Ideally, CLs should be reported together in budget documentation, for example, as part of a consolidated fiscal risk statement. If reporting on CLs is scattered throughout the budget documents, then it may be difficult to understand their total magnitude and potential impact.

  • Low effectiveness indicates that there is little reporting on and disclosure of CLs by SNGs, PCs, and PPPs. This may be by design, which would be evident in a low score on institutional design, or because legal and regulatory provisions are not complied with. It could also be that there is some information on CLs in budget documents, but that this is so fragmented that it is not possible to understand the impacts of these liabilities. For this score, few CLs are reported to central government. Low effectiveness is also indicated when CLs are reported for none or one of the three categories.

  • Medium effectiveness indicates that some CLs are reported and disclosed, but that there are important gaps in the reporting. A partial picture of fiscal risks related to SNGs, PCs, and PPPs is better than nothing, but is clearly not adequate for consistent fiscal risks analysis and management. For this score, some CLs are reported to central government, or CLs are reported for two of three categories.

  • High effectiveness implies that there is comprehensive reporting of CLs and that these are fully disclosed in budget documents. Most CLs are reported to central government and disclosed, or CLs are reported and disclosed for three of three categories. If reporting and disclosure are fully in line with legal requirements, there should be high scores on both design and effectiveness. In some case, practices may be better than legally required and the effectiveness score higher than the design score. Box 5.10 describes disclosure of CLs in the Slovak Republic.

Useful Data Series

article image

Disclosure of Contingent Liabilities in Slovak Republic

In Slovak Republic, contingent liabilities arising from major public investment projects are presented in government budget documents as an annex, regardless of who is responsible for the projects. By law, the central government does not guarantee liabilities incurred by SNGs.

Every year, the central government approves a general government budget covering a period of three years. It contains the state budget and a summary of budgets of general government institutions (Table 5.10.1). Annexes to the budget include general government contingent liabilities (including contingent liabilities of SNGs and state-owned enterprises) and “implicit liabilities.” The projected “availability-based” payments for PPPs (discounted expected payments for the remaining duration of the concessions) are reported as “implicit liabilities” and are expressed as a percentage of GDP. In 2018, the implicit liabilities connected to PPPs were estimated at 3.3 percent of GDP.

Table 5.10.1.

Slovak Republic Contingent Liabilities

(Euros)

article image
Source: Slovak Republic Budget 2018, Appendix 4.

Institution 4: Project Appraisal

Are project proposals subject to systematic project appraisal?

This institution addresses whether and how appraisals are used to determine if a project is likely to meet its stated objectives and achieve a defined threshold of spending efficiency. If an appraised project meets this threshold, it becomes eligible for budget selection (which is discussed under Institution 10). Appraisals analyze project benefits and their contributions to achieving objectives set in national or sectoral plans. As such, appraisals are part of the planning phase of public investment management.

  • The first dimension of this institution seeks to account for the coverage of projects that are subject to appraisal, and the overall rigor and independence of these appraisals. Because appraisals require specialized skills and are labor intensive, major projects are typically subject to more extensive appraisals than others.

  • The second dimension addresses the extent to which appraisals are based on standard methodology and central support, to ensure a consistent basis for the analytical process. Different, or uncoordinated, methodologies do not guarantee adequate appraisal of all projects and make cross-project comparisons (within and between types of projects) difficult or impossible.

  • The third dimension addresses how risks are considered in project appraisals. Project risks increase uncertainty regarding final costs and benefits. Because a wide range of possible costs and benefits complicates appraisal, risk identification and possible mitigation measures to narrow this range are an essential part of appraisal. Information on risks, and management of them, should also feed into project implementation plans, addressed in Institution 14. Appraisal methodologies vary based on the type of project, its size, and its unique context and characteristics. There is no universally accepted definition of project appraisal, but appraisal is generally considered as one stage in the project cycle (Figure 5.1). The appraisal phase is often divided in two subphases.

Figure 5.1.
Figure 5.1.

Standard Project Phases

Source: IMF staff.

Appraisal will usually include analysis of a project’s benefits, costs, and risks, from a social, economic, and financial perspective. Table 5.3 provides an overview of elements that are commonly assessed in a comprehensive project appraisal. In advanced appraisals, the analysis of project benefits and impacts may be extensive.

Table 5.3.

Key Elements of Comprehensive Project Appraisal

article image
Source: Allen and Tandberg 2021.

This institution focuses on projects that should be appraised by central government institutions, including PPPs. In some countries, projects implemented by PCs are subject to the same appraisal process, but this is not a requirement under this institution. Central government oversight of PC investment projects is discussed under Dimension 5.c.

Dimension 4.a: Are major capital projects subject to rigorous technical, economic, and financial analysis?

Questionnaire

article image

Definitions of Key Terms

article image

Institutional Design

This dimension aims to determine the efficiency of projects in the pool from which projects are selected for inclusion in the budget. If all projects, and particularly the major projects, have been subject to systematic and consistent appraisal, the selection of projects is likely to maximize the economic returns on scarce public investment resources (see Institution 10). In some low-income countries, many projects are financed by development partners (DPs) and international financial institutions (IFIs) and may be subject to appraisal by them. This should not impact the assessment of the institutional design for this dimension, but may affect the assessment of effectiveness, as will be discussed further. For the assessment of the institutional design, appraisal of externally financed projects by DPs and IFIs is not relevant—what is relevant here is appraisal by government entities (possibly on the basis of studies presented by those external entities, plus additional due diligence by government).

  • A low score indicates that there is no legally mandated mechanism for systematic appraisal of major projects. Requirements for appraisal may be completely missing, or they may be insufficient to ensure a rigorous appraisal process for all major projects, or several significant sectors or financial sources are exempted.

  • A medium score indicates that there will be legal requirements for systematic and rigorous appraisal of all major projects. Some sectors or financing sources may be exempted from the general appraisal process. If this is the case, there should be other mechanisms in place to ensure that these projects are subject to the same level of appraisal. For instance, if public-private partnerships are exempted from general appraisal rules, the partnership’s legal framework should have rules prescribing that appraisal is done with the same rigor.

  • A high score implies that there is systematic appraisal of all major projects and that appraisal results are either published or subject to independent external review. Publication should, as a minimum, include a description of the overall project, key issues, conclusions stemming from the analysis, assumptions made in the analysis, and any recommendations for modifying the project proposal. If appraisal results are made subject to external review, then it is expected that the external reviewers are qualified for the task and will have access to full information about the project and the appraisal that has been done. External review can be done by national as well as international experts.

Important Documents

article image

Effectiveness

The effectiveness assessment for this dimension should focus on the share of major projects that are subject to rigorous project appraisal. Do appraisal documents provide systematic and credible assessments, in line with legal or regulatory requirements? Do the documents describe realistic expected benefits and costs of projects? Are lower-value projects systematically identified and discarded or returned for further development and possible resubmission?

The assessment must be based on a sample of project appraisal documents. There are often major discrepancies between the formal requirements for project appraisal and the actual practices. Many countries have comprehensive regulations and manuals for project appraisal, including requirements for cost-benefit analysis, risk analysis, and implementation plan review, but have limited capacity to comply with these. In the actual project documents, these requirements may be ignored or only addressed in a minimalistic fashion. Data and assumptions may be far from realistic, suffering from optimism bias. Project “appraisal” documents are sometimes limited to basic technical design documents. Verification of actual project documents is therefore essential for the analysis of this dimension.

  • Low effectiveness indicates that few major projects are subject to systematic and rigorous appraisal, or that there is evidence that low-value projects are included in the pipeline with no requirement for resubmission. This may be related to the absence of a clear regulatory framework. In this case, both the effectiveness assessment and the design assessment will tend to be low. Low effectiveness may also be related to failure to comply with formal requirements for project appraisal or lack of capacity to carry out the required appraisals. In these cases, the effectiveness will be assessed lower than the institutional design. In some countries, the appraisal process never leads to projects being rejected and all projects are included in the project pipeline, regardless of the appraisal results and regardless of whether they are ready for implementation.

  • Medium effectiveness implies that some major projects are systematically and rigorously appraised, and there is no evidence that low-value projects are included in the pipeline. Under a rigorous process, some project appraisals are expected to contain recommendations for change and even requests for project resubmissions after revision.

  • High effectiveness indicates that most projects are subject to systematic and rigorous appraisal, with clear linkages to the decision to include projects in the pipeline; further, appraisals are realistic. Good practice indicates that all projects should be appraised, but the depth of the appraisal would reflect the value and the risks of the project. Only the projects that are assessed to have high value and are ready for implementation are included in the pipeline, and other projects are rejected or returned for further development. If externally funded projects are excluded from national procedures, there should be rules to ensure that these are subject to at least the same degree of scrutiny as other projects. Box 5.11 describes the project appraisal framework in Colombia. Colombia has not had a PIMA but is generally recognized for a well-designed and effective public investment planning framework. Box 5.12 describes the project appraisal framework in Timor-Leste, which has been strengthened substantially in recent years.

Some projects may be appraised by IFIs and DPs, but this is unlikely to constitute an effective overall framework for appraisal of major projects. Some IFIs, including the major development banks, have rigorous appraisal methodologies. However, many other financial institutions, as well as bilateral DPs, have much less rigorous approaches. In addition, external financing plans are often more focused on establishing a financial mechanism than on the specific projects to be financed under this plan. In these cases, the DPs’ appraisal of the financial mechanism will not constitute appraisal of the specific investment projects. In some cases, IFIs and DPs carry out appraisal of specific investment projects only after a financial mechanism has been approved. Project appraisal by IFIs and DPs will focus on the priorities and preferences of the institution conducting the appraisal, which is not always fully aligned with the government’s priorities and preferences.

Project Appraisal in Colombia

Colombia has an advanced framework for preparation, appraisal, and selection of public investment projects. The process comprises clearly defined stages: identification, preparation, appraisal, pre-selection and selection/programming. Appraisal is comprehensive, including financial, economic, and social costs and benefits, based on detailed project documents. Figure 5.11.1 describes the main elements in the process.

Figure 5.11.1.
Figure 5.11.1.

Colombia Project Development and Appraisal Process

Source: Colombia Ministry of Planning 2016.Note: EBI File summarizes the information contained in the formulation of the investment projects for the administration, strengthening, and rates. It helps the administration and the general public so they know the basic information of each of the investment projects that they execute. BPIN = Bank for National Investment Programs and Projects; CBA = cost-benefit analysis; IRR = internal rate of return; NPV = net present value; SRR = statutory reserve requirement.

Standardized Appraisal Methodology in Timor-Leste

Following the 2016 PI MA, Tim or-Leste has taken several steps to strengthen public investment management, including a standardized methodology for project appraisal. This appraisal process assesses the projects against eight criteria and sorts them in three major groups green (important and ready), yellow (important but not (ready), and red (ready but less important) (Figure 5.12.1).

Figure 5.12.1.

Several guidelines are related to project development and appraisal:

  • Project Appraisal Guideline. MPS, August 2017

  • Feasibility Study Guideline. MPS, May 2018

  • Model Terms of Reference for Infrastructure Fund Projects. MPS, February 2018

  • Project Brief Standard Form. MPS, 2017

  • Fund Administration Manual. MPS, December 2018

  • Ex Post Evaluation Guide. MPS, 2019

Source: Government of Timor-Leste 2020b. Note: IRR = internal rate of return; SDP = Strategic Development Plan.

If the effectiveness assessment is to be influenced by appraisals done by IFIs and DPs, the basis for this should be comprehensively described and documented. The share of different IFIs’ and DPs’ financing of the capital budget should be presented, together with a summary description of the appraisals that are conducted and of the separate due diligence and review done by the government, if any. This should be based on a representative sample of appraisal documents. To give a higher score on effectiveness than on design, there should be documentation showing that a significant share of major projects is subject to systematic, rigorous and consistent appraisal, and that this appraisal is based on the government’s priorities and preferences. For medium effectiveness, many major projects should be covered. For a high score, most of the public investment program should be subject to systematic appraisal.

Useful Data Series

article image

Dimension 4.b: Is there a standard methodology and central support for the appraisal of projects?

Questionnaire

article image

Definitions of Key Terms

article image

Institutional Design

The aim of this dimension is to determine whether appraisals have a known quality, and lead to results that can be used to compare projects. It is closely connected with Dimension 4.a discussed earlier. The existence of a standard methodology does not imply that lesser projects are required to follow the same procedures and methodologies as major projects—the effort and degree of sophistication usually depend on the importance of the project.

  • A low score indicates that there is neither a standard methodology nor central support for appraisal of major investment projects. While the existence of appraisal methodologies is relevant for the assessment of Dimension 4.a, the dimension assesses the standardization or coordination of methodologies among the several appraisal entities.

  • A medium score indicates that there is either a standard methodology or central support for appraisal of major investment projects. Methodologies are standard if they are presented in manuals or instructions issued by a responsible agency and are applicable to most projects in the capital budget. Central support usually means that one unit advises the entities proposing most capital projects in the central government budget. A ministry of planning or public works might provide central support as effectively as MoF. In some cases, there may be more than one unit involved in providing central support, based on a common methodology. Support exists when advising on appraisal methodologies is part of a unit’s responsibilities designated in budget instructions, a minister’s order, regulation, or law. Support is most clearly indicated by the issuance of manuals and instructions and the conduct of training. If the unit only provides guidance ad hoc, this does not constitute central support

  • A high score indicates that there is both a standard methodology and central support for appraisal of major investment projects.

Important Documents

article image

Effectiveness

The effectiveness assessment should cover to what extent standard appraisal methodologies are applied to project appraisals and contribute to the quality of these. This indicator is a measure of the support to achieving the objectives of Dimension 4.a. The effectiveness assessment will therefore have a strong focus on compliance with formal regulations. Externally financed projects may be appraised according to IFI standards, but these should be consistent with national methodologies. The assessment should therefore cover all major public investments, regardless of financing source and financing modalities.

  • Low effectiveness implies that standard appraisal methodology has limited impact on project appraisal. This may be because the methodology is missing, which would imply that the institutional design score also is low. It could also be because the methodology is incomplete or inadequate, or because it is not followed in practice. If effectiveness is low, few major projects fully benefit from the standard methodology.

  • Medium effectiveness indicates that a standard methodology is in place, but not fully applied to all major projects. Some major projects fully benefit from the standard methodology.

  • High effectiveness indicates that the standard methodology is used actively to ensure high-quality project appraisals for most major projects. The methodology is consistently applied for most major projects.

PIMAs should comment on any external methodologies for appraisal of externally funded projects, and their consistency with the national standard appraisal methodologies. Box 5.13 provides an example from the Slovak Republic, which has well-designed national methodologies and central support for domestically financed major projects, while projects financed by EU funds follow EU methodologies. Box 5.14 describes the choice of discount rates for cost-benefit analysis, a key methodological issue for project appraisal.

Project Appraisal Guidelines and Central Support in Slovak Republic

In Slovak Republic, investment projects financed by budget resources have been subject to cost-benefit analysis (CBA) since 2017. CBAs are mandated for major projects above €40 million and for IT projects above €10 million. Central support for state budget-funded major projects is rendered by the MoF’s Value-for-Money Division, which validates the calculations set out in the CBA before any project is eligible to enter the procurement stage. The Value-for-Money Division was created to assist ministries that do not have the ability to do CBA and to control the quality of CBA for projects. Even though the MoF’s opinion on the CBA of these major projects is not binding, it has a strong influence on government decisions.

Projects financed by EU funds are a significant share of public investment (see Figure 5.13.1). These are subject to comprehensive technical, economic, and financial analysis determined by EU rules and procedures, and there is central support for project appraisal. For the 2007–13 programming period the European Commission published a guideline for CBA that provided a common framework for project appraisal. This was subsequently updated for the programming period 2014–20. The European Commission requires the use of CBA for all major infrastructure projects above €50 million, regardless of the beneficiary. In Slovak Republic, all EU-funded projects follow the EU rules and conduct the required CBA. The state expertise involvement in the appraisal of projects is mandatory. Since 2017, the results of the appraisals have been published on the website of the Ministry of Transport, Construction and Regional Development.

Figure 5.13.1.
Figure 5.13.1.

Public Investment, by Source of Financing

Source: Slovak Republic PIMA 2019.

Useful Data Series

article image

Discount Rates for Cost-Benefit Analysis

The choice of discount rates is a key element of the methodology for project appraisal, and should be clearly defined in national guidance materials. A higher discount rate will generally make investments less beneficial and will favor projects for which benefits materialize quickly compared with longer-term projects. There are two main approaches to selecting appropriate discount rates:

  • Consumption behavior (normative) approach (C). The social discount rate is computed from the Ramsey equation: r=d+n*g, where d is time preference, n is the elasticity of marginal utility of consumption, and g is the growth rate of the economy.

  • Opportunity cost of capital (descriptive) approach (O). The discount rate is estimated by observing actual interest rates, for instance market rates for low-risk government bonds.

Treatment of risk is also a critical issue. Some countries apply risk-free interest rates and handle risk by scenario analyses of benefit and cost streams. Others include a risk premium in the discount rate. Discount rates may differ (be reduced) over time. Table 5.14.1 summarizes the approach to determining the discount rate for public investment appraisal in different countries.

Table 5.14.1.

Discount Rates in Selected Countries

article image
Source: Mouter 2018.

Dimension 4.c: Are risks taken into account in conducting project appraisals?

Questionnaire

article image

Definitions of Key Terms

article image

Institutional Design

The aim of this dimension is to identify uncertainties relating to the project and to minimize them. Thus, the quality of project appraisals will be improved. Table 5.4 gives an overview of some common risks related to major public investment projects.

  • A low score indicates that there is no regulatory requirement for systematic risk assessment in appraisals of major projects. Neither is risk assessment included in standard appraisal methodologies if they exist. Some risks may be required to be identified in project documents, but there is no analysis and no indication of probability or possible impacts. There are no regulatory requirements for consultations with stakeholders to identify possible risks.

  • A medium score indicates that appraisals of major projects are required to include a section devoted to risk assessment but there are no requirements for risk mitigation plans.

  • A high score indicates that both risk analysis and concrete plans for risk mitigation are required for major projects. Mitigation measures can be included in the appraisal document, procurement plan, project implementation plan, financial plan, or operating plan. If they are in documents other than the project appraisal, mitigation measures must be clearly identified as such and linked to risks identified in the appraisal.

Table 5.4.

Common Risks in Major Public Investment Projects

article image
Source: Monteiro and others 2020.

Important Documents

article image

Effectiveness

The effectiveness of risk assessments and risk mitigation plans depends on how they are used. Risk mitigation plans are not effective if they are not implemented. This may be confirmed by looking at a sample of major projects approved in the budget roughly five years before and for which an appraisal was completed and a risk mitigation plan prepared. If risks materialized but the mitigation plan was not implemented in material aspects, the mitigation plan is not effective. The focus is on domestically financed major projects, because national authorities will have limited influence over risk management practices for externally financed projects.

  • Low effectiveness implies that there is no systematic use of risk assessment in project appraisal or project implementation. There may be no formal requirements, as indicated by a low score on institutional design, and no informal or ad hoc approaches to incorporating risk considerations. Alternatively, risk assessments may be prepared but are too limited for risk management purposes. Few major investment projects include stringent analysis of project risks.

  • Medium effectiveness indicates that risk assessment plans are prepared for some project appraisals. Some major investment projects include stringent analysis of project risks.

  • High effectiveness indicates that risk analysis is systematically included in project development and actively used in project implementation. Most major investment projects include stringent analysis of project risks and risk mitigation plans. Estimates of project costs and benefits disclose the risks related to key assumptions, and there are contingency reserves to absorb a reasonable share of these risks. Risk mitigation plans are defined before project implementation and are used to manage risks during project implementation. Active risk management helps ensure that projects are implemented on budget and on time, and deliver the expected benefits. Box 5.15 describes the approach to assessing project risks in Norway, where all major projects (above €100 million) undergo detailed risk analysis.

Useful Data Series

article image

Project Risks and Contingency Reserves in Norway

In Norway, cost estimates for major investment projects are subject to mandatory, standardized risk analysis. The project estimates are decomposed into major cost components, key risks are defined, and a cost probability distribution is assigned to each risk factor. Because cost of risk is assymetric, expected total cost (the P50 estimate) is higher than the base cost estimate. Figure 5.15.1 illustrates the cost estimates for different risk factors, sorted by importance.

Figure 5.15.1.
Figure 5.15.1.

Risk Analysis in Norwegian Investment Projects

(Deviations from base estimates)

On the basis of this analysis, project allocations include two contingency items. The first is called expected additions and reflects the difference between the base cost estimate and the expected total cost (P50). This reserve is managed by the project implementing agency. The second item is called uncertainty reserve. It reflects the difference between P50 and P85 (85 percent delivery confidence). This reserve is managed by the supervisory ministry and is only released to the implementing agency after a formal procedure, whereby the implementing agency must explain and justify why cost estimates have increased.

Source: Drevland, Austeng, and Torp 2005.

Institution 5: Alternative Infrastructure Financing

Is there a favorable climate for the private sector, PPPs, and PCs to finance infrastructure?

This institution is intended to assess the climate for the private sector, PPPs, and PCs to finance economic infrastructure. The division of public and private responsibilities for economic infrastructure is not the same in all countries. For example, internationally, electricity is commonly generated and distributed by both public and private entities. If private firms find a stable environment in which they can achieve a fair return on long-term investment, responsibilities for some infrastructure can shift from the public sector to the private sector, thus relieving pressure on public finances.

Three dimensions seek to measure this climate:

  • The first dimension assesses if private firms are free to invest and are reasonably assured that they can earn a fair return. Because most economic infrastructure requires investments that are large and long term (often 30 years or more), private investors look for a consistent regulatory history or safeguards that are likely to continue into the future.

  • The next two dimensions look at two important classes of financial institutions that straddle the public and private sectors, and for which poor institutions can shift financing risks back to the public sector.

  • PPPs are a potential source of private finance and expertise but they must be structured carefully to ensure a fair allocation of risk and reward.

  • PCs form a large part of the economy in some countries. PCs exist, in part, to free management from political interference. But weak PC governance sometimes results in financial losses for the governments that own them, thus undermining the goal of having PCs relieve financial pressures on public finances.

Dimension 5.a: Does the regulatory framework support competition in contestable markets for economic infrastructure (for example, power, water, telecoms, and transport)?

Questionnaire

article image

Definitions of Key Terms

article image
article image

Institutional Design

This dimension aims to determine whether major infrastructure market environments are competitive, fair, and predictable, and thus encourage long-term private investments. Private investors will generally be cautious about entering markets where there is extensive government involvement, particularly if there is uncertainty about the government view on the future development of this market and lack of transparency about specific government interventions. If the government wants private investment in the market, it must demonstrate its intention to develop a competitive market. Elimination of restrictions on market entry and establishment of independent regulators are important signals in this regard.

  • A low score indicates that the legal and regulatory framework does not support competition and there are few incentives for private sector engagement. Domestic monopolies dominate many markets, and there are no or few economic regulators.

  • A medium score implies that the picture is mixed—some markets are open for competition and economic regulators have been established. For this score, there must be an institutional framework that supports competition in a least two important submarkets and at least two regulators must be established. In many countries the markets for telecommunications, bus transport, and electricity generation are among the first to be opened up for competition.

  • A high score indicates that the legal and regulatory framework supports competition in major economic infrastructure markets. For this score, there must be clear support for competition in at least two of the four major markets (electricity, water, telecommunications, and transport). A major market is competitive if the majority of its subsidiary specialized markets are competitive. There is also a requirement that regulators are independent and well established, and that there are at least two independent regulators, each covering different major infrastructure markets.

Important Documents

article image

Effectiveness

Effectiveness should be gauged by the numbers and types of market participants and their market shares. If all qualified firms are legally allowed to enter a market but have not done so, the market is not competitive. This may be the result of many possible factors. The effectiveness should be downgraded if there is any evidence of cartelization or collusion in those markets. That evidence may be produced by competition authorities or institutionalized observatories, or it may be highlighted by international reputable assessments. The actual independence of regulators is also an important factor to consider.

  • Low effectiveness implies that there is little actual competition in major infrastructure markets. There may be clear legal impediments to market access, which would show up in the assessment of institutional design. In addition, many countries place barriers to entry that are not transparent. Regulators may not have been established long enough, or demonstrated consistency in their policy making, to give confidence to investors that they can achieve a return on a long-term investment. If a market has been legally competitive for at least three years but there have been no new entrants, then the market should not be considered competitive. If the market shares of private companies are low, then effectiveness would usually be assessed as low.

  • Medium effectiveness indicates that private companies have significant market shares in at least two major markets. Although state-owned companies still may have a dominant role, market entry by private companies indicates that they see the market as attractive and expect their market share to grow in the future. This situation is common in many markets for electricity generation. Box 5.16 describes the electricity distribution market in Ireland, where there are several private companies but where the main PC still controls about 50 percent of the market.

  • High effectiveness indicates that private companies have high market shares in at least two major markets. This will often be a mature competitive market with international competition, and state-owned companies will be expected to play a minor role. In many countries, telecommunication markets will have reached this stage.

If an infrastructure market is legally competitive but firms in the market consistently lose money or receive substantial subsidies, then competition may be distorted. This suggests government intervention in the market, for instance, pressure on regulators to keep prices charged to consumers low. This is not uncommon in electricity or transport markets. Such infrastructure markets should not be viewed as fully competitive. However, this assessment should be cross-checked with the number of market participants. In principle, a well-defined subsidy program can be combined with a competitive market. In practice, this is difficult to achieve.

Competition in Infrastructure Markets in Ireland

Ireland’s economic infrastructure markets are either open to both domestic and international competition or regulated monopolies. The former applies to electricity generation and retail distribution, gas supply, telephone, and internet, while the transmission networks (gas, water, and electricity), which are by their nature natural monopolies, are closely regulated. The electricity market covers both the Republic of Ireland and Northern Ireland, and is interconnected with Great Britain, with three private sector players and two additional public sector generators competing with the incumbent Electricity Supply Board. The provision of broadband from larger national operators to smaller regional wireless broadband service providers.

The main domestic regulator, the Commission for Regulation of Utilities is responsible for setting the prices for many of the regulated monopolies. The Commission is organizationally, financially, and managerially autonomous. It sets prices for the transmission networks according to a set of economic criteria that take account of both impact on the consumer and maintenance of investment-grade credit ratings for the related companies. In practice, this means scrutinizing closely the company investment plans and the projections for domestic demand that underlie them to avoid over- or under investment. Other regulators include the rail regulator and the Commission for Communications Regulation.

Table 5.16.1.

Electricity Distribution in Ireland in the Second Quarter of 2019

article image
Sources: Ireland PIMA 2017; Commission for Regulation of Utilities 2019.

Useful Data Series

article image

Dimension 5.b: Has the government published a strategy/policy for PPPs and a legal/regulatory framework that guides the preparation, selection, and management of PPP projects?

Questionnaire

article image

Definitions of Key Terms

article image

Institutional Design

The purpose of this dimension is to assess the strategic and regulatory framework for development, appraisal, approval, and oversight of PPPs. A PPP strategy/policy is expected to promote some form of PPP and establish a procedure for approving PPPs. A general policy stating that no PPPs will be considered does not constitute a PPP strategy/ policy for the purpose of this institution. However, a formal str