Making the Most of Public Investment in MENA and CCA Oil-Exporting Countries
Author: Ms. Maria A Albino, Ms. Svetlana Cerovic, Mr. Francesco Grigoli, Mr. Juan C Flores, Mr. Javier Kapsoli, Mr. Haonan Qu, Mr. Yahia Said, Mr. Bahrom Shukurov, Mr. Martin Sommer, and Mr. SeokHyun Yoon

Contributor Notes

Authors’ E-mail Address: malbinowar@imf.org; scerovic@imf.org; fgrigoli@imf.org; jflores@imf.org; jkapsoli@imf.org; hqu@imf.org; ysaid@imf.org; bshukurov@imf.org; msommer@imf.org; syoon@imf.org
Publication Date:
12 Nov 2014
eISBN:
9781498311397
Language:
English
Keywords:
SDN; investment; MCDOEs; investment efficiency; efficiency; public investment; levels in MCDOEs; average cost; appraisal cycle; investment selection; public investment management Index
Download PDF (1.2 MB)

Over the past decade, rising oil prices have translated into high levels of public investment in most MENA and CCA oil exporters. This has prompted questions about the efficiency of public investment in generating growth and closing infrastructure gaps, as well as concerns about fiscal vulnerabilities. When public investment is inefficient, higher levels of spending may simply lead to larger budget deficits, without sufficiency increasing the quantity or quality of public infrastructure in support of economic growth. This paper examines the efficiency of public investment in the MENA and CCA oil exporters using several techniques, including a novel application of the efficiency frontier analysis, estimates of unit investment costs, and assessments of public investment processes. The analysis confirms that these oil exporters have substantial room to improve public investment efficiency. Reforms in the public financial and investment management systems are needed to achieve this objective.

Abstract

Over the past decade, rising oil prices have translated into high levels of public investment in most MENA and CCA oil exporters. This has prompted questions about the efficiency of public investment in generating growth and closing infrastructure gaps, as well as concerns about fiscal vulnerabilities. When public investment is inefficient, higher levels of spending may simply lead to larger budget deficits, without sufficiency increasing the quantity or quality of public infrastructure in support of economic growth. This paper examines the efficiency of public investment in the MENA and CCA oil exporters using several techniques, including a novel application of the efficiency frontier analysis, estimates of unit investment costs, and assessments of public investment processes. The analysis confirms that these oil exporters have substantial room to improve public investment efficiency. Reforms in the public financial and investment management systems are needed to achieve this objective.

I. Introduction

Strong oil revenues provide an opportunity for policymakers in oil-exporting countries to accelerate growth and promote diversification through efficient public investments that yield high social dividends. In particular, public investments could enable the buildup of a stock of physical capital into assets that enhance economic growth and overall social welfare. Vandycke (2013) and Sachs and Warner (1995) highlight the role of oil resources in the economic development of countries such as Australia, Canada, Norway, and the United States, and point out how countries such as Indonesia and Malaysia have used oil revenue to finance investments and made a “big push” in industrial development. Public investment thus can play a prominent role in boosting growth and long-term development (IMF, 2014), as well as in enhancing non-oil growth prospects (Collier and others, 2009). The extent to which public investment contributes to this goal, however, depends on its efficiency (Gupta and others, 2014).

Measuring investment efficiency, though challenging, is crucial to assessing whether public investment is yielding its expected results. The analytical challenges in measuring investment efficiency, which are not exclusive to oil-exporting countries, include ambiguity in defining suitable inputs and outputs, sensitivity to outliers, limited availability of appropriate cross-country data, and difficulties in differentiating the role of efficiency from other relevant factors affecting the productivity of public investment. For example, certain techniques for measuring efficiency rely on an estimate of public capital stock that is imperfect.5 Given these challenges, most of the analytical results in this paper are presented by country groups.

Several approaches to measuring public investment efficiency have been used in previous studies. The approaches include estimates of how much of a dollar of public investment actually translates into public capital (Pritchett, 2000); estimates of the unit cost of large infrastructure projects in Europe and Central Asia, covering two MCDOEs (Alexeeva and others, 2011); and public sector performance indicators, with a focus on spending efficiency in new European Union member states and emerging markets (Afonso and others, 2010). The IMF and World Bank have also developed an index for assessing the quality of the public investment management (PIM) systems in 71 developing countries (Dabla-Norris and others, 2012), including eight oil exporters and three MCDOEs.

This Staff Discussion Note uses several approaches to gauge the efficiency of public investment in MCDOEs. Section II discusses why improving public investment efficiency is a priority. Section III assesses public investment efficiency, and compares the performance of MCDOEs with that of benchmark groups, namely, with commodity exporters with strong institutions in other regions, and with a global sample.6 The methods used for cross-country comparisons include a novel application of the efficiency frontier analysis, calculations of unit costs for large investment projects, and assessments of public investment processes. Section IV explores the determinants of public investment efficiency using cross-country regressions and reviews two successful country frameworks for public investment management. Section V concludes with main findings and policy implications.

II. Why Improving Public Investment Efficiency is a Priority

Improving public investment efficiency in MCDOEs is a priority in part because the countries continue to have substantial infrastructure needs. For example, MCDOEs’ stock of paved roads is considerably smaller than in advanced economies (Figure 1). This is the case even for the GCC countries, the best performers within the MCDOE group. The gap vis-à-vis advanced economies is less pronounced in some other areas such as electricity generation capacity, where the GCC countries (but not the other MCDOEs) are already on par with advanced countries (Figure 2).

Figure 1.
Figure 1.

Paved Roads, 2011

(Kilometers per million people)

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Source:World Development Indicators, World Bank.
Figure 2.
Figure 2.

Electricity Generation Capacity, 2011

(1,000 kilowatts per million people)

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Source:World Development Indicators, World Bank.

At the same time, most of the MCDOEs have maintained high levels of public investment over the past decade.7 Since the early 2000s, and amid rising oil prices, public investment has increased in MCDOEs on the back of sustained high oil revenues and with the adoption of major infrastructure programs. Even during the global financial crisis in 2008/09, capital spending remained high and was part of the fiscal stimulus provided by governments of several MCDOEs to mitigate the impact of the crisis.8 Notwithstanding the recovery from the crisis and the unwinding of the fiscal stimulus in some countries, public investment levels in MCDOEs (as a share of GDP) remain almost twice as high as in other emerging markets, excluding China (Figure 3).

Figure 3.
Figure 3.

Public Capital Expenditure, 2001–12 1/2/

(In percent of GDP)

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Source: IMF, World Economic Outlook (WEO) database; and IMF staff calculations.1/ Refers to general government and excludes oil-related investment of public companies.2/ Does not include non-oil investment financed by the sovereign wealth fund.3/ Includes oil exporting countries in other regions as classified by the WEO.4/ Emerging markets excluding China and the MCDOEs.

In some countries, high capital spending has contributed to weaker fiscal positions and exacerbated fiscal vulnerability to sudden declines in oil prices. With sizable increases in public investment, the fiscal position of most MCDOEs has deteriorated, with balances turning into deficits in eight out of 15 countries. The growing fiscal vulnerability in most countries to a potential decline in oil prices is evidenced by the increase in budget breakeven oil prices (Figure 4), as highlighted in the IMF’s Fall 2013 Regional Economic Outlook for the Middle East and Central Asia.

Figure 4.
Figure 4.

Break-even Fiscal Oil Price, 2009 and 2013

(In U.S. dollars)

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Source: IMF, Fall 2013 Regional Economic Outlook for the Middle East and Central Asia.

Improving efficiency of public investment could thus help MCDOEs improve infrastructure quality without a deterioration in the fiscal position. This is an especially high priority in MCDOEs dealing with a relatively short horizon of resource production (e.g., Bahrain, Azerbaijan, and Oman, which have less than a generation left before their natural resources are exhausted).

III. Assessing Public Investment Efficiency

This Staff Discussion Note explores three alternative measures of public investment efficiency. The first measure refers to the relative efficiency in translating monetary inputs into infrastructure outputs (efficiency frontier analysis) (Figure 5); the second measure is an estimated unit cost of large infrastructure projects; and the third measure focuses on the quality of the public investment management system. In contrast to the approach of Pritchett (2000), these definitions of efficiency provide a measurement of efficiency or inefficiency relative to the best-performing countries in the sample.

Figure 5.
Figure 5.

Efficiency Frontier Analysis

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Source: Prepared by IMF staff.

A. Efficiency Frontier Analysis

Efficiency frontiers can be used to assess the relative efficiency of the MCDOEs in translating public investment expenditures (inputs) into infrastructure (outputs). The efficiency scores reflect how far any given country is from the production possibility frontier determined by the best performers across the globe (see Annex I for more details on the methodology). To our knowledge, this method has not been employed previously for assessing public investment efficiency.

Assessments of the efficiency of public investment are carried out with a two inputs–one output model:

  • Inputs: The key input is real public capital stock per capita in purchasing power parity (PPP) terms, estimated based on the Penn World Table database.9 The auxiliary input is per capita GDP, which is used as a proxy for the contributions of the private sector to infrastructure services. The variables are averaged over the period 2006–12, given the availability of data used on the output side.

  • Output: The output measure for infrastructure is proxied with the infrastructure component of the Global Competitiveness Indicator (GCI) developed by the World Economic Forum (averaged over 2006–12, given data availability).10 This infrastructure indicator (Figure 6) is largely qualitative, as it covers several dimensions of the quality of infrastructure derived from perception surveys, including transport (roads, railroads, ports) and electricity supply. This means that the calculated scores are not a direct measure of efficiency and they primarily capture relative, rather than absolute, performance. That said, the infrastructure indicator also includes quantitative measures of infrastructure components, such as telecommunication network per 100 people (fixed lines and mobile subscriptions) and available airline seat kilometers per week, reducing concerns about the use of a survey-based index for cross-country comparison.

Figure 6.
Figure 6.

Infrastructure Quality and GDP per Capita

(Index)

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Sources: World Economic Forum; and IMF, World Economic Outlook (WEO) database.

The GCI index of infrastructure quality is highly correlated with the physical measures of infrastructure, further easing any concerns about this survey-based dataset. The correlation coefficient between the country scores for infrastructure quality and the quantity of paved roads per capita is 0.7 (or equivalently, R2 = 0.44) (Figure 7).11 Electricity generation capacity per capita and phone lines per capita are also strongly correlated with the GCI infrastructure quality index, with the correlation coefficients above 0.6 (R2 = 0.43) (Figure 8).

Figure 7.
Figure 7.

Infrastructure Quality and Paved Roads (2011 or latest available)

(Index)

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Sources: Global Competitiveness Indicators; World Bank, World Development Indicators; and IMF, World Economic Outlook (for per capita measures).
Figure 8.
Figure 8.

Infrastructure Quality and Electricity Generation (2011 or latest available)

(Index)

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Sources: Global Competitiveness Indicators; World Bank, World Development Indicators; and IMF, World Economic Outlook (for per capita measures).

Two types of nonparametric frontiers are used to estimate the efficiency of public investment. The first is the partial frontier method called Partial Free Disposal Hull (PFDH) (or order-m efficiency), and the second is the Data Envelopment Analysis (DEA). The main difference between them is that the PFDH frontier benchmarks against a group of peers and uses re-sampling to improve robustness, reducing sensitivity to outliers, measurement errors, and other issues that are common in cross-country data. The DEA is clearly sensitive to the presence of outliers and could overestimate inefficiencies by benchmarking a country relative to only a few best performers in the sample. For assessing technical efficiency, this Staff Discussion Note uses an output-oriented analysis; therefore, efficiency scores imply the proportional amount by which output could be increased while leaving input (public stock of capital) unchanged.12

PFDH and DEA Efficiency Scores

The estimated efficiency scores suggest that the relative efficiency of MCDOEs tends to be lower than in non-MCD commodity-exporting countries.13 Within the MCDOEs, countries in the GCC perform better than the non-GCC MCD countries. The performance of the GCC group, however, is weaker than that of the non-MCD commodity exporters with strong institutions (e.g., Australia, Canada), or the best performers in the global sample (e.g., Germany, the United Kingdom). While the relative position of the groups is broadly unchanged with the use of different frontiers, the PFDH scores display a larger variability around the efficiency frontier (Figure 9).

Figure 9.
Figure 9.

Infrastructure Quality (Infrastructure Component – Global Competitiveness Indicator)

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

* Commodity exporters with strong institutions (WGI>90).Source: IMF staff estimates.

A comparison of the efficiency scores across country groups points to substantial room for efficiency improvements in MCDOEs. Under both methodologies, the MCDOEs’ performance ranks well when compared to low-income countries but lags that of advanced economies (Table 1). The magnitude of the inefficiency depends on the frontier under consideration. Under the DEA analysis, for example, the median efficiency score indicates that MCDOEs could increase the infrastructure quality by 18 percent with the same amount of investment.14

Table 1.

Relative Efficiency Scores by Income level

(Median efficiency score)

article image
Source: IMF staff estimates.

B. Project-level Cost Analysis

This section examines the efficiency of public investment by estimating and comparing the unit costs of large public infrastructure projects. Given limited data availability and the emphasis on large-scale projects, the section focuses on mass transit (namely, metros) and roads. Project-level data from the Zawya database are used to calculate the per-kilometer unit cost of mass transit projects and roads. The main caveat is that other factors that could legitimately affect the unit cost (e.g., cost of land and other inputs, quality, technical complexities, underground component, and geographical differences) are omitted due to the lack of comparable information. Despite these limitations, a broad comparison of unit investment costs across country groups may still be indicative of the general cost trends. Examples of earlier studies of unit investment costs include Alexeeva and others (2011), Flyvbjerg (2008), and Fox (2000).

Mass Transit

The project-level analysis reveals a substantial variation in the contracted cost estimates of mass transit systems in MCDOEs.15 The per-kilometer cost estimates of 14 such construction projects in MCDOEs ranges between $44 million and $104 million, based on contract values. Because of the risk of substantial cost escalation evidenced in the literature for infrastructure projects, the final costs of these mass transit projects are likely to be higher. Flyvbjerg, Skamris, and Buhl (2002) find an average cost overrun of 45 percent for rail-related transportation projects, with a higher average (65 percent) for projects outside Europe and North America.

The final cost estimate of mass transit projects in MCDOEs, assuming typical cost overruns, is higher than in other regions in only a few cases (Figure 10). With all mass transit projects but one still under construction in MCDOEs, this Staff Discussion Note approximates the possible final cost of such projects using the assumption of a 65 percent cost overrun from Flyvbjerg, Bruzelius, and Wee (2008), and converts this cost to 2002 prices using the U.S. construction price index. The 75 percent cost overrun observed in the only mass transit project completed among MCDOEs (Dubai) is similar to the average cost overrun of 65 percent found by Flyvbjerg, Skamris, and Buhl (2002) for projects outside Europe and North America. For all MCDOEs (except one), the final cost estimates are above the average cost of mass transit systems in Asia and Latin America. For two of five countries in the sample of MCDOEs, the final cost estimates are higher than those of mass transit projects in the United States.

Figure 10.
Figure 10.

Mass Transit Final Cost Estimates

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Sources: Zawya Projects; Flyvbjerg, Bruzelius, and Wee (2008); and IMF staff estimates.

If low wage costs are taken into account, all MCDOE mass transit projects appear more expensive than similar projects in the United States. Moreover, for three out of five MCDOEs, the final adjusted cost estimate is four to six times the adjusted estimate in the United States. However, an important qualifier to the comparison with the United States is that this calculation does not control for the cost of land or for different costs of raw materials, which MCDOEs must often import at an additional expense.

Roads

The construction costs calculated from data on large-scale road projects also point to substantial variation across MCDOEs.16 The per-kilometer-per-lane cost estimates for 16 road construction projects in MCDOEs vary from $0.2 million to $1.7 million, excluding the projects with the lowest and highest unit costs. The average cost estimate per kilometer per lane is $1.1 million. ($0.75 million excluding the outliers), which is broadly similar to the unit cost estimates for central and eastern European and central Asia countries implied by the estimates in Alexeeva and others (2011).

The literature and experience in the region suggest that the cost escalation of road and rail construction projects is less of a concern than that of mass transit projects.17 The Muscat Expressway in Oman, completed in 2012 with a cost close to $350 million, compared with a contract value of $330 million, shows a 6 percent cost overrun. In a sample of 258 transport projects in 20 mostly advanced countries, Flyvbjerg, Skamris, and Buhl (2002) identified average cost overruns of 28 percent. Overruns are usually associated with both optimism bias in project costing or inadequate controls during project execution.

C. Public Investment Management Index

This section discusses as an alternative measure of public investment efficiency an index that captures the institutional quality of public investment management. The Public Investment Management Index (PIMI) was developed by Dabla-Norris and others (2012) for 71 countries.18 The PIMI covers four main stages of the project cycle, and focuses on institutional processes associated with public investment spending allocated through the central government budget (Table 2).19 20 As with any index of this type, caution needs to be exercised when interpreting the quantitative results, since the PIMI criteria were selected using informed judgment.

Table 2.

Public Investment Management Index

article image
Source: Dabla-Norris and others (2012).

This Staff Discussion Note expands the PIMI to MCDOEs on the basis of responses provided by country authorities to a PIMI survey in late 2013. Overall, scores are available for eight out of 15 MCDOEs. The responses from country authorities were validated with IMF Country Desks, given the lack of required information through other sources. As a result of this process, the responses to some questions were revised downward (see aggregate PIMI scores for selected analytical groups in Annexes III-IV).

PIMI Scores

The available sample of MCDOEs outperforms oil exporters from other regions, but lags behind the best-performing countries (Figure 12). The PIMIs of the MCDOEs on average outperform countries in sub-Saharan Africa (SSA) and emerging Asia and the Pacific (AP), and score similarly to countries in Central and Eastern Europe (CEE) (see list of countries and groups in Annex III). This comparison requires some caution, however, given the limited coverage of oil exporters in other regions (six in SSA, one in Emerging AP, and two in Latin America and the Caribbean (LAC)). Despite this relatively good performance of the MCDOEs, the average overall score is about only half the score of strong performers such as Brazil and South Africa.

Figure 11.
Figure 11.

Average Cost Overruns

(Percent of original project cost)

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Source: Flyvbjerg, Skamris, and Buhl (2002).
Figure 12.
Figure 12.

Public Investment Management Index by Region

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Sources: Dabla-Norris and others (2012); country authorities; and IMF staff estimates.Note: Number of countries in parentheses.

Looking at the project cycle, MCDOEs’ performance is particularly weak at the appraisal and selection stages (Figure 13). The analysis of median scores suggests that the MCDOEs lag behind countries in the CEE and LAC regions in the appraisal and selection stages of the project cycle, but perform better than SSA. The main areas for improvement in MCDOEs include cost-benefit analysis, at least for large projects; medium-term planning and budgeting frameworks integrated with the annual budget; selection of investments on the basis of relevant sector strategies; and consideration of recurrent costs. In the implementation and evaluation stages, the MCDOEs score above the average of most of the other regions. Annex III elaborates further on the components of the PIMI and presents the performance of MCDOEs countries across all stages of the PIMI.

Figure 13.
Figure 13.

Public Investment Management Index Scores by Stage and Region

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Sources: Dabla-Norris and others (2012); country authorities; and IMF staff estimates.

IV. Explaining Public Investment Efficiency

A. Empirical Analysis

Cross-country regressions suggest that high public investment efficiency is generally associated with good institutional quality and low dependency on natural resource receipts:

  • Quality of institutions. Based on specification (1) in Table 3, a one standard deviation increase (about three percentage points in the quality of governance measures from the International Country Risk Guide, namely bureaucracy quality, corruption, and law and order) could improve public investment efficiency by 0.20 (i.e., about 22 percent of the average score in the sample). The result is consistent with the finding in other literature that in countries with weak institutional quality, governments may use capital spending as a vehicle for rent-seeking (Keefer and Knack, 2007; Grigoli and Mills, 2014), which leads to inefficient spending.

  • Natural resource receipts. Results from specification (1) also suggest a negative association between dependency on natural resources and public investment efficiency: an increase by one standard deviation in natural resource receipts (about 17 percent) could reduce the efficiency of public investment by 0.02, which is roughly 3 percent of the average score in the sample. However, the impact estimates are no longer significant in other specifications. The finding is consistent with Gelb and Grassman (2010), who provided evidence that the high volatility of natural resource revenues of oil and gas exporters has contributed to the poor quality of public spending.

  • Official development assistance (ODA). Results show that ODA is generally positively associated with investment efficiency, although the estimated coefficient is statistically insignificant.

Table 3.

Main Determinants of Public Investment Efficiency

article image
Note: Robust standard errors in parentheses.*** p<0.01, ** p<0.05, * p<0.1.

PFDH scores estimated using the infrastructure component of the GCI (2006-12).

ICRG measure on a scale from 0 to 30 with higher values indicating better quality governance.

WGI index.

Oil reserve dummy variable equals 1 if a country’s reserve horizon is greater than the median of all oil-exporting countries, based on BP data.

Standard deviation of annual percent changes in real public investment.

Using alternative indicators of institutional quality and including additional control variables do not generally affect the results. The regression analysis considers an alternative variable for the quality of institutions: the World Governance Indicator (WGI) from the World Bank (see specification (2)). The additional controls are public investment in percent of GDP, oil reserve horizon,21 and public investment volatility.22,23 The impact of institutional quality on public investment efficiency is significant under both alternative measures of institutional quality with additional control variables.24 The adverse impact of the public investment level on efficiency could reflect diminishing returns on investment and/or wasteful public spending, though the estimates are not statistically significant.25

In this context, the low rank of (non-GCC) MCDOEs on World Bank governance-related indicators could adversely impact the efficiency of public investment (Figure 14). Common arguments explaining inefficiencies in public investment in oil exporters are the abundance of oil revenues, which could weaken incentives to prioritize and carefully appraise projects; lack of demand for project appraisal and independent review due to the politicization of public investment decision-making; and the absence of a more medium-term perspective, which also weakens investment decisions (Rajaram and others, 2014).

Figure 14.
Figure 14.

Worldwide Governance Indicators, 2012

(Percentile rank)

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Sources: World Bank, WGI (government effectiveness, regulatory quality, rule of law and control of corruption); and IMF staff calculations.

B. Successful Case Studies

An in-depth analysis of successful frameworks for the appraisal and selection of investment projects in two resource-rich countries (Norway and Chile) can provide useful lessons for the MCDOEs that scored particularly low in these two areas of the PIMI. Rajaram and others (2014) point to the potential pitfalls of investment decisions when commodity-related revenue is increasing rapidly. In such a setting, investment decisions tend to be discrete, relatively discretionary, and therefore quite politicized. The two countries selected as case studies face similar challenges to MCDOEs in managing public investment amid abundant commodity-related revenues, but have achieved relative success in the outcomes of their public investment programs. The analysis of Norway’s framework is based on Samset and Volden (2013) and the analysis of Chile’s PIM is based on Gomez-Lobo (2012) and World Bank (2006).

Norway

Norway has developed a strong framework for fostering successful public investment outcomes. This two-stage framework—known as the Quality Assurance Scheme (QAS)—was adopted in 2000 to improve the quality of investment projects by establishing a system in which politics and administration are clearly separated (Figure 15).

Figure 15.
Figure 15.

Norway: Quality Assurance Scheme

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Source: Ministry of Finance, Norway.

The QAS has two “gateways.” The first gateway of the system focuses on the cost-benefit analysis, including alternatives, before a government’s decision is made. The second gateway is undertaken before a formal proposal to the Parliament is made and considers a project management strategy, including an independent consultant’s views on costing. The scheme has helped to prevent controversies about the ineffective use of public funds and brought more attention to cost estimates.

The QAS has had a positive effect on project savings, although project performance varies by sector. According to the data submitted to QAS during 2000–09, the total net saving for the projects under review, defined by the difference between the original budget and the final cost, was estimated at about 7 percent of total investment (Figure 16).26 About 80 percent of road projects have been completed within the cost frame. The average savings in the railway and construction sectors were smaller than in case of roads (Figure 17).

Figure 16.
Figure 16.

Norway: Deviation between Final Project Cost and Project Budget

(40 projects in total, percent)

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Source: Samset and Volden (2013).
Figure 17.
Figure 17.

Norway: Cost Overruns and Savings by Sector

(Percent of project budget)

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Source: Samset and Volden (2013).Note: Figures in parentheses indicate number of projects.

Chile

Chile’s National Investment System (NIS) has several institutional safeguards to ensure appropriate management of public investment projects. The Ministry of Planning is in charge of this system (Figure 18). By law, all public bodies, including ministries, local governments, and state-owned enterprises, must apply to the NIS for funding to undertake an investment project or program. The objective is to improve the quality of public investment by selecting the projects with the largest net present social value. More complex or expensive projects require an evaluation at each sequential stage of the initiative (identification, pre-feasibility, feasibility, and design) with a strong focus on managing total project costs over the life of each project. Under closer scrutiny, 5–8 percent of project proposals are rejected at initial screening. Specific mechanisms, as a distinctive feature of Chile, are in place to trigger a review of a project’s continued justification if there are material changes to project costs. Where the lowest tender is 10 percent or more above the estimated price, the project is subjected to a reappraisal.

Figure 18.
Figure 18.

Chile: Public Investment Project Appraisal Cycle

Citation: Staff Discussion Notes 2014, 010; 10.5089/9781498311397.006.A001

Source: World Bank (2006).

The NIS has served as a gatekeeper to ensure that only projects fulfilling these requirements receive an allocation of public resources. The NIS standardizes project presentation formats and establishes explicit application and evaluation processes for public funds. It also provides general as well as sector-specific methodological guidelines for the appraisal of projects and programs. Medium-term public investment envelopes are in place, and major, documented efforts have been put into building the capacity for project appraisal across the whole government. The NIS also has a system of checks and balances. In particular, the system separates the institution that reviews and approves the appraisal of projects from the institutions promoting projects. As a result, public projects the road and health sectors have started to generate cost savings in 2000s. The NIS has helped sustain a pipeline of appraised and approved projects that fulfill technical criteria and are eligible for budget funding.

A dedicated system for an ex-post assessment provides feedback to the NIS to improve project appraisal methodologies and refine development criteria to further increase public spending efficiency. The ex-post evaluation system has two components. One consists of reviewing the costs, implementation timeframes and compliance with the technical regulations just after a project is built. The other is an in-depth social impact assessment after the project has been ongoing for a sufficient period of time. Particularly, the ex-post assessment in the NIS features a centralized project information platform that serves as the basis for a historical analysis of the costs and demands of various types of projects.

V. Main Findings and Policy Implications

There is substantial room to improve public investment efficiency in MENA and CCA oil exporters. Improving the efficiency of sizable investment programs in these countries could help boost growth and speed up progress in realizing the development agenda. In particular, the MCDOEs’ performance seems weak relative to that of the best non-MCDOE performers in assessments using efficiency frontiers, project-level data, and PIMI scores. The results also indicate some variability in public investment efficiency within the MCDOEs, with GCC countries ranking generally higher than other MCDOEs. Some caution with these findings is needed, however, because of the limitations of the data and methodologies, as discussed in this Staff Discussion Note.

Stronger institutions could foster more efficient public investment. This is suggested by the regression analysis, the case studies, and the analysis of PIM frameworks. The task of developing strong institutions in MCDOEs is crucial because investment programs are large. Strengthening institutions, though challenging, should begin now. It will likely require sustained efforts over many years.

In the near term, MCDOEs could start with fairly simple measures to promote greater scrutiny of public investment projects while at the same launching broader reform efforts in this area. Such measures could include:

  • Increasing the transparency of data on key investment projects both over the project cycle (e.g., appraisal information, competitive procurement process, bidding statistics, cost/time overruns) and in the context of the budget process (e.g., objectives, costs of main investment projects, ex-post evaluations), as advocated by Barma and others (2012);

  • Preparing an infrastructure needs assessment, with the support of development partners such as the World Bank or the European Bank for Reconstruction and Development, to help guide medium-term sectoral strategies on infrastructure and monitor progress in closing infrastructure gaps; and

  • Undertaking in-depth and independent diagnosis of the current PIM system—also in coordination with development partners—to help identify country-specific priority areas for reform in the management of public investments.

Subsequently, and on the basis of the in-depth analysis of the PIM system, the MCDOEs could focus on modifying the framework for managing public investment to help improve its productivity. The PIMI analysis suggests that stronger medium-term budget frameworks, more thorough appraisal and selection of investment projects, and systematic ex post assessments could help strengthen the productivity of public investment. Looking forward, additional insights into PIM reform strategies in MCDOEs and other economies should be informed by the forthcoming IMF Policy paper “Making Public Investment More Efficient,” which will discuss in more detail which fiscal institutions are critical and can be strengthened to improve the efficiency of public investment.

Annex I. Estimating Efficiency Frontiers

The methodological framework for measuring the efficiency of production units has been widely used to gauge the efficiency of public spending (Herrera and Pang, 2005). This framework is based on a production function approach in which inputs are combined to produce outputs based on a given technology. To make this theoretical framework operational, researchers can choose between two families of empirical methods: parametric and nonparametric (Grigoli and Kapsoli, 2013). The first involves the estimation of an econometric model with the restriction of nonpositive errors. The latter is related to linear programming, where the data are enveloped with a piecewise linear hull. The most popular options for nonparametric efficiency analysis are data envelopment analysis (DEA) and free disposal hull (FDH). DEA uses a convex hull function while the FDH uses a nonconvex and staircase-shaped hull function.

Nonparametric methods have been criticized because they are very sensitive to the presence of measurement errors and outliers. On the other hand, parametric methods would require the introduction of several assumptions regarding the stochastic distribution of errors and the functional form underlining the model. They would also require a set of control variables measuring institutional quality and other determinants of output (Grigoli and Kapsoli, 2013)

To balance between quality and availability of information, this Staff Discussion note uses an intermediate methodology. We use a nonparametric method called partial frontier analysis. There are several methods in this category; we are using one called order-m efficiency (see Felder and Tauchmann, 2013, for more details) also known as partial free disposal hull (PFDH). It generalizes FDH by adding a layer of randomness to the estimation of the efficiency scores. Instead of benchmarking a decision-making unit relative to the best-performing peer in the sample, order-m compares each decision-making unit against the best performer in an artificial subsample of m peers. The sample is randomly drawn with replacement. This stage is repeated n times resulting in n pseudo-efficiency scores. Finally, order-m efficiency scores are calculated as the average of the n pseudo-efficiency scores.

This methodology attenuates the impact of extreme observations (outliers) on the efficiency scores. Also, as a partial frontier approach, order-m efficiency allows for super-efficient decision-making units located beyond the estimated efficiency frontier; therefore, efficiency scores for such decision-making units will be higher than one.

Application

For the paper we estimate efficiency scores using the above-described method in an output-oriented model. Efficiency scores should therefore be interpreted as the proportional amount by which output could be increased while leaving input consumption unchanged. As an output variable we used the quality of infrastructure index published by the World Economic Forum. As inputs we used estimations of public capital stock prepared by the IMF Fiscal Affairs Department using the Penn World Table 7.1. We also included per capita GDP as auxiliary input.

Annex II. Regression Analysis

The exercise covers the global sample and the partial free disposal hull (PFDH) scores estimated using the infrastructure quality output (infrastructure component of the Global Competitiveness Index) and all the other variables over the period 2006–12.

Quality of institutions is constructed using three International Country Risk Guide (ICRG) variables: bureaucracy quality, corruption, and law and order, with higher values indicating better quality governance.

Natural resource receipts capture a country’s dependence on its natural resources; the data come from the World Bank’s World Development Indicators database. They are measured by the average profits of total natural resource exports of an economy in percent of GDP.

Official development assistance (ODA) measure from the Organization for Economic Cooperation and Development is the average net disbursement in percent of GDP.

Summary of Statistics

article image

Result of the Impact of Institutional Quality on Efficiency Scores – Tobit Model

article image
Note: Robust standard errors in parentheses.*** p<0.01, ** p<0.05, * p<0.1.

DEA scores estimated using the infrastructure component of the GCI (2006-2012).

ICRG measure on a scale from 0 to 30 with higher values indicating better quality governance.

WGI index.

Oil reserve dummy variable equals to 1 if a country’s reserve horizon is greater than the median of all oil exporting countries based on BP data.

Standard deviation of annual percent changes in real public investment.

Annex III. Public Investment Management Index27

The Public Investment Management Index (PIMI) captures quality and efficiency across four main stages of the public investment management cycle: appraisal, selection, implementation, and evaluation. The basic processes and best practices associated with the strongest score (4) in each stage are described below.

Strategic Guidance and Project Appraisal

  • Nature of strategic guidance, and availability of sector strategies.

  • Transparency of appraisal standards.

  • Observed conduct of ex-ante appraisals.

  • Independent review of appraisals conducted.

The maximum score requires a well-defined public investment plan and/or sector strategies for most sectors, with full costing of recurrent expenditures and investment; a published document to detail appraisal standards; routinely undertaken economic appraisals for large projects; and independent checks by a regulator or office of appraisals.

Project Selection and Budgeting

  • Existence of medium-term planning framework and its integration in the budget.

  • Inclusion in the budget (or similar) of donor-funded projects.

  • Integration of recurrent and investment expenditures in budget.

  • Nature of scrutiny and funding supplied by the legislature, including its committees.

  • Public access to key fiscal information.

Maximum score requires multiyear forecasts and the clear subsequent setting of annual budget ceilings; detailed information for a large share of donor-funded projects; consistently selected investments; coverage of fiscal policies and medium-term fiscal framework by the legislature’s review; and publicly available information on key fiscal aggregates, external audit reports, and contract awards.

Project Implementation

  • Degree of open competition for awarding of contracts.

  • Nature of any complaint mechanism relating to procurement.

  • Funding flows during budget execution.

  • Existence and effectiveness of internal controls, such as commitment controls.

  • Effectiveness of system of internal audit.

A maximum score requires accurate data on the method used to award public contracts; an operative process for submission and timely resolution of procurement process complaints; the execution of more than 90 percent of the capital budget; broad expenditure commitment controls; and internal audits (that meet international standards) for all entities.

Project Evaluation and Audit

  • Degree to which ex-post evaluations are conducted.

  • Degree to which external audits are produced on a timely basis and scrutinized by the legislature.

  • The maintenance of asset registers, and/or asset values.

A maximum score requires ex-post evaluations routinely performed by the auditor general or the executive; audited expenditures (which should comply with auditing standards), including capital investments; and a complete and operational asset register.

MENA and CCA Oil Exporters: Details of Public Investment Management Index (Average)

article image
Source: Dabla-Norris and others (2012); country authorities; and IMF staff estimates.

List of Countries in the Public Investment Management Index by Region

article image
Source: Dabla-Norris and others (2012).

Annex IV. Summary of Scores by Analytical Country Groups28

article image
Source: IMF staff estimates.

References

  • Afonso, A., L. Schuknecht, and V. Tanzi. 2010. “Public Sector Efficiency: Evidence for New EU Member States and Emerging Markets.” Applied Economics 42 (17): 2147–164.

    • Search Google Scholar
    • Export Citation

  • Alexeeva V., C. Queiroz, and S. Ishihara. 2011. “Monitoring Road Works Contracts and Unit Costs for Enhanced Governance in Europe and Central Asia. Transport Papers No. TP-33, World Bank, Washington, DC.

    • Search Google Scholar
    • Export Citation

  • Barma N., K. Kaiser, T. Minh Le, and L. Viñuela. 2012. Rent to Riches: The Political Economy of Natural Resource-led Development. Washington, DC: World Bank.

    • Search Google Scholar
    • Export Citation

  • Collier P., F. Van der Ploeg, and A. Venables. 2009. “Managing Resource Revenues in Developing Countries,” OxCarre Research Paper 2009–14, University of Oxford, Oxford, UK.

    • Search Google Scholar
    • Export Citation

  • Dabla-Norris E., J. Brumby, A. Kyobe, Z. Mills, and C. Papageorgiou. 2012. “Investing in Public Investment: An Index of Public Investment Efficiency.” Journal of Economic Growth 17 (3): 235–66.

    • Search Google Scholar
    • Export Citation

  • Dabla-Norris E., R. Allen, L. Zanna, T. Prakash, E. Kvintradze, V. Lledo, I. Yackovlev, and S. Gollwitzer. 2011. “Budget Institutions and Fiscal Performance in Low-Income Countries.” IMF Working Paper 10/80, International Monetary Fund, Washington, DC.

    • Search Google Scholar
    • Export Citation

  • Felder, S., and H. Tauchmann. 2013. “Federal State Differentials in the Efficiency of Health Production: An Artifact of Spatial Dependence?” European Journal of Health Economics 14 (1): 21–39.

    • Search Google Scholar
    • Export Citation

  • Flyvbjerg, Bent. 2007. “Cost Overruns and Demand Shortfalls in Urban Rail and Other Infrastructure.” Transportation Planning and Technology 30 (1): 9–30.

    • Search Google Scholar
    • Export Citation

  • Flyvbjerg, Bent, N. Bruzelius, and B. Wee. 2008. “Comparison of Capital Costs per Route-Kilometre in Urban Rail,” European Journal of Transport and Infrastructure Research 8 (1): 17–30.

    • Search Google Scholar
    • Export Citation

  • Flyvbjerg Bent, M. Skamris, and S. Buhl. 2002. “Underestimating Costs in Public Work Projects: Error or Lie.” APA Journal 68 (3): 279–295.

    • Search Google Scholar
    • Export Citation

  • Fox, Halcrow, 2000, “World Bank Urban Transport Strategy Review – Mass Rapid Transit in Developing Countries”, Final Report prepared for Department for International Development, Lonely, July

    • Search Google Scholar
    • Export Citation

  • Gelb, A., and S. Grassman. 2010. “How Should Oil Exporters Spend Their Rents?” Working Paper 221, Center for Global Development, Washington, DC.

    • Search Google Scholar
    • Export Citation

  • Gomez-Lobo, A. 2012. “Institutional Safeguards for Cost Benefit Analysis: Lessons from the Chilean National Investment System.” Journal of Benefit-Cost Analysis 3 (1): 1–30.

    • Search Google Scholar
    • Export Citation

  • Grigoli, F., and J. Kapsoli. 2013. “Waste Not: The Efficiency of Health Expenditure in Emerging and Developing Countries.” IMF Working Paper 13/87, International Monetary Fund, Washington, DC.

    • Search Google Scholar
    • Export Citation

  • Grigoli F., and Z. Mills. 2014. “Institutions and Public Investment: An Empirical Analysis.” Economics of Governance 15 (2): 131–53.

    • Search Google Scholar
    • Export Citation

  • Gupta, S., A. Kangur, C. Papageorgiou, and A. Wane. 2014. “Efficiency-Adjusted Public Capital and Growth.” World Development 57 (C): 164–78.

    • Search Google Scholar
    • Export Citation

  • Herrera, S., and Pang, G. 2005. “Efficiency of Public Spending in Developing Countries: An Efficiency Frontier Approach,” Policy Research Working Paper No. 3645, World Bank, Washington, DC.

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund (IMF). 2014. “Is It Time For An Infrastructure Push? The Macroeconomic Effects on Public Investment.” Chapter 3 in World Economic Outlook, World Economic and Financial Surveys, Washington, DC, October.

    • Search Google Scholar
    • Export Citation

  • International Monetary Fund (IMF). Forthcoming. “Making Public Investment More Efficient.” Fiscal Affairs Department Policy Paper, Washington, DC.

    • Search Google Scholar
    • Export Citation

  • Keefer, P., and S. Knack. 2007. “Boondoggles, Rent-Seeking and Political Checks and Balances: Public Investment under Unaccountable Governments.” Review of Economics and Statistics 89 (3): 566–72.

    • Search Google Scholar
    • Export Citation

  • Mandl, U., A. Dierks, and F. Ilzkovitz. 2008. “The Effectiveness and Efficiency of Public Spending.” Economics Papers No. 301, European Commission, Brussels.

    • Search Google Scholar
    • Export Citation

  • Ostry J., A. Ghosh, and R. Espinoza. Forthcoming. “Public Debt, Public Investment, and Growth: Normative Considerations.”

  • Pritchett, L., 2000, “The Tyranny of Concepts: CUDIE (Cumulated, Depreciated, Investment Effort) is Not Capital.” Journal of Economic Growth Vol. 5, pp. 361–384.

    • Search Google Scholar
    • Export Citation

  • Rajaram, A., T. Minh Le, J. Brumby, and N. Biletska. 2010. “A Diagnostic Framework for Assessing Public Investment Management.” Policy Research Working Paper No. 5397, World Bank, Washington, DC.

    • Search Google Scholar
    • Export Citation

  • Rajaram, A., T. Minh Le, K. Kaiser, J. Kim, and J. Frank, Editors. 2014. The Power of Public Investment Management: Transforming Resources into Assets for Growth. Directions in Development, World Bank, Washington, DC.

    • Search Google Scholar
    • Export Citation

  • Sachs, J., and A. Warner. 1995. “Natural Resource Abundance and Economic Growth.” NBER Working Paper 5398, National Bureau of Economic Research, Cambridge, Massachusetts.

    • Search Google Scholar
    • Export Citation

  • Samset K., and G. Volden. 2013. “Investing for Impact: Lessons with the Norwegian State Project Model and the First Investment Projects That Have Been Subjected to External Quality Assurance.” Norwegian University of Science and Technology.

    • Search Google Scholar
    • Export Citation

  • Vandycke, N. 2013. “Natural Resources, Physical Capital, and Institutions.” Policy Research Working Paper No. 6586, World Bank, Washington, DC.

    • Search Google Scholar
    • Export Citation

  • World Bank. 2006. “Appraisal of Public Investment: Chile.” (Washington DC)

1

The staff team was led by Maria Albino-War and supervised by Martin Sommer, under the general guidance of Tim Callen and Alfred Kammer. Diana Kargbo-Sical and Sarah Knight provided editorial assistance. The authors are grateful to Min Zhu for encouraging the launch of this project. The paper incorporates helpful suggestions from Era Dabla-Norris, Daniela Gressani, Mark Horton, Catherine Anne Maria Pattillo, Ananthakrishnan Prasad, Carlo Sdralevich, Hossein Samiei, Marinus Verhoeven, James Brumby, and Felipe Zanna during internal IMF workshops. The authors also thank Benedict Clements (IMF staff), and Murray Petrie and Antonio Afonso (external reviewers) for detailed comments on an earlier draft of the paper.

5

These difficulties are also present in broader analysis aimed at measuring the efficiency of public spending. See Mandl, Dierks, and Ilzkovitz (2008) for a survey of different methods used for cross-country comparisons of the efficiency and effectiveness of public spending.

6

The global sample includes countries from all regions.

7

Refers to non-oil capital spending considering that the fiscal coverage from WEO is mainly of the general government. Capital spending for non-oil infrastructure in some MCDOEs could take place though outside the budget as part of investments of public enterprises (e.g., Saudi Arabia), or as investments of sovereign wealth funds (e.g., Kazakhstan and Turkmenistan). But limited data availability prevents such analysis.

8

This was not the case in several advanced economies during the crisis that experienced large increases in public debt almost entirely unrelated to public investment (Ostry and others, forthcoming).

9

See Gupta and others (2014) for the details on the estimations of public capital stock.

10

Due to data limitations, it is not possible to distinguish between public and private sector infrastructure.

11

The smallest 5 percent of countries (by land mass) are excluded as in these economies other forms of transportation can be more important.

12

An efficiency score of 0.8, for example, would imply that the infrastructure stock could be increased by 20 percent (or 1–0.2) with the same level of inputs.

13

Annex IV presents scores for selected analytical country groups. The charts present the maximum, minimum, and median of all the scores in the relevant group.

14

The efficiency scores derived from the different frontiers are not directly comparable. In particular, the PFDH allows the presence of super-efficient countries located beyond the production possibility frontier, assigning them efficiency scores higher than 1. Under the DEA, the maximum score assigned to countries in the frontier is 1.

15

Given data availability for mass transit projects, selected MCDOEs refer to Algeria and the GCC. The Zawya database includes projects in Algeria (1), Kuwait (1), Qatar (4), Saudi Arabia (4), and the United Arab Emirates (4).

16

For roads, the included construction projects are in Iraq (1), Kuwait (1), Oman (1), Qatar (4), Saudi Arabia (3), and the United Arab Emirates (6).

18

The index was constructed using diagnostics of countries’ public investment management (PIM) systems based on the World Bank and other donor assessments, budget survey databases, and expert surveys.

19

The basic processes and best practices underpinning the PIMI are presented in detail in Annex III.

20

Building on the work of Rajaram and others (2010), Rajaram and others (2014) develop an alternative public investment management framework consisting of eight critical features of the project cycle: guidance; appraisal; independent review; selection; implementation; adjustment; operation; and evaluation. The authors present a number of case studies (including resource-dependent countries outside MCD), assessing their practices against this benchmark.

21

The oil reserve horizon is a dummy variable equal to 1 if a country’s reserve horizon is greater than the median of all oil-exporting countries, based on BP data.

22

Dummy variable equal to 1 if the standard deviation of annual percent changes in real public investment is greater than the 90th percentile in the sample.

23

Additional variables explored in this analysis that did not affect the results and were statistically insignificant include the degree of urbanization, population density, and business environment.

24

The Budget Institutions Index from Dabla-Norris and others (2011) was used as another alternative measure of quality of institutions. The variable, while statistically significant, leads to a drop of about two-thirds of observations.

25

Annex II presents a detailed description and summary of statistics of all variables, as well as additional robustness checks, including the DEA scores and estimates from an alternative Tobit model (the Tobit analysis is used given that the DEA efficiency scores are between 0 and unity).

26

This analysis was based on 40 projects that have produced their final cost figures since the QAS was established. The 7 percent net saving, however, cannot be considered a measure of improvement relative to earlier periods nor a measure of “cost saving,” given that the approved budget may or may not have been accurate.

27

See Dabla-Norris and others (2012) for more detailed information on methodology.

28

Azerbaijan, Iraq, Uzbekistan, and Turkmenistan are excluded from the efficiency frontiers and Algeria, Bahrain, Iran, Oman, Saudi Arabia, United Arab Emirates and Uzbekistan from the PIMI assessment due to data availability.

Same Series

Other IMF Content

Other Publishers

Cover Staff Discussion Notes
Making the Most of Public Investment in MENA and CCA Oil-Exporting Countries
Author: Ms. Maria A Albino, Ms. Svetlana Cerovic, Mr. Francesco Grigoli, Mr. Juan C Flores, Mr. Javier Kapsoli, Mr. Haonan Qu, Mr. Yahia Said, Mr. Bahrom Shukurov, Mr. Martin Sommer, and Mr. SeokHyun Yoon
Volume/Issue:
Volume 2014: Issue 010
Publisher:
International Monetary Fund
ISBN:
9781498311397
ISSN:
2617-6750
Pages:
33
DOI:
https://doi.org/10.5089/9781498311397.006

Paved Roads, 2011

(Kilometers per million people)

Electricity Generation Capacity, 2011

(1,000 kilowatts per million people)

Public Capital Expenditure, 2001–12 1/2/

(In percent of GDP)

Break-even Fiscal Oil Price, 2009 and 2013

(In U.S. dollars)

Efficiency Frontier Analysis

Infrastructure Quality and GDP per Capita

(Index)

Infrastructure Quality and Paved Roads (2011 or latest available)

(Index)

Infrastructure Quality and Electricity Generation (2011 or latest available)

(Index)

Infrastructure Quality (Infrastructure Component – Global Competitiveness Indicator)

Mass Transit Final Cost Estimates

Average Cost Overruns

(Percent of original project cost)

Public Investment Management Index by Region

Public Investment Management Index Scores by Stage and Region

Worldwide Governance Indicators, 2012

(Percentile rank)

Norway: Quality Assurance Scheme

Norway: Deviation between Final Project Cost and Project Budget

(40 projects in total, percent)

Norway: Cost Overruns and Savings by Sector

(Percent of project budget)

Chile: Public Investment Project Appraisal Cycle