It is widely acknowledged that a scaling-up of public investment in low-income countries, particularly in infrastructure, is central to their development prospects. Arguments for significantly boosting investment in physical and social infrastructure to achieve sustained growth rest on the high returns to investment in capital scarce environments, and the pressing deficiencies in these areas. Historically, however, weaknesses in public investment management have resulted in inadequate returns to public and private investment in many low-income countries. Low returns to public investment arise from poor selection and implementation of projects due to limited information, waste and leakage of resources, and weak technical expertise. Private investment returns, in turn, are lowered by the lack of complementary public inputs. At the same time, a substantial scaling-up of public investment in a weak institutional environment runs the risk of potentially undermining its growth benefits as well as prospects for fiscal and debt sustainability.
Country efforts to “invest in the investment process” can play a key role in raising the returns on public and private investment, and in ensuring that the scaled-up investment reaps the required growth dividends, while maintaining fiscal and debt sustainability (Collier, 2008). This encompasses several aspects―country capacity to carry out technically sound and non-politicized project appraisal and selection, appropriate mechanisms for implementation, oversight, and monitoring of investment projects, and ex post evaluation. The transparency and accountability of these functions and processes contributes to ensuring that productive public investment is supported. Economic and institutional indicators that measure and capture these aspects can thus play a key role in guiding the assessment of the scope to increase productive public investment and its growth benefits.
This paper is the first to construct an index that captures different ex ante and ex post dimensions of various stages of the investment process. Specifically, it develops a composite index of the efficiency of the public investment management process for 71 countries (40 low-income countries). The paper draws upon country diagnostics on public investment management systems conducted by the World Bank, existing budget survey databases and assessments carried out by donors, supplemented by expert surveys.1
In contrast to existing assessments of investment efficiency and quality based on physical indicators, the index breaks new ground by examining the efficiency of the process underpinning investment.2 In particular, it records the quality and efficiency of the investment process across four consecutive stages: project appraisal, selection, implementation and evaluation. The index allows for benchmarking against the performance of different country groups and across regions, and provides a new dataset that could be utilized for cross-country analysis.
The remainder of this paper proceeds as follows. Section II provides a brief review of the literature on investment needs in low-income countries and the link between public investment, growth and investment efficiency. Section III describes the components of the index of efficiency of public investment management. Section IV describes the index construction, while Section V presents its statistical properties, and a comparison with other available indicators. Potential applications of the index are discussed in Section VI. Finally, Section VII draws conclusions.
II. Public Investment Scaling-up and Public Investment Management in Low-income countries: A Brief Review of the Literature
There is a broad consensus that a scaling-up of investment in low-income countries, particularly in infrastructure, is critical to achieve sustained growth.3 In many low-income countries, deficiencies in infrastructure, especially in energy, roads, and communication, reduce productivity at least as much as structural factors, such as bureaucracy, corruption and lack of financing Calderon and Serven (2008).4 Improvements in infrastructure not only directly raise the productivity of human and physical capital (for example, roads provide access to remote areas making private investment possible), but also indirectly, through lower transportation costs which increase economies of scale, productivity, and thus growth (Straub, 2008).
Infrastructure stocks vary considerably across regions of the developing world. As shown in Figure 1, Europe and Central Asia (ECA) have the highest stocks in all sectors, followed by the Middle East and North Africa (MNA) and Latin America and the Caribbean (LAC). The lowest stocks are in East Asia and the Pacific (EAP), South Asia, and Sub-Saharan Africa (SSA). More generally, low-income countries across all regions suffer from an infrastructure deficit in comparison to middle-income countries, with the gap widening over time (Foster et al., 2008). While estimates of infrastructure needs in low-income countries vary depending on the methodology employed, available estimates suggest large costs of addressing their infrastructure needs. The Africa Infrastructure Diagnostic (AICD), for instance, estimates overall annual infrastructure spending needs in low-income countries in SSA at US$93 billion, about 15 percent of the region’s GDP (Foster and Briceño-Garmendia, 2009).5
Figure 1.Infrastructure Endowments plotted against Income per Capita, by region
Note: Countries include both low- and middle-income countries
Source: Africa Infrastructure Country Diagnostic (AICD).
Recent studies suggest that the growth impact of increased infrastructure spending in low-income countries is potentially substantial. Calderon and Serven (2008), using physical indicators of infrastructure, find that if low-income countries halved their infrastructure gap, reaching the level in middle income countries, annual growth rates would increase by 2 percent. Calderon and Serven (2008) estimates that, if low-income countries in SSA reach the level of the regional leader (Mauritius), growth could increase by 2.3 percent; if they catch up to countries like South Korea, growth would increase by 2.6 percent.
In many developing countries, however, the link between public capital spending and capital stock accumulation, and hence growth, is weakened by evidence of low efficiency of public investment. The notion that public investment spending is equal to capital accumulation rests on the assumption that public investment is inherently productive. This assumption is particularly problematic in many low-income countries, as a high degree of inefficiency, waste, or corruption often distorts the impact of public spending on capital accumulation, leaving a trail of poorly executed and ineffective projects.
A large body of theoretical and empirical evidence recognizes the importance of the quality and efficiency in investment spending in determining the marginal productivity of investment, and its growth impact. Following Barro (1990), a large number of endogenous growth models show that productive government investment can raise the long-run rate of growth by permanently increasing the returns to other factors of production. More recent theoretical studies models show how inefficient and corrupt bureaucracies can interact with the provision of public infrastructure services, reducing the quality and effectiveness of public capital, firms’ incentives to invest, and hence growth (see for e.g., Chakraborty and Dabla-Norris, 2009).
The bulk of the empirical literature on the economic effects of public investment has focused on its long-run contribution to the level or growth rate of aggregate income or productivity. While far from unanimous, a number of studies suggest a positive relationship, particularly in the case of infrastructure investment. Recent studies, especially those using the physical indicators of infrastructure as proxies for the quality of infrastructure, find significantly positive effects of public capital on growth (Calderon and Serven, 2008). In contrast, findings are less robust among studies that use public investment flows or their cumulative value (Easterly and Rebelo, 1993; Keefer and Knack, 2007).6 As discussed above, this likely reflects the fact that investment spending may be a poor proxy for the accumulation of productive assets in developing countries owing to waste or corruption.
Another strand of this literature finds that the broader institutional context within which investment decisions are undertaken and the quality of project selection, management, and implementation play a crucial role in determining the return on investment and its growth dividends (Esfahani and Ramirez, 2003; Haque and Kneller, 2008). For instance, Flyvbjerg (2003) finds that large cost overruns, benefit shortfalls, waste, and low completion rates are common in major infrastructure projects in developing countries, and can be attributed to their poor selection, monitoring, and evaluation. In a similar vein, Collier et al. (2008) argue that the return on investment in many low-income countries is reduced by limited information and technical capacity for conducting rigorous ex ante appraisal, as well as misaligned incentives, extreme examples of which are corruption and rent seeking.
Public investment, particularly infrastructure, may also respond to political economy motives rather than simple economic efficiency considerations. For example, Henisz and Zelner (2006) present evidence that interest group pressure and the structure of political institutions affects investments by state-owned electric utilities. Guasch et al. (2007) show that weak operational frameworks increase the likelihood of political interference and make the expropriation of sunk investments more likely, jeopardizing the realization of medium term returns. Many of these problems are more acute in low-income countries.
The importance of the quality and efficiency of public investment spending has also been highlighted in arguments for granting countries additional fiscal space for productive investment. A number of studies argue that the failure to recognize the asset-creating nature of investment and the inter-temporal tradeoffs involved creates an anti-investment bias in developing countries, with negative consequences for growth (Easterly and Serven, 2008; Serven, 2007; Collier, 2008). These studies note, however, that public investments are likely to exhibit higher marginal productivity ex post if the government is able to ex ante select high return projects—thereby significantly cutting down on wasteful projects and insuring efficient utilization of fiscal resources for investment spending.
In summary, while the literature suggests that a scaling-up of investment in low-income countries is vital, the link with development outcomes depends critically on the quality and efficiency of public investment. This highlights the importance of going beyond discussions of spending levels and addressing issues of the broad institutional framework underpinning the provision of investment. In particular, assessing the quality of project selection, appraisal, implementation, and evaluation in a country can help identify the specific weaknesses that contribute to poor outcomes and guide appropriate institutional and technical remedies that could correct such failures. To this end, our paper is a first attempt to identify the strength of the public investment management process in developing countries.
III. Index of public investment management quality
In this section, we describe the components of an index of public investment management (PIMI) efficiency, drawing upon the existing literature, country-experiences with public investment management, and the Rajaram et al. (2010) diagnostics framework.7 The index aims to systematize available information regarding the desirable characteristics and functioning of identified stages of the public investment cycle.
Most available quantitative indices compile available information regarding the characteristics and functioning of the budget process, practices, and fiscal rules (see e.g., Dabla-Norris et al., 2010 and references therein). There are, to our knowledge, no available indices of the efficiency of public investment management, which is a related but frequently overlooked area of public financial management. Motivated by this gap in the literature, the main aim of this paper is to construct an index that is relevant for analyzing the de facto strength of public investment management institutions in low and middle-income countries.
A. Components of the Index
Consistent with the literature, we identify four major consecutive phases associated with public investment management: strategic guidance and project appraisal; project selection; project management and implementation; and project evaluation and audit.8 Under each of these stages, the emphasis is on capturing the basic processes and controls that are likely to yield efficient public investment decisions, while recognizing the role of institutions, capacity, and incentives. Each of these stages is made up of several individual components (17 in total). Box 1 provides a summary of the main dimensions and components, while Appendix I provides a detailed description of the scoring methodology and the data sources used.
Box 1.Key Aspects of the Public Investment Management Index (PIMI)
1. Strategic Guidance and Project Appraisal
- Nature of strategic guidance and availability of sector strategies
- Transparency of appraisal standards
- Observed conduct of ex ante appraisals
- Independent review of appraisals conducted
2. Project Selection and Budgeting
- Existence of medium term planning framework and its integration to the budget
- Inclusion in budget (or similar) for donor funded projects
- Integration of recurrent and investment expenditures in budget
- Nature of scrutiny and funding supplied by legislature, including its committees
- Public access to key fiscal information
3. Project Implementation
- Degree of open competition for award of contracts
- Nature of any complaints mechanism relating to procurement
- Funding flows during budget execution
- Existence and effectiveness of internal controls, such as commitment controls
- Effectiveness of system of internal audit
4. Project Evaluation and Audit
- Degree to which ex-post evaluations are conducted
- Degree to which external audits are produced on a timely basis and scrutinized by the legislature
The maintenance of asset registers, and/or asset values.
Strategic Guidance and Project Appraisal
Broad strategic guidance for public investment is an important starting point to anchor government decisions and to guide sector-level decision-makers (Spackman, 2001; Allen and Tommasi, 2001). Strategic guidance ensures that investments are chosen based on development policy priorities. Projects or programs that meet this first screening test need to undergo further scrutiny of their financial and economic feasibility and sustainability to avoid wasteful “white elephant” projects. Our index assesses these dimensions using the following criteria:
- We assess whether broad guidance is available, strategic plans are made, and costs are estimated. Guidance may be derived from a national plan or other medium- to long-term strategic document that establishes economy-wide development priorities at the highest decision-making levels.9 This should be supplemented by a sector level strategy or sub-sector level strategy that provides a more detailed translation of the overarching priorities into an articulated and costed sector investment strategy.
- Project appraisal or ex ante evaluation includes consideration of whether detailed standards for the conducting of appraisals are made available and whether these standards are applied.10 The project selection process should ensure that projects proposed for financing have been evaluated for their social (including environmental) and economic value. To do so effectively, governments should have formal and well publicized guidance on the technical aspects of project appraisal appropriate to the technical capacity of ministries and departments.11 Economic evaluation of projects should be commensurate with the scale and scope of the project – with larger projects requiring more rigorous tests of financial and economic feasibility and sustainability.12
- Our index also evaluates the extent to which appraisal standards are reinforced with some form of independent check. Where departments and ministries (rather than a central unit) undertake the appraisal, an independent peer review might be necessary in order to check any subjective, self-serving bias in the evaluation.
Project Selection and Budgeting
The process of appraising and selecting public investment projects needs to be linked with the budget cycle. Cross-country experience suggests that in the absence of proper integration, governments resort to borrowing without due consideration of the sustainability aspects, assets are inadequately maintained, and major projects suffer from poor management and performance (Jacobs, 2008). Our index assesses these dimensions using the following criteria:
- A medium-term framework that translates fiscal objectives or rules into a credible plan for the evolution of fiscal aggregates is important for evaluating the sustainability of the investment program.13 This is evaluated in two separate criteria: assessing the existence of multi-year forecasts and their linkage to annual budgetary policies; and the integration of recurrent and investment expenditures in the budget to determine whether multi-year current and new sector polices can be financed within annual aggregate fiscal targets.14
- Efficient investment requires sound decisions in the choice of investments, active management of the asset portfolio (including through disposals), and a budgetary process that allocates recurrent funding to operate and maintain existing assets. The latter is especially important for donor funded projects that create assets, which can be significant in many low-income countries, while operation and maintenance costs are borne by government. To this end, our index assesses the inclusion of information on donor-funded investment projects in the budget.
- In some settings, formal project selection checks are avoided by “jumping the fence”, or side-stepping controls put in place to keep out poor quality projects. A more formal review process through the budget committee or equivalent of the legislature, backed up by high levels of public disclosure, could assist in reinforcing the appraisal standards and gateways put in place.15 Our index also captures the extent to which the public has access to information about key fiscal aggregates, contract awards, and external audit reports.
Project implementation covers a wide-range of aspects, from timely budget execution and efficient procurement to sound internal budgetary monitoring and control that supports financial and program management. While problems in implementation can be a reflection of inefficiencies in the previous stages—related to poor project selection and budgetary integration—country experiences suggest that the absence of clear organizational arrangements, regular reporting and monitoring frameworks, and weak procurement practices, can result in chronic under-execution of investment budgets, rent seeking, and corruption. Our index evaluates the strength of project implementation along the following dimensions:
- We assess the strength of procurement practices using two indicators. We first assess the extent to which practices for awarding contracts are competitive. The objective is to secure low cost construction supported by a competitive procurement process that is free of collusion between suppliers and government purchasers. Second, we assess the existence and operation of a procurement complaints mechanism to provide adequate checks and balances in the process.16
- Interruptions in or unpredictability of funding flows, whether own-sourced or donor-financed, can undermine efficient implementation of projects and result in under-execution of the capital budget (World Bank 2004)17 Our index proxies for this by assessing the extent to which under-execution of capital budgets has been a chronic problem over the past three years. While the fact that resources in the capital budget are expended as planned may only provide a partial indication that individual projects are implemented efficiently, as capacity constraints could also be a key bottleneck, it can point to systematic problems.
- To reinforce the drive for efficiency and decrease corruption in project implementation, having appropriate internal controls in relevant ministries as well as a credible internal audit function in place for investment projects is important (Diamond 2006). To this end, our index also assesses the existence and effectiveness of internal controls, and of the internal audit function.
Project Audit and Evaluation
A desirable but often missing feature of public investment management systems in developing countries is an ex post evaluation of completed projects, which in its basic form focuses on the comparison of the project’s costs with those established during project design18 Sound facility operation also requires that comprehensive and reliable asset registers be maintained and subject to external audit.19 Inadequate asset registration systems make it difficult to maintain or account for physical property, and can result in leakage. Our index assesses the following dimensions:
- We assess whether ex-post evaluation of domestic projects are routinely undertaken and performed by the auditor general or the executive and whether investment projects are routinely subject to external audits.20 The former can provide guidance on how to improve future project design and implementation.
- The index assesses the extent to which asset registers or inventory of public sector property is maintained.
A few qualifications are in order before we turn to the specifics of index construction. Although our index is quite comprehensive, it does not attempt to provide an exhaustive catalogue of all aspects of public investment management. Practicalities associated with data availability constrain the choice of indicators that could be included. For instance, it would have been useful to include information on the extent to which countries fund maintenance of their assets. While higher funding does not necessarily translate into better-maintained assets, low funding or ineffective asset management programs can reduce the lives and productive values of assets, thereby, undermining their growth benefits.
It has also proven difficult to identify data in a number of other areas which are pertinent to an assessment of the efficiency of public investment management. For instance, monitoring project implementation would minimally involve a comparison of project progress relative to the implementation plan. However, at this stage, it has proven difficult to identify a suitable source of data on this issue. In addition, given the long-term nature of infrastructure projects, there could be changes to the underlying economic and social value of the project during the period that it has been devised, elaborated and implemented. As a result, it is important to have mechanisms in place to trigger a review of the project’s continued justification in light of material changes to project costs, schedule, or expected benefits (Squire 1984). It has also proven difficult to capture this aspect due to data constraints.
In other areas, while data may exist, their interpretation may be system-dependent such that their inclusion becomes problematic. For instance, the role of the legislature can be reflected in various ways – in amending budgets, in reviewing them, in introducing projects, or in stopping projects that are underway. In some countries, the legislature is the budget allocating entity, whereas in others, any attempts by the legislature to change budget allocations proposed by the executive could become a matter of ‘confidence’ in the executive. While various attempts were made to codify meaningful differences, the focus was relegated to roles that appear accepted across all jurisdictions – such as the scrutiny role and the requirements for the provision of public information.
Public investment management allocations tend to be subject to budget allocations, and frequently the public expenditures of state-owned enterprises and private partners. Thus, much of the actual data and control systems are determined by country-specific institutional arrangements. In terms of coverage, the focus of the institutional processes captured by our index is largely on public investment by central government entities (ministries, departments and agencies). Any distinctive issues that relate to the interface between public investment by central and sub-national governments or public private partnerships (PPP) are not directly included given data limitations.
The compilation of the index should, therefore, be seen as a first attempt to amass comparative information of interest – further work could focus on ways to resolve data compilation and comparison issues on other important dimensions of public investment management.21
IV. Index Construction
As described in the previous section, the index is composed of 17 indicators grouped into four stages of the public investment management cycle: (i) Strategic Guidance and Project Appraisal; (ii) Project Selection; (iii) Project Implementation; and (iv) Project Evaluation and Audit. To capture the efficiency of the public investment management process along each of the stages, we scored countries based on the different indicators and sub-indices, which were then combined to construct the overall index.
Most of the data used in the construction of the indices is qualitative in nature. For each question, a scale between 0 and 4 was used, with a higher score reflecting better public investment management performance. In answering the questions described in Box 1 and in assigning scores, it is inevitable that some degree of judgment was exercised. To minimize the degree of discretion, a set of coding rules was used, which can be found in Appendix I. The coding depended on the nature of the question. For some factual questions the coding was binary (0 or 4 score). Other questions allowed for a more-detailed scale for their answers, and hence greater differentiation across countries in terms of the various dimensions.
A. Data Collection
The construction of the index relied upon an extensive data collection effort. Data were compiled from a large number of sources including from World Bank Public Investment Management (PIM) case studies, PEFA assessment reports, the Budget Institutions database, World Bank Public Expenditure Reviews (PERs), World Bank Country Procurement Assessment Reviews, and World Bank Country Financial Accountability Assessments, and country websites (see Appendix II for a more detailed description of the data sources). The sources largely cover the 2007-2010 periods, and include 71 countries (40 low-income countries and 31 middle-income countries).
A wide variety of sources was necessary to create the index as no single database possessed either the country coverage or the appropriate systems’ details. The World Bank’s PIM diagnostic case studies and PERs were the primary source of information on how national PIM systems are actually functioning across different stages of the investment cycle. These relied on in-country discussion with country officials, review of published and unpublished material, and discussions with relevant stakeholders. For many countries, these sources were not sufficient to score all questions, particularly for project appraisal, project evaluation, and capital budget execution rates. As a result, to supplement the information, a short questionnaire was sent to World Bank public finance experts and country economists.
B. Weighting and Aggregation
Appropriate weighting of indicators into sub-indices, and subsequently into an aggregate index, is a crucial issue in index construction. Both our benchmark PIMI overall index and the four sub-indices are constructed using a simple arithmetic mean. For example, the Project Selection sub-index is the simple average of its five indicator terms, while the Project Evaluation sub-index is the simple average of its three indicator terms. The PIMI overall index is then derived as a simple average of the four sub-indices. The advantage of arithmetic averaging is that it is straightforward and transparent. In addition, the absence of strong priors over the weights of indicators in each of the sub-indices makes simple averaging the natural benchmark candidate in the literature (see for e.g., Knack et al., 2010).
To examine the robustness of our indices, we also considered alternative weighting schemes. As discussed below, the rank order correlations between the different approaches are high and significant, suggesting that the additive aggregation procedure used for the construction of the benchmark overall index is robust to alternative weighting schemes.
V. PIMI Country Scores
In this section, we describe the main features of the overall index and the four sub-indices. We also provide a comparison between our index and other related indices of institutional quality.
A. Country Scores
We begin by reporting country scores. The advantage of reporting scores is that they provide a metric for assessing country performance (for e.g., relative ranking of stronger and weaker performers or the difference between the mean index and countries falling in the bottom quartile of the sample distribution). We first discuss the overall-index scores before turning to a discussion of the sub-indices.
Scores Based on the Overall Index
Figure 2 illustrates the overall scores and the decomposition of the four sub-indices (the scores are reported in Table 1; item-by-item scores are available by the authors upon request). The most notable feature of Figure 2 is the large variation in the range of the index scores across countries. The mean overall index score for our sample is 1.68, while the standard deviation is 0.66. Not surprisingly, the top 5 countries are middle-income countries (South Africa, Brazil, Colombia, Tunisia and Thailand; Table 1a), while the weakest performers (Belize, Congo, Rep., Solomon Islands, Yemen, and W. Bank and Gaza) are largely low-income countries. The heat map presented in Figure 3 illustrates the sizeable variation in the investment process in our sample of developing countries from a slightly different angle.
Figure 2.PIMI Overall Index: Decomposition by Sub-Index Figure 3.Heat Map of PIMI Index and Sub-Index
|Overall Index||Overall Index|
|Country Name||Score||Country Name||Score|
|Bangladesh||2.04||Trinidad and Tobago||1.10|
|FYR Macedonia||1.93||Sierra Leone||1.03|
|Namibia||1.81||Sao Tome and Principe||0.90|
|El Salvador||1.77||West Bank and Gaza||0.80|
|Mongolia||1.72||Congo, Republic of||0.50|
Some basic descriptive analysis of the overall PIMI is suggestive of the relative strengths and weaknesses in public investment management processes across different country groups. As can be seen from Figure 4, across regions, ECA countries have relatively more developed public investment management processes, followed by countries in the LAC and EAP regions, while countries in the MENA and SSA regions trail behind. SSA countries, in particular, are characterized by weaknesses in all stages of the public investment management process, albeit with substantial cross-country variation. Project appraisal and evaluation are weak areas across all regions included in the sample.
Figure 4.PIMI Overall Index: Sub-groups
The bottom panel of Figure 4 shows that oil exporters have lower PIMI scores than the rest of the countries in the sample. This correlation between poor institutional quality and natural resource wealth is well documented in the theoretical and empirical literature (see Collier and Van der Ploeg, 2009, and references therein). In particular, it has been noted that resource revenue windfalls frequently increase incentives to misappropriate funds, thus discouraging incentives for a sound institutional and public investment management processes.
Next, we examine further how the index varies across countries on the basis of their level of economic development. While low-income countries, on average, have weaker public investment management processes than middle-income countries, this masks significant cross-country variation (Figure 5). In particular, a number of low-income countries have considerably higher scores than several middle-income countries. For instance, both Bolivia and Rwanda have PIMI scores above 2, surpassing the scores of 19 middle-income countries. The overlap of PIMI scores across middle- and low-income countries raises several interesting questions. For instance, why have relative strengths in public investment management efficiency in some low-income countries failed to translate into higher per capita GDP? And, conversely, how have several middle-income countries managed to grow out of poverty without exhibiting high PIMI scores?
Figure 5.PIMI Overall Index: MICs vs. LICs
One possible explanation is that the quality of public investment management only partially captures the relevant institutional processes that are necessary to achieve sustained growth. For instance, it could the case that institutions for managing public investment need to be complemented with other pertinent economic and political institutions in order to produce growth dividends. Specifically, in the context of growth regression analysis (discussed later) the variables of first order interest could be the interactions between PIMI and other institutional factors rather than the level effect of PIMI itself.
Another interesting observation is that middle- and low-income countries, on average, exhibit comparable scores for the project implementation stage of the investment process, while the largest differences between the two groups are in the project appraisal, selection and evaluation stages. It would be important to understand why countries across both income groups have made the greatest strides in the project implementation stage, and what may be the possible sources of variation in the remaining three stages of investment process. Clearly, these questions warrant further analysis that is beyond the scope of this paper.
In summary, the overall PIMI index shows significant variation across the 71 countries in our sample. While there are some important regional differences in the overall scores, project appraisal and evaluation stages of the investment process appear to be areas of weakness across all country groups. Dividing the sample by income groups reveals an interesting overlap in overall index scores across low- and middle-income countries that is worthy of further investigation. In addition, consistent with the literature, natural resource commodity exporters show a significantly lower PIMI scores than other countries.
Scores by Sub-Index
Beyond the large cross-country variation in overall scores described above, there is an even more notable variation for each of the sub-indices. This suggests that the observed differences in public investment management processes across countries stem largely from the substantial cross-country heterogeneity across the four stages of the investment process.
The cross-country variation in sub-index scores is illustrated in Table 1b. The first two columns of Table 1b report the scores of countries for the Appraisal sub-index, respectively. The entire potential range for scores (0 to 4) is encompassed, with 9 countries (mostly small island economies and countries in SSA) receiving a score of 0 for not having effective mechanisms for choosing public investment projects, and 2 countries (South Africa and Colombia) receiving the maximum score of 4. The variation across country scores in the sample is high, with a mean of 1.49 and a standard deviation of 1.09. In addition, it is evident that a large number of countries have low capacity to choose public investment projects (25 countries received a score of 1or lower).
|Trinidad and Tobago||0.00||2.40||1.33||0.67|
|Sao Tome and Princip||0.00||0.80||1.47||1.33|
|West Bank and Gaza||0.00||1.20||1.33||0.67|
|Congo, Republic of||0.00||1.20||0.80||0.00|
The second two columns of Table 1b report the scores of countries under the Selection sub-index, respectively. This sub-index shows somewhat lower variability as compared to the appraisal sub-index, possibly reflecting the fact that low-income countries have made greater strides in improving the quality of their budget institutions over the past two decades (see for e.g., Dabla-Norris et al., 2010). For instance, the sub-index mean is 1.89 with a standard deviation of 0.77. Only 8 countries have scores below 1, while 31 countries score 2 or higher. The Implementation sub-index exhibits higher mean scores relative to the previous investment stages (mean of 1.97), while the overall variability is again lower than in the appraisal stage. Top performers along this dimension are middle-income countries (Brazil, Thailand and El Salvador, with all three receiving a score of 3.33). Finally, we find that the mean score under the Evaluation sub-index (mean of 1.42), while similar to that for the appraisal index, is substantially lower than the means of the selection and implementation indices, suggesting that this is an area of relative weakness for a large number of countries in our sample.
To illustrate the usefulness of the disaggregated PIMI sub-indices for policy-relevant analysis, we focus on the public investment management processes in selected SSA countries. As shown in Figure 6, there is significant variation in the overall PIMI index score across the seven countries considered, with Burundi exhibiting a low overall score, and Rwanda a high score. More interestingly, the figure shows vastly different performance across the four stages of the investment management process, even among countries with virtually indistinguishable overall scores.
Figure 6.PIMI Index and its Subcomponents: Select Countries in SSA
Kenya, Tanzania and Uganda (the three largest countries in the East African Community) are cases in point. While all three countries have comparable scores on the overall PIMI (around 1.5), their performance under the PIMI sub-indices differs substantially. For instance, while Uganda exhibits relatively high scores in the project selection stage compared to its neighbors, it lags behind in the implementation and evaluation stages. By contrast, Tanzania appears to be performing relatively poorly in the appraisal stage. This example points to the large differences across countries in the relevant areas of weaknesses in the processes for managing public investment. Thus, policy actions to improve public investment capacity should be tailored around country-specific needs. This, in turn, suggests that the more aggregate approach adopted in this paper could usefully be complimented with more in-depth country-specific diagnostics.
The analysis of the PIMI sub-indices sheds new light on the diversity in public investment management processes across the countries in our sample. It is worth noting that the four sub-indices were chosen on the basis of a conceptual rather than a statistical classification. In this sense, even beyond their collective use in constructing the overall index, the sub-indices could independently be employed for analytical or policy work.
B. Comparisons of PIMI with Existing Institutional Indices
While strict comparisons with existing indices are difficult because none captures the efficiency of the investment process per se, it is still useful to assess whether our index provides meaningful information. Figure 7 presents rank pair-wise correlations of the overall PIMI index with five existing relevant indices: the Budget Institution index constructed by Dabla-Norris et al. (2010), Kaufman-Kraay governance indicators (including Government Effectiveness, the average of the Governance Indicators, and the Control of Corruption index) and the World Bank’s Country Policy and Institutional Assessment (CPIA) index, focusing specifically on the sub-CPIA index.22
Figure 7.Spearman Correlation between PIMI and Other Indices
Note: Government Effectiveness, Control of Corruption, and Average of Governance Indicators are the Kaufmann-Kraay indicators.
Spearman rank correlations of the overall PIMI with the sub-CPIA and the budget institution index are relatively high (0.5 and 0.6, respectively). PIMI is also positively correlated, albeit at lower levels, with the Kaufman-Kraay governance indicators. Overall, positive but not exceedingly high correlations with other existing indices instill confidence that the PIMI captures related but not identical information. In particular, it indicates that the PIMI includes information on specific dimensions of institutional quality not fully captured by other indices. Instead it can be viewed as a useful complement, capturing the public investment management component of the overall institutional environment.
We next investigate the relationship between the PIMI sub-indices and other available indices. As shown in Figure 8, the project selection and implementation sub-indices are the most highly correlated with the budget institutions index and the disaggregated governance indicators. In particular, spearman rank correlations with the budget institutions index are relatively high—0.51 and 0.52, respectively—reflecting the close association between the components captured in these sub-indices and public financial management more generally. Interestingly, project selection and implementation are more correlated to the Kaufmann-Kraay governance indicators than the appraisal and evaluation stages of the investment process. This could likely reflect the greater scope for waste and corruption at these stages of the investment process. For instance, without proper integration of investment projects into the budget, or formal project selection checks, the scope for leakage increases. Similarly, at the project implementation stage, weak procurement practices could render the investment process more prone to corruption.
Figure 8.Spearman Correlation between PIMI Sub-Index and Other Indices
Note: Government Effectiveness, Control of Corruption, and Average of Governance Indicators are the Kaufmann-Kraay indicators.
These results provide evidence of the complementarity between the components of PIMI and existing governance indicators. Moreover, the comparisons with other existing indices taken together, confirm that PIMI can be used to capture the efficiency of the investment process.
In this section, we show that our benchmark index incorporates a reasonable set of components by ensuring that specific indicators measuring similar aspects of the public investment management process were grouped together, consistent with measurement and index number theory. Finally, we show that the overall PIMI index is robust to alternative weighting schemes commonly used in the literature.
In this sub-section, we examine the correlations among indicators in each sub-index in order to assess whether indicators were appropriately classified into a particular sub-index. The second column in Table 2 presents the inter-item rank correlations of the four sub-indices. The third and fourth columns present the number of indicators in each sub-index and a “scale of reliability” coefficient. The scale of reliability coefficient is a 0-1 normalized statistic that measures how reliable the aggregation of a particular index is (the higher the reliability, the higher the statistic). It is a positive function of the average inter-item correlation and of the number of items included.
|No. of component items||Scale reliability coefficient|
|PIMI overall index||0.25||4||0.73|
|(average of 4 sub-indices)|
|PIMI-17 overall index||0.24||17||0.80|
|(average of all 17 indicators)|
As discussed above, the score on the appraisal sub-index was obtained by aggregating four indicators. The mean inter-item correlation among the four components is 0.38, resulting in a scale reliability coefficient for the sub-index of 0.70. Aggregating across five indicators similarly generated the scores on the selection and implementation sub-indices. The mean inter-item correlation among these sub-index components is 0.22 and 0.20, respectively, with coefficients of internal reliability of 0.57. Finally, the sub-index for project evaluation was obtained by averaging across three indicators, with a mean inter-item correlation of 0.18 and coefficient of reliability of 0.41.
The bottom of Table 2 also presents inter-sub-index correlations in the benchmark case. The average correlation among the four indices is 0.25, with a high coefficient of reliability of 0.73. The table also presents the average correlations of the PIMI-17, an alternative index where each of the 17 indicators are weighted equally (and not aggregated into sub-indices). As shown in columns 2 and 4, the results are very similar to those obtained under the benchmark PIMI, both in terms of the rank correlations and level of reliability. Importantly, the improved validity does not come at the expense of reliability; that is, the average inter-item correlations remain virtually unchanged, while the scale reliability coefficients increases somewhat when the overall index is constructed using the alternative weighting scheme.23
In summary, the results in Table 2 indicate that the average intra-sub-index correlations, (ranging from 0.18 to 0.38) justify the composition of the sub-indices, without raising concerns about multicollinearity. The associated reliability coefficient estimates (ranging from 0.41 to 0.70) provide a further indication that our sub-indices are reasonably constructed. The same conclusion is reached regarding the construction of the overall PIMI, as the relevant reliability coefficients are quite high (0.73 for the PIMI and 0.80 for the PIMI-17).
Alternative Weighting Schemes
Sensitivity analysis was carried out to check the robustness of our indices to alternative aggregating and weighting procedures (Table 3). In particular, different weights and assumptions about the degree of substitutability and complementarity of components were considered. First, we weighted each of our 17 indicators equally to produce the PIMI-17. Second, we used Principal Component Analysis (PCA), which is commonly used in the literature, to obtain an alternative PIMI-PCA index.
|Overall Index||Alternative Summary Index|
|Trinidad and Tobago||TTO||1.10||1.22||4.36|
|Sao Tome and Principe||STP||0.90||0.90||3.27|
|West Bank and Gaza||WBG||0.80||0.86||3.08|
|Congo, Republic of||COG||0.50||0.59||1.62|
The first pair of columns in Table 3 reports the scores from the benchmark PIMI (same as reported in Table 1a). The next two pairs of columns present two alternative indices—the PIMI-17 and the PIMI-PCA. Relative to the benchmark overall PIMI, the PIMI-17 gives more weight to the Selection and Implementation dimensions. The last pair of columns presents an overall index obtained using PCA.24
While these alternative weightings affect country scores, the resultant changes are not substantial and our main results are unaltered. The rank order correlations between the different approaches are high and significant, suggesting that the additive aggregation procedure described above is robust to alternative specifications. For example, the top 7 performers are identical under all three versions of the PIMI, and the same is true for the bottom 6 countries. Of course, the indicators and sub-indices can be aggregated in several other meaningful ways, some of which may be equally valid as those presented in Table 3.
VI. Potential applications of PIMI
The PIMI can be used to study a broad range of questions in the literature and could potentially add value to the often-inconclusive empirical evidence on the investment-growth nexus. Although not causal, scatter plots of the PIMI against real per capita GDP and growth reassuringly illustrate a positive correlation (Figure 9). While the PIMI index is time-invariant, like most institutional processes, the institutional aspects of the investment process captured by the index are likely to be slow moving. In what follows we briefly describe three potential applications.
Figure 9.PIMI with Growth and Income
A potential application of the investment efficiency index is in the valuation of capital stocks first explored by Pritchett (2000). As mentioned above, resources spent by the public sector do not frequently translate into commensurate additions to physical capital on account of low investment efficiency. Accounting for this difference could have important implications for estimating the returns to public sector capital more accurately, decomposing growth into total factor productivity and factor accumulation, and better capturing the effect of investment spending on growth. The measurement of capital stocks in low-income countries will be particularly important as a significant share of investment spending is carried out by the public sector.
Another empirical application of our index is in growth empirics. Cross-country growth regression estimation can test the efficiency with which public capital stocks are utilized and subsequently affect growth, including interactions between PIMI and other institutional and policy variables. An alternative empirical strategy is to use existing firm level data to assess the impact of public investment efficiency on private sector investment.
The PIMI could also be utilized in aggregate models, which hitherto largely assume that the capital accumulation equation is a function of perfectly efficient investment institutions, and thus one unit of investment translates into in one unit of physical capital. In fact, the index is currently utilized in an application by Berg et al. (2010) who develop a model to study the medium-to-long run macroeconomic effects of external-debt financed public investment expansions in low income countries. Their model makes explicit the investment-growth linkage. The efficiency of investment matters not only for the impact of the investment spending (flow) on the accumulation of investment stock, but also for the stock of existing investment and how efficiently this can be “leveraged” to affect growth. More broadly, incorporating PIMI into macro models of development is a notable avenue of future research that is likely to generate new knowledge and inform the policy debate.
This paper presents, for the first time, a multi-dimensional index of the quality and efficiency of the public investment management process in 71 developing countries. Drawing on multiple sources, the paper assembles the most comprehensive set of information to date pertaining to the investment process for a diverse group of countries, both in terms of regions and levels of economic development. The efficiency of the public investment process is proxied by constructing indices that aggregate indicators across four key stages of the investment process (appraisal, selection, implementation, and evaluation) to reflect institutional arrangements that can deliver the required growth benefits of scaled-up investment. In particular, the index seeks to identify the institutional features that minimize major risks and provide an effective process for managing public investments.
By exploring the sub-indices, in addition to the overall index, researchers and policy makers can break down and investigate different dimensions of the investment management process. Importantly, it can be a useful starting point for undertaking nuanced policy-relevant diagnostics and analysis and identification of specific areas where reform efforts should be prioritized. Over time, the index could be used to evaluate ongoing efforts at improving the investment environment in low-income. Several potential research applications of the index have been outlined.
- Search Google Scholar
- Export Citation
- Search Google Scholar
- Export Citation
- Search Google Scholar
- Export Citation
AllenR. and D.Tommasi2001 “Managing Public Expenditure: A Reference Book for Transition Countries” available via the internet www.sigmaweb.org/dataoecd/5/62/39367671.pdf.
BarroR. J.1990 “Government Spending in a Simple Endogenous Growth Model” Journal of Political EconomyVol. 98103-125.
BergA.J.GottschalkR.Portillo and F.Zanna2010 “The Macroeconomics of Medium-Term Aid Scaling-Up Scenarios” IMF Working Paper 10/160 (Washington: International Monetary Fund).
CalderonC. and L.Serven2008 “Infrastructure and Economic Development in Sub-Saharan Africa” World Bank Policy Research Working Paper No. 4712 (Washington: World Bank).
CaidenN. and A.Wildavsky1974Planning and Budgeting in Poor CountriesTwentieth Century FundUSA.
ChakrabortyS. and E.Dabla-Norris2009 “The Quality of Public Investment” IMF Working Paper 09/154 (Washington: International Monetary Fund).
CollierP. and VenablesA.2008 “Managing Resource Revenues: Lessons for Low Income Countries” African Economic Research Consortium 2008 Annual Conference.
CollierP.2010The Plundered Planet (New York: Oxford University Press).
DiamondJ.2006 “Budget System Reform in Emerging Economies: The Challenges and the Reform Agenda” IMF Occasional Paper No. 245 (Washington: International Monetary Fund).
De HaanJ.W.Romp and J.-E.Sturm2008 “Public Capital and Economic Growth: Key Issues for Europe” in: SchwartzG.A.CorbachoK.Funke(Eds.) Public Investment and Public-Private Partnership: Addressing Infrastructure Challenges and Managing Fiscal RisksPalgrave McMillan & International Monetary Fund pp. 11-20.
EasterlyW. and S.Rebelo1993 “Fiscal Policy and Economic Growth: An Empirical Investigation” Journal of Monetary EconomicsVol. 32417-458.
EasterlyW.T.Irwin and L.Serven2008 “Walking Up the Down Escalator: Public Investment and Fiscal Stability” World Bank Research ObserverVol. 23 pp. 37-56.
EsfahaniH. and M.Ramirez2003 “Institutions, Infrastructure, and Economic Growth” Journal of Development EconomicsVol. 70 pp. 443-477.
FlyvbjergB.N.Bruzeliua and W.Rothengatter2003Megaprojects and Risk: An Anatomy of Ambition (U.K., Cambridge: University Press, Cambridge).
FosterV.C.Briceno-Garmendia and K.Smits2008 “Financing Public Infrastructure in Sub-Saharan Africa: Patterns and Emerging Issues” AICD Background Paper No. 15 (Washington: World Bank).
GuaschJJ.Laffont and S.Straub2007 “Concessions of Infrastructure in Latin America: Government-led Renegotiation” Journal of Applied EconometricsVol. 221267-1294.
HaqueM. and R.Kneller2008 “Public Investment and Growth: The Role of Corruption” Centre for Growth and Business Cycle Research Discussion Paper Series No. 98 (U.K., Manchester: University of Manchester).
HeniszW. and B.Zelner2006 “Interest Groups, Veto Points, and Electricity Infrastructure Deployment” International OrganizationVol. 60263-286.
JacobsD.2008 “A Review of Capital Budgeting Practices” IMF Working Paper 08/160 (Washington: International Monetary Fund).
JonesL.P.Tandon and I.Vogelsang1990Selling Public Enterprises: A Cost-Benefit Methodology (Cambridge, Massachusetts: Massachusetts Institute of Technology).
KeeferP. and S.Knack2007 “Boondoggles, Rent-Seeking, and Political Checks and Balances: Public Investment under Unaccountable Governments” Review of Economics and StatisticsVol. 89 pp. 566-572.
KnackS.F. H.Rogers and N.Eubank2010 “Aid Quality and Donor Rankings” World Bank Policy Research Working Paper No. 5290 (Washington: World Bank).
PetrieM.2010 “Promoting Public Investment Efficiency: A Synthesis of Country Experiences” paper presented at the World Bank Preparatory Workshop, Promoting Public Investment Efficiency Global Lessons and Resources for Strengthening World Bank Support for Client Countries.
PritchettL.2000 “The Tyranny of Concepts: CUDIE (Cumulated, Depreciated, Investment Effort) Is Not Capital” Journal of Economic GrowthVol. 5 pp. 361-384.
RajaramA.T. M.LeN.BiletskaJ.Brumby2010 “A Diagnostic Framework for Assessing Public Investment Management” World Bank Policy Research Working Paper No. 5397 (Washington: World Bank).
ServenL.2007 “Fiscal Rules, Public Investment, and Growth” Policy Research Working Paper Series No. 4382 (Washington: World Bank).
Schiavo-CampoS. and D.Tommasi1999Managing Government Expenditure(Manila; Philippines: Asian Development Bank).
SchwartzG.A.Corbacho and K.Funke2008Public Investment and Public-Private Partnership: Addressing Infrastructure Challenges and Managing Fiscal Risks (Washington: International Monetary Fund).
SpackmanM.2002 “Multi-Year Perspective in Budgeting and Public Investment Planning” draft backgroundpaper for discussion at session III.1 of the OECD Global Forum on Sustainable Development: Conference on Financing Environmental Dimension of Sustainable Development OECDParisApril pp. 46.
SquireL. and H. G.van der Tak1984Economic Analysis of Project (Baltimore and London: The Johns Hopkins University Press).
StraubS.2008 “Infrastructure and Growth in Developing Countries: Recent Advanced and Research Challenges” World Bank Policy Research Working Paper No. 4460 (Washington: World Bank).
WebberD.2007 “Integrating Current and Development Budgets: A Four-Dimensional Process” OECD Journal on BudgetingVol. 7 pp. 93-107.
WebberD.2010 “From Cash Rationing to Credible and Effective Budget Execution” (Unpublished; Washington: World Bank).
|1a. Strategic Guidance|
|1||Are sector strategies prepared, including estimates of their costs, to guide identification of public investment projects?||Costed Sector Strategies||The score is 0 if there is no broad strategic guidance for public investment decisions OR sector strategies are not prepared; 1.33 if there is strategic guidance (PIP, PRSP, national development plan) AND sector strategies exist in several major sectors but are not costed; 2.67 if there is strategic guidance and partially costed sector strategies are prepared for major sectors and 4 if there is a well-defined public investment plan AND/OR sector strategies exist for most sectors with full costing of recurrent expenditures and investment.||Budget InstitutionsDatabase, PEFA, OECD, and ROSC|
|1b. Project Appraisal|
|2||Is there a published document which details appraisal standards?||Appraisal Standards||The score is 0 if there is no published document; and 4 if there is a published document which details appraisal standards.||PIM case studies, websites|
|3||Are economic appraisals (or cost/benefit analysis) routinely undertaken, at least for large projects?||Economic Appraisals||The score is 0 if there are no economic appraisals; 2 if economic appraisals are undertaken for large projects, but not uniformly; and 4 if economic appraisals are routinely undertaken for large projects.||PIM case studies, PERs, websites, FAD TA reports, WB documents|
|4||Is there an independent check or regulator of appraisals to ensure objectivity and quality of appraisals?||Independent Check||The score is 0 if there are no checks; 2 if there are checks, but coverage is compromised; and 4 if there are independent checks done by a regulator or office of appraisals.||PIM case studies, PERs, websites, FAD TA reports, WB documents|
|2a. Project Selection: Integration with Budget|
|5||Is there a medium-term planning and budgeting framework; is this framework integrated with the annual budget?||Medium-Term Planning and Integration||The score if 0 if the government does not prepare multi-year forecasts of fiscal aggregates; 2 if there are multi-year forecasts but there are no links with annual budget ceilings; and 4 if there are multi-year forecasts and the subsequent setting of annual budget ceilings is clear and differences are explained.||Budget Institutions Database, MTEF database, PEFA, PEFA reports, OECD, and ROSC|
|6||Are donor financed projects included in the budget?||Inclusion of information on donor-funded projects||The score is 0 if information on donor-financed projects is not included in the budget or information is seriously deficient; 2 if partial information is included; and 4 if detailed information for a large share of donor-funded projects is included.||Budget Institutions Database, PEFA reports, OECD, and IBP|
|7||Are investments selected on the basis of relevant sector strategies and recurrent cost implications?||Investment Selection||The score is 0 if budgeting for investment and recurrent expenditure are separate processes with no recurrent cost estimates being shared; 2 if many investment decisions have poorly defined links to sector strategies and their recurrent cost implications are included in forward budget estimates only in a few (but major) cases; and 4 if investments are consistently selected on the basis of relevant sector strategies and recurrent cost implications in accordance with sector allocations and included in forward budget estimates for the sector.||PEFA|
|2b. Project Selection: Role of the Legislature|
|8||What is the scope of Legislative scrutiny prior to voting on appropriations to fund projects?||Scope of the Legislature’s Scrutiny||The score is 0 if the legislature’s review is non-existent or extremely limited, or there is no functioning legislature; 2 if the legislature’s review covers fiscal policies and aggregates for the coming year as well as detailed estimates of capital and recurrent expenditure; and 4 if the legislature’s review covers fiscal policies, medium term fiscal framework and medium term priorities as well as details of expenditure.||PEFA|
|2c. Project Selection: Public Scrutiny|
|9||Does the government provide public access to key fiscal information?||Public Access to Key Fiscal information||The score is 0 if information on key fiscal aggregates, external audit reports, and contract awards is not publicly available; 2 if information on key fiscal aggregates but not external audit reports and contract awards is publicly available; and 4 if Information on all three is publicly available.||PEFA|
|3a. Project Implementation: Procurement|
|10||Is there open competition for award of contracts?||Open Competition for Award of Contracts||The score is 0 if there is insufficient data exists to assess the method used to award public contracts or the available data indicates that use of open competition is limited; 2 if less than 75% of contracts above the threshold are awarded on basis of open competition, but the data may not be accurate; and 4 if accurate data on the method used to award public contracts exists and shows that more than 75% of contracts above the threshold are awarded on the basis of open competition.||PEFA, OECD Procurement indicators, World BankCPAR, Open Budget Index|
|11||Is there existence and operation of a procurement complaints mechanism?||Complaints Mechanism||The score is 0 if no process is defined to enable submitting and addressing complaints regarding the implementation of the procurement process; 2 if a process exists for submitting and addressing procurement complaints, but it is designed poorly and does not operate in a manner that provides for timely resolution of complaints; and 4 if a process (defined by legislation) for submission and timely resolution of procurement process complaints is operative and subject to oversight of an external body with data on resolution of complaints accessible to public scrutiny.||PEFA|
|3b. Project Implementation: Timeliness|
|12||During the past three years, has there been chronic under-execution of capital budgets?||Capital Budget Execution||The score is 0 if less than 50% of the capital budget has been spent; 1.33 if 50-70% of the capital budget has been spent; 2.67 if 70-90% of the capital budget has been spent; and 4 if more than 90% of the capital budget has been spent.||Budget papers, capitalbudgets execution rates for African countries, PEFA|
|3c. Project Implementation: Internal controls and audits|
|13||Are there effective internal controls, including controls on expenditure commitments?||Existence and effectiveness of internal controls||The score is 0 if commitment control systems are generally lacking or routinely violated; 2 if such controls exist, but do not cover all expenditures, or are occasionally violated; and 4 if comprehensive expenditure commitment controls are in place and compliance with rules is high.||Budget Institutions Database, PEFA reports and ROSC|
|14||Is there an effective system of internal audit?||Internal audit||The score is 0 if there is no internal audit system; 2 if internal audits are functional for some entities and partially meet recognized international standards; and 4 if internal audits exist for all entities and generally meet international standards.||Budget Institutions Database, OECD, PEFAreports, and ROSC|
|4. Project Evaluation, Audit and Asset Management|
|15||Is ex-post evaluation of domestic projects routinely undertaken?||Evaluation Frequency||The score is 0 if ex-post evaluations or spending reviews are not routinely undertaken; 2 if there are post-completion spending reviews; and 4 if ex-post evaluations are routinely performed by the auditor general or the executive, and cover costs and benefits.||PIM case studies, PER, budget web-sites, World Bank reports?|
|16||Are external audits produced on at imely basis and scrutinized by the legislature?||External audit||The score is 0 if audits cover less than 50 percent of total central government expenditures, including capital investments; 2 if at least 50 percent or more of total central government expenditures are audited annually, including capital investments; and 4 if all expenditures, including capital investments, are audited and the full range of financial audits is in compliance with auditing standards.||Budget Institutions Database, PEFA reports and IBP|
|17||Does the government maintain an asset register or inventory of public sector property, equipment, vehicles, etc.?||Asset Register||The score is 0 if no there is no asset register; 2 if one exists but it is incomplete; and 4 if there is a complete and operational asset register.||PIM case studies, PERs, FAD TA reports, WB documents|
The data used in this study draw on the following main sources:
World Bank Public Investment Management Case Studies. These case studies developed out of the World Bank’s Diagnostic Framework for Assessing Public Investment Management. Countries were assessed among eight “must have” features of an efficient public investment system: Investment Guidance & Preliminary Screening; Formal Project Appraisal; Independent Review of Appraisal; Project Selection and Budgeting; Project Implementation; Project Adjustment; Facility Operation; and Project Evaluation. Twenty seven have had this diagnostic.
Public Expenditure and Financial Accountability (PEFA) assessments. The PEFA framework was developed between 2003 and 2005 as a joint undertaking of the World Bank, the European Commission, the U.K.’s Department for International Development (DFID), the Swiss State Secretariat for Economic Affairs, the Royal Norwegian Ministry of Foreign Affairs, the French Ministry of Foreign Affairs, and the IMF. Since 2005, the PEFA program conducts assessments, some of which are publicly available in the form of country reports, on the technical and institutional basis for sound budget governance covering a broad range of PFM performance indicators. It uses 28 indicators grouped in three areas: credibility of the budget; comprehensiveness and transparency and budget cycle. PEFA assessments are done every three years and cover 96 countries.
Budget Institutions Database. The database was developed by the IMF in 2010 to assess the various stages of budget institutions in low-income countries, and uses several indices and sub-indices to benchmark countries across income levels, regions, and different institutional arrangements. The index was constructed using 35 questions, covering two dimensions of the budgeting cycle: (1) planning and negotiation, approval, and implementation, and (2) the degree of centralization of budgetary decision-making; the existence and effectiveness of rules and controls; the sustainability and credibility of the budget as a key policy instrument; and its comprehensiveness and transparency. Seventy low-and-middle income countries are scored among these parameters.
OECD International Budget Practices and Procedures Database. The database, which was originally developed by the OECD, contains the results of the 2007 OECD survey of budget practices and procedures in OECD countries, the 2008 World Bank/OECD survey of budget practices and procedures in Asia and other regions, and the 2008 CABRI/OECD survey of budget practices and procedures in Africa. The database contains the results of surveys for the 30 OECD member countries and 67 developing countries from Africa, the Middle East, Eastern Europe, Asia, Latin America and the Caribbean. Questions cover most of the stages and several aspects of the budget cycle, including preparation, approval, execution, accounting and audit, and performance information. The questions are of the multiple-choice or check-the-box type.
Country Procurement Assessment Reports (CPAR). The diagnostic framework was developed in 1998 by the World Bank to analyze procurement policies, organization, and procedures in its member countries. These reports assess a country’s legal framework, procurement system organizational framework, procurement capacity building, system/institutions, procurement procedures/tools, decision-making and control system, anti-corruption initiatives and programs, private sector participation in the system, contract administration and management, and system for addressing complaints. To date, 112 reports have been completed for 93 countries.
Public Expenditure Reviews (PERs). PERs are core diagnostic studies prepared to help countries establish effective and transparent mechanisms to allocate and use available public resources in a way that promotes economic growth and helps in reducing poverty. As part of the World Bank’s country economic and sector work, PERs are undertaken to assist the Bank’s borrowers to understand their development problems and potential solutions as well as help illuminate the World Bank’s own country assistance strategy.
County Financial Accountability Assessments (CFAAs). CFAAs are a key diagnostic tool to describe the financial accountability arrangements in countries’ public and private sectors with the objective of strengthening these environments. CFAAs support both the exercise of the Bank’s fiduciary responsibilities and the achievement of its development objectives through assessing the strengths and weakness of countries’ accountability arrangements and identifying the risks that these may pose to the use of Bank and other public funds.
We thank Andy Berg, Hugh Bredenkamp, Paul Collier, Shanta Deverajan, Dominique Desruelle, Martine Guerguil, Sanjeev Gupta, Ricardo Hausmann, Daehaeng Kim, Stephen Knack, Aart Kraay, Lant Pritchett, Laure Radifer, Felipe Zanna, Kenichi Ueda, Duncan Last, Jean Pierre Nguenang and seminar participants at the IMF, the World Bank, the Public Investment Management Conference (Vietnam, 2010) and the Conference on Sustainable Investment Scaling Up In Low-Income Countries (Washington, 2010) for suggestions and comments. We gratefully acknowledge the help of many country economists at the World Bank and the IMF, too many to list them all, who assisted us with the data collection. Ke Wang provided excellent research assistance.
These include the Public Expenditure and Financial Accountability (PEFA) assessment framework and the OECD-World Bank budget database.
Outcome based indicators, such as mainline faults per 100 telephone calls for telecommunications, electricity generation losses as a percent of total electricity output, the percentage of paved roads in good condition, are commonly used in the literature to assess the quality of investment.
The massive infrastructure deficit in low-income countries, particularly in Sub-Saharan Africa, is viewed as a key bottleneck to achieving sustained growth (Commission on Growth and Development, 2008).
In this paper, we focus on pressing needs in physical infrastructure which is often a key input into social infrastructure and human development (Fay et al. 2005).
Similarly, in the Latin American region as a whole, the World Bank (2005) estimates that about 4 to 6 percent of GDP in capital spending per year is needed to catch up to countries that in 1980 had lower stocks of infrastructure, such as South Korea and China (see Fay and Morrisson, 2007).
See Romp and de Haan (2008) and Straub (2008a, b) for a survey of this literature.
The analytical framework developed by Rajaram et al. (2010) describes eight preferred or minimum features of a sound public investment system.
In low-income countries, the Poverty Reduction Strategy paper (PRSP) may serve as such a document (Rajaram et al., 2010).
The value of ex ante project evaluation depends on the quality of the analysis and capacity of staff with project evaluation skills; it has not proved possible to include an easy measure of such capacities.
There is a longstanding debate over the degree of emphasis on more complex and demanding techniques, such as cost-benefit and cost-effectiveness analysis (see Collier 2010, and Collier et. al, 2008). In many low- income countries, with weak technical capacity to undertake such assessments, the more basic and narrower elements of economic appraisal – such as whether there is a clear need for the project, clear and measurable objectives, consideration and analysis of options, including alternative options, could be more important (Glenday, 2010).
Medium-term budgeting frameworks (MTBFs) can play an important role in ensuring aggregate fiscal discipline while acting as a bridge between the goals and objectives of medium-term country developmentstrategies and the annual budget process. However, in the absence of basic budget institutions and adequate capacity they may not achieve their objectives. See, for example, Schiavo-Campo (1999).
Ideally, sector ministries and the ministry of finance should review forward costs of investment projects and their funding during budget preparation. However, budgeting for public investment remains poorly integrated into the formal budget preparation process in many countries (see Allen and Tommasi, 2001; and Webber 2007).
It should be noted, however, that the legislature itself could be a source of non-appraised projects. For instance, depending on the nature of constitutional and organic budget law discretion, some legislatures may have virtually unrestricted authority to substitute or add to the proposed projects to be funded in the budget.
While it is important to provide for a legal complaints mechanism in the procurement process to encourage public and legal scrutiny, there could be a danger of creating a litigious environment, where one supplier uses the threat of legal challenge to exert influence.
In an environment of considerable capacity, operating agencies should compile balance sheets, on which the value of assets created through new fixed capital expenditure, purchases or bequests is maintained (IMF 2001). The GFS manual identifies general-purpose assets (schools, office buildings etc.); infrastructure assets (highways, communication networks etc.), and heritage assets (historic, artistic significance etc.).
Even in aid-dependent countries, where there is reliance on donors to undertake reviews and evaluations of their projects, it is often the case that little systematic use is made of findings from donor evaluations to improve future project design and implementation (Petrie, 2010).
The World Bank has recently engaged in more than 27 in-depth PIM diagnostic assessments (see Petrie 2010). These have typically involved significant diagnostic efforts but also varying degrees of scope.
The sub-CPIA index is based on the 5 sub-components, two that are policy related (debt and fiscal) and three that capture the quality of public financial management environment (budget and financial management, public administration and transparency, and accountability and corruption in the public sector. Together with the PEFA reports, the sub-CPIA index is an important input into the capacity assessment process that establishes debt limits in Fund-supported programs (see, “Debt Limits in Fund-Supported Programs—Proposed New Guidelines,” SM/09/215).
A complete set of correlation matrices; correlations among the indicators in each sub-index, correlations among all 17 indicators, and correlations among the sub-indices, is available from the authors upon request.
Principal component analysis (PCA) transforms correlated variables into a smaller number of uncorrelated variables called principal components ranked according to how much variability they capture in the data (e.g. the first principal component is the one with most variability, and each succeeding component accounts for as much of the remaining variability as possible).