The Use of Performance Information in Resource Allocation1
The role of performance information in budget management is growing in advanced economies. In Italy, performance information will be critical for making the Spending Review a permanent part of the budget process. Italy generates significant performance information but there is less evidence that this is actively used by key budget decision makers. High quality performance information needs to be used more proactively to evaluate public policy and inform resource allocation decisions.
1. The role of performance information (PI) in budget management is growing in advanced economies and likely to increase in the future. Performance information (PI) is quantitative data on actual or expected results of government policies and programs, associated with data on incurred costs. PI on activities, results, outputs or outcomes are non-financial, while PI on inputs and costs are mostly financial. Nonfinancial PI may take different forms such as indicators, benchmarks, ratings, rankings, league tables, or scores. It allows assessment of effectiveness and efficiency of public policies and may be used in a vast array of budget or non-budget environments. More than 80 percent of OECD countries are now using PI to support budget management. This goes hand in hand with the development of program and multiyear budgeting as well as with public policy evaluations and spending reviews. With shrinking fiscal space, demand is growing for PI in order to strengthen the allocation process and identify saving opportunities.
2. Establishing a link between resources and results, however, is the challenge. In that regard, this resources-to-results link appears to be easier to establish at the micro level than at macro level. Usage of PI in budgeting process is contentious but, under certain conditions, possible. Italy, with its recent and robust performance management system, could use more effectively PI in its budget allocation process. Accordingly, this paper seeks to: (i) describe the international practices in terms of PI usage at different levels of resource allocation; (ii) discuss challenges of integrating PI into the budget process; and (iii) assess the Italian situation in this regard and make some recommendations.
B. International Practices and Lessons
3. International experience shows that the actual use of PI in allocation processes depends on its level of aggregation, from the macro to the micro (see Figure 1). The main lesson from international experience is simple: the lower the level of aggregation, the tighter the resources-to-results link that can be established. Therefore, the higher the level, the less PI is usable in the allocation process.
Figure 1.PI Usage at Different Allocation Levels
4. At the macro level, advanced countries tend to use PI with caution. At the level of policy evaluation and formulation, the horizons are long (medium to long term) and the policy segmentation is usually wide: 10 “functions” of government (COFOG), or 30 to 50 policy areas, as in Sweden, the United Kingdom, and France (35 “missions”). This large segmentation erases institutional subsector borders between fiscal players such as central or local governments. The expected results are typically outcomes. Efficiency analysis associated with benchmarks—both in space (between countries or agencies) and through time—is a good way to provide a rough assessment of public policy efficiency. However, while these quantitative exercises at the macro level are necessary they are not sufficient to set long-term budget priorities. In this respect, it is recognized that the use of PI is more relevant for intra sectoral prioritization than for inter sectoral prioritization: PI is helpful to choose between different education programs or between different health programs but not for prioritizing defense over justice or sport over agriculture. Concrete examples of usage of PI in allocation process at macro level are:
Public policy planning which requires the use of many PI, be it the traditional economic planning (still in use in some countries with comprehensive and centralized planning tradition such as Mexico and Turkey) or the ministerial/sectoral medium-term strategies that are commonplace in many EU countries (France, Sweden). The same goes for public policy evaluations which have enjoyed a recent resurgence in new forms such as strategic reviews (Australia, New Zealand, and United Kingdom). However, in these types of exercise, the resources-to-results link is quite loose and, frequently, strategic planning and resource allocation are two parallel processes, following separated procedures and steered by different institutions.
As it comes to the first step of spending reviews (SR),2 PI may help mapping possible “saving areas” across General Goverment expenditure (via matrices, crossing functional classification with economical classification and/or with institutional classification) and support policy makers design their saving strategy and priorities.
Medium-term expenditure framework (MTEF), which sets out medium-term budget priorities, may match these with indicators on expected results (be it outputs or outcomes), with medium-term target values.
5. At the intermediate level, PI is more commonly used and tighter forms of resources-to-results link are observed. At the budgeting level, be it at the executive preparation or parliamentary approval stage, the time horizon is shorter (one year, sometime two)3 and the policy segmentation is narrower: some hundreds of programs and/ or spending agencies. Here, institutional fragmentation appears to be an inescapable constraint: every level of government (local, social security and central) prepares its own budget.4 The expected results are outputs, often associated with outcomes. Hence, the rationale for a stronger resources-to-results link is more robust and usage of PI is easier than at the macro level, even if it is far from having replaced the classical incremental or legalistic budgeting methods. Concrete examples of usage of PI in allocation process at the intermediate level are:
The budget cycle starts with an assessment of the past spending (during past and current fiscal year) taking stock of the actual outlays and their associated results, in term of activities and outputs; this is useful to define the baseline estimates as well as to clarify the metrics of possible evolution, expansion or downsizing of current programs.
In the second step of spending reviews (see above), PI can also be useful for the definition and calibration of detailed savings.
The role of formula funding appears to be more and more important. Under this method, the amount of resources is calculated in using price and quantity equations, where (i) price may be standard costs (based on historical trends or normative patterns) or benefit scales; and (ii) quantities may be the number of outputs, activities or eligible beneficiaries. This type of formula funding provides transparent and ex ante mandatory rules, warranting predictability and limiting discretion. It is widely used in decentralized environments, namely in specific areas such as education and health, where central budget authorities have to fund a great number of agencies or local authorities, performing similar activities or delivering identical services.
6. At the micro level, PI is most extensively used. In this case, the time horizons are quite short, quarters or even months, and segmentation of delivery units is narrow. The internal allocation process within spending ministries, both at preparation and execution stages, involves a large use of PI on outputs, activities and costs and the resources-to-results link is widely applicable. In addition, PI helps support transparency and accountability arrangements that are needed in a decentralized environment where more flexibility and managerial discretion is granted as a condition for more demanding performance. Concrete examples of usage of PI in allocation process at the micro level are:
Performance contracts, or quasi contracts, which are in use in many countries (Australia, France, Sweden, United Kingdom and United States) as a managerial tool, linked to budget programs. Aimed at implementing programs, these contracts set out precise service delivery targets, in terms of outputs and/or activities and allocate budget appropriations accordingly.
Management control: directly inspired from the business sector, heads of public agencies make increasingly systematic use of PI to monitor the day-to-day operations and adapt resource allocation to the changing environment.
HR performance management: in limited areas such as the senior staff of some service delivery agencies, some countries have attempted to develop “performance pay” schemes. In these cases, a part of the compensation package is linked to the agency performance. However this type of arrangement, where the resources-to-results link is applied on an individual basis, remains infrequent, for technical as well socio-cultural reasons.
7. Some policy domains are more suited to PI than others. Education, health, social entitlements, police, or urban transportation are fertile areas for PI development, while justice, foreign affairs, or defense are considered less appropriate. Regulatory or control activities are also less suited to that type of approach and more discretionary resource allocation methods remain in place. Anywhere where a production function appears easy to formulate, especially in repetitive activities, using PI and resources-to-results links is relevant. Modular and progressive implementation of program budgeting is more desirable than an overnight systemic reform.
8. It is, nonetheless, important to keep in mind that most of PI is a proxy for public policy results. Result indicators, namely outcome indicators, are not the results themselves. So, when the actual value of an indicator diverges from the targeted path, the right reaction is to conduct further examination in order to understand the meaning and causes of this divergence, and then take relevant actions. Any mechanistic use of indicators should be avoided. Indicators should be taken as warning signs aimed at drawing policymakers and managers’ attention. Caution is warranted especially when indicators are deemed to trigger automatic rewards or sanctions. The PI system sometimes “hits the target but misses the point” (Hood, 2006).
9. Information asymmetry is one of the major shortcomings of PI. A great deal of indicators, namely on outputs, activities and inputs are generated by the delivery agencies, and hence are under their control. In the same vein, all players concerned by PI will tend to focus their attention and action on indicators’ performance targets, and neglect others. In some case, players will be tempted to manipulate the indicator itself as a substitute for actually improving underlying performance. This is summarized in Goodhart’s law: “a measure that becomes a target may thereby cease to be a good measure”. To mitigate possible distorting effects of PI, attention has to be paid to their incentive effects. Furthermore, independent control of data reliability and relevancy is needed. The selection, production and use of PI are sometimes overseen by independent bodies: either the general audit body like in Australia, New Zealand and the United Kingdom, or a specific administrative body as the French CIAP.5 In the United States, a law on PI quality was passed in 2000.
10. A cost-benefit analysis is needed to avoid PI overload. There is a tendency to multiply the number of indicators with the illusory hope of having a better reflection of the actual results and impacts. But too much data can also obscure issues. A limited and careful choice of PI is desirable. This selective approach to PI is not only necessary to prevent information congestion but also to spare resources: collecting PI is costly, especially on outcomes where data are outside the delivery agencies, which require costly panels, surveys, or polls to collect. A cost-benefit analysis should be made prior to embarking on an ambitious PI framework. Any PI which appears to be not actually used—that is triggering no decision—should be abandoned.
11. Performance does not improve overnight. Public sector managers need to reconfigure services, redeploy staff, and experiment with new modes of delivery. PI regimes therefore need to be linked to multi-year resource allocations which build in efficient savings not necessarily immediately but over time.
C. Strengthening the Use of Performance Information in the Budget Process
12. Usage of PI in budgeting at the national level is progressing only gradually. International experience shows that PI exists in national budget documentation, but its actual usage to directly determine the amount of public resources to be allocated to a given policy goal is limited. In most countries, several non-performance considerations are taken into account in the budget negotiation process: spending history, incrementalism, legal constraints, balance of power between Cabinet ministers, and political considerations. Even countries deeply engaged in performance budgeting recognize that PI often plays a secondary role in budget decisions and prefer to use the term of “performance informed budgeting” as opposed to “direct performance budgeting.”
13. Cultural reasons and conservatism may also explain this limited success of performance budgeting. Budgeters tend to still maintain their focus on inputs which is their historical core responsibility and skill. They often seem to fear to be drawn on the results field, where they are in a weak position since arguments for expenditure increase are many and often powerful. Spending ministry officials consider also that goals and results are their job and they reluctantly accept that Ministry of Finance officials address this type of issues. Beyond that, two more substantial types of reason are to be considered: methodological challenges are not always properly addressed and policymaker involvement remains weak.
14. A first methodological difficulty arises from attribution issues. The chain of causation from resources to outcomes is complex. A given outcome may result from several factors and the amount of public financial resources allocated is just one amongst many others. Not only is the list of these external factors long—private spending, other policy instruments, natural, and cultural environment—but the weighting of their relative contribution is very challenging (Annex 2). Most of these attribution issues are located in the output/outcomes link.
15. Another technical challenge is to have a full understanding of the value chain linking expenditure to results. It is necessary to analyze the several steps that transform financial resources into results, be it outcomes or outputs. In this value chain, inefficiencies may occur at every step. The longer the chain, the more difficult it is to understand. For instance, if an improvement of health outcomes is considered a desirable policy goal, a correlated increase of resources may not yield the expected results for several reasons: input prices may increase under the effect of a higher public demand, technical inefficiencies may worsen because of congestion issues and, the outputs mix deemed to contribute to outcomes may prove irrelevant. One cannot expect any strict proportionality between input changes and outcome changes, neither upward, nor downward.
16. Finally, policymakers are often not sufficiently involved in the choice of PI in budgeting activities. The PI chain should start from the top. The determination of desired outcomes is often contentious and the choice of indicators is a sensitive issue that depends on the goals assigned to a given policy. These goals are in politicians’ remit. PI cannot be developed only in experts’ circles, public managers and academics, without the involvement of ministers, parliamentarians, and citizens. Budgeting is a political activity: allocating resources to public policies implies setting priorities and making hard choices. Policymakers need to take responsibility: define clear and explicit policy goals and the PI that goes with them. This is an essential prerequisite to the development of any PI framework and performance budgeting system.
17. If properly addressed, these difficulties can be overcome. First, countries looking for more performance-oriented budgeting system should focus initially on outputs. This is for two reasons: (i) there are fewer attribution issues in the resources/outputs link than in the resources/outcomes link; and (ii) outputs are easily computable, via the costing of activities that yield them. Having not the same qualities, outcomes cannot be taken as a direct reference for budgeting. However, a gradual shift from outputs to outcomes can be contemplated over time as data improves and the relationship between resources, outputs, and outcomes becomes better understood. Second, there should be an explicit determination of policy goals without which neither PI framework nor performance budgeting may function. That underlines the need for political implication. In this respect, experience and realism suggest to select limited issues on which politicians’ attention might be solicited: setting clear goals and budget priorities. A greater personal implication of ministers in performance budgeting and management often appears illusory, given politicians’ agendas and timetables.
D. Performance Information in Italy
18. Italy is already producing and using a large array of PI. Beside academics and think tank research, PI is now widely in use in public administration. The introduction of significant public management and performance budgeting reforms, in two laws in 2009, has led to the production of many performance indicators in every policy area and the establishment of a public data bank. Local authorities have followed suit and many local budgets are also using PI. The strong decentralization movement has also spurred more systematic usage of PI to determine the financial transfers (and tax sharing arrangements) from central government to regions and municipalities.
19. The use of PI in budgeting is supported by a strong methodological and legal framework. The Ragioneria Generale dello Stato (RGS) has invested considerable time in studying the lessons from international experience, and the Italian performance budgeting approach seems to be in line with the best international practice. A September 2012 decree defines state-of-the-art indicators that are now an important component of a new type of budget documentation known as “integrative notes” describing the objectives and resources of government “missions” and “programs.” These “integrative notes” are produced in two successive versions: appended to each ministerial draft budget, the “ex ante” integrative notes describe the goals and the appropriated resources while ex post “integrative notes”—in the same format than the “ex ante” ones but appended to the budget reporting law—give an account on the actual use of resources and the achieved results. The same law also stipulates that every spending ministry should create an “evaluation unit” and develop training courses on spending analysis and performance measurement.
20. However, there is little indication that this performance information is being used by the main stake holders of the budgeting process. They are not used in the executive phase of budget preparation: the budget requests are not based on performance information but on legal requirements and other obligations. Consequently, the budget negotiations and the final Cabinet decisions are mainly driven by legal and political considerations. The parliamentary debate and final vote are carried out on the same ground: parliamentarians and their committees pay little attention to the “integrative notes” and the PI they contain. The same goes for the media which do not provide an account of the contents of the integrative notes and their PI. There is also little evidence that citizens, political parties and NGOs are taking this information into account in their discussions with policymakers. Budgeting and PI appear to be two parallel processes, the former leading to actual decisions, the latter producing formal documents.
21. Part of the reason for this may be that most of the PI are inward-oriented and bureaucratic in nature. They tend to focus on the internal functioning of governmental administrations which is of limited interest for policymakers, taxpayers, beneficiaries and their representatives. For example, one of the most frequent indicators is about the rate of budget execution. Another example is the number of meetings held or the number of administrative plans issued. Outcomes indicators, be they ultimate outcomes or intermediary outcomes, are very rare. So-called “output” indicators are more frequent, but many of them are obscure: some measure the degree of implementation of quarterly plans but no information is given on the content and ambition of these plans. Finally, the PI framework is mainly a bottom-up process: indicators are proposed and managed by frontline services and then compiled at the upper level. Top-down guidance from political level is very limited.
22. Another reason for the lack of attention paid may be that no linkage between resources and results is established in the “integrative notes.” Though these notes are made of two sections—the first one on programs, goals, and indicators and the second on resources calculation—there is no link made between the two. The second section shows that the criteria and methods of resource allocation focus on compliance to spending laws: historical, incremental or automatic budgeting seems to be prevailing.
23. A final reason for the lack of public attention be paid to nonfinancial performance, is the absence of any accountability arrangements for PI. The “integrative notes” do not identify the agencies in charge of each program or the name of the senior officials responsible for their management. Reporting requirements seem limited. There is no ex post assessment of results achieved or the effectiveness and efficiency of the use of resources in doing so. No incentives mechanisms exist linking the achievement of results to further resources. The performance-based compensation system for public services established under the 2009 legislation is not operational.
24. Politicians, ministers, as well as parliamentarians, should be involved in the design of the strategic components of the performance budgeting framework. One of their main responsibilities in that regard would be to set explicit policy goals for each “program.” Clear policy goals are the keystone of any PI framework and one of their main advantages. They require politicians to shift from vague, uncertain, implicit objectives to more precise and more demanding goals. The second strategic domain in which politicians should be involved is the choice of policy instruments they consider relevant to reach these policy goals: the output mix they deem necessary as well as the regulatory framework. Some procedures and documents of that type already exist in Italy. But they are complex, too detailed and preformatted: hence, after some years of experience, this appears to be a purely formal exercise without any real impact on policy making or management. Possible ways of reviving this exercise might comprise: (i) dramatically shortening the documents with a narrower focus on a few political issues; (ii) presenting options, with risk analysis, and not just routine ratifications; (iii) organizing a real, personal, and direct discussion between the top senior officials and their Minister; (iv) widely publicizing the final ministerial decisions inside and outside the Ministry; and (v) presenting this to the relevant Parliament committees. Thus, the present bottom-up PI framework would be reversed and replaced by a top-down process starting from high-level political goals, desegregated downwards in cascading layers.
25. The “integrative notes” should be combined with reports on ministerial activities prescribed by the 2009 laws. These two exercises, deriving from two laws adopted in 2009, are similar in term of objectives and content. Their ultimate aims are the same: transparency, performance, and accountability. Both are setting goals and describing activities of a given ministry. Both are sent each year to the Parliament. However they are produced by different channels and they demand a large amount of work in term of data collection and coordination. Efforts have already been made to better coordinate these two documents but performance indicators and costing methods are still different. For the sake of consistency and cost-efficiency, the best solution would be to merge the two documents. As a second best, a provisional solution might be to have a common production process and staff, based on common doctrine, method and data but to keep two separate documents.
26. Results indicators, primarily on outputs, should take a larger role. Output is the link between the two spheres of responsibility: the political one which decides outputs’ type and level and the managerial one which is responsible for its delivery. Moreover, outputs are easy to identify for they are tangible, the value chain between resources and outputs is quite easy to control and output indicators are cheap to produce since most of them are deriving from the delivery process. Other countries that have embarked up performance-based budgeting have typically started with output-based targets and then gradually migrated toward a more outcome-based approach. For Italy, the “integrative notes” should systematically comprise at least one output indicator reflecting the actual service provided to the final client (citizen, user/ beneficiary, or businesses).
27. The cost of delivering those outputs should be fully addressed in the “integrative notes.” The division of “integrative notes” into two separate sections should be reconsidered in order to allow a closer link between resources and results. Costing methods should be focused on “activities” that are producing the desired outputs: this might be the more convenient option, provided the main indirect costs (overhead, support function, etc.) are taken into consideration. Thus the allocation of resources in the budgeting process would be based on economical rationale rather than the current legalistic approach which refers to the amounts stipulated in the various ordinary laws. This may lead to a revision of the role of ordinary laws in spending matters with the view of giving a stronger authority to the budget laws.
28. Service delivery areas such as health, education, police as well as social entitlements or transfers to local authorities should be considered as a priority. These areas are well suited to the metrics of performance budgeting, indicators and costing. They also represent the bulk of public spending. Other policy areas such as foreign affairs are less adapted, for technical and cultural reasons: the amounts at stake are limited and resources allocation might use more classic methods. Therefore, improvement of “integrative notes” might be progressive starting with those ministries in which the conditions for success are most likely.
29. Local authorities should be encouraged to adopt similar performance budgeting arrangements and PI frameworks. Several local budgets are already using this type of approach. While no common mandatory method is advisable, it would be good to encourage cooperation and exchange of experience in creating forums and networks. However, greater attention to PI coordination seems to be necessary in policy areas that are common to central and local budgets (such as economic affairs and education). PI central/local coordination would also be helpful to inform the discussion on budget transfers to local authorities.
30. Quality control and independent scrutiny of PI should be supported by some procedural and institutional arrangements. Different options might be considered: the RGS could play this role from the center of government. A more participatory option would be to establish an inter-ministerial committee in charge of validating the PI designs in different ministries and agencies. Another solution might be to entrust the Court of Account (Corte dei Conti) with this task.
31. The choice, definition, and monitoring of PI should also engage the public. A first step in this direction would be communication: a limited number of major indicators, related to the political priorities of the day (competitiveness, youth employment, civil service) might be widely communicated and explained in the media. A more ambitious step would be participation: panels and polls (via internet, for instance) on public performance issues would allow citizens to give their opinion on public policies goals and results. “Citizen’s budget” projects, such as developed in several countries, might be encouraged. These initiatives might be another way to tie the Italian PI framework to the political agenda.
|Tertiary Education||Students taught||Number of degrees||Employability||Ratio students qualified/students with a job|
|Heart Health||Heart Surgeries||Number of successful heart surgeries||Life expectancy||Number of years of life after heart surgery|
|Poor Family Support||Family allowances||Number of allowances distributed||Fairness, Equity||Gini index|
|Youth Unemployment||Assistance to job seekers||Number of job seekers addressed||Youth Employability||Job seekers addressed/job seekers with a job|
What are the determinants of global health outcomes?
Health public spending is indeed often considered the first determinant. But there are many others, some under the direct control of the government, some others that are not, such as:
Health private spending, via co-payments or private provision and funding of health services, that are so entangled that it is practically impossible to attribute;
Many other public expenditures contribute to health, for example in the budgets of environment (better sewage), housing (better sanitation) or infrastructure (safer roads);
Other policy tools may contribute to achieve health goals: primarily, laws (on drugs, alcohol and tobacco use) that are sometimes more powerful tool than public spending; governmental communication may also play some role.
Several socio-cultural and environmental factors (climate, food habits, life behavior) outside of any direct government control.
AtkinsonAnthony2005 “Measurement of Government Output and Productivity for the National Accounts (United Kingdom: Palgrave Macmillan).
HoodChristopher2008 “Performance Measurement: Indicating the Way to Better Public Services” (Washington: World Bank).
HughesRichard2010 “Using Performance Information in the United Kingdom” in “Results Performance Budgeting and Trust in Government” (Washington: World Bank).
Manning and others 2006 “How and Why Should Government Activity be Measured” in Government at a Glance?” OECD (Paris: Organization for Economic Cooperation and Development).
PollittChristopher2005 “Performance Management in Practice: A Comparative Study of Executive Agencies” (Rotterdam: Erasmus University).
RobinsonMarc2013 “Spending Reviews” OECD (Paris: Organization for Economic Cooperation and Development).
RobinsonMarc2007 “Performance Budgeting” (New York: Palgrave Macmillan).
ScottGraham2008 “Some Observations on the Concepts and the State of the Art” in Performance Budgeting (Washington: World Bank).
Prepared by: Benoît Chevauchez (FAD)
In many countries, SR are a two-step process: the first one is aimed at targeting the possible savings fields, the second one is focused on identifying more precise savings, calculate their size and address implementation issues.
Such as in France with the 2+1 FY budget planning or as in the UK for the DEL expenditure.
And coordination of prioritization may be challenging.
Comité interministeriel d’audit des programmes.