Book
Share
Chapter

12 Performance Budgeting in England: Public Service Agreements

Author(s):
Marc Robinson
Published Date:
October 2007
Share
  • ShareShare
Show Summary Details
Author(s)
Peter C. Smith1

When it came to power in 1997, the Blair government sought to revolutionize the way in which UK public services are planned and delivered. In particular, it put in place mechanisms that sought:

  • to improve the way in which priorities are set between public services

  • to focus the actions of specific public services on explicit objectives and targets

  • to transmit national objectives clearly to local service providers

  • to improve coordination and performance of public services.

To this end, the principal initiatives that have been introduced are:

  • a system of explicit objectives and measurable national targets for government ministries, in the form of what are known as Public Service Agreements (PSAs)

  • a high-profile “delivery unit” charged with monitoring progress against PSA targets and where necessary demanding corrective action

  • a range of instruments designed by individual ministries to encourage local service providers to meet the national PSA targets.

The PSA system is a major departure for UK public services, seeking to introduce a level of coherence and transparency to policy-making that had never been attempted hitherto. It is a major test-bed for the principles of performance budgeting, and has without question delivered many benefits in the planning and delivery of public services. However, it has also led to serious tensions within some of those services, and exposed some major difficulties that arise in the implementation of performance budgeting. This chapter first summarizes the history of PSAs, and illustrates with an example from the health ministry. It then discusses some of the major issues that have arisen in seeking to implement the PSA regime, and assesses its effectiveness. The chapter ends with some comments on the relationship between PSAs and performance budgeting.

Background

The United Kingdom has traditionally been a highly centralized state, with a strong central government and circumscribed local autonomy. Since 1999 there has been some devolution of powers to national governments in Scotland and Wales, leading to increased divergence in policies. However, strong central control has remained a feature of English government, where weak regional and local government offer little counterbalance to national powers. Throughout this chapter, the discussion relates to performance budgeting systems put in place in England, which accounts for 84 percent of the population of the United Kingdom.

Responsibility for financial management of public expenditure rests with the finance ministry (known as Her Majesty’s Treasury), led within the government by the Chancellor of the Exchequer (who throughout the period under discussion was Gordon Brown). Traditionally, other government departments negotiated annual budgets with the Treasury. With the exception of some “demand led” spending (such as welfare payments) there has been a strong culture of adherence to such budgets within English government.

The Blair government came to power in 1997 with a commitment to evidence-based policy, to systematic priority-setting, and to explicit performance targets throughout the public services. This led Gordon Brown to undertake in 1998 a Comprehensive Spending Review (CSR) that set three-year budgets in advance for government departments. The break with annual budgets was a new departure, and was intended to offer departments more medium-term certainty within which to plan reforms. After the budgetary agreements were concluded, a set of Public Service Agreements with each department was announced, intended to signal priorities across the entire range of government activity. These were expressed in the form of about 600 specific objectives, many of which were expressed as a target in measurable form, and expected to be achieved within designated timeframes. (Access to these and all subsequent Treasury documents can be secured through <http://www.hm-treasury.gov.uk/spending_review/spend_index.cfm>.)

A distinctive feature of PSAs was the intention to focus on the outcomes of the public services, rather than operational activities of public service delivery. Although this principle was pursued as an ambition, for many departments the initial targets related more to processes, procedures or outputs. For example, a target for the Ministry of Defence was to “create new Joint Rapid Reaction Forces, which will be fully operational by October 2001.” A second feature of the PSAs was the incorporation within each of a productivity target, intended to signal the high priority given to value for money across all government activity. For example, an efficiency target of 2 percent per year was set for police services, and a system of annual efficiency plans introduced.

The PSA process signaled the Blair government’s determination to make the management of public services more transparent, and to give departments clear statements of priorities. However, in this first round, the detail, specificity and measurability of the PSA targets were highly variable. For example, the Home Office had a very specific target by 2001 to halve the time from arrest to sentence for persistent young offenders, alongside a much vaguer target to “improve victims’ and witnesses’ satisfaction with their treatment by the criminal justice system,” for which there was no baseline or measurement instrument specified. It was, moreover, also clear that the number of targets was excessive, seriously diminishing their impact.

The intention was to review PSAs biennially, so the first set of PSAs was reviewed two years later in the government’s first Spending Review (SR2000). Several principles were applied in the review, including a commitment to focusing on only the government’s key priorities, to moving towards a more widespread emphasis on outcomes rather than processes or outputs, and to adopting a longer time horizon. This resulted in a marked reduction in targets (from 600 to about 160), covering a reduced number of departments, and the introduction of four “cross-cutting” PSAs that introduced joint targets across departments.

Moreover, the published documentation associated with the 2000 PSAs became more systematic and uniform. The new format included for each ministry: an overarching aim; a set of broad objectives; specific, measurable targets associated with each target; a value for money (efficiency) target; and a statement of who was responsible for delivery (usually the relevant minister). The PSAs were supported by a series of Technical Notes giving detailed technical specifications of each target, including its formal definition and the measurement instrument to be used to judge success. In addition, a more operational Service Delivery Agreement was developed for each department, giving details of how the PSAs would be delivered.

The apparent reduction in the number of targets was to some extent illusory. The Service Delivery Agreements included many lower-level operational targets, intended to support planning and implementation associated with the departmental PSA. Moreover, although the Spending Review covered the usual three-year planning horizon, some of the targets now covered much longer periods. For example, the Ministry of Agriculture, Fisheries and Food, was asked to “reverse the long term decline in the number of farmland birds by 2020, as measured annually against underlying trends.” This highlights an important emerging feature of PSAs: that they may relate to time periods well beyond the budgetary time horizon. Progress towards such targets nevertheless continued to be monitored, to determine whether the department remained on track.

The next Spending Review occurred in 2002 (SR2002). The stated number of targets was further reduced from 160 to 130, and there was much improved continuity in their scope and definition. An important new development was the increased direct interest of the Prime Minister in the PSA process, reflecting the high profile given to the delivery of public services in his successful 2001 election campaign. This was manifest in the creation of the Prime Minister’s Delivery Unit (PMDU), with the objective of assuring progress towards key PSA targets, especially in health, education, crime, and transport. The PMDU seeks to “improve public services by working with departments to help them meet their PSA targets consistent with fiscal rules” (<http://www.cabinetoffice.gov.uk/pmdu/>). In doing so, it offers challenge and practical guidance to the relevant departments and provides regular performance updates directly to the Prime Minister.

The SR2004 offered further consolidation and refinement of the principles developed for the PSA regime. A key development was the abandonment of the value for money (VFM) targets. These had sometimes given rise to difficulties, such as contradicting a department’s other PSA targets (an issue discussed further below). Abolition of VFM targets was made possible by an Independent Review of Public Sector Efficiency, led by Sir Peter Gershon, which calculated (in £millions) concrete “efficiency” savings to be delivered by each department (Gershon, 2004). The review made many detailed recommendations on the scope for efficiency improvements, and the Office for Government Commerce became responsible for ensuring the efficiency programme is implemented. Another review (led by Sir Tony Atkinson) advocated new approaches towards measuring the productivity of government services in the national accounts, which are now being implemented by the Office for National Statistics (Atkinson, 2005).

The number of stated targets was further reduced to 126. Again, to some extent this reduction was illusory. Some previous targets were replaced by PSA “standards.” These indicated a continued level of performance that was required once a previous PSA target had been achieved. For example, the Department of Trade and Industry was given a standard to “maintain the UK’s standing as one of the best places in the world for online business.” Performance against standards continued to be monitored, but there was no requirement for further improvement. The PMDU rendered the detailed service delivery agreements largely redundant, and they were abandoned in SR2004.

At the time of writing, a second Comprehensive Spending Review is under way, which will report in 2007.

Case study: PSAs for the Department of Health

In order to illustrate the issues that emerge in setting departmental PSA targets, this section examines experience with the Department of Health. In common with other government ministries, the department has been set strategic targets as part of the PSA system since 1998. Its 2004 PSA targets are set out in Box 12.1. They are based on four broad objectives, as follows:

  1. improve the health of the population. By 2010 increase life expectancy at birth in England to 78.6 years for men and to 82.5 years for women

  2. improve health outcomes for people with long-term conditions

  3. improve access to services, in particular waiting times

  4. improve the patient and user experience

Detailed targets associated with the objectives are given in Box 12.1. At the bottom of the box, there are in addition four standards to be maintained, reflecting targets secured through previous PSAs that must continue to be achieved. A set of even more detailed Technical Notes accompanies the targets, giving the context, data sources, and measurement instruments. An example showing the Technical Note for the obesity target is given in Box 12.2.

Box 12.1.Department of Health Public Service Agreement targets, 2004

Objective I: Improve the health of the population. By 2010 increase life expectancy at birth in England to 78.6 years for men and to 82.5 years for women.

1. Substantially reduce mortality rates by 2010:

  • from heart disease and stroke and related diseases by at least 40% in people under 75, with at least a 40% reduction in the inequalities gap between the fifth of areas with the worst health and deprivation indicators and the population as a whole;

  • from cancer by at least 20% in people under 75, with a reduction in the inequalities gap of at least 6% between the fifth of areas with the worst health and deprivation indicators and the population as a whole; and

  • from suicide and undetermined injury by at least 20%.

2. Reduce health inequalities by 10% by 2010 as measured by infant mortality and life expectancy at birth.

3. Tackle the underlying determinants of ill health and health inequalities by:

  • reducing adult smoking rates to 21% or less by 2010, with a reduction in prevalence among routine and manual groups to 26% or less;

  • halting the year-on-year rise in obesity among children under 11 by 2010 in the context of a broader strategy to tackle obesity in the population as a whole; and

  • reducing the under-18 conception rate by 50% by 2010 as part of a broader strategy to improve sexual health.

Objective II: Improve health outcomes for people with long-term conditions.

4. To improve health outcomes for people with long-term conditions by offering a personalised care plan for vulnerable people most at risk; and to reduce emergency bed days by 5% by 2008, through improved care in primary care and community settings for people with long-term conditions.

Objective III: Improve access to services.

5. To ensure that by 2008 no one waits more than 18 weeks from GP referral to hospital treatment.

6. Increase the participation of problem drug users in drug treatment programmes by 100% by 2008 and increase year on year the proportion of users successfully sustaining or completing treatment programmes.

Objective IV: Improve the patient and user experience.

7. Secure sustained national improvements in NHS patient experience by 2008, as measured by independently validated surveys, ensuring that individuals are fully involved in decisions about their healthcare, including choice of provider.

8. Improve the quality of life and independence of vulnerable older people by supporting them to live in their own homes where possible by:

  • increasing the proportion of older people being supported to live in their own home by 1% annually in 2007 and 2008; and

  • increasing by 2008, the proportion of those supported intensively to live at home to 34% of the total of those being supported at home or in residential care.

Standards

  • A four hour maximum wait in Accident and Emergency from arrival to admission, transfer or discharge.

  • Guaranteed access to a primary care professional within 24 hours and to a primary care doctor within 48 hours.

  • Every hospital appointment booked for the convenience of the patient, making it easier for patients and their GPs to choose the hospital and consultant that best meets their needs.

  • Improve life outcomes of adults and children with mental health problems, by ensuring that all patients who need them have access to crisis services and a comprehensive Child and Adolescent Mental Health Service.

Source: HM Treasury (2004a).

Box 12.2.Example of a PSA Technical Note (SR 2002 joint Obesity Target for Department of Health and Department for Education and Skills)

PSA Target: Halting the year-on-year rise in obesity among children under 11 by 2010, in the context of a broader strategy to tackle obesity in the population as a whole

Scope: Children aged between 2 and 10 years (inclusive) in England.

Obesity: Prevalence of obesity as defined by the National BMI percentile classification (from the 1990 reference population from T J Cole et al) and measured through the Health Survey for England. Children above the 95th percentile of the 1990 reference curve are defined as obese.

Halt the year on year increase: Obesity in 2-10 year olds rose, on average, by 0.8% per year between 1995 and 2002. Halting the increase would mean no significant change in prevalence between the two three year periods 2005/06/07 and 2008/09/10.

Data source: Health Survey for England. We are also exploring with colleagues in DH and DfES the cost and feasibility of options for other sources of data in order to obtain more local level information.

Baseline year: Due to the small sample size, the baseline will be the weighted average for the three-year period 2002/03/04.

Target Year: By 31 December 2010, in practice this will mean 2010-11 financial year.

Reporting: Annually (aggregate trend data will be available every three years). The lag between the end of the collecting period and data being published is around 12-15 months.

OGD contributions to PSA: The delivery of this joint PSA target will be supported by a range of programmes including:

a) the joint DfES and DCMS PE, School Sport and Club Links project which seeks to increase the percentage of school children who spend a minimum of two hours each week on high quality PE and school sport within and beyond the curriculum;

b) the joint DfES and DH National Healthy Schools Programme which seeks to promote a whole school approach to healthy living;

c) the joint DfES and DH “Food in Schools” Programme which seeks to promote a whole school approach to a range of food issues.

Throughout the PSA regime, a central role of the Department of Health has been to devise operational instruments that transmit the national PSA targets to the local level. To this end, the most important initiative was the development of a system of “performance ratings” for individual National Health Service (NHS) organizations. Since 2001, every organization (including local health authorities and NHS providers of care) has been ranked annually on a four-point scale (zero to three stars) according to a series of about 40 performance indicators. The indicators are intended directly to reflect the objectives of the NHS, as embodied in the national PSA targets (Department of Health, 2001).

For each NHS organization, the indicators are combined according to a complex algorithm to produce the star rating. The most important determinant of an organization’s rating was its performance against a set of about ten “key indicators,” which hitherto have been dominated by measures of various aspects of patient waiting times. This was augmented by a composite measure of performance based on the 30 or so subsidiary indicators, combined in the form of a “balanced scorecard” view of the organization. Clinical quality comprised only a small element of the calculation. Since 2004, a new health care regulator, the Healthcare Commission, took over responsibility for preparing the star ratings from the health ministry. The performance indicators used to determine its ratings of acute hospitals in 2004 are shown in Box 12.3 (Healthcare Commission, 2004).

The most striking innovation associated with performance ratings was the introduction of very strong managerial incentives dependent on the level of attainment, which some commentators characterize as a regime of “terror” (Bevan and Hood, 2006a). The jobs of senior executives of poorly performing organizations came under severe threat, and the performance indicators (especially the key targets) became a prime focus of managerial attention. Rewards for performing well included some element of increased organizational autonomy. For example, the best performers in the acute hospital sector became eligible to apply for Foundation status, implying considerably greater autonomy from direct NHS control.

The response to performance ratings among NHS managers has been mixed. Many have criticized the system because of some of the apparently arbitrary way in which the ratings are calculated, and their sensitivity to small data fluctuations (Barker et al., 2004). However, there are some who acknowledge that the system does give managers better focus, and a real lever with which to affect organizational behaviour and clinical practice. Reaction among health care professionals has been less ambiguous, with a widespread view that they distort clinical priorities and undermine professional autonomy (Mannion et al., 2005). This is hardly surprising, as one of the aims of the national and local targets was precisely to challenge traditional clinical behavior, and to direct more attention to issues such as waiting times that had not always been a high priority.

There is, however, no doubt that performance ratings have delivered major improvements in the aspects of NHS care that they targeted (Bevan and Hood, 2006b), and have secured marked progress towards some of the PSA targets. For example, very long waits for non-urgent inpatient treatment, a prime focus of the PSA regime, have been rapidly eliminated. Moreover, compared to Wales and Scotland, which have not been subject to the PSA regime and have not implemented performance ratings, targeted aspects of English health care have improved markedly, even though funding levels are less than in the other two countries (Hauck and Street, forthcoming).

Less satisfactorily, the high-level PSAs shown in Box 12.1 include important public health targets under objective 1, such as reduced mortality rates from heart disease and cancer, reductions of health inequalities, and reduced rates of smoking, childhood obesity, and teenage pregnancy. Converting these high-level public health objectives into meaningful local targets through the medium of the performance ratings system has proved far less straightforward than in the waiting time domain. Public health has therefore not received anything like the sustained managerial attention enjoyed by the health service delivery targets (Marks and Hunter, 2005). There is therefore concern that local managers are concentrating on targeted and readily managed aspects of health care (most notably objective 3, waiting times) at the expense of less controllable and less immediate concerns, such as public health (objective 1).

Box 12.3.Indicators for constructing acute hospitals’ performance ratings, 2003-04

Key targets

12 hour wait standard for emergency inpatient admission

two week wait for first appointment for suspected cancer

achieving financial balance

satisfactory hospital cleanliness

improving working lives of staff

achieving targets for booking outpatient and elective appointments

outpatient waiting standards (21 weeks reducing to 17 weeks over course of the year)

non-emergency inpatient waiting standards (12 months reducing to 9 months over course of the year)

accident and emergency wait of 4 hours or less

Balanced scorecard indicators

Clinical focus

child protection

clinical governance composite indicator

clinical negligence

composite of participation in clinical audits

deaths following a heart bypass operation

deaths following selected non-elective surgical procedures

emergency readmission following discharge (adults)

emergency readmission following discharge for a fractured hip

indicator on stroke care

infection control

thrombolysis—30 minute door to needle time

infection control

Patient focus

A&E emergency admission waits (4 hours)

adult inpatient and young patient surveys: access and waiting

adult inpatient and young patient surveys: better information, more choice

adult inpatient and young patient surveys: building closer relationships

adult inpatient and young patient surveys: clean, comfortable, friendly place to be

adult inpatient and young patient surveys: safe, high-quality, coordinated care

better hospital food

breast cancer: 1 month diagnosis to treatment

breast cancer: 2 month GP urgent referral to treatment

cancelled operations

day case patient booking

delayed transfers of care

patient complaints

patients waiting longer than standard for revascularisation

six month inpatient waits thirteen week outpatient waits

Capacity and capability

implementing specialist appraisal

data quality on patient ethnic groups

information governance

junior doctors’ hours

staff opinion survey: health, safety and incidents

staff opinion survey: human resource management

staff opinion survey: staff attitudes

Source: Healthcare Commission, <http://ratings2004.healthcarecommission.org.uk/home.asp> (accessed April 26, 2005).

While retaining the principle of rating performance on a simple composite measure, it is noteworthy that the Healthcare Commission in 2006 implemented a major change to the assessment regime that pays more attention to a broader spectrum of performance, most notably clinical quality (Healthcare Commission, 2005). It places increased emphasis on self-reporting, and reports clinical performance and financial performance separately

Discussion

PSAs, and in particular the associated targets, have become a central element of political discourse in England. They have without question succeeded in shaping the priorities and delivery of public services, although whether that influence is for the good remains a matter of fierce debate. On the one side are those who claim that, by focusing on outcomes and stating firm measurable targets, they have helped to “modernize” public services. On the other side are those who claim that, through their simplistic view of priorities, they are undermining the traditional public service ethos and rendering those services dysfunctional.

PSA targets have certainly delivered noteworthy successes, such as the reduction in NHS waiting times, improvements in measured police performance, and focusing attention on numeracy and literacy in schools. However, alongside the manifest intended improvements in many of the measured PSA targets there are widespread reports of adverse side-effects in other, often unmeasured, aspects of public services (Bevan and Hood, 2006a). Many of these reports are anecdotal and may be apocryphal, but some have been credibly documented by the House of Commons Public Administration Select Committee (2003) and Bevan and Hood (2006b). Examples from health care include neglect of unmeasured aspects of performance (for example, clinical priorities being sacrificed in the pursuit of reduced waiting times), distorted behavior (such as refusing to admit patients to accident departments until a four-hour waiting time target was achievable), and fraud (manipulation of waiting lists).

Unintended and adverse responses such as these were readily predictable from the Soviet literature (Nove, 1980). They offer a powerful caution against a sole reliance on a targets regime for securing improvement, and the need to put in countervailing instruments where necessary (Smith, 1995). These might include: a strong national data audit and surveillance capacity; a system of professional inspection that monitors and reports on unintended consequences; careful scrutiny of performance beyond targets by organizational boards of governors; some sort of democratic “voice” in the control of local public service organizations; and empowerment of service users through improved information and systems of redress.

The Social Market Foundation (2005) summarized the criticisms of the targets regime under five headings:

  1. there are too many targets

  2. they are too rigid and undermine the morale of staff

  3. they have perverse and unintended consequences

  4. it is not always clear who is responsible for meeting the target

  5. the data are often not credible.

The history summarized above demonstrates the learning that has been necessary over its lifetime, and the need to change key elements of the PSA infrastructure in order to resolve such difficulties. This section summarizes some of the most important questions that have arisen in the development of the PSA system under eight headings: Who should choose the targets? What targets should be chosen? When should outcomes be used as a basis for targets? How should targets be measured and set? How should cross-departmental targets be handled? How should attainment be scrutinized? How should departmental objectives be transmitted to local organizations? How should efficiency be incorporated into the PSA regime?

Who should choose the targets?

In principle, it seems perfectly reasonable and indeed honorable for a legitimately elected government to set out its objectives and targets in the explicit fashion of the PSAs. The PSAs serve many purposes, but one is to enhance political accountability. Through the PSAs, the government can be held to account by parliament and the electorate, both for its choice of priorities, and for its performance against the targets. Indeed it is a sign of the success of the PSA process that much of the public debate surrounding targets now refers less to the principle of setting targets and more to the details of what those targets should be.

However, there remains disagreement about the processes by which priorities are chosen and targets are set. For example, many argue that the government’s excessive emphasis on waiting times in its targets for the National Health Service ignores the prime objective of health care, which is to improve health, and therefore poses a threat to clinical quality. Because of such outcomes, some have argued that the professionals delivering the public services should have a greater say in influencing the nature of the targets. There is an element of good sense in this principle, because the outcomes of many public services rely very heavily on the engagement and commitment of front-line professionals. And yet it is also the case that the priorities and working practices of those professionals can impede progress towards better performance. To some extent, the PSA process seeks to challenge traditional ways of delivering public services, and it will therefore at times come into conflict with the professions.

There are also those who argue that parliament should have a greater say in target setting. In this respect, a crucial role of parliament is to scrutinize the choice of priorities and the attainment of targets. However, it is difficult to see how the involvement of the legislature in choosing targets would enhance the PSA process. No government will pursue objectives with total commitment when it does not fully control their nature. (This principle also applies, of course, to the devolved organizations charged with delivering public services, and gives rise to some of the problems of morale and alienation discussed further below.)

It is also frequently suggested that service users should have more say in setting PSA targets. There is of course much to commend wide consultation with user groups in identifying priorities for improvement. However, setting objectives involves considerations beyond immediate users of a particular service, such as the taxpayer perspective, the interests of future users, and the interests of users of other services. The user perspective is therefore important, but cannot be the sole influence on priority setting, which in any case involves judgments about the relative importance of different user groups.

Thus, any prudent government seeking to implement a PSA-type process would be well-advised to consult many relevant stakeholders about the choice of objectives and the nature of the targets. However, uncritical accommodation of every interest group would render the PSA process meaningless, for example, leading to an unwieldy proliferation of priorities. A prime role of government is to balance conflicting claims on public resources, and PSAs should in the end be an explicit and succinct statement of the government’s choice in that respect.

What targets should be chosen?

Multiple objectives are a characteristic of public services—indeed it can be argued that the existence of multiple objectives is one of the defining characteristics of public services, and is one of the reasons why they cannot (at least in their entirety) be delivered by competitive markets.

However, one of the intentions of the PSA system is to focus on a limited number of objectives. This requires tough political choices to be taken. The 1998 suite of PSAs failed to recognize this, and therefore in many domains failed to have a detectable impact. Subsequent Spending Reviews addressed this issue by focusing on a greatly reduced number of targets. It is nevertheless important to note that some of the numerical reduction was deceptive—the health example above indicates how some targets became multidimensional, for example, seeking to address both overall health improvement and reductions in inequalities in health. Furthermore, as noted above, a number of previous targets have now been converted into standards, indicating a level of attainment secured in previous periods. To many, these retained the appearance of targets, albeit in a different guise.

It is noteworthy that, having identified a priority, the government has sought to include an associated objective into a department’s PSA, even when attainment is hard to measure. Indeed in recent Spending Reviews an attempt has been made to quantify attainment of even the most elusive objective, such as measuring improvements in “children’s communication, social and emotional development” through use of a new national survey instrument by the education ministry. Quantification is without question a good principle to pursue, as it will in general allow the government to set departments concrete targets. However, it does run the risk of distracting managerial attention from important qualitative aspects of performance, and suggests that reports of progress towards quantified targets should be accompanied by a narrative describing success and failure in more qualitative terms.

The move towards standards indicates that, as the Social Market Foundation suggests, targets should focus on domains where manifest change is required. If a domain is not included in the targets regime, this is not necessarily an indication that it is unimportant. Rather, it may suggest that it is not a priority for urgent change. The key focus of PSA targets should be where change is required, and maintenance of standards in other domains should be secured through other instruments, such as routine regulation, inspection, or market mechanisms.

More generally, it is noteworthy that successful ministries manage to maintain a balance between the pursuit of PSA targets and the maintenance of standards in other domains (Cabinet Office, 2006a). In contrast, the Home Office pursued its PSA targets with great vigor and some success, but was judged very adversely in a “capability review” that assessed its general fitness for purpose (Cabinet Office, 2006b).

When should outcomes be used as a basis for targets?

From the outset, the architects of the PSA system recognized that it is usually the outcomes of public services that matter to most service users and the broader public. In principle, the outcomes focus enables public service organizations to look beyond traditional ways of delivering their services and traditional organizational boundaries. It has had some marked successes in the English PSA system, such as a reorientation in police services from the traditional perspective of solving crime to a concern with crime prevention and reducing the fear of crime.

However, the focus on outcomes can give rise to difficulties. For example, some outcomes (such as wars averted by the Foreign Office) are intrinsically unmeasurable. Even if they can be measured, some outcomes (such as reduced mortality from smoking) can take years to materialize, beyond the lifetime of most governments. Furthermore, some outcomes (such as the alleviation of world poverty) are particularly vulnerable to influences beyond the control of the government department under scrutiny. Each of these difficulties offers the department an excuse for apparent failure, and can undermine the PSA process.

On the other hand, it is clear that the use of output measures can distort behaviour, and lead to unintended outcomes. For example, the Department for Education and Skills was asked in 2004 to “increase the stock of…registered childcare by 10%” by 2008. The real desired outcomes, as stated in SR2004 were, “supporting child development, removing barriers to parental employment and alleviating child poverty.” However, it is not at all clear that the chosen output target will necessarily address these issues. The target is readily subverted—for example, attainment might be secured merely by registering existing unregistered capacity. Or it might lead to perverse outcomes—the new places may be taken up predominantly by wealthy parents. Therefore, if the chosen output target is pursued without regard to the eventual outcomes, it may be ineffective. Additional assurance will be needed that the desired outcomes have indeed been secured. Yet if outcome data are available to provide that assurance, they should have formed the basis for the original target.

In short, outcome measures address what matters to the service user and the citizen, and are less vulnerable to distortion. It therefore seems unanswerable that outcomes should inform all PSA objectives. However, there will be occasions when a carefully chosen output or process measure—which evidence shows is clearly linked to the eventual outcome—may form a more effective basis for a target.

How should targets be measured and set?

A central feature of the PSA debate has been how—once objectives have been identified—the associated targets should be set, in terms of the measurement instrument to be used and the level of attainment to be required. A number of governmental agencies and regulators collaborated to advise departments on how they might set targets for their front-line delivery organizations (HM Treasury et al., 2001). Their report advocated the use of specific, measurable, achievable, relevant, and timed (SMART) targets, and the Treasury has sought to pursue these principles in the setting of PSA targets.

The Royal Statistical Society (Bird et al., 2005) put forward a more comprehensive set of desirable general principles for setting targets, as follows:

  • Indicators should be directly relevant to the primary objective, or be an obviously adequate proxy measure.

  • Definitions need to be precise but practicable.

  • Survey-based indicators, such as of user satisfaction, should use a shared methodology and common questions between institutions.

  • Indicators and definitions should be consistent over time.

  • Indicators and definitions should obviate, rather than create, perverse behaviors.

  • Indicators should be straightforward to interpret, avoiding ambiguity about whether the performance being monitored has improved or deteriorated.

  • Indicators that are not collected for the whole population should have sufficient coverage to ensure against misleading results; that is, potential bias compared to measuring the target population should be small.

  • Technical properties of the indicator should be adequate.

  • Indicators should have the statistical potential to exhibit or identify change within the intended timescale.

  • Indicators should be produced with appropriate frequency, disaggregation, and adjustment for context.

  • Indicators should conform to international standards if these exist.

  • Indicators should not impose an undue burden—in terms of cost, personnel, or intrusion—on those providing the information.

  • Measurement costs should be commensurate with the likely information gain.

The National Audit Office (2005, 2006a) scrutinized the data systems used to monitor and report progress against all PSA targets from SR2002, and found varying levels of success:

  • 30 percent were fit for purpose

  • 29 percent were broadly appropriate, but systems needed strengthening, such as improving controls over data collection and documentation, and improving checks on data obtained from external bodies

  • 18 percent were broadly appropriate, but disclosure needed strengthening to explain data limitations to the public

  • 12 percent were not fit for purpose, most commonly because of design problems, where the systems established did not measure adequately the aspects of performance included in the target

  • 6 percent were not yet established

  • for 5 percent it was too early to judge.

The Statistics Commission (2006) scrutinized all SR2004 targets in detail to assess whether the statistical evidence to support PSA targets was adequate for the purpose of achieving government policy objectives. Among its findings, it noted that:

  • there are some cases in which important technical aspects of targets are not explained in the Technical Notes

  • the complexity of some targets can become a serious obstacle to understanding and accountability

  • not all the data used to monitor targets were collected, managed, and published in line with the National Statistics Code of Practice

  • the transparency and independence of the qualitative assessments of performance were variable

  • long time-lags between the end of the target period and the data becoming available can be a serious weakness.

A number of approaches exist to overcome some of these weaknesses. For example, the Royal Statistical Society advocates adoption of a multi-stage measurement “protocol” for each target, that would explicitly explain all stages of the measurement process, from choice of indicator to publication of results. It also recommends publication of levels of uncertainty alongside all attainment measures. In the same vein, the Statistics Commission advocates publication of interim attainment measures for longer-term targets.

A particular feature of PSA targets is the specification of explicit levels of attainment to be achieved. This is an important element of the process, but it has been applied with inconsistent rigor. Some targets have been little more than unattainable aspirations, while others can be secured with little effort on the part of ministries. Furthermore, there are conflicting pressures within the targets regime. To be effective managerial instruments, targets should be stretching but attainable, suggesting (say) a one-in-three risk of failure. However, few governments would want to face parliament or the electorate with such a high proportion of failures. From an accountability perspective, a government would wish to feel there was a good chance of attaining all targets. It is difficult to see how this tension can be satisfactorily resolved, unless the political process becomes mature enough to recognize that some failure is inevitable and not necessarily adverse if progress is nevertheless being secured.

How should cross-departmental targets be handled?

A focus on outcomes sometimes gives rise to objectives that are not obviously attached to a particular ministry, leading to the need to specify “joint” targets that transcend departmental boundaries. These have given rise to particular difficulties in the PSA process. In a joint report, the National Audit Office and the Audit Commission (2006) have examined three particular joint targets requiring especially complex cross-departmental “delivery chains”: supplying affordable housing, promoting bus use, and halting the rise in child obesity. They find no ready solutions, but advocate much stronger collaboration between national and local government, and stronger engagement with non-governmental organizations. In short, such targets give rise to problems of coordination, persuasion and engagement that must be addressed if they are to be successful.

The Social Market Foundation recognized that some targets cannot be broken down into individual components, and therefore required a joint effort by two or more departments. However, it recommended that all targets should be unambiguously attributed to a single department, and that targets that must be delivered by more than one department should always have a “lead” department that takes responsibility for the meeting of the target. It is noteworthy that in launching the 2007 CSR, Gordon Brown announced that he would review areas that require cross-departmental collaboration, with a view to seeking innovative solutions for the challenges they pose.

How should attainment be scrutinized?

For a mechanism that has played such a central part in the recent development of English public services, there has been surprisingly little attention paid to the public reporting and scrutiny of attainment against targets. An exception was the report by the House of Commons Public Administration Select Committee (2003), which sought to identify attainment of 249 measurable targets from CSR1998. They found that 67.1 percent had been met; 7.6 percent had been partially met; 10.0 percent had been not met; and for 14.9 percent there was inadequate data on achievement.

These results were not readily available, and the Committee had to undertake research to secure them. In their original form, performance reports were found in a variety of formats, with varying levels of clarity, in the annual reports of individual ministries. The Treasury website merely offered links to these reports. The Committee recommended that progress towards targets and eventual attainment should be reported consistently and regularly on a single, authoritative website.

Within the UK parliamentary system, parliament appoints scrutiny committees for most ministries, and these would be the natural focus of holding the government accountable through routine reporting of progress towards PSA targets. However, systematic parliamentary scrutiny has not yet become routine, and instead scrutiny has been piecemeal, in the form, for example, of occasional reports from pressure groups, the media, and public service regulators.

A particularly important issue within the PSA regime has been the need for independent audit of the reliability of the data used to assess attainment. Hood (2006) reports a range of evidence to suggest that officials implementing the PSA system have given little explicit consideration to potential gaming responses and the associated data audit and regulatory requirements. Indeed, few within government have an interest in challenging information that reports apparent performance improvements and attainment of targets. Considerable popular scepticism has therefore developed about the veracity of information provided by the government on its own performance. The National Audit Office examines the processes for data collection, but is not in a position to assure the accuracy of all data (see Chapter 6 on performance auditing). In this respect, it is noteworthy that the government has announced its intention to make the Office for National Statistics more independent of government than hitherto. An important objective of this initiative is to dispel the perception that reports of government performance may be unreliable.

Far greater attention has been given within government to scrutiny of progress towards targets. The initial instrument for assuring implementation of PSAs was the set of service delivery agreements with departments. However, these proved unwieldy and ineffective, and they have been replaced by the Prime Minister’s Delivery Unit (PMDU). The PMDU has been a very important element of the more mature PSA system, indicating a perception within government that continuous monitoring, strong, and timely intervention powers, and continued political attention at the highest level has made an essential contribution to the longevity and sustained high profile of the system.

How should departmental objectives be transmitted to local organizations?

In most cases, attainment of national ministerial targets relies on securing satisfactory improvement in local organizations charged with delivering services. Therefore much depends on how the ministerial priorities are transmitted to local services. In the early rounds of the PSA process, such transmission mechanisms were often crude and ineffective. For example, the national objective to “increase the percentage of children who achieve level 4 in each of the Key Stage 2 English and maths tests …to 80% in English and 75% in maths” was initially translated by the education ministry into a simple-minded requirement that each school should secure that outcome, regardless of its existing level of attainment and the difficulty of the local circumstances.

Such approaches lead to manifest problems. Organizations already performing well have no incentive to improve, while those with disadvantaged populations may stand no chance of success, and become alienated. Indeed, if such regimes were sustained, it may become difficult to recruit public service workers in disadvantaged areas, exacerbating existing problems. As a result, many national departments have introduced more subtle target regimes for local organizations, seeking to encourage all to improve in the chosen measures, from whatever baseline they start.

However, the tension between national objectives and local discretion has become an important unresolved issue within the PSA regime, especially in the local government sector, where there should be some legitimate flexibility to set local priorities. The prevailing lack of flexibility was highlighted in a report by the Audit Commission (2003) that criticized the neglect of local government discretion in earlier PSAs.

In response, the Treasury set up a review of devolved decision-making to examine how national priorities and local flexibility can be accommodated within the PSA system. It has signaled an intention to move in the medium term towards the publication of local performance data as an alternative to national targets (HM Treasury, 2004b), with a view to allowing local people, rather than the national government, to hold local services to account for their chosen priorities and performance. However, while a policy of devolution clearly has relevance to services that are the responsibility of local governments, such as schools and personal social services, it has not yet been made clear how it local accountability will work in services (such as health care) that do not have a local democratic decision-making mechanism.

How should efficiency be incorporated into the PSA regime?

Most PSA targets address issues of effectiveness, as measured in the outcome targets. However, governments are also profoundly concerned with issues of cost-effectiveness, and so the Treasury has naturally sought to incorporate value for money considerations into the PSA system. From a theoretical perspective this may appear misguided—most departments are given a fixed budget, and the assumption must be that (if they spend the budget wisely) they will strive to secure their PSA objectives within that budget, and inefficiencies will naturally be driven out of the system. In short, issues of cost-effectiveness could be addressed naturally by ensuring that departments seek out maximum effectiveness within their budget constraints.

However, probably sensibly, the Treasury never believed that an emphasis on outcomes alone would ensure VFM, and they have therefore been seeking other ways of improving VFM. At first, this was attempted through the specification of an explicit VFM target within the PSA system. However, this gave rise to manifest anomalies. For example, the Department of Health VFM target was based on year-on-year improvements in the outputs of the NHS (such as hospital admissions, weighted for severity) divided by its expenditure. This target was weak for a number of reasons. First, it focused on outputs rather than outcomes, thereby breaching a guiding principle of PSAs and giving rise to obvious potential for distortion. And second, it was very selective in its coverage of outputs, and therefore offered only a partial glimpse of the department’s overall VFM. Implicitly it emphasized the department’s health care objectives, at the expense of health improvement.

The Treasury has therefore sought to implement other instruments alongside PSAs to ensure pursuit of VFM. For example, the implementation of Sir Peter Gershon’s detailed review of government efficiency included operational efficiency initiatives for specific ministries, such as improved procurement of IT services and relocation of civil servants to low-cost parts of the country.

More recently, it has become clear that VFM will become a central feature of the 2007 CSR, and the Treasury has published a paper discussing how VFM can be embedded into the PSA system (HM Treasury, 2006). The discussion paper raises a number of obvious approaches to promoting VFM, such as seeking out further operational efficiencies, in line with the Gershon agenda, and strengthening the capacity for financial management within government departments. However, it also discusses more radical approaches to securing VFM, such as empowering users (giving them more say in the shaping of their services and more choice of providers) and enhancing local democratic accountability.

Implicit in proposals such as these is a desire to focus the PSA targets on national priority outcomes only, and to move away from rigid central targets in other domains (perhaps relying on minimum local standards). This approach may address some of the criticisms of inflexibility inherent in the PSA system hitherto, and reduce the reported alienation of professionals charged with service delivery. However, to be successful it will require the provision of better-quality data, of relevance to service users and taxpayers, on the performance of local services. It will also require major enhancements to the weak systems of local democratic accountability currently in place for many public services.

Assessment

In introducing the PSA system in 1998, the government had a number of objectives (House of Commons Public Administration Select Committee, 2003):

  • offering a clear statement of what it is trying to achieve

  • giving a clear sense of direction and ambition

  • introducing a focus on delivering results

  • forming a basis for deciding what is and what is not working

  • improving accountability.

In broad terms, it is difficult to argue with the claim that it has been successful in these respects. However, the PSA regime remains a work in progress. It has introduced numerous challenges and anomalies, the most important of which are documented above. As experience unfolds, it is becoming clear that to be successful the PSA regime must be augmented by a number of other mechanisms. A series of departmental Capability Reviews by the Cabinet Office (2006a, 2006b) has noted that “while progress against PSAs and other top targets is necessary and welcome, it is not sufficient for delivering high-quality performance across the whole system.”

Some of the more important institutional requirements for the implementation of regimes such as the English PSA system include:

  • sustained political commitment to the PSA system, at the very highest level

  • a nimble central government organization (the PMDU) responsible for timely monitoring, reporting, and (where necessary) intervention

  • continued monitoring and regulation in domains not directly covered by PSA targets

  • high-quality performance management skills within the ministries

  • carefully crafted mechanisms for transmitting national targets to the local level

  • strong collaborative arrangements where necessary for domains that cross traditional ministerial boundaries

  • careful integration of central and local government priorities

  • mechanisms for ensuring VFM that are consistent with the PSA outcome targets

  • engagement, as appropriate, with relevant stakeholders, including user groups, professional organizations, and the voluntary sector.

A number of commentators have offered suggestions on the architecture of the targets regime. For example, the Social Market Foundation (2005) raises the following issues:

  • Targets should be set only when change is required or for aspects of public services which are exceptionally important.

  • There should be a fairly small number of targets in place at any one time.

  • While an outcome orientation is desirable, “process” and “input” targets may sometimes be appropriate, especially if the organization in question has limited influence over the outcome.

  • Targets add most value where other mechanisms, such as user choice and the threat of exit, or the contestability of providers, are not in place.

  • Proportionate sanctions and incentives are important. An organization missing a stretching target by a narrow margin should not be sanctioned for failure, but rather rewarded for its progress.

  • Public service targets should be fully integrated into departmental performance management, audit, and inspection regimes.

  • Joint targets that need to be delivered by more than one department should always have a “lead” department which takes responsibility for the meeting of the target.

  • Greater use could be made of targets relating to public satisfaction for a number of public service areas.

In addition, the Royal Statistical Society and Statistics Commission have given detailed guidance on technical aspects of performance measurement. The work of the National Audit Office emphasizes the need for improved quality of data, and there is clear evidence that genuinely independent scrutiny and audit of the data have become a central requirement of the PSA regime.

Performance budgeting seeks to strengthen the linkage between the funding and results of public services. On the face of it, the English PSA system offers immense potential in this respect. It applies a consistent approach to setting priorities across all government activity, focuses on the outcomes of government endeavor, and creates a comprehensive framework for measuring attainment. Moreover, it has been developed within the framework of the Treasury’s expenditure negotiations with ministries.

However, it remains the case that—notwithstanding rhetoric to the contrary—the link between the PSA system and the funding mechanism remains weak. In examining why this is the case, it is useful to think of three broad roles for the PSA system: setting priorities, measuring attainment, and offering incentives to ministries (James, 2004).

Without question, the PSA system offers a useful framework within which to set governmental priorities. It offers the finance ministry an opportunity to survey the whole domain of public services, to assess the relative merit of alternative claims on resources, and to choose objectives. In this respect, it contributes to the budgetary process, by requiring that departments expose their plans to critical scrutiny and receive funding conditional on acceptance of explicit targets.

However, there is no direct link between the PSA objectives set for a department and the funding it receives. In principle, the Treasury could make the allocation of a department’s budget conditional on acceptance of an ambitious PSA target. As noted above, PSA targets are probably strongest when signaling a change of priorities or a new policy direction. In these circumstances, additional money for a ministry can be made conditional. For example, the health ministry’s allocation of extra funding since 2000 was to some extent conditional on acceptance of targets embodied in its PSAs, and the more detailed NHS Plan (Department of Health, 2000). More usually, however, the detailed targets emerge from a process of negotiation after budgets have been set.

Another priority-setting area where PSAs can in principle make a major contribution is in cross-departmental domains, which historically have received low levels of attention from individual ministries. Although England is still struggling to find an entirely satisfactory mechanism for addressing cross-cutting objectives, the PSA system offers some promise for this especially difficult aspect of government. One possible future development may be to “carve out” budgets specifically for cross-departmental domains.

The retrospective measurement of attainment inherent in the PSA system can also furnish important evidence for expenditure reviews. First, it might identify “delinquent” ministries that are not making good use of their budgets. And second it might signal domains where government spending is highly effective (or ineffective), irrespective of the ministry’s competence. However, interpreting measures of attainment will always be problematic. The Soviet experience offers numerous examples of the perverse incentives that arise when budgets are based on attainment of central targets, and the associated distortionary consequences. Therefore, there is a strong case for viewing measures of attainment with caution, and maintaining a very loose linkage between performance against targets and departmental budgets.

In contrast to many previous “target” initiatives, a noteworthy feature of the PSA regime has been its success in securing sustained attention at the highest ministerial level. The responsibility for attainment of targets lies with ministers, and there is one instance of a minister resigning in response to poor performance against targets (the Education Minister, Estelle Morris). However, as noted above, there is in general rather weak public and parliamentary scrutiny of performance, and it is unusual for a minister to feel fatally exposed by poor performance. Furthermore, James (2004) found extensive use of stratagems to shift the blame for poor performance away from ministers. Nevertheless, the main (quite powerful) incentive operating on ministers will be to attain targets in order to maintain a reputation for competence. Senior civil servants will similarly have a concern for reputation.

James found limited evidence of any material incentives operating on departments more generally Perhaps the most direct incentive operating is the threat of receiving a “hard time” from the Treasury or the Delivery Unit. Serious underperformance will lead to escalating levels of intervention from the PMDU, with loss of autonomy and potential damage to reputation for those directly involved. The more recent “capability reviews” also have important incentive effects, although their remit extends well beyond the PSA system. However, there is little scope for using funding allocations as a major component of the incentive regime.

The National Audit Office (2006b) found that the majority of departments are increasingly using outcome-based performance information for reporting, accountability, and planning purposes. However, they found less use of PSA outcomes being made to inform financial management decisions or to maximise efficiency and effectiveness. This is perhaps where the major scope lies for strengthening the link between the PSA system and funding. In principle, the chosen PSA targets should inform the internal decisions of departments in evaluating existing activity, and assessing the virtues of new initiatives. A major challenge is therefore to embed the PSA principles within departmental decision-making in this way.

In conclusion, the English PSA system is an important new development with strong links to the principles of performance budgeting. It offers an immensely rich source of experience in seeking to apply consistent outcome-based performance criteria to the management of public services, and has without question delivered some major successes within the English public services. However, it has also exposed important implementation difficulties. It therefore merits careful examination by all those involved in the development of performance budgeting systems.

Note

I am indebted to Alex Hill (HM Treasury) for informal discussions and an invaluable paper on the history and mechanics of UK experience presented at the IMF performance budgeting seminar. I also benefited from membership of the Treasury’s Performance Information Panel, which scrutinized technical aspects of the PSA target-setting process, and participation in the Royal Statistical Society and Social Market Foundation commentaries on the PSA targets regime. I thank my fellow committee members and co-authors for their many perceptive insights, which may, I fear, not always be fully acknowledged in the text. I should also like to thank Marc Robinson for constructive comments and guidance. All errors and opinions remain my own responsibility.

References

    Other Resources Citing This Publication