Chapter

IV Budget Forecasting

Author(s):
Vladimir Klyuev, Martin Mühleisen, and Tamim Bayoumi
Published Date:
October 2007
Share
  • ShareShare
Show Summary Details
Author(s)
Martin Mühleisen, Stephan Danninger, David Hauner, Kornélia Krajnyák and Bennett Sutton1 

Canada’s strong fiscal record in recent years rests on a proven budgetary framework, including a well-established forecasting process. Canadian public finances are highly transparent, and the prudent fiscal policies of recent governments have enjoyed widespread public support. Beginning in the mid-1990s, these policies produced a string of budget surpluses that helped reduce federal debt (measured by the accumulated deficit) from almost 70 percent of GDP in 1996 to close to 40 percent in 2004. At that point, following a number of years with much larger than anticipated surpluses, the fiscal forecasting process went under review to ensure that “the [federal] government continues to use the most up-to-date economic and fiscal forecasting methods, and to benchmark Canadian practices against the best in the world” (Department of Finance Canada, 2004, p. 67). The IMF assisted this process by comparing Canadian central government budget forecasting with that of a benchmark group of other industrial countries.

This section summarizes the result of this comparison. The benchmark group consists of most other G-7 countries, as well as Australia and New Zealand (two commodity-exporting countries) and the Netherlands, Sweden, and Switzerland (smaller industrial countries with advanced budget practices).2 The analysis follows a two-pronged approach, covering both structural and quantitative aspects. The section first compares the institutional environment for fiscal forecasting and forecasting processes across the benchmark group. It then provides a description of budgetary forecast outcomes and presents the results of statistical analyses that, among other things, test for forecast bias and identify links between structural characteristics and forecast errors.

The results suggest that fiscal forecasting in Canada has been governed by one of the strongest institutional frameworks among the countries studied. Although Canada has no formal fiscal rule, recent governments have aimed for a modest surplus, which evolved into a de facto fiscal target. In support of this objective, Canada adopted a conservative approach to budgeting, with explicit prudence and contingency factors and a strong commitment to transparency and accountability. One particular strength is the explicit use of macroeconomic projections from a wide range of private forecasters for preparation of the budget. However, forecasts of fiscal variables are compiled by the Department of Finance with little participation from nongovernmental agencies. As is the case for many other countries, Canada could enhance public understanding of budgetary forecasts by providing more information on the assumptions and methods underlying the translation of the macroeconomic outlook into fiscal projections.

Quantitative analysis suggests that Canada’s budget projections of macroeconomic and fiscal aggregates were more cautious than in other countries between the mid-1990s and 2004. Measures for the distance between budget projections and actual outcomes were among the highest in the benchmark group. Moreover, forecast errors for both revenue and expenditure aggregates were consistently on the conservative side, making Canada the country that has most strongly underestimated its fiscal balance since 1995. Empirical tests indicate that the forecast errors are significantly different from zero and that both public and private forecasters were repeatedly surprised by the strength of the Canadian economy and fiscal performance, particularly in the late 1990s. Indeed, given the close link between tax revenues and the macro economy, stronger than expected growth appears to account for a considerable part of fiscal overperformance. The relatively volatile macroeconomic environment as well as institutional factors have also likely contributed to Canada’s conservative forecast bias.

The Institutional Environment

A country’s budget forecasting practices depend importantly on the legal and institutional structures governing fiscal policy. These structures need to be taken into account when comparing forecasting practices across countries, particularly as they can influence the accuracy of budget projections in a number of ways. This subsection looks at three factors that characterize the fiscal environment: (1) the distribution of fiscal authority between the legislature and the executive; (2) fiscal relations between the central and subnational governments; and (3) the presence of fiscal rules and other constraints limiting fiscal policy discretion.

Distribution of Fiscal Authority

The distribution of fiscal authority between the executive and legislative branch may affect the nature and quality of budget forecasts. For example, if substantial fiscal authority rests with the legislature, policy assumptions underlying the fiscal forecast of the executive branch may differ from fiscal measures taken, and the forecast quality could suffer correspondingly. Alternatively, the executive could face incentives to produce biased forecasts in order to influence the behavior of the legislature. For example, the executive could provide conservative revenue forecasts to keep spending pressures under control. By contrast, there would likely be fewer incentives for biased forecasts in cases where the legislature tends to approve the budget as drafted.

In Canada, the legislature has largely been focused on optimizing the budget process, as opposed to taking an active role in formulating the budget. The budget process reflects international best practices in many areas. For example, an OECD-World Bank survey (OECD and World Bank, 2003) finds that 19 out of 20 key aspects of the Canadian budget process are regulated by the constitution or by law (Table 4.1). Among the countries in the benchmark group, only the United States achieves a similar score.3 Moreover, Canada adheres to 10 of 13 OECD Best Practices in budget reporting, which is matched only by New Zealand and the United States.

Table 4.1.Indicators of the Relationship between Legislature and Executive
AustraliaCanadaFranceGermanyItalyNetherlandsNew ZealandSwedenUnited KingdomUnited States
Aspects of budget process regulated by the constitution or by law
Public funds can be spent only in programs as authorized by legislation×××××××××
The budget and financial reporting covers all central government transactions (including extrabudgetary transactions)×××××××
All budget transactions to be shown in gross terms××××××
The minister in charge of government finances has effective power over budget management×××××××
Individual government organizations are held accountable for the funds they collect and/or use××××××
Individual ministers are held accountable for the funds they collect and/or use×××××××
Requirements for independently audited financial accounting reports×××××××
Requirements for independently audited financial reports×××××
Conditions for use of contingency or reserve provisions×××××
Definition of public money×××××××××
Rules for the creation of extrabudgetary funds to special cases, authorized by separate statute×××××××
Authorize the government accounts into which all public money must be paid and from which expenditures are made only by appropriation of the parliament××××××××××
Roles for the parliament and the executive in the budget process and the relationship between the two branches with respect to budget responsibilities×××××××××
The form and structure of the annual budget law (or finance bill) to be voted by parliament××××××××
The definition of main headings and accounts in the annual budget law××××××××
The definition of the budget deficit and surplus××××××××
Legal basis for formulation and execution of budget, including the role and authorities of the ministry of finance/treasury and/or the central budget authority×××××××
Administrative/judicial sanctions for infractions of budget legislation××××××
The basis for management (internal) control and internal audit××××××××
Authorities and responsibilities for issuing and reporting on government guarantees××××××××
OECD Best Practices on central government budget reporting met
General overview of revenue and expenditure××××××××××
Detailed estimates of revenue and expenditure×××××××××
Citizen’s guide×××
Pre-budget report (general budget policy, aggregates)××××××
Long-term (10 to 40 years) outlook for public finances××××
Midyear report (s) on fiscal outlook××××××××
Report on tax expenditures××××××××
Statement of government assets××××××××
Special reports for old-age program finances×××××
Special reports for civil service pension×××××
Special reports on government debt×××××
Special reports on contingent liabilities×××
Pre-election report××
Number of months budget is submitted to legislature before the new fiscal year<2<22–44–62–44–6<22–4<2>6
Percentage share of expenditure for which appropriation acts need to be passed20–3030–4090–10090–10090–10090–10070–8030–40

Canadian budgets are usually passed as submitted without any changes. This appears to be a common feature in Westminster-style parliamentary systems and in other countries where the executive enjoys reliable support in the legislature, such as Australia, New Zealand, the United Kingdom, and Sweden (OECD and World Bank, 2003). The role of Canada’s parliament is circumscribed by the following:

  • Parliament receives the budget relatively late, less than two months before the start of the fiscal year. A quarter of the fiscal year has typically elapsed by the time the budget is approved. In contrast, legislatures of other countries receive the budget two to six months before the new fiscal year; it is even earlier in the United States. The late budget submission may be partly attributable to the use of accrual accounting, which requires the use of information that becomes available late in the fiscal year.

  • Only a relatively small part of total expenditure is funded by appropriation laws. As mandatory spending in Canada does not require annual funding legislation, new appropriations cover only about 30–40 percent of spending. This is similar to arrangements in Australia and the United States, but contrasts sharply with other countries. In the United Kingdom, appropriation laws cover 70–80 percent of total expenditure, and coverage can reach 90–100 percent in continental Europe.

  • As in many parliamentary systems, the Canadian legislature has limited powers to change the submitted budget. Parliament can reduce, but not increase, funding for line items, but has otherwise only the choice of approving or rejecting the government’s spending proposals. Only parliaments in Australia and New Zealand—which have to approve or reject the budget as a whole—are more constrained. Some restrictions also apply in France and Switzerland, while legislatures in Germany, Italy, the Netherlands, Sweden, and the United States are free to change every aspect of the budget proposal.

  • The executive would suffer strong consequences if parliament voted against any budget proposal. The budget vote is considered a vote of confidence in many countries, but political tradition in Canada (as in Australia, New Zealand, and the United Kingdom) goes further. In these countries, the executive would customarily step down if parliament voted against any single aspect of the budget (OECD, 2002).

As a result, there is little indication that executive-legislative relations should affect the accuracy of Canadian budget forecasts more than in other countries. The legislature’s limited role in the annual budget process appears to provide few incentives for providing biased forecasts; it also constrains the potential loss of forecast quality resulting from modifications prior to passage. At the same time, relatively stringent process rules and reporting requirements would seem conducive to forecast accuracy.

Fiscal Relations with Subnational Governments

The structure of intergovernmental relations also has implications for budgetary forecasting. From a technical perspective, the volatility of fiscal outcomes at the center is likely to be higher if significant transfers to the subnational levels are provided on a cost-sharing rather than a block-grant basis, given the scope for adjustments after the fact. However, there are also circumstances that may contribute to a deliberate bias in fiscal projections, such as when fiscal targets are set at the central government level even though the central government has limited control over the behavior of subnational governments.

Canadian provinces enjoy substantial financial independence, but transfers to provinces account for an important share of central government spending:

  • The center’s share in general government is smaller in Canada than in any of the comparator countries. Combined, Canada’s subnational governments are about as large as the central government (Table 4.2). This reflects the comparatively high number of policy responsibilities falling on subnational governments, including the country’s universal health care system.

  • Provinces have a high share of own-source revenues (85 percent), including from tax revenues shared with the central government (Figure 4.1). They are also free to determine their overall fiscal aggregates as well as most expenditure allocations—among the benchmark countries, only the subnational governments in Sweden and the United States have as much leeway. Canadian provinces also can borrow without federal limits—as in France, the Netherlands, New Zealand, and Sweden.4

  • However, transfers to other levels of government are a more important budget item in Canada than in other benchmark countries.5 Intergovernmental transfers are substantial even when compared to GDP, and the relatively small size of the center further inflates their size relative to other federal expenditures (Table 4.3).

Figure 4.1.Influence of Subnational Governments

Table 4.2.Share of Spending by Subnational Governments1
Federal countries
Australian.a.
Canada56.5
Germany36.1
Switzerlandn.a.
United States40.0
Unitary countries
France18.6
Italy29.7
Netherlands34.2
New Zealandn.a.
Sweden43.4
United Kingdom25.9
Table 4.3.Consolidated Central Government Expenditure Shares, 20031(In percent of total expenditure)
AustraliaCanadaFrance2GermanyItaly3NetherlandsNew ZealandSweden2SwitzerlandUnited Kingdom2United StatesRank, Canada
General public services26.630.37.713.720.622.19.023.514.35.212.21
of which:
Public debt transactions5.09.45.95.813.95.55.47.43.69.22
Defense6.55.85.23.62.63.62.95.74.97.419.14
Public order and safety0.93.01.70.44.43.74.13.20.65.11.46
Economic affairs6.26.010.06.75.76.06.89.411.56.97.08
Environment protection0.20.60.20.11.70.40.00.53
Housing and community amenities0.71.31.00.91.70.51.50.60.81.32.04
Health14.22.716.919.312.710.416.52.921.116.423.410
Recreation, culture, and religion0.91.50.70.12.00.92.20.80.51.40.24
Education9.32.110.00.410.510.821.26.42.612.52.69
Social protection34.546.644.354.838.241.535.947.243.742.532.03
Memorandum item:
Transfers to subnational governments427.231.48.614.312.926.11.013.727.519.119.11
Sources: IMF, Government Financial Statistics; OECD Revenue Statistics; and IMF staff calculations.

Uncertainties about revisions to the level of intergovernmental transfers and shared tax revenues may pose difficulties for fiscal forecasting. Fiscal arrangements in Canada provide for considerable payments from the federal government to the provinces. Under some arrangements, the final amounts are usually not determined by the end of a given fiscal year, giving rise to adjustments in subsequent years. Due to the relatively large size of transfers relative to other government expenditures, revisions can sometimes have a notable impact on the federal fiscal forecast:

  • The amount of equalization transfer payments, which are provided as unconditional block grants, was until recently subject to considerable uncertainty. Prior to 2004, these transfers were subject to significant adjustments, owing to statistical revisions of provincial tax bases and population size (Box 4.1). In 2004, however, the government reached an agreement with the provinces to place equalization transfers on a more predictable basis, including by eliminating retroactive adjustments to the overall amount of transfers.

  • Adjustments also are made necessary because the central government collects personal and corporate income taxes on behalf of nine and seven provinces, respectively, as well as Canada Pension Plan payroll contributions. These collections represent about 35 percent of federal revenue. Gross income and payroll tax revenues are divided on a preliminary basis throughout the year, but the actual split is only known after all relevant tax returns are assessed—usually toward the end of the following fiscal year.

Box 4.1.Equalization Transfers in Canada

Equalization transfers are designed to reduce disparities in tax-raising capacity between provinces. The transfers are provided as general-purpose block grants, channeling federal funds to provinces with below- average revenue- raising capacity. The definition of “revenue-raising capacity” is based on a comparison between per capita revenue raised and the per capita revenue each individual province could raise if it levied national average tax rates on each of the sources of provincial revenue. Each province’s revenue-raising capacity is then compared to that of the average of the five middle-income provinces (British Columbia, Manitoba, Saskatchewan, Ontario, and Quebec) on a per capita basis. Total equalization entitlements are determined as:

where

  • Eij =entitlement under revenue source j in province i

  • Bj =the tax base for revenue source j in the representative provinces

  • P =the population of the representative provinces

  • Bij =the tax base for revenue source j in province i

  • Pi =the population of province i

  • τj =the national average tax rate for revenue source j

The size of equalization transfers has been subject to considerable uncertainty. Initially, inputs to the formula determining entitlements are based on estimates for the current fiscal year. As these data are revised in subsequent years—for example, if a new census is taken or final tax revenue data become available—entitlements are modified and positive or negative ex post payments are made (see Table). Between FY 2000–01 and FY 2003–04, the magnitude of adjustments ranged between −21 percent to 8 percent of annual transfers, equivalent to a margin of up to ⅛ percent of GDP.

In October 2004, the government announced a new equalization framework. This included a new legislated level of overall equalization entitlements starting in 2005– 06, with a built-in growth rate of 3.5 percent annually.

Calculation of 2000–01 Equalization Transfers(in billions of Canadian dollars)
Nfld.P.E.I.N.S.N.B.Que.Man.Sask.Total
Payments through February 20011.00.21.21.14.31.10.19.0
Third estimate (February 2001)1.10.31.31.25.41.20.210.8
Fourth estimate (October 2001)1.10.31.31.25.41.20.310.8
Fifth estimate (February 2002)1.10.31.41.35.21.30.310.8
Sixth estimate (October 2002)1.10.31.41.35.31.30.210.9
Seventh estimate (February 2003)1.10.31.41.35.31.30.210.9
Final estimate (September 2003)1.10.31.41.35.41.30.210.9
Source: Department of Finance.Notes: Nfld. =Newfoundland; P.E.I. =Prince Edward Island; N.S. = Nova Scotia; N.B. = New Brunswick; Que. = Quebec; Man. = Manitoba; Sask. = Saskatchewan.
Source: Krelove, Stotsky, and Vehorn (1997).

Fiscal Rules and Other Constraints

Fiscal policy rules may improve fiscal discipline, but the costs of violating budget targets may also lead to cautionary biases. Governments that face incentives to improve their budget planning and implementation processes by implication have better prospects for meeting fiscal forecasts.6 On the other hand, asymmetric consequences for not meeting budget targets may lead to the incorporation of both explicit and implicit prudence factors in the forecast (see, for example, Zellner, 1986).

Unlike in many other countries, fiscal policy in Canada is not constrained by budget rules imposed under the constitution or the law (Table 4.4).7 Most advanced countries have adopted some form of rule, including targets for both the overall balance or expenditure, and require embedding fiscal plans within a medium-term framework.8 The monitoring of these objectives is usually accompanied with rigorous reporting requirements for comparing plans with outturns. For example, the EU Commission mandates that Stability Reports include a section on the general economic policy strategy, macroeconomic forecasts, and budgetary projections, as well as a series of standardized tables to enable the evaluation of the projections. In Australia, New Zealand, and the United Kingdom, fiscal planning is guided by legislation specifically aimed at enhancing transparency and accountability.

Table 4.4.Fiscal Policy Rules and Transparency Laws
Type of RuleFiscal Transparency Law
Deficit/debtGolden ruleExpenditure ceiling
AustraliaYes
Canada1
FranceSGP2
GermanySGP2Yes
ItalySGP2
NetherlandsSGP2Real
New ZealandYes
Sweden2% surplusNominal
SwitzerlandNominal
United KingdomDebtYesYes
United States
Source: IMF staff.

Between 1998 and 2006, Canada followed a de facto fiscal rule of “budget balance or better,” with performance observed on a relatively stringent basis and defined specific fiscal targets aimed at achieving this goal.9 The political commitment to this target, whose asymmetry was derived from long-term fiscal sustainability considerations, gave it a role similar to quantitative fiscal policy targets in a rules-based system. The target appeared even stronger than in many countries, both because performance was observed on an annual basis instead over the medium term and because the target was expressed in nominal terms and thus more difficult to achieve during a downturn than a GDP ratio. Forecasting performance has also been closely monitored and plays an important role in assessing the government’s track record in implementing its policy plans.

Canada also adheres to a strict budget planning framework. Along with adoption of the new fiscal targets, fiscal forecasting practices were fundamentally overhauled in the mid-1990s. A key objective of these reforms was to improve the credibility of economic and fiscal forecasts in response to a rapid buildup of public debt. Financial markets had begun to discount the government’s fiscal policy plans because economic assumptions were consistently overoptimistic. A summary description of the current organization of the forecasting process is given in Box 4.2.

During the years under study, Canada placed significant emphasis on prudent forecasts, which could have affected forecast accuracy. While macroeconomic forecasts are obtained from a panel of private sector forecasters, fiscal forecasts contain an explicit cautionary bias—the so-called prudence factor.10 In addition, the budget included a contingency reserve to cushion against unforeseen economic developments. In 2004, the prudence factor and the contingency reserve amounted to Can$1 billion and Can$3 billion, respectively, for both the 2004–2005 and 2005–06 budget projections. Although on a smaller scale than in Canada, other countries also use cautious economic assumptions or specific reserves. For example, in the Netherlands, formal arrangements have been in place for utilizing funds from unexpected overperformance of the fiscal balance (Blondal and Kristensen, 2002).

Box 4.2.Fiscal Forecasting Arrangements

In 1994 and 1995, Canada implemented significant changes to the budget formulation process. The government adopted a new public expenditure management system, a two-year rolling planning horizon was introduced, and the forecasting process was revamped. This system was refined in 1999 by publishing five-year fiscal forecasts in the fiscal midyear reports and by being more explicit about prudent planning assumptions in fiscal forecasts.

For the macroeconomic forecast, the Department of Finance surveys approximately 20 private sector forecasters each quarter after the National Accounts are released. Average annual private sector forecasts of real GDP growth, inflation, labor market indicators, and interest and exchange rates form the basis of the government’s macroeconomic assumptions. To ensure model consistency, the department may refine these assumptions in meetings with outside economists. The department feeds the assumptions thus gained into its internal macroeconomic model (the Canadian Economic and Fiscal Model) to construct aggregate revenue and expenditure projections consistent with the private sector forecast.

The detailed revenue and expenditure forecast is produced by the Department of Finance and respective spending agencies. Within the department, the forecasts are principally generated by the Fiscal Policy Division, although some smaller elements of the revenue forecast, for example, the low-income rebate of the VAT, are forecast by the Department’s Tax Policy Branch using micro-simulation models. Similarly, the department’s Economic Development and Corporate Finance Branch and certain Crown corporations are also consulted and provide information to help formulate the nontax revenue component of the revenue forecast. Other departments provide spending forecasts based on three-year business plans, which are reviewed by the Treasury Board Secretariat.

Since 1999, five-year fiscal forecasts have been prepared by private sector forecasters and are published in the Economic and Fiscal Update published in the fall. These forecasts cover broad fiscal aggregates on a general government basis. Based on this forecast, central government projections are again provided by the Department of Finance, with the 2004 Update presenting details on how the central government data have been derived from the private sector’s general government forecast.

In addition to fiscal rules, expenditure discretion in Canada is constrained by relatively high debt service costs and other nondiscretionary expenditure. In particular, the share of interest payments is the second highest among the 11 countries, despite the recent decline in public debt, while the share of social protection is the third highest (see Table 4.3).11 Moreover, as noted, the share of transfers to other levels of government is far higher in Canada than in most of the benchmark countries.

Fiscal Forecasting Practices in International Comparison

The importance of fiscal forecasts for budget planning purposes raises process and transparency issues. While solid technical capacities are necessary for high-quality forecast outcomes, forecasting performance also tends to be boosted by an open budget preparation process, including the involvement of nongovernmental agencies, public access to information, and regular reviews of forecasting performance (IMF, 2001). This subsection contrasts technical aspects of Canada’s fiscal forecasting arrangements with other countries and assesses its transparency aspects.12

The role of fiscal forecasts in the Canadian budget process is similar to practices in other benchmark countries (Table 4.5).13 In the majority of these countries, responsibility for budget preparation is assigned to one government agency (the ministry of finance or treasury), but the process usually involves collaboration with other government agencies. Forecasts are framed within a medium-term horizon in all countries, mostly in the form of a rolling three- to five-year forecasting framework (for example, euro area countries are required to prepare five-year fiscal plans), although the period for which fiscal plans are binding, or for which greater detail is presented, is typically much shorter. In Canada, budget preparation is based on a two-year framework, although since 1999 the government has also prepared five-year fiscal forecasts as part of the mid-year fiscal update.

Table 4.5.Key Institutional Characteristics of the Fiscal Forecasting Process
Characteristics of the Forecasting Process
Budget

Authority
Forecasting

Horizon
Macroeconomic

forecast
Revenue and expenditure
AustraliaTreasury, Department of Finance and AdministrationRolling three-year budget forecastTreasury internal: based on extensive consultation processGovernment internal; revenue derived from interaction between spreadsheet-based forecast and econometric model; expenditure supplied by spending agencies
CanadaFinance Department and Treasury Board SecretariatRolling two-year budget forecasts; aggregate fiscal forecasts for five yearsAverage of private forecastersRevenue and expenditure: two-year budget forecast prepared internally by experts group and respective spending agencies; five-year forecast in midyear based on forecast of private sector
GermanyMinistry of Finance (MoF)Five-year (SGP)MoF internal: after consultations with forecasting agenciesRevenue: based on consensus among expert group with nongovernmental participation; expenditure: government internal supplied by spending agencies
NetherlandsMinistry of FinanceRolling three-year budget forecast; five years at aggregate level (SGP)Independent public agency for coalition period; MoF otherwiseRevenue: by independent public agency for four-year coalition plan; MoF internal revenue forecast for individual budget years; expenditure forecasts by spending agencies
SwedenMinistry of FinanceRolling three-year budget forecast; five years at aggregate level (SGP)MoF internal: model-driven, benchmarked against other public sector forecastsMoF internal; revenue model driven benchmarked against other public sector forecasters; expenditure prepared by spending agencies
SwitzerlandMinistry of FinanceRolling three-year budget forecastForecast by expert group comprising MoF, central bank, and statistical officeGovernment internal; revenue: iterative process between different departments in the MoF; expenditure supplied by spending agencies
United KingdomTreasuryFive-year budget forecast; aggregate long-term projectionsTreasury: iterative process between econometric model and micro-based fiscal forecastsGovernment internal; revenue: iteration between Treasury macromodel and micro-based expert models in revenue department; expenditure prepared by spending agencies
FranceMinistry of FinanceFive-year (SGP)MoF: Forecasting DirectorateGovernment internal; revenue: iteration between various departments in the MoF; expenditure: forecasts made by the MoF’s Budget Directorate in coordination with spending ministries
ItalyMinistry of Finance and EconomyFive-year (SGP)
New ZealandTreasuryFour-year budget forecastIterative spreadsheet-based forecast including views of expert panel, business, and senior staff from TreasuryGovernment internal: two revenue forecasts prepared and published separately by Treasury and revenue administration; based on micro- and macromodels with consistency check with macroeconomic forecasts and assessment against views of practitioners (tax talks); Treasury forecast used in budget; expenditure forecasts prepared by spending agencies
United StatesExecutive (Office of Management and Budget)Five-year budget forecastPresident’s forecast assessed by Congressional Budget Office leading to congressional budget resolution that establishes major fiscal aggregates to constrain the decision making of the appropriations, taxing, and authorizing committees
Sources: OECD and World Bank (2003); country authorities; and IMF country desks.Notes: Forecasting horizon includes budget year. MoF = ministry of finance; SGP = Stability and Growth Pact.

Canada relies more than other countries on macro-economic forecasts by private forecasters (Table 4.6; see also Box 4.2). In most benchmark countries, the agency responsible for the budget develops its economic forecast in-house, using econometric and spreadsheet-based models. These estimates are often supplemented with information gained from consultations with nongovernmental forecasters or the business sector. In some countries, no outside agency is formally involved and quality control is left to benchmarking against other forecasting agencies (for example, Sweden). The main trade-off between the two approaches is that greater involvement of outside agencies may boost forecast credibility, whereas a broader consultation process may imply the use of less systematic forecasting techniques, which may make it more difficult to pinpoint the cause of forecast errors.

Table 4.6.Fiscal Forecasting: Quality Assurance
Involvement of

Nongovernmental Agencies1
Ex Post Assessment of

Forecasting Performance2
Availability of

Information on Fiscal

Performance3
Macro

forecast
Revenue

forecast
SelfExternalScore on detail

and regularity
AustraliaMediumLowRegularOccasionalMedium
CanadaHighMediumRegularOccasionalHigh
FranceMediumLowRegularRegularHigh
GermanyMediumHighOccasionalOccasionalLow
ItalyLowLowNoLow
NetherlandsMediumMediumRegularNoLow
New ZealandMediumMediumRegularOccasionalHigh
SwedenLowLowOccasionalNoLow
SwitzerlandLowLowOccasionalOccasional
United KingdomLowLowRegular, legal requirementRegularHigh
United StatesRegularHigh
Sources: OECD and World Bank (2003); and data provided by country authorities.

As in most of the other countries, revenue and expenditure forecasts in Canada are prepared by the ministry of finance. The degree to which the forecasting process is formalized varies quite significantly across countries. Some countries prepare stylized forecasts with some cross-checks against sectoral and revenue experts (Sweden, Switzerland). Others use detailed model-driven processes and microdata-based models maintained by technical experts (Australia, France, and the United Kingdom). In Canada, there is little direct involvement of outside agencies in preparing revenue and expenditure forecasts for the annual budget. However, projections for the midyear fiscal update are compiled by a small group of private forecasters, providing an independent view of the medium-term implications of current fiscal policies. Other countries have assigned similar tasks to independent agencies. For example, the U.S. Congressional Budget Office regularly provides 10-year projections of major economic and fiscal variables, based on fiscal policies as legislated by the U.S. Congress. Australia assesses its fiscal forecast through an extensive consultation process with outside experts and the business sector.

The Canadian public has relatively broad access to budgetary information. A comparison of the detail of published fiscal information shows that Canada scores high relative to countries in the benchmark group (see Table 4.6). The primary budget documents available to the public are the annual Budget Plan (usually released in February or March) and the Economic and Fiscal Update prepared midyear. Both provide economic and fiscal forecasts with detailed explanations of anticipated future developments. The level and detail of published information is comparatively high.

However, the closed nature of the budget compilation process implies that forecast risks may not be widely understood, limiting public debate on this aspect. As for many other countries, Canada provides relatively little information on the key assumptions and methods underlying the use of macroeconomic assumptions in budget forecasts, making it difficult for outsiders to distinguish between fiscal forecasting performance and errors arising from implicit prudence factors.14 Some countries in the benchmark group are more inclusive in this regard. In Germany, for example, tax revenue forecasts are based on the consensus of a technical expert group with participation of nongovernmental agencies, providing some assurances that fiscal forecasts are untainted by policy objectives. In Australia and New Zealand, the government is legally required to demonstrate, at the time the budget is issued, that budget policies are consistent with long-term fiscal objectives, including by establishing a clear link between policy objectives, forecasts, and outcomes. This requirement has led to a greater emphasis on forecast outcomes, with performance assessments being used to gauge the realism of new budget plans.

Unlike most benchmark countries, the Canadian government provides regular and detailed analyses of its fiscal forecasting performance, despite the lack of an explicit legal requirement. Only a few countries mandate such reports on an annual basis (Australia, New Zealand, and the United Kingdom). However, despite the lack of an explicit legal requirement, the government’s Annual Financial Report analyzes fiscal results for the previous fiscal year, including by listing the sources of deviations from initial forecasts. The government also initiated a comprehensive review of its forecasting performance in 1994. A special task force reviewed the accuracy of the Department of Finance’s economic and fiscal forecasts and their role in the budget planning process, initiating changes in the budget process. A more focused review and consultations with a group of private sector economists in 1999 led to a more explicit treatment of the prudence factor and the introduction of five-year fiscal forecasts beginning with the Economic and Fiscal Update in that year (see Box 4.2).

Assessing Forecast Accuracy

Data problems generally limit the analysis of fiscal forecasting performance across countries. A number of studies have compared the accuracy of macroeconomic forecasts by private sector economists and international organizations (Artis, 1996; Artis and Marcellino, 2001; Ash, Smyth, and Heravi, 1998; Batchelor, 2001; Isiklar, Lahiri, and Loungani, 2005; Loungani, 2000; Öller and Barot, 2000). But most analyses of budget projections have focused on a single country, given the difficulties in obtaining a cross-country data set of budget forecasts. More recently, two studies have analyzed budgetary forecasts for a group of relatively homogenous countries (the euro area), with one suggesting that the size of forecast errors may depend on structural characteristics of a country’s budgetary framework (Strauch, Hallerberg, and von Hagen, 2004), and the other calling for independent budget forecasting agencies on the basis of significant forecast biases (Jonung and Larch, 2004).

Information obtained for this study provided sufficient detail to compare recent Canadian central government budget forecasts with benchmark countries. At a minimum, most budgets provide three to four years of information for key macroeconomic and fiscal variables, including actual or estimated values for the preceding year, an estimate or projection for the current year, and projections for one or two future years.15 Most budgets are also compiled near the beginning of a new fiscal year, with the result that the values of economic and fiscal variables reported for the prior year are generally at or close to their final revision. This allows the use of historical data reported in the budget as a basis for comparison against projections contained in earlier budgets.16 A description of available data is contained in Box 4.3.

Budget projections are evaluated against subsequent budget “actuals,” which provides two advantages over using fully revised values. First, data revisions (caused, for example, by changes in the coverage of government accounts) may be retroactively applied to fiscal outcomes but not to past budget projections. Therefore, revised historical data cannot be used to measure the accuracy of projections made before a revision came into force. Under this analysis’s definition of forecast errors, data losses are limited to at most two to three observations around the time a revision is introduced. Moreover, this method is also “fair” in that it focuses on the information that was available to forecasters at the time and that underlay economic agents’ expectations.

On this basis, a comparison of forecast errors shows notable differences between Canada and other benchmark countries. For example, projections for real GDP growth in Canada appear to have been on the optimistic side in the early 1990s and more cautious during the high-growth phase in the second half of the 1990s.17 A similar pattern can be observed in the United States, whereas German and Swiss budget forecasters appear to have maintained a more optimistic outlook over time. On the other hand, Canadian fiscal forecasts appear to have been consistently one-sided between the mid-1990s and 2004, whereas most other countries have reported two-sided errors. Before proceeding to a more formal evaluation, however, several words of caution are in order.

Data Caveats

Reflecting the idiosyncratic nature of every country’s budget process, the empirical analysis remains complicated by data limitations. The most important constraints are the following:

  • Time series of consistent forecasts and budget outcomes are relatively short (often with fewer than 10 observations), limiting the power of statistical tests. Many countries updated their budget formats and forecasting methods in the early to mid-1990s. This has generally increased the level of information provided but also has resulted in structural breaks related to the adoption of new budget concepts and coverage. Although the coverage of revenue and expenditure data is broadly similar across most countries, there are limits to how closely they can be compared. For example, while tax categories are relatively similar, some countries include social insurance contributions as government revenues. Moreover, sources for nontax revenues (which may include receipts from asset sales, royalties from natural resources, or frequency spectrum fees, to name a few) tend to differ significantly across countries.

  • Comparing expenditure subcategories appears to be particularly difficult. For example, the distinction between discretionary and mandatory spending components—each of which poses a different challenge to budget forecasters—is difficult to obtain for most countries or can only be approximated. Similarly, data on transfers to other levels of government are not provided on a consistent basis.

  • Checks for internal consistency and structural breaks may not have captured all data anomalies. These checks resulted in the rejection of a considerable number of data points. However, given relatively scant institutional knowledge of the information contained in government budgets more than a few years back, only obvious statistical outliers were eliminated.

Importantly, revised forecasts published in midyear budget updates or other publications are also not considered in this study. In many countries, the government provides updated budget projections in the course of the fiscal year—for example, in Canada’s Economic and Fiscal Update or in convergence programs provided by countries in the euro area. Other public bodies (such as the U.S. Congressional Budget Office) often conduct complementary analyses of fiscal developments. Including such information, however, would have greatly increased the cost of collecting and preparing a consistent dataset.

This may exacerbate problems caused by policy shifts that are implemented midyear. For example, there was a relatively large U.S. fiscal “error,” which underlines the difficulties in limiting the focus of this study to annual budget documents. If negotiations over fiscal measures conclude considerably later than a budget has been published, there may be a higher likelihood that policy outcomes differ from underlying assumptions, possibly resulting in a significant deviation of fiscal projections from outcomes. However, such deviations would be policy-driven and not the responsibility of budget forecasters.18

Macroeconomic Forecasts

The remainder of this subsection presents a formal comparison of forecast errors since 1995, separated into macroeconomic and fiscal projections. The mean error (ME) and root mean squared error (RMSE) for one-year forecasts of key macroeconomic variables are presented in Table 4.7. The mean error is the simple average of forcast errors over 1995–2003, providing an indication of the direction of forecast errors. The RMSE, defined as the square root of the mean of the errors squared, is independent of the error sign and therefore is a better measure of the size of forecast errors. Limiting the sample to the years indicated focuses the analysis on the period during which the current Canadian forecasting methodology was in force. Moreover, longer time series were not available for many countries, and 2005 budgets were not yet released in most cases.

Table 4.7.Descriptive Statistics of One-Year Budget Forecast Errors, 1995–20031
AustraliaCanadaFranceGermanyItalyNetherlandsNew ZealandSwedenSwitzerlandUnited KingdomUnited States
Macroeconomic variables
Nominal GDP−0.0066−0.02360.00350.02440.0002−0.0111−0.0037−0.02710.0116−0.0170−0.0168
0.02900.03440.01420.02920.01380.02960.02090.03570.02810.01990.0390
89899994969
Real GDP growth−0.0500−0.47500.68330.96110.13950.47780.03330.72500.7078−0.1500−0.4333
1.15921.72111.23641.33931.85771.58301.66371.46791.46980.98931.6809
98698964959
GDP deflator0.2859−0.17500.18330.5611−0.16110.25570.57620.00570.1669
1.25361.05220.36230.77920.68891.24170.96090.90380.3510
98690904959
Unemployment rate0.40000.08750.23330.35000.2000−0.35000.2778
0.66380.28340.44470.60050.57970.62650.8149
98003964009
Fiscal variables
Government revenue−0.0154−0.03790.01050.0155−0.0180−0.0278−0.0175−0.0329−0.0209−0.00850.0027
0.04660.06200.03130.04640.02800.12320.02880.03510.08400.02760.0921
89696684969
Tax revenue−0.0207−0.02920.00860.02260.00240.0001−0.0409−0.00550.0049
0.05100.05690.02860.05070.05420.02440.04300.02620.0993
89690994069
Personal income tax0.0093−0.02730.06050.0199−0.0063−0.0257−0.0194−0.0145
0.02340.05370.10320.07130.02150.03600.04350.1524
29090694069
Corporate income tax−0.0686−0.06940.13520.03880.0371−0.03870.00650.0987
0.10680.16520.47880.18030.10350.21940.10930.2340
29090694069
Social insurance taxes−0.0885−0.0168−0.0004
0.14860.02340.0277
20000000069
Indirect taxes−0.0276−0.01600.03490.00870.0015−0.1304−0.00170.0772
0.04070.06030.09260.03570.04550.20430.00780.1039
29090694069
Other revenue−0.0434−0.1861−0.0550−0.3883−0.20910.0343−0.06490.0642
0.12440.23500.15890.55020.25920.07300.16950.2241
89090584069
Government expenditure−0.00620.0082−0.0111−0.00070.0076−0.01720.00220.00820.01100.00720.0027
0.02880.02580.01780.02340.02610.06780.00920.01460.02220.01000.0209
89696687969
Mandatory expenditure−0.0020−0.02250.0159
0.04350.03940.0314
09090000009
Discretionary expenditure−0.00510.0568−0.0221
0.03620.07150.0340
09090000009
Interest expenditure−0.07500.02450.01870.0079−0.0131−0.02000.03640.00930.0295
0.10400.04580.13810.05660.12600.05010.15030.0816
97096994019
Fiscal balance−0.8025−6.54271.97921.5599−2.42180.7900−1.9792−0.0811−3.0378−1.29850.0711
5.69137.94283.62465.06693.71094.20893.29542.57858.76063.258510.8211
89896684969
GDP ratios
Government revenue−0.1375−0.27230.1727−0.0934−0.3433−1.5255−0.5984−0.3000−0.36430.32640.4333
0.66450.70710.43550.42480.67213.39931.09941.46630.89410.85071.4397
89696584969
Government expenditure−0.08750.5204−0.1671−0.28880.4600−1.07930.08480.2883−0.00760.94850.3000
1.02560.71850.37470.53171.36001.98850.62330.52780.41871.08961.0654
89696587969
Fiscal balance−0.1111−1.11460.36250.1954−0.7867−0.31060.1331−0.0250−0.3567−0.5122−0.0778
1.39501.36370.66260.59261.22741.99980.48211.20131.05771.27052.0367
99896584969
Source: IMF staff calculations.

The evidence suggests that economic growth in Canada has on average been ½ percentage point higher than budget projections in recent years. Canadian projections of nominal GDP and real GDP growth show higher RMSEs than for those of most other countries, and Canadian mean errors are at the negative end among the benchmark countries. Decomposing the RMSE into its two components indicates that this result appears to be mostly a function of the large mean error, given that the standard deviation of Canadian forecast errors has not been as high as for many other benchmark countries. This could suggest that Canadian forecasters adopted a relatively consistent forecast bias, whereas the deviations for other countries are spread more equally on the positive and the negative side (statistical tests of this hypothesis are presented in the next subsection).

Canadian forecasters also underestimated GDP inflation by 0.2 percentage points on average, but short-term unemployment trends were anticipated quite well. Projection errors for increases in the GDP deflator show a distribution similar to the growth forecast, with high RMSEs and a mean at the negative end among the sample countries. By contrast, the one-year forecast of the unemployment rate exhibited a lower RMSE and a lower (positive) mean error than for other countries.

These findings indicate that Canadian budget forecasters generally adopted a conservative view of macroeconomic developments over the past 10 years. Errors made in forecasting major macroeconomic variables are internally consistent. On average, growth and inflation were stronger than expected, and unemployment rates lower than anticipated. The projection of nominal GDP also suffers from the fact that Canadian forecasters underestimated base-year GDP by about 1 percent on average—the largest negative value in the benchmark group.19 Macroeconomic prudence adjustment through the 1998 budget—affecting about half of all sample years for Canada—is estimated to account for 0.1 percentage points of the mean real growth forecast error and for half as much of the mean GDP inflation error.

Box 4.3.Data Overview

Australia

Annual budgets are usually presented in May, two months before the start of the fiscal year in July. Forecast data begins with the 1984/85 budget.

  • Budgets present activities of the general government, which includes central, state/territory, and local governments.

  • Beginning in FY1999/2000, Australia moved from a cash to an accrual accounting basis, but subsequent budgets reported most items on both a cash and an accrual basis. For the sake of consistency, the dataset uses cash forecasts for all fiscal variables, except interest expenses which were only available on an accrual basis from FY1999/2000 to FY2004/05. In FY1999/2000 and FY2000/01, individual, corporate, indirect, and other taxes are omitted from the dataset because they were not reported on a cash basis.

  • From FY1984/85 through FY 1993/94, there were no revenue projections beyond the budget year—that is, two-year projections were omitted. Projections for real GDP growth and unemployment were also limited to the next fiscal year.

  • Final outcomes for FY 1996/97 were not reported in the FY 1998/99 budget and had to be substituted with estimates reported in the FY1997/98 budget.

Canada

Data were provided in electronic form by the Department of Finance. Canadian budgets are usually published in February, two months before the start of the fiscal year on April 1.

  • Projections for FY 2000–01 come from the Budget Update for FY1999–2000, which was published in October 2000.

  • Mandatory expenditures include transfer payments; discretionary expenses are defined as program costs.

  • Actual outcomes are generally taken from annual financial reports of the government. Annual financial reports are published sufficiently long after the close of the fiscal year to properly estimate accruals transactions.

France

Data were provided in electronic form by French national authorities. French budgets are usually published in September, with the fiscal year starting on January 1.

  • Forecast data begins with FY1996. Personal income, corporate income, and excise and other tax revenue data are not available for FY1996 and FY1997.

Germany

German budgets are published in September for the next fiscal year starting on January 1.

  • Forecast data begins with FY1990. Variables directly affected by the 1990 reunification have been omitted.

  • Data on mandatory expenditures comprise government wages and salaries and transfer payments. Discretionary expenditures include acquisition of goods and services and capital spending.

Italy

Italian budget proposals are published in the “Documento di Programmazione Economico-Finanziaria” (DPEF) between May and July, a half-year before the start of the next fiscal year in January. Data provided by the national authorities reach back to FY1989.

  • Personal and corporate income, excise, and other tax revenue data were not available.

  • Central government data for FY2000 and FY2001 were not available.

  • For FY1990–FY1998, DPEFs did not report final outcomes for either fiscal or macroeconomic variables, and so estimated outcomes from the previous budget are used as the final outcomes.

Netherlands

Data were provided in electronic form by Dutch national authorities. Dutch budgets are published in September, preceding the fiscal year starting on January 1.

  • Forecast data begin with FY1995 and cover general government.

  • Most projections were limited to the one-year time frame.

New Zealand

New Zealand publishes its “Budget Economic and Fiscal Update” (BEFU) in May, prior to the start of the fiscal year on July 1. Growth and unemployment data were pulled directly from BEFU documents; all other observations came from: http://www.treasury.govt.nz/fiscaldata/default.asp.

  • Projection data were available for FY1994/95 through FY2004/05, except for growth and unemployment projections which begin with FY1998/99.

Sweden

Outcome and projection data for Sweden were taken from “Appendix 2: Svensk Economi” of the annual budget bill. The bill is published in September, four months prior to the start of the fiscal year on January 1. Data were available for FY1997 through FY2005, with the exception of FY2000.

  • Revenues and the fiscal balance were provided on a general government basis. Budgetary expenditure is on a central government basis.

  • Data for personal income, corporate income, and excise and other tax revenue were not available for FY1997 and FY1998.

Switzerland

Data were provided in electronic form by Swiss national authorities. Swiss budgets are published in October, with the fiscal year beginning on January 1.

  • Forecast data begin with FY1990.

  • Data for personal income, corporate income, and excise and other tax revenue were not available.

United Kingdom

The U.K. government usually publishes its “Budget Report” in March, shortly before the start of the fiscal year in April. Data only cover budgets published under the current framework since FY1997/98.

  • The “Budget Report” refers primarily to the public sector, although general government aggregates are shown for most years.

  • The current U.K. fiscal framework separates the current and capital budgets. For consistency purposes, current and capital expenditures were consolidated. Total outlays are the sum of current expenditure and net investment.

  • The headline balance concept used was “Net borrowing” inclusive of net windfall tax receipts and associated spending (WTAS), asset sales, and depreciation.

United States

Federal government data was obtained from “Historical Tables: Budget of the U.S. Government,” which is usually published in February, eight months before the start of the fiscal year on October 1.

  • Interest expense is recorded on a net basis.

  • For FY1984/85 through FY1990/91, mandatory spending was defined as “total, relatively uncontrollable outlays” and discretionary spending as “total, relatively controllable outlays.”

  • Prior to FY1990/91, nominal output is reported as gross national product (GNP). Beginning with FY1990/91, nominal output is reported as gross domestic product (GDP).

Consensus Forecasts

Private sector forecast data for real GDP growth rate (calendar-year basis) and the headline budget deficit value (in local currency) come from Consensus Economics, Inc. Consensus Economics publishes updated estimates for the current and next calendar/fiscal year every month. The data for this study are drawn from the month in which authorities released their budget documents.

Fiscal Forecasts

A similarly conservative approach appears to have been applied to Canada’s fiscal projections. An analysis of revenue and expenditure projections generally finds Canada among the group of countries with relatively weak forecast accuracy (as measured by the RMSE). Moreover, compared to the benchmark group, the average error takes on one of the largest negative values for revenues and one of the largest positive values for expenditures. Taken together, this implies that Canada has the largest negative mean error for the overall deficit forecast, even after allowing for economic prudence and contingency factors.20

On the revenue side, projections of personal income tax revenue and revenue from the Goods and Services Tax (GST) and the Manufacturers’ Sales Tax (MST) contributed most to the overall forecast error. As far as subcomponents of tax revenue are concerned, Canadian RMSEs are generally not as large relative to those of other countries for aggregate revenues. What makes Canada stand out, however, is that the mean error for all subcomponents is negative, compared to at least one positive error for all of the other five countries for which similar data are available. It is the accumulation of small but persistently negative errors, rather than large forecast errors, that make Canadian forecasters appear relatively pessimistic.

Deviations on the expenditure side appear partly driven by smaller-than-expected debt servicing costs. For all countries, expenditure forecasts have been significantly more accurate than revenue forecasts, as evident from substantially lower MEs and RMSEs. Canada has been no exception as far as mandatory and discretionary expenditure items are concerned. However, interest payments were on average 2 percent lower than projected, leading to an average forecast error of 0.1 percent of GDP.21

Even when scaled by the size of GDP, Canadian fiscal forecasts appear unusually conservative. When forecast errors are defined as the difference between actual and projected GDP ratios, Canada still has the largest negative ME compared to the benchmark group, although the RMSEs are in a more moderate range. Canada may have been helped by the fact that forecast accuracy improves once revenues are expressed as GDP ratios, given that the elasticity of tax revenues is close to unitary in many countries. On the other hand, projections of expenditure-to-GDP ratios suffer particularly from GDP forecast errors as nominal expenditures tend to be more closely in line with budget targets.

Statistical Analysis of Forecast Outcomes

This subsection uses statistical tests to further explore the forecast characteristics. First, tests are used to check for the presence of a forecast bias and to check whether projections are efficient in the sense that they use all information available at the time of the forecast. Second, budget projections for GDP growth and the fiscal balance are compared with private sector consensus forecasts. Third, using structural information described previously, country data are pooled to test whether variables describing the forecasting environment have a significant impact on projection outcomes.

Bias and Efficiency Tests

A series of statistical tests confirms a forecasting bias in some components of Canada’s macroeconomic and fiscal forecasts (Table 4.8). The tests suggest that, between 1995 and 2003, the mean and median of the forecasts for nominal GDP as well as for total and nontax government revenue were significantly different from zero.22 This places Canada in a group with Germany, New Zealand, Sweden, and the United Kingdom, which all exhibit a consistent bias in either the macro forecast or aggregate fiscal revenues or expenditures. By comparison, Australia, France, Italy, the Netherlands, and the United States are largely free of such bias.

Table 4.8.Results of Forecast Error Median and Mean Tests1
AustraliaCanadaFranceGermanyItalyNetherlandsNew Zealand2Sweden2SwitzerlandUnited KingdomUnited States
Nominal GDP− (8)SWVCc (9)− (8)SWVCc (9)− (9)− (9)− (9)C (4)− (9)SWVC (6)− (9)
Real GDP growth− (9)− (8)− (6)SWVCc (9)− (8)− (9)− (6)− (4)− (9)− (5)− (9)
GDP inflation3− (9)− (8)− (6)Cc (9)− (9)− (4)Cc (9)− (5)c (9)
Unemployment rate3C (9)− (8)− (3)C (9)− (6)− (4)− (9)
Government revenue− (8)WVCc (9)− (6)− (9)Cc (6)− (6)WVC (8)VC (4)− (9)− (6)− (9)
Tax revenue− (8)− (9)− (6)− (9)− (9)− (9)VC (4)− (6)− (9)
of which:
Personal income tax3− (2)− (9)C (9)− (6)− (9)C (4)− (6)− (9)
Corporate income tax3− (2)− (9)− (9)− (6)− (9)− (4)− (6)− (9)
Social insurance taxes3− (2)C (6)− (9)
Indirect taxes3− (2)− (9)− (9)− (6)− (9)− (4)− (6)C (9)
Other revenue− (8)SWVCc (9)− (9)SWVCc (5)SWVCc (8)− (4)− (6)− (9)
Government expenditure− (8)− (9)Cc (6)− (9)− (6)− (6)− (8)SWVc (7)SW (9)WVC (6)− (9)
Mandatory expenditure− (9)C (9)WV (9)
Discretionary expenditure− (9)SWVCc (9)SWVC (9)
Interest expenditureSWVCc (9)− (7)S (9)− (6)− (9)c (9)− (4)− (0)− (9)
Fiscal balance3− (8)C (9)− (8)− (9)C (6)− (6)C (8)− (4)− (9)− (6)− (9)
Mean tests including growth terms4
Government revenue− (8)Gg (9)ng (6)g (9)Nng (6)nGg (6)NG (8)NG (4)Gg (9)n (6)− (9)
Tax revenue− (8)g (9)ng (6)− (9)g (9)− (9)NG (4)− (6)− (9)
Other revenuen (7)Gg (9)− (9)NnGg (5)NnGg (8)− (4)− (6)− (9)
Government expenditure− (8)g (9)Nng (6)− (9)− (6)− (6)− (8)N (4)− (9)g (6)− (9)
Mandatory expenditure− (9)− (9)− (9)
Discretionary expenditure− (9)g (9)NG (9)
Interest expenditureNnGg (8)Nn (7)− (9)g (6)− (9)n (9)− (4)− (9)
Source: IMF staff calculations. See Mühleisen and others (2005) for a description of the underlying methods.

The tests also underline that it is the aggregation of small, unidirectional forecast errors that leads to an overall bias in growth and revenue estimates in Canada. For example, both real GDP growth and GDP inflation forecasts have a negative mean error that is not statistically different from zero. However, the hypothesis of a zero nominal GDP error (to which both the growth and inflation error contribute) is clearly rejected. Similarly, the mean errors of individual tax revenue components were not significant at the 10 percent level, unlike the statistically significant aggregate revenue forecast error. Nontax revenues, which account for about 10 percent of total revenues, also appear strongly downward biased.

Errors in the output projection tend to explain a substantial share of revenue errors across most countries, including Canada. In a second battery of mean tests, forecast errors for macroeconomic variables were added to the right-hand side of the test regression. Whereas inflation and unemployment rate forecast errors failed to affect test outcomes, either nominal GDP or real growth errors eliminated much of the apparent bias in revenue forecasts across most countries. In the case of Canada, the null hypothesis of unbiased forecasts was no longer rejected once nominal GDP errors were included, suggesting a close approximation of the country’s tax base.23 Given the typically small share of unemployment assistance and other cyclically sensitive components in total government expenditure, it is not surprising that macroeconomic variables appear to have a lesser influence on the outcome of expenditure projections, with exceptions including Sweden and Switzerland and some spending components in the United States and Germany.

Finally, tests of forecast efficiency suggest that Canadian budget forecasters may not have employed all of the information available. Under an “efficient” forecasting process, forecasters would update their forecasting models to take into account any source of systematic forecast errors, such as a permanent improvement of a country’s growth prospects. As a result, forecast errors would at least be independently if not normally distributed. This hypothesis is rejected for Canadian growth and revenue estimates, as well as for a number of variables for Germany, the Netherlands, the United Kingdom, and the United States (Table 4.9). Consistent with the results of this test, Canada is also one of the few countries to exhibit strong autocorrelation in both tax and nontax revenue errors.

Table 4.9.Results of Efficiency Tests
AustraliaCanadaFranceGermanyItalyNethelandsNew Zealand2Sweden2SwitzerlandUnited KingdomUnited States
Joint significance tests1
Nominal GDP−(8)FC (9)−(8)FC (9)C (9)C (9)−(9)−(4)−(9)FC (6)−(9)
Real GDP growth−(9)C (8)−(6)FC (9)FC (8)−(9)−(6)−(4)−(9)C (5)FC (9)
GDP inflation2−(9)−(8)FC (6)FC (9)−(9)−(4)FC (9)−(5)−(9)
Unemployment rate2FC (9)−(8)C (3)FC (9)−(6)−(4)−(9)
Government revenue−(8)C (9)−(6)−(9)C (6)FC (6)−(8)FC (4)−(9)C (6)C (9)
Tax revenue−(8)−(9)C (6)−(9)−(9)−(9)FC (4)−(6)C (9)
of which:
Personal income tax2−(9)FC (9)−(6)−(9)FC (4)C (6)−(9)
Corporate income tax2C (9)−(9)FC (6)−(9)−(4)C (6)FC (9)
Social insurance taxes2FC (6)−(9)
Indirect taxes3−(9)−(9)FC (6)−(9)−(4)−(6)FC (9)
Other revenue−(8)FC (9)FC (9)FC (5)FC (8)−(4)FC (6)FC (9)
Government expenditure−(8)−(9)−(6)−(9)−(6)−(6)−(8)−(7)−(9)FC (6)C (9)
Mandatory expenditure−(9)−(9)FC (9)
Discretionary expenditure−(9)FC (9)FC (9)
Interest expenditureFC (9)−(7)−(9)−(6)FC (9)−(9)C (4)−(9)
Error autocorrelation2
Nominal GDP3
Real GDP growth1
Government revenue132
Tax revenue233
Other revenue31
Government expenditure2
Source: IMF staff calculations. See Mühleisen and others (2005) for a description of the underlying methods.

Budget versus Private Sector Forecasts

One measure of comparing budget forecasts against each other is to study how they hold up against private sector forecasts in the respective country. For that purpose, one-year budget forecasts were compared with consensus projections for growth and the fiscal balance, taken from the month when the corresponding budget was released. Descriptive statistics for consensus projection errors reveal that their magnitude is generally close to those for budget forecast errors and that neither growth nor fiscal forecast errors are consistently larger for public or private forecasters across countries.

Differences in government and private sector forecast errors in Canada are relatively small. Private sector forecasts exhibit a slightly smaller RMSE for growth and fiscal forecasts than those of the government, similar to the cases of Italy and New Zealand. Although the difference in the growth forecast appears rather minor—reflecting the fact that budget forecasts are largely based on macroeconomic projections provided by private forecasters—the test of RMSE equality is rejected at relatively high confidence levels. As for the fiscal forecast, anecdotal evidence suggests that the private sector is usually focusing on the underlying budgetary balance (the simple difference between federal revenues and expenditures, excluding the economic prudence and contingency reserve; Figure 4.2). The difference in RMSEs indeed becomes statistically insignificant once that concept is used.

Figure 4.2.Canada: Fiscal Balance Forecast Errors

(Forecast error in percent of size of government)

Source: IMF staff calculations.

Tests for statistical dominance have also proved inconclusive. While a visual inspection suggests that the difference between the two sets of projections is small relative to the magnitude of the overall error, a formal test can also be used to analyze whether one of the forecasts statistically encompasses the other. These tests often yield inconclusive results—such as when coefficients are estimated with similar magnitude but opposite sign, as in the case of the Canadian growth forecast. The fiscal forecast contained in Canada’s budgets appears somewhat weak relative to the consensus, but the only clear-cut cases of statistical dominance relate to fiscal forecasts in Italy and New Zealand, where the private sector appears to have a clear edge over the government, and vice versa in France.

Factors Affecting Forecast Errors

This subsection relates forecast performance to major characteristics of the fiscal environment, as well as to measures of underlying economic volatility. The approach follows Strauch, Hallerberg, and von Hagen (2004), who analyzed whether budget forecasts by euro area countries were influenced by elections or institutional factors. Accordingly, some of the information collected in subsections of this paper has also been used for empirical testing (a list of variables is contained in Table 4.10).

Table 4.10.Potential Factors Affecting Forecast Outcomes
VariableDescription
Federal structure (dummy variable)Presence of a federal political structure.
Fiscal rule (dummy)Presence of a fiscal rule (see Table 4.4).
Expenditure ceiling (dummy)Presence of format expenditure ceiling.
Deficit ceiling (dummy)Presence of a formal deficit ceiling.
AppropriationShare of budget expenditure subject to appropriation (midpoint of range; see Table 4.1)
Regulatory framework (dummy)Number of aspects regulated by the constitution or by law (see Table 4.1).
Budget reportingNumber of OECD Best Practices met (see Table 4.1).
Accountability framework (dummy)Positive response to the question whether a formal comparison is made between the medium-term fiscal policy objectives and the government’s annual budget with explanations given for any deviations.
Performance assessment (dummy)Regular, occasional, or no external ex post assessment of forecasting performance (see Table 4.6).
Budget lead time (dummy)Average number of months between submission of the budget and the budget vote (see Table 4.1).
Prudential framework (dummy)Combination of “Prudential framework 1” and “Prudential framework 2.”
Prudential framework 1 (dummy)Positive response to the question whether there is an explicit “prudence” factor built into the economic assumptions which reduces the final economic estimates by a set amount.
Prudential framework 2 (dummy)Positive response to the question whether growth assumption underpinning the medium-term fiscal framework contains a margin of “prudence” vis-à-vis the forecast.
Stable tax revenueAverage share of personal income, social security, and indirect tax revenue in total revenue (1991–2002).
Mandatory expenditureAverage share of mandatory expenditure in total central government expenditure.
TransfersShare of transfer payments to subnational governments in total central government expenditure (see Table 4.3).
Sources: OECD and World Bank (2003); and IMF staff calculations.

The paper also tests the hypothesis that strong fluctuations in a country’s economy could affect the accuracy of budget forecasts. For example, commodity-exporting countries such as Australia, Canada, and New Zealand could be expected to suffer from larger and more frequent exogenous shocks than other countries. Given economic models’ limitations in predicting turning points, this could make economic projections more difficult.

Indeed, Canada has experienced greater macroeconomic volatility than many other countries:

  • Overall, Canada registered the third highest output volatility among benchmark countries between 1990 and 2003. Short-term interest rates also fluctuated relatively strongly during that period, but other macroeconomic variables, including consumer price inflation, business sector wages, and the nominal effective exchange rate, remained comparatively stable.

  • However, fiscal aggregates have not been significantly more volatile in Canada than in other countries. Volatility in Canada’s expenditure-to-GDP ratios was higher than in many benchmark countries. This could partly reflect policy-induced changes in the expenditure ratio, such as cutbacks in spending on economic affairs (subsidies) and social protection related to consolidation in the 1990s, as well as sharp reductions in public debt payments. By contrast, Canada’s revenue volatility (measured relative to the size of GDP) has been lower than in any of the other 10 countries—with the exception of corporate income tax revenue, which may have been particularly affected by export volatility.24

The results suggest that structural characteristics of the fiscal environment have limited explanatory power for cross-country differences in forecast errors. For the most important variables contained in budget forecasts, a series of simple OLS regressions of MEs and RMSEs on a constant and one of the structural variables yields few significant results.25 The conservative stance of Canada’s forecasts is consistent with some of the findings, but there are also counterintuitive relationships:

  • There is some evidence that stronger accountability reduces the RMSE for the growth and tax revenue forecast, but that a federal structure has the opposite effect.

  • In countries where the budget is presented to the legislature early, revenues appear to be harder to forecast. However, this result may be influenced by a coincidence with recent policy shifts in the United States, which has the largest budget lead time.

  • There is weak evidence that deficit and expenditure ceilings coincide with conservative revenue estimates.

  • Fiscal rules are associated with overly optimistic forecasts, although the same applies to countries with a high share of voted appropriations. A higher share of mandatory expenditure is positively correlated with the forecast error for government spending.26

On the other hand, the evidence that forecasts tend to be more conservative in the presence of macroeconomic and fiscal volatility is relatively strong. In particular, a more volatile GDP growth environment pushes growth and, by implication, revenue forecast errors downward while leaving expenditure forecasts unaffected.

In some equations, volatility indicators and institutional features were found to be jointly significant. A combination of growth volatility and prudence indicators was found to provide the best explanation for fluctuations in MEs and RMSEs across benchmark countries, with volatility being consistently and more strongly significant across the range of regressions carried out. Paradoxically, a more formalized accountability framework and stricter requirements for assessing fiscal policy were found to be associated with overly optimistic expenditure forecasts. This may be due to “adverse selection”—formal accountability may have been strengthened particularly in countries with expenditure discipline problems.

It remains unclear whether these findings can fully explain the difference between forecast errors in Canada and other countries. On the one hand, the existence of a mean error/bias for growth and revenue forecasts in Canada appears to be fully explained by a combination of prudence indicators and macro volatility. For example, the predicted value for the mean error of Canada’s nominal GDP forecasts is close to the actual value (Figure 4.3), suggesting that forecasters in other countries would likely arrive at the same outcome if they were operating in Canada’s forecasting environment. On the other hand, the RMSE—which is a better measure for overall forecast quality—appears little affected by macro volatility, and Canada remains the country with the second highest residual in the bottom chart of Figure 4.3. Further research—based on more comprehensive data and more refined economic models— would be needed to shed greater light on the relationship between the fiscal forecasting environment and forecast accuracy.27

Figure 4.3.Impact of GDP Volatility on Forecast Quality

(Forecast errors of growth rates in annual percent change)

Source: IMF staff calculations.

1Estimation of the mean error between real GDP growth forecasts and the actual results regressed against the volatility of real GDP growth and the budget lead time.

2Estimation of the root mean squared error between real GDP growth forecasts and the actual results regressed against the volatility of real GDP growth and prudence indicators.

Conclusions

This study suggest that, between the mid-1990s and 2004, Canadian budgets followed a cautious forecasting approach. A descriptive analysis shows Canada with larger and more conservative fiscal forecast errors than most other countries. The study also finds that Canada’s aggregate forecast error is composed of small but consistently one-sided errors in fiscal subcomponents, which appears characteristic of a conservative forecasting approach.

A considerable part of this outcome appears to be related to a forecast bias in the macroeconomic component. This finding may be partly a consequence of Canada’s economic environment at the time, given the link between macroeconomic volatility and pessimistic growth projections. Moreover, Canadian forecasters were not unique in underestimating the global boom of the late 1990s. Although prudence adjustments in budgets of the middle to late 1990s also led to a slight increase in forecast errors, macro projections were likely affected by the fact that Canada unexpectedly outperformed other industrial countries throughout much of the period.

However, other factors are also likely to have played a role. Budget forecasters have had to cope with considerable uncertainty relating to the size of provincial transfers and tax-sharing arrangements, which were exacerbated by the relatively large size of provincial budgets relative to the federal government. Moreover, the economic literature suggests that a conservative budgeting approach constitutes a rational response to a regime where the costs of missing a fiscal target are both high and asymmetric, as has been the case in Canada over the past 10 years.

Canada could benefit from further improving the transparency of its budgetary forecasts. Given the importance of restoring public confidence in government finances in the mid-1990s, the consequences of running into deficit were considerably higher than those of achieving a surplus. As Canada’s fiscal situation has improved, it is unclear the extent to which the relative costs of missing budget targets have changed. However, Canada could benefit from opening up the forecasting process, for example, by involving private forecasters in producing revenue estimates. Equally important, providing more information about critical parts of the forecasting process—in particular, the assumptions and methods used for transforming macroeconomic forecasts into fiscal projections—would invite greater outside scrutiny, helping to improve forecast quality and bolster public confidence in budget projections.28

The authors are grateful to colleagues in the participating countries as well as in the IMF’s European and Asia and Pacific Departments for their cooperation in acquiring and analyzing the data for this study.

Japanese fiscal policy in the middle to late 1990s was largely implemented through supplementary budget requests, which would complicate a comparison of its budget projections with other countries. Japan was therefore not included in the benchmark group.

Switzerland was not part of the OECD-World Bank survey.

See OECD and World Bank (2003). In the United States, many states have fixed limits in their constitutions. Similar to most comparator countries, the Canadian federal government does not guarantee the debt of subnational governments.

Equalization transfers (to reduce economic disparities among provinces) and transfers for health and social spending are the most important transfers, amounting to 1 and 3 percent of GDP, respectively.

For example, the introduction of fiscal policy constraints in euro area countries led to the adoption of binding multiyear targets, supplemented with more detailed descriptions of countries’ fiscal plans.

Canada’s “Fiscal Spending Control Act” was in force only between 1991 and 1994.

See Kopits and Symansky (1998) and Dabán and others (2003) for a detailed discussion of fiscal policy rules. The EU’s Stability and Growth Pact mandates that deficits do not exceed 3 percent and that the debt-to-GDP ratio remains less than 60 percent. Medium-term targets must be authorized by the legislature in Italy and the United States.

The conservative government that took power in 2006 abandoned this rule in favor of targeting (without committing to) a moderate surplus of about ¼ percent of GDP.

Prudence was incorporated into the fiscal projections used for the budgets from 1994 until 1998 by explicitly adopting economic assumptions that were more pessimistic than the average of the private sector economic forecasts, including higher interest rates and weaker economic growth.

The share of interest payments decreased from 20 percent in 1990 to 9 percent in 2003.

For a more complete analysis, see Mühleisen and others (2005).

Sources for this information include country responses to a short staff questionnaire, an OECD and World Bank survey on budget institutions (OECD and World Bank, 2003), and available IMF Reports on the Observance of Standards and Codes. The questionnaire covered the development and organization of the forecasting process, as well as arrangements for quality control and transparency.

Beginning with the 2004 Economic and Fiscal Update, the government committed to provide additional information on how national-accounts-based fiscal projections provided by private sector forecasters translate into the accounting framework used in the budget.

Given the small number of countries providing medium-term projections, forecasts for three and more years were not considered. Also, central government forecasts were not available for a number of countries, and so general government forecasts were used.

For a description of available data and methodological issues, see Mühleisen and others (2005).

Errors are defined as projected minus actual values. A negative value therefore implies that the outcome has exceeded expectations, and vice versa.

Indeed, the consequences of U.S. tax and spending measures were well anticipated at the time of passage.

For this study, the base year (or “in-year”) is the year preceding the budget year (for example, the base year for the FY2004–05 budget is FY2003–04). Although a similarly large base-year error was only found for the United States, cross-country comparisons involving the GDP deflator suffer from the fact that inflation forecasts were not available for some countries and had to be calculated as the difference between the nominal and real GDP growth rates, with base-year values substituting for actual values.

Economic prudence and contingency were categorized neither as revenue nor expenditure, with the result that the discrepancy between projected and actual deficits in Canada is larger than the difference between the revenue and expenditure errors. Redefining the projected deficit as the difference between revenue and expenditure projections corrects for this.

The forecast error for debt service charges also stems partly from a prudence adjustment to the interest rate forecast in the late 1990s, although this effect could not be quantified.

See Muhleisen and others (2005) for a description of the tests.

Among countries with a significant nominal GDP coefficient, the measured elasticity of revenue errors was between 1½ and 2, with Canada in the middle (1½) and the United States at the high end.

For comparing volatility across countries, fiscal aggregates have been divided by GDP. Sources of volatility include policy changes, such as enhanced public expenditure programs in the United Kingdom after 2000, expenditure cuts in Canada or Sweden during the 1990s, or tax cuts in the United States. The results are not corrected for this fact, both because it can be argued that volatility stemming from policy changes also contributes to a more difficult forecasting environment, and because estimates of non-policy-induced volatility are not available for most countries.

Each of these regressions is run with a maximum sample of only 11 observations, depending on the number of countries for which information was available.

The results are robust in the sense that they hold even if different countries are removed from the sample.

Panel estimations are particularly affected by data shortcomings and contribute little additional information. However, time dummies for the late 1990s have generally been significant in regressions covering fiscal variables, suggesting that surprises from a strong global growth environment have not been confined to Canada.

Examples include Australia and New Zealand, which have adopted transparency legislation to boost public understanding of fiscal forecasts, whereas in other countries—such as Germany and the Netherlands—academic bodies or independent agencies participate in the forecast. IMF (2002) also provides suggestions for expanding the information content of the Economic and Fiscal Update.

    Other Resources Citing This Publication