Structural Models in Real Time1
Author:
Kevin Clinton https://isni.org/isni/0000000404811396 International Monetary Fund

Search for other papers by Kevin Clinton in
Current site
Google Scholar Close
,
Marianne Johnson https://isni.org/isni/0000000404811396 International Monetary Fund

Search for other papers by Marianne Johnson in
Current site
Google Scholar Close
,
Mr. Jaromir Benes
Search for other papers by Mr. Jaromir Benes in
Current site
Google Scholar Close
,
Mr. Douglas Laxton
Search for other papers by Mr. Douglas Laxton in
Current site
Google Scholar Close
, and
Mr. Troy D Matheson
Search for other papers by Mr. Troy D Matheson in
Current site
Google Scholar Close

Contributor Notes

Author’s E-Mail Address: jbenes@imf.org; kclinton@rogers.com; mjohnson@bank-banque-canada.ca; dlaxton@imf.org; tmatheson@imf.org
Publication Date:
01 Mar 2010
eISBN:
9781451963625
Language:
English
Keywords:
WP
Download PDF (318.3 KB)

This paper outlines a simple approach for incorporating extraneous predictions into structural models. The method allows the forecaster to combine predictions derived from any source in a way that is consistent with the underlying structure of the model. The method is flexible enough that predictions can be up-weighted or down-weighted on a case-by-case basis. We illustrate the approach using a small quarterly structural and real-time data for the United States.

Abstract

This paper outlines a simple approach for incorporating extraneous predictions into structural models. The method allows the forecaster to combine predictions derived from any source in a way that is consistent with the underlying structure of the model. The method is flexible enough that predictions can be up-weighted or down-weighted on a case-by-case basis. We illustrate the approach using a small quarterly structural and real-time data for the United States.

I. Introduction

Informed monitoring, using judgment, can outperform short-term forecasts of structural models.1 And since announcements, special factors and the uncertain timing of some events – for example, the bursting of an evident market bubble – have important sporadic effects, there will always be a role for judgment in forecasting. A common practice at central banks, even where the staff forecast is model based, is to set the values for the current quarter (the nowcast) and next quarter by closely monitoring high frequency indicators (HFIs). Staff update forecasts through the quarter as new information arrives, not just to increase accuracy, but also to provide a narrative on the outlook that incorporates the most recent events. Leeper (2003), however, criticizes the theoretical ambiguity and lack of transparency that this process may involve. In principle, it would be better, for inter-quarter updates, as well as more formal quarterly exercises, to incorporate arriving data directly into the model, the structure of which embodies the staff’s view of the relevant characteristics of the economy. Such a process would oblige forecasters to provide a systematic rationale for their adjustments; and monitoring would be more systematically replicable, and less dependent on particular experts.

This paper takes up Leeper’s suggestion, describing a practical methodology to improve the real-time forecasting of a model, through systematic input of outside information. The latter may come from separate higher-frequency model forecasts or from judgment. The resulting forecast for a given variable will be a weighted average of the extraneous predictions (there may be more than one) and the unmodified current model forecast. The weights attached to each may be imposed by judgement, varied with the forecast horizon, or calibrated on the basis of past performance.

To illustrate the methodology for the United States, we use a Small Quarterly Structural Model (SQSM) that incorporates a strong real-financial linkage (Carabenciov and others, 2008, 2008a). The results illustrate the extent to which intra-quarterly information improves short-term forecasting performance. Examining the estimated standard deviations around predictions derived from our procedure reveals that the conventional practice of assuming that short-term predictions are actual data understates the degree of uncertainty around the resulting forecasts.

II. Background

Structural macroeconomic models make use of theoretical and empirical relationships between economic and financial variables. As simplifications of reality, they do not attempt to predict well over very short horizons. Central banks in the industrialized countries typically conduct a quarterly exercise that involves a formal model-based forecast, with a multi-year horizon, and less formal intra-quarter updates of the near-term outlook, perhaps on a monthly or weekly schedule (e.g., Macklem, 2002). For the model-based forecast, the staff set current-quarter values of the variables (the nowcast) largely on the basis of their monitoring of the most recent information, which includes an array of high-frequency data and news on current developments. In essence, the forecasters override the model, to impose a short-term forecast that looks more realistic given the available information. From these initial conditions, the model then provides forecast paths for the endogenous variables.

A few researchers, however, have been working in the direction advocated by Leeper, to derive the short-term forecast from the model, with the relevant high frequency data as input. Boivin and Giannoni (2006, 2008) develop a dynamic factor model framework to incorporate rich datasets into a Dynamic Stochastic General Equilibrium (DSGE) model. Their results demonstrate that high-frequency data may have significant content for the estimation, as well as the forecasting performance, of structural models: the additional information may compensate for missing variables and measurement error. Mestre and McAdam (2008) do not use high frequency data, but compare forecasts of a structural model and time-series models. They find, for the euro area, that simple models of observed residuals can substantially improve short-term forecasting accuracy of both structural and time-series models. Gomez and others (2009) combine the DSGE model of the Central Bank of Colombia with mixed frequency variables not in the model itself. At a given point in time, this poses a missing observation problem, which the authors solve by mixed interval smoothing with the Kalman filter. This technique, however, has yet to be adapted to real-time forecasting.

In contrast to our approach, most of the high-frequency literature concerns the automation of the current analysis process, and does not have as a starting point an economic model suitable for medium-term forecasts and policy analysis. Data rich forecasting approaches include:

In an international study, Jakaitiene and Dees (2009) evaluate various versions of these techniques, and global versus individual-country approaches. The results are not clear cut, as no one method dominates.

III. The Real Time Problem

Asynchronous release of data provides a data set that has a ‘jagged edge’ at each point in time, as illustrated in Table 1 for the variables in SQSM.

Table 1:

Data available at the beginning of quarter Q

article image
X denotes available data.

In this case, at the beginning of the first month of any quarter Q, the forecaster has a set of data comprising: quarter Q –1 for the interest rate (R) and an indicator of credit conditions (BLT); quarter Q – 2 for GDP; quarter Q – 2 for the CPI and the unemployment rate (U), as well as the first two months of Q – 1.

IV. Four Options for Incorporating External Information Into a Forecast

A. Model-Based Forecast with Data Set Truncation

One might truncate the entire data set, so that forecasting begins from the most recent quarter where a complete quarterly data set is available. A quarterly forecast made at the beginning of the first month of a quarter would use actual data for Q – 2 as initial conditions. Higher-frequency data arriving later would be ignored. This method produces forecasts that are consistent with the model, but it has the obvious, and serious, disadvantage of wasting timely information.

B. Model-Based Forecast with Incomplete Data Set

One might use all available quarterly data, and solve the model for the missing quarterly observations. In the example, we would derive estimates from the model for Q – 1, and onwards, conditional on the interest rate and BLT data up to Q – 1, and on the data for other variables up to Q – 2. While this approach is less wasteful than the previous option, it still neglects higher frequency (e.g. monthly and daily) data from Q – 1, and Q.

C. Hard Tunes — Setting Values in A Model-Based Forecast Based on Extraneous Analysis

Forecasters at central banks and other institutions routinely use estimates derived outside their main forecasting model to fill in for incomplete data. Recent information from a variety of sources – such as equations based on ‘leading indicators’, high-frequency financial data, business and labor market news, surveys of business and consumer sentiment, and so on – informs an estimate for last quarter, Q – 1, and a nowcast for current quarter, Q. The methods used to construct the estimates may vary over time, and may not be easily replicable.2

This is the process that Leeper criticizes. Knowledge about the economy embodied in the structural model is ignored. Moreover, it is difficult to provide quantitative measures of the uncertainty in such predictions.

D. Soft Tunes — Combined Approach

We propose to impose off-model information on the estimates of the structural model by adding more structure to the model’s measurement equations - the equations relating the state variables of the model to the observed data. This methodology is flexible, both in terms of the information that can be incorporated, and its timing.3

Different types of data can be incorporated: higher-frequency observations can be used as indicators for quarterly variables; external estimates; or dynamic-factor models that summarize the information content of large quantities of variables. Information may relate to future months or quarters, e.g. the Federal funds futures market rate, or to official announcements, e.g. on the budget, or to an external projection, e.g. on population growth.

To illustrate, consider a extraneous prediction for quarterly inflation:

π t = π t ̂ + ε t ( 1 )

where πt is inflation, π̂t is a prediction of inflation, and εt is a prediction error. Suppose we use a simple bridge equation, based on the monthly data, to predict the quarterly inflation rate for period t. As the quarter proceeds, step by step the monthly information accumulates, and the prediction error for inflation declines until, when the inflation observation for the quarter arrives, the prediction error falls to zero. Essentially, our approach is to add the prediction equation (1) to the measurement equations of our structural model, raising the weight on π̂t relative to the pure model estimate πt as new data arrive.

The influence that the prediction of inflation has on the final model estimate is governed by the variance of the prediction error εt. Essentially, this approach nests the pure-model and hard-tuning methods described above. With var(εt) = ∞ the combined approach reduces to the pure-model approach, while var(εt) = 0 implies hard tuning.

A key difference between our method for solving the real-time problem and those outlined above is that we can allow multiple predictions for a single variable to enter the model. We could have, for example, n bridge equations for inflation in the current quarter:

π t = π ^ t 1 + ε t 1 ⋮ ⋮ ⋮ ⋮ π t = π ^ t n + ε t n ( 2 )

The weight within the augmented structural model on any single prediction, π^ti, would then depend on relative size of the estimate of var(εti). Roughly speaking, the final model estimate of inflation will be a weighted average of the n outside predictions and a model estimate, where the weights are inversely related to the standard deviations of the prediction errors. Notice that the structure is consistent with a wide range of alternative sources of predictions. For example, the bridge equations could be designed to exploit cross-correlation amongst the prediction errors.

In the case of a model- or equation-based prediction, the magnitude of the prediction error can be easily derived. Forecasters making judgment-based predictions, on the other hand, need to quantify the expected size of the errors around their predictions. As recommended by Leeper, this makes the process of incorporating short-term forecasts into the structural model more transparent. It could also be used to make short-term forecasters more accountable for their judgments over time.

V. Application

We conduct a recursive experiment in which we make quarterly forecasts in each month from 1998Q1 to 2009Q2. The forecasts are made in simulated real time using the actual data that were available at the time.4 The model, SQSM, is similar to the U.S. sector of the Global Projection Model (Carabenciov and others, 2008).

We experiment with 3 of the approaches described above: data set truncation; hard tunes; and soft tunes. With one exception, our hard tune for a variable is the simple average of the available monthly data. Thus, the forecast for the current quarter made in the second month is the number reported for the first month (averaged or grossed up to a quarterly rate as appropriate); likewise, the forecast in the third month is based on the data available for the first two months. The exception is GDP growth, for which we use the Blue Chip Economic Indicators (BCEI) nowcast and one-quarter ahead forecast, which BCEI update (approximately) on the 10th of each month.

Each time a forecast is made, we calculate the variances of the prediction errors from performance up to that point – and hence re-weight the various predictors. For example, the relevant variance for the soft tune of the unemployment rate for the nowcast in the third month of the quarter is the historical variance of the errors of the 2-month average as a predictor of the quarter.

VI. Application: Real-Time Data

We examine forecasting accuracy over 0-to-12-quarter horizons. For the purposes of this paper, monthly forecasts are made following the release of the CPI, and incorporate new information in discrete monthly batches, with data arrival on the actual historical release dates. The calendar in table 2 shows the sequence of forecasts and the associated data releases for a representative quarter.

Table 2:

Indicative calendar of data releases (example is 2009Q1)

article image

The main variables are: the Federal funds rate (monthly average); Bank Lending Tightening, BLT, based on the Senior Loan Officer Survey (quarterly); the unemployment rate (monthly); the core CPI inflation rate (monthly); and BCEI forecasts of quarterly real GDP (updated monthly).5 BCEI calculate the average, or consensus, forecast from a survey of 50 U.S. business economists and professional forecasters conducted in the first few days of each month. We use the consensus forecast to illustrate the use of an extraneous estimate in the model forecast.

VII. Application: The Small Quarterly Structural Model (SQSM)

A. Behavioral Equations

A.1 Output gap

Equation 3 is a behavioral equation that relates the output gap (yt=Yt−Y¯t) to its own lead and lagged values, lagged values of the short-term real interest rate gap (rrgapt), shocks to bank lending as represented by the variable ηt, and a disturbance term:

y t = β 1 y t − 1 + β 2 y t + 1 − β 3 rrgap t − 1 − θ η t + ε t y ( 3 )

All variables in this equation are gaps, i.e. deviations from equilibrium values.

A.2 Core inflation

Equation 4 links core inflation to its own history (a 1-quarter lag), and to its future rate as predicted by the model (a 4-quarter lead). It also includes the lagged output gap, as per the usual short-run Phillips curve trade-off, and a disturbance term.

π t = λ 1 π 4 t + 4 + ( 1 − λ 1 ) π 4 t − 1 + λ 2 y t − 1 − ε t π ( 4 )

A.3 Interest rate

Equation 5 is a Taylor-type, forward-looking, policy reaction function that determines the short-term nominal interest rate as a function of: the long-run equilibrium rate, rr¯t; the deviation of the expected year-on-year rate of inflation 3 quarters in the future from the policy target πtar; and the output gap.

rs t = ( 1 − γ 1 ) [ rr ¯ t + π 4 t + 3 + γ 2 ( π 4 t + 3 − π tar ) + γ 4 y t ] + γ 1 rs t − 1 + ε t rs ( 5 )

The equation also includes a disturbance term (εtrs) to allow for discretionary policy actions. The equation applies only in so far as it predicts a non-negative interest rat the model recognizes the zero lower bound. This nonlinearity is a material feature o: the model in an environment of deflationary risks.

A.4 Unemployment rate

Equation 6 is a dynamic version of Okun’s law where the unemployment gap is a function of its lagged value, the contemporaneous output gap and a disturbance term (εtu).

u t = α 1 u t − 1 + α 2 y t + ε t u ( 6 )

The historical correlation between changes in the output gap, and future changes in the unemployment gap, provides information that improves empirical estimates of these unobservable variables.

A.5 Financial-real linkage

Carabenciov and others (2008) find that changes in bank credit conditions can account for a large part of the fluctuations in U.S. economic activity. Equation 7 writes BLTt as a function of the equilibrium value, BLTt¯, which is defined to be a random walk (equation 8).

BLT t = BLT t ¯ − κ y t + 4 + ε t BLT ( 7 )
BLT t ¯ = BLT ¯ t − 1 + ε t BLT ¯ ( 8 )

Equation 7, assumes bank-lending practices normally depend on the expected behavior of the economy 4 quarters ahead. A positive output gap (high capacity utilization) tends to ease lending conditions, whereas a negative gap (excess capacity), creates tighter conditions. The stochastic term, εtBLT, captures exogenous changes to lending policies; and a distributed lag of this variable, ηt enters the equation for the output gap:

η t = 0.04 ε t − 1 BLT + 0.08 ε t − 2 BLT + 0.12 ε t − 3 BLT + 0.16 ε t − 4 BLT + 0.20 ε t − 5 BLT + 0.16 ε t − 6 BLT + 0.12 ε t − 7 BLT + 0.08 ε t − 8 BLT + 0.04 ε t − 9 BLT ( 9 )

The values of the coefficients of equation 9 impose a hump-shaped response to a change in εtBLT, peaking at the 5th quarter.

A.6 Stochastic processes and model definitions

Shocks may affect both the level and growth rate of potential output. Shocks to the level can be permanent, while the shocks to the growth rate can result in highly persistent deviations in potential growth from the long-run steady-state growth rate. In equation 10, potential output, Ȳt, is equal to its own lagged value plus the quarterly growth rate (gtY¯/4) plus a disturbance term (εtY¯) that can cause permanent level shifts in potential GDP.

Y ¯ t = Y ¯ t − 1 + g t Y ¯ / 4 + ε t Y ¯ ( 10 )

In equation 11, potential growth may diverge for a while from the steady-state rate gY¯ss, following a one-off disturbance (εtgY¯). The return to gY¯ss is gradual, at a pace determined by the value of the partial adjustment coefficient, τ.

g t Y ¯ = τ g Y ¯ ss + ( 1 − τ ) g t − 1 Y ¯ + ε t g Y ¯ ( 11 )

A similar set of relationships holds for NAIRU. In equation 12, U¯t is a function of its past value, a disturbance term of some persistence, gU¯, and a temporary disturbance term, εtU¯.

U ¯ t = U ¯ t − 1 + g t U ¯ + ε t U ¯ ( 12 )

Equation 13, sets gtU¯ as a function of its lagged value and the disturbance term εtgU¯. This specification allows the NAIRU to be affected by both level and persistent growth shocks.

g t U ¯ = ( 1 − α 3 ) g t − 1 U ¯ + ε t g U ¯ ( 13 )

Equation 14 defines the real interest rate, rrt, as the difference between the nominal interest rate, rs, and the expected rate of inflation for the subsequent quarter.

rr t = rs t − π t + 1 ( 14 )

Equation 15 defines rrgapt, the real interest rate gap, as the difference between rrt and its equilibrium value, rrt¯,

rrgap t = rr t − rr ¯ t ( 15 )

while equation 16 defines rrt¯, the equilibrium real interest rate, as a function of the steady-state real interest rate, rr¯ss. It may diverge from the steady state for extended periods in response to a stochastic shock, εtrr¯.

rr ¯ t = ρ rr ¯ ss + ( 1 − ρ ) rr ¯ t − 1 + ε t rr ¯ ( 16 )

A.7 Stuctural cross correlations of disturbances

The model contains two structural correlations (both positive) across error terms:

  • between εtgY¯ and εty, capturing the idea that a positive shock to potential output growth will generate an increase in expected permanent income, which raises current spending, even before the level of potential output itself increases, such that the output gap, increases

  • between εtY¯and εtπ (the error term in the inflation equation has a negative sign). This implements the notion that a positive supply shock puts downward pressure on costs and prices.

These correlations roughly mimic impulse-response functions from the DSGE model ofthe U.S. economy in Juillard and others (2007, 2008).

A.8 Bayesian parameter estimates

We estimate the parameters using U.S. data ranging from 1994Q1 to 2008Q4. Prior means, prior standard deviations, posterior modes, and posterior standard deviations for the Bayesian estimation can be found in tables 5, 6 and 7 in the appendix.

Table 3:

Current quarter RMSEs, 1998Q1 to 2009Q2

article image
Forecasts made after CPI release
Table 4:

One-quarter ahead RMSEs, 1998Q1 to 2009Q2

article image
Forecasts made after CPI release

VIII. Results

A. Real Time Forecasts

We focus on forecasts made in the period January 2007 to September 2009, which of course covers the recent financial crisis. In figure 1, each grey line presents the real-time quarterly forecasts made in final month of the quarter. The blue line represents actual outcomes.

Figure 1:
Figure 1:

Real time forecasts made in the final month of the quarter and actual data, 2007Q1 to 2009Q3

Citation: IMF Working Papers 2010, 056; 10.5089/9781451963625.001.A001

The bottom panel shows the forecasts for the BLT variable. In premininary work, we found that the inclusion of this variable in the model greatly improved forecast accuracy, reducing the short-term root mean squared error (RMSE) of the GDP forecast by around a third. Throughout 2008, as the survey reported increasingly severe credit tightening, and the outlook for the economy deteriorated. This illustrates that the additional information in high-frequency data may assume crucial importance during a crisis, or more generally during periods of high uncertainty, when the regular relationships built into the model may cease to hold.

B. Forecast Accuracy

Table 3 shows RMSEs for the nowcasts following the release of the CPI data in months 1, 2, and 3.

At the beginning of the quarter, data are available for Q –1, with the exception of GDP, which is only available up to Q – 2. This leads to a substantial difference in the first month between the RMSEs for GDP forecasts from the model with truncated quarterly data, and those for hard and soft tunes. The BCEI forecast evidently contains valuable high-frequency information that truncation forces out. For all other variables, the first month of data generates very little change, and the RMSEs for the model-based and the tuned cases are very similar.

In months 2 and 3, the release of both Q – 1 data and a revised BCEI outlook help to improve the GDP nowcasts further. The arrival of the monthly data on inflation, the interest rate, and the unemployment rate cause large declines in the RMSEs of the tuned nowcasts of these variables.

Overall, the hard tunes produce much better nowcast accuracy than the model with truncated data, and slightly better accuracy than the soft tunes. Generally, at this very short horizon the model errors are quite large relative to the errors in the prediction equations. This is recognized in the soft-tunes case by a high weight on the extraneous forecasts relative to the model predictions.

The medium-term forecasts are not generally affected by the inclusion of high-frequency data. Tables 8 to 12 provide RMSEs for forecast horizons of up to 12 quarters. At horizons beyond one year, the RMSEs tend to widen and stabilize. One might note that the RMSEs reported here are swollen by the crisis (and the unusual forecast errors that accompanied it); more generally, the forecast accuracy of SQSM at longer horizons is comparable with that of the other models.

Table 4 provides RMSEs for one-quarter-ahead forecasts. The arrival of the monthly information, and in particular the updating of the BEA’s monthly estimates of the previous quarter’s GDP, as well as the BCEI outlook, clearly lowers the RMSEs as the quarter progresses.

Tuning improves on the model forecast with truncated data by a wider margin as more intra-quarter data become available.

Generally, these results confirm that the model just with truncated quarterly data is not very good at short-term forecasting. By incorporating the high-frequency indicators, we substantially improve the model-based forecast. Morever, working within the framework of the model establishes coherence between near- and medium-term forecasts.

C. Uncertainty

The model with soft tunes can produce more realistic measures of uncertainty than the model with hard tunes in real time. In this representative example, assume that the forecaster is in the beginning of January 2009 and aims to make forecasts after the release of each piece of data in the calendar (table 2). Following each release, we can compute the model-implied standard deviations around the nowcasts and forecasts. These standard deviations represent the uncertainty around the estimates and can be used to produce fan charts in real time. For our purposes, however, we use these standard deviations as measures of uncertainty.

Figures 2 to 6 show the evolution of the derived standard deviations after each data release in 2009Q1. The vertical axis gives the size of a single standard deviation for the variable in question, for both hard and soft tunes. The horizontal axis shows the release dates as in table 2. The 4 panels in each figure refer to forecasts over horizons from 2009Q1 itself (the nowcast) to 2009Q4, respectively.

Figure 2:
Figure 2:

Uncertainty under estimated with hard tunes: Uncertainty around year-on-year GDP growth forecasts after each data release in 2009Q1

Citation: IMF Working Papers 2010, 056; 10.5089/9781451963625.001.A001

Figure 3:
Figure 3:

Uncertainty under estimated with hard tunes: Uncertainty around year-on-year core inflation forecasts after each data release in 2009Q1

Citation: IMF Working Papers 2010, 056; 10.5089/9781451963625.001.A001

Figure 4:
Figure 4:

Uncertainty under estimated with hard tunes: Uncertainty around nominal fed funds rate forecasts after each data release in 2009Q1

Citation: IMF Working Papers 2010, 056; 10.5089/9781451963625.001.A001

Figure 5:
Figure 5:

Uncertainty under estimated with hard tunes: Uncertainty around unemployment forecasts after each data release in 2009Q1

Citation: IMF Working Papers 2010, 056; 10.5089/9781451963625.001.A001

Figure 6:
Figure 6:

Uncertainty under estimated with hard tunes: Uncertainty around the output gap forecasts after each data release in 2009Q1

Citation: IMF Working Papers 2010, 056; 10.5089/9781451963625.001.A001

Consider first the model with soft tunes. Not surprisingly, we find that the uncertainty around the near-term forecasts drops with each data release through the quarter. In the case of output growth, the release of the 2008Q4 GDP data on the 30th of January, and the updating of the BCEI forecast on February 10, have a clear effect on uncertainty in the short-term forecast. For inflation, the largest reduction occurs with the release of the CPI for January, on February 20. The results for the interest rate and the unemployment rate are similar, with the data release associated with the variable being forecast causing the largest decline in uncertainty.

For the model with hard tunes, we treat the first two quarters of the forecast as if they were data, setting the standard deviation of the prediction errors to zero. Thus, the degree of uncertainty around short-term predictions drops to zero on the relevant reporting day. Clearly, this understates the degree of uncertainty. As evident from tables 3 and 4, the estimates from the model with hard tunes contain errors, yet these are not accounted for in the model’s estimates of uncertainty. In contrast, the calculation for the model with soft tunes does factor in the prediction errors, providing a more realistic representation of uncertainty. Indeed, the distance between the lines for soft and hard tunes (the grey zone in figures 2 to 6) gauges the minimum extent to which uncertainty is understated for the hard tunes.

IX. Conclusion

The purpose of this paper is to outline a transparent and consistent approach to updating model forecasts in real time. More specifically, we use new information, as it is released, to revise forecasts with a quarterly structural model. The information may derive from any source, e.g. high frequency data, leading indicators, or expert forecasts. This addresses the argument made by Leeper (2003) in favor of a unified theoretical approach to short- and long-run forecasting. In practical terms, use of this approach enables forecasters to derive the implications of new information quickly and transparently.

We apply the methodology to a small structural model and real-time data for the United States. The results indicate the extent to which short-term forecast accuracy improves through the quarter as new data are released (there is generally little effect on forecast horizons beyond a couple of quarters). They also suggest that a conventional practice, whereby forecasters impose a short-term forecast on the basis of their reading of current developments, leads to the degree of uncertainty in the short-term outlook to be understated.

Future work will apply the methodology to a wider range of models, and develop additional ways of updating intra-quarter predictions.

References

  • Beneš, J., D. Laxton, and T. Matheson, “Adding Extraneous Information to A Structural Model”, IMF Working Paper, forthcoming.

  • Boivin, J., and M. Giannoni, 2006, “DSGE Models in a Data-Rich Environment,” NBER Working Paper T0332. (Cambridge, Massachusetts, National Bureau of Economic Research).

    • Search Google Scholar
    • Export Citation

  • Boivin, J., and M. Giannoni, 2008, “Optimal Monetary Policy in a Data-Rich Environment,” draft, June 13.

  • Camacho, M., and G. Perez-Quiros, 2009, “Ñ-STING: España Short Term INdicator of Growth,” Central Bank of Spain Working Paper 0912.

    • Search Google Scholar
    • Export Citation

  • Carabenciov, I, I. Ermolaev, C. Freedman, M. Juillard, O. Kamenik, D. Korshunov, D. Laxton, and J. Laxton, 2008, “A Small Quarterly Multi-Country Projection Model,” IMF Working Paper 08/279 (Washington: International Monetary Fund)

    • Search Google Scholar
    • Export Citation

  • Carabenciov, I, and J. Laxton, 2008a, “A Small Quarterly Multi-Country Projection Model with Real-Financial Linkages and Oil Prices,” IMF Working Paper 08/280 (Washington: International Monetary Fund).

    • Search Google Scholar
    • Export Citation

  • Evans, M.D.D., 2005, “Where are We Now?”: Real Time Estimates of the Macro Economy,” The International Journal of Central Banking, pp. 127–75, September.

    • Search Google Scholar
    • Export Citation

  • Giannone, D., Reichlin, L. and D. Small, 2008, “Nowcasting: The Real-Time Informational Content of Macroeconomic Data,” Journal of Monetary Economics Vol. 55(4), pp. 665–76.

    • Search Google Scholar
    • Export Citation

  • Gomez, A.G., M. Lavan, R.G. Diego, and L. E. Rojas, 2009, “Monetary Policy Forecasting in a DSGE Model with Data that is Uncertain, Unbalanced and About the Future,” Borradores de Economia No. 559.

    • Search Google Scholar
    • Export Citation

  • Groen, J. and G. Kapentanios, 2008, “Revisiting Useful Approaches to Data-Rich Macroeconomic Forecasting,” Federal Reserve Bank of New York Staff Report No. 327, October.

    • Search Google Scholar
    • Export Citation

  • Jakaitiene, A., and S. Dées, 2009, “Forecasting the World Economy in the Short-term,” ECB Working Paper Series No 1059 (Frankfurt: European Central Bank).

    • Search Google Scholar
    • Export Citation

  • Juillard, M., O. Kamenik, M. Kumhof, and D. Laxton, 2007, “Measures of Potential Output from An Estimated DSGE Model of the United States,” paper presented at a workshop on Issues in Measuring Potential Output (Ankara Turkey).

    • Search Google Scholar
    • Export Citation

  • Juillard, M., O. Kamenik, M. Kumhof, and D. Laxton, 2008, “Optimal Price Setting and Inflation Inertia in a Rational Expectations Model,” Journal of Economic Dynamics & Control, Vol. 32, pp. 2584–2621.

    • Search Google Scholar
    • Export Citation

  • Leeper, E., 2003, “An Inflation Reports Report,” Sveriges Riksbank Economic Review, Vol. 2003:3, pp. 94–118.

  • Macklem, Tiff, 2002, “Information and Analysis for Monetary Policy: Coming to a Decision,” Bank of Canada Review, Summer.

  • Mestre, R., and P. McAdam, 2008, “Is Forecasting With Large Models Informative? Assessing the Role of Judgement in Forecasts,” ECB Working Paper No 950 (Frankfurt: European Central Bank).

    • Search Google Scholar
    • Export Citation

  • Mariano, R.S., and Y. Murasawa., 2003, “A New Coincident Index of Business Cycles Based on Monthly and Quarterly Series,” Journal of Applied Econometrics, Vol. 18(4), pp. 427–43.

    • Search Google Scholar
    • Export Citation

  • Rojas, A., E.M. Bańbura and G. Rünstler, 2008, “Estimating and Forecasting the Euro Area Monthly National Accounts from a Dynamic Factor Model,” ECB Working Paper No. 953 (Frankfurt: European Central Bank).

    • Search Google Scholar
    • Export Citation

  • Romer, C.D and D. H. Romer, 2000, “Federal Reserve Information and the Behavior of Interest Rates,” The American Economic Review, Vol. 90, pp. 459–457.

    • Search Google Scholar
    • Export Citation

  • Sims, C.A., 2002, “The Role of Models and Probabilities in The Monetary Policy Process,” Brookings Panel on Economic Activity, Vol. 2, pp. 1–62.

    • Search Google Scholar
    • Export Citation

  • Tulip, P., 2005, “Has Output Become More Predictable? Changes in Greenbook Forecast Accuracy,” Finance and Economics Discussion Series, No. 2005-31, Federal Reserve Board, Washington, D.C.

    • Search Google Scholar
    • Export Citation

  • Zheng, I.Y., and J. Rossiter, 2006, “Using Monthly Indicators to Predict Quarterly GDP,” Bank of Canada Working Paper 2006-26.

Appendix: Data Definitions

article image
Table 5:

Results From Posterior maximization (parameters)

article image
Table 6:

Results From Posterior Maximization (standard deviation of structural shocks)

article image
Table 7:

Results from posterior parameters (correlation of structural shocks)

article image
Table 8:

Quarter-on-quarter GDP growth: RMSE, 1998Q1 to 2009Q2

article image
Forecasts made after CPI release
Table 9:

Year-on-year GDP growth: RMSE, 1998Q1 to 2009Q2

article image
Forecasts made after CPI release
Table 10:

Year-on-year core (CPI) inflation: RMSE, 1998Q1 to 2009Q2

article image
Forecasts made after CPI release
Table 11:

Fed Funds Rate: RMSE, 1998Q1 to 2009Q2

article image
Forecasts made after CPI release
Table 12:

Unemployment rate: RMSE, 1998Q1 to 2009Q2

article image
Forecasts made after CPI release
1

The views expressed in this paper are those of the authors and do not necessarily represent those of the IMF or IMF policy. The forecasts presented in this paper are not official IMF forecast. We thank Sharon Kozicki and Larry Schembri for useful comments, and Chunan Cao and Serhat Solmaz for their valuable research assistance. Marianne Johnson is a Research Advisor at the Bank of Canada. The remaining authors work in the Fund’s Economic Modeling Unit of the Research Department. The programs used in this paper can be downloaded from www.douglaslaxton.org

1

For example, Romer and Romer (2000) and Sims (2002) document the relative accuracy of Federal Reserve “Greenbook” forecasts. Tulip (2005) finds that the short-run (but not longer-run) accuracy of the Greenbook forecasts increased after 1984.

2

In practice, of course, forecasters conduct external evaluations of forecast errors and this information about the variance of the error term is often incorporated into the model-based forecast analysis.

3

The technical details of the methodology can be found in Beneš., Laxton, and Matheson (2010).

5

More detailed data descriptions can be found in the appendix.

Same Series

Other IMF Content

Other Publishers

  • Collapse
  • Expand
Cover IMF Working Papers
Structural Models in Real Time
Author:
Kevin Clinton
,
Marianne Johnson
,
Mr. Jaromir Benes
,
Mr. Douglas Laxton
, and
Mr. Troy D Matheson
Volume/Issue:
Volume 2010: Issue 056
Publisher:
International Monetary Fund
ISBN:
9781451963625
ISSN:
1018-5941
Pages:
35
DOI:
https://doi.org/10.5089/9781451963625.001
  • Figure 1:
    View in gallery
    Figure 1:

    Real time forecasts made in the final month of the quarter and actual data, 2007Q1 to 2009Q3

  • Figure 2:
    View in gallery
    Figure 2:

    Uncertainty under estimated with hard tunes: Uncertainty around year-on-year GDP growth forecasts after each data release in 2009Q1

  • Figure 3:
    View in gallery
    Figure 3:

    Uncertainty under estimated with hard tunes: Uncertainty around year-on-year core inflation forecasts after each data release in 2009Q1

  • Figure 4:
    View in gallery
    Figure 4:

    Uncertainty under estimated with hard tunes: Uncertainty around nominal fed funds rate forecasts after each data release in 2009Q1

  • Figure 5:
    View in gallery
    Figure 5:

    Uncertainty under estimated with hard tunes: Uncertainty around unemployment forecasts after each data release in 2009Q1

  • Figure 6:
    View in gallery
    Figure 6:

    Uncertainty under estimated with hard tunes: Uncertainty around the output gap forecasts after each data release in 2009Q1

Figure 1:

Real time forecasts made in the final month of the quarter and actual data, 2007Q1 to 2009Q3
Figure 2:

Uncertainty under estimated with hard tunes: Uncertainty around year-on-year GDP growth forecasts after each data release in 2009Q1
Figure 3:

Uncertainty under estimated with hard tunes: Uncertainty around year-on-year core inflation forecasts after each data release in 2009Q1
Figure 4:

Uncertainty under estimated with hard tunes: Uncertainty around nominal fed funds rate forecasts after each data release in 2009Q1
Figure 5:

Uncertainty under estimated with hard tunes: Uncertainty around unemployment forecasts after each data release in 2009Q1
Figure 6:

Uncertainty under estimated with hard tunes: Uncertainty around the output gap forecasts after each data release in 2009Q1