Systematizing Macroframework Forecasting: High-Dimensional Conditional Forecasting with Accounting Identities
Author:
Mr. Sakai Ando
Search for other papers by Mr. Sakai Ando in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0003-2785-4375
and
Mr. Taehoon Kim
Search for other papers by Mr. Taehoon Kim in
Current site
Google Scholar
PubMed
Close

Forecasting a macroframework, which consists of many macroeconomic variables and accounting identities, is widely conducted in the policy arena to present an economic narrative and check its consistency. Such forecasting, however, is challenging because forecasters should extend limited information to the entire macroframework in an internally consistent manner. This paper proposes a method to systematically forecast macroframework by integrating (1) conditional forecasting with machine-learning techniques and (2) forecast reconciliation of hierarchical time series. We apply our method to an advanced economy and a tourism-dependent economy using France and Seychelles and show that it can improve the WEO forecast.

Abstract

Forecasting a macroframework, which consists of many macroeconomic variables and accounting identities, is widely conducted in the policy arena to present an economic narrative and check its consistency. Such forecasting, however, is challenging because forecasters should extend limited information to the entire macroframework in an internally consistent manner. This paper proposes a method to systematically forecast macroframework by integrating (1) conditional forecasting with machine-learning techniques and (2) forecast reconciliation of hierarchical time series. We apply our method to an advanced economy and a tourism-dependent economy using France and Seychelles and show that it can improve the WEO forecast.

1. Introduction

Forecasting a macroframework, which consists of a potentially large number of macroeconomic variables and accounting identities, is a useful analytical tool to present an underlying economic narrative and check the narrative’s consistency. For example, forecasters can formalize a public-investment-driven growth story by specifying the paths of public investment and GDP. They can support the story by adjusting private investment, consumption, and imports based on assumptions about crowd-in/out, fiscal multiplier, and import leakage. With the forecasts of the current account balance, they can check whether the path of gross national saving is consistent with the story. Because of such flexibility and discipline imposed by the macroeconomic statistics, macroframework forecasting is widely used in policy institutions, including international financial institutions and individual countries.

However, macroframework forecasting is challenging since (1) forecasters often have information only on a subset of variables, which they must use to forecast other less informed variables, and (2) they need to ensure that all the accounting identities are satisfied. For example, forecasters often have prior information about GDP and fiscal variables from consensus forecasts and government budget plans. Depending on the country, forecasters might also be informed about other variables, such as commodity prices, oil production, tourists, etc. However, it is rare for forecasters to be informed about all the macro variables, so they must extend the forecast on a subset of variables to the rest, using the rule of thumb of stable GDP ratios, regressions with hand-picked variables, etc. Typically, such an extension does not satisfy all the accounting identities, so forecasters choose a variable to be residual and let it absorb all the forecast errors. When the residual looks unreasonable, forecasters iterate the process to adjust the macroframework. Such an ad-hoc approach not only incurs human error but also is resource-intensive since every time the GDP forecast changes, the impact cascades to the entire macroframework, and forecasters need to repeat the entire process.

To tackle the challenges, this paper proposes a method to systematize macroframework forecasting. As an input, forecasters provide the forecast on a subset of the variables and historical data (known variables). The method generates the forecast for the rest of the macroframework (unknown variables) in two steps. First, each unknown variable is forecasted using the known variables. Although OLS can do the task when the sample size is large, we use elastic net since macroframework typically consists of a smaller sample size in the time dimension than the number of variables. Second, the forecasts of all unknown variables are projected to the space defined by the accounting identities. We restrict the accounting identities to be affine in this paper so that the projection problem can be solved in closed form, and thus, the method can accommodate a large number of variables.

The method integrates the two strands of literature, high-dimensional conditional forecasting and forecast reconciliation of hierarchical time series. High-dimensional conditional forecasting via penalized regression has been studied by Uematsu and Tanaka (2018) in the context of forecasting quarterly US GDP. Banbura et. al. (2015) study the conditional forecasting of quarterly euro area macroeconomic and financial variables using vector autoregressions (VAR) and dynamic factor model (DFM) based on Kalman filtering methods. Our first step is a variant of the methods in the literature, but those papers do not consider the accounting identities in the macroframework, so we complement the first step with a second step reconciliation. Forecast reconciliation of hierarchical time series has been applied to quarterly Australian GDP by Athanasopoulos et. al. (2020) Capistran et. al. (2010) forecast inflation using a variant of the reconciliation method. These papers, however, do not consider conditional forecasting. Unconditional forecasting is often not convenient in practice since forecasters may want some variables to take a specific path and want the rest to be consistent with them. We extend the reconciliation method of Wickramasuriya et. al. (2019) by allowing not only unconditional forecasting but also conditional forecasting, so that the two strands of literature can be combined seamlessly.

To illustrate an example, we apply our method to French data in World Economic Outlook (WEO). We forecast 18 real sector variables in the GDP expenditure approach using the forecast of 35 variables (GDP, fiscal variables, and current account balance) and the historical data of all 53 variables. France is chosen since (i) data quality, including the available sample size and the number of variables, is high, and (ii) government is so large in the economy that fiscal variables can be expected to be informative about real sector variables. We evaluate the performance of the proposed method against the WEO forecast using the five April vintages from 2016 to 2020. The result shows that the method generates smaller root mean squared forecast errors than WEO forecasts by 20 percent on average. We provide another example of Seychelles in Annex 2 to illustrate an application to a tourism-dependent economy.

Our results suggest that the proposed method can systematize macroframework forecasting, but caveats need to be noted. The method relieves forecasters from agonizing over less-informed variables and allows them to focus on supplying known variables and checking the resulting macroframework forecasts. There are, however, two caveats to be noted. First, the method forecasts macroframework using historical correlation. Thus, the Lucas critique (Lucas, 1976) applies, i.e., a new policy that moves variables in a way that has never been observed can result in large forecast errors. If forecasters know the impact of such policy, they should embed the knowledge in the known variables instead of relying on historical correlation. Second, the method uses the forecasts of the known variables supplied by forecasters as inputs. Thus, if the inputs are not accurate, the forecast of the unknown variables will be inaccurate. If forecasters have weak priors on some variables, they should include them in the unknown variables to avoid spilling over to the rest of the macroframework. That being said, as in the literature of conditional forecasting, forecasters can supply the path of known variables that is not likely to happen but is policy-relevant for scenario analysis.

2. General Framework

This section explains notation and describes the method in two steps. Since the method is more general than the macroframework setting, we keep the terminology and notation general. We also omit the description of data pre-processing in this section for simplicity and provide an example of data transformations in the section of country example.

We denote the set of variables in the data by r and the constraints by (C, d). The data consist of m > 0 variables. Among them, k of them are called known variables, denoted by rtk, and have T + h samples, where h is the forecast horizon. The rest mk variables are called unknown variables, denoted by rtu, and have T samples.

rtu:(mk)×1,rtk:k×1,rt=[rturtk]:m×1.(1)

For ease of notation, we will denote the set of the unknown and known variables by u and k. The data do not move freely and satisfy

Ctrt=dt,(2)

where Ct and dt are known (m – n) x m matrix and (m-n)xl vector. We assume that the submatrix of Ct that corresponds to the unknown variables has full rank so that redundant constraints, such as the constraints that do not include unknown variables or those that differ only in the part corresponding to known variables, have already been dropped. Thus, the number of free variables is n < m. We focus on a non-trivial case k < n, since otherwise, the forecast of known variables and constraints can pin down the forecast of unknown variables.

Table 1 shows the relationship between known and unknown variables. Note that the known variables can be a large object since they can include the lags of the variables. The objective of the method is to fill the shaded NaN cells with the forecast r˜tu,t=T+1,...,T+h such that the constraints are satisfied

Ct[r˜turtk]=dt,t=T+1,...,T+h.(3)
Table 1.

Data Structure

article image

2.1 Step 1: Forecasting Each Unknown Variable

The first step of our method is to forecast unknown variables without imposing the constraints. There can be many ways to do this step. One candidate is OLS or VAR if the sample size is sufficiently larger than the number of predictors. In macroframework, however, the sample size in the time dimension is often smaller than the number of predictors, especially in annual data. Forecasters can use quarterly data to mitigate the problem, but they face trade-offs since many macroeconomic variables are only available in annual frequency.

To accommodate the situations where the sample size is smaller than the number of predictors, we use the elastic net with time series cross-validation. 1 Since the elastic net is an extension of OLS, the coefficients can be compared with the values in the literature for a sanity check, although they cannot be interpreted as causality. The elastic net is also an extension of the Lasso and ridge regression. As the Lasso, the elastic net learns a sparse model so that many coefficients are estimated to be zero. The elastic net inherits the stability of ridge regression when the predictors are highly correlated. Time series cross-validation is adopted to mitigate overfitting and maintain the time series structure in estimation.

More specifically, the algorithm minimizes the cross-validated mean squared error subject to hx and L2 penalties. Since the penalization is not scale-invariant, the data are standardized.

r¯it=ritμiσi,μi=1TΣτ=1Trit,σi=1T1Στ=1T(ritμi)2,i=1,...,m,t=1,...,T.(4)

This step applies to all the variables that are not constant over time. For constant variables, the constant values themselves become the forecast. For each i ∈ u the parameters solve

(β^(i,T),λ^1(i,T),λ^2(i,T))=argminβ=(β1,...,βk),λ1,λ2TSCV(1TΣt=1T(r¯itu+β1r¯itk+...+βkr¯ktk)2+λ1Σi=1k|βi|+λ2Σi=1kβi2),(5)

where (i, T indicates that the parameter is for i ∈ u and estimated using data up to T, and TSCV means time series cross-validation in which successive training sets are supersets of those that come before them, as visualized in Table 2.

Table 2.

Time Series Split

article image

We set the number of folds in the cross-validation to be five, which is the default parameter value of the scikit-learn package in Python. Thus, the last five observations become the test set by turns. The objective is to find the set of parameters (β,λ12) that minimizes the mean of the errors across the folds. The estimated coefficients are used to construct the first step forecast for the standardized variables

r¯itu=β^1(i,T)r¯1tk+...+β^k(i,T)r¯ktk,t=T+1,...,T+h.(6)

The first step forecast r¯itu can be obtained by transforming the standardized forecast r¯itu back to the original scale.

r¯itu=μi+σir¯itu,iu,t=T+1,...,T+h.(7)

It is possible that the elastic net chooses β^k(i,T)=0 for all k, in which case the forecast is the historical mean r^itu=μi. The forecast of the known variable remains the same r^tk=rtk. Note that the first step forecast r^t does not necessarily satisfy the constraints (3).

2.2 Step 2: Forecast Reconciliation

The second step ensures that the constraints (3) are satisfied by projecting the first step forecast r^tu on the space defined by the constraints. Fix t = T + 1, ...T + h. Given the first step forecast r^t, the second step forecast r^t solves

r^t=argminr^=[r^ur^k]12(r^tr¯)W^1(r^tr˜)s.t.{Ctr˜=dtr˜k=rtk,(8)

where the weight matrix Ŵ is an estimate of the one-step-ahead first step forecast error W=VT(r^T+1rT+1*), and rT+1* is a random variable representing the true value that is unobservable at time T. The weight implies that the smaller the forecast error of the first step forecast r^itu is, the closer the second step forecast r^itu is to the first step forecast r^itu. Intuitively, the variables with small forecast errors are used to forecast the rest of the variables through the constraints. Geometrically, the solution r^t is a point in the space, defined by the constraints {r˜m:Ctr˜=dt,r˜k=rtk}, that minimizes the distance to the first step forecast r^t.2

This formulation is similar to the method proposed by Wickramasuriya et. al. (2019) but differs in two aspects. First, Wickramasuriya et. al. (2019) show that the optimal weight that minimizes the trace of the forecast error covariance matrix Σi=1mVT(ritrit*) is forecast-horizon-dependent Wt=VT(r^trt),t=T+1,...,T+h3. They note, however, that it is challenging to estimate more than one-step-ahead forecast error in practice t > T + 1 and propose a simplifying assumption Wt = ktWT+1 where kt is a scalar that depends on the horizon t = T + 1,... ,T + h. Since the constant kt does not change the solution of (8), we use an estimate of WT+1 as the weight. Second, the constraints fix the known variables r˜k=rtk. This additional constraint is the main difference from Wickramasuriya et. al. (2019) and enables the forecast reconciliation technique to be naturally extended to conditional forecast.

We simplify the problem (8) by substituting out the known variables. Let Ut and Kt denote the transpose of (m — k) x m and kxm submatrices of the constraint matrix Ct that correspond to the unknown and known variables, i.e.,, Ct=[Ut'Kt']. Let Ŵ denote the (m – k) x (m – k) submatrix of the weight matrix Ŵ that corresponds to the unknown variables. By substituting r^k=rtk and noting r^tk=rtk, the problem reduces to

r˜tu=argminr˜m12(r^tur˜u)W1(r^tur˜u)s.t.Ut'r˜u+Kt'rtk=dt.(9)

The weight matrix Ŵ is constructed by shrinking the sample covariance of forecast errors. Let Tin denote the number of forecast errors. Applying the first step to the subsamples leads to

r^itu=μi+σi{β^1(i,t1)r¯1tk+...+β^k(i,t1)r¯ktk},iu,t=TTin+1,...,T.(10)

Note that β^(i,t1) means the coefficient for iu estimated using the subsample up to t - 1. Thus, the steps 1 and 2 run (Tin + 1) x u regressions in total, and all the forecast errors are from out-sample forecasts. The sample covariance matrix of the forecast errors is

V^=1Tin1(e^e˜)(e^e˜),e^=[r^TTin+1urTTin+1ur^TurTu],e˜=1Tin1Tin×1Σt=TTin+1T(r^turtu).(11)

The sample covariance matrix V^ can be full rank when Tin is large. In macroframework, however, the sample size in the time dimension is often smaller than the number of predictors, especially in annual data. Thus, as in Wickramasuriya et. al. (2019), we retain full rank by shrinking the sample covariance matrix V^

W^=λdiag(V^)+(1λ)diag(V^),λ=ΣijV^(ρij)Σijρij2(12)

where diag(V^) is a matrix where diagonal elements coincide with V^ and off-diagonal elements are 0, and λ is the shrinkage parameter developed by Shafer and Strimmer (2005) with ρ and V^(ρ) defined by4

z={diag(V^)}12(e^e˜),ρ=1Tin1zz,V^(ρ)=Tin(zz)(zz)(Tin1)3ρρTin(Tin1).(13)

The symbol ⊙ denotes Hadamard product.

Given the weight matrix Ŵ, we can solve the second step forecast r˜t in closed form as in Theorem 1. The closed-form solution allows a large number of variables to be reconciled in the second step and makes the two steps amenable to a high-dimensional environment.

Theorem 1. Suppose Ut and Ŵ are full rank. The second step forecast r˜tu defined by (9) can be written as

r˜itu=r˜ituW^Ut(Ut'W^Ut)1(Ctr^tdt).(14)

Proof. See Annex 1.

3. Country Example

In this section, we discuss an application of our general framework. To illustrate the application in a simple example, we forecast one-year ahead GDP expenditure approach subcomponents conditional on fiscal variables and a few headline variables such as GDP and external balance. The forecasts of these conditional variables tend to be more available to forecasters than the unknown variables thanks to government budget plans, consensus forecasts, and other sources. Thus, the example illustrates how an agnostic forecaster, who does not have much country-specific knowledge, can apply the general framework.

In practice, however, forecasters should use their country-specific knowledge to determine the known variables and supply their forecasts. For example, the forecast of tourists might be informative for tourism-dependent economies, the forecast of commodity prices and production plans might be informative for commodity-trading economies, and the construction schedule of infrastructure might be informative if the economic impact is expected to be large. Annex 2 provides an example of a tourism-dependent economy, estimating contributions to Seychelles’ GDP conditional on services exports.

3.1 Data

We use French data retrieved from World Economic Outlook (WEO). The IMF publishes the WEO data which contain historical data and forecasts up to five years ahead. Individual country team in the IMF produces each country’s data based on available information, including official statistics, consultation with the country authorities and private-sector experts, and the team’s own analysis. The data are then checked for internal consistency by a dedicated team, including accounting identities. We choose French data because (i) a rich set of variables is available, and (ii) the size of the government is the largest among the economies with rich sets of variables. Thus, fiscal variables can be expected to be informative about the behavior of the economy.

Table 3 summarizes the list of variables that we use for this exercise. The known variables before including lags consist of 2 types of GDP, 4 GDP subcomponents that are related to government, 28 fiscal variables, and 1 external current account balance. The variables starting with N denote national accounts, G denote government finance statistics, and B denote balance of payments. We include all variables that start from G, reflecting our agnostic approach. The unknown variables are 18 subcomponents of the GDP expenditure approach that are not directly related to the government. In total, 53 variables are used in the analysis. The national accounts variables in WEO are restricted to be GDP expenditure approach, but forecasters can include the variables from production and income approaches, which can be useful when the path of a specific industry or worker-specific subsidies play an important role as in the COVID-19 pandemic.

Table 3.

List of WEO Variables

article image

Note that the list contains not only the most disaggregated variables but also their aggregates. As Wickramasuriya et. al. (2019) note, aggregated data tend to have a higher signal-to-noise ratio than highly disaggregated data in general. It is reasonable to believe that this is also the case with macroframework. For example, those who report the statistics may misclassify consumption into investment, which could result in volatile GDP subcomponents and stable aggregates. Thus, we include variables from various levels.

To assess the performance, we use seven years of WEO vintages from 2016 April to 2022 April. We choose these vintages since the same set of constraints can be imposed on the same set of unknown variables. The number of known variables increases over time. The sample starts from 1980 in all the vintages. Each vintage contains historical data available as of April. As of 2022, French quarterly national accounts are published with a two-month lag for the first estimate and a three-month lag for the detailed figures. The performance of a forecast in a year is assessed using the same year’s value in the next year’s vintage.

3.2 Accounting Identities

We consider 11 constraints. Since the constraints are time-invariant, we drop the time subscript. The GDP expenditure approach equation gives seven constraints.

NGDP=NCG+NCPNC+NIG+NIPNINTDD+NXG+NXSNXNMG+NMPNSNFB+NSDGDP.(15)

We do not include the real version of the equation since chain-linked volumes are not additive. Investment can also be divided into gross fixed capital formation and changes in inventories. The former can be written as the sum of private and public subcomponents.

NI=NFI+NINW,NFI=NFIG+NFIP.(16)

Gross national savings can be written as the sum of investment and current account balance or the sum of gross national private and public saving.

NGS=NI+bca,NGS=NGSG+NGSP.(17)

3.3 Variables to Forecast

Instead of forecasting the unknown variables in level, we forecast their shares in GDP. It is typical to forecast normalized series in practice for cross-country comparison. In this example, we forecast the unknown variables’ shares of GDP. One can forecast the levels directly, although a rule of thumb is to transform data into stationary series. As Hyndman and Athanasopoulos (2021) note, spurious regression generally will not continue to work into the future even though it might appear to give reasonable short-term forecasts. In general, the performance of a forecasting method could change depending on the transformation.

3.4 Step 1: Forecasting Each Unknown Variable

In step 1, most variables are normalized by GDP and differenced. Nominal and real GDPs are transformed into growth rates. All other variables are normalized by nominal GDP and then subtracted by its one-year lag to remove trends. We do not use growth rates for all variables since some variables take zero or negative values, including the current account balance. An alternative transformation is the contribution to GDP as in Annex 2.

The number of folds in the cross-validation uses the default value, s = 5. Thus, the elastic net selects the model that minimizes the forecast error for the last five years. We include the lags of all variables to capture delayed responses, such as the impact of government expenditure on investment. The number of lags is chosen to be two, following the format of the selected economic indicators table in France Article IV consultation staff report. (IMF, 2021) Thus, the number of variables used in the elastic net is 159 = 53 x 3, of which the number of unknown variables is 18 and that of known variables is 141. The number of forecasts uses the same number as the cross-validation Tin = s = 5, so the elastic net is applied 108 = 6 x 18 times in total.

The first step forecast r^ is obtained by un-differencing the output from the elastic net. We apply the elastic net with time-series cross-validation to the differenced series and obtain the first step forecast of the unknown variables’ shares of GDP r^ by reverting the differencing. The differencing can be reversed by adding the previous year’s observation. The un-differencing simplifies the constraints in the second step by making them time-invariant, but one can instead keep the differencing and impose time-variant constraints.

3.5 Step 2: Forecast Reconciliation

The second step forecast r^ is obtained by projecting the first step forecast of the unknown variables’ shares of GDP r^ on the space defined by the constraints. Note that the constraints for the shares of GDP are affine and time-invariant. For example, the GDP expenditure approach equation becomes

NTDDNGDP+NFBNGDP+NSDGDPNGDP=1.(18)

The second step forecast r^ reflects the information of known variables rk and satisfy all the accounting identities.

3.6 Performance Assessment

The forecast performance is assessed using the root mean squared error (RMSE). Let r˜tu be the second step forecast of unknown variables for year t. Forecast error is the difference from the true value rt*, which is defined as the year t observation of year t + 1 vintage. For example, the true value of a variable for 2016 is the 2016 observation in 2017 vintage. Although it is possible that the data are further revised in later vintages, we use the one-year ahead vintage so that the forecast and true value maintain the same time lag for all years. Each unknown variable’s forecast error is aggregated over vintages using RMSE, and each forecasting method is summarized as the mean RMSE over the variables.

RMSEi(r˜)=16Σν20162021(r˜ivurivu*)2,iu;RMSE(r˜)=1|u|Σi=1|u|RMSEi(r˜),(19)

where |u| is the number of unknown variables.

For comparison, we calculate the RMSE for three other methods. First, the main benchmark is WEO forecast rWE0. Second, we use the first step forecast f to assess the improvement due to reconciliation in the second step. Third, we use true values r*, i.e., the same years’ observation in one-year ahead vintage, to construct the in-sample forecasts. Although the third method is an in-sample forecast and uses the information that is not available to forecasters, it is expected to give the lower bound of forecast error. The performance of each method is measured by replacing r˜withrWEO,r^,andr˜* in (20).

Figure 1shows that the second step forecast r˜ improves WEO forecast by around 20 percent. The mean RMSE of WEO forecast rWEO is around 0.0154, the first step forecast r^ is around 0.0145, the second step r˜ is 0.0123, and the second step with true values r˜* is 0.0121. Essentially, the second step forecast r˜ improves WEO forecast rWEO by around 20 percent on average. The result suggests that the method not only reduces burden for forecasters but also can result in performance gain.

Figure 1.
Figure 1.

Mean RMSE of Four Forecasting Methods

Citation: IMF Working Papers 2022, 110; 10.5089/9798400211683.001.A001

The chart shows mean root mean squared error for four forecasting methods: WEO forecast, 1st step forecast, 2nd step forecast, and 2nd step forecast conditional on true values. The 2nd step forecast improves WEO forecast by around 20 percent on average.

The difference in performance, however, is not uniform. Figure 2 shows the RMSE for each unknown variable’s share of GDP. The second step forecast r˜ tends to improve WEO forecast rWEO for domestic variables, such as consumption and investment, more than external variables, such as imports of services. Since the current account balance is the only known variable related to external sector in this exercise, the result suggests that it might be useful to include more external sector variables, such as commodity prices, other countries GDP growth, etc. Annex 3 shows the forecast error of each unknown variable over time. One can see that the second step forecast error and WEO forecast error behave similarly for most variables and years, although the performance of several variables diverge in 2020.

Figure 2.
Figure 2.

RMSE of Unknown Variables

Citation: IMF Working Papers 2022, 110; 10.5089/9798400211683.001.A001

The chart shows the root mean squared error of each unknown variable. The 2nd step forecast improves the WEO forecast for most unknown variables.

4. Discussion

The country example in section 3 suggests that the method can be useful for many other situations related to macroframework forecasting. The country example described GDP expenditure approach subcomponents, but accounting identities constrain GDP production and income approaches as well as all other macroeconomic statistics (price, fiscal, external, monetary, etc). Thus, the method can be scaled up to the macroframework of an entire economy. The country example also uses only one economy, but all economies are constrained by trade and financial linkages. For example, the sum of all economies’ current accounts equals zero, and the sum of all countries’ financial liabilities equals the sum of financial assets minus monetary golds, up to statistical discrepancies. Thus, the method can be scaled up to the macroframework of the entire world if ample data are available.

However, there are some caveats. Since the first step forecast uses historical correlation, the Lucas critique (Lucas, 1976) applies, i.e., a new policy that moves variables in a way that has never been observed can result in large forecast errors. Also, note that the coefficients in the first step should not be interpreted as causal effects. If forecasters know the impact of such policies, they should embed the knowledge in the known variables instead of relying on historical correlation. This also applies to non-policy shocks that move variables in an unprecedented manner. When there is a structural break in historical correlation, that year’s forecast error is likely to be large due to overfitting. From the next year, the time series cross-validation in the first step avoids overfitting, and the large forecast error is reflected in the weight matrix, so the value of historical data is discounted.

The method could perform poorly when the forecasts of known variables are inaccurate. Biased forecasts on the known variables will bias those on the unknown variables especially when the known variables have been informative in the historical data. If forecasters have weak priors on some variables, they should include them in the unknown variables to avoid spilling over to the rest of the macroframework. Forecasters, however, may want to specify the known variables’ forecasts that are not likely to happen but are policy-relevant for scenario analysis. Forecasters can also reflect the uncertainty on the known variables by specifying the distribution of known variables, randomly drawing a sample from it, and applying the method to obtain the forecasts of the unknown variables.

5. Conclusion

We have proposed a method to forecast macroframework in a systematic manner. We proposed the method in a general framework and applied it to the context of forecasting macroeconomic variables of France using WEO data. The result suggests that combining high-dimensional conditional forecasting and forecast reconciliation of hierarchical time series can accommodate a large number of variables, systematize the macroframework forecasting, and improve the WEO forecast.

The analysis motivates future research in various directions. Extending the general framework to other situations, such as multilateral consistency, is an interesting application. Applying the framework to mixed-frequency data could improve macroframework forecasting by incorporating the information from higher frequency data and insights from nowcasting literature. Another direction is to extend the general framework by allowing log-linear constraints since some macroeconomic accounting identities take the shape of multiplication. One may also apply the framework conversely to search important variables. We leave these topics for future research.

Annex 1. Proof of Theorem 1

Set the Lagrangian

L=12(r^tur˜u)W^1(r^tur˜u)+μ(Ut'r˜u+Kt'rtkdt).

Taking the derivative with respect to r˜u leads to

W^1(r^tur˜u)Utμ=0r˜u=r^t+1uW^Utμ.

Multiplying both sides by Ut and using the constraint give

Utr˜u=Ut'r^tuUt'W^Utμ=dtKt'rtk.

Since Ut and Ŵ are full rank, Ut'W^Ut is invertible. Thus, the Lagrange multiplier is

μ=(Ut'W^Ut)1(Ut'r^tu+Kt'rtkdt)=(Ut'W^Ut)1(Ctr^tdt).

Substituting the Lagrange multiplier back to the first-order condition gives

r˜u=r^tuW^Ut(Ut'W^Ut)1(Ctr^tdt).

Annex 2. Another Country Example: Seychelles

This section shows an alternative country example using Seychelles, a tourism-dependent economy with a rich set of variables. The example shows that the 2nd step forecast is comparable with the WEO forecast for national accounts variables related to domestic activities and improves the WEO forecast for those related to external activities.

Seychelles’ example uses the same WEO vintages as France and a smaller number of variables. The list of variables used in the analysis is provided in Table 4. We replace fiscal variables with exports of services which include inbound tourism. The private and public breakdown of gross national savings is dropped since they are not available in the WEO database. We also drop real GDP so that normalization by nominal GDP gives a natural interpretation.

Table 4.

List of WEO Variables

article image

The object to forecast is contributions to GDP. Specifically, let yit denote ith variable. The contribution of ith variable to nominal GDP is defined as

rit=yityit1NGDPt1.(20)

We apply this transformation to all variables including nominal GDP itself, in which case contribution coincides with growth rate, and known variables that are not constrained by the accounting identities.

The accounting identities for the contributions remain linear. For example, the GDP expenditure approach equation, analogous to (18) can be expressed as

NTDDtNTDDt1NGDPt1+NFBtNFBt1NGDPt1+NSDGDPtNSDGDPt1NGDPt1NGDPtNGDPt1NGDPt1=0.(21)

The result suggests that the 2nd step forecast r˜ improves the WEO forecast mainly through the variables related to external activities. The left chart of Figure 3 shows the mean RMSE. The order of estimators is similar to Figure 1, but the gain from conditioning true values is larger. The right chart shows that the 2nd step forecast r˜ improves the WEO forecast mainly through the variables related to external activities. Imports and exports, as well as their breakdown into goods and services, are improved. In contrast, the WEO forecast performs better for the variables related to domestic activities such as consumption and investment.

Figure 3.
Figure 3.

Mean RMSE of Four Forecasting Methods (Left) and RMSE of Each Unknown Variable (Right)

Citation: IMF Working Papers 2022, 110; 10.5089/9798400211683.001.A001

The left chart shows mean root mean squared error for four forecasting methods: WEO forecast, 1st step forecast, 2nd step forecast, and 2nd step forecast conditional on true values. The 2nd step forecast improves WEO forecast by around 15 percent on average. The right chart shows root mean squared error of each unknown variable. The 2nd step forecast improves WEO forecast for the variables related to external activities.

Annex 3. Forecast Error of Unknown Variables

uA001fig01

One-year ahead forecast error of unknown variable over year 1/3

Citation: IMF Working Papers 2022, 110; 10.5089/9798400211683.001.A001

uA001fig02

One-year ahead forecast error of unknown variable over year 2/3

Citation: IMF Working Papers 2022, 110; 10.5089/9798400211683.001.A001

uA001fig03

One-year ahead forecast error of unknown variable over year 3/3

Citation: IMF Working Papers 2022, 110; 10.5089/9798400211683.001.A001

References

  • Athanasopoulos, George, Puwasala Gamakumara, Anastasios Panagiotelis, Rob Hyndman, Mohamed Affan (2020), “Hierarchical forecasting,” Macroeconomic forecasting in the era of big data.

    • Search Google Scholar
    • Export Citation
  • Banbura, Marta, Domenico Giannone, Michele Lenza (2015), “Conditional forecasts and scenario analysis with vector autoregressions for large cross-sections,” International Journal of Forecasting.

    • Search Google Scholar
    • Export Citation
  • Capistran, Carlos, Manuel Ramos-Francia (2009), “Does Inflation Targeting Affect the Dispersion of Inflation Expectations?Journal of Money, Credit and Banking.

    • Search Google Scholar
    • Export Citation
  • Gagliardini, Patrick, Elisa Ossola, Olivier Scaillet (2020), “Estimation of large dimensional conditional factor models in finance,” Handbook of Econometrics.

    • Search Google Scholar
    • Export Citation
  • Hyndman, Rob, George Athanasopoulos (2021), “Forecasting: principles and practice, 3rd edition,” OTexts.

  • International Monetary Fund (2021), “France: 2021 Article IV Consultation-Press Release; Staff Report and Statement by the Executive Director for France.” IMF Country Report.

    • Search Google Scholar
    • Export Citation
  • Lucas, Robert (1976), “Econometric policy evaluation: A critique,” Carnegie-Rochester Conference Series on Public Policy.

  • Schafer, Juliane, Korbinian Strimmer (2005), “A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics,” Statistical Applications in Genetics and Molecular Biology.

    • Search Google Scholar
    • Export Citation
  • Taieb, Souhaib Ben, Bonsoo Koo (2019), “Regularized Regression for Hierarchical Forecasting Without Unbiasedness Conditions,” in the 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining.

    • Search Google Scholar
    • Export Citation
  • Uematsu, Yoshimasa, Shinya Tanaka (2018), “High-dimensional macroeconomic forecasting and variable selection via penalized regression,” Econometrics Journal.

    • Search Google Scholar
    • Export Citation
  • Wichramasuriya, Shanika, George Athanasopoulos, Rob Hyndman (2019), “Optimal Forecast Reconciliation for Hierarchical and Grouped Time Series Through Trace Minimization,” Journal of the American Statistical Association.

    • Search Google Scholar
    • Export Citation
  • Wichramasuriya, Shanika, Berwin Turlach, Rob Hyndman (2020), “Optimal non-negative forecast reconciliation,” Statistics and Computing.

    • Search Google Scholar
    • Export Citation
*

We thank all participants of IMF seminars.

1

The algorithm is available in the scikit-learn package of python. An alternative method is dimension reduction such as conditional factor models as surveyed in Gagliardini et. al. (2020). We leave the horse race for future research.

2

The formulation also assumes that there are no inequality constraints. Wickramasuriya et. al. (2020) propose an optimal non-negative forecast reconciliation, although it may incur some bias.

3

The formulation assumes that the 1st step forecast is unbiased. Taieb and Koo (2019) propose a method without the assumption, although it is more computationally complex.

4
Each element of the matrix V^(ρ) can be written as
V^(ρ)=Tin(Tin1)3{Σt=TTin+1T(zitzjt)2(1TinΣTTin+1Tzitzjt)2}.
  • Collapse
  • Expand
Systematizing Macroframework Forecasting: High-Dimensional Conditional Forecasting with Accounting Identities
Author:
Mr. Sakai Ando
and
Mr. Taehoon Kim