Introduction

The Covid crisis highlighted the need for a systematic analysis of high-frequency indicators of economic activity. While in normal times the dynamics of real economic activity can generally be sufficiently characterized and explained by conventional month lyand quarterly aggregate indicators—e.g.. industrial production or gross domestic product—during crisis times, businesses, market analysts, and policymakers need higher-frequency information about economic activity. The importance of timely information was vividly illustrated in 2020, when the COVID pandemic hit the global economy with unprecedented intensity and unpredictability. In Europe for instance, the economy recorded in the second quarter its largest contraction since World War II. However, the first official GDP growth estimates of the second quarter for European countries were published only in mid-July, too late to assess the magnitude of the shock and calibrate the appropriate policy response. Fortunately, other economic indicators had been available much earlier. A question arose as to whether a variety of higher frequency data from different sources could be systematically analyzed to provide sufficiently accurate information on economic conditions timely enough to usefully inform policymaking.

Over the last decade, at least in advanced countries, data availability has become less of a constraint thanks to new technologies, more efficient at collecting and compiling data. As a result, new information, mainly from non-official sources—such as market surveys, sentiment indicators, Google search results, air quality, etc.—have become available. These indicators—frequently available and possibly capable of quickly registering sharp changes in economic conditions—can supplement standard economic indicators produced and validated by national statistical offices, to help gauge the economic situation at the aggregate level.

Furthermore, methods that can deal with large datasets containing highly correlated variables of different frequencies and information quality, have also been developed. For more than 20 years, models based on dynamic factor methodology have been prominent nowcasting tools to quantitatively assess the state of the economy in the current period or over the very near future. More recently, and not surprisingly, machine learning methods have gained popularity among economists and have been deployed also in nowcasting.

This paper describes an effort to strengthen nowcasting capacity at the IMF’s European department in early 2020. It adds to the growing literature on nowcasting in several ways. First, it motivates and compiles datasets of standard variables and new ones, such as Google search and air quality, including country-specific databases of selected European economies and their transformations. Second, beyond applying standard dynamic factor model (DFM), we employ several machine learning (ML) algorithms to nowcast GDP growth. Third, the paper com pares the performance of alternative methods across an heterogenous group of European economies, during normal and crisis times.

The paper is structured as follows. Part 2 provides a non-exhaustive literature overview focusing mostly on DFMs and machine learning methods. Part3 discusses data that could inform about an economy’s position and their statistical treatment. Part4 provides a non-technical summary of DFMs and selected machine learning approaches. Part 5 presents our main results and compares them across alternative techniques, including during normal and crisis episodes. Part6 concludes.

Literature Review

Economists in academia and policy institutions have sought to predicting quarterly GDP and turning points of the economic cycle over the near term using higher frequency indicators for a long time. Nowcasting models, as we know them today, rely on the idea originally formulated by Sargent and Sims (1977) that the salient features of business cycle fluctuations can be captured by a small number of factors that could be estimated from data. Stock and Watson (1989) were among the first to construct indices of business cycle using factor models.

The early models had the state-space representation and were estimated by maximum-likelihood and relied on only a limited number of variables. The latent common factors were estimated by Kalm an Filter. The first model by Stock and Watson (1989) extracted a single common factor from a small set of monthly core indicators (employment, industrial production,sales, and income).

Overtime, these models were expanded and improved to deal with a much larger number of variables and factors used to construct different business cycle indicators. Giannone et al. (2005) and Watson (2005) showed that the findings by Sargent and Sims (1997) also apply to high-dimensional datasets and proposed to formulate factor models using state-space specification to reduce the dimensionality of large economic systems, i.e. to extract unknown common factors. While the large number of variables doesn’t impose restriction on feasibility of estimation (Doz et al (2012)), it increases the estimation time; hence the need for selectivity when it comes to picking what variables go into a model.

Since then, factor models have been further refined and become widely used tools to monitor economic conditions. The model of Giannone, Reichlin, and Small (2008) was first implemented to nowcast GDP at the Board of Governors of the Federal Reserve based on a project that started in 2003.Various versions of this model have been built for different economies and implemented in other central banks, including the European Central Bank (ECB, 2008) and the International Monetary Fund (Matheson (2011), Taheri Sanjani (2013)). The models have been continuously refined. In a series of papers, D’Agostino et al. (2011) and Giannone et al. (2013) further developed the DFM methodology and improved prediction properties of this group of models by further improving the Kalman-Filter estimation methodlogy and optimizing the data selection process.

DFMs have been widely used during the COVID-19 pandemic. Sampi and Jooste (2020) introduced Google mobility indicators into the DFM model to capture the economic effects of the pandemic on industrial production in Latin America and the Carribean while Lewis et al. (2020a, 2020b) produced a Weekly Economic Index (WEI) to measure real activity. Other work includes Guer in (2020), Chapman and Desai (2021), Antolin-Diaz et al. (2021), or Corona et al. (2021).

While DFMs have became mainstream tools to nowcast GDP growth, new techniques have emerged, based on statistical learning and machine learning. These techniques not only introduce alternative ways to estimate coefficients but also perform data driven variables and model selection that may lead to smaller deviations between actual and predicted outcomes, i.e. better out-of-sample prediction. About thirty years ago, Fagan and Fell (1991) did not include non-linear methods and neural networks in their overview of techniques for short-term economic forecasting, since they were still in their infancy as far as economics was concerned and not widely used by practitioners. Since then, a lot has changed and, while this is a relatively new area, the number of articles related to machine learning using big data in economics has been growing fast due to two factors.

• First, new data have became available. For example, Google trends data have become a new source of information. Choi and Varian (2009a) showed that using specific keywords, Google trends data may help predict the present, especially when economic conditions change rapidly. For exam pie, Varian and Choi (2009b), Tuhkuri (2016) used Google search data to predict current unemployment.

• Second, a large number of algorithms have been developed and improved. Hastie et al. (2009, and 2015) or Gareth et al. (2015) are classic textbooks discussing the main tools. However, the examples they provide are mostly outside of economics.

As a result, machine learning techniques have become popular among economists. Varian (2014), Belloni et al. (2014), and Storm (2020) provide an overview of some of the tools for manipulating and analyzing big data in economics. Amat et.al. (2018) apply machine learning methods to predict floating exchange rates of twelve industrial countries and conclude that these methods perform better than tools typically used in the literature. Similarly, using a large scanner panel dataset with highly correlated variables, Bajari et al (2015) apply alternative machine learning methods to estimate consumer demand and find that the machine learning methods predict out-of-sample demand much more accurately than standard panel data and logistic models. Chakraborty and Joseph (2017) discuss machine learning in the context of central banking and policy analyses. Beyond discussing main machine learning concepts and models, they also present three case studies using machine learning at the central bank. Kohlscheen (2021) examines drivers of inflation relying on regression trees and random forests. Basuchoudhary et.al. (2014) motivate and apply different machine learning algorithms (Artificial Neural Network, Boosting, Bootstrap Aggregating, Random Forest predictors and Regression Tree)to predict long-term economic growth and the likelihood of recessions. They use the machine learning algorithms to assess the significance of various variables—previously identified by research—in predicting economic growth. Athey (2019) com pares machine learning methods to traditional econometrics, discusses main advantages of machine learning, and provides examples of using these methods for prediction and inference in policy analysis. She also predicts that machine learning will have a large impact on how empirical work is conducted.

Although machine learning methods have been primarily developed for prediction, they have started being applied also for nowcasting. Richardson et. al. (2020) apply machine learning algoriths to a dataset of about 600 domestic and international variables to predict New Zealand GDP growth. The study shows that machine learning algorithms—mainly boosted trees, support vector machine, and neural networks—are able to outperform a simple autoregressive model and DFM. Applying several machine learning algorithms on a dataset of quarterly macroeconomic and financial data to nowcast Indonesia’s GDP growth, Muchishaet. al. (2021) arrive at a similar conclusion that all machine learning models outperform AR(1) bench mark while Random Forest showed the best performance. Without adding data from more granular and novel data sources, Jung et. al. (2018) still find that machine learning methods—Elastic Net, Super Learner, and Recurring Neural Network algorithms—can outperform traditional statistical methods. Tiffin (2016) applies machine learning algorithms in case of Lebanon—a country with long delays in the publication of its GDP. Using a dataset of 19 variables, he shows how Elastic Net and Random Forest can be used to nowcast Lebanese GDP growth before official data is released. Bolhuis and Rayner (2020) apply alternative machine learning algorithms to nowcast the Turkish GDP growth. In addition,to further lower forecast errors,the authors combine individual machine learning models into ensembles. They find that the machine learning models can reduce forecast errors by at Ieast30percentwhen compared to traditional models. In response to rapidly changing environmend during the COVID pandemic, Woloszko (2020) constructed the OECD Weekly Tracker of GDP growth by including Google trends data highlighting the predictive power of specific keywords, including “bankruptcies”, “economic crisis”, “investment”, “luggage” and “mortgage”. On a sample of OECD and G20 countries, they find that on average the model’s forecast error is 17 percent lower than that of an AR(1) model. The model also captures a sizeable share of business cycle variation and detects well the downturn and subsequent rebound of the economy.

In addition to DFM and machine learning, nowcasting models can be based on other approaches, including for example bridge equations, mixed data sampling (MIDAS) or mixed-frequency VARs. Bridge equations gained in popularity mainly due to its simplicity and low technical requirements. The method relies on single equation regressions of quarterly GDP growth on a small number of high-frequency variables (such as industrial production or surveys) aggregated to quarterly data by averaging, summing up, or taking another transformation (Baffigi et al., 2004). To form a forecast of the variable of interest, high-frequency (explanatory) variables are forecast using separate time series models with a potentially large numbers lags, resulting in a large number of parameters to be estimated. Thus, while parsimonious, the main disadvantage of bridge equations models is a potentially large number of parameters that needs to be estimated. MIDAS approach addresses this issue, e.g., the large number of parameters that needs to be estimated, by replacing the unrestricted lag polynomial structure by non-linear functional forms. While MIDAS approach achieves the model’s parsimony, its parameters need to be estimated by non-linear estimation methods, which may complicate empirical work. Mixed frequency VARs are an alternative to single-equation models. These models operate at the highest frequency of data, at which, however, not all variables are necessarily available. Lower frequency variables are assumed to be periodically missing and represented through a state space model.

Over the last decade, these and other new approaches have been introduced to near-term forecasting and nowcasting literature, all of them having their pros and cons (Camacho and Perez-Quiros (2013)). The remainder of this paper will focus on nowcasting models based on DFMs and machine learning algorithms.

Data

The basic idea of nowcasting is to exploit a diverse set of timely information that is available before an official release of a target variable.¹ As such, data selection and transformation is key to the success of nowcasting.

New technologies and approaches to data collection contributed to wider data availability, allowing economists to rely on large datasets. The so-called high-dimensional datasets or big data often include a number of explanatory variables that is close to, or even exceeds, the number of observations. The motivation behind those large datasets has been to maximize the information set and thus reduce the risk of bias due to the omission of information.

Unfortunately, the prediction capacity of a fitted model does not always improve when introducing additional explanatory variables (Hastie et al (2015)). Specifically, introducing additional explanatory variables into a model increases the risk of over fitting the model—as potentially noisy variables are assigned non-zero coefficients. In addition, even if explanatory variables are relevant to the model, large datasets often consist of interconnected clusters of information, within which variables are highly correlated or present similar concepts.² In general, excessive collinearity among explanatory variables results in the large variance of the estimated coefficients, which may overweight the benefits of reduced omitted variable bias.

To address issues arising from large datasets—which have become limiting especially in case of classical statistical techniques such as linear regression—and to fully exploit their potential, new techniques and algorithms have been developed. Recently developed techniques and algorithms have been instrumental in dealing with large datasets (see Models and Methodology section). Some of the models penalize the introduction of additional variables and some of them have the capacity to fully exclude variables from the sample and thus shrink the size of the original dataset. While this is an important feature that makes the practitioner’s task easier, carefully selecting timely and informative variables has remained critical to nowcasting in the big data environment.

Following standard practices in nowcasting of GDP growth, our sample contains a wide range of official and nonofficial, hard and soft, and proxy data. The data used are published with different frequencies and cover main aspects of the economy and the external environment.³The dataset does not include detailed sectoral data and uses only headline figures. For example, industrial production is included only once on the aggregate level, although disaggregated data are available at the same time. This practice is in line with the literature showing that disaggregated information neither improves, nor harms the quality of the nowcast.⁴

The dataset also excludes variables available only since 2020. Proxy of economic activity available at frequency close to real time (e.g., Google mobility indicators, dining reservations, hotel occupancy rates, etc.) have gained in popularity over the last decade, and the COVID pandemic has accelerated their use in short-term forecasting. However, while available with high frequency, the time series of most of these indicators are often very short and thus does not allow to estimate, train, and test the performance of alternative models on historical data.⁵ As such, Google trends, Oxford Stringency Indices and air quality data are the only high (weekly) frequency variables included in our sample.

The total number of variables ranges between 20 to 60 depending on the characteristics of the economy for which our set of models is applied, and data availability. Data can be organized in the following clusters:

• i. Firm and production date: industrial output, housing and construction indicators such as manufacturing production and industry turnovers.

• ii. Surveys or forward-looking indicators:business surveys, economic sentiment survey, and consumer confidence.

• iii. Labor market variables.unemployment, wages, and employment data.

• iv. Financial variable, stock prices, various interest rates, stock of credit and deposits

• v. Prices: producer price index, CPI, house prices, commodity price index

• vi. Foreign trade.exports and imports volumes and values.

• vii. External environmentopenness to trade, reliance on Global Value Chain, and main trading partners’ economic variables.

• viii. Other variables capturing specific features of an economy: reliance on tourism ⁶, commodity export, vulnerabilities to climate change, presence of domiciled multinational companies in a country among others.

• ix. Additional indicators: air quality index, hotel occupancy rates, Oxford Stringency Indices, frequency of google searches for specific words that may proxy current position in the business cycle.^{7 8}

Before applying the nowcasting models, the variables need to be transformed:

• First, all variables with frequency higher than quarterly (mostly weekly and monthly) must be transformed into quarterly date. The DFM takes advantage of the mixed-frequency data by using bridge equation and Kalman filter. while in the case of the ML models, weekly and monthly data are transformed to quarterly frequency, so all the variables in the final dataset are on quarterly basis.⁹

• Second, as in the case of standard econometric tools, time series must be seasonally adjusted and detrended. In general, transforming quarterly variables in levels into year-over-year growth rates addresses both seasonality and non-stationarity of the time series in the dataset. A smaller number of variables, such as unemployment rate—already expressed as percent—or outcomes from surveys are either kept unchanged in levels or transformed to year-over-year differences.¹⁰

Models and Methodology

Once the data have been collected and transformed, we need methods to(i) extract information contained in a large number of variables and (ii) address issues arising from large datasets, e.g. collinearity and risk of overfitting. The methods also need to be able to deal with incomplete data and data of different frequencies. The goal is to be able to process new information in real time in a systematic and objective manner.

The remainder of this section discusses main ideas behind techniques applied in this work: (i) DFMs, and(ii) the tools relying on machine learning.¹¹

Machine Learning

ML algorithms have become widely used in analyzing large datasets for several reasons:¹

• - Flexibility: ML algorithms are flexible and effective in capturing patterns in the data. As such, ML techniques often outperform traditional techniques at learning from large datasets and predicting complex and nonlinear relationships.

• - Variable selection.Most of the ML algorithms limit over fitting by actively and directly seeking the most suitable set of indicators to predict target variables.

• - Cross-validation: ML algorithms fine-tune the model parameters by iteratively partitioning the sample into training and testing subsamples and gauging the pseudo out-of-sample performance with test subsample to minimize the forecast error.

On the other hand, the research on the interpretability of ML algorithms has been scarce so far (Carvalho et al 2019) and ML algorithms are often seen as black boxes with a clear trade-off between model performance and interpretability (Linardatos et al 2020). In addition, some ML algorithms seem less robust to missing observations or incomplete dataset than DFM.²

^{>Table 1.}

A Brief Introduction to ML Algorithms

^{>Table 1.}

A Brief Introduction to ML Algorithms

Methods	Description
LASSO, Ridge, Elastic Net	LASSO (least absolute shrinkage and selection operator), Ridge regression and Elastic net are modified linear regression methods introducing different types of regularizations (a penalty imposed on the use of coefficients) to enhance the prediction accuracy. Compared to traditional regression methods, these methods can avoid dimensionality and overfitting, but still face the challenge of linearity.
Support Vector Machine (SVM)	SVM constructs hyperplanes to partition combinations of explanatory variables and makes a point forecast for each of the sections, like kernel regression with regularization. SVM can overcome the drawbacks of linear regression models including linearity, collinearity, overfitting, and high dimension issue. The performance, however, depends on proper selection of the kernel function and regularization parameters. Complicated kernel function or parameters on the other way may limit SVM’s interpretability.
Random Forest (RF)	RF is a combination of forecasts from many individual regression trees. As a non-parametric algorithm, RF can also overcome the main drawbacks of linear regression models, including linearity, collinearity, overfitting, and issues related to high dimensionality of datasets. Nonetheless, while flexible, RF has a limited capability of predicting extreme or outlier events. Besides, given the complex structure of RF, it lacks interpretability.
Neural Network (NN)	Atypical NN is a multi-layer non-linear method which maps a series of inputs into a target output. A network consists of layers composed of artificial neurons (or nodes). Each neuron (i) receives inputs from neurons in previous layers, (ii) applies a function to produce a single output, and (iii) sends the output to other neurons in the next layer. Eventually, a final set of nodes is mapped into the target output. Being very flexible, NN allow for alternative functional forms in each neuron as well as for different structure of individual layers. As a sophisticated and flexible algorithm, NN has been proven to bean immensely powerful tool for prediction, addressing drawbacks of the traditional regression methods. Such sophistication and flexibility, however, significantly limit the interpretability of predictions from NN.

View Table

^{>Table 1.}

A Brief Introduction to ML Algorithms

Methods	Description
LASSO, Ridge, Elastic Net	LASSO (least absolute shrinkage and selection operator), Ridge regression and Elastic net are modified linear regression methods introducing different types of regularizations (a penalty imposed on the use of coefficients) to enhance the prediction accuracy. Compared to traditional regression methods, these methods can avoid dimensionality and overfitting, but still face the challenge of linearity.
Support Vector Machine (SVM)	SVM constructs hyperplanes to partition combinations of explanatory variables and makes a point forecast for each of the sections, like kernel regression with regularization. SVM can overcome the drawbacks of linear regression models including linearity, collinearity, overfitting, and high dimension issue. The performance, however, depends on proper selection of the kernel function and regularization parameters. Complicated kernel function or parameters on the other way may limit SVM’s interpretability.
Random Forest (RF)	RF is a combination of forecasts from many individual regression trees. As a non-parametric algorithm, RF can also overcome the main drawbacks of linear regression models, including linearity, collinearity, overfitting, and issues related to high dimensionality of datasets. Nonetheless, while flexible, RF has a limited capability of predicting extreme or outlier events. Besides, given the complex structure of RF, it lacks interpretability.
Neural Network (NN)	Atypical NN is a multi-layer non-linear method which maps a series of inputs into a target output. A network consists of layers composed of artificial neurons (or nodes). Each neuron (i) receives inputs from neurons in previous layers, (ii) applies a function to produce a single output, and (iii) sends the output to other neurons in the next layer. Eventually, a final set of nodes is mapped into the target output. Being very flexible, NN allow for alternative functional forms in each neuron as well as for different structure of individual layers. As a sophisticated and flexible algorithm, NN has been proven to bean immensely powerful tool for prediction, addressing drawbacks of the traditional regression methods. Such sophistication and flexibility, however, significantly limit the interpretability of predictions from NN.

Results

We applied the DFM and ML models to a set of six European countries and examined whether some of the models tend to outperform our benchmark model—AR(1)—and the other models.⁴. Tables 2 to 4 present RMSE of individual nowcasting models and the AR(1)benchmark model.⁵The tables provide the results for the six above mentioned countries during three sample periods. Table 2 covers the full sample of available data up to 2021Q1 while Table 3 and 4 provide results for the pre-COVID norma/period (data up to 2019Q4) and the COVID pandemic period (2020Q1 to 2021Q1), respectively. Chart 1 illustrates the variety of fit quality we obtained across countries, models, and time periods. The findings from this exercise are specific to our sample of countries and should not be generalized.

The key findings from the results include the following:

• Both our DFM and ML models add value, especially during times of hightened volatility. Except for Ireland and Malta, the DFM and ML models outperform AR(1)in a large majority of cases when applyied to the whole sample (Table 2). The AR(1) performs quite well during a period of relatively stable growth and frequently outperforms the other nowcasting models (Table 3). For example, the AR(1) benchmark model performed better than several ML methods during normal times in case of Austria, Hungary, Malta, and Poland. However, and not surprisingly. the strength of the DFM and ML models improved markedly during the COVID-19 pandemic (Table 4). In particular, except for Ireland and Malta, the Ridge regression and Lasso models could significantly reduce the average forecast errors ranging from 40 to 75 percent relative to the AR(1) benchmark during the pandemic. For DFM, the forecast errors are about 50 percent lower than the AR(1) bench mark across all countries.

• The performance of individual models greatly differs across sub-samples (time periods). For example, Ridge Regression and DFM perform well for Poland and Hungary, respectively, during normal times but did not do well during COVID-19 period. Chart 1—middle panel “Not So Good”—provides selected country exam pies of methods with differing performance during pre-and during COVID periods. Tables 3 and 4 show a full set of the RMSEs for pre-and during COVID periods. As expected,the RMSEs during normal times (pre-COVID) are lower than that during the COVID period, where all countries experienced highly volatile economic growth. Focusing on the period prior to the pandemic, Table 3 shows that the DFM performs relatively well—and often better than many ML model—in case of many countries and is among the best performers for Austria, Hungary, and Portugal. Nonetheless, the DFM model performed worse than the AR(1) benchmark for Malta and Poland. For ML methods, Gaussian Process Regression (GPReg) generates the lowest RSMEs for Ireland and Portugal during normal times, while Linear Regression (LinReg) performs the best for Poland. The results are quite different when considering nowcasting performances during the COVID period (Table 4). Particularly, several ML methods appeared to perform better at capturing turning points—notably during 2020Q2–2020Q3—when comparing to both the AR(1) benchmark and the DFM. This finding is in line with several papers, including Richardson, Mulder, and Vehbi (2018); and Soybilgen and Yazgan (2021).

• The performances of individual DFM and ML models can also greatly differ across countries. Tables 2–4 and Panel Chart 1 demonstrate that there is no one-size-fit-all model. In other words, the best model that captures GDP dynamics are different across countries and across time. For exam pie, on the one hand, the DFM generates the lowest RMSE for Malta, while Ridge and LASSO perform the best for Austria, Hungary, and Poland. On the other hand, LASSO does not track well actual GDP growth in case of Malta and Ireland.

Examples of Model Performances

Citation: IMF Working Papers 2022, 052; 10.5089/9798400204425.001.A001

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Examples of Model Performances

Citation: IMF Working Papers 2022, 052; 10.5089/9798400204425.001.A001

• Download Figure
• Download figure as PowerPoint slide

Examples of Model Performances

Citation: IMF Working Papers 2022, 052; 10.5089/9798400204425.001.A001

• Download Figure
• Download figure as PowerPoint slide

^{Table 2.}

Full Sample-Model Performance (RMSE) Until 2021Q1

Source: IMF staff calculation. Note: the green highlighted cells indicate models with lower RMSE against the AR1 benchmark, while the bold red values indicate a model with the lowest RMSE for each country.

^{Table 2.}

Full Sample-Model Performance (RMSE) Until 2021Q1

Country	Austria	Hungary	Ireland	Malta	Poland	Portugal
Backtest period	2017Q1–2021Q1	2016Q1–2021Q1	2015Q2–2021Q1	2012Q-2021Q1	2015Q2–2021Q1	2015Q3–2021Q1
Benchmark- RMSE
AR1	3 7	3.8	7.7	3.571	2.11	4.0
Machine Learning Models-RMSE
EHel_SEm	1.29	1.22	20.34	3.67	1.64	1.18
EHet_SE1	1.27	1.27	20.34	3.62	1.61	1.18
Lasso_SEm	1.33	1.21	20.13	3.61	1.71	1.20
Lasso_SE1	1.30	1.30	20.11	3.59	1.67	1.17
Ridge_SEm	1.26	1.20	20.44	3.74	1.45	1.25
Riclge_SE1	1.23	1.24	20.44	3.63	1.45	1.25
RFor	3.03	3.16	7.66	4.35	1.80	3.02
SVMachine	1.65	1.39	15.87	4.14	1.60	1.62
LinReg	1 38	1.56	14.44	5.57	2.20	1.53
GPReg	2.76	2.71	6.62	4.19	1.53	3.62
NN	1.82	2.07	8.09	3.94	1.98	3.05
NNdeep	2.40	2.02	7.57	3.32	2.25	2.56
NNrstm	3.01	2.57	8.39	5.06	2.22	2.69
NNUIstm	2.78	2.13	7.59	4.55	2.45	2.87
NNconv	2.11	2.22	10.46	4.50	1.96	1.71
Dynamic Factor Model-R MSE
DFM	1.80	2.11	7.13	2.77	1.71	2.60

Source: IMF staff calculation. Note: the green highlighted cells indicate models with lower RMSE against the AR1 benchmark, while the bold red values indicate a model with the lowest RMSE for each country.

View Table

^{Table 2.}

Full Sample-Model Performance (RMSE) Until 2021Q1

Country	Austria	Hungary	Ireland	Malta	Poland	Portugal
Backtest period	2017Q1–2021Q1	2016Q1–2021Q1	2015Q2–2021Q1	2012Q-2021Q1	2015Q2–2021Q1	2015Q3–2021Q1
Benchmark- RMSE
AR1	3 7	3.8	7.7	3.571	2.11	4.0
Machine Learning Models-RMSE
EHel_SEm	1.29	1.22	20.34	3.67	1.64	1.18
EHet_SE1	1.27	1.27	20.34	3.62	1.61	1.18
Lasso_SEm	1.33	1.21	20.13	3.61	1.71	1.20
Lasso_SE1	1.30	1.30	20.11	3.59	1.67	1.17
Ridge_SEm	1.26	1.20	20.44	3.74	1.45	1.25
Riclge_SE1	1.23	1.24	20.44	3.63	1.45	1.25
RFor	3.03	3.16	7.66	4.35	1.80	3.02
SVMachine	1.65	1.39	15.87	4.14	1.60	1.62
LinReg	1 38	1.56	14.44	5.57	2.20	1.53
GPReg	2.76	2.71	6.62	4.19	1.53	3.62
NN	1.82	2.07	8.09	3.94	1.98	3.05
NNdeep	2.40	2.02	7.57	3.32	2.25	2.56
NNrstm	3.01	2.57	8.39	5.06	2.22	2.69
NNUIstm	2.78	2.13	7.59	4.55	2.45	2.87
NNconv	2.11	2.22	10.46	4.50	1.96	1.71
Dynamic Factor Model-R MSE
DFM	1.80	2.11	7.13	2.77	1.71	2.60

Source: IMF staff calculation. Note: the green highlighted cells indicate models with lower RMSE against the AR1 benchmark, while the bold red values indicate a model with the lowest RMSE for each country.

^{Table 3.}

Pre-COVID Samples- Model Performance (RMSE) Until 2019Q4

Source: IMF staff calculation. Note: the green highlighted cells indicate models with lower RMSE against the AR1 benchmark, while the bold red values indicate a model with the lowest RMSE for each country.

^{Table 3.}

Pre-COVID Samples- Model Performance (RMSE) Until 2019Q4

Country	Austria	Hungary	Ireland	Mata	Poland	Portugal
Backtest period	2017Q1–2019Q4	2016Q1–2019Q4	2015Q2–2019Q4	2012Q-2019Q4	2015Q2–2019Q4	2015Q3–2019Q4
Benchmark- RMSE
AR1	0.57	0.9	7.6	2.11	0.7	0.5
Machine Learning Models-RMSE
EINet_SEm	0.57	1.13	6.28	3.15	0.70	0.42
EINet_SE1	0.61	1.14	6.28	3.14	0.70	0.43
Lasso_SEm	0.60	1.12	6.53	3.07	0.66	0.41
Lasso_SE1	0.60	1.19	6.57	3.05	0.66	0.41
Ridge_SEm	0.58	1.11	6.24	3.14	0.71	0.44
Ridge_SE1	0.60	1.11	6.24	3.14	0.71	0.44
RFor	0.69	0.83	8.46	4.13	1.01	0.93
SVMachine	0.65	1.30	6.19	1.03	0.77	0.43
LhReg	0.60	0.36	7.80	3.52	0.66	0.41
GPReg	0.75	0.71	4.79	2.70	0.67	0.35
NN	0.90	0.94	6.33	3.40	0.81	0.76
NNdeep	1.02	1.26	6.19	2.68	0.83	0.66
NNIstm	0.54	0.78	8.72	4.09	1.23	0.47
NNMstm	1.01	0.77	7.26	3.02	1.15	0.51
NNconv	0.90	1.34	6.68	2.69	0.92	0.45
Dynamic Factor Model-RMSE
DFM	0.47	0.60	7.43	2.36	0.92	0.46

Source: IMF staff calculation. Note: the green highlighted cells indicate models with lower RMSE against the AR1 benchmark, while the bold red values indicate a model with the lowest RMSE for each country.

View Table

^{Table 3.}

Pre-COVID Samples- Model Performance (RMSE) Until 2019Q4

Country	Austria	Hungary	Ireland	Mata	Poland	Portugal
Backtest period	2017Q1–2019Q4	2016Q1–2019Q4	2015Q2–2019Q4	2012Q-2019Q4	2015Q2–2019Q4	2015Q3–2019Q4
Benchmark- RMSE
AR1	0.57	0.9	7.6	2.11	0.7	0.5
Machine Learning Models-RMSE
EINet_SEm	0.57	1.13	6.28	3.15	0.70	0.42
EINet_SE1	0.61	1.14	6.28	3.14	0.70	0.43
Lasso_SEm	0.60	1.12	6.53	3.07	0.66	0.41
Lasso_SE1	0.60	1.19	6.57	3.05	0.66	0.41
Ridge_SEm	0.58	1.11	6.24	3.14	0.71	0.44
Ridge_SE1	0.60	1.11	6.24	3.14	0.71	0.44
RFor	0.69	0.83	8.46	4.13	1.01	0.93
SVMachine	0.65	1.30	6.19	1.03	0.77	0.43
LhReg	0.60	0.36	7.80	3.52	0.66	0.41
GPReg	0.75	0.71	4.79	2.70	0.67	0.35
NN	0.90	0.94	6.33	3.40	0.81	0.76
NNdeep	1.02	1.26	6.19	2.68	0.83	0.66
NNIstm	0.54	0.78	8.72	4.09	1.23	0.47
NNMstm	1.01	0.77	7.26	3.02	1.15	0.51
NNconv	0.90	1.34	6.68	2.69	0.92	0.45
Dynamic Factor Model-RMSE
DFM	0.47	0.60	7.43	2.36	0.92	0.46

Source: IMF staff calculation. Note: the green highlighted cells indicate models with lower RMSE against the AR1 benchmark, while the bold red values indicate a model with the lowest RMSE for each country.

^{Table 4.}

During COVID-19 Sample- Model Performance (RMSE) Between 2020Q1–2021Q1

Source: IMF staff calculation. Note: the green highlighted cells indicate models with lower RMSE against the AR1 benchmark, while the bold red values indicate a model with the lowest RMSE for each country.

^{Table 4.}

During COVID-19 Sample- Model Performance (RMSE) Between 2020Q1–2021Q1

Country	Austria	Hungary	Ireland	Malta	Poland	Portugal
Backtest period	2020Q1 -2021Q1	2020Q1–2021Q1	2020Q1–2021Q1	2020Q1–2021Q1	2020Q1–2021Q1	2020Q1–2021Q1
Benchmark- RMSE
AR1	6.86	7.6	8.2	8.0	5.1	8.4
Machine Learning Models-RMSE
EINet_EEm	2.21	1.49	42.84	7.81	2.85	2.40
EINet_SE1	2.14	1.60	42.84	7.65	2.85	2.40
Lasso_SEm	2.2B	1.47	42.23	7.22	2.96	2.47
Lasso_SE1	2.20	1.60	42.16	7.62	3.06	2.40
Ridae_SEm	2.15	1.42	43.09	7.84	2.84	2.55
Ridge_SE1	2.06	1.56	43.09	7.80	2.80	2.55
RFor	5.49	6.30	4.97	9.98	5.54	6.23
SVMacnine	2.88	1.65	32.61	7.38	2.75	3.39
LinReg	2.38	2.08	27.74	12.10	4.32	3.18
GPReg	4.95	5.41	11.10	11.13	5.26	7.74
NN	3.05	3.89	12.74	12.73	5.02	6.38
NNdeep	4.12	3.47	11.40	8.38	6.15	5.34
NNtitm	5.48	5.08	6.97	9.73	5.30	5.70
NNtjIstm	4.87	4.15	8.71	10.13	5.97	6.07
NNcoiw	3.61	3.86	18.85	7.54	5 46	3.57
Dynamic Factor Model-RMSE
DFM	3.39	4.20	5.87	4.85	4.02	5.94

Source: IMF staff calculation. Note: the green highlighted cells indicate models with lower RMSE against the AR1 benchmark, while the bold red values indicate a model with the lowest RMSE for each country.

View Table

^{Table 4.}

During COVID-19 Sample- Model Performance (RMSE) Between 2020Q1–2021Q1

Country	Austria	Hungary	Ireland	Malta	Poland	Portugal
Backtest period	2020Q1 -2021Q1	2020Q1–2021Q1	2020Q1–2021Q1	2020Q1–2021Q1	2020Q1–2021Q1	2020Q1–2021Q1
Benchmark- RMSE
AR1	6.86	7.6	8.2	8.0	5.1	8.4
Machine Learning Models-RMSE
EINet_EEm	2.21	1.49	42.84	7.81	2.85	2.40
EINet_SE1	2.14	1.60	42.84	7.65	2.85	2.40
Lasso_SEm	2.2B	1.47	42.23	7.22	2.96	2.47
Lasso_SE1	2.20	1.60	42.16	7.62	3.06	2.40
Ridae_SEm	2.15	1.42	43.09	7.84	2.84	2.55
Ridge_SE1	2.06	1.56	43.09	7.80	2.80	2.55
RFor	5.49	6.30	4.97	9.98	5.54	6.23
SVMacnine	2.88	1.65	32.61	7.38	2.75	3.39
LinReg	2.38	2.08	27.74	12.10	4.32	3.18
GPReg	4.95	5.41	11.10	11.13	5.26	7.74
NN	3.05	3.89	12.74	12.73	5.02	6.38
NNdeep	4.12	3.47	11.40	8.38	6.15	5.34
NNtitm	5.48	5.08	6.97	9.73	5.30	5.70
NNtjIstm	4.87	4.15	8.71	10.13	5.97	6.07
NNcoiw	3.61	3.86	18.85	7.54	5 46	3.57
Dynamic Factor Model-RMSE
DFM	3.39	4.20	5.87	4.85	4.02	5.94

Source: IMF staff calculation. Note: the green highlighted cells indicate models with lower RMSE against the AR1 benchmark, while the bold red values indicate a model with the lowest RMSE for each country.

To operationalize our findings, we developed an integrated, fully-automated nowcasting tool. The tool automatically (i) collects and treats the dataset; (ii) applies a suit of DFM and ML models to the dataset to perform backtest (charts and indicators of forecasting accuracy); (iii) re-estimate the model each time new data becomes available and produces a nowcast of the current quarter GDP growth; and, (iv) generates an aggregated output for all the methods across the whole period and the subsamples. Once variables to form the dataset have been selected, the tool can be easily applied to any country, subject to data availability (Annex [2]).

Summary and Conclusion

The COVID-19 pandemic has underscored the need for timely data and methods, which allow for the assessment of economies’current health. We motivate, compile, and discuss datasets for six European economies and apply and compare traditional and new nowcasting methods for the period 1995Q1 -2021Q4 as well as during normal times (1995–2019) and during the COVID-19 pandemic (2020Q1 up to 2021 q4). The datasets comprise standard variables suggested by the literature as well as new variables that have become recently available such as Google Trends and air quality. We provide lists of variables, and their transformations, for six European countries that can serve as exam pies for other economies. To avoid issues linked to large datasets (collinearity between explanatory variables and model overfitting), we apply methods capable of reducing the dimension of the original dataset. As a representative of a standard approach, we apply a DFM following the New York Fed tradition. Building on the growing evidence of the usefulness of machine learning methods in economics, we introduce several ML algorithms to our exercise.

Overall, we find that the tools we applied add value and have the power to inform the nowcast of the current quarter GDP growth. Applying alternative methods and comparing their predictive capability on historical data, we find that a strong majority of our methods outperform the AR(1) benchmark model. Specifically, applying the ML methods reduced the average forecast errors up to 75 percent and the DFM reduced the fore cast error by half compared to the AR(1) model across all countries. Nonetheless, we found that there is no one-size-fits-all method that would outperform the remaining methods in case of all countries under all circumstances. Although our results apply to a specific set of countries and time episodes—and cannot necessarily be generalized—we find that our DFM models tend to perform better during normal times but not as much during times of heightened volatility. In other words, DFMs did not predict the sharp contraction of2020Q2tothe extent as the ML methods we used did. On the contrary, most of the ML methods performed strongly at identifying turning points, while it seems to overvalue the effect of minor changes in some of the explanatory variables on GDP growth during normal times. Therefore, when using the techniques we applied, the practitioner needs to carefully analyze back-testing results and apply judgment as to what models can be trusted to provide the most informative nowcast.