A. Rationales for Central Banks to Publish Autonomous Factor Forecasts

Banks’ demand for reserves is motivated by three factors: (1) the reserve requirement; (2) autonomous factors (which changes have an exogenous impact on banks’ reserves); and (3) a possible demand for excess reserves. Central banks need to forecast these three factors to calibrate their OMOs until the end of the maintenance period and keep excess reserves low at the end of the maintenance period (Kraenzlin and Schlegel 2012). In so doing, central banks can anchor short-term rates close to the policy rate and prevent them from drifting lower if reserves are in excess of the assessed needs, or higher if the supplied reserves are insufficient to meet market liquidity needs (Hilton and Hrung 2010).

Looking at the details of banks’ refinancing needs, some items are highly predictable. The reserve requirement is set by the central bank. The target is set before the beginning of the maintenance period. Thus, there is no uncertainty over the reserves to be maintained on average by the end of the maintenance period.

The demand for excess reserves is expected to be small, stable, and, thus, predictable in well-functioning money markets. In tranquil times, banks are unlikely to keep excess reserves remunerated at a rate lower than their funding cost—so long as they are confident of their access to the market. However, the demand for excess reserves could become large, volatile, and, thus, impossible to predict if risk perception increases and the money market functioning deteriorates, causing liquidity to dry up, market segmentation to emerge, rates and volume volatility to increase, and uneven market access across counterparties to develop.

The forecast exercise concentrates on autonomous factors that show some level of volatility over maintenance periods and that the central bank can forecast with some level of accuracy. To illustrate, Box 1 describes the challenges of forecasting autonomous factors in the Eurosystem. Besides currency in circulation, holdings at central banks by non-financial institutional clients (e.g., governments) are a main driver of autonomous factors in the Eurosystem.

From the market point of view, the publication of AFF is important (Bindseil 2001). Banks wish to maintain reserves at the central bank to make daily payments and to fulfill the reserve requirement obligations at the end of the maintenance period. Therefore, bank treasurers will assess their refinancing needs over the maturity of the central bank OMOs based on reserve targets. The assessment will determine their participation in the central bank’s operations. However, they do not know the positions of all other banks, and thus cannot assess market conditions over the maturity of the upcoming refinancing operation.

Central banks usually have good information on the expected changes in their balance sheet items that correspond to autonomous factors. If a bank’s treasurer happens to be short, but knows (via the autonomous factors publication) that the rest of the market is long, he or she will likely try to find market funding to meet liquidity needs instead of turning to the central bank’s refinancing operations. The opposite is true if the forecast reveals tighter liquidity conditions in the market due to a change in the autonomous factors; the treasurer is likely to secure refinancing at the central bank operation rather than tapping the market and taking the risk of paying more than the central bank refinancing rate.

In an environment of fixed rate and full allotment, it becomes even more important that banks make informed decisions on whether to satisfy their refinancing needs either at the central bank or in the market. We assume that there are two types of counterparties: (1) banks with significant excess reserves and no refinancing needs (L1); and (2) banks with refinancing needs (L2). Under this allotment method, banks in group L2 could obtain unlimited refinancing for seven days at the central bank (as long as collateral is available) for a predetermined rate Rc, or borrow in the overnight market for seven consecutive days at rate R, which is not determined ex ante. The challenge for banks in L2 is to minimize their overall cost of refinancing by getting the right mix of central bank and market funding.

If counterparties in L2 were certain about the supply of reserves over the main refinancing operation (MRO)¹ week (S), they could request at the central bank refinancing operation the exact difference between their refinancing needs (BL2) and the autonomous factor-dictated supply of reserves (S). Consequently, counterparties in L2 can collectively bid at the refinancing operation, which will produce in sum enough excess reserves in cash-rich banks in L1 to drive short-term rates in the market to the deposit facility rate. The AFF are supposed to provide the information on the expected supply of reserves (S). However, if banks in L2 fail to collectively bid for a sufficient amount based on the information available, short-term rates could become volatile (“coordination failure”).

Market segmentation complicates bidding coordination of L2 banks at the MRO. Market segmentation is the result of restrictive credit line (CCL), due to the lack of confidence between counterparties, in particular, between banks with refinancing needs of the group L2 and cash-rich banks of the group L1. Therefore, only part of the autonomous reserve supply S will be available in the market, depending on the credit policy of cash-rich banks.

Alternatively, market segmentation encourages counterparties to hold precautionary excess reserves. Therefore, counterparties in L2 have a demand for excess reserves, which increases their refinancing need BL2 independently from developments in autonomous factors. In such cases, the demand for excess reserves becomes more difficult to predict.

Since the introduction of fixed rate and full allotment in October 2008, the Eurosystem has experienced different levels of excess reserves. There have been periods in which the existing stock of excess liquidity has not been high enough to absorb all liquidity needs of all banks with structural or contingency liquidity needs. In the context of stricter bilateral credit lines, country risk limits, and collateral concentration limits from Counterparty Clearing Houses (CCP), these episodes led to money market strains as the market found it difficult to accommodate smoothly such concentrated spikes of liquidity need. Misjudging upcoming changes in liquidity needs could therefore lead to high short-term rates (above the Eurosystem refinancing cost), such as what happened in 2009–2011 and again in 2014.

The informational advantage of central banks is even more important in monetary unions where counterparties may face challenges in collecting information from beyond their national borders. Furthermore, the ability counterparties to collect and process the available information could vary substantially, depending on their size and degree of sophistication. Moreover, forecasting challenges are heavily influenced by economy-specific factors. As the forecasting exercise is decentralized, local liquidity managers should be in a position to collect the best information available. In the Eurosystem, for example, each NCB of the euro area forecasts the autonomous factors in their balance sheet and the European Central Bank (ECB) aggregates the NCB’s forecasts to publish the Eurosystem’s autonomous factor forecast.

B. International Practices in Autonomous Factor Forecast Publication

The cross-economy analysis of the publication of autonomous factors allows a descriptive overview of international practices, providing some insights into underlying motivations. To gain these insights, we compare the publication practices of a sample of 36 other European Union (EU) and non-euro economies, both advanced (17) and emerging markets (19).

For our sample, we add nine EU non-euro countries (Bulgaria, Croatia, Czech Republic, Denmark, Hungary, Poland, Romania, Sweden, and United Kingdom) as well as five EU candidates (Albania, Montenegro, North Macedonia, Serbia, and Turkey) to the UE, which provide the European comparison against the Eurosystem framework. Furthermore, we complement the international comparison by adding the remaining 12 economies of the IMF grouping of Advanced Economies (Australia, Canada, Hong Kong SAR, Iceland, Israel, Japan, Korea, New Zealand, Norway, Singapore, Switzerland, and United States) as well as the remaining 2 OECD members (Chile and Mexico) and 8 OECD accession and partner countries (Brazil, China, Colombia, Costa Rica, India, Indonesia, Russia, and South Africa).

Of the 36 economies in our sample, 16 publish some sort of forecast, which can be either a forecast of autonomous factors or banks’ accounts at the central bank (“liquidity” forecast) based on a projection of autonomous factors (Figure 1). Most central banks publish the sum of AFF without details, but a few (Denmark and Russia) provide forecasts for individual autonomous factors. Six economies provide a public target either for the allotment of their OMOs (allotment target) or for the balance of banks’ accounts at the central bank (liquidity forecast). The remaining 14 monetary authorities do not publish a forecast, but rather some liquidity history which includes an explicit figure, just their balance sheets, or only broad monetary aggregates. Finally, the Federal Reserve publishes the results of a survey that asks banks the minimum level of reserves they would be comfortable holding, which is, to our knowledge, the only initiative to publish an estimate of counterparties’ demand for reserves.

View Full Size

International Publication Practices of Autonomous Factor Forecasts

Citation: IMF Working Papers 2019, 296; 10.5089/9781513522005.001.A001

Source: Central banks.

• Download Figure
• Download figure as PowerPoint slide

In terms of distribution across advanced and emerging economies, the publication practice is notably similar with only a slightly greater tendency in emerging economies to not publish.

In the following, we scrutinize the incidence of publication practices and distinguish six dimensions: (1) exchange rate arrangement; (2) monetary policy framework; (3) monetary policy implementation framework; (4) OMO allotment method; (5) matching the OMO tenor; and (6) publication frequency. The results are tabulated in the respective panel of Figure 2.

View Full Size

International Comparisons of Publishing Practices of Autonomous Factor Forecasts (Number of Central Banks)

Citation: IMF Working Papers 2019, 296; 10.5089/9781513522005.001.A001

Source: Central banks, Annual Report on Exchange Rate Arrangements and Exchange 2018.Note: The economies that publish a forecast, target, or history are clustered by the exchange rate regime.

• Download Figure
• Download figure as PowerPoint slide

Exchange rate arrangement. Using the IMF Annual Report on Exchange Arrangements and Exchange Restrictions (2018 AREAER), we compare the evidence of publishing autonomous factors across exchange rate regimes. Our sample covers seven exchange rate regimes, including systems ranging from currency board to free floating. We observe relatively more central banks that publish AFF with flexible exchange rate regimes than fixed exchange rate arrangements. This finding reflects the preference of these economies to use exchange rates as their operational target, rather than interest rates or liquidity conditions. Furthermore, monetary authorities that are active in foreign exchange markets often have unforeseen changes in autonomous factors, owing to changes in net foreign assets. Specifically, the changes in net foreign assets due to FX interventions in fixed exchange rate arrangements are difficult to predict beyond the settlement horizon of FX transactions, which is spot (t+2) in most cases. Only changes in liquidity shorter than the settlement date of FX interventions are known with certainty.

Monetary policy regime. Our sample covers all monetary policy regimes of the IMF AREAER: from exchange rate anchors to monetary aggregate targets, inflation targeting, and other regimes. The density of economies with both an inflation target and providing some target forecast for autonomous factors is large. This finding supports earlier evidence that economies with exchange rate targets do not publish a forecast. In addition, monetary authorities with monetary aggregate targets also do not publish forecasts, although they could provide some guidance on their operational target by publishing medium-term projections of currency in circulation and short-term forecasts of banks’ accounts at the central bank. Consequently, it is necessary to focus on the details of inflation targeting regimes.

Interest rate framework. We distinguish three main interest rate frameworks: (1) corridor systems (in which the central bank strives to provide the exact amount of refinancing that banks need over a short-term horizon); and (2) floor systems (in which the central bank aims at keeping more reserves in the system than what banks need over a short-term horizon); as well as (3) others. Most economies in our sample operate in a corridor system (22), but there is still a sizable number of them operating a floor system (10), and a few with other systems (4). The first category requires an accurate autonomous factor forecast to reach the neutral allotment (which is the allotment for which the supply of reserves equals the demand for reserves of central bank counterparties). In contrast, the second category, floor system, tolerates a larger quantity of excess reserves and relies more on short-term fine tuning operations. Our findings support this reasoning: corridor systems more often publish a forecast than floor systems (or other systems). Yet, a notable number of monetary authorities publish forecasts or targets within a floor system, implying that short-horizon forecasts and their publication remain relevant for calibrating and fine-tuning operations under a floor system, particularly as the policy signal can be more blurred (Beirne 2012).

Allotment mechanism. Allotment methods of OMOs can be broken down into: (1) variable rate tender and calibrated allotment; (2) fixed rate and full allotment; and (3) others. We find that most economies in our sample use some form of calibrated allotment for their OMOs (28). Some stand out, allotting full amounts at a fixed rate (6). For three monetary authorities, we either do not have information about their allotment mechanisms or it is not applicable for them (e.g., they are currently not conducting OMOs).

AFF are necessary to calibrate OMOs, but their publication is less necessary as the central bank keeps control over the allotment. Under the fixed rate and full allotment method, the bidding of banks determines the allotment. This is consistent with a neutral allotment, under the assumption that banks will not keep excess reserves voluntarily. Forecasts are, thus, not necessary for the central bank to calibrate the allotment of its OMOs. However, the publication of forecast is useful for better-informed bidding by the central bank’s counterparties.

Our findings suggest that targets are only published in an environment of calibrated allotments, which is the main reason for publishing any targets. For those that do not calibrate the allotment of their OMOs, the sample is divided between (1) publishing forecasts and (2) publishing history, underlining the trade-off explained above. Fixed rate full allotment is a slight majority for publishing forecast.

Publishing synchronized with OMOs, and publication frequency. The publication usually covers the maturity of the main OMOs (Figure 2, Panel 5). However, there is a notable number of economies publishing a forecast or target independently from OMOs. Publication is surprisingly frequent, with nine economies providing a forecast or target every day and one even publishing intra-daily updates (Figure 2, Panel 6). Large known transactions within the OMO period seem worth disclosing (Gray 2008) as the central bank can provide the banking sector much greater reassurance, with the major benefit of better anchoring of short-term rates. Therefore, it is a common international practice to provide additional updates outside of the OMO cycle. In contrast, the publication of liquidity history approximates the IMF Statistical Data Dissemination Standard of the monthly publication rhythm.

In terms of format, the Eurosystem is one of the few central banks in our review that publishes AFF in stock but not in flow. The publication in stocks provides two advantages. First, stocks are easier to reconcile with central bank balance sheet developments; and, second, the comparison of AFR and AFF at two points in time is easier to comprehend. In contrast, the change in perspective might better facilitate bidding behavior in OMOs, as it is related to the change in liquidity conditions. Over time, the publication format seems to be a matter of habit, convenience, and internal procedures, which is not easy to grasp and quantify in a cross-economy study.

C. History of Autonomous Factor Forecast Publication in the Eurosystem

The Eurosystem decided to publish regularly its average AFF in June 2000 in the context of its MRO announcement (normally occurring on Monday afternoon, for a tender executed on Tuesday morning and operations settled on Wednesday).² In March 2004, the Eurosystem amended the timing of the reserve maintenance period to align the start of the maintenance period with the settlement of any MRO, thereby preventing rate changes within a given maintenance period, and reducing the MRO maturity from two weeks to one. At the same time, it decided to publish a benchmark allotment amount for the MRO in addition to the autonomous factor forecast. This benchmark allotment amount is the ECB estimate of the liquidity needed to be injected to fulfill banks’ liquidity needs. The ECB is not bound by the benchmark allotment amount and could decide to inject more liquidity than the benchmark (loose allotment), or less than the benchmark (tight allotment).

The benchmark calculation includes an excess liquidity objective: null or low under neutral liquidity allotment (2000–08) or a stable (but not enforced) level of excess reserves under the fixed rate and full allotment (2008–14). As such, counterparties could infer the reserve objective of the Eurosystem based on the benchmark and the autonomous factor forecast if the two cover the same period.

The Eurosystem publishes different forecast horizons for the autonomous factor forecast (nine days) and the benchmark (seven days). The nine-day forecast is published at the time of the MRO announcement. The seven-day forecast is published after the MRO allotment. Based on the seven-day forecast, counterparties can replicate the benchmark calculation with published data. This practice still prevails under the fixed rate and full allotment, although bidding by counterparties determines the MRO allotted amount (not the published benchmark).

At the beginning of each maintenance period, the NCBs and the ECB jointly prepare AFF for all days of the full maintenance period. Consecutively, this forecast is updated for the rest of the period up to a minimum of 11 days. Then, the Eurosystem produces a forecast for the next 11 days until the beginning of the next maintenance period. These forecasts are not published.

For the published AFF, the Eurosystem amended its liquidity publication in March 2004 with a time horizon covering normally nine days. Unless Monday is non-settlement day in the euro area payment system, liquidity publication runs from Monday to the Tuesday of the next week, as the MRO is normally announced on Monday and allotted on Tuesday (Figure 3). The period includes the seven-day maturity of the MRO (Wednesday to Tuesday) and two extra days (the first Monday and Tuesday).

View Full Size

Schedule of a Standard Autonomous Factor Publication

Citation: IMF Working Papers 2019, 296; 10.5089/9781513522005.001.A001

Source: European Central Bank.

• Download Figure
• Download figure as PowerPoint slide

For the first week of the maintenance period, however, the number of days in the estimate is reduced to seven (Wednesday to Tuesday) because the data from the expired maintenance period do not provide useful information for the smooth fulfilment of the current maintenance period. Hence, they are not incorporated into the calculation of the benchmark. Thus, about 70 percent of the estimates commenced on a Monday, and 22 percent on a Wednesday. The rest reflects changes in the starting day of the estimates due to holidays. In terms of the forecast horizon, 62 percent of the estimates were provided a nine-day horizon and 20 percent a seven-day horizon (the first week of the maintenance period). The rest is generally longer estimates (12–13 days) due to longer MRO periods, such as over holidays.

In March 2004, the Eurosystem also decided to publish an update of the autonomous factor estimate on the allotment day of the MRO (that is, on the Tuesday in a standard MRO week). The update covers the same period as the Monday announcement’s estimate, but substitutes the forecast of the first Monday with the AFR on that day. Therefore, an eight-day estimate can be derived from the Tuesday update by removing the Monday realized value and calculating the average of the remaining forecast horizon. The update is published with the allotment of the MRO; therefore, this forecast cannot be used to bid at this operation. Such an update, however, helps the market to disentangle allotment decisions that deviate from the initially announced benchmark.

A. Forecast Accuracy

The ECB publishes its forecasts of autonomous factor balances on a weekly basis. A forecast is published when an MRO is announced (first forecast), and an updated forecast (second forecast) is published when the MRO allotment is announced the next day. This paper covers all the published AFF from June 26, 2000, to August 19, 2019.

The realized autonomous liquidity factors are published daily, together with data on OMOs, recourse to the lending facility, use of the deposit facility, current account holdings, and reserve requirements. This detailed information has been available since January 1, 1999. ³

The ECB has adjusted its publication schemes several times over the years, which creates a challenge for computing the AFR. From 1999 to 2009, the liquidity effect from AFR was published as one series. From 2010 to 2013, the same liquidity effect was reported, but also included reserves injected through an asset purchase program called the Securities Markets Program (SMP). However, for this period, the amount injected under SMP was not published, so it must be deducted from ECB’s tender operations history.⁴ These records contain information about the fine-tuning operations designed to absorb liquidity effects from the SMP operations during this period.

Since 2014, the time series was further modified to include the liquidity effect from all other asset purchase programs. The AFR, then, became the reported series plus the operations through all of ECB’s asset purchase programs. These include the Covered Bond Purchase Program (CBPP), CBPP2, CBPP3, SMP, Asset-Backed Securities Purchase Program, Public Sector Purchase Program, and Corporate Sector Purchase Program.

A second challenge for this computation is to remove realized data from the second forecast. The first forecast and the second forecast cover the same period, except that one day has passed between them. Therefore, the second forecast includes one day (the first day) of realized data. Based on data published by the ECB, we removed the first day realized data from the forecast to obtain eight-day forecast-only data (instead of the nine-day publication).

The resulting liquidity forecasts and AFR are plotted in Figure 4 for the first forecast and Figure 5 for the second forecast. The forecast errors are the difference between the AFF and the AFR. At first glance, the difference between realized and forecast autonomous factors did not show large or persistent discrepancies relative to the size of the autonomous factors. However, there were small and short-lasting differences reflecting forecast errors. The average error, in absolute term, is EUR 2.2 billion for the first forecast and EUR 1.7 billion for the second forecast. Twice the standard error of the first forecast error amounted to EUR 25 billion. The first forecast errors exceeded EUR 25 billion in 4 percent of the periods; the same amount was exceeded only 2.4 percent of the time for the second forecast. This points to a notable improvement of the forecasts between the first and the second.

View Full Size

Autonomous Factors: Realized Versus First Forecast

Citation: IMF Working Papers 2019, 296; 10.5089/9781513522005.001.A001

Source: European Central Bank, authors’ calculations.

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Autonomous Factors: Realized Versus Second Forecast

Citation: IMF Working Papers 2019, 296; 10.5089/9781513522005.001.A001

Source: European Central Bank, authors’ calculations.

• Download Figure
• Download figure as PowerPoint slide

In the following, the accuracy of Eurosystem liquidity forecasts, which indicates how closely the forecast traces reality, is measured as the RMSE:

where e is the forecast error defined as the difference between the forecast and the AFR. A smaller RMSE indicates lower degree of forecast error, hence higher accuracy. A 90-forecast-period moving window is used to compute the RMSE for the whole series (Figure 6) to show the changes in accuracy over time. Because the RMSE is not linear, a larger error has more impact than a smaller one, which seems reasonable as large autonomous factor forecast errors are presumed to have the largest market impact.

View Full Size

Autonomous Factor Forecast Root Mean Square Error

(Moving Window)

Citation: IMF Working Papers 2019, 296; 10.5089/9781513522005.001.A001

Source: European Central Bank, authors’ calculations.

• Download Figure
• Download figure as PowerPoint slide

The RMSE was close to one from October 2003 to December 2007, indicating a high degree of forecast accuracy during this period. It broadly corresponds to the period between the March 2004 revision of the Eurosystem’s liquidity management framework and the introduction of the fixed rate and full allotment in October 2008 (hereafter, the “new system”). In March 2004, the Eurosystem reduced the maturity of its operations from two weeks to one week and stated that its policy rate would not change during a maintenance period. Step increases in the RMSE, such as in December 2007 and December 2016, reflect disproportionally large forecast errors. Despite a drop in 2009, the RMSE remained above its best performances after 2008 as relatively larger forecast errors became more frequent.

The revised forecasts (“2^nd forecasts”) tend to have a higher accuracy than the initial forecasts (“1^st forecasts”). This reflects the improvement in the forecasts due to the incorporation of the most up-to-date information, as well as the inclusion of one day of AFR in the forecast average.

Besides the two-sided error metric, we evaluate the biasedness of the forecasts. The forecasting bias is assessed using an ordinary least squares regression of the first difference in Eurosystem forecasts to the first difference in AFR based on the equation:

Where,

• ΔAFR_t is the log-difference of the realized autonomous factor for the MRO week (t),

• ΔAFF_t is the log-difference of the autonomous factor forecast for the MRO week (t), and

• e_t is the residual, representing the estimated forecast errors. They reflect deviations from the relationship between realized and estimated values of autonomous factors, including the presence of any possible forecasting bias.

It is important to use differentiated variables for two reasons. First, from an economic point of view, what matters for the change in liquidity conditions and bidding in OMOs is the change in autonomous factors. The current level of autonomous factors is usually known to market participants, at least with the publication of the central bank balance sheet. Second, from an econometric point of view, using the level of autonomous factors might induce non-stationarity issues.

Under the assumption that forecasts are unbiased, realized autonomous factors should move proportionally to their estimated value without a drift (that is, a constant). As such, the estimated coefficients should have the following values:

• The realized value should be perfectly elastic to the estimate: β = 1. An estimated value different from one would indicate a bias conditional to the change in the forecast (that is, the bias change with the size of the autonomous factor change).

• The constant should be null: c=0. A non-null estimated drift would reflect an unconditional bias in the forecast. It would mean that the change in autonomous factors over the forecast period would be systematically over- or underestimated.

We propose testing the null hypothesis of both conditional and unconditional forecast bias together as the joint Wald test of (β = 1 and c = 0. Acknowledging the likely presence of structural break, we estimate regression (1) and infer from it the p-value of the Wald joint test for a rolling window of 90 forecasting periods (about 90 weeks) from 2000 to 2019.

The test rejects the null assumption of conditional and unconditional unbiased forecast for 37 periods of 90 days out of 953 periods (4 percent) from 2000 to 2019 for the first forecast and for 119 periods (12.5 percent) for the second forecast (Figure 7). It is generally more difficult to reject a bias in the 1^st forecast than in the 2^nd forecast, suggesting that the additional information has an impact not only on the accuracy of the forecast but also on possible bias in the forecast. Biased forecasts are noticeably more frequent after 2008 than before. The longest stretch of unbiased forecast corresponds to the period 2004 to 2008.

View Full Size

Forecast Bias Test P-value

(Rolling Basis)

Citation: IMF Working Papers 2019, 296; 10.5089/9781513522005.001.A001

Sources: European Central Bank, authors’ calculations.Note: The figure presents the p-value of the joint test. A value above 5 percent means that the null hypothesis of no-bias could not be rejected.

• Download Figure
• Download figure as PowerPoint slide

The Eurosystem forecasts tend to overestimate the increase in autonomous factors. Figure 8 shows the estimates of the coefficient β, which represent the forecast conditional bias. An estimated coefficient of less than one means that the forecaster expected a larger change in the autonomous factor than the one that actually happened. A perfect forecast would require that β equal one. The estimated coefficient is significant for all periods in the rolling regression estimates and remains below one for each period. However, evidence of structural breaks in forecast errors suggests that the forecasting bias could have changed through time. It seems to converge to one from 2004 to 2008. After 2008, the coefficient fluctuated between 0.9 and 0.5, suggesting a larger and persistent overestimation bias.

View Full Size

Conditional Bias Coefficient for 1^st and 2^nd Forecasts

Citation: IMF Working Papers 2019, 296; 10.5089/9781513522005.001.A001

Sources: European Central Bank, authors’ calculations.Note: A statistic above 1 or below -1 means that the null hypothesis that the unconditional bias (the constant) is null can be rejected.

• Download Figure
• Download figure as PowerPoint slide

The result suggests that Eurosystem forecasts are usually not subject to an unconditional bias, which means that they are not systematically off by a fixed amount. The Student statistic test for the constant c in equation 1 never exceeds a critical value of 1.5, which is low for this test.

An indication of forecast accuracy could be found in the R-square of the regression used to estimate the conditional and unconditional forecast bias. A higher R-square indicates smaller estimated forecast errors (e_t in equation 1) for a given forecasting bias. The accuracy of the forecast reached its highest level during the new system period and errors were minimal (Figure 9). During the same period, the assumption of a biased forecast was rejected, which suggests that the forecast was both unbiased and accurate. The accuracy then dropped after 2008, especially for the 1^st forecast. Thus, during this period, the forecast presents both a bias and lower accuracy.

View Full Size

R-Square of Conditional and Unconditional Forecast Bias Regression

Citation: IMF Working Papers 2019, 296; 10.5089/9781513522005.001.A001

Sources: European Central Bank, authors’ calculations.Note: A statistic above 1 or below -1 means that the null hypothesis that the unconditional bias (the constant) is null can be rejected.

• Download Figure
• Download figure as PowerPoint slide

B. Forecast Reliability

Reliability is the persistence of accurate forecasts, or a low probability of significant “surprises.” If forecast errors are, on average, low but there are some significant deviations, this could induce a premium on the demand for excess reserves as an insurance against “surprises.”

The distribution of forecast errors appears to be leptokurtic, meaning that the distribution peak is higher around the mean value and has higher densities of value at extreme ends than a normal distribution. Therefore, outlier errors are more frequent and more extreme than if the forecast errors were following a normal distribution.

The distributions are positively skewed, meaning the right tail of the distribution is long. Therefore, the likelihood of overestimating AFR (positive forecast errors) is higher that the likelihood of underestimating them (negative forecast errors). That said, the fat tail of the distribution is on the left as the largest forecast errors (“outliers”) underestimate AFR.

From July 2000 to August 2019, forecast errors are concentrated around smaller values for the second forecast than for the first forecast (Figure 10). The first forecast experienced larger (outliers) and more frequent underestimation errors than the second forecast. Besides outliers, the distributions also indicate the same probability of overestimating AFR in the first and the second forecast.

View Full Size

Distribution of First and Second Forecast Errors

Citation: IMF Working Papers 2019, 296; 10.5089/9781513522005.001.A001

Source: European Central Bank, authors’ calculations.

• Download Figure
• Download figure as PowerPoint slide

We use the period March 2004 to October 2008 (248 forecast) as a benchmark for high-forecast quality as it corresponds to the period following significant revision to the operational framework and before the major change in market conditions due to the global financial crisis. The rest of the sample include 804 forecasts prepared between July 2000 and February 2004 and between November 2008 and August 2019.

Forecast errors of the second forecast, which has the best forecasting capacity, are notably more concentrated around smaller values during the benchmark period than in the rest of the sample (Figure 11). While outlier underestimation errors are less frequent over the benchmark period, the difference between average errors and the outliers during the benchmark period makes the benchmark distribution more leptokurtic than the distribution of errors for the rest of the period. Besides outlier underestimation errors, the probability of overestimating AFR appear slightly higher during the benchmark period that the rest of the sample.

View Full Size

Distribution of Second Forecast Errors: Different Periods

Citation: IMF Working Papers 2019, 296; 10.5089/9781513522005.001.A001

Sources: European Central Bank, authors’ calculations.

• Download Figure
• Download figure as PowerPoint slide

C. Factors Influencing Forecast Errors

In this subsection, we focus on explaining forecast errors by analyzing the correlates of the forecast errors. This analysis allows us to derive the supporting factors and the impediments to high-quality forecast. Using public information as correlates can be used to test the efficiency of the forecast (cf. Frankel and Froot 1987; and Patton and Timmermann 2012). We tested this by regressing forecast errors based on information that was known when the forecast was made. The forecast is said to be “strong” in efficiency if the forecast errors are not significantly correlated with the additional information (i.e., the forecaster has used all the information available). If public information can explain only part of the forecast errors, the forecast itself is not using all information that was available at the time. If the forecast is not efficient, it is not as useful as it could be.

The analysis of the changes in accuracy and bias indicators already hint at the factors that help explain forecast errors. The improvements from the old to the new system indicate that the parameters of the forecast, such as its length and the contemporaneousness of the information used, can influence accuracy and bias. In addition, assuming that the Eurosystem models used to predict currency in circulation perform well, most of the forecast errors should come from the accounts of non-financial clients of the Eurosystem (e.g., governments). This means that market conditions, which determine how well these clients can manage their accounts in the Eurosystem, are likely an important factor in forecast quality. For instance, the financial crisis, market segmentation, and a near-zero money market interest rate environment reduced investment opportunities in the money market for NCB clients, which increased the likelihood of unexpected change in their accounts.

1^st forecast (results presented in Table 1)

Equation 3 breaks down the sources of forecasting errors:

Where,

• e_lst,t is the absolute value of the forecast errors calculated as the difference between AFR and the 1^st forecast.

• d_L stands for the length of the forecast in numbers of days. The forecast length is usually nine days, but is reduced to seven days for the first operation of the maintenance period. A longer forecasting horizon is expected to increase forecast errors.

• d_p stands for the periods identified in the history of the Eurosystem AFF. The period March 2004-October 2008, during which the forecasts seem to have been at their best, is used as a reference period.

• X_t is a vector of exogenous variables, reflecting market segmentation, such as the sovereign spread (computed as the difference between German 10-year yields and the median of other euro area sovereign yields of the same maturity), excess reserves as a percentage of reserve requirements (under the assumption that excess reserves are an indication of money market segmentation), and overnight volumes as reported by the EONIA panel (under the assumption that higher volumes reflect a less segmented money market).

• δZ_t is the EONIA, which is expected to control for the low interest rate environment.

2^nd forecast (results presented in Table 2)

Where,

• e_2nd,t is the absolute value of the forecast errors, calculated as the difference between AFR and the 2^nd forecast.

• e_lst is the forecast errors on of the 1^st forecast introduced as an explanatory variable of the error of the 2^nd forecast. This variable tests the use of the information obtained between the 1^st and 2^nd forecasts by the forecaster (akin to an error correction term). A coefficient significantly less than one but different from zero would reflect a reduction of the errors based on more contemporaneous information.

Both the 1^st and 2^nd forecast errors are significantly correlated with the set of variables presented above, suggesting that forecasts do not qualify as “strong” in efficiency. The 2^nd forecast, while resulting in smaller errors and less biased, is less efficient than the 1^st forecast, as reflected in a higher R-square for the full specification of the 2^nd forecast error compared with the 1^st forecast error (column 2 of Tale 1 and 2). This is due mainly to the contribution of the 1^st forecast error as an explanatory variable of the 2^nd forecast error. However, only part of the 1^st forecast errors is reproduced in the 2^nd forecast errors, showing that one day of additional information notably reduces forecast errors.

Using the full specification in equation 3, the 1^st forecast errors are significantly correlated with the sovereign spreads and excess reserves—both of which are indicators of market segmentation. They have the expected signs as well as the dummy variable March 2004 to October 2008, during which forecast errors were lower, as anticipated (column 2 in Table 1). The 2^nd forecast errors (equation 4 and column 2 in Table 2) are significantly correlated with excess reserves and EONIA volumes—both of which are also indicators of market segmentation, the 1^st forecast errors (as discussed earlier), and the dummy variable March 2004 to October 2008. These results corroborate the assumption that segmentation increases forecast errors.

Two differences between 1^st and 2^nd forecast errors regressions are worth noting. First, the 1^st forecast errors appear somewhat correlated with the previous period 1^st forecast errors, indicating that the information arising from previous forecast errors is not fully factored in the forecast exercise. This would suggest some degree of inefficiency in the forecasting model in the spirit of Frankel and Froot (1987). This result is not corroborated, though, for the 2^nd forecast errors. Second, the 2 ^nd forecast errors are negatively correlated with EONIA levels, reflecting the possible contribution of the low interest rate environment to forecast errors. This result is not corroborated, though, for the 1^st forecast errors.

Finally, columns 3 to 9 in Table 1 and Table 2 present the bivariate correlates of each variable and the forecast errors. They all show significant correlations with the expected signs, except for the correlation between the length of the forecast horizon and the 1^st forecast errors. Looking at the R-squares of the full specification with those of the bi-variate correlates, it appears that the main contribution arises from the market segmentation indicators (either excess reserves or EONIA volume) for the 1^st forecast errors, and from the 1^st forecast errors itself for the 2^nd forecast errors.

Table 1.

Correlation Between 1^st Forecast Errors and Other Variables

Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 Sources: European Central Bank, authors’ calculations. Note: EONIA refers to the Euro Overnight Index Average. L1 indicates the lag by one period of a variable.

Table 1.

Correlation Between 1^st Forecast Errors and Other Variables

VARIABLES	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)
Sovereign spread	1.988*** (0.463)	1.836*** (0.501)	3.160*** (0.379)
EONIA	-0.192 (0.151)	-0.180 (0.152)		-1.989*** (0.125)
Excess reserves	0.981*** (0.137)	0.987*** (0.137)			1.282*** (0.0640)
EONIA volume	-0.0143 (0.0168)	-0.00699 (0.0178)				-0.257*** (0.0140)
Length of forecast horizon	-0.00360 (0.188)	0.0262 (0.194)					0.397*** (0.139)
L1. 1st forecast error	0.118* (0.0658)	0.115* (0.0653)						0.402*** (0.0285)
Mar 2004/Oct 2008		-0.716** (0.308)							-5.282*** (0.309)
Constant	2.475 (2.023)	2.239 (2.066)	3.445*** (0.324)	8.376*** (0.289)	2.966*** (0.234)	13.04*** (0.469)	1.927 (1.209)	3.179*** (0.266)	6.524*** (0.297)
Observations	1,029	1,029	1,030	1,036	1,036	1,036	1,036	1,035	1,036
R-squared	0.333	0.334	0.063	0.197	0.280	0.246	0.008	0.161	0.084

Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 Sources: European Central Bank, authors’ calculations. Note: EONIA refers to the Euro Overnight Index Average. L1 indicates the lag by one period of a variable.

View Table

Table 1.

Correlation Between 1^st Forecast Errors and Other Variables

VARIABLES	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)
Sovereign spread	1.988*** (0.463)	1.836*** (0.501)	3.160*** (0.379)
EONIA	-0.192 (0.151)	-0.180 (0.152)		-1.989*** (0.125)
Excess reserves	0.981*** (0.137)	0.987*** (0.137)			1.282*** (0.0640)
EONIA volume	-0.0143 (0.0168)	-0.00699 (0.0178)				-0.257*** (0.0140)
Length of forecast horizon	-0.00360 (0.188)	0.0262 (0.194)					0.397*** (0.139)
L1. 1st forecast error	0.118* (0.0658)	0.115* (0.0653)						0.402*** (0.0285)
Mar 2004/Oct 2008		-0.716** (0.308)							-5.282*** (0.309)
Constant	2.475 (2.023)	2.239 (2.066)	3.445*** (0.324)	8.376*** (0.289)	2.966*** (0.234)	13.04*** (0.469)	1.927 (1.209)	3.179*** (0.266)	6.524*** (0.297)
Observations	1,029	1,029	1,030	1,036	1,036	1,036	1,036	1,035	1,036
R-squared	0.333	0.334	0.063	0.197	0.280	0.246	0.008	0.161	0.084

Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 Sources: European Central Bank, authors’ calculations. Note: EONIA refers to the Euro Overnight Index Average. L1 indicates the lag by one period of a variable.

Table 2.

Correlation between 2^nd Forecast Errors and Other Variables

Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 Sources: European Central Bank, authors’ calculations. Note: EONIA refers to the Euro Overnight Index Average. L1 indicates the lag by one period of a variable.

Table 2.

Correlation between 2^nd Forecast Errors and Other Variables

VARIABLES	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)	(10)
Sovereign spread	0.737** (0.311)	0.146 (0.380)	1.647*** (0.358)
EONIA	0.230 (0.172)	0.977*** (0.350)		-1.587*** (0.148)
Excess reserves	0.258** (0.125)	0.249** (0.124)			0.866*** (0.0605)
EONIA volume	-0.0368** (0.0162)	-0.0472*** (0.0174)				-0.188*** (0.0136)
Length of forecast horizon	-0.145 (0.203)	-0.135 (0.199)					-0.0193 (0.183)
1st forecast error	0.440*** (0.0655)	0.430*** (0.0633)						0.519*** (0.0208)
L1. 2nd forecast error	0.00894 (0.0494)	-0.00593 (0.0519)							0.297*** (0.0337)
Mar 2004/Oct 2008		-3.080*** (1.031)								-5.395*** (0.310)
Constant	2.764 (2.032)	3.706* (2.097)	3.555*** (0.345)	6.293*** (0.263)	2.680*** (0.251)	9.854*** (0.428)	4.878*** (1.612)	1.381*** (0.219)	3.316*** (0.273)	6.311*** (0.305)
Observations	797	797	799	805	805	805	805	804	804	805
R-squared	0.463	0.470	0.026	0.125	0.203	0.191	0.000	0.437	0.088	0.140

Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 Sources: European Central Bank, authors’ calculations. Note: EONIA refers to the Euro Overnight Index Average. L1 indicates the lag by one period of a variable.

View Table

Table 2.

Correlation between 2^nd Forecast Errors and Other Variables

VARIABLES	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)	(10)
Sovereign spread	0.737** (0.311)	0.146 (0.380)	1.647*** (0.358)
EONIA	0.230 (0.172)	0.977*** (0.350)		-1.587*** (0.148)
Excess reserves	0.258** (0.125)	0.249** (0.124)			0.866*** (0.0605)
EONIA volume	-0.0368** (0.0162)	-0.0472*** (0.0174)				-0.188*** (0.0136)
Length of forecast horizon	-0.145 (0.203)	-0.135 (0.199)					-0.0193 (0.183)
1st forecast error	0.440*** (0.0655)	0.430*** (0.0633)						0.519*** (0.0208)
L1. 2nd forecast error	0.00894 (0.0494)	-0.00593 (0.0519)							0.297*** (0.0337)
Mar 2004/Oct 2008		-3.080*** (1.031)								-5.395*** (0.310)
Constant	2.764 (2.032)	3.706* (2.097)	3.555*** (0.345)	6.293*** (0.263)	2.680*** (0.251)	9.854*** (0.428)	4.878*** (1.612)	1.381*** (0.219)	3.316*** (0.273)	6.311*** (0.305)
Observations	797	797	799	805	805	805	805	804	804	805
R-squared	0.463	0.470	0.026	0.125	0.203	0.191	0.000	0.437	0.088	0.140

Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 Sources: European Central Bank, authors’ calculations. Note: EONIA refers to the Euro Overnight Index Average. L1 indicates the lag by one period of a variable.