Online learning and forecast combination in unbalanced panels

ABSTRACT

This article evaluates the performance of a few newly proposed online forecast combination algorithms and compares them with some of the existing ones including the simple average and that of Bates and Granger (1969). We derive asymptotic results for the new algorithms that justify certain established approaches to forecast combination including trimming, clustering, weighting, and shrinkage. We also show that when implemented on unbalanced panels, different combination algorithms implicitly impute missing data differently, so that the performance of the resulting combined forecasts are not comparable. After explicitly imputing the missing observations in the U.S. Survey of Professional Forecasters (SPF) over 1968 IV-2013 I, we find that the equally weighted average continues to be hard to beat, but the new algorithms can potentially deliver superior performance at shorter horizons, especially during periods of volatility clustering and structural breaks.

KEYWORDS:

• Machine learning
• Recursive algorithms
• SPF forecasts
• missing data

JEL CLASSIFICATION:

• C22
• C53
• C14

Acknowledgement

An earlier version of the article was presented at the New York Camp Econometrics VI (Lake Placid, April 2011) and the 17th International Panel Data Conference (Montreal, July 2011). We thank Cheng Hsiao, Tom Wansbeek, three anonymous referees and the handling editor for many helpful comments.

Notes

See Capistrán and Timmermann (2009), Issler and Lima (2009), and Smith and Wallis (2009).

Aiolfi and Timmermann (2006) document such crossings in the context of model-based forecasts.

See Rapach and Strauss (2005), Leung and Barron (2006), Rapach and Strauss (2007), Fan et al. (2008), Inoue and Kilian (2008), Sanchez (2008), and Altavilla and Grauwe (2010).

Note that m_t exists provided E|y_t| < ∞, which is satisfied under our assumptions made in this section.

Yang (2004) assumes uniform boundedness of the fourth moments in Proposition 3.

t ≫ t_o means t is much greater than t_o.

The results in Proposition 1 also apply to the case where only some of the forecasts approach long run efficiency.

Results from additional experiments with varying sample sizes, number of forecasters, and specifications of heterogeneity in forecasters’ performance are available from the authors.

We observe no clear relationship between performance and participation. Capistrán and Timmermann (2009) and Genre et al. (2013) report similar findings.

We have also considered the Winsorized mean method where the top and bottom 5% of individual forecasts are winsorized. Although this maintains the variability of individual forecasts more than trimming, its performance is very similar to that of TM, and is therefore omitted.

For choosing the optimum learning rate, Sancetta (2010) proposes two methods in addition to choosing the learning rate ex post. Based on our results, in most cases, the performance of MLS algorithm is found to be insensitive to how the learning rate is chosen, similar to the findings in (Sancetta (2010), p. 613). Results obtained using automatically-determined, data-dependent learning rate are available upon request.

In what follows, we report only the results from using the second imputation method. Also, we omit results related to the median, trimmed mean, and Winsorized mean combination methods because they did not contribute any additional insight to our findings in the article.

Only exceptions are a few L₁-AFTER losses for UNEMP for current and one-quarter-ahead forecasts where the errors are very small fractions, so that their squares become much less that their absolute values.

See for example, Larrick and Soll (2006), Yaniv and Milyavsky (2007), Vul and Pashler (2008), Herzog and Hertwig (2009), and Soll and Larrick (2009).