Dynamic decision making with predictive panels

Abstract

This paper studies the dynamics of realized accuracy obtained with predictive panel models. A decision maker is affected by a loss of accuracy from an estimated model with respect to out-of-sample data. We investigate the link between this loss of accuracy and changes in the distribution of the underlying data from the estimation phase (in-sample) to the out-of-sample tests. We then model the norms of distributional changes with positive autoregressive processes in order to predict the loss of accuracy. Based on two different financial datasets, our empirical results show that our indicators have a strong explanatory power over realized portfolio returns.

Keywords:

• Predictive regressions
• Panel models
• Error decomposition
• Out-of-sample accuracy
• Distribution shifts

JEL:

• C33
• C58
• G17

Acknowledgements

We thank two anonymous referees and an anonymous associate editor for their comments and suggestions that helped us significantly improve the quality of our manuscript. Remaining errors are our own.

Notes

1 This issue is hardly new and we, for example, refer to Quionero-Candela et al. (2009) for a collection of articles on this topic when models rely on machine learning.

2 We work in the usual 2-dimensional setting for ${\mathit{X}}_t,$ but recent advances consider higher dimensions with tensors (see Schosser (2022)).

3 Theoretically, Eichner and Wagener (2011) have shown the impact of mean-variance portfolio compositions when means or covariances change.

4 We consider that panel models nest simple regressions as particular cases, when there is only one time index.

5 This assumption can be relaxed and in this case, a new term would appear in the form of ${\mathit{y}}_{\text{oos}}^{\prime }{\mathit{y}}_{\text{oos}}-{\mathit{y}}_{\text{is}}^{\prime }{\mathit{y}}_{\text{is}}.$ This addition does not carry much insight and burdens the terms unnecessarily.

6 Other choices of norms alter the results only marginally. The correlations between time-series of $D_t$ for different types of norms is above 95% (and sometimes above 99%): all norms experience the exact same co-movements. We tested the Frobenius norm and the $L^1$ (maximum absolute column sum). The results are available upon request.

7 As expected, subsamples are associated with fewer significant coefficients, but this comes from smaller sizes (Lin et al. (2013)).

8 See Aknowledgements section.

9 Alternatively, recent methods aim to mitigate the sensitivity to estimation or misspecification risk, see van Staden et al. (2021) and the references therein.

10 Using alternative weighting schemes (e.g., capitalization-based) introduces biases (e.g., towards large firms) which pollutes the signal from the predictions.

11 Another popular task in financial modelling is volatility forecasting (see, e.g., Meligkotsidou et al. (2019)), and in this case, a dedicated auto-regressive model with exogenous predictors should be developed, but this is out of the scope of the present paper.

12 https://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html