Forecasting stock market returns over multiple time horizons

Abstract

In this paper, we seek to demonstrate the predictability of stock market returns and explain the nature of this return predictability. To this end, we introduce investors with different investment horizons into the news-driven, analytic, agent-based market model developed in Gusev et al. [Algo. Finance, 2015, 4, 5–51]. This heterogeneous framework enables us to capture dynamics at multiple timescales, expanding the model’s applications and improving precision. We study the heterogeneous model theoretically and empirically to highlight essential mechanisms underlying certain market behaviours, such as transitions between bull and bear markets and the self-similar behaviour of price changes. Most importantly, we apply this model to show that the stock market is nearly efficient on intraday timescales, adjusting quickly to incoming news, but becomes inefficient on longer timescales, where news may have a long-lasting nonlinear impact on dynamics, attributable to a feedback mechanism acting over these horizons. Then, using the model, we design algorithmic strategies that utilize news flow, quantified and measured, as the only input to trade on market return forecasts over multiple horizons, from days to months. The backtested results suggest that the return is predictable to the extent that successful trading strategies can be constructed to harness this predictability.

Keywords:

• Stock market dynamics
• Return predictability
• Price feedback
• Market efficiency
• News analytics
• Sentiment evolution
• Agent-based models
• Ising
• Dynamical systems
• Synchronization
• Self-similar behaviour
• Regime transitions
• News-based strategies
• Algorithmic trading

Acknowledgements

We are grateful to LGT Capital Partners for partially funding this project and to Dow Jones & Company for providing access to the Factiva.com news archive. We would like to thank a number of people who provided their help over the course of this work: John Orthwein for editing this paper and contributing ideas on its readability; Ed King and Maxim Zhilyaev for helping with data retrieval and management; George Coplit, Matthias Feiler, Roger Hilty, Christian Jung and Francesco Valenti for useful comments on backtest set-up and robustness; and Vitaliy Godlevsky for discussions regarding market behaviours on short timescales and other helpful comments.

Notes

¹ For example, competition among high-frequency trading firms has increased the trade execution speed from roughly 100 ms to 10 μs over the last decade.

² There is a large body of research that examines, typically applying regression methods, the predictive power of observable variables, such as the dividend yield and many others; see, for example, Fama and French (1988, 1989), Campbell and Shiller (1988a, 1988b), Baker and Wurgler (2000), Campbell and Thompson (2008), Cochrane (2008). However, the evidence for return prediction remains inconclusive: e.g. Ferson et al. (2003, 2008), Goyal and Welch (2003, 2008). The model of market dynamics that we develop here is fundamentally nonlinear, indicating that causal relations among the variables are substantially more complex than regression dependence. It follows that the standard approach to return prediction may be ill-suited to capture this predictability; e.g. Novy-Marx (2014) vividly pointed out this limitation by extending stock market predictive regressions to a number of rather implausible variables, such as sunspot activity and planetary motion. We propose an alternative approach that combines theoretical models with empirical data to explore whether stock market returns can be predicted in an economically significant manner.

³ The idea that the observations of price changes may generate a feedback loop that significantly affects market dynamics is not new (see a review by Shiller 2003). However, its application for return forecasting, which is the subject of the present work, is nontrivial due to nonlinear behaviours induced by it.

⁴ A family of models, named after Ising (1925), developed originally to explain the phenomenon of ferromagnetism via the interaction of discrete atomic spins in an external magnetic field and later broadly applied to study problems in social and economic dynamics (see reviews by Castellano et al. 2009, Lux 2009, Slanina 2014, Sornette 2014). We take note of two recent works that share common ground with Gusev et al. (2015). First, Franke (2014), referencing Lux’s (1995) analytic stock market model, studied a generic sentiment-driven economic model with feedback, which has some features similar to those found in Gusev et al. (2015). Second, Carro et al. (2015) investigated the influence of exogenous information on sentiment dynamics in the stock market, which is also a central theme in Gusev et al. (2015).

⁵ This approach contrasts with that of the established agent-based financial models, where market dynamics are sought to emerge, primarily, through the interaction among agents pursuing different trading strategies, such as the influential work by Lux and Marchesi (1999) among many others.

⁶ Extensive research has been done on empirical measures of sentiment—which include indices based on periodic surveys of investor opinion; various proxies such as trading volume, call vs. put contracts and others; and applications of machine learning and rule-based techniques for parsing financial news and social media content—and on their correlation with price movement (e.g. Antweiler and Frank 2004, Brown and Cliff 2004, Baker and Wurgler 2007, Das and Chen 2007, Tetlock 2007, Loughran and McDonald 2011, Lux 2011, Da et al. 2014). Alternatively, Gusev et al. (2015) suggested a rule-based parsing methodology for measuring h and calculated sentiment s(h) from this empirical h, using the homogeneous model described in this section.

⁷ Gusev et al. (2015, equation (13), figure 12). The dot denotes the derivative with respect to time. Parameters are positive, while s_* can take any sign. The parameter values, estimated using the empirical data, are provided in Table 1 of that same paper. We note that equation (1b), with h as an exogenous variable, was originally obtained by Suzuki and Kubo (1968) in the context of a purely statistical mechanics problem.

⁸ This form of the equation neglects the impact of direct interaction between the agents, omitting the terms proportional to h and s in the argument of the hyperbolic tangent (Gusev et al. 2015: equation (12c)).

⁹ The all-to-all interaction mode is the leading-order approximation for a general interaction topology in this model.

¹⁰ Gusev et al. (2015) estimated τ_s to be around one month.

¹¹ Cont and Bouchaud (2000) addressed heterogeneity in opinion formation as a topological problem within the framework of percolation theory from chemistry and physics, leading to the emergence of clusters of investors with shared sentiment. Following this work, a number of percolation models have been proposed that replicate some of distinctive market behaviours. However, as mentioned above, the results are sensitive to the choice of topology in a model and it is difficult to economically justify any one particular topology choice.

¹² Similarly, the average memory time span can be presumed proportional to the investment time horizon. We note that the agent-based market model developed in a series of publications by Levy, Levy and Solomon (see Levy et al. 2000) includes investors with different memory time spans.

¹³ The basic argument is as follows: Consider an investor who has just allocated capital to the market. The following day, this investor is unlikely to amend her allocation unless her sentiment has changed. This is because the capital that the investor has already deployed reflects this same level of sentiment. Therefore, ignoring external constraints, investment allocations can be presumed to be driven by the change in sentiment on timescales where the investors’ memory of past sentiment levels persists . Conversely, on longer timescales investors would invest or divest depending on the level of sentiment itself since their previous allocation decisions would not be linked in their memory to the previous levels of sentiment. Since capital flows lead to price changes, these two asymptotic views can be superposed to yield the approximate price-sentiment relation in the form of equation (1a). For details, see Gusev et al. (2015, Section 1.3.1).

¹⁴ This equation should be treated as an average relation applicable under normal market conditions or over extended time periods. In particular, it is not expected to hold during spikes in trading activity, such as those accompanying market crashes. Additionally, it may not be true for groups with very long investment horizons because on the corresponding timescales effects due to the finite size of the market can affect the assumed linear dependence between the horizon and the amount of investment capital. Nevertheless, as a first approximation, this relation will prove helpful for gaining insight into market dynamics on the relevant timescales.

¹⁵ Since and , the term in (5a) is dominated by the sentiment of long-term investors, that is s_i corresponding to large τ_i. Also, equation (5b) implies that s_i varies by O(1) over τ_i, i.e. the longer the investment horizon, the slower the sentiment variation. Therefore, contributes to price development over the long term, e.g. months and years, which enables us to replace it in the leading order by a constant growth rate.

¹⁶ As β₁ increases, the potential well undergoes a bifurcation from a single-well U-shape to a double-well W-shape at β₁ = 1. The potential is symmetric for δ = 0; positive δ breaks the symmetry, making the part of the well where sentiment is positive deeper and the part where sentiment is negative more shallow.

¹⁷ The motion on the (s, h)-plane bears resemblance to the motion on the -plane.

¹⁸ This analysis is relevant for the particles with . In Section 3.4, we will show that the ‘ultra-light’ particles with possess no intrinsic dynamics, adjusting instead to the dynamics of ‘heavier’ particles.

¹⁹ These constraint equations constitute nontrivial first integrals of motion, out of which N – 1 are independent. The independent first integrals reduce the degrees of freedom of system (7) from 2N to N + 1, which matches the number of equations in the dynamical system (6).

²⁰ Synchronization is ubiquitous among the behaviours of interacting nonlinear oscillators: e.g. Pikovsky et al. (2001) provide an in-depth treatment of various synchronization effects in coupled oscillator systems.

²¹ On daily intervals, ξ_t is modelled as normally distributed white noise with zero mean and unit variance. However, we have chosen ξ_t to have a small positive intraday autocorrelation on the assumption that news events are positively correlated on intraday time intervals; the autocorrelation is zero over the intervals of one day or longer.

²² In this paper, we do not consider the effect of a slowly varying β₁ on regime transitions, studied in detail by Gusev et al. (2015) for N = 1. These authors showed that β₁ slightly increased during the bull markets and slightly decreased during the bear markets (while remaining above unity), affecting the shape of the potential well and, therefore, the probability of regime transitions.

²³ Incidentally, the behaviour of these two investor groups resembles that of the two types of investors ubiquitous in the market modelling literature: fundamental traders and systematic traders.

²⁴ A more detailed discussion about the measurement of h can be found in Gusev et al. (2015). We only note here that these authors proposed to treat each news item as if it were a ‘sales pitch’ aimed at investors to buy or sell the market. This allowed them to employ ideas from marketing research and put forward an argument that h—information patterns that explicitly mention the direction of expected market movement—impacts investors most. In practice, news about current and recent market price changes, which make up the bulk of the study’s relevant news volume, can also influence investors. As such, this information is included in the measurement of h, along with information concerning anticipated market movement.

²⁵ This result holds for model (6) with N ≥ 2. For simplicity, we show its derivation in the case N = 2.

²⁶ Strictly speaking, although s₁ spends most of its time (~τ_h) on the isocline, where its velocity is close to zero, it can also leave the isocline and move briefly (~τ₁) along a trajectory in its vicinity (figure 7). Therefore, is nearly zero at all times, except for brief moments when the trajectory departs the isocline, so that the average contribution in (9c) due to s₁ is small as compared to s₂ and can be neglected.

²⁷ This approximation does not work in a system with N = 1, as there exist for which the term in (9c) cannot be neglected. As a result, for large the coupling between s₁ and h can be strong enough to cause a limit cycle dynamic. The situation is different in systems with N ≥ 2 that simulate market dynamics with a greater precision. There, can in average be neglected in comparison with () in (9c), so that the motion of s₁ is completely determined in this case by the dynamics between h and s_i≠1.

²⁸ This analytical result, which follows from equations (10) and (11), was also verified by direct numerical simulations. In addition, we note that equations (10) and (11) can be obtained by rescaling system (9) using a dimensionless time variable and then inspecting the leading-order balance on relevant timescales; we have chosen an informal derivation above for the sake of preserving the readability of this section.

²⁹ This term is applied in a collective sense, comprising financial analysts, newspaper journalists, market commentators, finance bloggers and other participants who communicate their market views through mass media.

³⁰ For example, we swim by using water’s resistance to create momentum; however, this strategy would fail if we were the size of bacteria: for micro-organisms, water appears as viscous as honey for humans, forcing them to evolve unique propulsion techniques, such as corkscrew-like locomotion mechanisms among others.

³¹ We follow the steps of a similar derivation for N = 1 in Gusev et al. (2015, Appendix C). That appendix also provides a detailed analysis of the phase portrait geometry, including the bifurcations of equilibrium points and the formation of a stable limit cycle.