How Market Sentiment Drives Forecasts of Stock Returns

Abstract

We reveal a novel channel through which market participants’ sentiment influences how they forecast stock returns: their optimism (pessimism) affects the weights they assign to fundamentals. Our analysis yields four main findings. First, if good (bad) “news” about dividends and interest rates coincides with participants’ optimism (pessimism), the news about these fundamentals has a significant effect on participants’ forecasts of future returns and has the expected signs (positive for dividends and negative for interest rates). Second, in models without interactions, or when market sentiment is neutral or conflicts with news about dividends and/or interest rates, this news often does not have a significant effect on ex ante or ex post returns. Third, market sentiment is largely unrelated to the state of economic activity, indicating that it is driven by non-fundamental considerations. Moreover, market sentiment influences stock returns highly irregularly, in terms of both timing and magnitude. This finding supports recent theoretical approaches recognizing that economists and market participants alike face Knightian uncertainty about the correct model driving stock returns.

Keywords:

• Stock-return forecasts
• Fundamentals
• Market sentiment
• Structural change
• Model ambiguity

JEL Classifications::

• G12
• G14
• G41
• C58

Acknowledgements

The authors are participants in the Program on Knightian Uncertainty Economics at the Institute for New Economic Thinking (INET). The authors are grateful to INET for continuing support of this research.

Notes

1 For seminal studies documenting REH models’ difficulties in accounting for the role of fundamentals in explaining lower-frequency returns, see Shiller (1981) and Mehra and Prescott (1985). For more recent studies on these models’ empirical difficulties see, for example, Welch and Goyal (2008) and Ang and Bekaert (2007).

2 For examples of such studies of the U.S. stock market, see Baker and Wurgler (2006), Tetlock (2007), Garcia (2013), Mangee (2017), and references therein.

3 Greenwood and Shleifer use seven proxies summarizing different surveys of investors’ forecasts. An important contribution of their paper is to show that these proxies are highly correlated with market participants’ decisions to invest their capital in mutual funds. Furthermore, Greenwood and Shleifer show that the seven proxies co-move strongly and positively, even though the surveys that underpin them rely on very different methodologies. This consistency buttresses their argument that survey evidence is not just “meaningless noise” (p. 715).

4 Daily Wall Street Journal “Abreast of the Market” reports were collected through the Proquest textual data retrieval system. Our measure for market sentiment relies on the dictionary of words indicating positive and negative sentiment developed by Loughran and McDonald (2011).

5 Tetlock, Saar-Tsechansky, and Macskassy (2008) find that this is particularly true if the news articles reference fundamentals. Garcia (2013) meanwhile finds that most of sentiment’s predictive power is concentrated during recessions. These are suggestive of the interactions and asymmetries we examine more explicitly in this paper.

6 For examples of such studies, see Andersen et al. (2007) and Pearce and Roley (1985).

7 Barsky and De Long (1993) were among the first studies to suggest that in explaining stock-price fluctuations, the dividend growth rate is better approximated as non-stationary rather than assuming that it is mean-constant.

8 These three threshold levels and two measures of dividend news yield six alternative specifications of stock returns.

9 Mian and Sankaraguruswamy (2012) finds that sentiment impacts the announcement effect of firm-specific earnings.

10 For an influential contribution to the development of models that represent change with probabilistic rules, and an authoritative recent review, see Hamilton (1988, 2008). For a seminal behavioral-finance model that formalizes with a stochastic process the role of market sentiment in how participants forecast returns, see Barberis, Shleifer and Vishny (1998).

11 The historical dataset can be found at http://www.econ.yale.edu/∼shiller/data.htm.

12 For a detailed examination of the problems posed by the non-stationarity of fundamental variables in specifying stock returns, see Frydman and Stillwagon (2018).

13 We use the TextMining (TM) package available in R, which is based on the vector space method. The general results are robust to other normalizations, for example dividing by total word count or using only the unique positive and negative words each month.

14 Conventional ADF unit-root tests with auto-selected lag-length for ${\widehat{R}}_t,$ $R_t,$ $\Delta d_t,$ $\Delta i_t$ and ${\mathit{Sent}}_t$ generate t-statistics(p-values) of −6.04(.000), −14.703(.000), −2.893(.047), −8.283(.000), and −6.654(.000), respectively.

15 See, for example, De Bondt and Thaler (1985), Cutler, Poterba, and Summers (1991), Chopra, Lakonishok, and Ritter (1992), Fama and French (1992), Lakonishok, Shleifer, and Vishny (1994), and La Porta (1996).

16 In models of ex post returns, even when including sentiment additively, we find effects of dividends. This is consistent with other studies on sentiment and ex post returns. See Table A4 in the Appendix.

17 The indicators or dummy variables are selected in blocks using the Autometrics tree search algorithm (Doornik 2009). The block search produces a higher “breakdown point” and much greater robustness to outliers compared to OLS or alternatives like the least absolute deviation estimator.

18 We also regressed sentiment on our measures of dividend and interest-rate news. Dividends were highly insignificant (|t|<0.5), and interest rates were significant with the “wrong” sign. The positive correlation between sentiment and interest rates could be interpreted as reverse causation: optimism has caused rising interest rates, rather than rising interest rates leading to an improved outlook for stocks. For this reason, we rely on the orthogonalization with industrial production.

19 Structural change in how fundamentals drive ex post returns has been documented in many studies, including Pastor and Stambaugh (2001), Pettenuzzo and Timmermann (2011), Rapach and Wohar (2006), Paye and Timmermann (2006), Ang and Timmermann (2012), and Mangee (2016). Frydman and Stillwagon (2018) find structural breaks in the relationship between survey measures of ex ante returns and fundamentals.

20 There is a long literature in the foreign exchange market rejecting REH in survey data. See Stillwagon (2014) and references therein. Coibion and Gorodnichenko (2012) similarly reject full information REH for household and professional inflation expectations.

21 The ex post model is not rejected however according to the one-step-ahead Chow forecast test, where the model is allowed to update recursively. By contrast, the breakpoint Chow test examines parameter instability for each recursive sub-sample relative to the full sample.

22 Stillwagon and Sullivan (2019) find that while a Markov switching model approximates the exchange-rate process within sample, it is difficult to ascertain the number of states required for such an approximation, particularly out of sample. Moreover, Stillwagon and Sullivan demonstrate that the Markov switching model involving the best-performing number of regimes ex post, which gives the model a very favorable bias, could not outperform a random walk out of sample.

23 Hendry has pioneered econometric studies focusing on the inherent difficulties in modeling structural change with probabilistic rules. In a series of papers and books, he has demonstrated not only that macroeconomic models experience structural breaks, but also that these breaks are often triggered by historical events. See Hendry and Doornik (2014), Hendry (2018), and references therein.

24 For econometric evidence, see Ang and Timmermann (2012) and references therein. Frydman, Goldberg, and Mangee (2015) provide descriptive evidence that 20% of events that triggered movements in US stock prices between 1993 and 2009 were, at least in part, non-repetitive.