Enhanced Portfolio Optimization

Abstract

Portfolio optimization should provide large benefits for investors, but standard mean–variance optimization (MVO) works so poorly in practice that optimization is often abandoned. Many of the approaches developed to address this issue are surrounded by mystique regarding how, why, and whether they really work. So, we sought to simplify, unify, and demystify optimization. We identified the portfolios that cause problems in standard MVO, and we present here a simple “enhanced portfolio optimization” method. Applying this method to industry momentum and time-series momentum across equities and global asset classes, we found significant alpha beyond the market, the 1/N portfolio, and standard asset pricing factors.

Disclosure: The authors report no conflicts of interest. AQR Capital Management is a global investment management firm that may or may not apply similar investment techniques or methods of analysis as described here. The views expressed here are those of the authors and not necessarily those of AQR. Lasse Heje Pedersen gratefully acknowledges support from Center for Financial Frictions (Grant No. DNRF102).

Editor’s Note:

Submitted 13 August 2020

Accepted 17 November 2020 by Stephen J. Brown

Acknowledgments

We thank Michele Aghassi, Stephen Brown (the editor), Ben Davis, Victor DeMiguel, Antti Ilmanen, Roni Israelov, Bryan Kelly, Lorenzo Garlappi, Ernst Schaumburg, and Raman Uppal for helpful comments and Matthew Silverman and Jusvin Dhillon for excellent research assistance.

Notes

¹ A large literature has addressed estimation noise—for example, Ledoit and Wolf (2003, 2004) on noise in variance–covariance matrices and Black and Litterman (1992) on noise in expected returns.

² Note that this result is not simply the same as saying that averaging portfolios improves performance (as shown by Tu and Zhou 2011). We found that EPO can work even better. For example, if we first compute the standard MVO portfolio without shrinkage, $x^{w=0}$, and the solution with full shrinkage, $x^{w=1}$, and then take the average of these, $a{x}^{w=0}+\left(1-a\right)x^{w=1}$, the result does not work as well as our EPO method for any $a$, especially if the MVO is particularly ill behaved. The EPO method first shrinks and then optimizes, not the other way around, which is useful because shrinking the correlations stabilizes the optimization process.

³ We unify several leading approaches to optimization, but EPO obviously does not nest all methods. Roncalli (2013) and Bruder, Gaussel, Richard, and Roncalli (2013) reviewed various methods of regularizing MVO, including a discussion of the eigendecomposition of the variance–covariance matrix similar to our problem portfolios, showing that the risk of these portfolios is low. We additionally show that the expected return of problem portfolios is too high (see Panel B of Figure 1) and that large EPO shrinkage can help address both these problems. DeMiguel et al. (2009), considering 14 methods of optimization, found that none consistently outperformed the simple 1/N portfolio. Some methods do show promise in outperforming the 1/N portfolio, however, such as methods that constrain the portfolio norm (Jagannathan and Ma 2003; DeMiguel, Garlappi, Nogales, and Uppal 2009), methods based on ambiguity aversion (Garlappi, Uppal, and Wang 2007), methods that average several approaches (Tu and Zhou 2011), and methods that apply careful MVO with good inputs (Allen, Lizieri, and Satchell 2019).

⁴ Although a version of EPO can be shown to be equivalent to Black and Litterman (1992), there are several differences. Indeed, Black and Litterman always shrank toward the market portfolio, whereas we consider a general anchor (or no anchor); they considered long–short “view portfolios,” whereas we simply consider signals about expected returns, such as industry momentum or time-series momentum, and we allow “double shrinkage”— of both the estimated expected returns and the variance–covariance matrix. Most importantly, our contribution is to unify this approach with other optimization methods by showing the link to correlation shrinkage (which is not clear from the equations in Black and Litterman, p. 42), by presenting a simple, new, and powerful way to operationalize the method, and by documenting empirically how it works.

⁵ Appendix A describes a method to stabilize the risk model that is more sophisticated than shrinking correlations called “random matrix theory” (RMT). We have found empirically, however, that EPO works as well with simple correlation shrinkage as with RMT.

⁶ The variance of $\eta$ is proportional to $\Sigma$ in order to capture the idea that true fluctuations in expected returns are correlated across correlated assets (similar to the assumption made in Point 7 of the appendix of Black and Litterman 1992). Expressed in a different way, the PC portfolios have expected returns $P\text{'}{\sigma }^{-1}\mu =\gamma P\text{'}{\sigma }^{-1}\Sigma a+P\text{'}{\sigma }^{-1}\eta$, where the random fluctuation term, $P\text{'}{\sigma }^{-1}\eta$, has variance $\tau P\text{'}{\sigma }^{-1}\Sigma {\sigma }^{-1}P=\tau D$, implying that the expected returns of the least important principal components vary the least.

⁷ To understand the anchor at a deeper level, consider again the case of $\eta =0$. In this case, the expected excess return on any asset—say, asset number 1, is $\text{E}\left(r_1\right)=\gamma \left(1,0,\dots ,0\right)\Sigma a=\gamma \text{ cov}\left(r_1,r_a|s\right)$. Using this relationship for anchor portfolio $a$ and solving for $\gamma =\text{E}\left(r_{\text{a}}\right)/\text{var}\left(r_a|s\right)$, we get $\text{E}\left(r_1\right)=\left[\text{cov}\left(r_1,r_a|s\right)/\text{var}\left(r_a|s\right)\right]\text{E}\left(r_a\right)=:{\beta }_{1,a}\text{E}\left(r_a\right)$. If $a$ is the market portfolio, this relationship is simply the conditional capital asset pricing model (CAPM). Hence, Equation 12$\mu =\gamma \Sigma a+\eta ,$ (12) defining $\mu$ means that the CAPM holds, on average, but $\eta$ pushes the expected returns around in such a way that the CAPM does not always hold exactly, resulting in trading opportunities. More generally, Equation 12$\mu =\gamma \Sigma a+\eta ,$ (12) says that the anchor is the tangency portfolio when there are no shocks ($\eta =0$).

⁸ To our knowledge, the specification of Equation 15$\underset{x}{\max }\underset{\mu }{\min }\left[(x-a{)}^{\prime }\mu -\frac{\gamma }{2}{x}^{\prime }\Sigma x\right]$$\text{subject to}$$\mu \in \left\{\overline{\mu } | {\left(\overline{\mu }-s\right)}^{\prime }{\Lambda }^{-1}\left(\overline{\mu }-s\right)\le c^2\right\}.$ (15) and its solution is new, but Fabozzi et al. (2010) considered a version of Equation 15$\underset{x}{\max }\underset{\mu }{\min }\left[(x-a{)}^{\prime }\mu -\frac{\gamma }{2}{x}^{\prime }\Sigma x\right]$$\text{subject to}$$\mu \in \left\{\overline{\mu } | {\left(\overline{\mu }-s\right)}^{\prime }{\Lambda }^{-1}\left(\overline{\mu }-s\right)\le c^2\right\}.$ (15) that is simpler in two ways: First, whereas we consider a general Λ, Fabozzi et al. assumed that Λ equals Σ, which means that there is no shrinkage of the variance–covariance matrix, and second, Fabozzi et al. did not have an anchor portfolio.

⁹ The assumption of independence of errors in the expected returns across securities, $\Lambda =\lambda V,$ implies that the error in the measurement of the expected return of the principal components has a variance given by $P\text{'}{\sigma }^{-1}\left(\lambda V\right){\sigma }^{-1}P= \lambda I$, where $I$ is the identity matrix. That is, errors of all the principal components are independent and of equal magnitude.

¹⁰ Alternatively, we can think of the anchor being $a=0$, which gives the same result as Equation 20$EP{O}^s\left(w\right)=\frac{1}{\gamma }{\Sigma }_w^{-1}s,$ (20) up to a constant that can be absorbed in the risk aversion coefficient. However, we think of the anchor as also being the EPO portfolio with full shrinkage, $w=1$, implying that $a=\left(1/\gamma \right)V^{-1}s$ is the more natural interpretation of Equation 20$EP{O}^s\left(w\right)=\frac{1}{\gamma }{\Sigma }_w^{-1}s,$ (20) .

¹¹ Investors can also avoid specifying $\gamma$ altogether by solving an equivalent optimization that maximizes expected returns subject to a maximum volatility constraint, thus specifying a volatility target in lieu of $\gamma$.

¹² Choosing $\gamma$ may be done in several other, related ways, some of which work better than others. For example, although $\gamma$ in Equation 22$$EP{O}^a\left(w\right)={\Sigma }_w^{-1}\left[\left(1-w\right)\frac{\sqrt{a^{\prime }\tilde{\Sigma }a}}{\sqrt{s^{\prime }{\Sigma }_w^{-1}\tilde{\Sigma }{\Sigma }_w^{-1}s}}s+wVa\right].$$ (22) equalizes the variance of the anchor with that of $(1/\text{γ)}{\Sigma }_w^{-1}s\text{,}$ one could also replace the latter with the variance of the standard MVO solution, $(1/\text{γ)}{\tilde{\Sigma }}^{-1}s\text{,}$ but this is a poor choice if the standard MVO is ill behaved. Ao et al. (2019) and Raponi et al. (2020) also considered methods where $\gamma$ is based on variance.

¹³ Specifically, the general EPO is the solution to a Lavrentiev regularization (Lavrentiev 1967), and the simple EPO is the solution to a Tikhonov regularization. The simple EPO can also be seen as a ridge regression of a vector of 1s on the matrix of realized returns when risk and expected returns are estimated by their sample counterparts.

¹⁴ If we had included non-USD currency pairs, then the variance–covariance matrix would not be of full rank because, for example, EUR–USD, EUR–JPY, and USD–JPY are linked through a triangular arbitrage.

¹⁵ Available at https://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html.

¹⁶ The annualized variance of instrument $i$ was estimated as ${\left({\sigma }_t^i\right)}^2=261{\displaystyle {\sum }_{k=0,\dots ,\infty }\left(1-\delta \right){\delta }^k}{\left(r_{t-1-k}^i-{\overline{r}}_t^i\right)}^2,$ where ${\overline{r}}_t^i$ is the exponentially weighted average return computed similarly, 261 annualizes the daily returns, and $\delta$ was chosen to achieve a center of mass of ${\displaystyle {\sum }_{k=0,\dots ,\infty }\left(1-\delta \right){\delta }^{k}k=\delta /\left(1-\delta \right)=60}\text{ days}$. The correlations were estimated by first computing covariance and volatilities in the corresponding way—using 3-day returns with 150-day center of mass—and then computing the correlations as ratios of the covariances to the product of the volatilities. We required at least 300 days of data to be available for an asset before it entered the covariance matrix.

¹⁷ In other words, the covariance of assets $i$ and $j$ is estimated as $\left[1/(K-1)\right]{\displaystyle \sum }_{k=1,\dots ,K}\left(r_{t-k}^i-{\overline{r}}_t^i\right)\left(r_{t-k}^j-{\overline{r}}_t^j\right)$.

¹⁸ Some studies have considered longer time horizons—for example, past five-year returns. Past long-term returns, however, predict returns negatively, if at all, perhaps because securities that have risen in price over a long time have become expensive (De Bondt and Thaler 1985). Alas, comparing optimization methods using a faulty signal of expected returns is not informative.

¹⁹ Babu et al. (2020) reported a median time-series momentum Sharpe ratio per asset of 0.34 per year (i.e., 0.10 per month) for traditional assets.

²⁰ Indeed, this coefficient implies that the EPO portfolio with full shrinkage, $EP{O}^s\left(w=100\%\right)=(1/{\gamma }_t)V_t^{-1}{s}_t$, has a notional exposure to asset $i$ that matches that of Moskowitz et. al (2012) given in Appendix A. That is, $EP{O}^s{\left(w=100\%\right)}^i =(1/{\gamma }_t)\left[s_t^i/{\left({\sigma }_t^i\right)}^2\right]=(1/n_t)\left(40\%/{\sigma }_t^i\right) \text{sign}\left(r_{t-12,t}^i\right)$.

²¹ Because the number of assets in our sample varied over time, we scaled the realized and ex ante average returns and volatilities to preserve the trace of the correlation matrix—that is, ensuring that the sum of variances would equal the largest number of assets in our sample, 55.

²² From 1963 to 2018, the five Fama–French factors realized Sharpe ratios between 0.27 and 0.49 and the equal-weighted portfolio of all five factors realized a Sharpe ratio of 0.93.

²³ Estimates of the eigenvectors are kept equal to the sample eigenvectors to make the estimate of the correlation matrix rotational invariant, meaning that rotating the data by some orthogonal matrix rotates the estimator in the same way (see Ledoit and Wolf 2012; Bun et al. 2017).

²⁴ The Woodbury matrix identity shows a way to rewrite the inverse of a sum of matrices and, using the Woodbury formula, we see that $\left[I-{\left(\text{τ}\Sigma +\Lambda \right)}^{-1}\text{τ}\Sigma \right]={\left(I+{\Lambda }^{-1}\text{τ}\Sigma \right)}^{-1}={\left[{\Lambda }^{-1}\left(\Lambda +\text{τ}\Sigma \right)\right]}^{-1}={\left(\text{τ}\Sigma +\Lambda \right)}^{-1}\Lambda .$.

²⁵ We can also write the regression in a simpler way, $\frac{1}{\text{γ}}s=\Sigma x+\epsilon$, as we do when we consider the Lavrentiev regularization. When we use the standard ridge regression on this simpler equation, we get $x=\frac{1}{\gamma }{\left({\Sigma }^2+\text{λ}I\right)}^{-1}\Sigma s$, so we have written the regression differently to avoid the $\Sigma$-squared.