What is the value of the cross-sectional approach to deep reinforcement learning?

References

• Aboussalah, A.M. and Lee, C.-G., Continuous control with stacked deep dynamic recurrent reinforcement learning for portfolio optimization. Expert Syst. Appl., 2020, 140, 112891.
   Google Scholar
• Algoet, P.H. and Cover, T.M., Asymptotic optimality and asymptotic equipartition properties of log-optimum investment. Ann. Probab., 1988, 16, 876–898.
   Web of Science ®Google Scholar
• Almahdi, S. and Yang, S.Y., An adaptive portfolio trading system: A risk-return portfolio optimization using recurrent reinforcement learning with expected maximum drawdown. Expert Syst. Appl., 2017, 87, 267–279.
   Web of Science ®Google Scholar
• Andrews, D.W.K., Cross-section regression with common shocks. Econometrica, 2005, 73(5), 1551.
   Web of Science ®Google Scholar
• Bekiros, S.D., Heterogeneous trading strategies with adaptive fuzzy actor–critic reinforcement learning: A behavioral approach. J. Econ. Dyn. Control, 2010, 34(6), 1153–1170.
   Google Scholar
• Bertsekas, D.P., Dynamic programming and optimal control, Vol. 1, 1995 (Athena Scientific: Belmont).
   Google Scholar
• Bertsekas, D.P. and Shreve, S.E., Stochastic Optimal Control: The Discrete-time Case, Vol. 23, 1978 (Academic Press: Cambridge).
   Google Scholar
• Breiman, L., Optimal gambling systems for favorable games. In Berkeley Symposium on Mathematical Statistics and Probability, 1961 (Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability: Berkeley).
   Google Scholar
• Cornuejols, G. and Tutuncu, R., Optimization Methods in Finance, 2006 (Cambridge University Press: Pittsburgh).
   Google Scholar
• Cover, T., Universal portfolios. Math. Finance, 1991, 1(1), 1–29.
   Google Scholar
• Deng, Y., Bao, F., Kong, Y., Ren, Z. and Dai, Q., Deep direct reinforcement learning for financial signal representation and trading. IEEE. Trans. Neural. Netw. Learn. Syst., 2017, 28(3), 653–664.
   PubMed Web of Science ®Google Scholar
• Elman, J.L., Finding structure in time. Cogn. Sci., 1990, 14(2), 179–211.
   Web of Science ®Google Scholar
• Fischer, T.G., Reinforcement learning in financial markets – A survey. Institute for Economics, 2018.
   Google Scholar
• Grondman, I., Busoniu, L., Lopes, G.A. and Babuska, R., A survey of actor-critic reinforcement learning: Standard and natural policy gradients. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.), 2012, 42(6), 1291–1307.
   Web of Science ®Google Scholar
• Györfi, L. and Schäfer, D., Nonparametric prediction. In Advances in Learning Theory: Methods, Models and Applications, pp. 339–354, 2003 (Budapest University of Technology and Economics: Budapest).
   Google Scholar
• Györfi, L., Lugosi, G. and Udina, F., Nonparametric kernel-based sequential investment strategies. Math. Finance, 2006, 16, 337–357.
   Web of Science ®Google Scholar
• Györfi, L., Urbán, A. and Vajda, I., Kernel-based semi-log-optimal empirical portfolio selection strategy. Int. J. Theor. Appl. Finance, 2007, 10, 505–516.
   Google Scholar
• Haugh, M. and Lo, A., Computational challenges in portfolio management. Comput. Sci. Eng., 2001, 3, 54–59.
   Web of Science ®Google Scholar
• Hesterberg, T., Importance sampling in multivariate problems. In Proceedings of the Statistical Computing Section, American Statistical Association 1987 Meeting, pp. 412–417, 1987 (American Statistical Association: San Francisco).
   Google Scholar
• Hutter, F., Hoos, H.H. and Leyton-Brown, K., Sequential model-based optimization for general algorithm configuration. In International Conference on Learning and Intelligent Optimization, pp. 507–523, 2011 (International Conference on Learning and Intelligent Optimization: Rome).
   Google Scholar
• Jiang, Z., Xu, D. and Liang, J., A deep reinforcement learning framework for the financial portfolio management problem. ArXiv:1706.10059, 2017.
   Google Scholar
• Kelly, J.L., A new interpretation of information rate. The Bell Syst. Tech. J., 1956, 35(4), 917–926.
   Google Scholar
• Konda, V.R. and Tsitsiklis, J.N., Actor-critic algorithms. In Proceedings of the conference on Advances in Neural Information Processing Systems, pp. 1008–1014, 2000 (The conference on Advances in Neural Information Processing Systems: Denver).
   Google Scholar
• Li, B. and Hoi, S., Online portfolio selection: A survey. ACM Comput. Surv., 2014, 46(3), 1–33.
   Web of Science ®Google Scholar
• Li, H., Dagli, C.H. and Enke, D., Short-term stock market timing prediction under reinforcement learning schemes. In Proceedings of the IEEE international symposium on Approximate Dynamic Programming and Reinforcement Learning, pp. 233–240, 2007 (IEEE: Honolulu).
   Google Scholar
• Liang, Z., Chen, H., Zhu, J., Jiang, K. and Li, Y., Adversarial deep reinforcement learning in portfolio management. ArXiv:1808.09940, 2018.
   Google Scholar
• Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D. and Wierstra, D., Continuous control with deep reinforcement learning. Preprint arXiv:1506.02971v4, 2016.
   Google Scholar
• Lu, D.W., Agent inspired trading using recurrent reinforcement learning and LSTM neural networks. arxiv, 2017. https://arxiv.org/pdf/1707.07338.pdf.
   Google Scholar
• Maringer, D. and Ramtohul, T., Regime-switching recurrent reinforcement learning for investment decision making. Comput. Manage. Sci., 2012, 9, 89–107.
   Google Scholar
• Markowitz, H., Portfolio selection. J. Finance., 1952, 7, 77–91.
   Web of Science ®Google Scholar
• Markowitz, H., Portfolio theory as I still see it. Ann. Rev. Financ. Econ., 2010, 2, 1–23. URL https://doi.org/https://doi.org/10.1146/annurev-financial-011110-134602.
   Web of Science ®Google Scholar
• Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S. and Hassabis, D., Human-level control through deep reinforcement learning. Nature, 2015, 518, 529–533.
   PubMed Web of Science ®Google Scholar
• Moody, J. and Saffell, M., Learning to trade via direct reinforcement. IEEE Trans. Neural Net., 2001, 12(4), 875–889.
   PubMed Web of Science ®Google Scholar
• Necchi, P.G., Policy gradient algorithms for the asset allocation problem. In Politecnico di Milano, Scuola di Ingegneria Industriale e dell' Informazione, 2016 (Politecnico di Milano: Milan).
   Google Scholar
• Neuneier, R., optimal asset allocation using adaptive dynamic programming. In Advances in Neural Information Processing Systems, 1996 (Advances in Neural Information Processing Systems: Denver).
   Google Scholar
• Puterman, M.L., Markov Decision Processes: Discrete Stochastic Dynamic Programming, 1994 (John Wiley & Son: Hoboken, NJ).
   Google Scholar
• Schulman, J., Levine, S., Abbeel, P., Jordan, M. and Moritz, P., Trust region policy optimization. In Proceedings of the 32nd International Conference on Machine Learning, 2017 (Proceedings of the 32nd International Conference on Machine Learning: Sydney).
   Google Scholar
• Schulman, J., Wolski, F., Dhariwal, P., Radford, A. and Klimov, O., Proximal policy optimization algorithms. ArXiv preprint arXiv:1707.06347, 2017.
   Google Scholar
• Sharpe, W.F., The sharpe ratio. J. Portfolio Manag., 1994, 21(1), 49–58.
   Web of Science ®Google Scholar
• Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D. and Riedmiller, M., Deterministic policy gradient algorithms. Proceedings of the 31st Int. Conference Machine Learning (ICML), 2014, 32(1), 387–395.
   Google Scholar
• Sutton, R. and Barto, A., Reinforcement Learning: An Introduction, 2017 (The MIT Press: Cambridge).
   Google Scholar
• van Roy, B., Temporal-difference learning and applications in finance. In Computational Finance Conference, 1999 (MIT Press: New York).
   Google Scholar
• Wang, Y., Wang, D., Zhang, S., Feng, Y., Li, S. and Zhou, Q., Deep q-trading. CSLT Technical Report-20160036, 2017.
   Google Scholar
• Wooldridge, J.M., Part 1: Regression analysis with cross sectional data. In Introductory Econometrics: A Modern Approach (4th ed.), 2009 (South-Western Cengage Learning: Boston).
   Google Scholar
• Zarkias, K.S., Passalis, N., Tsantekidis, A. and Tefas, A., Deep reinforcement learning for financial trading using price trailing. In International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019 (IEEE: Brighton).
   Google Scholar
• Zhengyao, J. and Liang, J., Cryptocurrency portfolio management with deep reinforcement learning. In Intelligent Systems Conference (IntelliSys), 2017 (IEEE: London).
   Google Scholar