Data-driven optimal tracking control of discrete-time linear systems with multiple delays via the value iteration algorithm

References

• Aggarwal, J. K., Banks, H. T., & Eller, D. H. (1969). Optimal control of linear time-delay systems. IEEE Transactions on Automatic Control, 14(6), 678–687 . https://doi.org/10.1109/TAC.1969.1099301
   Web of Science ®Google Scholar
• Bertsekas, D. P. (2008). Neuro-dynamic programming. In C. Floudas & P. Pardalos (Eds.), Encyclopedia of optimization. Boston, MA: Springer.
   Google Scholar
• Dreyfus, S. E., & Stuart, E. (1962). Applied dynamic programming. Princeton University Press.
   Google Scholar
• Garate-Garcia, A., Marquez-Martinez, L. A., & Moog, C. H. (2011). Equivalence of linear time-delay systems. IEEE Transactions on Automatic Control, 56(3), 666–670. https://doi.org/10.1109/TAC.2010.2095550
   Web of Science ®Google Scholar
• Ha, I. J., & Gilbert, E. G. (1987). Robust tracking in nonlinear systems. IEEE Transactions on Automatic Control, 32(9), 763–771. https://doi.org/10.1109/TAC.1987.1104710
   Web of Science ®Google Scholar
• Kirk, D. E. (1970). Optimal control theory: An introduction. Prentice-Hall.
   Google Scholar
• Kiumarsi, B., Lewis, F. L., Modares, H., Karimpour, A., & Naghibi, M. B. (2014). Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics. Automatica, 50(4), 1167–117. https://doi.org/10.1016/j.automatica.2014.02.015
   Web of Science ®Google Scholar
• Kiumarsi, B., Lewis, F. L., Naghibi-Sistani, M. B., & Karimpour, A. (2015). Optimal tracking control of unknown discrete-time linear systems using input–output measured data. IEEE Transactions on Cybernetics, 45(12), 2770–2779. https://doi.org/10.1109/TCYB.2014.2384016
   PubMed Web of Science ®Google Scholar
• Kolmanovskii, V., & Myshkis, A. (1999). Introduction to the theory and applications of functional differential equations. Dordrecht: Kluwer Academy.
   Google Scholar
• Lewis, F., & Vrabie, D. (2009). Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits & Systems Magazine, 9(3), 32–50. https://doi.org/10.1109/MCAS.2009.933854
   Web of Science ®Google Scholar
• Liu, H. L., Tang, G. Y., & Yang, X. (2016). Minimum-energy control for time-varying systems with multiple state and input delays. International Journal of Systems Science, 47(12), 3036–3043. https://doi.org/10.1080/00207721.2015.1068883.
   Web of Science ®Google Scholar
• Liu, M., & You, J. (2012). Observer-based controller design for networked control systems with sensor quantisation and random communication delay. International Journal of Systems Science, 43(10), 1901–1912. https://doi.org/10.1080/00207721.2011.555013
   Web of Science ®Google Scholar
• Liu, Y., Zhang, H., Luo, Y., & Han, J. (2016). ADP based optimal tracking control for a class of linear discrete-time system with multiple delays. Journal of the Franklin Institute, 353(9), 2117–2136. https://doi.org/10.1016/j.jfranklin.2016.03.012
   Web of Science ®Google Scholar
• Liu, Y., Zhang, H., Yu, R., & Qu, Q. (2018). Data-driven optimal tracking control for discrete-time systems with delays using adaptive dynamic programming. Journal of the Franklin Institute, 355(13), 5649–5666. https://doi.org/10.1016/j.jfranklin.2018.06.013
   Web of Science ®Google Scholar
• Lu, X., Liu, R., Lv, C., Wang, N., Zhang, Q., Wang, H., Zhang, G., & Liang, X. (2021). Control for networked control systems with multiplicative noises, packet dropouts and multiple delays. International Journal of Systems Science, 3, 1–12. https://doi.org/10.1080/00207721.2021.1929556
   Web of Science ®Google Scholar
• Lu, X., Zhang, Q., Liang, X., Wang, H., Sheng, C., & Zhang, Z. (2020). Optimal control for networked control systems with multiple delays and packet dropouts. International Journal of Advanced Robotic Systems, 17(3). https://doi.org/10.1177/1729881420913765
   PubMed Web of Science ®Google Scholar
• Mehrmann, V. (1995). Algebraic Riccati equations (Peter Lancaster and Leiba Rodman). Siam Review, 38(4), 694–695. https://doi.org/10.1137/1038142
   Google Scholar
• Palanisamy, M., Modares, H., Lewis, F. L., & Aurangzeb, M. (2015). Continuous-time Q-learning for infinite-horizon discounted cost linear quadratic regulator problems. IEEE Transactions on Cybernetics, 45(2), 165–176. https://doi.org/10.1109/TCYB.2014.2322116
   PubMed Web of Science ®Google Scholar
• Pindyck, R. (1972). The discrete-time tracking problem with a time delay in the control. IEEE Transactions on Automatic Control, 17(3), 397–398. https://doi.org/10.1109/TAC.1972.1099975
   Web of Science ®Google Scholar
• Prokhorov, D. V., Santiago, R. A., & Ii, D. C. W. (1995). Adaptive critic designs: A case study for neurocontrol. Neural Networks: The Official Journal of the International Neural Network Society, 8(9), 1367–1372. https://doi.org/10.1016/0893-6080(95)00042-9
   Web of Science ®Google Scholar
• Richard, J. P. (2003). Time-delay systems: An overview of some recent advances and open problems. Automatica, 39(10), 1667–1694. https://doi.org/10.1016/S0005-1098(03)00167-5
   Web of Science ®Google Scholar
• Rosenblueth, J. F. (1992). Strongly and weakly relaxed controls for time delay systems. SIAM Journal on Control and Optimization, 30(4), 856–866. https://doi.org/10.1137/0330047
   Web of Science ®Google Scholar
• Su, H., & Tang, G. Y. (2016). Observer-based approximate optimal tracking control for time-delay systems with external disturbances. International Journal of Systems Science, 47(12), 2837–2846. https://doi.org/10.1080/00207721.2015.1034300
   Web of Science ®Google Scholar
• Sun, S., & Xiao, W. (2013). Optimal linear estimators for systems with multiple random measurement delays and packet dropouts. International Journal of Systems Science, 44(2), 358–370. https://doi.org/10.1080/00207721.2011.601347
   Web of Science ®Google Scholar
• Uchida, K., Shimemura, E., Kubo, T., & Abe, N. (1988). The linear-quadratic optimal control approach to feedback control design for systems with delay. Automatica, 24(6), 773–780. https://doi.org/10.1016/0005-1098(88)90053-2
   Web of Science ®Google Scholar
• Valeria, S., Szyld, D. B., & Marlliny, M. (1972). On two numerical methods for the solution of large-scale algebraic Riccati equations. IMA Journal of Numerical Analysis, 34(3), 904–920. https://doi.org/10.1093/imanum/drt015
   Web of Science ®Google Scholar
• Wang, F. Y., Jin, N., Liu, D., & Wei, Q. (2011). Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound. IEEE Transactions on Neural Networks, 22(1), 24–36. https://doi.org/10.1109/TNN.2010.2076370
   PubMed Web of Science ®Google Scholar
• Werbos, P. (1991). A menu of designs for reinforcement learning over time. Cambridge, MA: MIT Press.
   Google Scholar
• Yan, G., Fang, M., & Xu, J. (2019). Analysis and experiment of time-delayed optimal control for vehicle suspension system. Journal of Sound and Vibration, 446(4), 144–158. https://doi.org/10.1016/j.jsv.2019.01.015
   Web of Science ®Google Scholar
• Yang, L., Luo, Y., & Zhang, H. (2014). Adaptive dynamic programming for discrete-time LQR optimal tracking control problems with unknown dynamics. 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, Orlando, FL, USA: IEEE.
   Google Scholar
• Yin, Y. Z., Yang, Z. L., Yin, Z. X., & Xu, F. (2018). Optimal control of LQR for discrete time-varying systems with input delays. International Journal of Systems Science, 49(5), 1021–1031. https://doi.org/10.1080/00207721.2018.1438535
   Web of Science ®Google Scholar
• Yuan, C., & Wu, F. (2017). Consensus for multi-agent systems with time-varying input delays. International Journal of Systems Science, 48(14), 2956–2966.
   Web of Science ®Google Scholar
• Zhang, H., Liu, Y., Xiao, G., & Jiang, H. (2020). Data-based adaptive dynamic programming for a class of discrete-time systems with multiple delays. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 50(2), 432–441. https://doi.org/10.1109/TSMC.6221021
   Web of Science ®Google Scholar
• Zhang, H., Wei, Q., & Luo, Y. (2008). A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm. IEEE Transactions on Systems Man & Cybernetics Part B, 38(4), 937–942. https://doi.org/10.1109/TSMCB.2008.920269
   PubMed Web of Science ®Google Scholar
• Zhang, J., Zhang, H., Luo, Y., & Feng, T. (2014). Model-free optimal control design for a class of linear discrete-time systems with multiple delays using adaptive dynamic programming. Neurocomputing, 135(6), 163–170. https://doi.org/10.1016/j.neucom.2013.12.038
   Web of Science ®Google Scholar
• Zhang, J., Zhang, H., Wang, B., & Cai, T. (2016). Nearly data-based optimal control for linear discrete model-free systems with delays via reinforcement learning. International Journal of Systems Science, 47(5–8), 1563–1573. https://doi.org/10.1080/00207721.2014.941147
   Web of Science ®Google Scholar