58
Views
0
CrossRef citations to date
0
Altmetric
Research Article

Infinite-time robust optimal output tracking of continuous-time linear systems using undiscounted reinforcement learning

&
Received 16 Jan 2024, Accepted 29 May 2024, Published online: 18 Jun 2024

References

  • Amirparast, A., & Hosseini Sani, S. K. (2023). A robust optimal distributed control design for simultaneous voltage regulation and current sharing in dc microgrid. IET Smart Grid, 6(6), 653–665. https://doi.org/10.1049/stg2.v6.6
  • Amirparast, A., & Hosseini Sani, S. K. (2023). Undiscounted reinforcement learning for infinite-time optimal output tracking and disturbance rejection of discrete-time LTI systems with unknown dynamics. International Journal of Systems Science, 54(10), 2175–2195. https://doi.org/10.1080/00207721.2023.2221240
  • Amirparast, A., & Hosseini Sani, S. K. (2024). Off-policy reinforcement learning algorithm for robust optimal control of uncertain nonlinear systems. International Journal of Robust and Nonlinear Control, 34(8), 5419–5437. https://doi.org/10.1002/rnc.v34.8
  • Anderson, B. D. O., Moore, J. B., & Naidu, D. S. (1995). Optimal control: Linear quadratic methods (Vol. 40). Institute of Electrical and Electronics Engineers.
  • Artstein, Z., & Leizarowitz, A. (1985). Tracking periodic signals with the overtaking criterion. IEEE Transactions on Automatic Control, 30(11), 1123–1126. https://doi.org/10.1109/TAC.1985.1103851
  • Bertsekas, D. (2019). Reinforcement learning and optimal control. Athena Scientific.
  • Chen, C., Xie, L., Xie, K., Lewis, F. L., & Xie, S. (2022). Adaptive optimal output tracking of continuous-time systems via output-feedback-based reinforcement learning. Automatica, 146, 110581. https://doi.org/10.1016/j.automatica.2022.110581
  • Gale, D. (1967). On optimal development in a multi-sector economy. The Review of Economic Studies, 34(1), 1–18. https://doi.org/10.2307/2296567
  • Gao, W., & Jiang, Z.-P. (2016). Adaptive dynamic programming and adaptive optimal output regulation of linear systems. IEEE Transactions on Automatic Control, 61(12), 4164–4169. https://doi.org/10.1109/TAC.2016.2548662
  • Haddad, W. M., & Bernstein, D. S. (1988). Optimal output feedback for non-zero set point regulation: The discrete-time case. International Journal of Control, 47(2), 529–536. https://doi.org/10.1080/00207178808906029
  • Jiang, Y., & Jiang, Z.-P. (2014). Robust adaptive dynamic programming and feedback stabilization of nonlinear systems. IEEE Transactions on Neural Networks and Learning Systems, 25(5), 882–893. https://doi.org/10.1109/TNNLS.5962385
  • Kamalapurkar, R., Walters, P., Rosenfeld, J., & Dixon, W. (2018). Reinforcement learning for optimal feedback control. Springer.
  • Karimi-Ghartemani, M., Khajehoddin, S. A., Jain, P., & Bakhshai, A. (2011). Linear quadratic output tracking and disturbance rejection. International Journal of Control, 84(8), 1442–1449. https://doi.org/10.1080/00207179.2011.605908
  • Kirk, D. E. (2004). Optimal control theory: An introduction. Courier Corporation.
  • Kiumarsi, B., Lewis, F. L., & Jiang, Z.-P. (2017). H∞ control of linear discrete-time systems: Off-policy reinforcement learning. Automatica, 78, 144–152. https://doi.org/10.1016/j.automatica.2016.12.009
  • Kiumarsi, B., Lewis, F. L., Modares, H., Karimpour, A., & Naghibi-Sistani, M.-B. (2014). Reinforcement q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics. Automatica, 50(4), 1167–1175. https://doi.org/10.1016/j.automatica.2014.02.015
  • Kleinman, D. (1968). On an iterative technique for riccati equation computations. IEEE Transactions on Automatic Control, 13(1), 114–115. https://doi.org/10.1109/TAC.1968.1098829
  • Lee, J. Y., Park, J. B., & Choi, Y. H. (2014). Integral reinforcement learning for continuous-time input-affine nonlinear systems with simultaneous invariant explorations. IEEE Transactions on Neural Networks and Learning Systems, 26(5), 916–932.
  • Lewis, F. L., & Liu, D. (2013). Reinforcement learning and approximate dynamic programming for feedback control. John Wiley & Sons.
  • Lewis, F. L., & Vrabie, D. (2009). Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits and Systems Magazine, 9(3), 32–50. https://doi.org/10.1109/MCAS.7384
  • Lewis, F. L., Vrabie, D., & Syrmos, V. L. (2012). Optimal control. John Wiley & Sons.
  • Lewis, F. L., Vrabie, D., & Vamvoudakis, K. G. (2012). Reinforcement learning and feedback control: Using natural decision methods to design optimal adaptive controllers. IEEE Control Systems Magazine, 32(6), 76–105. https://doi.org/10.1109/MCS.2012.2214134
  • Lin, F. (2007). Robust control design: An optimal control approach. John Wiley & Sons.
  • Lin, F., & Brandt, R. D. (1998). An optimal control approach to robust control of robot manipulators. IEEE Transactions on Robotics and Automation, 14(1), 69–77. https://doi.org/10.1109/70.660845
  • Liu, D., Yang, X., Wang, D., & Wei, Q. (2015). Reinforcement-learning-based robust controller design for continuous-time uncertain nonlinear systems subject to input constraints. IEEE Transactions on Cybernetics, 45(7), 1372–1385. https://doi.org/10.1109/TCYB.2015.2417170
  • Luo, B., Wu, H.-N., & Huang, T. (2014). Off-policy reinforcement learning for H∞ control design. IEEE Transactions on Cybernetics, 45(1), 65–76. https://doi.org/10.1109/TCYB.2014.2319577
  • Luo, B., Wu, H.-N., & Li, H.-X. (2014). Adaptive optimal control of highly dissipative nonlinear spatially distributed processes with neuro-dynamic programming. IEEE Transactions on Neural Networks and Learning Systems, 26(4), 684–696.
  • Modares, H., & Lewis, F. L. (2014). Linear quadratic tracking control of partially-unknown continuous-time systems using reinforcement learning. IEEE Transactions on Automatic Control, 59(11), 3051–3056. https://doi.org/10.1109/TAC.9
  • Modares, H., Lewis, F. L., & Jiang, Z.-P. (2015). H∞ tracking control of completely unknown continuous-time systems via off-policy reinforcement learning. IEEE Transactions on Neural Networks and Learning Systems, 26(10), 2550–2562. https://doi.org/10.1109/TNNLS.2015.2441749
  • Modares, H., Lewis, F. L., & Jiang, Z.-P. (2016). Optimal output-feedback control of unknown continuous-time linear systems using off-policy reinforcement learning. IEEE Transactions on Cybernetics, 46(11), 2401–2410. https://doi.org/10.1109/TCYB.2015.2477810
  • Perrusquía, A., & Yu, W. (2020a). Discrete-time H2 neural control using reinforcement learning. IEEE Transactions on Neural Networks and Learning Systems, 32(11), 4879–4889. https://doi.org/10.1109/TNNLS.2020.3026010
  • Perrusquía, A., & Yu, W. (2020b). Neural H2 control using continuous-time reinforcement learning. IEEE Transactions on Cybernetics, 52(6), 4485–4494. https://doi.org/10.1109/TCYB.2020.3028988
  • Rizvi, S. A. A., & Lin, Z. (2018). Output feedback q-learning for discrete-time linear zero-sum games with application to the h-infinity control. Automatica, 95, 213–221. https://doi.org/10.1016/j.automatica.2018.05.027
  • Vamvoudakis, K. G., Wan, Y., Lewis, F. L., & Cansever, D. (2021). Handbook of reinforcement learning and control. Springer.
  • Vrabie, D., Pastravanu, O., Abu-Khalaf, M., & Lewis, F. L. (2009). Adaptive optimal control for continuous-time linear systems based on policy iteration. Automatica, 45(2), 477–484. https://doi.org/10.1016/j.automatica.2008.08.017
  • Wang, D. (2019). Robust policy learning control of nonlinear plants with case studies for a power system application. IEEE Transactions on Industrial Informatics, 16(3), 1733–1741. https://doi.org/10.1109/TII.9424
  • Wang, D., He, H., & Liu, D. (2017). Adaptive critic nonlinear robust control: A survey. IEEE Transactions on Cybernetics, 47(10), 3429–3451. https://doi.org/10.1109/TCYB.2017.2712188
  • Willems, J. L., & Mareels, I. M. Y. (2004). A rigorous solution of the infinite time interval lq problem with constant state tracking. Systems & Control Letters, 52(3-4), 289–296. https://doi.org/10.1016/j.sysconle.2004.01.001
  • Xiao, G., Zhang, H., Zhang, K., & Wen, Y. (2018). Value iteration based integral reinforcement learning approach for H∞ controller design of continuous-time nonlinear systems. Neurocomputing, 285, 51–59. https://doi.org/10.1016/j.neucom.2018.01.029
  • Zhao, J., & Lv, Y. (2022). Output-feedback robust control of systems with uncertain dynamics via data-driven policy learning. International Journal of Robust and Nonlinear Control, 32(18), 9791–9807. https://doi.org/10.1002/rnc.v32.18

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.