A brief survey on nonlinear control using adaptive dynamic programming under engineering-oriented complexities

ABSTRACT

Nonlinear dynamics is frequently encountered in practical applications. Adaptive dynamic programming (ADP), which is implemented via actor/critic neural networks with excellent approximation capabilities, is appropriate to be used in finding the solution for the control problem in the presence of known/unknown nonlinear dynamics. The objective of this paper is to introduce state-of-the-art ADP-based algorithms and survey the recent advances in the ADP-based control strategies for nonlinear systems with various engineering-oriented complexities. Firstly, the main motivation of the ADP-based algorithms is thoroughly discussed, and the way of implementing the ADP-based algorithms is highlighted. Then, the latest research results concerning ADP-based control policy design for nonlinear systems are reviewed in detail, Finally, we conclude the survey by outlining the challenges and possible research topics in the future.

KEYWORDS:

• Nonlinear dynamics
• adaptive dynamic programming
• nonlinear control
• neural networks
• engineering-oriented complexities

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Date sharing is not applicable to this article as no new data were created or analysed in this study.

Additional information

Funding

This work was supported in part by the National Natural Science Foundation of China [grant number 62273087], the Shanghai Pujiang Program of China [grant number 22PJ1400400] and the Natural Science Foundation of Heilongjiang Province of China [grant number ZD2022F003].

Notes on contributors

Yuhan Zhang

Yuhan Zhang received the B.Eng. degree in electronic information science and technology from the Shandong University of Science and Technology, Qingdao, China, in 2016, and the M.Sc. degree in marketing from the University of Nottingham, Nottingham, UK, in 2017. She is currently pursuing the Ph.D. degree in control science and engineering from Shandong University of Science and Technology, Qingdao, China. Yuhan Zhang is a very active reviewer for many international journals.

Lei Zou

Lei Zou received the Ph.D. degree in control science and engineering in 2016 from Harbin Institute of Technology, Harbin, China. He is currently a Professor with the College of Information Science and Technology, Donghua University, Shanghai, China. From October 2013 to October 2015, he was a visiting Ph.D. student with the Department of Computer Science, Brunel University London, Uxbridge, U.K. His research interests include control and filtering of networked systems, moving-horizon estimation, state estimation subject to outliers, and secure state estimation. Professor Zou serves (or has served) as an Associate Editor for IEEE/CAA Journal of Automatica Sinica, Neurocomputing, International Journal of Systems Science, and International Journal of Control, Automation and Systems, a Senior Member of IEEE, a Member of Chinese Association of Automation, a Regular Reviewer of Mathematical Reviews, and a very active reviewer for many international journals.

Yang Liu

Yang Liu received the B.Sc. degree and the Ph.D. degree in the Department of Automation at Tsinghua University, Beijing, China, in 2010 and 2016, respectively.Since 2021, he has been working as a Research Associate in the Department of Aeronautical and Automotive Engineering, Loughborough University, U.K. His research interests include system security, complex networks as well as stochastic systems. He is a Member of IEEE and a Member of Chinese Association of Automation; and a very active reviewer for many international journals.

Derui Ding

Derui Ding received both the B.Sc. degree in Industry Engineering in 2004 and the M.Sc. degree in Detection Technology and Automation Equipment in 2007 from Anhui Polytechnic University, Wuhu, China, and the Ph.D. degree in Control Theory and Control Engineering in 2014 from Donghua University, Shanghai, China. He is currently a Senior Research Fellow with the School of Science, Computing and Engineering Technologies, Swinburne University of Technology, Melbourne, VIC, Australia. His research interests include nonlinear stochastic control and filtering, as well as distributed control, filtering and optimization. He received the 2021 Nobert Wiener Review Award from IEEE/CAA Journal of Automatica Sinica, the 2020 and 2022 Andrew P. Sage Best Transactions Paper Awards from the IEEE Systems, Man, and Cybernetics (SMC) Society, and the IET Premium Awards 2018. He is a Senior Member of the Institute of Electrical and Electronic Engineers (IEEE). He is serving as an Associate Editor for IEEE Transactions on Industrial Informatics, IEEE/CAA Journal of Automatica Sinica, Neurocomputing and IET Control Theory & Applications.

Jun Hu

Jun Hu received the B.Sc. degree in information and computation science and M.Sc. degree in applied mathematics from Harbin University of Science and Technology, Harbin, China, in 2006 and 2009, respectively, and the Ph.D. degree in control science and engineering from Harbin Institute of Technology, Harbin, China, in 2013. From September 2010 to September 2012, he was a Visiting Ph.D. Student in the Department of Information Systems and Computing, Brunel University, U.K. From May 2014 to April 2016, he was an Alexander von Humboldt research fellow at the University of Kaiserslautern, Kaiserslautern, Germany. From January 2018 to January 2021, he was a research fellow at the University of South Wales, Pontypridd, U.K. He is currently Professor in the Department of Mathematics, Harbin University of Science and Technology, Harbin 150080, China. His research interests include nonlinear control, filtering and fault estimation, time-varying systems and complex networks. He has published more than 80 papers in refereed international journals. Dr. Hu serves as a reviewer for Mathematical Reviews, as an editor for Neurocomputing, Journal of Intelligent and Fuzzy Systems, Neural Processing Letters, Systems Science and Control Engineering, and as a guest editor for International Journal of General Systems and Information Fusion.