Adaptive dynamic programming-based decentralised control for large-scale nonlinear systems subject to mismatched interconnections with unknown time-delay

Abstract

This paper addresses decentralised optimal control problems for large-scale nonlinear systems subject to mismatched interconnections with unknown time-delay by adaptive dynamic programming. To eliminate the effects of mismatched interconnections with unknown time-delay, a novel local value function is constructed to transform the decentralised control problem into an optimal control problem. A local robust observer is established to identify the bound of the unknown interconnections. Then, based on the observer-critic architecture, the decentralised optimal control policy is achieved by solving local Hamiltonian–Jacobi–Bellman equation via local policy iteration algorithm. The stability of the closed-loop large-scale nonlinear system is guaranteed to be uniformly ultimately bounded by implementing a set of decentralised control policies. Simulation examples demonstrate the effectiveness of the proposed scheme.

Keywords:

• Adaptive dynamic programming
• large-scale nonlinear systems
• decentralised control
• unknown mismatched interconnection
• unknown time-delay
• reinforcement learning

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported in part by the National Natural Science Foundation of China [grant number 61533017, [grant number 61973330], [grant number 61773075], [grant number 61603387], in part by the Guangdong Introducing Innovative and Enterpreneurial Teams of ‘The Pearl River Talent Recruitment Program’ [grant number 2019ZT08X340], in part by the Early Career Development Award of SKLMCCS [grant number 20180201], and in part by the State Key Laboratory of Synthetical Automation for Process Industries [grant number 2019-KF-23-03].

Notes on contributors

Qiuye Wu

Qiuye Wu received his B.S. degree in Automation from Beijing Institute of Technology at Zhuhai, Zhuhai, China, in 2017, and the M.S. degree in Control Science and Engineering from Guangdong University of Technology, Guangzhou, China, in 2020. He is currently pursuing the Ph.D. degree in Control Science and Engineering at the School of Automation, Guangdong University of Technology, Guangzhou, China. His research interests include optimal control, neural networks, reinforcement learning, and adaptive dynamic programming.

Bo Zhao

Bo Zhao received his B.S. degree in Automation, and Ph.D. degree in Control Science and Engineering, all from Jilin University, Changchun, China, in 2009 and 2014, respectively. He was a Post-Doctoral Fellow with the State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China, from 2014 to 2017. Then, he joined the State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China, from 2017 to 2018. He is currently an Associate Professor with the School of Systems Science, Beijing Normal University, Beijing, China. He has authored or co-authored over 80 journal and conference papers. His research interests include adaptive dynamic programming, robot control, fault diagnosis and tolerant control, optimal control, and artificial intelligence-based control. He has authored or co-authored over 80 journal and conference articles. He was the secretary of Adaptive Dynamic Programming and Reinforcement Learning Professional Committee of Chinese Automation Association (CAA), and the secretary of 2017 the 24th International Conference on Neural Information Processing. He is IEEE Member, Asian-Pacific Neural Network Society (APNNS) Member and CAA Member.

Derong Liu

Derong Liu received the Ph.D. degree in electrical engineering from the University of Notre Dame, Notre Dame, IN, USA, in 1994. He was a Staff Fellow with General Motors Research and Development Center, Warren, MI, USA, from 1993 to 1995. He was an Assistant Professor with the Department of Electrical and Computer Engineering, Stevens Institute of Technology, Hoboken, NJ, USA, from 1995 to 1999. He joined the University of Illinois at Chicago, Chicago, IL, USA, in 1999, and became a Full Professor of Electrical and Computer Engineering, and of Computer Science, in 2006. He served as the Associate Director of the State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Beijing, China, from 2010 to 2015. He is currently a Full Professor with the School of Automation, Guangdong University of Technology, Guangzhou, China. He has authored or co-authored 19 books. Prof. Liu was a recipient of the Michael J. Birck Fellowship from the University of Notre Dame, in 1990, the Harvey N. Davis Distinguished Teaching Award from the Stevens Institute of Technology, in 1997, the Faculty Early Career Development Award from the National Science Foundation in 1999, the University Scholar Award from the University of Illinois, from 2006 to 2009, and the Overseas Outstanding Young Scholar Award from the National Natural Science Foundation of China in 2008. He was selected for the ‘100 Talents Program’ by the Chinese Academy of Sciences in 2008. He was the Editor-in-Chief of the IEEE Transactions on Neural Networks and Learning Systems from 2010 to 2015. He is the Editor-in-Chief of Artificial Intelligence Review (Springer). He is a Fellow of the International Neural Network Society and the International Association of Pattern Recognition.