Model-free adaptive control design for nonlinear discrete-time processes with reinforcement learning techniques

ABSTRACT

This paper deals with the model-free adaptive control (MFAC) based on the reinforcement learning (RL) strategy for a family of discrete-time nonlinear processes. The controller is constructed based on the approximation ability of neural network architecture, a new actor-critic algorithm for neural network control problem is developed to estimate the strategic utility function and the performance index function. More specifically, the novel RL-based MFAC scheme is reasonable to design the controller without need to estimate y(k+1) information. Furthermore, based on Lyapunov stability analysis method, the closed-loop systems can be ensured uniformly ultimately bounded. Simulations are shown to validate the theoretical results.

KEYWORDS:

• Model-free adaptive control
• reinforcement learning
• discrete-time nonlinear systems
• data-driven control

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported in part by the Funds of the National Natural Science Foundation of China [grant numbers 61621004 and 61420106016], the Fundamental Research Funds for the Central Universities [grant number N130604005], and the Research Fund of State Key Laboratory of Synthetical Automation for Process Industries [grant number 2013ZCX03].

Notes on contributors

Dong Liu

Dong Liu received the B.S. degree in mathematics and applied mathematics and M.S. degree in operational research and cybernetics from Shenyang Normal University, China, in 2009 and 2012, respectively. He is currently pursuing the Ph.D. degree at the College of Information Science and Engineering, Northeastern University. His current research interests include data-driven control, model-free adaptive control, event-triggered control and reinforcement learning control.

Guang-Hong Yang

Guang-Hong Yang received the B.S. and M.S. degrees in mathematics, and Ph.D. degree in control theory and control engineering from Northeast University, Shenyang, China, in 1983, 1986, and 1994, respectively. From 2001 to 2005, he was aResearch Scientist/Senior Research Scientist with the National University of Singapore, Singapore. He is currently a professor and the dean with the College of Information Science and Engineering, Northeastern University. His current research interests include fault-tolerant control, fault detection and isolation, cyber physical systems, and robust control. Dr. Yang is an Associate Editor for the International Journal of Control, Automation and Systems, the International Journal of Systems Science, the IET Control Theory and Applications and the IEEE Transactions on Fuzzy Systems.