Online adaptive optimal control for continuous-time nonlinear systems with completely unknown dynamics

Abstract

An online adaptive optimal control is proposed for continuous-time nonlinear systems with completely unknown dynamics, which is achieved by developing a novel identifier-critic-based approximate dynamic programming algorithm with a dual neural network (NN) approximation structure. First, an adaptive NN identifier is designed to obviate the requirement of complete knowledge of system dynamics, and a critic NN is employed to approximate the optimal value function. Then, the optimal control law is computed based on the information from the identifier NN and the critic NN, so that the actor NN is not needed. In particular, a novel adaptive law design method with the parameter estimation error is proposed to online update the weights of both identifier NN and critic NN simultaneously, which converge to small neighbourhoods around their ideal values. The closed-loop system stability and the convergence to small vicinity around the optimal solution are all proved by means of the Lyapunov theory. The proposed adaptation algorithm is also improved to achieve finite-time convergence of the NN weights. Finally, simulation results are provided to exemplify the efficacy of the proposed methods.

Keywords:

• adaptive control
• optimal control
• approximate dynamic programming
• system identification
• nonlinear systems

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 61203066), the Open Research Project of the State Key Laboratory of Industrial Control Technology, Zhejiang University, China (No. ICT1548), and in part by the Marie Curie Intra-European Fellowships AECE Project under Grant FP7-PEOPLE-2013-IEF-625531.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

National Natural Science Foundation of China [grant number 61203066]; Open Research Project of the State Key Laboratory of Industrial Control Technology, Zhejiang University, China [grant number ICT1548]; Marie Curie Intra-European Fellowships: AECE Project [grant number FP7-PEOPLE-2013-IEF-625531].