Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints

Abstract

In this paper, an adaptive reinforcement learning-based solution is developed for the infinite-horizon optimal control problem of constrained-input continuous-time nonlinear systems in the presence of nonlinearities with unknown structures. Two different types of neural networks (NNs) are employed to approximate the Hamilton–Jacobi–Bellman equation. That is, an recurrent NN is constructed to identify the unknown dynamical system, and two feedforward NNs are used as the actor and the critic to approximate the optimal control and the optimal cost, respectively. Based on this framework, the action NN and the critic NN are tuned simultaneously, without the requirement for the knowledge of system drift dynamics. Moreover, by using Lyapunov’s direct method, the weights of the action NN and the critic NN are guaranteed to be uniformly ultimately bounded, while keeping the closed-loop system stable. To demonstrate the effectiveness of the present approach, simulation results are illustrated.

Keywords:

• adaptive control
• input constraints
• neural networks
• optimal control
• reinforcement learning

View correction statement:

Erratum

Additional information

Funding

This work was supported in part by the National Natural Science Foundation of China [grant number 61034002], [grant number 61233001], [grant number 61273140].