Optimal trajectory tracking of uncertain nonlinear continuous-time strict-feedback systems with dynamic constraints

Abstract

A novel method for optimal trajectory tracking of uncertain nonlinear systems by using adaptive dynamic programming (ADP)-based integral reinforcement learning (IRL) and neural networks (NNs) is proposed. The method utilises an actor-critic framework with optimal backstepping to minimise a discounted value function and uses a single-layer NN identifier for estimating the unknown dynamics. For safety assurance, a time-varying barrier Lyapunov function (TVBLF) is used in the control design to handle the constraints. A novel weight-tuning law by using the control input error, integral Bellman error, and the NN identifier is used for the actor-critic framework. A novel lifelong learning (LL)-based method for critic NN is utilised in an optimal framework by incorporating the Bellman error to mitigate catastrophic forgetting in multitasking scenarios. Stability analysis using Lyapunov stability theory is obtained for the overall closed-loop system. Simulations of leader-follower mobile robot formation control show a 25% reduction in the cost in multitasking scenarios.

Keywords:

• Optimal control
• strict feedback
• nonlinear systems
• reinforcement learning
• neural networks
• lifelong learning

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The project or effort undertaken was or is sponsored by the Office of Naval Research [grant numbers N00014-21-1-2232, N00014-23-1-2195, N00014-24-1-2338] and Army Research Office Cooperative Agreement W911NF-21-2-0260 and W911NF-22-2-0185, and Intelligent Systems Center at Missouri University of Science and Technology, Rolla, MO, USA.