Robustness improvement of optimal control in terms of RBFNN with empirical model reduction and transfer learning

Figures & data

Figure 1. Flowchart of the RBFNN optimal control algorithm with model reduction and transfer learning for linear systems.

Figure 2. Quanser-Servo2 Inverted Pendulum system hardware setup (Quanser, 2022).

Table 1. Parameters of the rotary pendulum system.

Symbol	Description	Value
$m_r$	Mass of rotary arm	0.095 kg
r	Length of rotary arm	0.085 m
$J_r$	Moment inertial of rotary arm	$2.29\times {10}^{-4}\mathrm{kg}\cdot {\mathrm{m}}^2$
$b_r$	Viscous damping of rotary arm	${10}^{-3}\mathrm{N}\cdot \mathrm{m}\cdot \mathrm{s}/\mathrm{rad}$
$m_p$	Mass of pendulum	0.024 kg
$L_p$	Length of pendulum	0.129 m
l	Half length of pendulum	0.0645 m
$J_p$	Moment inertial of pendulum	$1.33\times {10}^{-4}\mathrm{kg}\cdot {\mathrm{m}}^2$
$b_p$	Viscous damping coefficient	$5\times {10}^{-5}\mathrm{N}\cdot \mathrm{m}\cdot \mathrm{s}/\mathrm{rad}$
g	Gravitational acceleration	9.81 $\mathrm{m}\cdot {\mathrm{s}}^2$

Table 2. Summary of reduced order model matrices from model-based gramians and empirical gramians.

Model-based gramians	Empirical gramians
$A_r=\left[\begin{array}{ccc}0& -2.0421& 0\\ 0& 13.7774& 0\\ 0& 0& -5.4132\end{array}\right]$	$A_r=\left[\begin{array}{ccc}13.7791& 0.2460& -0.0002\\ -0.2614& -21.9083& 0.0014\\ -0.0011& -0.0389& 0.0842\end{array}\right]$
$B_r=\left[\begin{array}{c}1.1392\\ -0.7163\\ -1.9942\end{array}\right]$	$B_r=\left[\begin{array}{c}-1.0746\\ 1.5235\\ -1.9277\end{array}\right]$
$C_r=\left[\begin{array}{ccc}4.00& 0& 1.9936\\ 0& -1.4653& -0.0477\end{array}\right]$	$C_r=\left[\begin{array}{ccc}-0.4058& -0.9037& -1.6948\\ -0.9951& -1.1802& -0.0585\end{array}\right]$

Table 3. Hankel singular values of the rotary pendulum.

State	Hankel singular value ${\omega }_i$
1	∞
2	∞
3	0.3673
4	0.0304

Figure 3. Top: Relative output errors of the reduced order model by the balanced truncation and empirical balanced truncation. Below: The input signal.

Figure 4. Tracking the square wave of the rotary army in simulations. In the legend, ‘LQR’ denotes the LQR control designed with the original model; ‘RBFNN’ denotes the RBFNN control designed with the model-based BT; ‘Empirical RBFNN’ denotes the RBFNN control designed with the empirical BT.

Figure 5. Comparisons of tracking response $\theta (t)$ of the rotary army in simulations. In the legend, ‘RBFNN_Empirical’ denotes the RBFNN control designed with the empirical BT; ‘LQR_Empirical BT’ denotes the LQR control designed with the empirical BT; ‘LQR_Model-based BT’ denotes the LQR control designed with the model-based BT.

Figure 6. Comparisons of tracking response $\alpha (t)$ of the rotary army in simulations. Legends are the same as in Figure 5.

Figure 7. The closed-loop tracking responses $\theta (t)$ of the rotary arm of Quanser-Servo2. Legends are the same as in Figure 4.

Figure 8. The closed-loop responses $\theta (t)$ of the rotary arm under various controls for balancing the inverted pendulum of Quanser-Servo2. Top: Responses before retraining. Bottom: Responses after retraining. Legends are the same as in Figure 4.

Figure 9. The closed-loop tracking response $\theta (t)$ of the rotary arm of Quanser-Servo2. Legends are the same as in Figure 4.

Figure 10. The closed-loop response $\alpha (t)$ of the pendulum in the rotary arm tracking control of Quanser-Servo2. Legends are the same as in Figure 4.

Table 4. Summary of control performance for LQR, RBFNN, empirical RBFNN, retrained RBFNN and retrained empirical RBFNN.

Case	Algorithm	$e_{\mathrm{RMS}}$	$u_{\mathrm{effort}}$
	LQR	4.54	4667.64
	RBFNN	2.31	4080.89
Stabilization	Retrained RBFNN	1.83	2769.3
	Empirical RBFNN	2.86	4343.8
	Retrained Empirical	0.57	3210.1
	LQR	4.67	7410.1
Trajectory	RBFNN	4.08	7468.4
Tracking	Retrained RBFNN	2.77	6667.5
	Empirical RBFNN	4.34	6794.8
	Retrained Empirical	3.21	8380.3

Figure 11. Robustness comparisons of all the controls under consideration. Top: The closed-loop angle response $\theta (t)$ of the rotary arm in balancing control of Quanser-Servo2. Bottom: Disturbance $d(t)$. Legends are the same as in Figure 4.

Figure 12. Disturbances to the second order nonlinear system.

Figure 13. Performance comparison of RBFNN, Poly-NN control and LQR controls for the nonlinear system in Equation (61(61) $\begin{array}{r}\begin{array}{rl}{\dot{x}}_1& =x_1+x_2-x_1(x_1^2+x_2^2)\\ {\dot{x}}_2& =-x_1+x_2-x_2(x_1^2+x_2^2)+u\end{array}\end{array}$(61) ). Top: The control $u(t)$. Middle: The response $x_1(t)$. Bottom: The response $x_2(t)$.

Figure 14. Comparison of spatial distribution of RBFNN and Poly-NN optimal controls u as a function of the state $\mathbf{x}$. Left: The control $u(\mathbf{x})$ plotted in the training region $X_{s1}\in [-1,1]\times [-1,1]$. Right: The control $u(\mathbf{x})$ plotted beyond the training region into the larger region $X_{s2}\in [-2,2]\times [-2,2]$.

Figure 15. Robustness of the RBFNN and LQR controls with respect to the model uncertainty β. The vertical dash lines mark the critical value of β, beyond which the closed-loop system becomes unstable.

Figure A1. Comparison of RBFNNs and LQR control performances for the linear 2D system. Top: Control $u(t)$. Middle: Response $x_1(t)$. Bottom: Response $x_2(t)$. The initial condition of the system is $\mathbf{x}(0)=[1,1{]}^T$.

Quanser (2022). QUBE-Servo2. https://www.quanser.com/products/qube-servo-2/

 Google Scholar