An online self-adaptive RBF network algorithm based on the Levenberg-Marquardt algorithm

Figures & data

Figure 1. Structure of RBF networks.

Table 1. Algorithmic depiction of OSA-RBFNN.

Initialization set the length of the sliding window L there is one hidden node in the network for k = 1:NMAX move new samples $\left(x_k,y_k\right)$ to the sliding windows, then remove the first sample calculate $e_{rmse}$ of the sample in the current sliding window using Equation (15)(15) $e_{rmse}=\sqrt{\frac{\sum _{i=1}^L{\beta }_i{\left(y_i-o_i\right)}^2}{L}.}$(15) (step=1) for step = 2:max_step find maximum of abs error of $e_{rmse}$ if the fixed training steps is reached and $e_{rmse}>e_{obj}$ ($e_{obj}$ is the target value of the network training error) create a new hidden node at the location of maximum error by setting weight the width of the new RBF node to 1. calculate Jacobian vector $j_p$ using Equations (9)(9) $j_p=\left[\begin{array}{c}\frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{\omega }_0},\frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{\omega }_1}\dots \frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{\omega }_h}\dots \frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{\omega }_H},\frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{c}_{1,1}}\dots \frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{c}_{1,i}}\dots \frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{c}_{1,I}}\dots \frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{c}_{h,1}}\dots \\ \frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{c}_{h,i}}\dots \frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{c}_{H,1}}\dots \frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{c}_{H,i}}\dots \frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{c}_{H,I}},\frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{\sigma }_1}\dots \frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{\sigma }_h}\dots \frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{\sigma }_H}\end{array}\right].$(9) ~(12(12) $\frac{\mathrm{\partial }{e}_p}{\mathrm{\partial }{\sigma }_h}=-\frac{{\omega }_h{\phi }_h\left(X_p\right)\parallel X_p-c_h{\parallel }^2}{{\sigma }_h^2}.$(12) ) calculate sub quasi-Hessian matrix $q_p=j_p^T{j}_p$, and gradient vector ${\eta }_p=j_p^T{e}_p$ if check the hidden nodes and the node cannot activate prune the hidden node end if check redundant hidden nodes in the RBF network, and the Euclidean distance between the center of two hidden nodes is less than the threshold. merge the two hidden nodes using Equation (17)(17) $\left\{\begin{array}{c}{w}_i=w_i+w_j,\\ c_i=\left(c_i+c_j\right)/2,\\ {\sigma }_i=max\left({\sigma }_i,{\sigma }_j\right).\end{array}\right.$(17) end end calculate quasi-Hessian matrix Q using Equation (8)(8) $Q=\sum _{p=1}^P{q}_p,q_p=j_p^T{j}_p$(8) ;Using Equations (6)(6) $\mathrm{g}=\sum _{p=1}^P{\eta }_p$(6) and (7(7) ${\eta }_p=j_p^T{e}_p$(7) ) to calculate the gradient vector g. update RBF network parameters using Equation (3)(3) ${\mathrm{\Delta }}_{k+1}={\mathrm{\Delta }}_k-{\left(Q_k+{\mu }_{k}I\right)}^{-1}{g}_k$(3) ; calculate new output using Equation (16)(16) ${\varphi }_h^k\left(x_k\right)=w_h^{k}exp\left(-\frac{\parallel x_k-c_h{\parallel }^2}{{\sigma }_h}\right),r_h^k=\parallel \frac{{\varphi }_h^k}{{\varphi }_{max}^k}\parallel .$(16) , and sign the state of the node. calculate RMSE(step) while $e_{rmse}$ of the sample in the current sliding window is not reduced adjust the ${\mu }_k$ parameter using Equation (3)(3) ${\mathrm{\Delta }}_{k+1}={\mathrm{\Delta }}_k-{\left(Q_k+{\mu }_{k}I\right)}^{-1}{g}_k$(3) endwhile ($e_{rmse}$ is not reduced) if RMSE(step)<desired error, then break endfor(max_step) if RMSE(k)<desired error, then break(new RBF node loop) endfor(main loop)

Figure 2. The attribution of the true and estimated centers.

Table 2. Performance comparison of true and estimated value on object function.

True center	(0.3,0.2)	(0.7,0.2)	(0.1,0.5)	(0.9,0.5)	(0.3,0.8)	(0.7,0.8)
Estimated center	(0.3110, 0.2000)	(0.6798, 0.1993)	(0.0988, 0.5000)	(0.8799, 0.5103)	(0.3000, 0.7994)	(0.6981, 0.8022)
True width	0.1	0.1	0.144	0.144	0.1	0.1
Estimated width	0.0998	0.0999	0.1425	0.1496	0.0992	0.0995
True weight	1	1	1	1	1	1
Estimated weight	1.0299	1.0073	1.0200	0.9887	1.0021	1.0003

Figure 3. Chaotic behavior of x(t) and x(t + 50).

Figure 4. The performance of online learning.

Figure 5. The performance of RMSE.

Figure 6. The change in hidden nodes.

Figure 7. The testing error of 500 samples.

Table 3. The performance of comparison of different algorithms.

Algorithms	Number of Hidden Nodes	Training RMSE	Testing Error
eTS	99	-^a	-^a
RAN	51	0.0316	0.0427
MRAN	47	0.0274	0.0319
OSAMNN	35	0.0319	-^a
OSA-RBFNN	39	0.0057	0.0072

Note: “^a”Results not listed in the original paper.