Locally regularised two-stage learning algorithm for RBF network centre selection

Abstract

Nonlinear system models constructed from radial basis function (RBF) networks can easily be over-fitted due to the noise on the data. While information criteria, such as the final prediction error (FPE), can provide a trade-off between training error and network complexity, the tunable parameters that penalise a large size of network model are hard to determine and are usually application dependent. This article introduces a new locally regularised, two-stage stepwise construction algorithm for RBF networks. The main objective is to produce a parsimonious network that generalises well over unseen data. This is achieved by utilising Bayesian learning within a two-stage stepwise construction procedure to penalise centres that are mainly interpreted by the noise. Specifically, each output layer weight is assigned a hyperparameter, a large value of such a parameter forcing the associated output layer weight to be near to zero. Sparsity is achieved by removing irrelevant RBF centres from the network. The efficacy of proposed algorithm from the original two-stage construction method is retained. Numerical analysis shows that this new method only needs about half of the computation involved in the locally regularised orthogonal least squares (LROLS) alternative. Results from two simulation examples are presented to show that the nonlinear system models resulting from this new approach are superior in terms of both sparsity and generalisation capability.

Keywords:

• Bayesian learning
• local regularisation
• two-stage stepwise regression
• linear-in-the-parameters
• RBF

Acknowledgements

J. Deng wishes to thank Queens University Belfast for the award of an ORS scholarship to support his doctoral studies.