A model-averaging approach for smoothing spline regression

ABSTRACT

This article considers nonparametric regression problems and develops a model-averaging procedure for smoothing spline regression problems. Unlike most smoothing parameter selection studies determining an optimum smoothing parameter, our focus here is on the prediction accuracy for the true conditional mean of Y given a predictor X. Our method consists of two steps. The first step is to construct a class of smoothing spline regression models based on nonparametric bootstrap samples, each with an appropriate smoothing parameter. The second step is to average bootstrap smoothing spline estimates of different smoothness to form a final improved estimate. To minimize the prediction error, we estimate the model weights using a delete-one-out cross-validation procedure. A simulation study has been performed by using a program written in R. The simulation study provides a comparison of the most well known cross-validation (CV), generalized cross-validation (GCV), and the proposed method. This new method is straightforward to implement, and gives reliable performances in simulations.

KEYWORDS:

• Bagging
• Cross validation
• Nonparametric regression
• Smoothing parameter
• Smoothing spline

MATHEMATICS SUBJECT CLASSIFICATION:

• Primary 62E17
• Secondary 62C05

Additional information

Funding

This work was supported in part by the China Postdoctoral Science Foundation (2016M600157), the National Natural Science Foundation of China (11171002), and Beijing Natural Science Foundation.