Nonparametric Regression With Basis Selection From Multiple Libraries

Abstract

New nonparametric regression procedures called BSML (Basis Selection from Multiple Libraries) are proposed in this article for estimating a complex function by a linear combination of basis functions adaptively selected from multiple libraries. Different classes of basis functions are chosen to model various features of the function, for example, truncated constants can model change points in the function, while polynomial spline representers may be used to model smooth components. The generalized cross-validation (GCV) and covariance inflation criteria are used to balance goodness of fit and model complexity where the model complexity is estimated adaptively by either the generalized degrees of freedom or covariance penalty. The cross-validation (CV) method is also considered for model selection. Spatially adaptive regression and model selection in multivariate nonparametric regression will be used to illustrate the flexibility and efficiency of the BSML procedures. Extensive simulations show that the BSML procedures are more adaptive than some well-known existing nonparametric regression methods. Analyses of real datasets are used to illustrate the BSML procedures. This article has supplementary materials online.

KEY WORDS:

• Covariance inflation criterion
• Covariance penalty
• Generalized cross-validation
• Generalized degrees of freedom
• Overcomplete bases
• Smoothing spline ANOVA
• Spatial adaptivity

ACKNOWLEDGMENTS

Yuedong Wang’s research was supported by a grant from the National Science Foundation (DMS-0706886).