Smoothing Spline Semiparametric Nonlinear Regression Models

Abstract

We consider the problem of modeling the mean function in regression. Often there is enough knowledge to model some components of the mean function parametrically. But for other vague and/or nuisance components, it is often desirable to leave them unspecified and to be modeled nonparametrically. In this article, we propose a general class of smoothing spline semiparametric nonlinear regression models (SNRMs) which assumes that the mean function depends on parameters and nonparametric functions through a known nonlinear functional. SNRMs are natural extensions of both parametric and nonparametric regression models. They include many popular nonparametric and semiparametric models such as the partial spline, varying coefficients, projection pursuit, single index, multiple index, and shape-invariant models as special cases. Building on reproducing kernel Hilbert spaces (RKHS), the SNRMs allow us to deal with many different situations in a unified fashion. We develop a unified estimation procedure based on minimizing penalized likelihood using Gauss–Newton and backfitting algorithms. Smoothing parameters are estimated using the generalized cross-validation (GCV) and generalized maximum likelihood (GML) methods. We derive Bayesian confidence intervals for the unknown functions. A generic and user-friendly R function is developed to implement our estimation and inference procedures. We illustrate our methods with analyses of two real datasets and evaluate finite-sample performance by simulations.

Datasets, computer code and derivation of posterior covariances are available in the online supplements.

Keywords:  :

• Backfitting
• Gauss–Newton algorithm
• Nonlinear functional
• Nonparametric regression
• Penalized likelihood
• Semiparametric models
• Smoothing parameter