Choosing a Range for the Amount of Smoothing in Nonparametric Regression

Abstract

Most nonparametric regression estimates smooth the observed data in the process of generating a suitable estimate. The resulting curve estimate is sensitive to the amount of smoothing introduced, and it is important to examine the estimate over a range of smoothing choices. An objective way to specify such a range is to estimate the optimal value of the smoothing parameter (bandwidth) with respect to some loss function and then report a confidence interval for this parameter estimate. This interval suggests examining three regression estimates: an undersmoothed curve evaluated at the lower endpoint of the confidence interval, an oversmoothed curve evaluated at the upper endpoint, and also an optimal curve chosen by the data-based method. This article describes two strategies for constructing such confidence intervals using asymptotic approximations and simulation techniques. Suppose that data are observed from the model Y_k = f(t_k ) + e_k (1 ≤ k ≤ n), where f is a function that is twice differentiable and {e_k } are mean-zero, independent, random variables. A cubic smoothing spline estimate of f is considered where the smoothing parameter is chosen using generalized cross-validation. Confidence intervals are constructed for the smoothing parameter that minimizes average squared error. This is done using the asymptotic distribution of the cross-validation function and by a version of the bootstrap. Although this bootstrap method involves more computation, it yields confidence intervals that tend to have a shorter width. In general, this second method is easier to implement since it avoids the problem of deriving the complicated formulas for the asymptotic variance. Moreover, for spline estimates one can significantly reduce the amount of computation through a useful orthogonal decomposition. Although this work focuses on smoothing spline estimates where the smoothing parameter is found by generalized cross-validation, the basic ideas in this article are not limited to this pair of methods. Regardless of how the smoothing parameter is selected, it is useful to consider under- and oversmoothed curve estimates to investigate the sensitivity. A confidence interval for the smoothing parameter remains a useful way for specifying this range.

Key Words:

• Bandwidth selection
• Bootstrap
• Confidence intervals
• Cross-validation
• Smoothing splines