Variable Selection in Kernel Regression Using Measurement Error Selection Likelihoods

ABSTRACT

This article develops a nonparametric shrinkage and selection estimator via the measurement error selection likelihood approach recently proposed by Stefanski, Wu, and White. The measurement error kernel regression operator (MEKRO) has the same form as the Nadaraya–Watson kernel estimator, but optimizes a measurement error model selection likelihood to estimate the kernel bandwidths. Much like LASSO or COSSO solution paths, MEKRO results in solution paths depending on a tuning parameter that controls shrinkage and selection via a bound on the harmonic mean of the pseudo-measurement error standard deviations. We use small-sample-corrected AIC to select the tuning parameter. Large-sample properties of MEKRO are studied and small-sample properties are explored via Monte Carlo experiments and applications to data. Supplementary materials for this article are available online.

KEYWORDS:

• Bandwidth selection
• Feature selection
• LASSO
• Nadaraya–Watson
• Nonparametric regression
• Solution path

Supplementary Materials

Appendix A provides an overview of the SWW framework and derives the MEKRO estimator in Eq. (3). Appendix B sketches a proof for selection consistency of MEKRO. Appendix C is an additional numerical study to address MEKRO.s performance when X follows a Gaussian distribution.

Acknowledgments

We thank the referees, Associate Editor, and Editor for alerting us to additional references and for their thoughtful comments and suggestions that greatly improved the paper.

Funding

K. White was funded by NSF grant DMS-1055210, NIH grant P01CA142538, and NIH training grant T32HL079896. L. Stefanski was funded by NSF grant DMS-1406456, NIH grants R01CA085848 and P01CA142538. Y. Wu was funded by NSF grant DMS-1055210 and NIH/NCI grants R01-CA149569 and P01-CA142538.

Notes

1 Copyright (c) 1995–1996 by The University of Toronto, Toronto, Ontario, Canada. All Rights Reserved.

2 Updated: 8 Oct. 1996. Accessed: 2 Mar. 2014.