Kernel Estimators for Univariate Binary Regression

Abstract

We present a rather thorough investigation of the use of kernel-based nonparametric estimators of the binary regression function in the case of a single covariate. We consider various versions of Nadaraya–Watson and local linear estimators, some involving a single bandwidth and others involving two bandwidths. The locally linear logistic estimator proves to be a good single-bandwidth estimator, although the basic Nadaraya–Watson estimator also fares quite well. Two-bandwidth methods show great potential when bandwidths are selected with knowledge of the target function, but much of their potential vanishes when data-based bandwidths are used. Likelihood cross-validation and plug-in approaches are the data-based bandwidth selection methods tested; both prove quite useful, with a preference for the latter. Adaptive two-bandwidth methods retain particularly good performance only in certain special situations (and separate estimation of the two bandwidths as for optimal density estimation is never recommended). We therefore propose a hybrid estimation procedure in which the local linear logistic estimator is used unless the ratio of (robust) variances of the covariate in the success and failure groups is greater than 2, in which case we switch to a two-bandwidth Nadaraya–Watson-type estimator, each using plug-in bandwidth selection.

KEY WORDS :

• Bandwidth selection
• Cross-validation
• Local linear
• Logistic
• Nadaraya-Watson
• Nonparametric regression
• Plug-in
• Two bandwidths