Variable Selection and Estimation for Misclassified Binary Responses and Multivariate Error-Prone Predictors

Abstract

In statistical analysis or supervised learning, classification has been an attractive topic. Typically, a main goal is to adopt predictors to characterize the primarily interested binary random variables. To model a binary response and predictors, parametric structures, such as logistic regression models or probit models, are perhaps commonly used approaches. However, due to the convenience of data collection, existence of non-informative variables as well as inevitability of measurement error in both responses and predictors become ubiquitous. The simultaneous appearance of these complex features make data analysis become challenging. To address those concerns, we propose a valid inferential method to deal with measurement error and handle variable selection simultaneously. Specifically, we focus on logistic regression or probit models, and propose estimating functions by incorporating corrected responses and predictors. After that, we develop the boosting procedure with error-eliminated estimating functions accommodated to do variable selection and estimation. To justify the proposed method, we examine the convergence of the boosting algorithm and rigorously establish the theoretical results. Through numerical studies, we find that the proposed method accurately retains informative predictors and gives precise estimators, and its performance is generally better than that without measurement error correction. The supplementary materials of this article, including proofs of theoretical results and computer code, are available online.

KEYWORDS:

• Boosting
• Error elimination
• Measurement error
• Regression calibration
• Supervised learning

Acknowledgments

The author would like to thank his master student, QinYing OuYang, for helping edit the initial programming code and thank Lingyu Cai for helping revise the programming code and rerun new numerical studies during the revision process. The author appreciates the Editor, an Associate Editor, and two referees for their useful comments that significantly improve the initial article.

Additional information

Funding

Chen’s research was supported by National Science and Technology Council with grant ID 110-2118-M-004-006-MY2.