A stepwise regression algorithm for high-dimensional variable selection

Abstract

We propose a new stepwise regression algorithm with a simple stopping rule for the identification of influential predictors and interactions among a huge number of variables in various statistical models. Like conventional stepwise regression, at each forward selection step, a variable is included in the current model if the test statistic of the enlarged model with the predictor against the current model has the minimum $p$-value among all the candidates and is smaller than a predetermined threshold. Instead of using conventional information types of criteria, the threshold is determined by a lower percentile of the beta distribution. We conducted extensive simulation studies to evaluate the performance of the proposed algorithm for various statistical models and found it to be very competitive and robust compared with several popular high-dimensional variable selection methods.

Keywords:

• forward regression
• genome-wide association study
• high-dimensional data
• lasso
• minimum p-value

Acknowledgements

The authors are grateful to the AE and referees for the helpful comments and valuable suggestions. The authors thank Professor W.H. Pan of Academia Sinica for providing the GWAS data and the permission of use as a real example in this study. The work was supported in part by the National Science Council of Taiwan grant NSC 99-2118-M-001-004-MY2.