Bootstrapping some GLM and survival regression variable selection estimators

Abstract

Inference after variable selection is a very important problem. This paper derives the asymptotic distribution of many variable selection estimators, such as forward selection and backward elimination, when the number of predictors is fixed. Under strong regularity conditions, the variable selection estimators are asymptotically normal, but generally the asymptotic distribution is a nonnormal mixture distribution. The theory shows that the lasso variable selection and elastic net variable selection estimators are $\sqrt{n}$ consistent estimators of $\mathit{\beta }$ when lasso and elastic net are consistent estimators of $\mathit{\beta }.$ A bootstrap technique to eliminate selection bias is to fit the variable selection estimator ${\widehat{\mathit{\beta }}}_{VS}^{*}$ to a bootstrap sample to find a submodel, then draw another bootstrap sample and fit the same submodel to get the bootstrap estimator ${\widehat{\mathit{\beta }}}_{\mathit{MIX}}^{*}.$ Bootstrap confidence regions were used for hypothesis testing.

Keywords:

• Backward elimination
• bagging
• confidence region
• forward selection

MATHEMATICS SUBJECT CLASSIFICATION:

• Primary 62F40
• Secondary 62J12

Acknowledgments

The authors thank the referees and Editors for their work.