The Influence of Variable Selection: A Bayesian Diagnostic Perspective

Abstract

Variable selection is ubiquitous in statistical practice. Forward selection, backwards elimination, or a combination thereof are among the most popular techniques. Statistical texts teach direct interpretation of coefficients after variable selection. For scientific purposes, the desired interpretation is unconditional on the variable selection. In this article an influence analysis of variable selection is performed from a Bayesian viewpoint. Variable selection is shown to be surprisingly influential. A new statistic is recommended for routine examination during variable selection.

Key Words:

• Backward elimination
• Diagnostic
• Forward selection
• Generalized linear model
• Model selection