A Simulation of Biased Estimation and Subset Selection Regression Techniques

Abstract

This study compared three biased estimation and four subset selection regression techniques to least squares in a large-scale simulation. The parameters relevant to a comparison of the techniques involved were systematically varied over wide ranges. A parameter of importance not used in previous major simulations of subset techniques, the proportion of independent variables in the data that were superfluous, was included. The major result is that neither biased estimation nor subset selection demonstrated a consistent superiority over the other, excluding stepwise and principal component regression, both of which performed poorly.

KEY WORDS:

• Ridge regression
• Stepwise regression
• Principal component regression