Ridge regression and the Lasso: how do they do as finders of significant regressors and their multipliers?

Abstract

A simulation study is done to compare Ridge Regression (RR) and the Lasso, under the assumption of a linear model, by calculating four metrics: the squared distance, from the true coefficients, of estimated coefficients that are both statistically significant and true; the proportion of true regressors discovered; the squared distance, from the true predictions, of the predictions made using the estimated coefficients that are only statistically significant (but not necessarily true); and the chance that no estimated coefficient is true. Results indicate that RR surpasses the Lasso in regard to all of these metrics. This indicates that RR can add value to model discovery, in Economics and the Sciences, by continuing to employ the key concept of statistical significance in the classical sense to find true regressors. This is important, because it allows the “manufacturer” of models to focus on the process generating the data, if indeed there is one. And, thus, provide important feedback on the outputs of other fashionable competitors, such as Machine Learning, with their pervasive “black-box” focus on prediction as an end in itself.

KEYWORDS:

• Lasso
• Ridge regression
• Statistical significance

Acknowledgements

I thank the anonymous referees, and the Editor-in-Chief, for asking me a question that made me think much more about the symbolic form of my manuscript; and how symbolism can transparently represent my reasoning, which in the prior version of my manuscript was mostly embedded in computer code and output and thus harder to follow. The prior version of my manuscript was posted on the Social Science Research Network (SSRN) preprint server in August 2019 (https://ssrn.com/abstract=3437930); and I thank SSRN for offering this option.