Ridge regression and the Lasso: how do they do as finders of significant regressors and their multipliers?

References

• Austin, P. C., and E. W. Steyerberg. 2015. The number of subjects per variable required in linear regression analyses. Journal of Clinical Epidemiology 68 (6):627–36. doi:10.1016/j.jclinepi.2014.12.014.
   PubMed Web of Science ®Google Scholar
• Bendel, R. B., and A. A. Afifi. 1977. Comparison of stopping rules in forward stepwise regression. Journal of the American Statistical Association 72 (357):46–53. doi:10.2307/2286904.
   Web of Science ®Google Scholar
• Breiman, L. 2001a. Statistical modeling. Statistical Science 16 (3):199–231. doi:10.1214/ss/1009213726.
   Web of Science ®Google Scholar
• Breiman, L. 2001b. Random forests. Machine Learning 45 (1):5–32. doi:10.1023/A:1010933404324.
   Web of Science ®Google Scholar
• Brook, R. J., and T. Moore. 1980. On the expected length of the least squares coefficient vector. Journal of Econometrics 12 (2):245–46. doi:10.1016/0304-4076(80)90010-X.
   Web of Science ®Google Scholar
• Efron, B., and R. Tibshirani. 1993. An introduction to the bootstrap. Boca Raton, FL: Chapman and Hall.
   Google Scholar
• Efron, B., T. Hastie, I. Johnstone, and R. Tibshirani. 2004. Least angle regression. The Annals of Statistics 32 (2):407–99. doi:10.1214/009053604000000067.
   Web of Science ®Google Scholar
• Efroymson, M. A. 1960. Multiple regression analysis. In Mathematical methods for digital computers. New York: Wiley.
   Google Scholar
• Frank, I. E., and J. H. Friedman. 1993. A statistical view of some chemometrics regression tools. Technometrics 35 (2):109–35. doi:10.1080/00401706.1993.10485033.
   Web of Science ®Google Scholar
• Gana, R. 1995. Ridge regression estimation of the linear probability model. Journal of Applied Statistics 22 (4):537–39. doi:10.1080/757584790.
   Web of Science ®Google Scholar
• Gana, R. 2018. Immigration: How does it impact the wages of native workers?. Available at SSRN https://ssrn.com/abstract=3247320. doi:10.2139/ssrn.3247320.
   Google Scholar
• Gana, R., and S. Vasudevan. 2019. Ridge regression estimated linear probability model predictions of O-glycosylation in proteins with structural and sequence data. BMC Molecular and Cell Biology 20 (1):21. doi:10.1186/s12860-019-0200-9.
   PubMed Web of Science ®Google Scholar
• Goldberger, A. S. 1964. Econometric theory. New York: Wiley.
   Google Scholar
• Golub, G. H., M. Heath, and G. Wahba. 1979. Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21 (2):215–23. doi:10.1080/00401706.1979.10489751.
   Web of Science ®Google Scholar
• Greenberg, E., and R. P. Parks. 1997. A predictive approach to model selection and multicollinearity. Journal of Applied Econometrics 12 (1):67–75. doi:10.1002/(SICI)1099-1255(199701)12:1<67::AID-JAE427>3.0.CO;2-W.
   Web of Science ®Google Scholar
• Gruber, M. H. J. 2014. Regression estimators – A comparative study. Boston, MA: Academic Press.
   Google Scholar
• Gruber, M. H. J. 2017. Improving efficiency by shrinkage – The James-Stein and Ridge Regression estimators. New York: Marcel Dekker Inc.
   Google Scholar
• Hocking, R. R. 1976. The analysis and selection of variables in linear regression. Biometrics 32 (1):1–49. doi:10.2307/2529336.
   Web of Science ®Google Scholar
• Hoerl, A. E., R. W. Kannard, and K. F. Baldwin. 1975. Ridge regression: some simulations. Communications in Statistics - Simulation and Computation 4 (2):105–23. doi:10.1080/03610917508548342.
   Web of Science ®Google Scholar
• Hoerl, A. E., and R. W. Kennard. 1970. Ridge regression: biased estimation of nonorthogonal problems. Technometrics 12 (1):55–67. doi:10.1080/00401706.1970.10488634.
   Web of Science ®Google Scholar
• Hoerl, A. E., and R. W. Kennard. 1976. Ridge regression iterative estimation of the biasing parameter. Communications in Statistics - Theory and Methods 5 (1):77–88. doi:10.1080/03610927608827333.
   Web of Science ®Google Scholar
• Hoerl, R. W., J. H. Schuenemeyer, and A. E. Hoerl. 1986. A simulation of biased estimation and subset selection regression techniques. Technometrics 28 (4):369–80. doi:10.2307/1268986.
   Web of Science ®Google Scholar
• Keynes, J. M. 1938. The Collected Writings of John Maynard Keynes, volume 14, pp. 299–300.
   Google Scholar
• Lawless, J. F., and P. Wang. 1976. A simulation study of ridge and other regression estimators. Communications in Statistics - Theory and Methods 5 (4):307–23. doi:10.1080/03610927608827353.
   Web of Science ®Google Scholar
• Mallows, C. L. 1973. Some comments on Cp. Technometrics 15 (4):661–75. doi:10.1080/00401706.1973.10489103.
   Web of Science ®Google Scholar
• Matsumoto, M., and T. Nishimura. 1998. Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator. ACM Transactions on Modeling and Computer Simulation 8 (1):3–30. doi:10.1145/272991.272995.
   Google Scholar
• Monyak, J. T. 1998. Mean squared error properties of the ridge regression estimated linear probability model, unpublished PhD dissertation, University of Delaware.
   Google Scholar
• Myers, R. H. 1990. Classical and modern regression with applications. Technometrics 29:377–378.
   Google Scholar
• Obenchain, R. L. 1977. Classical F-tests and confidence regions for ridge regression. Technometrics 19 (4):429–39. doi:10.1080/00401706.1977.10489582.
   Web of Science ®Google Scholar
• Osborne, M., B. Presnell, and B. Turlach. 2000. A new approach to variable selection in least squares problems. IMA Journal of Numerical Analysis 20 (3):389–404. doi:10.1093/imanum/20.3.389.
   Web of Science ®Google Scholar
• Piegorsch, W. W., and G. Casella. 1989. The early use of matrix diagonal increments in statistical problems. SIAM Review 31 (3):428–34. doi:10.1137/1031089.
   Web of Science ®Google Scholar
• Taylor, H. L., S. C. Banks, and J. F. McCoy. 1979. Deconvolution with the Ɩ1 norm. Geophysics 44 (1):39–52. doi:10.1190/1.1440921.
   Web of Science ®Google Scholar
• Santosa, F., and W. W. Symes. 1986. Linear inversion of band-limited reflection seismograms. SIAM Journal on Scientific and Statistical Computing 7 (4):1307–30. doi:10.1137/0907087.
   Web of Science ®Google Scholar
• Tibshirani, R. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B (Methodological) 58 (1):267–88. doi:10.1111/j.2517-6161.1996.tb02080.x.
   Web of Science ®Google Scholar