Bootstrapping some GLM and survival regression variable selection estimators

References

• Akaike, H. 1973. Information theory and an extension of the maximum likelihood principle. In Proceedings, 2nd international symposium on information theory, ed. B. N. Petrov and F. Csakim, 267–81. Budapest: Akademiai Kiado.
   Google Scholar
• Bickel, P. J., and J. J. Ren. 2001. The bootstrap in hypothesis testing. In State of the art in probability and statistics: Festschrift for William R. van Zwet, ed. M. de Gunst, C. Klaassen, and A. van der Vaart, 91–112. Hayward, CA: The Institute of Mathematical Statistics.
   Google Scholar
• Breiman, L. 1996. Bagging predictors. Machine Learning 24 (2):123–40. doi:10.1023/A:1018054314350.
   Web of Science ®Google Scholar
• Buckland, S. T., K. P. Burnham, and N. H. Augustin. 1997. Model selection: An integral part of inference. Biometrics 53 (2):603–18. doi:10.2307/2533961.
   Web of Science ®Google Scholar
• Burr, D. 1994. A comparison of certain bootstrap confidence intervals in the Cox model. Journal of the American Statistical Association 89 (428):1290–302. doi:10.2307/2290992.
   Web of Science ®Google Scholar
• Charkhi, A., and G. Claeskens. 2018. Asymptotic post-selection inference for the Akaike information criterion. Biometrika 105 (3):645–64. doi:10.1093/biomet/asy018.
   Web of Science ®Google Scholar
• Claeskens, G., and N. L. Hjort. 2008. Model selection and model averaging. New York, NY: Cambridge University Press.
   Google Scholar
• Cook, R. D., and L. Forzani. 2018. Big data and partial least squares prediction. Canadian Journal of Statistics 46 (1):62–78. doi:10.1002/cjs.11316.
   Web of Science ®Google Scholar
• Cook, R. D., and L. Forzani. 2019. Partial least squares prediction in high-dimensional regression. The Annals of Statistics 47 (2):884–908. doi:10.1214/18-AOS1681.
   Web of Science ®Google Scholar
• Cook, R. D., and S. Weisberg. 1999. Applied regression including computing and graphics. New York, NY: Wiley.
   Google Scholar
• Cox, D. R. 1972. Regression models and life-tables. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 34 (2):187–220. doi:10.1111/j.2517-6161.1972.tb00899.x.
   Web of Science ®Google Scholar
• Efron, B. 1979. Bootstrap methods, another look at the jackknife. The Annals of Statistics 7 (1):1–26. doi:10.1214/aos/1176344552.
   Web of Science ®Google Scholar
• Efron, B. 1982. The jackknife, the bootstrap and other resampling plans. Philadelphia, PA: SIAM.
   Google Scholar
• Efron, B., and R. Tibshirani. 1986. Bootstrap methods for standard errors, confidence intervals, and other methods of statistical accuracy (with discussion). Statistical Science 1 (1):54–77. doi:10.1214/ss/1177013815.
   Google Scholar
• Efron, B., T. Hastie, I. Johnstone, and R. Tibshirani. 2004. Least angle regression (with discussion). The Annals of Statistics 32 (2):407–51. doi:10.1214/009053604000000067.
   Web of Science ®Google Scholar
• Ewald, K., and U. Schneider. 2018. Uniformly valid confidence sets based on the lasso. Electronic Journal of Statistics 12 (1):1358–87. doi:10.1214/18-EJS1425.
   Web of Science ®Google Scholar
• Fan, J., and R. Li. 2001. Variable selection via noncave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96 (456):1348–60. doi:10.1198/016214501753382273.
   Web of Science ®Google Scholar
• Freedman, D. A. 1981. Bootstrapping regression models. The Annals of Statistics 9 (6):1218–28. doi:10.1214/aos/1176345638.
   Web of Science ®Google Scholar
• Frey, J. 2013. Data-driven nonparametric prediction intervals. Journal of Statistical Planning and Inference 143 (6):1039–48. doi:10.1016/j.jspi.2013.01.004.
   Web of Science ®Google Scholar
• Friedman, J., T. Hastie, N. Simon, and R. Tibshirani. 2015. glmnet: Lasso and elastic-net regularized generalized linear models. R package version 2.0. http://cran.r-project.org/package=glmnet.
   Google Scholar
• Friedman, J., T. Hastie, and R. Tibshirani. 2010. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 33 (1):1–22. doi:10.18637/jss.v033.i01.
   PubMed Web of Science ®Google Scholar
• Guan, L., and R. Tibshirani. 2020. Post model-fitting exploration via a “next-door” analysis. Canadian Journal of Statistics 48 (3):447–70. doi:10.1002/cjs.
   PubMed Web of Science ®Google Scholar
• Hall, P. 1988. Theoretical comparisons of bootstrap confidence intervals (with discussion). The Annals of Statistics 16 (3):927–85. doi:10.1214/aos/1176350933.
   Web of Science ®Google Scholar
• Hastie, T., R. Tibshirani, and M. Wainwright. 2015. Statistical learning with sparsity: The lasso and generalizations. Boca Raton, FL: CRC Press Taylor & Francis.
   Google Scholar
• Hillis, S. L., and C. S. Davis. 1994. A simple justification of the iterative fitting procedure for generalized linear models. The American Statistician 48 (4):288–89. doi:10.1080/00031305.1994.10476082.
   Web of Science ®Google Scholar
• Hjort, G., and N. L. Claeskens. 2003. The focused information criterion. Journal of the American Statistical Association 98 (464):900–45. doi:10.1198/016214503000000819.
   Web of Science ®Google Scholar
• Knight, K., and W. J. Fu. 2000. Asymptotics for lasso-type estimators. The Annals of Statistics 28 (5):1356–78. doi:10.1214/aos/1015957397.
   Web of Science ®Google Scholar
• Lee, S. M. S., and Y. Wu. 2018. A bootstrap recipe for post-model-selection inference under linear regression models. Biometrika 105 (4):873–90. doi:10.1093/biomet/asy046.
   Web of Science ®Google Scholar
• Leeb, H., and B. M. Pötscher. 2003. The finite-sample distribution of post-model selection estimators and uniform versus nonuniform approximations. Econometric Theory 19 (01):100–42. doi:10.1017/S0266466603191050.
   Web of Science ®Google Scholar
• Leeb, H., B. M. Pötscher, and K. Ewald. 2015. On various confidence intervals post-model-selection. Statistical Science 30 (2):216–27. doi:10.1214/14-STS507.
   Web of Science ®Google Scholar
• Lindenmayer, D. B., R. Cunningham, M. T. Tanton, H. A. Nix, and A. P. Smith. 1991. The conservation of arboreal marsupials in the montane ash forests of central highlands of Victoria, South-East Australia: III. The habitat requirement’s of Leadbeater’s possum Gymnobelideus Leadbeateri and models of the diversity and abundance of arboreal marsupials. Biological Conservation 56 (3):295–315. doi:10.1016/0006-3207(91)90063-F.
   Web of Science ®Google Scholar
• Lu, S., Y. Liu, L. Yin, and K. Zhang. 2017. Confidence intervals and regions for the lasso by using stochastic variational inequality techniques in optimization. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 79 (2):589–611. doi:10.1111/rssb.12184.
   Web of Science ®Google Scholar
• Lumley, T. 2009. leaps: Regression subset selection. R package version 2.9. https://cran.r-project.org/package=leaps.
   Google Scholar
• Mallows, C. 1973. Some comments on Cp. Technometrics 15 (4):661–76. doi:10.2307/1267380.
   Web of Science ®Google Scholar
• Meinshausen, N. 2007. Relaxed lasso. Computational Statistics & Data Analysis 52 (1):374–93. doi:10.1016/j.csda.2006.12.019.
   Web of Science ®Google Scholar
• Nelder, J. A., and R. W. M. Wedderburn. 1972. Generalized linear models. Journal of the Royal Statistical Society, A 135 (3):370–80. doi:10.2307/2344614.
   Web of Science ®Google Scholar
• Ning, Y., and H. Liu. 2017. A general theory of hypothesis tests and confidence regions for sparse high dimensional models. The Annals of Statistics 45 (1):158–95. doi:10.1214/16-AOS1448.
   Web of Science ®Google Scholar
• Olive, D. J. 2017a. Linear regression. New York, NY: Springer.
   Google Scholar
• Olive, D. J. 2017b. Robust multivariate analysis. New York, NY: Springer.
   Google Scholar
• Olive, D. J. 2018. Applications of hyperellipsoidal prediction regions. Statistical Papers 59 (3):913–31. doi:10.1007/s00362-016-0796-1.
   Web of Science ®Google Scholar
• Olive, D. J. 2021. Prediction and statistical learning. Online course notes. http://parker.ad.siu.edu/Olive/slearnbk.htm.
   Google Scholar
• Olive, D. J., and D. M. Hawkins. 2005. Variable selection for 1D regression models. Technometrics 47 (1):43–50. doi:10.1198/004017004000000590.
   Web of Science ®Google Scholar
• Olive, D. J., R. C. Rathnayake, and M. G. Haile. 2021. Prediction intervals for GLMs, GAMs, and some survival regression models. Communications in Statistics: Theory and Methods. Advance online publication. doi:10.1080/03610926.2021.1887238.
   Web of Science ®Google Scholar
• Pelawa Watagoda, L. C. R., and D. J. Olive. 2020. Comparing six shrinkage estimators with large sample theory and asymptotically optimal prediction intervals. Statistical Papers. Advance online publication. doi:10.1007/s00362-020-01193-1.
   Web of Science ®Google Scholar
• Pelawa Watagoda, L. C. R., and D. J. Olive. 2021. Bootstrapping multiple linear regression after variable selection. Statistical Papers 62 (2):681–700. doi:10.1007/s00362-019-01108-9.
   Web of Science ®Google Scholar
• Pötscher, B. 1991. Effects of model selection on inference. Econometric Theory 7 (2):163–85. doi:10.1017/S0266466600004382.
   Web of Science ®Google Scholar
• Pratt, J. W. 1959. On a general concept of “in Probability”. The Annals of Mathematical Statistics 30 (2):549–58. doi:10.1214/aoms/1177706267.
   Google Scholar
• R Core Team. 2018. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. www.R-project.org.
   Google Scholar
• Rinaldo, A., L. Wasserman, and M. G’Sell. 2019. Bootstrapping and sample splitting for high-dimensional, assumption-free inference. The Annals of Statistics 47 (6):3438–69. doi:10.1214/18-AOS1784.
   Web of Science ®Google Scholar
• Schomaker, M. 2012. Shrinkage averaging estimation. Statistical Papers 53 (4):1015–34. doi:10.1007/s00362-011-0405-2.
   Web of Science ®Google Scholar
• Schomaker, M., and C. Heumann. 2014. Model selection and model averaging after multiple imputation. Computational Statistics & Data Analysis 71:758–70. doi:10.1016/j.csda.2013.02.017.
   Web of Science ®Google Scholar
• Schwarz, G. 1978. Estimating the dimension of a model. The Annals of Statistics 6 (2):461–64. doi:10.1214/aos/1176344136.
   Web of Science ®Google Scholar
• Sen, P. K., and J. M. Singer. 1993. Large sample methods in statistics: An introduction with applications. New York, NY: Chapman & Hall.
   Google Scholar
• Shao, J. 1993. Linear model selection by cross-validation. Journal of the American Statistical Association 88 (422):486–94. doi:10.1080/01621459.1993.10476299.
   Web of Science ®Google Scholar
• Shao, J., and D. S. Tu. 1995. The jackknife and the bootstrap. New York, NY: Springer.
   Google Scholar
• Simon, N., J. Friedman, T. Hastie, and R. Tibshirani. 2011. Regularization paths for Cox's proportional hazards model via coordinate descent. Journal of Statistical Software 39 (5):1–13. doi:10.18637/jss.v039.i05.
   PubMed Web of Science ®Google Scholar
• Su, W. J. 2018. When is the first spurious variable selected by sequential regression procedures? Biometrika 105 (3):517–27. doi:10.1093/biomet/asy032.
   Web of Science ®Google Scholar
• Su, Z., and R. D. Cook. 2012. Inner envelopes: Efficient estimation in multivariate linear regression. Biometrika 99 (3):687–702. doi:10.1093/biomet/ass024.
   Web of Science ®Google Scholar
• Tibshirani, R. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 58 (1):267–88. doi:10.1111/j.2517-6161.1996.tb02080.x.
   Web of Science ®Google Scholar
• Tibshirani, R. J., A. Rinaldo, R. Tibshirani, and L. Wasserman. 2018. Uniform asymptotic inference and the bootstrap after model selection. The Annals of Statistics 46 (3):1255–87. doi:10.1214/17-AOS1584.
   Web of Science ®Google Scholar
• Venables, W. N., and B. D. Ripley. 2010. Modern applied statistics with S. 4th ed. New York, NY: Springer.
   Google Scholar
• Wang, H., and S. Z. F. Zhou. 2013. Interval estimation by frequentist model averaging. Communications in Statistics - Theory and Methods 42 (23):4342–56. doi:10.1080/03610926.
   Web of Science ®Google Scholar
• Wieczorek, J. A. 2018. Model selection and stopping rules for high-dimensional forward selection. Ph.D. Thesis, Carnegie Mellon University.
   Google Scholar
• Yang, Y. 2003. Regression with multiple candidate models: Selecting or mixing? Statistica Sinica 13 (3):783–809.
   Web of Science ®Google Scholar
• Zhang, J. 2020. Consistency of MLE, LSE and M-estimation under mild conditions. Statistical Papers 61 (1):189–99. doi:10.1007/s00362-017-0928-2.
   Web of Science ®Google Scholar
• Zhao, P., and B. Yu. 2006. On model selection consistency of lasso. Journal of Machine Learning Research 7:2541–63.
   Web of Science ®Google Scholar
• Zhou, M. 2001. Understanding the Cox regression models with time–change covariates. The American Statistician 55 (2):153–55. doi:10.1198/000313001750358491.
   Web of Science ®Google Scholar
• Zou, H., and T. Hastie. 2005. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 67 (2):301–20. doi:10.1111/j.1467-9868.2005.00503.x.
   Web of Science ®Google Scholar