On the Performance of Horseshoe Priors for Inducing Sparsity in Structural Equation Models

References

• Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In B. N. Petrov & F. Csaki (Eds.), Second international symposium on information theory. Akademiai Kiado.
   Google Scholar
• Allen, D. M. (1974). The relationship between variable selection and data agumentation and a method for prediction. Technometrics, 16, 125–127. https://doi.org/10.1080/00401706.1974.10489157
   Web of Science ®Google Scholar
• Bainter, S. A., McCauley, T. G., Fahmy, M. M., Goodman, Z. T., Kupis, L. B., & Rao, J. S. (2023). Comparing Bayesian variable selection to lasso approaches for applications in psychology. Psychometrika, 88, 1032–1055. https://doi.org/10.1007/s11336-023-09914-9
   PubMed Web of Science ®Google Scholar
• Betancourt, M. (2018a). Bayes sparse regression. https://betanalpha.github.io/assets/case/_studies/bayes/_sparse/_regression.html
   Google Scholar
• Betancourt, M. (2018b). A conceptual introduction to Hamiltonian Monte Carlo. https://arxiv.org/pdf/1701.02434.pdf
   Google Scholar
• Bollen, K. A. (1989). Structural equations with latent variables. John Wiley and Sons.
   Google Scholar
• Bollen, K. A., & Curran, P. J. (2006). Latent curve models: A structural equation perspective. John Wiley & Sons.
   Google Scholar
• Bürkner, P.-C. (2017). brms: An R package for Bayesian multilevel models using Stan. Journal of Statistical Software, 80, 1–28. https://doi.org/10.18637/jss.v080.i01
   Web of Science ®Google Scholar
• Bürkner, P.-C. (2018). Advanced Bayesian multilevel modeling with the R package brms. The R Journal, 10, 395–411. https://doi.org/10.32614/RJ-2018-017
   Google Scholar
• Brooks, S. P., & Gelman, A. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7, 434–455. https://doi.org/10.2307/1390675
   Web of Science ®Google Scholar
• Carvalho, C. M., Polson, N. G., & Scott, J. G. (2009). Handling sparsity via the horseshoe. In Proceedings of the twelth international conference on artificial intelligence and statistics (pp. 73–80). PMLR.
   Google Scholar
• Carvalho, C. M., Polson, N. G., & Scott, J. G. (2010a). The horseshoe estimator for spare signals. Biometrika, 97, 465–480. https://doi.org/10.1093/biomet/asq017
   Web of Science ®Google Scholar
• Carvalho, C. M., Polson, N. G., & Scott, J. G. (2010b). The horseshoe estimator for sparse signals. Biometrika, 97, 465–480. https://doi.org/10.1093/biomet/asq017
   Web of Science ®Google Scholar
• Clyde, M. A., & George, E. I. (2004). Model uncertainty. Statistical Science, 19, 81–84. https://doi.org/10.1214/088342304000000035
   Web of Science ®Google Scholar
• Common Core of Data. (2020). https://nces.ed.gov/ccd/.
   Google Scholar
• Dawid, A. P. (1982). The well-calibrated Bayesian. Journal of the American Statistical Association, 77, 605–610. https://doi.org/10.1080/01621459.1982.10477856
   Web of Science ®Google Scholar
• Dawid, A. P. (1984). Statistical theory: The prequential approach. Journal of the Royal Statistical Society, Series A, 147, 278–202. https://doi.org/10.2307/2981683
   Web of Science ®Google Scholar
• Depaoli, S. (2021). Bayesian structural equation modeling. Guilford Publications.
   Google Scholar
• Depaoli, S., Kaplan, D., & Winter, S. (2023). Foundations and extensions of Bayesian structural equation modeling. In R. Hoyle (Ed.), Handbook of structural equation modeling (2nd ed.). Guilford Publications, Inc.
   Google Scholar
• Draper, D. (1995). Assessment and propagation of model uncertainty (with discussion). Journal of the Royal Statistical Society (Series B), 57, 55–98.
   Google Scholar
• Draper, D. (2013). Bayesian model specification: Heuristics and examples. In Bayesian theory and applications (pp. 483–498). Oxford University Press.
   Google Scholar
• Gelman, A., Hwang, J., & Vehtari, A. (2014). Understanding predictive information criteria for Bayesian models. Statistics and Computing, 24, 997–1016. https://doi.org/10.1007/s11222-013-9416-2
   Web of Science ®Google Scholar
• Gelman, A., Simpson, D., & Betancourt, M. (2017). The prior can often only be understood in the context of the likelihood. Entropy, 19, 555. https://doi.org/10.3390/e19100555
   Web of Science ®Google Scholar
• George, E. I., & McCulloch, R. E. (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association, 88, 881–889. https://doi.org/10.1080/01621459.1993.10476353
   Web of Science ®Google Scholar
• Goodrich, B., Gabry, J., Ali, I., Brilleman, S. (2022). rstanarm: Bayesian applied regression modeling via Stan (R package version 2.21.3). https://mc-stan.org/rstanarm/
   Google Scholar
• Gronau, Q. F., & Wagenmakers, E. J. (2019). Limitations of Bayesian leave-one-out cross-validation for model selection. Computational Brain & Behavior, 2, 35–47. https://doi.org/10.1007/s42113-018-0022-4
   PubMedGoogle Scholar
• Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12, 55–67. https://doi.org/10.1080/00401706.1970.10488634
   Web of Science ®Google Scholar
• Hoerl, R. W. (1985). Ridge analysis 25 years later. The American Statistician, 39, 186–192. https://doi.org/10.2307/2683926
   Web of Science ®Google Scholar
• Hoeting, J. A., Madigan, D., Raftery, A., & Volinsky, C. T. (1999). Bayesian model averaging: A tutorial. Statistical Science, 14, 382–417. https://doi.org/10.1214/ss/1009212519
   Web of Science ®Google Scholar
• Hoffman, M. D., & Gelman, A. (2014). The No-U-Turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, 15, 1593–1623. http://jmlr.org/papers/v15/hoffman14a.html
   Web of Science ®Google Scholar
• Hollenbach, F. M., & Montgomery, J. M. (2020). Bayesian model selection, model comparison, and model averaging. In The SAGE handbook of research methods in political science and international relations (pp. 937–960). Sage.
   Google Scholar
• Hsiang, T. C. (1975). A Bayesian view on ridge regression. The Statistician, 24, 267–268. https://doi.org/10.2307/2987923
   Google Scholar
• Jacobucci, R., & Grimm, K. J. (2018). Comparison of frequentist and Bayesian regularization in structural equation modeling. Structural Equation Modeling, 25, 639–649. https://doi.org/10.1080/10705511.2017.1410822
   PubMed Web of Science ®Google Scholar
• Kaplan, D. (2009). Structural equation modeling: Foundations and extensions (2nd ed.). Sage Publications.
   Google Scholar
• Kaplan, D. (2021). On the quantification of model uncertainty: A Bayesian perspective. Psychometrika, 86, 215–238. https://doi.org/10.1007/s11336-021-09754-5
   PubMed Web of Science ®Google Scholar
• Kaplan, D. (2023). Bayesian statistics for the social sciences (2nd ed.). Guilford Press.
   Google Scholar
• Kaplan, D., & Lee, C. (2015). Bayesian model averaging over directed acyclic graphs with implications for the predictive performance of structural equation models. Structural Equation Modeling, 23, 343–353.
   Web of Science ®Google Scholar
• Karimova, D., Leenders, R. T. A., Meijerink-Bosman, M., & Mulder, J. (2023). Separating the wheat from the chaff: Bayesian regularization in dynamic social networks. Social Networks, 74, 139–155. https://doi.org/10.1016/j.socnet.2023.02.006
   Web of Science ®Google Scholar
• Leamer, E. E. (1978). Specification searches: Ad hoc inference with nonexperimental data. Wiley.
   Google Scholar
• Lindley, D. V. (2007). Understanding uncertainty. Wiley.
   Google Scholar
• Merkle, E. C., Fitzsimmons, E., Uanhoro, J., & Goodrich, B. (2021). Efficient Bayesian structural equation modeling in Stan. Journal of Statistical Software, 100, 1–22. https://doi.org/10.18637/jss.v100.i06
   Web of Science ®Google Scholar
• Merkle, E. C., & Rosseel, Y. (2018). blavaan: Bayesian structural equation models via parameter expansion. Journal of Statistical Software, 85, 1–30. https://doi.org/10.18637/jss.v085.i04
   PubMed Web of Science ®Google Scholar
• Mitchell, T. J., & Beauchamp, J. J. (1988). Bayesian variable selection in linear regression. Journal of the American Statistical Association, 83, 1023–1032. https://doi.org/10.1080/01621459.1988.10478694
   Web of Science ®Google Scholar
• OECD. (2010). Pisa 2009 results: Executive summary. OECD.
   Google Scholar
• Park, T., & Casella, G. (2008). The Bayesian lasso. Journal of the American Statistical Association, 103, 681–686. https://doi.org/10.1198/016214508000000337
   Web of Science ®Google Scholar
• Piironen, J., & Vehtari, A. (2017). Sparsity information and regularization in the horseshoe and other shrinkage priors. Electronic Journal of Statistics, 11, 5018–5051. https://doi.org/10.1214/17-EJS1337SI
   Web of Science ®Google Scholar
• Plummer, M. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. Technische Universität Wien, Vienna, Austria. https://www.R-project.org/conferences/DSC-2003/Proceedings/Plummer.pdf.
   Google Scholar
• Polson, N. G., & Scott, J. G. (2011). Shrink globally, act locally: Sparse Bayesian regularization and prediction. In Bayesian statistics 9. Oxford University Press.
   Google Scholar
• Raftery, A. (1995). Bayesian model selection in social research (with discussion). In P. V. Marsden (Ed.), Sociological methodology (Vol. 25, pp. 111–196). Blackwell. https://doi.org/10.2307/271063
   Google Scholar
• Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & van der Linde, A. (2002). Bayesian measures of model complexity and fit (with discussion). Journal of the Royal Statistical Society Series B: Statistical Methodology, 64, 583–639. https://doi.org/10.1111/1467-9868.00353
   Web of Science ®Google Scholar
• Stan Development Team. (2021). Stan modeling language users guide and reference manual, version 2.26 [Computer software manual]. https://mc-stan.org.
   Google Scholar
• Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions. Journal of the Royal Statistical Society. Series B (Methodological), 36, 111–133. https://doi.org/10.1111/j.2517-6161.1974.tb00994.x
   Web of Science ®Google Scholar
• Stone, M. (1977). An asymptotic equivalence of choice of model by cross-validation and Akaike’s criterion. Journal of the Royal Statistical Society. Series B (Methodological), 39, 44–47. https://doi.org/10.1111/j.2517-6161.1977.tb01603.x
   Web of Science ®Google Scholar
• Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 58, 267–288. https://doi.org/10.1111/j.2517-6161.1996.tb02080.x
   Web of Science ®Google Scholar
• U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. (2022). National Assessment of Educational Progress NAEP. (https://nces.ed.gov/nationsreportcard/)
   Google Scholar
• van de Schoot, R., Depaoli, S., King, R., Kramer, B., Märtens, K., Tadesse, M. G., Vannucci, M., Gelman, A., Veen, D., Willemsen, J., & Yau, C. (2021). Bayesian statistical modelling. Nature Reviews Methods Primers, 1, 1–26. https://doi.org/10.1038/s43586-020-00001-2
   Google Scholar
• van Erp, S. (2020). A tutorial on Bayesian penalized regression with shrinkage priors for small sample sizes. In R. van de Schoot & M. Miočević (Eds.), Small sample size solutions (pp. 71–84). Taylor and Francis.
   Google Scholar
• van Erp, S., Oberski, D. L., & Mulder, J. (2019). Shrinkage priors for Bayesian penalized regression. Journal of Mathematical Psychology, 89, 31–50. https://doi.org/10.1016/j.jmp.2018.12.004
   Web of Science ®Google Scholar
• Vehtari, A., Gabry, J., Yao, Y., Gelman, A. (2019). loo: Efficient leave-one-out cross-validation and WAIC for Bayesian models (R package version 2.1.0). https://CRAN.R-project.org/package=loo
   Google Scholar
• Vehtari, A., Gelman, A., & Gabry, J. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27, 1413–1432. https://doi.org/10.1007/s11222-016-9696-4
   Web of Science ®Google Scholar
• Watanabe, S. (2010). Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. Journal of Machine Learning Research, 11, 3571–3594.
   Web of Science ®Google Scholar
• Yao, Y., Vehtari, A., Simpson, D., & Gelman, A. (2018). Using stacking to average Bayesian predictive distributions (with discussion). Bayesian Analysis, 13, 917–1007. https://doi.org/10.1214/17-BA1091
   Web of Science ®Google Scholar
• Zou, H. (2006). The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101, 1418–1429. https://doi.org/10.1198/016214506000000735
   Web of Science ®Google Scholar
• Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society Series B: Statistical Methodology, 67, 301–320. https://doi.org/10.1111/j.1467-9868.2005.00503.x
   Web of Science ®Google Scholar