Should Regularization Replace Simple Structure Rotation in Exploratory Factor Analysis?

References

• Arruda, E. H., & Bentler, P. M. (2017). A regularized GLS for structural equation modeling. Structural Equation Modeling, 24, 657–665. doi:10.1080/10705511.2017.1318392
   Web of Science ®Google Scholar
• Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 16, 397–438. doi:10.1080/10705510903008204
   Web of Science ®Google Scholar
• Asparouhov, T., & Muthén, B. (2012). General random effect latent variable modeling: Random subjects, items, contexts, and parameters. Retrieved from http://www.statmodel.com/download/NCME12.pdf
   Google Scholar
• Asparouhov, T., & Muthén, B. (2018). SRMR in Mplus. Retrieved from http://www.statmodel.com/download/SRMR2.pdf
   Google Scholar
• Beauducel, A. (2018). Recovering Wood and McCarthy’s ERP-prototypes by means of ERP-specific procrustes-rotation. Journal of Neuroscience Methods, 295, 20–36. doi:10.1016/j.jneumeth.2017.11.011
   PubMed Web of Science ®Google Scholar
• Bentler, P. M., & Yuan, K. H. (1999). Structural equation modeling with small samples: Test statistics. Multivariate Behavioral Research, 34, 181–197. doi:10.1207/S15327906Mb340203
   PubMed Web of Science ®Google Scholar
• Bernaards, C. A., & Jennrich, R. I. (2005). Gradient projection algorithms and software for arbitrary rotation criteria in factor analysis. Educational and Psychological Measurement, 65, 676–696. doi:10.1177/0013164404272507
   Web of Science ®Google Scholar
• Browne, M. W. (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36, 111–150. doi:10.1207/S15327906MBR360105
   Web of Science ®Google Scholar
• Carvalho, C. M., Chang, J., Lucas, J. E., Nevins, J. R., Wang, Q., & West, M. (2008). High-dimensional sparse factor modelling: Applications in gene expression genomics. Journal of the American Statistical Association, 103, 1438–1456. doi:10.1198/016214508000000869
   PubMed Web of Science ®Google Scholar
• Choi, J., Oehlert, G., & Zou, H. (2010). A penalized maximum likelihood approach to sparse factor analysis. Statistics and Its Interface, 3, 429–436. doi:10.4310/SII.2010.v3.n4.a1
   Web of Science ®Google Scholar
• DiStefano, C. (2002). The impact of categorization with confirmatory factor analysis. Structural Equation Modeling: A The Impact of Categorization with Confirmatory Factor Analysis, 9, 37–41. doi:10.1207/S15328007SEM0903_2
   Google Scholar
• DiStefano, C., & Morgan, G. B. (2014). A comparison of diagonal weighted least squares robust estimation techniques for ordinal data. Structural Equation Modeling, 21, 425–438. doi:10.1080/10705511.2014.915373
   Web of Science ®Google Scholar
• Donoho, D., & Stodden, V. (2006). Breakdown point of model selection when the number of variables exceeds the number of observations. The 2006 IEEE International Joint Conference on Neural Network Proceedings (pp. 1916–1921). Vancouver, Canada. doi:10.1109/IJCNN.2006.246934
   Google Scholar
• Donoho, D. L. (2006). For most large underdetermined systems of linear equations the minimal L1-norm solution is also the sparsest solution. Communications on Pure and Applied Mathematics, 59, 797–829. arXiv: 0912.3599. doi:10.1002/cpa.20132
   Web of Science ®Google Scholar
• Ertel, S. (2011). Exploratory factor analysis revealing complex structure. Personality and Individual Differences, 50, 196–200. doi:10.1016/j.paid.2010.09.026
   Web of Science ®Google Scholar
• Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4, 272–299. arXiv: 99 [1082-989X].doi:10.1037/1082-989X.4.3.272
   Web of Science ®Google Scholar
• Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties variable selection via nonconcave penalized likelihood and its oracle prope. Source Journal of the American Statistical Association, 96, 1348–1360. doi:10.1198/016214501753382273
   Web of Science ®Google Scholar
• Genz, A., Bretz, F., Tetsuhisa, M., Xuefei, M., Leisch, F., Scheipl, F., & Hothorn, T. (2017). mvtnorm: Multivariate normal and t distributions. Retrieved from http://cran.r-project.org/package=mvtnorm
   Google Scholar
• Glymour, C., Madigan, D., Pregibon, D., & Smyth, P. (1997). Statistical themes and lessons for data mining. Data Mining and Knowledge Discovery, 1, 11–28. doi:10.1023/A:1009773905005
   Web of Science ®Google Scholar
• Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning. Elements, 1, 337–387. doi:10.1007/b94608
   Google Scholar
• Hendrickson, A. E., & White, P. O. (1964). PROMAX: A quick method for rotation to oblique simple structure. British Journal of Statistical Psychology, 17, 65–70. doi:10.1111/j.2044-8317.1964.tb00244.x
   Web of Science ®Google Scholar
• Hirose, K. (2016). Simple structure estimation via prenet penalization. arXiv: 1607.01145. Retrieved from http://arxiv.org/abs/1607.01145
   Google Scholar
• Hirose, K., & Konishi, S. (2012). Variable selection via the weighted group lasso for factor analysis models. Canadian Journal of Statistics, 40, 345–361. doi:10.1002/cjs.11129
   Web of Science ®Google Scholar
• Hirose, K., & Yamamoto, M. (2014). Estimation of an oblique structure via penalized likelihood factor analysis. Computational Statistics and Data Analysis, 79, 120–132. arXiv: 1302.5475. doi:10.1016/j.csda.2014.05.011
   Web of Science ®Google Scholar
• Hirose, K., & Yamamoto, M. (2015a). Sparse estimation via nonconcave penalized likelihood in factor analysis model. Statistics and Computing, 25, 863–875. arXiv: 1205.5868. doi:10.1007/s11222-014-9458-0
   Web of Science ®Google Scholar
• Hirose, K., & Yamamoto, M. (2015b). Sparse estimation via nonconcave penalized likelihood in factor analysis model. Statistics and Computing, 25, 863–875. arXiv: 1205.5868. doi:10.1007/s11222-014-9458-0
   Web of Science ®Google Scholar
• Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12, 55. doi:10.2307/1267351
   Web of Science ®Google Scholar
• Huang, P.-H., Chen, H., & Weng, L.-J. (2017a). A penalized likelihood method for structural equation modeling. Psychometrika, 82, 329–354. doi:10.1007/s11336-017-9566-9
   PubMed Web of Science ®Google Scholar
• Huang, P.-H., Chen, H., & Weng, L.-J. (2017b). A penalized likelihood method for structural equation modeling. Psychometrika, 82, 329–354. doi:10.1007/s11336-017-9566-9
   PubMed Web of Science ®Google Scholar
• Jacobucci, R. (2017). regsem: Regularized structural equation modeling, 1–13. arXiv: 1703.08489. Retrieved from http://arxiv.org/abs/1703.08489
   Google Scholar
• Jacobucci, R., & Grimm, K. J. (2018). Comparison of frequentist and Bayesian regularization in structural equation modeling. Structural Equation Modeling, 00, 1–11. doi:10.1080/10705511.2017.1410822
   Web of Science ®Google Scholar
• Jacobucci, R., Grimm, K. J., & McArdle, J. J. (2016). Regularized structural equation modeling. Structural Equation Modeling, 23, 555–566. arXiv: 15334406. doi:10.1080/10705511.2016.1154793
   PubMed Web of Science ®Google Scholar
• James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning. In G. Casella, S. Fienberg, & I. Olkin (Eds.), Springer texts in statistics. New York, NY: Springer New York. doi: 10.1007/978-1-4614-7138-7.
   Google Scholar
• Jennrich, R. I. (2004). Rotation to simple loadings using component loss functions: The orthogonal case. Psychometrika, 69, 257–273. doi:10.1007/BF02295943
   Web of Science ®Google Scholar
• Jennrich, R. I. (2006). Rotation to simple loadings using component loss functions: The oblique case. Psychometrika, 71, 173–191. doi:10.1007/s11336-003-1136-B
   Web of Science ®Google Scholar
• Jin, S., Moustaki, I., & Yang-Wallentin, F. (2018). Approximated penalized maximum likelihood for exploratory factor analysis: An orthogonal case. Psychometrika, 83, 628–649. doi:10.1007/s11336-018-9623-z
   Web of Science ®Google Scholar
• Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183–202. doi:10.1007/BF02289343
   Web of Science ®Google Scholar
• Jung, S., & Lee, S. (2011). Exploratory factor analysis for small samples. Behavior Research Methods, 43, 701–709. doi:10.3758/s13428-011-0077-9
   PubMed Web of Science ®Google Scholar
• Jung, S., & Takane, Y. (2007). Regularized common factor analysis. New Trends in Psychometrics, 1, 1–10.
   Google Scholar
• Kaiser, H. F. (1958). The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23, 187–200. doi:10.1007/BF02289233
   Web of Science ®Google Scholar
• Kaiser, H. F. (1959). Computer program for varimax rotation in factor analysis. Educational and Psychological Measurement, 19, 413–420. doi:10.1177/001316445901900314
   Web of Science ®Google Scholar
• Kosinski, M., Wang, Y., Lakkaraju, H., & Leskovec, J. (2016). Mining big data to extract patterns and predict real-life outcomes. Psychological Methods, 21, 493–506. arXiv: arXiv:1011.1669v3. doi:10.1037/met0000105
   PubMed Web of Science ®Google Scholar
• Lorenzo-Seva, U., & Ten Berge, J. M. F. (2006). Tucker’s congruence coefficient as a meaningful index of factor similarity. Methodology, 2, 57–64. doi:10.1027/1614-2241.2.2.57
   Google Scholar
• Mai, Y., Zhang, Z., & Wen, Z. (2018). Comparing exploratory structural equation modeling and existing approaches for multiple regression with latent variables. Structural Equation Modeling: A Multidisciplinary Journal, 25, 737–749. doi:10.1080/10705511.2018.1444993
   Web of Science ®Google Scholar
• Marcoulides, G. A., & Drezner, Z. (2003). Model specification searches using ant colony optimization algorithms. Structural Equation Modeling: A Multidisciplinary Journal, 10, 154–164. doi:10.1207/S15328007SEM1001_8
   Web of Science ®Google Scholar
• Marcoulides, G. A., Drezner, Z., & Schumacker, R. E. (1998). Model specification searches in structural equation modeling using tabu search. Structural Equation Modeling: A Multidisciplinary Journal, 5, 365–376. doi:10.1080/10705519809540112
   Web of Science ®Google Scholar
• Marcoulides, G. A., & Ing, M. (2013). The use of generalizability theory in language assessment. In A. J. Kunnan (Ed.), The companion to language assessment (pp. 1207–1223). Hoboken, NJ, USA: John Wiley & Sons, Inc. doi:10.1002/9781118411360.wbcla014.
   Google Scholar
• Marcoulides, K. M., & Falk, C. F. (2018). Model specification searches in structural equation modeling with R. Structural Equation Modeling: A Multidisciplinary Journal, 00, 1–8. doi:10.1080/10705511.2017.1409074
   Web of Science ®Google Scholar
• Marsh, H. W., Lüdtke, O., Muthén, B., Asparouhov, T., Morin, A. J. S., Trautwein, U., & Nagengast, B. (2010). A new look at the big five factor structure through exploratory structural equation modeling. Psychological Assessment, 22, 471–491. doi:10.1037/a0019227
   PubMed Web of Science ®Google Scholar
• Marsh, H. W., Muthén, B., Asparouhov, T., Lüdtke, O., Robitzsch, A., Morin, A. J. S., & Trautwein, U. (2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to students’ evaluations of university teaching. Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 439–476. doi:10.1080/10705510903008220
   Google Scholar
• Morin, A. J. S., Marsh, H. W., & Nagengast, B. (2013). Exploratory structural equation modeling: An introduction. In G. R. Hancock & R. O. Mueller (Eds.), Structural Equation Modeling: A Second Course (pp. 395–436; 2nd ed.). Charlotte, NC: Information Age Publishing, Inc.
   Google Scholar
• Mulaik, S. A. (2010). Foundations of factor analysis (2nd ed.). Boca Raton, FL: CRC press.
   Google Scholar
• Muthén, B., & Asparouhov, T. (2011). Bayesian SEM: A more flexible representation of substantive theory web tables. Psychological Methods, 17, 313–335. doi:10.1037/a0026802
   Web of Science ®Google Scholar
• Muthén, B. O. (2004). Mplus technical appendices. Los Angeles, CA: Muthén & Muthén. Retrieved from https://www.statmodel.com/download/techappen.pdf
   Google Scholar
• Ning, L., & Georgiou, T. T. (2011). Sparse factor analysis via likelihood and L1-regularization. Proceedings of the IEEE Conference on Decision and Control (pp. 5188–5192). Orlando, FL. doi:10.1109/CDC.2011.6161415
   Google Scholar
• R Core Team. (2018). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.r-project.org/
   Google Scholar
• Rabe-Hesketh, S., Skrondal, A., & Zheng, X. (2007). Multilevel structural equation modeling. North-Holland, The Netherlands: Elsevier B.V. doi:10.1016/B978-044452044-9/50013-6
   Google Scholar
• Revelle, W. (2016). psych: Procedures for psychological, psychometric, and personality research. Evanston, IL: Northwestern University.
   Google Scholar
• Sass, D. A., & Schmitt, T. A. (2010). A comparative investigation of rotation criteria within exploratory factor analysis. Multivariate Behavioral Research, 45, 73–103. doi:10.1080/00273170903504810
   PubMed Web of Science ®Google Scholar
• Schmitt, T. A., & Sass, D. A. (2011). Rotation criteria and hypothesis testing for exploratory factor analysis: Implications for factor pattern loadings and interfactor correlations. Educational and Psychological Measurement, 71, 95–113. doi:10.1177/0013164410387348
   Web of Science ®Google Scholar
• Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461–464. doi:10.1214/aos/1176344136
   Web of Science ®Google Scholar
• Thurstone, L. L. (1935). The vectors of mind: Multiple-factor analysis for the isolation of primary traits. Chicago, IL: University of Chicago Press. doi:10.1037/10018-000
   Google Scholar
• Thurstone, L. L. (1954). An analytical method for simple structure. Psychometrika, 19, 173–182. doi:10.1007/BF02289182
   Google Scholar
• Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 267–288. doi:10.1111/j.2517-6161.1996.tb02080.x
   Web of Science ®Google Scholar
• Trendafilov, N. T. (2014). From simple structure to sparse components: A review. Computational Statistics, 29, 431–454. doi:10.1007/s00180-013-0434-5
   Web of Science ®Google Scholar
• Trendafilov, N. T., & Adachi, K. (2015). Sparse versus simple structure loadings. Psychometrika, 80, 776–790. doi:10.1007/s11336-014-9416-y
   PubMed Web of Science ®Google Scholar
• Trendafilov, N. T., Fontanella, S., & Adachi, K. (2017). Sparse exploratory factor analysis. Psychometrika. doi:10.1007/s11336-017-9575-8
   Web of Science ®Google Scholar
• Yamamoto, M., Hirose, K., & Nagata, H. (2017). Graphical tool of sparse factor analysis. Behaviormetrika, 44, 229–250. doi:10.1007/s41237-016-0007-3
   Google Scholar
• Yamamoto, M., & Jennrich, R. I. (2013). A cluster-based factor rotation. British Journal of Mathematical and Statistical Psychology, 66, 488–502. doi:10.1111/bmsp.12007
   PubMed Web of Science ®Google Scholar
• Yates, A. (1987). Multivariate exploratory data analysis: A perspective on exploratory factor analysis. Albany. NY, USA: State Univ. of New York Press.
   Google Scholar
• Zhang, C. H. (2010). Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 38(2), 894–942. doi:10.1214/09-AOS729
   Google Scholar
• Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society. Series B: Statistical Methodology, 67, 301–320. doi:10.1111/j.1467-9868.2005.00503.x
   Web of Science ®Google Scholar