Using Exploratory Bifactor Analysis to Understand the Latent Structure of Multidimensional Psychological Measures: An Example Featuring the WISC-V

References

• Ackerman, R. A., Donnellan, M. B., & Robins, R. W. (2012). An item response theory analysis of the narcissistic personality inventory. Journal Of Personality Assessment, 94, 141–155. doi:10.1080/00223891.2011.645934
   PubMed Web of Science ®Google Scholar
• Baltes, P. B., Cornelius, S. W., Spiro, A., Nesselroade, J. R., & Willis, S. L. (1980). Integration versus differentiation of fluid/crystallized intelligence in old age. Developmental Psychology, 16, 625–635. doi:10.1037/0012-1649.16.6.625
   Web of Science ®Google Scholar
• Beaujean, A. A. (2013). Factor analysis using R. Practical Assessment, Research, and Evaluation, 18, 1–11. Retrieved from: http://pareonline.net/pdf/v18n4.pdf
   Google Scholar
• Beaujean, A. A. (2014). R syntax to accompany best practices in exploratory factor analysis (2014) by Jason Osborne. Retrieved from: https://dl.dropboxusercontent.com/u/18489687/EFAbook/BestPracticesEFA_Rcode.pdf
   Google Scholar
• Beaujean, A. A. (2015). John Carroll’s views on intelligence: Bi-factor vs. higher-order models. Journal of Intelligence, 3, 121–136. doi:10.3390/jintelligence3040121
   Google Scholar
• Bernaards, C. A., & Jennrich, R. I. (2005). Gradient projection algorithms and software for arbitrary rotation criteria in factor analysis. Educational and Psychological Measurement, 65, 676–696. doi:10.1177/0013164404272507
   Web of Science ®Google Scholar
• Bonifay, W., & Cai, L. (2017). On the complexity of item response theory models. Multivariate Behavioral Research, 52, 465–484. doi:10.1080/00273171.2017.1309262
   PubMed Web of Science ®Google Scholar
• Bonifay, W., Lane, S. P., & Reise, S. P. (2017). Three concerns with applying a bifactor model as a structure of psychopathology. Clinical Psychological Science, 5, 184–186. doi:10.1177/2167702616657069
   Web of Science ®Google Scholar
• Brouwer, D., Meijer, R. R., & Zevalkink, J. (2013). On the factor structure of the Beck Depression Inventory–II: G is the key. Psychological Assessment, 25, 136–145. doi:10.1037/a0029228
   PubMed Web of Science ®Google Scholar
• Browne, M. W. (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36, 111–150. doi:10.1207/s15327906mbr3601_05
   Web of Science ®Google Scholar
• Canivez, G. L. (2016). Bifactor modeling in construct validation of multifactored tests: Implications for understanding multidimensional constructs and test interpretation. In K. Schweizer & C. DiStefano (Eds.), Principles and methods of test construction: Standards and recent advancements (pp. 247–271). Gottingen, Germany: Hogrefe.
   Google Scholar
• Canivez, G. L., & Watkins, M. W. (2016). Review of the Wechsler Intelligence Scale for Children–Fifth Edition: Critique, commentary, and independent analyses. In A. S. Kaufman, S. E. Raiford, & D. L. Coalson (Eds.), Intelligent testing with the WISC-V (pp. 683–702). Hoboken, NJ: Wiley.
   Google Scholar
• Canivez, G. L., & Kush, J. C. (2013). WAIS-IV and WISC-IV structural validity: Alternate methods, alternate results. Commentary on Weiss et al. (2013a) and Weiss et al. (2013b). Journal of Psychoeducational Assessment, 31, 157–169. doi:10.1177/0734282913478036
   Web of Science ®Google Scholar
• Canivez, G. L, & Watkins, M. W. (2010a). Exploratory and higher-order factor analyses of the Wechsler Intelligence Scale–Fourth Edition (WAIS–IV) adolescent subsample. School Psychology Quarterly, 25, 223–235. doi:10.1037/a0022046
   Web of Science ®Google Scholar
• Canivez, G. L, & Watkins, M. W. (2010b). Investigation of the factor structure of the Wechsler Intelligence Scale–Fourth Edition (WAIS–IV): Exploratory and higher-order factor analyses. Psychological Assessment, 22, 827–836. doi:10.1037/a0020429
   PubMed Web of Science ®Google Scholar
• Canivez, G. L., Watkins, M. W., & Dombrowski, S. C. (2016). Factor structure of the Wechsler Intelligence Scale for Children–Fifth Edition: Exploratory factor analyses with the 16 primary and secondary subtests. Psychological Assessment, 28, 975–986. doi:10.1037/pas0000238
   PubMed Web of Science ®Google Scholar
• Canivez, G. L., Watkins, M. W., & Dombrowski, S. C. (2017). Structural validity of the Wechsler Intelligence Scale for Children–Fifth Edition: Confirmatory factor analyses with the 16 primary and secondary subtests. Psychological Assessment, 29, 458–472. doi:10.1037/pas0000358
   PubMed Web of Science ®Google Scholar
• Carroll, J. B. (1983). Studying individual differences in cognitive abilities: Through and beyond factor analysis. In R. F. Dillon & R. R. Schmeck (Eds.), Individual differences in cognition (pp. 1–33). New York, NY: Academic Press.
   Google Scholar
• Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. New York, NY: Cambridge University Press.
   Google Scholar
• Chen, F. F., West, S. G., & Sousa, K. H. (2006). A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41, 189–225. doi:10.1207/s15327906mbr4102_5
   PubMed Web of Science ®Google Scholar
• Cliff, N. (1983). Some cautions concerning the application of causal modeling methods. Multivariate Behavioral Research, 18, 115–126. doi:10.1207/s15327906mbr1801_7
   PubMed Web of Science ®Google Scholar
• Dombrowski, S. C. (2013). Investigating the structure of the WJ III Cognitive at school age. School Psychology Quarterly, 28, 154–169. doi:10.1037/spq0000010
   PubMed Web of Science ®Google Scholar
• Dombrowski, S. C. (2014a). Exploratory bifactor analysis of the WJ III Cognitive in adulthood via the Schmid-Leiman procedure. Journal of Psychoeducational Assessment, 32, 330–341. doi:10.1177/0734282913508243
   Web of Science ®Google Scholar
• Dombrowski, S. C. (2014b). Investigating the structure of the WJ III Cognitive in early school age through two exploratory bifactor analysis procedures. Journal of Psychoeducational Assessment, 32, 483–494. doi:10.1177/0734282914530838
   Web of Science ®Google Scholar
• Dombrowski, S. C., Canivez, G. L., & Watkins, M. W. (2018). Factor structure of the 10 WISC–V primary subtests in four standardization age groups. Contemporary School Psychology, 22, 90–104. doi: 10.1007/s40688-017-0125-2.
   Google Scholar
• Dombrowski, S. C., Canivez, G. L., Watkins, M. W., & Beaujean, A. (2015). Exploratory bifactor analysis of the Wechsler Intelligence Scale for Children—Fifth Edition with the 16 primary and secondary subtests. Intelligence, 53, 194–201. doi:10.1016/j.intell.2015.10.009
   Web of Science ®Google Scholar
• Dombrowski, S. C., & Watkins, M. W. (2013). Exploratory and higher order factor analysis of the WJ III full test battery: A school aged analysis. Psychological Assessment, 25, 442–455. doi:10.1037/a0031335
   PubMed Web of Science ®Google Scholar
• Dombrowski, S. C., Watkins, M. W., & Brogan, M. J. (2009). An exploratory investigation of the factor structure of the Reynolds Intellectual Assessment Scales (RIAS). Journal of Psychoeducational Assessment, 27, 494–507. doi:10.1177/0734282909333179
   Web of Science ®Google Scholar
• Dziuban, C. D., & Shirkey, E. C. (1974). When is a correlation matrix appropriate for factor analysis? Some decision rules. Psychological Bulletin, 81, 358–361. doi:10.1037/h0036316
   Web of Science ®Google Scholar
• Ebesutani, C., McLeish, A. C., Luberto, C. M., Young, J., & Maack, D. J. (2014). A bifactor model of anxiety sensitivity: Analysis of the Anxiety Sensitivity Index–3. Journal of Psychopathology and Behavioral Assessment, 36, 452–464. doi:10.1007/s10862-013-9400-3
   Web of Science ®Google Scholar
• Flora, D. B., LaBrish, C., & Chalmers, R. P. (2012). Old and new ideas for data screening and assumption testing for exploratory and confirmatory factor analysis. Frontiers in Psychology, 3, 1–21. doi:10.3389/fpsyg.2012.00055
   PubMed Web of Science ®Google Scholar
• Gelman, A., & Loken, E. (2013). The garden of forking paths: Why multiple comparisons can be a problem when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited a head of time. Location: Author. Unpublished manuscript available at http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf
   Google Scholar
• Gignac, G. E. (2005). Revisiting the factor structure of the WAIS-R: Insights through nested factor modeling. Assessment, 12, 320–329. doi:10.1177/1073191105278118
   PubMed Web of Science ®Google Scholar
• Gignac, G. E. (2008). Higher-order models versus direct hierarchical models: G as superordinate or breadth factor? Psychology Science Quarterly, 50, 21–43.
   Google Scholar
• Gignac, G. E., & Watkins, M. W. (2013). Bifactor modeling and the estimation of model-based reliability in the WAIS-IV. Multivariate Behavioral Research, 48, 639–662. doi:10.1080/00273171.2013.804398
   PubMed Web of Science ®Google Scholar
• Holzinger, K. J., & Swineford, F. (1937). The bi-factor method. Psychometrika, 2, 41–54.
   Google Scholar
• Jennrich, R. I., & Bentler, P. M. (2011). Exploratory bi-factor analysis. Psychometrika, 6, 537–549.
   Google Scholar
• Jennrich, R. I., & Bentler, P. M. (2012). Exploratory bi-factor analysis: The oblique case. Psychometrika, 77, 442–454. doi:10.1007/s11336-012-9269-1
   PubMed Web of Science ®Google Scholar
• Loehlin, J. C., & Beaujean, A. A. (2016a). Latent variable models: An introduction to factor, path, and structural equation analysis (5th ed.). New York, NY: Routledge.
   Google Scholar
• Loehlin, J. C., & Beaujean, A. A. (2016b). Syntax companion for Latent variable models: An introduction to factor, path, and structural equation analysis (5th ed.). New York, NY: Routledge.
   Google Scholar
• Mansolf, M., & Reise, S. P. (2015). Local minima in exploratory bifactor analysis. Multivariate Behavioral Research, 50, 738. doi:10.1080/00273171.2015.1121127
   Web of Science ®Google Scholar
• Mansolf, M., & Reise, S. P. (2016). Exploratory bifactor analysis: The Schmid-Leiman orthogonalization and Jennrich-Bentler analytic rotations. Multivariate Behavioral Research, 51, 698–717. doi:10.1080/00273171.2016.1215898
   PubMed Web of Science ®Google Scholar
• Myers, N. D., Martin, J. J., Ntoumanis, N., Celimli, S., & Bartholomew, K. J. (2014). Exploratory bifactor analysis in sport, exercise, and performance psychology: A substantive-methodological synergy. Sport, Exercise, and Performance Psychology, 3, 258–272. doi:10.1037/spy0000015
   Web of Science ®Google Scholar
• Office for the Management and Budget of the White House. (2003). Circular A4. Washington, DC: Author.
   Google Scholar
• Paxton, P., Curran, P. J., Bollen, K. A., Kirby, J., & Chen, F. (2001). Monte Carlo experiments: Design and implementation. Structural Equation Modeling, 8, 287–312. doi:10.1207/S15328007SEM0802_7
   Web of Science ®Google Scholar
• R Development Core Team. (2019). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
   Google Scholar
• Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47, 667–696. doi:10.1080/00273171.2012.715555
   PubMed Web of Science ®Google Scholar
• Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95, 129–140. doi:10.1080/00223891.2012.725437
   PubMed Web of Science ®Google Scholar
• Reise, S. P., Moore, T. M., & Haviland, M. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scores. Journal of Personality Assessment, 92, 544–559. doi:10.1080/00223891.2010.496477
   PubMed Web of Science ®Google Scholar
• Revelle, W. (2012). psych: Procedures for psychological, psychometric, and personality research (version 1.2.4) [computer software]. Evanston, IL: Northwestern University.
   Google Scholar
• Revelle, W., & Wilt, J. (2013). The general factor of personality: A general critique. Journal of Research in Personality, 47, 493–504. doi:10.1016/j.jrp.2013.04.012
   PubMed Web of Science ®Google Scholar
• Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016). Applying bifactor statistical indices in the evaluation of psychological measures. Journal of Personality Assessment, 98, 223–237. doi:10.1080/00223891.2015.1089249
   PubMed Web of Science ®Google Scholar
• Saltelli, A., Tarantola, S., Campolongo, F., & Ratto, M. (2004). Sensitivity analysis in practice: A guide to assessing scientific models. Hoboken, NJ: Wiley.
   Google Scholar
• Sass, D. A., & Schmitt, T. A. (2010). A comparative investigation of rotation criteria within exploratory factor analysis. Multivariate Behavioral Research, 45, 73–103. doi:10.1080/00273170903504810
   PubMed Web of Science ®Google Scholar
• Schmid, J., & Leiman, J. M. (1957). The development of hierarchical factor solutions. Psychometrika, 22, 53–61. doi:10.1007/BF02289209
   Web of Science ®Google Scholar
• Watkins, M. W, Dombrowski, S. C, & Canivez, G. L. (2018). Reliability and factorial validity of the Canadian Wechsler Intelligence Scale for Children–Fifth Edition. International Journal Of School And Educational Psychology, 6, 252–265. doi: 10.1080/21683603.2017.1342580
   Google Scholar
• Wechsler, D. (2014b). Wechsler intelligence scale for children–fifth edition technical and interpretive manual. Bloomington, MN: Pearson.
   Google Scholar
• Widaman, K. F., & Herringer, L. G. (1985). Iterative least squares estimates of communality: Initial estimate need not affect stabilized value. Psychometrika, 50, 469–477. doi:10.1007/BF02296264
   Web of Science ®Google Scholar
• Wood, J. M., Tataryn, D. J., & Gorsuch, R. L. (1996). Effects of under- and overextraction on principal axis factor analysis with varimax rotation. Psychological Methods, 1, 354–365. doi:10.1037/1082-989X.1.4.354
   Web of Science ®Google Scholar
• Yung, Y.-F., Thissen, D., & McLeod, L. (1999). On the relationship between the higher-order factor model and the hierarchical factor model. Psychometrika, 64, 113–128. doi:10.1007/BF02294531
   Web of Science ®Google Scholar