Analyzing and Comparing Univariate, Multivariate, and Bifactor Generalizability Theory Designs for Hierarchically Structured Personality Traits

References

• American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME) (2014). Standards for educational and psychological testing. American Educational Research Association.
   Google Scholar
• American Psychological Association (APA) (1954). Technical recommendations for psychological tests and diagnostic techniques. Psychological Bulletin, 51(2, Pt.2), 1–38. https://doi.org/10.1037/h0053479
   Google Scholar
• Ark, T. K. (2015). Ordinal generalizability theory using an underlying latent variable framework [Doctoral dissertation]. University of British Columbia.
   Google Scholar
• Brennan, R. L. (2001). Generalizability theory. Springer-Verlag.
   Google Scholar
• Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555
   Web of Science ®Google Scholar
• Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. Wiley.
   Google Scholar
• Cronbach, L. J., Rajaratnam, N., & Gleser, G. C. (1963). Theory of generalizability: A liberalization of reliability theory. British Journal of Statistical Psychology, 16(2), 137–163. https://doi.org/10.1111/j.2044-8317.1963.tb00206.x
   Web of Science ®Google Scholar
• Cronbach, L. J., Schönemann, P., & McKie, D. (1965). Alpha coefficients for stratified-parallel tests. Educational and Psychological Measurement, 25(2), 291–312. https://doi.org/10.1177/001316446502500201
   Web of Science ®Google Scholar
• DeYoung, C. G., Quilty, L. C., & Peterson, J. B. (2007). Between facets and domains: 10 aspects of the Big Five. Journal of Personality and Social Psychology, 93(5), 880–896. https://doi.org/10.1037/0022-3514.93.5.880
   PubMed Web of Science ®Google Scholar
• Feinberg, R. A., & Wainer, H. (2014). A simple equation to predict a subscore’s value. Educational Measurement: Issues and Practice, 33(3), 55–56. https://doi.org/10.1111/emip.12035
   Web of Science ®Google Scholar
• Geiser, C., & Lockhart, G. (2012). A comparison of four approaches to account for method effects in latent state-trait analyses. Psychological Methods, 17(2), 255–283. https://doi.org/10.1037/a0026977
   PubMed Web of Science ®Google Scholar
• Haberman, S. J. (2008). When can subscores have value? Journal of Educational and Behavioral Statistics, 33(2), 204–229. https://doi.org/10.3102/1076998607302636
   Web of Science ®Google Scholar
• John, O. P., Donahue, E. M., & Kentle, R. L. (1991). The Big Five Inventory—versions 4a and 54. University of California, Berkeley, Institute of Personality and Social Research.
   Google Scholar
• Jorgensen, T. D. (2021). How to estimate absolute-error components in structural equation models of generalizability theory. Psych, 3(2), 113–133. https://doi.org/10.3390/psych3020011
   Google Scholar
• Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., Rosseel, Y. (2022). semTools: Useful tools for structural equation modeling. R package version 0.5-6. https://CRAN.R-project.org/package=semTools
   Google Scholar
• Kumar, S. S., Merkin, A. G., Numbers, K., Sachdev, P. S., Brodaty, H., Kochan, N. A., Trollor, J. N., Mahon, S., & Medvedev, O. (2022). A novel approach to investigate depression symptoms in the aging population using generalizability theory. Psychological Assessment, 34(7), 684–696. https://doi.org/10.1037/pas0001129
   PubMed Web of Science ®Google Scholar
• Le, H., Schmidt, F. L., & Putka, D. J. (2009). The multifaceted nature of measurement artifacts and its implications for estimating construct-level relationships. Organizational Research Methods, 12(1), 165–200. https://doi.org/10.1177/1094428107302900
   Web of Science ®Google Scholar
• Lord, F. M. (1956). Sampling error due to choice of split in split-half reliability coefficients. The Journal of Experimental Education, 24(3), 245–249. https://doi.org/10.1080/00220973.1956.11010545
   Web of Science ®Google Scholar
• Lyndon, M. P., Medvedev, O. N., Chen, Y., & Henning, M. A. (2020). Investigating stable and dynamic aspects of student motivation using generalizability theory. Australian Journal of Psychology, 72(2), 199–210. https://doi.org/10.1111/ajpy.12276
   Web of Science ®Google Scholar
• McCrae, R. R., & Costa, P. T. (2010). NEO Inventories professional manual. Psychological Assessment Resources.
   Google Scholar
• Medvedev, O. N., Krägeloh, C. U., Narayanan, A., & Siegert, R. J. (2017). Measuring mindfulness: Applying generalizability theory to distinguish between state and trait. Mindfulness, 8(4), 1036–1046. https://doi.org/10.1007/s12671-017-0679-0
   Web of Science ®Google Scholar
• Miller, Y. R., Medvedev, O. N., Hwang, Y.-S., & Singh, N. N. (2021). Applying generalizability theory to the Perceived Stress Scale to evaluate stable and dynamic aspects of educators’ stress. International Journal of Stress Management, 28(2), 147–153. https://doi.org/10.1037/str0000207
   Web of Science ®Google Scholar
• Paterson, J., Medvedev, O. N., Sumich, A., Tautolo, E.-S., Krägeloh, C. U., Sisk, R., McNamara, R. K., Berk, M., Narayanan, A., & Siegert, R. J. (2018). Distinguishing transient versus stable aspects of depression in New Zealand Pacific Island children using generalizability theory. Journal of Affective Disorders, 227, 698–704. https://doi.org/10.1016/j.jad.2017.11.075
   PubMed Web of Science ®Google Scholar
• Rajaratnam, N., Cronbach, L. J., & Gleser, G. C. (1965). Generalizability of stratified-parallel tests. Psychometrika, 30(1), 39–56. https://doi.org/10.1007/BF02289746
   PubMed Web of Science ®Google Scholar
• Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95(2), 129–140. https://doi.org/10.1080/00223891.2012.725437
   PubMed Web of Science ®Google Scholar
• Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016a). Applying bifactor statistical indices in the evaluation of psychological measures. Journal of Personality Assessment, 98(3), 223–237. https://doi.org/10.1080/00223891.2015.1089249
   PubMed Web of Science ®Google Scholar
• Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016b). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137–150. https://doi.org/10.1037/met0000045
   PubMed Web of Science ®Google Scholar
• Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02
   Web of Science ®Google Scholar
• Saucier, G. (1994). Mini-markers: A brief version of Goldberg’s unipolar Big-Five markers. Journal of Personality Assessment, 63(3), 506–516. https://doi.org/10.1207/s15327752jpa6303_8
   PubMed Web of Science ®Google Scholar
• Schmidt, F. L., Le, H., & Ilies, R. (2003). Beyond alpha: An empirical investigation of the effects of different sources of measurement error on reliability estimates for measures of individual differences constructs. Psychological Methods, 8(2), 206–224. https://doi.org/10.1037/1082-989X.8.2.206
   PubMed Web of Science ®Google Scholar
• Soto, C. J., & John, O. P. (2017). The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power. Journal of Personality and Social Psychology, 113(1), 117–143. https://doi.org/10.1037/pspp0000096
   PubMed Web of Science ®Google Scholar
• Steyer, R., Ferring, D., & Schmitt, M. J. (1992). States and traits in psychological assessment. European Journal of Psychological Assessment, 8(2), 79–98. https://doi.org/10.1027/1015-5759/a000413
   Web of Science ®Google Scholar
• Thorndike, R. L. (1951). Reliability. In E. F. Lindquist (Ed.), Educational measurement (pp. 560–620). American Council on Education.
   Google Scholar
• Tryon, R. C. (1957). Reliability and behavior domain validity: Reformulation and historical critique. Psychological Bulletin, 54(3), 229–249. https://doi.org/10.1037/h0047980
   PubMed Web of Science ®Google Scholar
• van Bork, R., Rhemtulla, M., Sijtsma, K., & Borsboom, D. (2022). A causal theory of error scores. Psychological Methods, 1–20. Advance online publication. https://doi.org/10.1037/met0000521
   Web of Science ®Google Scholar
• Vispoel, W. P., Hong, H., & Lee, H. (2023). Benefits of doing generalizability theory analyses within structural equation modeling frameworks: Illustrations using the Rosenberg Self-Esteem Scale [Teacher’s corner]. Structural Equation Modeling: A Multidisciplinary Journal, 1–17. Advance online publication. https://doi.org/10.1080/10705511.2023.2187734
   Web of Science ®Google Scholar
• Vispoel, W. P., Hong, H., Lee, H., & Jorgensen, T. R. (in press). Analyzing complete generalizability theory designs using structural equation models. Applied Measurement in Education,
   Web of Science ®Google Scholar
• Vispoel, W. P., & Lee, H. (2023). Merging generalizability theory and bifactor modeling to improve psychological assessments. Psychology and Psychotherapy: Review Study, 7, 1–4. https://crimsonpublishers.com/pprs/pdf/PPRS.000652.pdf
   Google Scholar
• Vispoel, W. P., Lee, H., Chen, T., & Hong, H. (2023a). Using structural equation modeling techniques to reproduce and extend ANOVA-based generalizability theory analyses for psychological assessments. Psych, 5(2), 249–273. https://doi.org/10.3390/psych5020019
   Google Scholar
• Vispoel, W. P., Lee, H., Chen, T., & Hong, H. (2023b). Extending applications of generalizability theory-based bifactor model designs. Psych, 5(2), 545–575. https://doi.org/10.3390/psych5020036
   Google Scholar
• Vispoel, W. P., Lee, H., Hong, H., & Chen, T. (2023). Applying multivariate generalizability theory to psychological assessments. Psychological Methods, 1–23. Advance online publication. https://doi.org/10.1037/met0000606
   PubMed Web of Science ®Google Scholar
• Vispoel, W. P., Lee, H., & Hong, H. (2023). Analyzing multivariate generalizability theory designs within structural equation modeling frameworks [Teacher’s corner]. Structural Equation Modeling: A Multidisciplinary Journal, 1–19. Advance online publication. https://doi.org/10.1080/10705511.2023.2222913
   Web of Science ®Google Scholar
• Vispoel, W. P., Lee, H., Xu, G., & Hong, H. (2022). Expanding bifactor models of psychological traits to account for multiple sources of measurement error. Psychological Assessment, 34(12), 1093–1111. https://doi.org/10.1037/pas0001170
   PubMed Web of Science ®Google Scholar
• Vispoel, W. P., Lee, H., Xu, G., & Hong, H. (2023). Integrating bifactor models into a generalizability theory based structural equation modeling framework. The Journal of Experimental Education, 91(4), 718–738. https://doi.org/10.1080/00220973.2022.2092833
   Web of Science ®Google Scholar
• Vispoel, W. P., Morris, C. A., & Kilinc, M. (2018a). Applications of generalizability theory and their relations to classical test theory and structural equation modeling. Psychological Methods, 23(1), 1–26. https://doi.org/10.1037/met0000107
   PubMed Web of Science ®Google Scholar
• Vispoel, W. P., Morris, C. A., & Kilinc, M. (2018b). Practical applications of generalizability theory for designing, evaluating, and improving psychological assessments. Journal of Personality Assessment, 100(1), 53–67. https://doi.org/10.1080/00223891.2017.1296455
   PubMed Web of Science ®Google Scholar
• Vispoel, W. P., Morris, C. A., & Kilinc, M. (2018c). Using G-theory to enhance evidence of reliability and validity for common uses of the Paulhus Deception Scales. Assessment, 25(1), 69–83. https://doi.org/10.1177/1073191116641182
   PubMed Web of Science ®Google Scholar
• Vispoel, W. P., Morris, C. A., & Kilinc, M. (2018d). Using generalizability theory to disattenuate correlation coefficients for multiple sources of measurement error. Multivariate Behavioral Research, 53(4), 481–501. https://doi.org/10.1080/00273171.2018.1457938
   PubMed Web of Science ®Google Scholar
• Vispoel, W. P., Morris, C. A., & Kilinc, M. (2019). Using generalizability theory with continuous latent response variables. Psychological Methods, 24(2), 153–178. https://doi.org/10.1037/met0000177
   PubMed Web of Science ®Google Scholar
• Vispoel, W. P., & Tao, S. (2013). A generalizability analysis of score consistency for the Balanced Inventory of Desirable Responding. Psychological Assessment, 25(1), 94–104. https://doi.org/10.1037/a0029061
   PubMed Web of Science ®Google Scholar
• Vispoel, W. P., Xu, G., & Kilinc, M. (2021). Expanding G-theory models to incorporate congeneric relationships: Illustrations using the Big Five Inventory. Journal of Personality Assessment, 103(4), 429–442. https://doi.org/10.1080/00223891.2020.1808474
   PubMed Web of Science ®Google Scholar
• Vispoel, W. P., Xu, G., & Schneider, W. S. (2022a). Interrelationships between latent state-trait theory and generalizability theory in a structural equation modeling framework. Psychological Methods, 27(5), 773–803. https://doi.org/10.1037/met0000290
   PubMed Web of Science ®Google Scholar
• Vispoel, W. P., Xu, G., & Schneider, W. S. (2022b). Using parallel splits with self-report and other measures to enhance precision in generalizability theory analyses. Journal of Personality Assessment, 104(3), 303–319. https://doi.org/10.1080/00223891.2021.1938589
   PubMed Web of Science ®Google Scholar
• Wan, C., Chen, Y., Gao, L., Zhang, Q., Quan, P., & Sun, X. (2020). Development and validation of the peptic ulcer scale under the system of quality of life instruments for chronic diseases based on classical test theory and generalizability theory. BMC Gastroenterology, 20(1), 422. https://doi.org/10.1186/s12876-020-01562-y
   PubMed Web of Science ®Google Scholar