Choosing Between Cronbach’s Coefficient Alpha, McDonald’s Coefficient Omega, and Coefficient H: Confidence Intervals and the Advantages and Drawbacks of Interpretive Guidelines

References

• American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing (U.S.). (2014). Standards for educational and psychological testing. https://www.apa.org/science/programs/testing/standards
   Google Scholar
• Cheung, G. W., Cooper-Thomas, H. D., Lau, R. S., & Wang, L. C. (2023). Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations. Asia Pacific Journal of Management. Advanced online publication. https://doi.org/10.1007/s10490-023-09871-y
   Google Scholar
• Cho, E. (2016). Making reliability reliable: A systematic approach to reliability coefficients. Organizational Research Methods, 19(4), 651–682. https://doi.org/10.1177/1094428116656239
   Web of Science ®Google Scholar
• Cho, E. (2022). The accuracy of reliability coefficients: A reanalysis of existing simulations. Psychological Methods. Advanced online publication. https://doi.org/10.1037/met0000475
   Web of Science ®Google Scholar
• Cho, E., & Kim, S. (2015). Cronbach’s coefficient alpha: Well known but poorly understood. Organizational Research Methods, 18(2), 207–230. https://doi.org/10.1177/1094428114555994
   Web of Science ®Google Scholar
• Cortina, J. M., Sheng, Z., Keener, S. K., Keeler, K. R., Grubb, L. K., Schmitt, N., Tonidandel, S., Summerville, K. M., Heggestad, E. D., & Banks, G. C. (2020). From alpha to omega and beyond! A look at the past, present, and (possible) future of psychometric soundness in the Journal of Applied Psychology. The Journal of Applied Psychology, 105(12), 1351–1381. https://doi.org/10.1037/apl0000815
   PubMed Web of Science ®Google Scholar
• Council for Accreditation of Counseling and Related Educational Programs. (2024). 2024 CACREP Standards. Author.
   Google Scholar
• Creswell, J. W., & Creswell, D. J. (2023). Research design: Qualitative, quantitative, and mixed methods approaches (6th ed.). Sage.
   Google Scholar
• Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555
   Web of Science ®Google Scholar
• Deng, L., & Chan, W. (2017). Testing the difference between reliability coefficients alpha and omega. Educational and Psychological Measurement, 77(2), 185–203. https://doi.org/10.1177/0013164416658325
   PubMed Web of Science ®Google Scholar
• Doval, E., Viladrich, C., & Angulo-Brunet, A. (2023). Coefficient alpha: The resistance of a classic. Psicothema, 35(1), 5–20. https://doi.org/10.7334/psicothema2022.321
   PubMed Web of Science ®Google Scholar
• Dunn, T., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. https://doi.org/10.1111/bjop.12046
   PubMed Web of Science ®Google Scholar
• Edwards, A. A., Joyner, K. J., & Schatschneider, C. (2021). A simulation study on the performance of different reliability estimation methods. Educational and Psychological Measurement, 81(6), 1089–1117. https://doi.org/10.1177/0013164421994184
   PubMed Web of Science ®Google Scholar
• Egberink, I. J. L., Janssen, N. A. M., & Vermeulen, C. S. M. (2009–2016). COTAN Documentatie [COTAN documentation]. Boom. https://www.cotandocumentatie.nl/
   Google Scholar
• Evers, A., Hagemeister, C., Hostmalingen, A., Lindley, P., Muniz, J., Sjoberg, A. (2013). EFPA review model for the description and evaluation of psychological and educational tests. Test review form and notes for reviewers. Version 4.2.6. https://psyktestbarn.r-bup.no/cms/files/913
   Google Scholar
• Flora, D. B. (2020). Your coefficient alpha is probably wrong, but which coefficient omega is right? A tutorial on using R to obtain better reliability estimates. Advances in Methods and Practices in Psychological Science, 3(4), 484–501. https://doi.org/10.1177/2515245920951747
   Google Scholar
• Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9(4), 466–491. https://doi.org/10.1037/1082-989X.9.4.466
   PubMed Web of Science ®Google Scholar
• George, D., & Mallery, P. (2003). SPSS for windows step by step: A simple guide and reference. 11.0 update (4th ed.) Allyn & Bacon.
   Google Scholar
• Green, S. B., & Yang, Y. (2015). Evaluation of dimensionality in the assessment of internal consistency reliability: Coefficient alpha and omega coefficients. Educational Measurement: Issues and Practice, 34(4), 14–20. https://doi.org/10.1111/emip.12100
   Web of Science ®Google Scholar
• Guttman, L. (1945). A basis for analyzing test-retest reliability. Psychometrika, 10(4), 255–282. https://doi.org/10.1007/BF02288892
   PubMed Web of Science ®Google Scholar
• Hancock, G. R., & Mueller, R. O. (2001). Rethinking construct reliability within latent variable systems. In R. Cudeck, & D. Sörbom (Eds.), Structural equation modeling: Present and future—A festschrift in honor of Karl Jöreskog (pp. 195–216) Scientific Software International.
   Google Scholar
• Hayes, A., & Coutts, J. (2020). Use Omega rather than Cronbach’s alpha for estimating reliability. But…. Communication Methods and Measures, 14(1), 1–24. https://doi.org/10.1080/19312458.2020.1718629
   Web of Science ®Google Scholar
• Heise, D. R., & Bohrnstedt, G. W. (1970). Validity, invalidity, and reliability. Sociological Methodology, 2(1), 104–129. https://doi.org/10.2307/270785
   Google Scholar
• Hoekstra, R., Vugteveen, J., Warrens, M. J., & Kruyen, P. M. (2019). An empirical analysis of alleged misunderstandings of coefficient alpha. International Journal of Social Research Methodology, 22(4), 351–364. https://doi.org/10.1080/13645579.2018.1547523
   Web of Science ®Google Scholar
• Hoyt, C. (1941). Test reliability estimated by analysis of variance. Psychometrika, 6(3), 153–160. https://doi.org/10.1007/BF02289270
   Google Scholar
• IBM Corp. (2020). IBM SPSS Statistics for Windows (Version 27.0) [Computer software]. https://www.ibm.com/support/pages/downloading-ibm-spss-statistics-27
   Google Scholar
• Institute for Personality and Ability Testing. (2014). 16PF Questionnaire: International reference manual. https://www.16pf.com/wp-content/uploads/Resources-16pf-International-Reference-Manual-2014.pdf
   Google Scholar
• Jamovi Project. (2023). Jamovi (Version 2.3) [Computer Software]. https://www.jamovi.org
   Google Scholar
• Kalkbrenner, M. T. (2021). Enhancing assessment literacy in professional counseling: A practical overview of factor analysis. The Professional Counselor, 11(3), 267–284. https://doi.org/10.15241/mtk.11.3.267
   Google Scholar
• Kalkbrenner, M. T. (2023). Alpha, omega, and H internal consistency reliability estimates-reviewing these options and when to use them. Counseling Outcome Research and Evaluation, 14(1), 77–88. https://doi.org/10.1080/21501378.2021.1940118
   Google Scholar
• Kalkbrenner, M. T., & Flinn, R. E. (2022). Psychometric properties of The Mental Distress Response Scale among first-generation college students. Journal of College Student Retention: Research, Theory & Practice. Advanced online publication. https://doi.org/10.1177/15210251221123041
   Google Scholar
• Kelley, K., & Cheng, Y. (2012). Estimation of and confidence interval formation for reliability coefficients of homogeneous measurement instruments. Methodology, 8(2), 39–50. https://doi.org/10.1027/1614-2241/a000036
   Web of Science ®Google Scholar
• Kelley, K., & Pornprasertmanit, S. (2016). Confidence intervals for population reliability coefficients: Evaluation of methods, recommendations, and software for composite measures. Psychological Methods, 21(1), 69–92. https://doi.org/10.1037/a0040086.supp
   PubMed Web of Science ®Google Scholar
• Kuder, G. F., & Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2(3), 151–160. https://doi.org/10.1007/BF02288391
   Google Scholar
• Lenz, A. S., Ault, H., Balkin, R. S., Barrio Minton, C., Erford, B. T., Hays, D. G., Kim, B. S. K., & Li, C. (2022). Responsibilities of users of standardized tests (RUST-4E): Prepared for the Association for Assessment and Research in Counseling. Measurement and Evaluation in Counseling and Development, 55(4), 227–235. https://doi.org/10.1080/07481756.2022.2052321
   Web of Science ®Google Scholar
• Malkewitz, C. P., Schwall, P., Meesters, C., & Hardt, J. (2023). Estimating reliability: A comparison of Cronbach’s α, McDonald’s ωt and the greatest lower bound. Social Sciences & Humanities Open, 7(1), 100368. https://doi.org/10.1016/j.ssaho.2022.100368
   Google Scholar
• McDonald, R. P. (1999). Test theory: A unified treatment. Lawrence Erlbaum.
   Google Scholar
• McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412–433. https://doi.org/10.1037/met0000144
   PubMed Web of Science ®Google Scholar
• Mvududu, N. H., & Sink, C. A. (2013). Factor analysis in counseling research and practice. Counseling Outcome Research and Evaluation, 4(2), 75–98. https://doi.org/10.1177/2150137813494766
   Google Scholar
• Nájera Catalán, H. (2019). Reliability, population classification and weighting in multidimensional poverty measurement: A Monte Carlo study. Social Indicators Research, 142(3), 887–910. https://doi.org/10.1007/s11205-018-1950-z
   Web of Science ®Google Scholar
• Neukrug, E. S., & Fawcett, C. R. (2020). Essentials of testing and assessment: A practical guide for counselors, social workers, and psychologists (Enhanced 3rd ed.). Cengage
   Google Scholar
• Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. McGraw-Hill.
   Google Scholar
• Oosterwijk, P. R., van der Ark, L. A., & Sijtsma, K. (2019). Using confidence intervals for assessing reliability of real tests. Assessment, 26(7), 1207–1216. https://doi.org/10.1177/1073191117737375
   PubMed Web of Science ®Google Scholar
• Padilla, M. A., & Divers, J. (2013). Bootstrap interval estimation of reliability via coefficient omega. Journal of Modern Applied Statistical Methods, 12(1), 78–89. https://doi.org/10.22237/jmasm/1367381520
   Google Scholar
• Raykov, T., & Marcoulides, G. A. (2015). A direct latent variable modeling based method for point and interval estimation of coefficient alpha. Educational and Psychological Measurement, 75(1), 146–156. https://doi.org/10.1177/0013164414526039
   PubMed Web of Science ®Google Scholar
• Raykov, T., & Marcoulides, G. A. (2019). Thanks coefficient alpha, we still need you!. Educational and Psychological Measurement, 79(1), 200–210. https://doi.org/10.1177/0013164417725127
   PubMed Web of Science ®Google Scholar
• Revelle, W. (2023, September). Using R and the psych package to find ω. https://personality-project.org/r/psych/HowTo/omega.pdf
   Google Scholar
• Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107–120. https://doi.org/10.1007/s11336-008-9101-0
   PubMed Web of Science ®Google Scholar
• Sijtsma, K., & Pfadt, J. M. (2021). Part II: On the use, the misuse, and the very limited usefulness of Cronbach’s alpha: Discussing low bounds and correlated errors. Psychometrika, 86(4), 843–860. https://doi.org/10.1007/s11336-021-09789-8
   PubMed Web of Science ®Google Scholar
• Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2(1), 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd
   PubMedGoogle Scholar
• Trizano-Hermosilla, I., & Alvarado, J. M. (2016). Best alternatives to Cronbach’s alpha reliability in realistic conditions: Congeneric and asymmetrical measurements. Frontiers in Psychology, 7(1), 769–769. https://doi.org/10.3389/fpsyg.2016.00769
   PubMedGoogle Scholar
• Watkins, M. (2017). The reliability of multidimensional neuropsychological measures: From alpha to omega. The Clinical Neuropsychologist, 31(6-7), 1113–1126. https://doi.org/10.1080/13854046.2017.1317364
   PubMed Web of Science ®Google Scholar
• Xiao, L., & Hau, K.-T. (2023). Performance of coefficient alpha and its alternatives: Effects of different types of non-normality. Educational and Psychological Measurement, 83(1), 5–27. https://doi.org/10.1177/00131644221088240
   PubMed Web of Science ®Google Scholar
• Zhang, Z., & Yuan, K. H. (2016). Robust coefficients alpha and omega and confidence intervals with outlying observations and missing data: Methods and software. Educational and Psychological Measurement, 76(3), 387–411. https://doi.org/10.1177/0013164415594658
   PubMed Web of Science ®Google Scholar
• Zinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005). Cronbach’s α, Revelle’s β, and Mcdonald’s ωH: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70(1), 123–133. https://doi.org/10.1007/s11336-003-0974-7
   Web of Science ®Google Scholar