Item Response Theory: A Modern Measurement Approach to Reliability and Precision for Counseling Researchers

References

• American Educational Research Association (AERA), American Psychological Association, & National Council on Measurement in Education (Eds.). (2014). Standards for educational and psychological testing. American Educational Research Association.
   Google Scholar
• Ames, A. J., & Penfield, R. D. (2015). An NCME instructional module on item-fit statistics for item response theory models. Educational Measurement: Issues and Practice, 34(3), 39–48. https://doi.org/10.1111/emip.12067
   Web of Science ®Google Scholar
• Andrich, D. A. (1982). An index of person separation in latent trait theory, the traditional KR.20 indices and the Guttman scale response pattern. Education Research and Perspectives, 9, 95–104.
   Google Scholar
• Andrich, D., & Marais, I. (2019). A course in rasch measurement theory: Measuring in the educational, social and health sciences. Springer Singapore. https://doi.org/10.1007/978-981-13-7496-8
   Google Scholar
• Balkin, R. S., & Kleist, D. M. (2022). Counseling research: A practitioner-scholar approach. John Wiley & Sons.
   Google Scholar
• Balkin, R. S., & Lenz, A. S. (2021). Contemporary issues in reporting statistical, practical, and clinical significance in counseling research. Journal of Counseling & Development, 99(2), 227–237. https://doi.org/10.1002/jcad.12370
   Web of Science ®Google Scholar
• Bond, T. G., Yan, Z., & Heene, M. (2020). Applying the Rasch model. Fundamental measurement in the human sciences (4th ed.). Routledge.
   Google Scholar
• Bonifay, W. (2019). Multidimensional item response theory (1st ed.). Sage.
   Google Scholar
• Briggs, D. C. (2022). Historical and conceptual foundations of measurement in the human sciences: Credos and controversies. Routledge.
   Google Scholar
• Cheng, Y., Yuan, K. H., & Liu, C. (2012). Comparison of reliability measures under factor analysis and item response theory. Educational and Psychological Measurement, 72(1), 52–67. https://doi.org/10.1177/0013164411407315
   Web of Science ®Google Scholar
• Chou, Y.-T., & Wang, W.-C. (2010). Checking dimensionality in item response models with principal component analysis on standardized residuals. Educational and Psychological Measurement, 70(5), 717–731. https://doi.org/10.1177/0013164410379322
   Web of Science ®Google Scholar
• Cook, R. M. (2021). Addressing missing data in quantitative counseling research. Counseling Outcome Research and Evaluation, 12(1), 43–53. https://doi.org/10.1080/21501378.2019.1711037
   Google Scholar
• Cook, R. M., Fye, H. J., & Wind, S. A. (2021). An examination of the Counselor Burnout Inventory using item response theory in early career post-master’s counselors. Measurement and Evaluation in Counseling and Development, 54(4), 233–250. https://doi.org/10.1080/07481756.2020.1827439
   Web of Science ®Google Scholar
• Cook, R. M., McKibben, W. B., & Wind, S. A. (2018). Supervisee perception of power in clinical supervision: The power dynamics in supervision scale. Training and Education in Professional Psychology, 12(3), 188–195. https://doi.org/10.1037/tep0000201
   Web of Science ®Google Scholar
• Cook, R. M., Sackett, C. R., & Wind, S. A. (2023a). The development of the meaningful experiences in counseling scale. Measurement and Evaluation in Counseling and Development, 56(4), 313–328. https://doi.org/10.1080/07481756.2022.2148110
   Web of Science ®Google Scholar
• Cook, R. M., Wind, S. A., & Fye, H. J. (2023b). Development of the trauma-informed practice scale - supervision version (TIP-SV). Measurement and Evaluation in Counseling and Development, 56(1), 13–32. https://doi.org/10.1080/07481756.2022.2034480
   Web of Science ®Google Scholar
• Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555
   Web of Science ®Google Scholar
• De Ayala, R. J. (2021). The theory and practice of item response theory. Routledge.
   Google Scholar
• De Champlain, A. F. (2010). A primer on classical test theory and item response theory for assessments in medical education. Medical Education, 44(1), 109–117. https://doi.org/10.1111/j.1365-2923.2009.03425.x
   PubMed Web of Science ®Google Scholar
• Donny, E. C., Denlinger, R. L., Tidey, J. W., Koopmeiners, J. S., Benowitz, N. L., Vandrey, R. G., al’Absi, M., Carmella, S. G., Cinciripini, P. M., Dermody, S. S., Drobes, D. J., Hecht, S. S., Jensen, J., Lane, T., Le, C. T., McClernon, F. J., Montoya, I. D., Murphy, S. E., Robinson, J. D., … Hatsukami, D. K. (2015). Randomized trial of reduced-nicotine standards for cigarettes. The New England Journal of Medicine, 373(14), 1340–1349. https://doi.org/10.1056/NEJMsa1502403
   PubMed Web of Science ®Google Scholar
• Embretson, S. E. (1996). The new rules of measurement. Psychological Assessment, 8(4), 341–349. https://doi.org/10.1037/1040-3590.8.4.341
   Web of Science ®Google Scholar
• Engelhard, G. (2013). Invariant measurement: Using rasch models in the social, behavioral, and health sciences. Routledge.
   Google Scholar
• Engelhard, G., & Wang, J. (2020). Rasch models for solving measurement problems (Vol. 187). Sage. https://us.sagepub.com/en-us/nam/rasch-models-for-solving-measurement-problems/book267292
   Google Scholar
• Engelhard, G., & Wind, S. A. (2021). A history of Rasch measurement theory. In B. E. Clauser & M. Bunch (Eds.), The history of educational measurement: Key advances in theory, policy, and practice (pp. 343–360). Routledge.
   Google Scholar
• Erford, B. T., Chang, C. Y., Crockett, S. A., Byrd, R., Johnsen, S. T., MacInerney, E. K., Menzies, A., Milowsky, A. I., Saks, J., Wills, L., Zhang, X., Alder, C., Anderson, B., Barstack, S., Bradford, E., Choi, J., Cummings, J. A., Fuller, A., Gayowsky, J., … Yu, C. (2023). An omnibus synthesis of author and article publication characteristics in 22 counseling journals from 2010 to 2019. Measurement and Evaluation in Counseling and Development, 56(3), 187–208. https://doi.org/10.1080/07481756.2023.2224028
   Web of Science ®Google Scholar
• Fye, H. J., Wind, S. A., & Cook, R. M. (2023). The trauma-informed practices scale – school counselor version (TIPS-SCV). Measurement and Evaluation in Counseling and Development. Advance online publication. https://doi.org/10.1080/07481756.2023.2243263
   Web of Science ®Google Scholar
• Kalkbrenner, M. T. (2023). Alpha, omega, and H internal consistency reliability estimates: Reviewing these options and when to use them. Counseling Outcome Research and Evaluation, 14(1), 77–88. https://doi.org/10.1080/21501378.2021.1940118
   Google Scholar
• Kalkbrenner, M. T., Gainza Perez, M. A., & Hubbard, J. S. (2024). Development and initial validation of scores on the lifestyle practices and health consciousness inventory-2: brief version. Measurement and Evaluation in Counseling and Development, (57)1, 1–14. https://doi.org/10.1080/07481756.2023.2193339
   Google Scholar
• Killian, T., Peters, H. C., & Floren, M. (2023). Development and validation of the multicultural and social justice counseling competencies-inventory. Measurement and Evaluation in Counseling and Development, 56(4), 329–346. https://doi.org/10.1080/07481756.2022.2160357
   Web of Science ®Google Scholar
• Lee, S. M., Baker, C. R., Cho, S. H., Heckathorn, D. E., Holland, M. W., Newgent, R. A., Ogle, N. T., Powell, M. L., Quinn, J. J., Wallace, S. L., & Yu, K. (2007). Development and initial psychometrics of the Counselor Burnout Inventory. Measurement and Evaluation in Counseling and Development, 40(3), 142–154. https://doi.org/10.1080/07481756.2007.11909811
   Web of Science ®Google Scholar
• Lenz, A. S., Ho, C. M., Rocha, L., & Aras, Y. (2021). Reliability generalization of scores on the Post-Traumatic Growth Inventory. Measurement and Evaluation in Counseling and Development, 54(2), 106–119. https://doi.org/10.1080/07481756.2020.1747940
   Web of Science ®Google Scholar
• Linacre, J. M. (1998). Structure in Rasch residuals: Why principal components analysis (PCA)? Rasch Measurement Transactions, 12(2), 636.
   Google Scholar
• Masters, G. N. (2018). Partial credit model. In W. J. van der Linden (Ed.), Handbook of item response theory (Vol. 1, pp. 109–126). CRC Press.
   Google Scholar
• McKibben, W. B., & Silvia, P. J. (2016). Inattentive and socially desirable responding: Addressing subtle threats to validity in quantitative counseling research. Counseling Outcome Research and Evaluation, 7(1), 53–64. https://doi.org/10.1177/21501378156131
   Google Scholar
• McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412–433. https://doi.org/10.1037/met0000144
   PubMed Web of Science ®Google Scholar
• Mellenbergh, G. J. (1996). Measurement precision in test score and item response models. Psychological Methods, 1(3), 293–299. https://doi.org/10.1037/1082-989X.1.3.293
   Web of Science ®Google Scholar
• Miller, Y. R., Medvedev, O. N., Hwang, Y.-S., & Singh, N. N. (2021). Applying generalizability theory to the Perceived Stress Scale to evaluate stable and dynamic aspects of educators’ stress. International Journal of Stress Management, 28(2), 147–153. https://doi.org/10.1037/str0000207
   Web of Science ®Google Scholar
• Morris, C. S., Ingram, P. B., Mitchell, S. M., & Victor, S. E. (2023). Screening utility of the PHQ-2 and PHQ-9 for depression in college students: Relationships with substantive scales of the MMPI-3. Measurement and Evaluation in Counseling and Development: The Official Publication of the Association for Measurement and Evaluation in Counseling and Development, 56(3), 254–264. https://doi.org/10.1080/07481756.2022.2110899
   PubMed Web of Science ®Google Scholar
• Muraki, E. (1997). A generalized partial credit model. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 153–164). Springer. https://doi.org/10.1007/978-1-4757-2691-6_9
   Google Scholar
• Muraki, E., & Muraki, M. (2018). Generalized partial credit model. In W. J. van der Linden (Ed.), Handbook of item response theory (Vol. 1, pp. 127–138). CRC Press.
   Google Scholar
• Poynton, T. A., DeFouw, E. R., & Morizio, L. J. (2019). A systematic review of online response rates in four counseling journals. Journal of Counseling & Development, 97(1), 33–42. https://doi.org/10.1002/jcad.12233
   Web of Science ®Google Scholar
• Radloff, L. S. (1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1(3), 385–401. https://doi.org/10.1177/014662167700100306
   Web of Science ®Google Scholar
• Rasch, G. (1960). Probabilistic models for some intelligence and achievement tests. (Expanded edition, 1980). University of Chicago Press.
   Google Scholar
• Raykov, T., & Marcoulides, G. A. (2011). Introduction to psychometric theory. Routledge.
   Google Scholar
• Reckase, M. D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational Statistics, 4(3), 207–230. https://doi.org/10.3102/10769986004003207
   Google Scholar
• Reise, S. P., Ainsworth, A. T., & Haviland, M. G. (2005). Item response theory: Fundamentals, applications, and promise in psychological research. Current Directions in Psychological Science, 14(2), 95–101. https://doi.org/10.1111/j.0963-7214.2005.00342.x
   Web of Science ®Google Scholar
• Revelle, W. (2016). psych: Procedures for Personality and Psychological Research (1.6.9) [ Computer software]. Northwestern University. https://CRAN.R-project.org/package=psych
   Google Scholar
• Robitzsch, A., Kiefer, T., & Wu, M. (2020). TAM: Test Analysis Modules (3.5-19) [Computer software]. https://CRAN.R-project.org/package=TAM
   Google Scholar
• Samejima, F. (2018). Graded response models. In W. J. van der Linden (Ed.), Handbook of item response theory (Vol. 1, pp. 95–108). CRC Press.
   Google Scholar
• Seol, H. (2016). Using the bootstrap method to evaluate the critical range of misfit for polytomous Rasch fit statistics. Psychological Reports, 118(3), 937–956. https://doi.org/10.1177/0033294116649434
   PubMed Web of Science ®Google Scholar
• Sijtsma, K., & Meijer, R. R. (2007). Nonparametric item response theory and special topics. In C. R. Rao & S. Sinhary (Eds.), Handbook of statistics: Psychometrics (Vol. 26, pp. 719–747). lsevier.
   Google Scholar
• Smith, R. M. (2004). Fit analysis in latent trait models. In E. V. Smith & R. M. Smith (Eds.), Introduction to Rasch measurement (pp. 73–92). JAM Press.
   Google Scholar
• Sriken, J., Johnsen, S. T., Smith, H., Sherman, M. F., & Erford, B. T. (2022). Testing the factorial validity and measurement invariance of college student scores on the Generalized Anxiety Disorder (GAD-7) scale across gender and race. Measurement and Evaluation in Counseling and Development, 55(1), 1–16. https://doi.org/10.1080/07481756.2021.1902239
   Web of Science ®Google Scholar
• van der Linden, W. J., & Hambleton, R. K. (Eds.). (1997). Handbook of modern item response theory. Springer-Verlag.
   Google Scholar
• Vidales, C. A., Vogel, D. L., & Levant, R. F. (2023). The Self-Stigma of Seeking Help (SSOSH) Scale: Measurement invariance across men from different backgrounds. Measurement and Evaluation in Counseling and Development, 57(1), 15–29. https://doi.org/10.1080/07481756.2022.2160356
   Web of Science ®Google Scholar
• Walker, A. A., Jennings, J. K., & Engelhard, G. (2018). Using person response functions to investigate areas of person misfit related to item characteristics. Educational Assessment, 23(1), 47–68. https://doi.org/10.1080/10627197.2017.1415143
   Web of Science ®Google Scholar
• Wells, C. S., & Hambleton, R. K. (2016). Model fit with residual analyses. In W. J. van der Linden (Ed.), Handbook of item response theory (Vol. 2, pp. 395–413). CRC Press.
   Google Scholar
• Wilson, M. (2011). Some notes on the term: “wright map. Rasch Measurement Transactions, 25(3), 1331.
   Google Scholar
• Wind, S. A. (2022). Exploring rating scale functioning for survey research. SAGE Publications.
   Google Scholar
• Wind, S. A., Cook, R. M., & McKibben, W. B. (2021). Supervisees’ of differing genders and races perceptions of power in supervision. Counselling Psychology Quarterly, 34(2), 275–297. https://doi.org/10.1080/09515070.2020.1731791
   Web of Science ®Google Scholar
• Wind, S. A., & Schumacker, R. E. (2021). Exploring the impact of missing data on residual-based dimensionality analysis for measurement models. Educational and Psychological Measurement, 81(2), 290–318. https://doi.org/10.1177/0013164420939634
   PubMed Web of Science ®Google Scholar
• Wolfe, E. W. (2013). A bootstrap approach to evaluating person and item fit to the Rasch model. Journal of Applied Measurement, 14(1), 1–9.
   PubMedGoogle Scholar
• Wu, M., & Adams, R. J. (2013). Properties of Rasch residual fit statistics. Journal of Applied Measurement, 14(4), 339–355.
   Google Scholar
• Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8(2), 125–145. https://doi.org/10.1177/014662168400800201
   Web of Science ®Google Scholar
• Zanon, C., Baptista, M. N., Rubin, M., Topkaya, N., Şahin, E., Brenner, R. E., Vogel, D. L., & Mak, W. W. S. (2023). Measuring family support in Australia, Brazil, Hong Kong, and Turkey: A psychometric investigation. Measurement and Evaluation in Counseling and Development. Advance online publication. https://doi.org/10.1080/07481756.2023.2219009
   Web of Science ®Google Scholar
• Zhu, P., Liu, Y., Luke, M. M., & Wang, Q. (2022). The development and initial validation of the cultural humility and enactment scale in counseling. Measurement and Evaluation in Counseling and Development, 55(2), 98–115. https://doi.org/10.1080/07481756.2021.1955215
   Web of Science ®Google Scholar