Item Response Theory: A Modern Measurement Approach to Reliability and Precision for Counseling Researchers

Abstract

The purpose of this article is to discuss reliability and precision through the lens of a modern measurement approach, item response theory (IRT). Reliability evidence in the field of counseling is primarily generated using Classical Test Theory (CTT) approaches, although recent studies in the field of counseling have shown the benefits of using IRT approaches to explore measurement precision. We discuss the theoretical foundations and assumptions of CTT and IRT, and examine how modern measurement theory (i.e. IRT) poses advantages for capturing measurement precision. We use an example analysis to demonstrate the indices of measurement precision in IRT approaches (e.g. standard errors, person-fit, item-fit, targeting, residual-based fit statistics). Finally, we discuss practical and clinical implications for counseling researchers for using IRT approaches to measure precision, including insights into precision for persons, items, and measurement invariance as well as the utility of brief and adaptive scales.

SIGNIFICANCE STATEMENT

Using modern measurement theory approaches (i.e. item response theory), evidence of precision is generated for persons, items, and other relevant facets, and these indices can inform needed scale revisions, identify for whom the theorized model is a good fit, and the items that are well-targeted for the population of interest.

Keywords:

• Item response theory
• quantitative research
• counseling research
• reliability
• precision

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Ryan M. Cook

Ryan M. Cook, PhD, LPC, ACS, is an Associate Professor in the Department of Educational Studies in Psychology, Research Methodology, and Counseling at the University of Alabama. His research interests include clinical supervision, issue of counselor development, and client-focused care.

Stefanie A. Wind

Stefanie A. Wind, PhD, is an Assistant Professor of Educational Measurement at the University of Alabama. Her primary research interests include the exploration of methodological issues in the field of educational measure­ment, with emphases on methods related to rater-mediated assessments, rating scales, Rasch models and item response theory models, and nonparametric item response theory, as well as applications of these methods to substantive areas related to education.