Reliable for Whom? Inferring and Reporting Reliability across Diverse Populations

Abstract

We describe best practices in reporting reliability estimates in counseling research with consideration to precision, generalization, and diverse populations. We provide a historical context to reporting reliability estimates, the limitations of past practices, and new methods to address reliability generalization. We highlight best practices including consideration of reliability estimates as function of scores, not scale, diverse sample, hierarchy of reporting reliability estimates, and methods of reliability generalization.

SIGNIFICANCE STATEMENT

Best practices in evaluating and reporting reliability estimates in counseling research are addressed. We consider the interpretation of reliability estimates across diverse populations; limitations of reliability estimates for common, published measures; and the reporting of reliability estimates in the practice of research and publication.

Keywords:

• reliability
• assessment
• diversity
• meta-synthesis
• reporting

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Richard S. Balkin

Richard S. Balkin, PhD, LPC, NCC is a Professor and Chair for the Department of Leadership and Counselor Education at The University of Mississippi. His research interests include counseling outcomes, research methods and statistics, counseling youth, and forgiveness. Dr. Balkin is a Fellow of the American Counseling Association.

Quentin Hunter

Quentin Hunter, PhD, LPC is an assistant professor in the Department of Leadership & Counseling at Eastern Michigan University and associate editor for the Journal of Counseling and Development and the International Journal for the Advancement of Counselling. His counseling experience includes work in school and clinical mental health settings.

Bradley T. Erford

Bradley T. Erford, PhD, is a professor and director in the human development counseling and human development studies programs of the Department of Human and Organizational Development in the Peabody College at Vanderbilt University, past-president of the American Counseling Association, and editor-emeritus of Measurement and Evaluation in Counseling and Development.