Reliable for Whom? Inferring and Reporting Reliability across Diverse Populations

References

• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1985). Standards for educational and psychological testing (2nd ed.). American Psychological Association.
   Google Scholar
• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing (3rd ed.). American Psychological Association.
   Google Scholar
• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.
   Google Scholar
• American Psychological Association. (2020). Publication manual for the American Psychological Association (7th ed.). American Psychological Association.
   Google Scholar
• Balkin, R. S. (2023). Addressing limitations of p in counseling research: Reporting Bayes factor bound and effect size precision. Counseling Outcome Research and Evaluation, 14(2), 167–173. https://doi.org/10.1080/21501378.2023.2226386
   Google Scholar
• Balkin, R. S., Harris, N., Freeman, S. J., & Huntington, S. (2014). The Forgiveness Reconciliation Inventory: An instrument to process through issues of forgiveness and conflict. Measurement and Evaluation in Counseling and Development, 47(1), 3–13. https://doi.org/10.1177/0748175613497037
   Web of Science ®Google Scholar
• Balkin, R. S., & Kleist, D. M. (2023). Counseling research: A practitioner-scholar approach (2nd ed.). American Counseling Association.
   Google Scholar
• Balkin, R. S., & Lenz, A. S. (2021). Contemporary issues in reporting statistical, practical, and clinical significance. Journal of Counseling & Development, 99(2), 227–237. https://doi.org/10.1002/jcad.12370
   Web of Science ®Google Scholar
• Balkin, R. S., Neal, S. A., Stewart, K. D., Hendricks, L., & Litam, S. D. (2022). Spirituality and relational health among Black Americans. Journal of Counseling & Development, 100(4), 412–420. https://doi.org/10.1002/jcad.12436
   Web of Science ®Google Scholar
• Bardhoshi, G., Duncan, K., & Erford, B. T. (2016). Psychometric meta-analysis of the English version of the Beck Anxiety Inventory. Journal of Counseling & Development, 94(3), 356–373. https://doi.org/10.1002/jcad.12090
   Web of Science ®Google Scholar
• Beck, A. T., & Steer, R. A. (1993). Beck anxiety inventory manual. Pearson Assessment.
   Google Scholar
• Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Manual for the Beck depression inventory (2nd ed.). Pearson Assessment.
   Google Scholar
• Bonett, D. G. (2002). Sample size requirements for testing and estimating coefficient alpha. Journal of Educational and Behavioral Statistics, 27(4), 335–340. https://doi.org/10.3102/10769986027004335
   Web of Science ®Google Scholar
• Charter, R. A. (2003). Study samples are too small to produce sufficiently precise reliability coefficients. The Journal of General Psychology, 130(2), 117–129. https://doi.org/10.1080/00221300309601280
   PubMed Web of Science ®Google Scholar
• Cohen, J. (1962). The statistical power of abnormal-social psychological research: A review. The Journal of Abnormal and Social Psychology, 65(3), 145–153. https://doi.org/10.1037/h0045186
   PubMed Web of Science ®Google Scholar
• Dowd, E. T. (1998). Test review of beck anxiety inventory [1993 Edition]. In J. C. Impara & B. S. Plake (Eds.), The thirteenth mental measurements yearbook. Buros Institute of Mental Measurements.
   Google Scholar
• Erford, B. T. (2021). Assessment for counselors. Pearson.
   Google Scholar
• Erford, B. T., Johnson, E., & Bardoshi, G. (2015). Meta-analysis of the English Version of the Beck Depression Inventory (BDI-II). Measurement and Evaluation in Counseling and Development, 49(1), 3–33. https://doi.org/10.1177/0748175615596783
   Web of Science ®Google Scholar
• Erford, B. T., Savin-Murphy, J. A., & Butler, C. (2010). Conducting a meta-analysis of counseling outcome research: Twelve steps and practical procedures. Counseling Outcome Research and Evaluation, 1(1), 19–43. https://doi.org/10.1177/2150137809356682
   Google Scholar
• Graham, J. M., Diebels, K. J., & Barnow, Z. B. (2011). The reliability of relationship satisfaction: A reliability generalization meta-analysis. Journal of Family Psychology, 25(1), 39–48. https://doi.org/10.1037/a0022441.supp
   PubMed Web of Science ®Google Scholar
• Hedges, L. V., & Olkin, I. (1985). Statistical methods for metaanalysis. Academic Press.
   Google Scholar
• Henson, R. K., & Thompson, B. (2002). Characterizing measurement error in scores across studies: Some Recommendations for conducting “reliability generalization” studies. Measurement and Evaluation in Counseling and Development, 35(2), 113–127. https://doi.org/10.1080/07481756.2002.12069054
   Web of Science ®Google Scholar
• Jhangiani, R. S., Chiang, I.-C A., Cuttler, C., & Leighton, D. C. (2019). Research methods in psychology (4th ed.). Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
   Google Scholar
• Lenz, A. S., Ault, H., Balkin, R. S., Barrio Minton, C., Erford, B. T., Hays, D. G., Kim, B. S. K., & Li, C. (2022). Responsibilities of users of standardized tests (RUST-4E): Prepared for the Association for Assessment and Research in Counseling. Measurement and Evaluation in Counseling and Development, 55(4), 227–235. https://doi.org/10.1080/07481756.2022.2052321
   Google Scholar
• Liang, B., Tracy, A., Taylor, C. A., Williams, L. M., Jordan, J. V., & Miller, J. B. (2002). The relational health indices: A study of women’s relationships. Psychology of Women Quarterly, 26(1), 25–35. https://doi.org/10.1111/1471-6402.00040
   Web of Science ®Google Scholar
• López-Pina, J. A., Sánchez-Meca, J., López-López, J. A., Marín-Martínez, F., Núñez-Núñez, R. M., Rosa-Alcázar, A. I., Gómez-Conesa, A., & Ferrer-Requena, J. (2015). The Yale–Brown obsessive compulsive scale: A reliability generalization meta-analysis. Assessment, 22(5), 619–628. https://doi.org/10.1177/1073191114551954
   PubMed Web of Science ®Google Scholar
• Schmidt, F., & Hunter, J. (2013). History, development, evolution, and impact of validity generalization and meta-analysis methods, 1975–2001. In K. R. Murphy (Ed), Validity generalization (pp. 42–76). Psychology Press.
   Google Scholar
• Schmidt, C. K., Piontkowski, S., Raque-Bogdan, T. L., & Ziemer, K. S. (2014). Relational health, ethnic identity, and well-being of college stu- dents of color: A strengths-based perspective. The Counseling Psychologist, 42(4), 473–496. https://doi.org/10.1177/0011000014523796
   Web of Science ®Google Scholar
• Segal, D. L., & Coolidge, F. L. (2018). Reliability. In M. H. Bornstein (Ed.), The SAGE encyclopedia of lifespan human development (p. 1835). SAGE Publications.
   Google Scholar
• Shepperd, J. A., Emanuel, A. S., Dodd, V. J., & Logan, H. L. (2016). The reliability of psychological instruments in community samples: A cautionary note. Journal of Health Psychology, 21(9), 2033–2041. https://doi.org/10.1177/1359105315569859
   PubMed Web of Science ®Google Scholar
• Thompson, B. (2002). Score reliability: Contemporary thinking on reliability issues. Sage publications.
   Google Scholar
• Trochim, W. M. K. (2023). The research methods knowledge base. Atomic Dog. https://conjointly.com/kb/
   Google Scholar
• Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58(1), 6–20. https://doi.org/10.1177/00131640121971059
   Web of Science ®Google Scholar
• Vacha-Haase, T., & Thompson, B. (2011). Score reliability: A retrospective look back at 12 years of reliability generalization studies. Measurement and Evaluation in Counseling and Development, 44(3), 159–168. https://doi.org/10.1177/0748175611409845
   Web of Science ®Google Scholar
• Waller, N. G. (1998). Test review of beck anxiety inventory. [1993 Edition]. In J. C. Impara, & B. S. Plake (Eds.), The thirteenth mental measurements yearbook. http://marketplace.unl.edu/buros/
   Google Scholar