Advanced search
Publication Cover
Measurement and Evaluation in Counseling and Development Volume 57, 2024 - Issue 2: Score Reliability
Submit an article Journal homepage
339
Views
1
CrossRef citations to date
0
Altmetric
Methods Plainly Speaking

Intercoder Reliability for Use in Qualitative Research and Evaluation

Monica L. Colemana University of Mississippi, Oxford, MS, USACorrespondence[email protected]
ORCID Iconhttps://orcid.org/0000-0002-0513-0221View further author information
ORCID Icon,
Moira Raganb Ripple Effect, Rockville, MD, USAORCID Iconhttps://orcid.org/0009-0003-1404-6675View further author information
ORCID Icon &
Tahani Daric University of Detroit Mercy, Detroit, MI, USAORCID Iconhttps://orcid.org/0000-0002-5873-5435View further author information
ORCID Icon
Pages 136-146 | Published online: 29 Feb 2024

References

  • American Counseling Association. (2014). ACA code of ethics. https://www.counseling.org/docs/default-source/ethics/2014-code-of-ethics.pdf
  • Bardhoshi, G., & Erford, B. T. (2017). Processes and procedures for estimating score reliability and precision. Measurement and Evaluation in Counseling and Development, 50(4), 256–263. https://doi.org/10.1080/07481756.2017.1388680
  • Berkovich, I. (2018). Beyond qualitative/quantitative structuralism: The positivist qualitative research and the paradigmatic disclaimer. Quality & Quantity, 52(5), 2063–2077. https://doi.org/10.1007/s11135-017-0607-3
  • Cheung, K. K. C., & Tai, K. W. H. (2023). The use of intercoder reliability in qualitative interview data analysis in science education. Research in Science & Technological Education, 41(3), 1155–1175. https://doi.org/10.1080/02635143.2021.1993179
  • Chouinard, J. A. (2013). The case for participatory evaluation in an era of accountability. American Journal of Evaluation, 34(2), 237–253.
  • Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104
  • Coleman, M. L. (2022). The use of counseling skills within evaluative contexts. Counseling Outcome Research and Evaluation, 13(1), 22–29. https://doi.org/10.1080/21501378.2022.2025771
  • Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). SAGE Publications, Inc.
  • Dari, T., Laux, J. M., Liu, Y., & Reynolds, J. (2019). Development of community-based participatory research competencies: A Delphi study identifying best practices in the collaborative process. The Professional Counselor, 9(1), 1–19. https://doi.org/10.15241/td.9.1.1
  • Feng, G. C. (2015). Mistakes and how to avoid mistakes in using intercoder reliability indices. Methodology, 11(1), 13–22. https://doi.org/10.1027/1614-2241/a000086
  • Freelon, D. (2013). ReCal OIR: Ordinal, interval, and ratio intercoder reliability as a web service. International Journal of Internet Science, 8(1), 10–16.
  • IBM Corp. (2021). IBM SPSS Statistics for Windows, Version 28.0.
  • Krippendorff, K. (2004). Reliability in content analysis: Some common misconceptions and recommendations. Human Communication Research, 30(3), 411–433. https://doi.org/10.1111/j.1468-2958.2004.tb00738.x
  • Kurasaki, K. S. (2000). Intercoder reliability for validating conclusions drawn from open-ended interview data. Field Methods, 12(3), 179–194. https://doi.org/10.1177/1525822X0001200301
  • Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.
  • Lenz, A. S., Ault, H., Balkin, R. S., Barrio Minton, C., Erford, B. T., Hays, D. G., Kim, B. S. K., & Li, C. (2022). Responsibilities of users of standardized tests (RUST-4E): Prepared for the Association for Assessment and Research in Counseling. Measurement and Evaluation in Counseling and Development, 55(4), 227–235. https://doi.org/10.1080/07481756.2022.2052321
  • Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Sage Publications.
  • Lumivero (2020). NVivo (Version 14). www.lumivero.com.
  • MacPhail, C., Khoza, N., Abler, L., & Ranganathan, M. (2016). Process guidelines for establishing Intercoder Reliability in qualitative studies. Qualitative Research, 16(2), 198–212.
  • Marshall, C., & Rossman, G. B. (1999). Designing qualitative research (3rd ed.) Sage Publications.
  • McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276–282. https://doi.org/10.11613/BM.2012.031
  • Miles, M. B., Huberman, A. M., & Saldaña, J. (2020). Qualitative data analysis: A methods sourcebook (4th ed.) SAGE Publications.
  • O’Connor, C., & Joffe, H. (2020). Intercoder reliability in qualitative research: Debates and practical guidelines. International Journal of Qualitative Methods, 19, 1–13. https://doi.org/10.1177/1609406919899220
  • Su, N. (2018). Positivist qualitative methods. SAGE Publications. https://doi.org/10.4135/9781526430212
  • Wilson, J., Gabriel, L., & James, H. (2014). Observing a client’s grieving process: Bringing logical positivism into qualitative grief counselling research. British Journal of Guidance & Counselling, 42(5), 568–583. https://doi.org/10.1080/03069885.2014.936823

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Order Reprints Request Corporate Permissions

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

Request Academic Permissions

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.

Related research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.