1,404
Views
11
CrossRef citations to date
0
Altmetric
ARTICLES

Are rating scales really better than checklists for measuring increasing levels of expertise?

ORCID Icon & ORCID Icon
Pages 46-51 | Published online: 20 Aug 2019
 

Abstract

Background: It is a doctrine that OSCE checklists are not sensitive to increasing levels of expertise whereas rating scales are. This claim is based primarily on a study that used two psychiatry stations and it is not clear to what degree the finding generalizes to other clinical contexts. The purpose of our study was to reexamine the relationship between increasing training and scoring instruments within an OSCE.

Approach: A 9-station OSCE progress test was administered to Internal Medicine residents in post-graduate years (PGY) 1–4. Residents were scored using checklists and rating scales. Standard scores from three administrations (27 stations) were analyzed.

Findings: Only one station produced a result in which checklist scores did not increase as a function of training level, but the rating scales did. For 13 stations, scores increased as a function of PGY equally for both checklists and rating scales.

Conclusion: Checklist scores were as sensitive to the level of training as rating scales for most stations, suggesting that checklists can capture increasing levels of expertise. The choice of which measure is used should be based on the purpose of the examination and not on a belief that one measure can better capture increases in expertise.

Acknowledgements

The authors wish to thank Katherine Scrowcroft for her help with this manuscript.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Glossary

Progress tests, checklists, rating scales: Progress tests are an assessment design in which learners at all levels are tested simultaneously on the same test but the content focuses on the knowledge or skills of a learner who has completed their studies. Feedback allows comparisons between groups of learners at different stages of learning and within any learner across their training.

Additional information

Funding

This study was funded by a research grant awarded to Timothy Wood by the Faculty of Medicine, University of Ottawa.

Notes on contributors

Timothy J. Wood

Timothy J. Wood, PhD, is a Professor at the Department of Innovation in Medical Education, University of Ottawa, Ottawa, Canada.

Debra Pugh

Debra Pugh, MD, MHPE, is an Associate Professor at the Department of Medicine, University of Ottawa, at the Ottawa Hospital, and Medical Education Advisor at Medical Council of Canada, Ottawa Canada.

Log in via your institution

Log in to Taylor & Francis Online

PDF download + Online access

  • 48 hours access to article PDF & online version
  • Article PDF can be downloaded
  • Article PDF can be printed
USD 65.00 Add to cart

Issue Purchase

  • 30 days online access to complete issue
  • Article PDFs can be downloaded
  • Article PDFs can be printed
USD 771.00 Add to cart

* Local tax will be added as applicable

Related Research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.