Twelve tips for developing an OSCE that measures what you want

References

• Bartman I, Smee S, Roy M. 2013. A method for identifying extreme OSCE examiners. Clin Teach. 10:27–31.
   PubMedGoogle Scholar
• Bloch R, Norman G. 2012. Generalizability theory for the perplexed: a practical introduction and guide: AMEE Guide No. 68. Med Teach. 34:960–992.
   PubMed Web of Science ®Google Scholar
• Bloch R, Norman G. 2015. G-String IV program. http://fhsperd.mcmaster.ca/g_string/download.html
   Google Scholar
• Brannick MT, Erol-Korkmaz HT, Prewett M. 2011. A systematic review of the reliability of objective structured clinical examination scores. Med Educ. 45:1181–1189.
   PubMed Web of Science ®Google Scholar
• Casey PM, Goepfert AR, Espey EL, Hammoud MM, Kaczmarczyk JM, Katz NT, Neutens JJ, Nuthalapaty FS, Peskin E. Association of Professors of Gynecology and Obstetrics Undergraduate Medical Education Committee. 2009. To the point: reviews in medical education-the objective structured clinical examination. Am J Obstet Gynecol. 200:25–34.
   PubMed Web of Science ®Google Scholar
• Coderre S, Woloschuk W, McLaughlin K. 2009. Twelve tips for blueprinting. Med Teach. 31:322–324.
   PubMed Web of Science ®Google Scholar
• Cömert M, Zill JM, Christalle E, Dirmaier J, Härter M, Scholl I. 2016. Assessing communication skills of medical students in objective structured clinical examinations (OSCE)-A Systematic Review of Rating Scales. PLoS One. 11:e0152717.
   PubMed Web of Science ®Google Scholar
• Cook DA, Brydges R, Ginsburg S, Hatala R. 2015. A contemporary approach to validity arguments: a practical guide to Kane’s framework. Med Educ. 49:560–575.
   PubMed Web of Science ®Google Scholar
• Currie GP, Sivasubramaniam S, Cleland J. 2016. Sequential testing in a high stakes OSCE: determining number of screening tests. Med Teach. 38:708–714.
   PubMed Web of Science ®Google Scholar
• Crick JE, Brennan RL. 1983. GENOVA program. https://education.uiowa.edu/centers/center-advanced-studies-measurement-and-assessment/computer-programs.
   Google Scholar
• Daniels VJ, Bordage G, Gierl MJ, Yudkowsky R. 2014. Effect of clinically discriminating, evidence-based checklist items on the reliability of scores from an Internal Medicine residency OSCE. Adv in Health Sci Educ. 19:497–506.
   PubMed Web of Science ®Google Scholar
• Daniels VJ, Surgin C, Lai H. 2016. Enhancing formative feedback of an OSCE through tablet scoring. Med Educ. 50 Supplement 1:28. doi:10.1111/medu.13057
   Google Scholar
• Denison A, Bate E, Thompson J. 2016. Tablet versus paper marking in assessment: feedback matters. Perspect Med Educ. 5:108–113.
   PubMed Web of Science ®Google Scholar
• Downing SM. 2004. Reliability: on the reproducibility of assessment data. Med Educ. 38:1006–1012.
   PubMed Web of Science ®Google Scholar
• Frank JR, Snell L, Sherbino J, editors. 2015. CanMEDS 2015 physician competency framework. Ottawa: Royal College of Physicians and Surgeons of Canada.
   Google Scholar
• Fuller R, Homer M, Pell G, Hallam J. 2017. Managing extremes of assessor judgment within the OSCE. Med Teach. 39:58–66.
   PubMed Web of Science ®Google Scholar
• Gingerich A, van der Vleuten CP, Eva KW, Regehr G. 2014. More consensus than idiosyncrasy: categorizing social judgments to examine variability in Mini-CEX ratings. Acad Med. 89:1510–1519.
   PubMed Web of Science ®Google Scholar
• Gofton WT, Dudek NL, Wood TJ, Balaa F, Hamstra SJ. 2012. The Ottawa surgical competency operating room evaluation (O-SCORE): a tool to assess surgical competence. Acad Med. 87:1401–1407.
   PubMed Web of Science ®Google Scholar
• Govaerts MJ, Van de Wiel MW, Schuwirth LW, Van der Vleuten CP, Muijtjens AM. 2013. Workplace-based assessment: raters’ performance theories and constructs. Adv in Health Sci Educ. 18:375–396.
   PubMed Web of Science ®Google Scholar
• Harden RM. 1990. Twelve tips for organizing an objective structured clinical examination (OSCE). Med Teach. 12:259–264.
   PubMed Web of Science ®Google Scholar
• Harden RM, Stevenson M, Downie WW, Wilson GM. 1975. Assessment of clinical competence using objective structured examination. Br Med J. 1:447–451.
   PubMed Web of Science ®Google Scholar
• Hastie MJ, Spellman JL, Pagano PP, Hastie J, Egan BJ. 2014. Designing and implementing the objective structured clinical examination in anesthesiology. Anesthesiology. 120:196–203.
   PubMed Web of Science ®Google Scholar
• Hodges BD, Hollenberg E, McNaughton N, Hanson MD, Regehr G. 2014. The psychiatry OSCE: a 20-year retrospective. Acad Psychiatry. 38:26–34.
   PubMed Web of Science ®Google Scholar
• Homer M, Pell G, Fuller R. 2017. Problematizing the concept of the “borderline” group in performance assessments. Med Teach. 39:469–475.
   PubMed Web of Science ®Google Scholar
• Jefferies A, Simmons B, Tabak D, McIlroy JH, Lee KS, Roukema H, Skidmore M. 2007. Using an objective structured clinical examination (OSCE) to assess multiple physician competencies in postgraduate training. Med Teach. 29:183–191.
   Google Scholar
• Kane MT. 2013. Validating the interpretations and uses of test scores. J Educ Meas. 50:1–73.
   Web of Science ®Google Scholar
• Khan KZ, Gaunt K, Ramachandran S, Pushkar P. 2013. The objective structured clinical examination (OSCE): AMEE guide no. 81. Part II: organisation and administration. Med Teach. 35:e1447–e1463.
   PubMed Web of Science ®Google Scholar
• Kreptul D, Thomas RE. 2016. Family medicine resident OSCEs: a systematic review. Educ Prim Care. 27:471–477.
   PubMed Web of Science ®Google Scholar
• Messick S. 1989. Validity. In: Linn RL, editor. Educational measurement. 3rd edn. New York (NY): American Council on Education and Macmillan. p. 13–103.
   Google Scholar
• Nulty DD, Mitchell ML, Jeffrey CA, Henderson A, Groves M. 2011. Best practice guidelines for use of OSCEs: maximising value for student learning. Nurse Educ Today. 31:145–151.
   PubMed Web of Science ®Google Scholar
• Patrício MF, Julião M, Fareleira F, Carneiro AV. 2013. Is the OSCE a feasible tool to assess competencies in undergraduate medical education? Med Teach. 35:503–514.
   PubMed Web of Science ®Google Scholar
• Pell G, Fuller R, Homer M, Roberts T. International Association for Medical Education 2010. How to measure the quality of the OSCE: a review of metrics - AMEE guide no. 49. Med Teach. 32:802–811.
   PubMed Web of Science ®Google Scholar
• Pell G, Fuller R, Homer M, Roberts T. 2013. Advancing the objective structured clinical examination: sequential testing in theory and practice. Med Educ. 47:569–577.
   PubMed Web of Science ®Google Scholar
• Pugh D, Bhanji F, Cole G, Dupre J, Hatala R, Humphrey-Murto S, Touchie C, Wood TJ. 2016. Do OSCE progress test scores predict performance in a national high-stakes examination? Med Educ. 50:351–358.
   PubMed Web of Science ®Google Scholar
• Pugh D, Halman S, Desjardins I, Humphrey-Murto S, Wood TJ. 2016. Done or almost done? Improving OSCE checklists to better capture performance in progress tests. Teach Learn Med. 28:406–414.
   PubMed Web of Science ®Google Scholar
• Pugh D, Regehr G. 2016. Taking the sting out of assessment: is there a role for progress testing? Med Educ. 50:721–729.
   PubMed Web of Science ®Google Scholar
• Pugh D, Smee S. 2013. Guidelines for the development of objective structured clinical examination (OSCE) Cases. Ottawa: Medical Council of Canada.
   Google Scholar
• Roch SG, Woehr DJ, Mishra V, Kieszczynska U. 2012. Rater training revisited: an updated meta-analytic review of frame-of-reference training. J Occup Organ Psychol. 85:370–395.
   Web of Science ®Google Scholar
• Sandilands DD, Gotzmann A, Roy M, Zumbo BD, De Champlain A. 2014. Weighting checklist items and station components on a large-scale OSCE: is it worth the effort? Med Teach. 36:585–590.
   PubMed Web of Science ®Google Scholar
• Schuwirth LW, Van der Vleuten CP. 2011. Programmatic assessment: from assessment of learning to assessment for learning. Med Teach. 33:478–485.
   PubMed Web of Science ®Google Scholar
• Setyonugroho W, Kennedy KM, Kropmans TJ. 2015. Reliability and validity of OSCE checklists used to assess the communication skills of undergraduate medical students: a systematic review. Patient Educ Couns. pii: S0738-3991:00277–00273.
   Google Scholar
• Smith V, Muldoon K, Biesty L. 2012. The objective structured clinical examination (OSCE) as a strategy for assessing clinical competence in midwifery education in Ireland: a critical review. Nurse Educ Pract. 12:242–247.
   PubMedGoogle Scholar
• Strand A, Daniels VJ. 2017. Improving Learning Outcomes through Immediate OSCE Score Sheet Review. Med Educ. 51(Suppl 1):114.
   Google Scholar
• Sturpe DA. 2010. Objective structured clinical examinations in doctor of pharmacy programs in the United States. Am J Pharm Educ. 74:148.
   PubMed Web of Science ®Google Scholar
• Swanson DB, van der Vleuten CP. 2013. Assessment of clinical skills with standardized patients: state of the art revisited. Teach Learn Med. 25 (Suppl 1):S17–S25.
   PubMed Web of Science ®Google Scholar
• Tamblyn R, Abrahamowicz M, Brailovsky C, Grand’Maison P, Lescop J, Norcini J, Girard N, Haggerty J. 1998. Association between licensing examination scores and resource use and quality of care in primary care practice. JAMA. 280:989–996.
   PubMed Web of Science ®Google Scholar
• Walsh M, Bailey PH, Koren I. 2009. Objective structured clinical evaluation of clinical competence: an integrative review. J Adv Nurs. 65:1584–1595.
   PubMed Web of Science ®Google Scholar
• Yousuf N, Violato C, Zuberi RW. 2015. Standard setting methods for pass/fail decisions on high-stakes objective structured clinical examinations: a validity study. Teach Learn Med. 27:280–291.
   PubMed Web of Science ®Google Scholar