Entrustment versus performance scale in high-stakes OSCEs: Rater insights and psychometric properties

References

• Acai A, Cupido N, Weavers A, Saperson K, Ladhani M, Cameron S, Sonnadara RR. 2021. Competence committees: the steep climb from concept to implementation. Med Educ. 55(9):1067–1077.
   PubMed Web of Science ®Google Scholar
• Bearman M, Ajjawi R, Bennett S, Boud D. 2021. The hidden labours of designing the objective structured clinical examination: a practice theory study. Adv Health Sci Educ. 26(2):637–651.
   PubMed Web of Science ®Google Scholar
• Berendonk C, Schirlo C, Balestra G, Bonvin R, Feller S, Huber P, Jünger E, Monti M, Schnabel K, Beyeler C. 2015. The new final clinical skills examination in human medicine in Switzerland: essential steps of exam development, implementation and evaluation, and central insights from the perspective of the national Working Group. GMS Zeitschrift Für Medizinische Ausbildung. 32(4):Doc40.
   PubMedGoogle Scholar
• Boursicot KA, Roberts TE, Burdick WP. 2019. Structured assessments of clinical competence. In: swanwick T, Forrest K, O’Brien BC, editors. Understanding medical education: evidence, theory, and practice. Oxford: John Wiley & Sons; p. 335–345.
   Google Scholar
• Braun V, Clarke V. 2006. Using thematic analysis in psychology. Qual Res Psychol. 3(2):77–101.
   Google Scholar
• Brennan RL. 2001. Generalizability theory – statistics for social and behavioral sciences. Berlin: springer.
   Google Scholar
• CarlLee S, Rowat J, Suneja M. 2019. Assessing entrustable professional activities using an orientation OSCE: identifying the gaps. J Grad Med Educ. 11(2):214–220.
   PubMedGoogle Scholar
• Chen CH, McNamara M, Teherani A, Cate OT, O'Sullivan P. 2016. Developing entrustable professional activities for entry into clerkship. Acad Med. 91(2):247–255.
   PubMed Web of Science ®Google Scholar
• G_String. 2019. A Windows wrapper for urGENOVA. [accessed 2019 Sept 12]. https://github.com/G-String-Legacy/G_String.
   Google Scholar
• Guttormsen S, Beyeler C, Bonvin R, Feller S, Schirlo C, Schnabel K, Schurter T, Berendonk C. 2013. The new licencing examination for human medicine: from concept to implementation. Swiss Med Wkly. 143:w13897.
   PubMed Web of Science ®Google Scholar
• Halman S, Fu AYN, Pugh D. 2020. Entrustment within an objective structured clinical examination (OSCE) progress test: bridging the gap towards competency-based medical education. Med Teach. 42(11):1283–1288.
   PubMed Web of Science ®Google Scholar
• Harden RM, Stevenson M, Downie WW, Wilson G. 1975. Assessment of clinical competence using objective structured examination. Br Med J. 1(5955):447–451.
   PubMed Web of Science ®Google Scholar
• Hauer KE, Oza SK, Kogan JR, Stankiewicz CA, Stenfors-Hayes T, Cate Ot, Batt J, O'Sullivan PS. 2015. How clinical supervisors develop trust in their trainees: a qualitative study. Med Educ. 49(8):783–795.
   PubMed Web of Science ®Google Scholar
• Hauer KE, ten Cate O, Boscardin C, Irby DM, Iobst, W, O'Sullivan PS. 2014. Understanding trust as an essential element of trainee supervision and learning in the workplace. Adv Health Sci Educ. 19(3):435–456.
   PubMed Web of Science ®Google Scholar
• Heeneman S, de Jong LH, Dawson LJ, Wilkinson TJ, Ryan A, Tait GR, Rice N, Torre D, Freeman A, van der Vleuten CPM. 2021. Ottawa 2020 consensus statement for programmatic assessment–1. Agreement on the principles. Med Teach. 43(10):1139–1148.
   PubMed Web of Science ®Google Scholar
• Hodges B, McIlroy JH. 2003. Analytic global OSCE ratings are sensitive to level of training. Med Educ. 37(11):1012–1016.
   PubMed Web of Science ®Google Scholar
• Holzhausen Y, Maaz A, Cianciolo AT, ten Cate O, Peters H. 2017. Applying occupational and organizational psychology theory to entrustment decision-making about trainees in health care: a conceptual model. Perspect Med Educ. 6(2):119–126. English.
   PubMed Web of Science ®Google Scholar
• Holzhausen Y, Maaz A, März M, Sehy V, Peters H. 2019. Exploring the introduction of entrustment rating scales in an existing objective structured clinical examination. BMC Med Educ. 19(1):1–9.
   PubMed Web of Science ®Google Scholar
• Hsieh H-F, Shannon SE. 2005. Three approaches to qualitative content analysis. Qual Health Res. 15(9):1277–1288.
   PubMed Web of Science ®Google Scholar
• Huwendiek S, Jung D, Schirlo C, Huber P, Balestra G, Guttormsen S, Berendonk C. 2020. The introduction of a standardised national licensing exam as a driver of change in medical education: a qualitative study from Switzerland. Med Teach. 42(10):1163–1170.
   PubMed Web of Science ®Google Scholar
• Huynh H, Feldt LS. 1976. Estimation of the Box correction for degrees of freedom from sample data in randomized block and split-plot designs. Journal of Educational Statistics. 1(1):69–82.
   Google Scholar
• Ilgen JS, Ma IW, Hatala R, Cook DA. 2015. A systematic review of validity evidence for checklists versus global rating scales in simulation‐based assessment. Med Educ. 49(2):161–173.
   PubMed Web of Science ®Google Scholar
• Krings R, Feller S, Wittwer I, Schnabel K, Steinlin M, Huwendiek S. 2021. Elementary school children as standardized patients in a summative OSCE – a mixed-method study according to the Ottawa criteria for good assessment. Med Teach. 43(10):1170–1178.
   PubMed Web of Science ®Google Scholar
• Lavelle E, Vuk J, Barber C. 2013. Twelve tips for getting started using mixed methods in medical education research. Med Teach. 35(4):272–276.
   PubMed Web of Science ®Google Scholar
• Malau‐Aduli BS, Hays RB, D’Souza K, Smith AM, Jones K, Turner R, Shires L, Smith J, Saad S, Richmond C, et al. 2021. Examiners’ decision‐making processes in observation‐based clinical examinations. Med Educ. 55(3):344–353.
   PubMed Web of Science ®Google Scholar
• Maudsley G. 2011. Mixing it but not mixed-up: mixed methods research in medical education (a critical narrative review). Med Teach. 33(2):e92–e104.
   PubMed Web of Science ®Google Scholar
• Newble D. 2004. Techniques for measuring clinical competence: objective structured clinical examinations. Med Educ. 38(2):199–203.
   PubMed Web of Science ®Google Scholar
• Patrício MF, Julião M, Fareleira F, Carneiro AV. 2013. Is the OSCE a feasible tool to assess competencies in undergraduate medical education? Med Teach. 35(6):503–514.
   PubMed Web of Science ®Google Scholar
• Pell G, Fuller R, Homer M, Roberts T. 2010. How to measure the quality of the OSCE: a review of metrics–AMEE guide no. 49. Med Teach. 32(10):802–811.
   PubMed Web of Science ®Google Scholar
• Pinilla S, Kyrou A, Maissen N, Klöppel S, Strik W, Nissen C, Huwendiek S. 2021. Entrustment decisions and the clinical team: a case study of early clinical students. Med Educ. 55(3):365–375.
   PubMed Web of Science ®Google Scholar
• Regehr G, MacRae H, Reznick RK, Szalay D. 1998. Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Acadmic Medicine. 73:993–997.
   PubMed Web of Science ®Google Scholar
• Rekman J, Gofton W, Dudek N, Gofton T, Hamstra SJ. 2016. Entrustability scales: outlining their usefulness for competency-based clinical assessment. Acad Med. 91(2):186–190.
   PubMed Web of Science ®Google Scholar
• Rotthoff T, Kadmon M, Harendza S. 2021. It does not have to be either or! Assessing competence in medicine should be a continuum between an analytic and a holistic approach. Adv Health Sci Educ. 26(5):1659–1673.
   PubMed Web of Science ®Google Scholar
• ten Cate O, Carraccio C, Damodaran A, Gofton W, Hamstra SJ, Hart DE, Richardson D, Ross S, Schultz K, Warm EJ, et al. 2021. Entrustment decision making: extending Miller’s pyramid. Acad Med. 96(2):199–204.
   PubMed Web of Science ®Google Scholar
• ten Cate O, Schwartz A, Chen HC. 2020. Assessing trainees and making entrustment decisions: on the nature and use of entrustment-supervision scales. Acad Med. 95(11):1662–1669.
   PubMed Web of Science ®Google Scholar
• Torre D, Rice NE, Ryan A, Bok H, Dawson LJ, Bierer B, Wilkinson TJ, Tait GR, Laughlin T, Veerapen K, et al. 2021. Ottawa 2020 consensus statements for programmatic assessment–2. Implementation and practice. Med Teach. 43(10):1149–1160.
   PubMed Web of Science ®Google Scholar
• Valentine N, Shanahan EM, Durning SJ, Schuwirth L. 2021. Making it fair: learners’ and assessors’ perspectives of the attributes of fair judgement. Med Educ. 55(9):1056–1066.
   PubMed Web of Science ®Google Scholar
• Weller J, Misur M, Nicolson S, Morris J, Ure S, Crossley J, Jolly B. 2014. Can I leave the theatre? A key to more reliable workplace-based assessment. Br J Anaesth. 112(6):1083–1091.
   PubMed Web of Science ®Google Scholar
• Yeates P, Moult A, Cope N, McCray G, Fuller R, McKinley R. 2022. Determining influence, interaction and causality of contrast and sequence effects in OSCEs. Med Educ. 56(3):292–302.
   PubMed Web of Science ®Google Scholar