The use of EPA assessments in decision-making: Do supervision ratings correlate with other measures of clinical performance?

References

• Association of American Medical Colleges. 2014. Core Entrustable professional activities for entering residency, Curriculum Developer’s Guide. Washington, DC: AAMC.
   Google Scholar
• Bransen D, Govaerts MJB, Sluijsmans DMA, Driessen EW. 2020. Beyond the self: the role of co-regulation in medical students’ self-regulated learning. Med Educ. 54(3):234–241.
   PubMed Web of Science ®Google Scholar
• Bray MJ, Bradley EB, Martindale JR, Gusic ME. 2020. Implementing systematic faculty development to support an EPA-based program of assessment: strategies, outcomes and lessons learned. Teach Learn Med. 2020:1–31.
   Web of Science ®Google Scholar
• Bullock JL, Lai CJ, Lockspeiser T, O’Sullivan PS, Aronowitz P, Dellmore D, Fung C-C, Knight C, Hauer KE. 2019. In pursuit of honors: a multi-institutional study of students’ perceptions of clerkship evaluation and grading. Acad Med. 94(11S):S48–S56.
   PubMed Web of Science ®Google Scholar
• Chen H, Carrie W, Sjoukje van den Broek E, ten Cate O. 2015. The case for use of entrustable professional activities in undergraduate medical education. Acad Med. 90(4):431–436.
   PubMed Web of Science ®Google Scholar
• Cohen J. 2013. Statistical power analysis for the behavioral sciences. New York: Routledge.
   Google Scholar
• “Competencies Required of the Contemporary Physician.” 2021. Undergraduate Medical Education Curriculum; [accessed 2021 Jun 25]. https://med.virginia.edu/ume-curriculum/curriculum/competencies-required-of-the-contemporary-physician/.
   Google Scholar
• Cook DA, Brydges R, Ginsburg S, Hatala R. 2015. A contemporary approach to validity arguments: a practical guide to Kane’s framework. Med Educ. 49(6):560–575.
   PubMed Web of Science ®Google Scholar
• Cook DA, Hatala R. 2016. Validation of educational assessments: a primer for simulation and beyond. Adv Simul. 1:31.
   PubMedGoogle Scholar
• Durning SJ, Cation LJ, Markert RJ, Pangaro LN. 2002. Assessing the reliability and validity of the mini-clinical evaluation exercise for internal medicine residency training. Acad Med. 77(9):900–904.
   PubMed Web of Science ®Google Scholar
• Englander R, Flynn T, Call S, Carraccio C, Cleary L, Fulton TB, Garrity MJ, Lieberman SA, Lindeman B, Lypson ML, et al. 2016. Toward defining the foundation of the MD degree: core entrustable professional activities for entering residency. Acad Med. 91(10):1352–1358.
   PubMed Web of Science ®Google Scholar
• Epstein RM. 2007. Assessment in medical education. N Engl J Med. 356(4):387–396.
   PubMed Web of Science ®Google Scholar
• Hall AK, Rich J, Damon Dagnone J, Weersink K, Caudle J, Sherbino J, Frank JR, Bandiera G, Van Melle E. 2020. It’s a Marathon, not a sprint: rapid evaluation of competency-based medical education program implementation. Acad Med. 95(5):786–793.
   PubMed Web of Science ®Google Scholar
• Hauer KE, Iverson N, Quach A, Yuan P, Kaner S, Boscardin C. 2018. Fostering Medical students’ lifelong learning skills with a dashboard, coaching and learning planning. Perspect Med Educ. 7(5):311–317.
   PubMed Web of Science ®Google Scholar
• Hodges B. 2003. Validity and the OSCE. Med Teach. 25(3):250–254.
   PubMed Web of Science ®Google Scholar
• Hodwitz K, Kuper A, Brydges R. 2019. Realizing one’s own subjectivity: assessors’ perceptions of the influence of training on their conduct of workplace-based assessments. Acad Med. 94(12):1970–1979.
   PubMed Web of Science ®Google Scholar
• Humphrey-Murto S, Côté M, Pugh D, Wood TJ. 2018. Assessing the validity of a multidisciplinary mini-clinical evaluation exercise. Teach Learn Med. 30(2):152–161.
   PubMed Web of Science ®Google Scholar
• Keeley MG, Gusic ME, Morgan HK, Aagaard EM, Santen SA. 2019. Moving toward summative competency assessment to individualize the Postclerkship Phase. Acad Med. 94(12):1858–1864.
   PubMed Web of Science ®Google Scholar
• Kelleher M, Kinnear B, Sall D, Schumacher D, Schauer DP, Warm EJ, Kelcey B. 2020. A reliability analysis of entrustment-derived workplace-based assessments. Acad Med. 95(4):616–622.
   PubMed Web of Science ®Google Scholar
• Kogan JR, Conforti LN, Yamazaki K, Iobst W, Holmboe ES. 2017. Commitment to change and challenges to implementing changes after workplace-based assessment rater training. Acad Med. 92(3):394–402.
   PubMed Web of Science ®Google Scholar
• Kogan JR, Holmboe ES, Hauer KE. 2009. Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review. JAMA. 302(12):1316–1326.
   PubMed Web of Science ®Google Scholar
• Lee V, Brain K, Martin J. 2017. Factors influencing mini-cex rater judgments and their practical implications: a systematic literature review. Acad Med. 92(6):880–887.
   PubMed Web of Science ®Google Scholar
• Lörwald AC, Lahner F-M, Mooser B, Perrig M, Widmer MK, Greif R, Huwendiek S. 2019. Influences on the implementation of mini-CEX and DOPS for Postgraduate Medical trainees’ learning: a grounded theory study. Med Teach. 41(4):448–456.
   PubMed Web of Science ®Google Scholar
• Lupi CS, Ownby AR, Jokela JA, Cutrer WB, Thompson-Busch AK, Catallozzi M, Noble JM, Amiel JM. and Association of American Medical Colleges Core Entrustable Professional Activities for Entering Residency Faculty Development Concept Group. 2018. Faculty development revisited: a systems-based view of stakeholder development to meet the demands of entrustable professional activity implementation. Acad Med. 93(10):1472–1479.
   PubMed Web of Science ®Google Scholar
• Mann K, van der Vleuten C, Eva K, Armson H, Chesluk B, Dornan T, Holmboe E, Lockyer J, Loney E, Sargeant J. 2011. Tensions in informed self-assessment: how the desire for feedback and reticence to collect and use it can conflict. Acad Med. 86(9):1120–1127.
   PubMed Web of Science ®Google Scholar
• McGill DA, van der Vleuten CPM, Clarke MJ. 2011. Supervisor assessment of clinical and professional competence of medical trainees: a reliability study using workplace data and a focused analytical literature review. Adv Health Sci Educ Theory Pract. 16(3):405–425.
   PubMed Web of Science ®Google Scholar
• Meyer EG, Chen HC, Uijtdehaage S, Durning SJ, Maggio LA. 2019. Scoping review of entrustable professional activities in undergraduate medical education. Acad Med. 94(7):1040–1049.
   PubMed Web of Science ®Google Scholar
• Mortaz Hejri S, Jalili M, Masoomi R, Shirazi M, Nedjat S, Norcini J. 2020. The utility of mini-clinical evaluation exercise in undergraduate and postgraduate medical education: a BEME review: BEME Guide No. 59. Med Teach. 42(2):125–142.
   PubMed Web of Science ®Google Scholar
• Ney EM, Shea JA, Kogan JR. 2009. Predictive validity of the Mini-Clinical Evaluation Exercise (Mcex): do medical students’ mCEX ratings correlate with future clinical exam performance? Acad Med. 84(10 Suppl):S17–S20.
   PubMed Web of Science ®Google Scholar
• Norcini JJ, Blank LL, Arnold GK, Kimball HR. 1995. The Mini-CEX (Clinical Evaluation Exercise): a preliminary investigation. Ann Intern Med. 123(10):795–799.
   PubMed Web of Science ®Google Scholar
• Norcini J, Anderson B, Bollela V, Burch V, Costa MJ, Duvivier R, Galbraith R, Hays R, Kent A, Perrott V, et al. 2011. Criteria for good assessment: consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach. 33(3):206–214.
   PubMed Web of Science ®Google Scholar
• Norcini J, Burch V. 2007. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach. 29(9):855–871.
   PubMed Web of Science ®Google Scholar
• O’Dowd E, Lydon S, O’Connor P, Madden C, Byrne D. 2019. A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011–2018. Med Educ. 53(3):234–249.
   PubMed Web of Science ®Google Scholar
• Parsons AS, Kon RH, Plews-Ogan M, Gusic ME. 2020. You can have both: coaching to promote clinical competency and professional identity formation. Perspect Med Educ. 10:57–63.
   Web of Science ®Google Scholar
• Pelgrim E. a M, Kramer AWM, Mokkink HGA, van den Elsen L, Grol RPTM, van der Vleuten CPM. 2011. In-training assessment using direct observation of single-patient encounters: a literature review. Adv Health Sci Educ Theory Pract. 16(1):131–142.
   PubMed Web of Science ®Google Scholar
• Shorey S, Lau TC, Lau ST, Ang E. 2019. Entrustable professional activities in health care education: a scoping review. Med Educ. 53(8):766–777.
   PubMed Web of Science ®Google Scholar
• Shumway JM, Harden RM. and Association for Medical Education in Europe. 2003. AMEE Guide No. 25: the assessment of learning outcomes for the competent and reflective physician. Med Teach. 25(6):569–584.
   PubMed Web of Science ®Google Scholar
• Tekian A, Harden RM, Cook DA, Steinert Y, Hunt D, Norcini J. 2020. Managing the tension: from innovation to application in health professions education. Med Teach. 42(3):333–339.
   PubMed Web of Science ®Google Scholar
• Tekian A, Ten Cate O, Holmboe E, Roberts T, Norcini J. 2020. Entrustment decisions: implications for curriculum development and assessment. Med Teach. 42(6):698–704.
   PubMed Web of Science ®Google Scholar
• ten Cate O. 2005. Entrustability of professional activities and competency-based training. Med Educ. 39(12):1176–1177.
   PubMed Web of Science ®Google Scholar
• Ten Cate O, Schwartz A, Chen HC. 2020. Assessing trainees and making entrustment decisions: on the nature and use of entrustment-supervision scales. Acad Med. 95(11):1662–1669.
   PubMed Web of Science ®Google Scholar
• Ten Cate O, Taylor DR. 2020. The recommended description of an entrustable professional activity: AMEE Guide No. 140. Med Teach. 2020:1–9.
   Web of Science ®Google Scholar
• van der Vleuten CPM, Schuwirth LWT. 2005. Assessing professional competence: from methods to programmes. Med Educ. 39(3):309–317.
   PubMed Web of Science ®Google Scholar
• Van Melle E, Frank JR, Holmboe ES, Dagnone D, Stockley D, Sherbino J. and International Competency-based Medical Education Collaborators. 2019. A core components framework for evaluating implementation of competency-based medical education programs. Acad Med. 94(7):1002–1009.
   PubMed Web of Science ®Google Scholar