Viewing Readiness-for-Residency through Binoculars: Mapping Competency-Based Assessments to the AAMC’s 13 Core Entrustable Professional Activities (EPAs)

Abstract

Construct: The construct being assessed is readiness-for-residency of graduating medical students, as measured through two assessment frameworks. Background: Readiness-for-residency of near-graduate medical students should be but is not consistently assessed. To address this, the Association of American Medical Colleges (AAMC), in 2014, identified and described 13 core Entrustable Professional Activities (EPAs), which are tasks that all residents should be able to perform unsupervised upon entering residency. However, the AAMC did not initially provide measurement guidelines or propose standardized assessments. We designed Night-onCall (NOC), an immersive simulation for our near-graduating medical students to assess and address their readiness-for-residency, framed around tasks suggested by the AAMC’s core EPAs. In adopting this EPA assessment framework, we began by building upon an established program of competency-based clinical skills assessments, repurposing competency-based checklists to measure components of the EPAs where possible, and designing new checklists, when necessary. This resulted in a blended suite of 14 checklists, which theoretically provide substantive assessment of all 13 core EPAs. In this paper, we describe the consensus-based mapping process conducted to ensure we understood the relationship between competency and EPA-based assessment lenses and could therefore report meaningful feedback on both to transitioning students in the NOC exercise. Approach: Between January-November 2017, five clinician and two non-clinician health professions educators at NYU Grossman School of Medicine conducted a rigorous consensus-based mapping process, which included each rater mapping each of the 310 NOC competency-based checklist items to lists of entrustable behaviors expected of learners according to the AAMC 13 core EPAs. Findings: All EPAs were captured to varying degrees by the 14 NOC checklists (overall Intraclass Correlation Coefficient (ICC) = 0.77). Consensus meetings resolved discrepancies and improved ICC values for three (EPA-9, EPA-10, EPA-12) of the four EPAs that initially showed poor reliability. Conclusions: Findings suggest that with some limitations (e.g., EPA-7 “form clinical questions/retrieve evidence”) established competency-based assessments can be repurposed to measure readiness-for-residency through an EPA lens and both can be reported to learners and faculty.

Keywords:

• Entrustable professional activities
• EPAs
• assessment
• undergraduate medical education
• competency-based medical education

Acknowledgements

We deeply appreciate the long-term financial support for this project from Dr. Thomas Riles, Associate Dean for Medical Education and Technology at the NYU Grossman School of Medicine (retired), founding director of the New York Simulation Center (NYSIM), and founder of WISE-MD and WISE-onCall, both distributed by Aquifer, as well as the Josiah Macy Jr. Foundation, which now supports the Night-onCall consortium.

Disclosure statement

No potential conflict of interest was reported by the authors.

Previous presentations

An earlier version of this work was presented at: SGIM 2018 in Denver CO, USA and AAMC 2018 in Austin TX, USA.

Funding

The author(s) reported there is no funding associated with the work featured in this article.