Competency milestones for medical students: Design, implementation, and analysis at one medical school

Abstract

Competency-based assessment seeks to align measures of performance directly with desired learning outcomes based upon the needs of patients and the healthcare system. Recognizing that assessment methods profoundly influence student motivation and effort, it is critical to measure all desired aspects of performance throughout an individual’s medical training. The Accreditation Council for Graduate Medical Education (ACGME) defined domains of competency for residency; the subsequent Milestones Project seeks to describe each learner’s progress toward competence within each domain. Because the various clinical disciplines defined unique competencies and milestones within each domain, it is difficult for undergraduate medical education to adopt existing GME milestones language. This paper outlines the process undertaken by one medical school to design, implement and improve competency milestones for medical students. A team of assessment experts developed milestones for a set of focus competencies; these have now been monitored in medical students over two years. A unique digital dashboard enables individual, aggregate and longitudinal views of student progress by domain. Validation and continuous quality improvement cycles are based upon expert review, user feedback, and analysis of variation between students and between assessors. Experience to date indicates that milestone-based assessment has significant potential to guide the development of medical students.

Acknowledgements

The authors acknowledge Stan Hamstra, PhD for his helpful review and feedback on a draft of this paper.

The content reflects the views the authors and does not necessarily represent the views of AMA or other participants in this Initiative.

Disclosure statement

The authors for this article have no interests to report.

Funding

This publication was prepared with financial support from the American Medical Association (AMA) as part of the Accelerating Change in Medical Education Initiative.

Notes on contributors

Kimberly D. Lomis, MD is associate dean for undergraduate medical education, professor of surgery, and professor of medical education and administration at Vanderbilt University School of Medicine. Her scholarly interests focus around curricular innovation, competency-based assessment, and change management.

Regina G. Russell, MEd, MA is the director of learning system outcomes for the MD program at Vanderbilt University School of Medicine. Her higher education experience includes student services administration, sociology and social work education, learner assessment and program evaluation. Current work includes measurement of medical student outcomes and the impact of organizational environments on learning.

Mario Davidson, PhD, MA (Mathematics Education), MS (Statistics), MAS (Mathematics) is a biostatistician for Vanderbilt University Medical Center. In his career, he has worked in medical and statistical education as an instructor and researcher. He is currently the course director for the Statistical Collaborator course presented in the Department of Biostatistics.

Amy E. Fleming, MD, MHPE is the associate dean for medical student affairs at the Vanderbilt University School of Medicine and associate professor of pediatrics, division of hospital medicine at Monroe Carell Jr Children's Hospital at Vanderbilt. Her research interests span learning communities in medical education, portfolios, learning goals, and mentoring.

William B. Cutrer, MD, MEd is the assistant dean of undergraduate medical education, and associate professor of Pediatrics, Critical Care Medicine at Vanderbilt University School of Medicine. He has published and presented widely on how students learn in the workplace and how to help them more effectively.

Geoffrey M. Fleming, MD FAAP is a Pediatric Intensive Care Unit physician with a history of education pursuits including Program Director for an ACGME accredited fellowship training program and Vice President for Continuous Professional Development in an ACCME accredited institution. He has been involved extensively in education administration leadership at both the local and national level through workshop development, course administration, educational design committees and advocacy.

Cathleen C. Pettepher, PhD, is a Professor of Cancer Biology and Medical Education and Administration and Assistant Dean of Medical Student Assessment at Vanderbilt University School of Medicine. Her scholarly interests have focused on designing and implementing curricular innovations and the development, implementation and evaluations of value of peer assessments in professional growth and identify formation.

Bonnie Miller, MD, MHHC is senior associate dean for health science education at Vanderbilt University School of Medicine and Executive Vice-President for Educational Affairs at Vanderbilt University Medical Center, Professor of Medical Education and Administration and Professor of Clinical Surgery. Her scholarly interests include curriculum innovation and the factors that influence the professional formation of physicians.

Glossary

Competency-based assessment (CBA)

Orients the assessment of learner performance to graduate outcome abilities. It is organized around competencies derived from an analysis of societal and patient needs. Ideally, a major portion of the assessments should be performed in the context of the clinical workplace and should be criterion-referenced. The assessment process is designed to facilitate the developmental progression of competence.

Adapted from:

1. The role of assessment in competency-based medical education. Eric S. Holmboe, Jonathan Sherbino, Donlin M. Long, Susan R. Swing, Jason R. Frank, and for the International CBME Collaborators. Medical Teacher Vol. 32, Iss. 8,2010

2. Toward a definition of competency-based education in medicine: a systematic review of published definitions. Jason R. Frank, Rani Mungroo, Yasmine Ahmad, Mimi Wang, Stefanie De Rossi, and Tanya Horsley. Medical Teacher Vol. 32, Iss. 8,2010

Milestones

Are discrete points or stages in a developmental progression. Milestones frameworks in medical education mirror the Dreyfus model of learner development from novice to expert. Well-articulated milestones for a specific competency provides a shared observational approach for assessors and a clear frame of reference for learners.

Adapted from:

1. Frameworks for learner assessment in medicine: AMEE Guide No. 78. Louis Pangaro and Olle ten Cate. Medical Teacher Vol. 35, Iss. 6, 2013.

2. Accreditation Council for Graduate Medical Education Milestones implementation.