On the validity of summative entrustment decisions

Abstract

Health care revolves around trust. Patients are often in a position that gives them no other choice than to trust the people taking care of them. Educational programs thus have the responsibility to develop physicians who can be trusted to deliver safe and effective care, ultimately making a final decision to entrust trainees to graduate to unsupervised practice. Such entrustment decisions deserve to be scrutinized for their validity. This end-of-training entrustment decision is arguably the most important one, although earlier entrustment decisions, for smaller units of professional practice, should also be scrutinized for their validity. Validity of entrustment decisions implies a defensible argument that can be analyzed in components that together support the decision. According to Kane, building a validity argument is a process designed to support inferences of scoring, generalization across observations, extrapolation to new instances, and implications of the decision. A lack of validity can be caused by inadequate evidence in terms of, according to Messick, content, response process, internal structure (coherence) and relationship to other variables, and in misinterpreted consequences. These two leading frameworks (Kane and Messick) in educational and psychological testing can be well applied to summative entrustment decision-making. The authors elaborate the types of questions that need to be answered to arrive at defensible, well-argued summative decisions regarding performance to provide a grounding for high-quality safe patient care.

Keywords:

• Assessment: general
• learning outcomes: decision-making
• teaching and learning: work-based

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Claire Touchie

Claire Touchie, MD, MHPE, FRCPC, is chief medical education officer at the Medical Council of Canada and professor of medicine at the University of Ottawa, Ottawa, Ontario, Canada.

Benjamin Kinnear

Benjamin Kinnear, MD, MEd, is associate professor of internal medicine and pediatrics at the University of Cincinnati College of Medicine/Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio, USA.

Daniel Schumacher

Daniel Schumacher, MD, PhD, MEd, is associate professor of pediatrics at Cincinnati Children’s Hospital Medical Center/University of Cincinnati College of Medicine, Cincinnati, Ohio, USA.

Holly Caretta-Weyer

Holly Caretta-Weyer, MD, MHPE, is assistant professor of emergency medicine at the Stanford University School of Medicine, Palo Alto, California, USA.

Stanley J. Hamstra

Stanley J. Hamstra, PhD, is professor of surgery at the University of Toronto, Toronto, Ontario, Canada, and research consultant, Accreditation Council for Graduate Medical Education, Chicago, Illinois, USA.

Danielle Hart

Danielle Hart, MD, MACM, is associate professor of emergency medicine at Hennepin Healthcare and the University of Minnesota, Minneapolis, Minnesota, USA.

Larry Gruppen

Larry Gruppen, PhD, is professor of learning health sciences at the University of Michigan Medical School, Ann Arbor, Michigan, USA.

Shelley Ross

Shelley Ross, PhD, is associate professor in the Department of Family Medicine at the University of Alberta, Edmonton, Alberta, Canada.

Eric Warm

Eric Warm, MD, is professor of medicine and internal medicine program director, University of Cincinnati College of Medicine Center, Cincinnati, Ohio, USA.

Olle ten Cate

Olle ten Cate, PhD, is professor of medical education and senior scientist at the Center for Research and Development of Education at University Medical Center Utrecht, The Netherlands.