Raising the Bar: Reassessing Standards for Procedural Competence

Abstract

Background: How often passing standards for clinical skills examinations should be reexamined is unknown. Purpose: The goal is to determine if improved resident performance affects the passing standard imposed by expert judges. Method: In 2006, we set a passing standard for a central venous catheter insertion clinical skills examination using an expert panel. From 2007 to 2010, trainee scores improved steadily. In 2010, a new expert panel provided judgments for the examination using 4 consecutive years of resident performance data. Interrater reliabilities and test–retest reliability (stability) were calculated. Passing standards from 2006 and 2010 were compared. Results: Judgments provided were reliable and stable. The new passing standard was 88% for internal jugular and 87% for subclavian central venous catheter insertion compared to 79% for both sites in 2006. Conclusions: Cumulative performance data influenced experts to set a more stringent minimum passing standard. Standards should be regularly reviewed to ensure they are fair and appropriately rigorous.

Acknowledgments

The Excellence in Academic Medicine Act is supported by the Illinois Department of Healthcare and Family Services. Dr. McGaghie's contribution was supported in part by the Jacob R. Suker, MD, professorship in medical education and by grant UL 1 RR 025741 from the National Center for Research Resources, National Institutes of Health (NIH). The NIH had no role in the preparation, review, or approval of the manuscript. We thank the Northwestern University internal medicine residents for their dedication to education and patient care. We acknowledge Dr. Douglas E. Vaughan for his support and encouragement of this work.