Assessing Advanced Communication Skills via Objective Structured Clinical Examination: A Comparison of Faculty Versus Self, Peer, and Standardized Patient Assessors

Abstract

Construct: The construct addressed in this study is assessment of advanced communication skills among senior medical students. Background: The question of who should assess participants during objective structured clinical examinations (OSCEs) has been debated, and options discussed in the literature have included peer, self, standardized patient, and faculty assessment models. What is not known is whether same-level peer assisted learning can be utilized for formative assessment of advanced communication skills when no faculty, standardized patients, or other trained assessors are involved in providing feedback. If successful, such an educational model would optimize resource utilization and broaden the scope of topics that could be covered in formative OSCEs. Approach: The investigators developed a 4-station formative OSCE focused on advanced communication skills for senior medical students, and evaluated the concordance of assessment done by same-level peers, self, standardized patients, and faculty for 45 students. After each station, examinees completed a self-assessment checklist and received checklist-based assessment and verbal feedback from same-level peers only. Standardized patients completed checklist-based assessments outside the room, and faculty did so after the OSCE via video review; neither group provided direct feedback to examinees. The investigators assessed inter-rater agreement and mean difference scores on the checklists using faculty score as the gold standard. Findings: There was fair to good overall agreement among self, same-level peer, standardized patient, and faculty-assessment of advanced communication skills. Relative to faculty, peer and standardized patient assessors overestimated advanced communication skills, while self-assessments underestimated skills. Conclusions: Self and same-level peer-assessment may be a viable alternative to faculty assessment for a formative OSCE on advanced communication skills for senior medical students.

Keywords:

• OSCE
• communication skills
• assessment
• feedback
• standardized patient

Acknowledgments

The authors would like to thank John Encandela, PhD, Barbara Hildebrand, Alan Kliger, MD, David Rosenthal, MD, Anna Reisman, MD, Veronika Shabanova, PhD, Tracy Yale, and the other members of the Clinical Skills Instruction team, Capstone Course team, and Standardized Patient Program at Yale School of Medicine who helped with the development, deployment, and evaluation of this project.

Conflict of interest

Dr. Fortin is a coauthor of Smith’s Patient-Centered Interviewing: An Evidence-Based Method, 4th Ed., McGraw Hill Education, New York, 2019; all proceeds from the textbook go to the Academy of Communication in Healthcare. The authors have no other conflicts to disclose.

Approvals

This study received exemption from review by the Yale University Institutional Review Board due to the educational nature of the project on March 5, 2017 (IRB Protocol ID 2000020576).

Previous presentations

Poster presentation, Society of General Internal Medicine, National Meeting. Denver, CO. April 2018.

Poster presentation, Pediatric Academic Societies, National Meeting. Toronto, ON. May 2018.