Reliability and benefits of medical student peers in rating complex clinical skills

Abstract

Background: Peers have been shown to be reliable raters in an objective structured clinical exam (OSCE). While the literature supports the use of senior level students in rating basic clinical skills, little is known about the reliability of peers of the same level of training in assessing complex clinical skills.

Aim: To investigate the reliability of student peers of the same level of training in rating complex clinical skills in a geriatric OSCE.

Methods: Peer (n = 115) ratings were compared to faculty ratings using correlation and generalizability analysis. Paired Wilcoxon Signed-Rank test was used to establish peer learning benefits.

Results: Reliability of the OSCE was moderately strong (G-coefficient = 0.70) with strong correlations between peer and faculty ratings for the overall OSCE (r = 0.78, p = 0.001) and for each case (r = 0.70–0.85, p = 0.001). Generalizability analysis indicated that raters contributed minimally to score variance. Peers reported gaining learning benefits from the rating process.

Conclusion: Peer raters of the same level of training can provide accurate ratings of complex clinical tasks and can serve as an important resource in assessing student performance in an OSCE. The peer review process can also serve an important role in enhancing student learning.

Acknowledgements

The authors wish to thank especially Michael Nagoshi, MD, from the University of Hawaii for his ongoing support and willingness to offer his expertise and OSCE cases to advance the project. The authors likewise acknowledge the support within Rowan University School of Osteopathic Medicine (formerly UMDNJ-School of Osteopathic Medicine) including Anita Chopra, MD, Director, New Jersey Institute for Successful Aging; Sima C. Bennett, MPH, RD, Director, Clinical Education and Assessment Center; Patrick Stewart, MBA, Senior Technologist and Paul Krueger, DO, former Associate Dean for Academic Affairs for their dedication to the success of the project.

Declaration of interest: The authors report no conflict of interest. The authors alone are responsible for the content and writing of this article.