R.I.M.E. and reason: multi-station OSCE enhancement to neutralize grade inflation

ABSTRACT

To offset grade inflation, many clerkships combine faculty evaluations with objective assessments including the Medical Examiners Subject Examination (NBME-SE) or Objective Structured Clinical Examination (OSCE), however, standardized methods are not established. Following a curriculum transition removing faculty clinical evaluations from summative grading, final clerkship designations of fail (F), pass (P), and pass-with-distinction (PD) were determined by combined NBME-SE and OSCE performance, with overall PD for the clerkship requiring meeting this threshold in both. At the time, 90% of students achieved PD on the Internal Medicine (IM) OSCE resulting in overall clerkship grades primarily determined by the NBME-SE. The clerkship sought to enhance the OSCE to provide a more thorough objective clinical skills assessment, offset grade inflation, and reduce the NBME-SE primary determination of the final clerkship grade. The single-station 43-point OSCE was enhanced to a three-station 75-point OSCE using the Reporter-Interpreter-Manager-Educator (RIME) framework to align patient encounters with targeted assessments of progressive skills and competencies related to the clerkship rotation. Student performances were evaluated pre- and post-OSCE enhancement. Student surveys provided feedback about the clinical realism of the OSCE and the difficulty. Pre-intervention OSCE scores were more tightly clustered (SD = 5.65%) around a high average performance with scores being highly negatively skewed. Post-intervention OSCE scores were more dispersed (SD = 6.88%) around a lower average with scores being far less skewed resulting in an approximately normal distribution. This lowered the total number of students achieving PD on the OSCE and PD in the clerkship, thus reducing the relative weight of the NMBE-SE in the overall clerkship grade. Student response was positive, indicating the examination was fair and reflective of their clinical experiences. Through structured development, OSCE assessment can provide a realistic and objective measurement of clinical performance as part of the summative evaluation of students.

KEYWORDS:

• OSCE
• clerkship
• assessment
• grade inflation
• RIME
• internal medicine

Acknowledgments

The authors would like to thank the University of Kansas Health System, Department of Internal Medicine, standardized patients, piloting students, and many faculty who were essential to the development and implementation of the OSCE across multiple campuses. The authors would like to thank Scott Owings and Ali Jahansooz who provided input into the early phases of OSCE development to align clinical content, assessment goals, and OSCE structure along with Cayla Teal who helped develop review and establish post-intervention grade thresholds following score review. The authors would like to thank Ali Jahansooz, Cynthia Kibet, and Brad Daily who continue to serve on the tri-campus grading committee. The authors would lastly like to give a very special thank you for the many people in our tri-campus Clinical Skills teams including David Becker, Julie Mack, Ralph Course, and Carey Rawson who were essential initial OSCE development, piloting, technological innovation and support, and standardized patient development, who along with Sarah Talley, Brad Daily and Mary Vernon continued implementation and development of the OSCE across multiple campuses.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Ethical approval

This study was formally submitted at our institution’s IRB for review and was deemed a quality improvement project due to the required nature for all students’ participation in the OSCE experience and therefore IRB approval was waived 4 September 2021.

Data

No outside data sources. All data was obtained by and interpreted by the authors.

Additional information

Funding

This project was funded by a one-time Medical Education Mentored Grant through The Academy of Medical Educators at the University of Kansas School of Medicine in 2021.