Evaluation using simulated students for reliability of multiple standardized-patients scoring in clinical performance examinations

Abstract

Background: The consistency of rating among different standardized patients (SPs) is very important in clinical performance examinations (CPX).

Aims: We evaluated the usefulness of simulated students to assess the reliability of a group of standardized patients in a CPX.

Methods: Five SPs in each case were trained to simulate the same patient in a CPX. Ten fourth-year medical student volunteers were selected to act as simulated students. After the simulated students encountered the first SP, they reviewed a video of their performance and sequentially repeated the performance with the other 4 SPs.

Results: The average maximum difference (Δmax ± SD) of multiple SPs’ percent scores on a simulated student was 18 ± 7. In 10 pairs of SPs who gave largest Δmax on a given student, the median Δmax was 30 (range, 26–38) while median difference of percent score in the faculty scoring on the same simulated student was 3 (range, 0–7). Five problematic SPs whose ratings were significantly different from 3 or more other SPs’ ratings in each case were identified.

Conclusions: Simulated students may be a useful means to check the consistency of rating among different SPs in a CPX.

Additional information

Notes on contributors

Wan Beom Park

WAN BEOM PARK is assistant professor of office of medical education, Seoul National University College of Medicine, Seoul, Republic of Korea.

Sung A. Lee

SUNG A LEE is SP educators of Seoul•Kyounggi Clinical Performance Examination Consortium, Republic of Korea.

Eun A. Kim

EUN A KIM is SP educators of Seoul•Kyounggi Clinical Performance Examination Consortium, Republic of Korea.

Jin Gwack

JIN GWACK is house staff of department of preventive medicine, Seoul National University College of Medicine, Seoul, Republic of Korea.

Keun-Young Yoo

KEUN-YOUNG YOO is professor of department of preventive medicine, Seoul National University College of Medicine, Seoul, Republic of Korea.

Jwa-Seop Shin

JWA-SEOP SHIN is assistant professor of office of medical education, Seoul National University College of Medicine, Seoul, Republic of Korea.

Yoon Sung Lee

YOON SUNG LEE is director of office of medical education, Seoul National University College of Medicine, Seoul, Republic of Korea