A study of Two Clinical Performance Scores: Assessing the Psychometric Characteristics of a Combined Score Derived from Clinical Evaluation Forms and OSCEs

Abstract

Background/Purpose - It is important to improve the quality of clinical skill assessments. In addition to using the OSCE, the clinical skills of medical students are assessed with clinical evaluation forms (CEFs). The purpose of this study is to examine the psychometric characteristics of an OSCE/CEF composite score.

Methods - This study included 2 medical student classes from a large medical school. Students completed approximately 12 OSCEs and were rated over 33 times on a CEF. The reliability of the CEF and OSCE were estimated. A correlation between the mean OSCE score and the mean CEF was calculated and corrected for attenuation. Classical methods were used to examine composite score reliability.

Results - For both classes there was a statistically significant correlation between the CEF and OSCE (r = .27 & .42, p = .003 & .0001). The disattenuated correlations were .44 and .68. Weighting the OSCE in the composite score as high as .4 was associated with only a small decrease in composite reliability.

Conclusion - These results demonstrate that assessment information based on simulated and actual patient encounters can be combined into a composite. Since a composite score may provide a more valid measure of clinical performance, this study supports using a combined CEF and OSCE measure. Keywords: psychometrics, composite score, clinical skills, OSCE, reliability

Key words:

• psychometrics
• composite score
• clinical skills
• OSCE
• reliability