Criterion validity of a competency-based assessment center in medical education – a 4-year follow-up study

Abstract

Introduction

Core competencies have progressively gained importance in medical education. In other contexts, especially personnel selection and development, assessment centers (ACs) are used to assess competencies, but there is only a limited number of studies on competency-based ACs in medical education. To the best of our knowledge, the present study provides the first data on the criterion-related validity of a competency-based AC in medical education.

Methods

We developed an AC tailored to measure core competencies relevant to medical education (social-ethical, communicative, self, and teaching) and tested its validity in n=30 first-year medical students using 3- to 4-year follow-up measures such as (a) objective structured clinical examinations (OSCE) on basic clinical skills (n=26), (b) OSCE on communication skills (n=21), and (c) peer feedback (n=18). The AC contained three elements: interview, group discussion, and role play. Additionally, a self-report questionnaire was provided as a basis for the interview.

Results

Baseline AC average score and teaching competency correlated moderately with the communication OSCE average score (r=0.41, p=0.03, and r=0.38, p=0.04, respectively). Social-ethical competency in the AC showed a very strong convergent association with the communication OSCE average score (r=0.60, p<0.01). The AC total score also showed a moderate correlation with the overall peer feedback score provided in Year 4 (r=0.38, p=0.06). In addition, communicative competency correlated strongly with the overall peer feedback (r=0.50, p=0.02). We found predominantly low and insignificant correlations between the AC and the OSCE on basic clinical skills (r=−0.33 to 0.30, all p's>0.05).

Conclusion

The results showed that competency-based ACs can be used at a very early stage of medical training to successfully predict future performance in core competencies.

• assessment
• educational
• student selection
• competency-based education
• self-assessment
• communication
• assessment center

Acknowledgements

We would like to thank the students, the raters, Mrs. Babette Dufrenne for her support in organizing the project, and Mrs. Bianca Raski for her support in data search.

Conflict of interest and funding

All authors declare no conflict of interest. The project was financed in part by tuition fees.