Undergraduate student assessors in a formative OSCE station

Dear Sir

The nature of marking and rating in OSCEs is such that any examiner, who has been trained, can assess a station. Bucknall et al. (2007) have shown that peer assessment has a high correlation with the faculty assessment of Basic Life Support (BLS) in healthcare students.

We took a cohort of undergraduate medical students sitting a formative OSCE (November 2011) and looked at the BLS station. Candidates were assessed by two independent OSCE assessors: a staff and a student assessor. Prior to the assessment, both assessors underwent separate but identical training specific to the station. All assessors remained blind to each other's marking and rating with resultant data assessed for differences within and between assessor groups.

We considered the component skill tasks of the BLS station and compared the marks recorded for each candidate (n = 75) by each assessor. The spread of marks assigned by assessor groups were not normally distributed and therefore we employed the non-parametric Kruskal–Walis test to analyse difference. Variation in marking amongst student assessors (n = 3) was not significant while the variation in marking amongst staff assessors was significant (p < 0.01, n = 5).

The variation between average staff assessor marks and average student assessor marks for a given candidate was examined using the Wilcoxon match-pairs-ranks test, with Spearman's rank correlation test showing effective pairing (p < 0.0001). The former illustrated a significant difference between the marks given to a candidate by staff assessors compared to the marks given to the same candidate by a student assessor (p < 0.002).

Analysing the marks awarded to specific tasks indicated that the two groups of assessors differed in their marking of tasks related to airway management, simultaneous assessment for breathing and pulse, notifying emergency services and integration of the BLS algorithm. This study could be extended by examining which group of assessors provided more lenient or stringent marking in particular sections of BLS as this may point toward differences in marking related to clinical experience or even the lack of familiarity with up to date guidance. This pilot study supplements early work done on the subject of student OSCE assessors.