How experience affects assessment – a longitudinal study of assessment of university students’ essays

ABSTRACT

The study shows how experience changes the way examiners assess students’ essays. Using a large number of written assessments, 192 examiners were followed over a period of five years, and the contents of their assessments were analysed both quantitatively and qualitatively. Multilevel regression analysis revealed that, with experience, examiners tended to make more positive remarks and provide more instructions. Qualitative analysis of the assessments showed that, with experience, some examiners tended to use less menu-marking and write more freely, providing instructions for the students rather than limiting themselves to evaluative text only. The assumption that it would be harder to govern examiners with more experience found no support in our study.

KEYWORDS:

• Criterion referenced assessment
• autonomy
• experience
• written assessments
• longitudinal study

Acknowledgements

We would like to thank the editor and the two anonymous reviewers of Studies in Higher Education for their thoughtful comments on earlier drafts of this article.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. We use the terms ‘protocols’, ‘assessments’ and ‘examinations’ interchangeably, because each examination corresponds to exactly one protocol which contains exactly one assessment.

2. The analysis was carried out using the lme4 package for the R language for statistical programming (Bates et al. 2015; R Core Team 2015).

Additional information

Funding

The authors acknowledge support from the Committee for Educational Science, Swedish Research Council [2010–4930], Project title: ‘I usually do it my way’ – the relationship between freedom, governance and quality in the assessment of essays with the Teacher Training Education at the University of Gothenburg as an example.