Improving marking reliability of scientific writing with the Developing Understanding of Assessment for Learning programme

Abstract

The Developing Understanding of Assessment for Learning (DUAL) programme was developed with the dual aims of improving both the quality and consistency of feedback students receive and the students’ ability to use that feedback to improve. DUAL comprises a range of processes (including marking rubrics, sample reports, moderation discussions and peer-review activities) which support explicit knowledge exchange between staff, and between staff and students and the development of tacit knowledge of standards in all participants. This paper describes the study which quantified the extent to which the DUAL programme improved reliability and efficiency of marking of first year biology reports by a large team of laboratory demonstrators. Marker reliability increased after participation in DUAL, with demonstrators marking laboratory reports significantly closer to the marks awarded by an expert marker of the discipline. Efficiency was also improved since time taken to mark the reports did not increase significantly, despite the fact that demonstrators were adapting to a new system of marking. Demonstrators valued the programme and nominated the marking rubric, feedback code and moderation discussion as the most useful elements. The community of practice built through joint participation in the DUAL programme created a supportive working environment for demonstrators and strengthened the collaboration between the discipline and academic language and learning lecturers.

Keywords:

• moderation of assessment
• inter-rater reliability
• marking efficiency
• scientific writing
• feedback

Acknowledgements

Many thanks to our team of laboratory demonstrators: Anna, Cathy, Heather, Natasha, Nicky, Jemima, Pam, Stacey and Thea who enthusiastically embraced the DUAL programme in its first incarnation. Thanks to Tom Angelo for providing valuable advice in the early stages of this project and to members of the First Year Biology Learning & Teaching Group for their support and encouragement. Thanks also to Pauline Ross for providing an example of a rubric developed for biology laboratory reports. Many thanks to Tom Angelo, Tai Paseta, Kate Chanock, Elizabeth Johnson, Meg Rosse and Pam Hurst for providing helpful comments on an earlier draft of this manuscript. This project was funded by a School of Life Sciences Teaching and Learning Grant and La Trobe University’s Curriculum Teaching and Learning Centre, and was approved by the Human Ethics Committee FHEC08/R3.