When precedence sets a bad example for reform: conceptions and reliability of a questionable high stakes assessment practice in Norwegian universities

Abstract

The validity of the Norwegian university grading standard has been called into serious question. The implicit standards used for assessing exams and the reliability of that understanding among examiners and psychology students were investigated in three studies. Studies 1 and 2 investigated the implicit standards that examiners used when assessing exams and the implicit standards that students presume examiners use in that process. Study 3 investigated how reliable expert examiners’ assessments actually are. The validity of the grading standards is challenged by findings that indicate that examiners and students shared similar but not identical conceptions of them, and tests of grader reliability revealed significant variance at critical thresholds. The findings are discussed in terms of their relevance to educational reforms currently going on in Norwegian higher education. The human cost of not completely reforming past practices is highlighted, and six concrete improvements are suggested.

Acknowledgement

The author wishes to thank the anonymous reviewers who provided insightful comments on how to think about the matters of grading criteria and standards. She also wishes to thank the following students for their assistance in collecting the data represented in this paper: Per Matti Aslaksen, Beate Brinchmann, Turid Hafstad, Andreas Harnang, Camilla Kj⊘ndahl, Jane Kj⊘ter⊘e, Bente Stellander and Vanja Stoltenberg all helped with the faculty interview study (Study 1). Erlend K. Lorentzen, Anne‐Lise Myhre, and Kine Aase helped with the student interview study (Study 2). Likewise, Siv Jacklin, Lars Forn and Odin Hjemdal helped lay the foundation for these studies in earlier work that explored how students develop their implicit theories about the grading practices.