On the reliability of high-stakes teacher assessment

Abstract

For a number of reasons, increasing reliance is being placed on teacher assessment in high-stakes contexts in many countries around the world. Simultaneously, countries that have for some time relied to greater or lesser degrees on teacher assessment for high-stakes purposes are in the process of questioning the validity of that reliance. In principle, teacher assessment has an important role to play in increasing assessment validity by complementing testing to cover subject domains more comprehensively than otherwise would be possible. But what is the evidence regarding the reliability of teacher assessment in high-stakes contexts? The answer is that the evidence is limited and often ambiguous. Research has revealed that teachers can be influenced by a number of construct-irrelevant factors as they work towards their judgements, factors such as gender, socio-economic background, effort and behaviour, that risk biasing their assessments. And when considering construct-relevant achievement evidence teachers are often expected to use verbal or semi-verbal sets of criteria, such as level descriptions, which typically require a degree of subjective interpretation in application and so are themselves a source of unwanted variation in judging standards. Arguably the most effective strategy for addressing these issues is participation in consensus moderation. Yet there have been few attempts to provide evidence of the effectiveness of moderation in practice. The potential value of, and the growing reliance upon, teacher assessment in high-stakes applications demand that evaluation of consensus moderation become a built-in part of the process.

Keywords:

• summative
• high-stakes
• teacher assessment
• moderation
• reliability

Acknowledgements

This paper owes much to the constructive feedback on earlier drafts of the two special issue guest editors and the three anonymous reviewers.

Notes

1. The GCSE examinations are taken at the statutory school-leaving age of 16. The GCE comprises the AS and A-level examinations, which are taken during the next two years of schooling. The AS is a lower level, taken on average in four subjects, whilst A is a higher level, taken on average in three subjects which will already have been amongst those studied at AS level.

2. QCDA was the Qualifications and Curriculum Development Authority, successor to the QCA.