Public perceptions of reliability in examination results in England

Abstract

Building on findings from existing qualitative research into public perceptions of reliability in examination results in England, a questionnaire was developed and administered to samples of teachers, students and employers to study their awareness of and opinions about various aspects of reliability quantitatively. Main findings from the study include: there was substantial variability in the understanding of reliability concepts and attitudes to unreliability in examination results among the respondents; the majority of the respondents appeared to understand the assessment process and the factors that affect students’ performances in examinations; to a degree, the respondents also understood the factors that could introduce inconsistency in examination results; the respondents showed various degrees of experience of the examination process and acceptance of measurement error in examination results; the level of tolerance of the respondents for assessment error to some degree was positively correlated to the level of belief about the examinations system, knowledge of aspects of unreliability and approaches to trust. Based on findings from this study and those from other studies, steps that should be taken to communicate with the public about reliability of assessment outcomes have been proposed.

Keywords:

• reliability of results
• measurement error
• human error
• public perceptions of reliability

Acknowledgements

The authors would like to thank Jo Taylor and Laura Daly for their assistance in project management and data collection. Thanks are also due to the two anonymous referees for their constructive comments on early drafts of the paper.