Do students with well‐aligned perceptions of question difficulty perform better?

Abstract

Effective criterion referenced assessment requires grade descriptors to clarify to students what skills are required to gain higher grades. But do students and staff actually have the same perception of the grading system, and if so, do they perform better than those whose perceptions are less accurately aligned with those of staff? Since students’ learning is influenced by their perception of the skills required to obtain high marks, it is important to learn what students think is being tested. The aims were thus to:

1. compare the student and staff perceptions of the grade descriptors associated with multiple‐choice questions (MCQs) and the factors that may underlie these student perceptions;

2. ascertain if students perform better when their perceptions of the grade descriptors are well‐aligned with staff judgements.

Students studying biochemistry or physics were provided with appropriate online MCQs and were asked to indicate the correct content answers and their perception of the level of difficulty (as indicated by the appropriate grade descriptor) for each question. Detailed feedback on understanding of content and level of difficulty for each question was later provided online. A key finding is that student and staff agreement on perception of question difficulty is only about 50%, for either topic. The differences were significantly correlated with prior knowledge, entry level and exam mark for physics. The implications of these findings to the wider tertiary education community are discussed.

Keywords:

• standards‐referenced assessment
• grade descriptors
• student perceptions
• student performance

Acknowledgements

The authors thank Dr Mary Jane Mahoney from the Faculty of Health Sciences and Ms Lyn Melville from Ferngood Educational Consultancy for their technical expertise in setting up the questions in the WebCT format. The authors also acknowledge and appreciate the support given by the Faculty of Health Sciences in the form of a Learning and Teaching Research Grant.