The influence of distractor strength and response order on MCQ responding

Abstract

Multiple-choice questions (MCQs) play a key role in standardised testing and in-class assessment. Research into the influence of within-item response order on MCQ characteristics has been mixed. While some researchers have shown preferential selection of response options presented earlier in the answer list, others have failed to replicate these results. This paper investigates a possible explanation for these mixed findings by assessing the influence of distractor strength on MCQ response order effects. A real-world assessment was administered to 232 undergraduates in which the response order within items was systematically varied. A generalised multilevel model was then used to show a significant interaction between distractor strength and response position with regard to response behaviour. Furthermore, the effect was found to be independent of student ability. These findings have implications for MCQ test construction, minimising the negative consequences of MCQ test exposure and promoting improved test-taking strategies.

Keywords:

• Examination
• metacognition
• assessment theory
• test