A randomized controlled trial comparing instructions regarding unsafe response options in a MCQ examination

Abstract

Background: Marking of multiple choice type examinations often just takes account of the correct responses. This may encourage guessing of incorrect and potentially unsafe responses. Ideally responses should contain a high proportion of correct, unsafe and the use of ‘don’t know’ response rather than incorrect.

Aim: This study explored the effect of instructions on responses.

Methods: Fourth- and fifth-year students sitting for an optional multiple choice examination were randomized to receive one of four instruction options: number-correct marking (the control group); alert to unsafe; mark deduction for unsafe; or correction for guessing.

Results: A total of 210 students sat the test. For the fourth-year cohort, compared with the control group, being alerted to unsafe and mark deduction for unsafe or incorrect responses were associated with graduated increases in the use of ‘don’t know’ and reductions in incorrect responses. For the fifth-year cohort, there were no differences in responses between options. The fifth-year cohort, gave more correct, and fewer incorrect, unsafe and ‘don’t know’ responses.

Conclusions: Both the year group and instructions had an effect. Being alerted to potentially unsafe responses, even when there is no mark deduction penalty, had an effect, although mark deduction still had the greatest effect. Assessment instructions may give subliminal messages that have important consequences.

Additional information

Notes on contributors

Mike Tweed

MIKE TWEED is an Associate Dean of Medical Education at University of Otago Wellington. His research interests relate to assessment of medical undergraduates.

Tim Wilkinson

TIM WILKINSON is an Associate Dean of Medical Education at University of Otago Christchurch. His research interests relate to assessment of medical undergraduates.