The use of distractors in multiple-choice questions: a medical student perspective

Dear editor

We read with considerable interest the study by Rahma et al that aimed to assess the impact of varying the number of distractors in multiple-choice questions (MCQs).¹ It was noticeable that in reducing the number of distractors from four to three, the discriminatory power of the examination increased, while the number of nonfunctional distractors was reduced. In demonstrating this, the group sheds an important light on the need for higher-quality distractors in making assessments adequately discriminatory and reliable. Indeed, as fourth-year medical students who have undertaken many diverse formats of MCQs, we concur that the quality of distractors, as opposed to their quantity, has a greater bearing on the depth of knowledge required to arrive at an answer.

At Imperial College London, medical school examinations include MCQs and Objective Structured Clinical Examinations (OSCEs). Simulated scenarios such as OSCEs are unique in their authentic representation of the clinical setting and their demand of comprehensive medical knowledge.² In our experience, validation from prior written examinations instills confidence in the knowledge base required for OSCEs. Therefore, we appreciate the necessity of written examinations and the importance of studies that aim to advance the quality of such assessments.

However, we believe MCQs fail to simulate the stepwise approach of clinical diagnosis. With the provision of options, the student’s ability to produce a list of differential diagnoses, or appropriate management plan, is not assessed. Crucially, both skills are essential facets of patient management in the clinical setting. A recently described form of assessment at Imperial College London is Very Short Answer questions (VSAs), which eliminate distractors altogether and require the student to provide a short answer. This compels the student to independently produce a list of options from which he/she determines the true answer: a practice more reflective of attaining a diagnosis in the clinical setting.

Sam et al demonstrated that a group of students who were given the same fifteen questions sequentially as VSAs and MCQs scored higher in the latter format.³ This suggests that where MCQs may reward students for a superficial knowledge, VSAs demand a deeper understanding of concepts. Poor-quality distractors in MCQs could allow students to arrive at the answer by a process of elimination as opposed to deduction. Furthermore, students may rely on cues and implausible distractors,³ circumventing the need for thorough understanding. It should also be noted that Rahma et al and Sam et al both recognized that creating four good distractors may be challenging for examiners, especially when core knowledge is being tested.^1,3 Therefore, VSAs may benefit both the examiner and examinee.

We therefore believe VSAs provide a more appropriate mode of assessment, as heralded by their introduction into the fifth-year curriculum at Imperial College London. However, further research is imperative to validate the replacement of MCQs by VSAs in medical school examinations.

Disclosure

The authors report no conflicts of interest in this communication.