A Novel Student-Led Approach to Multiple-Choice Question Generation and Online Database Creation, With Targeted Clinician Input

Abstract

Problem: Multiple-choice questions (MCQs) are the main method of assessing medical student knowledge. As a result there is a high demand from medical students for formative MCQs. However, teaching staff rarely have the time or incentive to develop high-quality formative questions, focusing instead on material for high-stakes assessments. Intervention: We have developed a novel student-led approach involving an interactive online question database, created by medical students for medical students. We adopted a staged approach to create an online bank of formative MCQ questions. First, students write MCQs following a standardized format. Questions are then peer-reviewed by other students, discussing relevant clinical topics, guidelines, and journals to improve question quality. The questions are then scrutinized by specialist doctors and academics. Next, questions are piloted online. Finally, question performance is evaluated statistically. This 5-stage student-led process produced a bank of more than 200 MCQs in three months. Context: This intervention was carried out by two final-year medical student leads at Cardiff University School of Medicine, UK. Final-year students were recruited to write and peer-review questions, and senior content specialists were recruited from the department. After piloting and evaluation of the questions, the question bank was made available as a learning resource to all medical students at Cardiff University. Outcome: Objective analysis of the created MCQs (discrimination indices and distractor analysis) indicated that the random sample of questions piloted were of high quality. When the questions were made available as online tests to approximately 600 students, usage data revealed that 2,800 tests were taken over a 3-month period, indicating that the resource was popular. In addition, subjective feedback from students question writers/reviewers was gathered via free text feedback forms and was invariably positive. We plan to continue the question generation process in Cardiff and would encourage other medical schools to adopt this approach. Lessons Learned: Our 5-stage approach can generate a large volume of high-quality MCQs, addressing the demand from students for formative MCQ questions, with minimal teaching staff input. The project's benefits go beyond the creation of the resource, as involving students in the writing, review, and presentation of questions itself is useful pedagogically.

Keywords:

• multiple-choice questions (MCQs)
• single-best answer
• student peer-review
• online question bank

ACKNOWLEDGMENTS

Benjamin H. L. Harris and Jason L. Walsh are co–first authors.

SUPPLEMENTAL DATA

Supplemental data for this article can be accessed on the publisher's website at http://dx.doi.org/10.1080/10401334.2015.1011651.