Sure, or unsure? Measuring students’ confidence and the potential impact on patient safety in multiple-choice questions

Abstract

Background: Multiple-choice questions (MCQs) provide useful information about correct and incorrect answers, but they do not offer information about students’ confidence.

Methods: Ninety and another 81 medical students participated each in a curricular neurology multiple-choice exam and indicated their confidence for every single MCQ. Each MCQ had a defined level of potential clinical impact on patient safety (uncritical, risky, harmful). Our first objective was to detect informed (IF), guessed (GU), misinformed (MI), and uninformed (UI) answers. Further, we evaluated whether there were significant differences for confidence at correct and incorrect answers. Then, we explored if clinical impact had a significant influence on students’ confidence.

Results: There were 1818 IF, 635 GU, 71 MI, and 176 UI answers in exam I and 1453 IF, 613 GU, 92 MI, and 191 UI answers in exam II. Students’ confidence was significantly higher for correct than for incorrect answers at both exams (p < 0.001). For exam I, students’ confidence was significantly higher for incorrect harmful than for incorrect risky classified MCQs (p = 0.01). At exam II, students’ confidence was significantly higher for incorrect harmful than for incorrect benign (p < 0.01) and significantly higher for correct benign than for correct harmful categorized MCQs (p = 0.01).

Conclusions: We were pleased to see that there were more informed than guessed, more uninformed than misinformed answers and higher students’ confidence for correct than for incorrect answers. Our expectation that students state higher confidence in correct and harmful and lower confidence in incorrect and harmful MCQs could not be confirmed.

Acknowledgements

The authors would like to thank Dr. Christine Schäfer for her support and advice on working with the ILIAS/Examenedit platform.

Disclosures statement

Rafael Rangel has nothing to declare. Leona Möller reports personal fees from UCB Pharma. A. Strzelczyk reports personal fees and grants from Desitin Arzneimittel, Eisai, LivaNova, Sage Therapeutics, UCB Pharma and Zogenix, outside the submitted work. Helmut Sitter has nothing to declare.

Glossary

• MCQ/MCQs: Multiple-choice question(s).

• ILIAS: German for “Integrated Learning, Information and Work Cooperation System”.

Notes on contributors

Rafael Henrique Rangel, MD, contributed to research project: conception, organization, execution, statistical analysis; manuscript: writing. Rafael Henrique Rangel works at the department of Anaesthesiology, Klinikum Kassel, Germany.

Leona Möller, MD, contributed to research project: conception, organization, execution; manuscript: review and critique. Leona Möller works at the Department of Neurology, Philipps University Marburg, Germany.

Helmut Sitter, PhD, contributed to statistical analysis: design and execution.

Tina Stibane, PhD, contributed to research project: conception, organization; manuscript: review and critique; statistical analysis: design and execution. Tina Stibane is head of the department of Medical Education at Philpps University Marburg, Germany.

Adam Strzelczyk, MD, MHBA, contributed to research project: conception, organization; statistical analysis: design and execution; manuscript: review and critique. Adam Strzelczyk is a senior physician at the Epilepsy Center Frankfurt Rhine-Main, Goethe University, Frankfurt.