Sure, or unsure? Measuring students’ confidence and the potential impact on patient safety in multiple-choice questions

References

• Archer JC. 2010. State of the science in health professional education: effective feedback. Med Educ. 44:101–108.
   PubMed Web of Science ®Google Scholar
• Berner ES, Graber ML. 2008. Overconfidence as a cause of diagnostic error in medicine. Am J Med. 121:S2–S23.
   PubMed Web of Science ®Google Scholar
• Bornstein BH, Emler AC. 2001. Rationality in medical decision making: a review of the literature on doctors' decision-making biases. J Eval Clin Pract. 7:97–107.
   PubMed Web of Science ®Google Scholar
• Bortz J, Schuster C. 2010. Statistik für Human- und Sozialwissenschaftler [Internet]. 7. vollständig überarbeitete und erweiterte Auflage. Berlin (Heidelberg): Springer-Verlag; 660 p. (Springer-Lehrbuch). ISBN: 9783642127694. ger. http://site.ebrary.com/lib/alltitles/docDetail.action?docID=10448295.
   Google Scholar
• Boulet J, van Zanten M. 2014. Ensuring high-quality patient care: the role of accreditation, licensure, specialty certification and revalidation in medicine. Med Educ. 48:75–86.
   PubMed Web of Science ®Google Scholar
• Burton RF. 2002. Misinformation, partial knowledge and guessing in true/false tests. Med Educ. 36:805–811.
   PubMed Web of Science ®Google Scholar
• Butler AC, Karpicke JD, Roediger HL. III. 2008. Correcting a metacognitive error: feedback increases retention of low-confidence correct responses. J Exp Psychol: Learn Mem Cogn. 34:918.
   PubMed Web of Science ®Google Scholar
• Butler AC, Roediger HL. 2008. Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing. Mem Cognit. 36:604–616.
   PubMed Web of Science ®Google Scholar
• Case SM, Swanson DB. 1998. Constructing written test questions for the basic and clinical sciences. Philadelphia (PA): National Board of Medical Examiners.
   Google Scholar
• Chi MTH, Kristensen AK, Roscoe R. 2012. Misunderstanding emergent causal mechanism in natural selection. In: Evolution challenges: integrating research and practice in teaching and learning about evolution. New York (NY): Oxford University Press; p. 145–173.
   Google Scholar
• Chi MTH, Roscoe RD, Slotta JD, Roy M, Chase CC. 2012. Misconceived causal explanations for emergent processes. Cognit Sci. 36:1–61.
   PubMed Web of Science ®Google Scholar
• Chi MTH. 1992. Conceptual change within and across ontological categories: Examples from learning and discovery in science. In: Giere RN, Feigl H, editors. Cognitive models of science: Minnesota studies in the philosophy of science. Minneapolis (MN): University of Minnesota Press; p. 129–186.
   Google Scholar
• Cohen J. 1992. Statistical power analysis. Curr Dir Psychol Sci. 1:98–101.
   Google Scholar
• Cox KR. 1976. How did you guess? Or, what do multiple-choice questions measure? Med J Aust. 1:884–886.
   PubMed Web of Science ®Google Scholar
• Croskerry P, Norman G. 2008. Overconfidence in clinical decision making. Am J Med. 121:S24–S29.
   PubMed Web of Science ®Google Scholar
• Curtis DA, Lind SL, Boscardin CK, Dellinges M. 2013. Does student confidence on multiple-choice question assessments provide useful information? Med Educ. 47:578–584.
   PubMed Web of Science ®Google Scholar
• Curtis DA, Lind SL, Dellinges M, Schroeder K. 2012. Identifying student misconceptions in biomedical course assessments in dental education. J Dent Educ. 76:1183–1194.
   PubMed Web of Science ®Google Scholar
• Dressel PL, Schmid J. 1953. Some modifications of the multiple-choice item. Educ Psychol Meas. 13:574–595.
   Web of Science ®Google Scholar
• Echternacht GJ, Boldt RF, Sellman WS. 1972. Personality influences on confidence test scores. J Educ Meas. 9:235–241.
   Web of Science ®Google Scholar
• Echternacht GJ. 1972. The use of confidence testing in objective tests. Rev Educ Res. 42:217–236.
   Web of Science ®Google Scholar
• Field A. 2009. Discovering statistics using SPSS. London: Sage Publications.
   Google Scholar
• Friedman C, Gatti G, Elstein A, Franz T, Murphy G, Wolf F. 2001. Are clinicians correct when they believe they are correct? Implications for medical decision support. Stud Health Technol Informat. 84:454–458.
   PubMedGoogle Scholar
• Hamdy H. 2006. Blueprinting for the assessment of health care professionals. Clin Teach. 3:175–179.
   Google Scholar
• Hevner K. 1932. A method of correcting for guessing in true-false tests and empirical evidence in support of it. J Soc Psychol. 3:359–362.
   Google Scholar
• Hopkins KD, Hakstian AR, Hopkins BR. 1973. Validity and reliability consequences of confidence weighting. Educ Psychol Meas. 33:135–141.
   Web of Science ®Google Scholar
• Hunt DP. 2003. The concept of knowledge and how to measure it. J Intellect Capital. 4:100–113.
   Google Scholar
• Jacobs. 1971. Correlates of unwarranted confidence in responses to objective test items. J Educ Meas. 8:15–20.
   Web of Science ®Google Scholar
• Kampmeyer D, Matthes J, Herzig S. 2015. Lucky guess or knowledge: a cross-sectional study using the Bland and Altman analysis to compare confidence-based testing of pharmacological knowledge in 3rd and 5th year medical students. Adv Health Sci Educ Theory Pract. 20:431–440.
   PubMed Web of Science ®Google Scholar
• Kolbitsch J, Ebner M, Nagler W, Scerbakov N. 2008. Can confidence assessment enhance traditional multiple-choice testing. Interactive Computer Aided Learning, ICL2008.
   Google Scholar
• Lichtenstein S, Fischhoff B. 1977. Do those who know more also know more about how much they know? Org Behav Hum Perform. 20:159–183.
   Web of Science ®Google Scholar
• Mann D. 1993. The relationship between diagnostic accuracy and confidence in medical students. Paper presented at: Annual Meeting of the American Educational Research Association; Apr 12–16; Atlanta, GA. 14p.
   Google Scholar
• McCoubrie P. 2004. Improving the fairness of multiple-choice questions: a literature review. Med Teach. 26:709–712.
   PubMed Web of Science ®Google Scholar
• Odom AL, Barrow LH. 2007. High school biology students' knowledge and certainty about diffusion and osmosis concepts. School Sci Math. 107:94–101.
   Google Scholar
• Pallant J. 2010. SPSS survival manual: a step by step guide to data analysis using SPSS. Maidenhead (Australia): Open University Press/McGraw-Hill.
   Google Scholar
• Pugsley L, McCrorie P. 2007. Improving medical education: improving patient care. Teach Teach Educ. 23:314–322.
   Web of Science ®Google Scholar
• Razali NM, Wah YB. 2011. Power comparisons of Shapiro–Wilk, Kolmogorov–Smirnov, Lilliefors and Anderson–Darling tests. J Stat Model Anal. 2:21–33.
   Google Scholar
• Salehi M, Sadighi F, Bagheri MS. 2015. Comparing confidence-based and conventional scoring methods: the case of an English grammar class. J Teach Lang Skills. 33:123–152.
   Google Scholar
• Schuwirth LWT, van der Vleuten CPM. 2004. Different written assessment methods: what can be said about their strengths and weaknesses? Med Educ. 38:974–979.
   PubMed Web of Science ®Google Scholar
• Soderquist HO. 1936. A new method of weighting scores in a true–false test. J Educ Res. 30:290–292.
   Google Scholar
• Sözbilir M. 2003. A review of selected literature on students’ misconceptions of heat and temperature. Boğaziçi Univ J Educ. 20:25–41.
   Google Scholar
• Swanwick T. 2014. Understanding medical education: evidence, theory, and practice. 2nd ed. Chichester (West Sussex): Wiley Blackwell; p. xiii, 506.
   Google Scholar
• Tarrant M, Ware J, Mohammed AM. 2009. An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis. BMC Med Educ. 9:1.
   PubMed Web of Science ®Google Scholar
• Taylor AK, Kowalski P. 2004. Naïve psychological science: the prevalence, strength, and sources of misconceptions. Psychol Rec. 54:15.
   Web of Science ®Google Scholar
• Treagust D. 1986. Evaluating students’ misconceptions by means of diagnostic multiple choice items. Res Sci Educ. 16:199–207.
   Google Scholar
• Tweed M, Purdie G, Wilkinson T. 2017. Low performing students have insightfulness when they reflect-in-action. Med Educ. 51:316–323.
   PubMed Web of Science ®Google Scholar
• Wakabayashi T, Guskin K. 2010. The effect of an ”unsure” option on early childhood professionals’ pre-and post-training knowledge assessments. Am J Eval. 31:486–498.
   Web of Science ®Google Scholar
• Wandersee JH, Mintzes JJ, Novak JD. 1994. Research on alternative conceptions in science. Handbook Res Sci Teach Learn. 177:210.
   Google Scholar