Comparing the Performance of ChatGPT-4 and Medical Students on MCQs at Varied Levels of Bloom’s Taxonomy

References

• Chen X, Xie H, Zou D, Hwang G-J. Application and theory gaps during the rise of artificial intelligence in education. Artl Intel. 2020;1:100002.
   Google Scholar
• Meo SA, Al-Masri AA, Alotaibi M, Meo MZS, Meo MOS. ChatGPT knowledge evaluation in basic and clinical medical sciences: Multiple choice question examination-based performance. Health care. 2023;11(14). doi:10.3390/healthcare11142046
   Google Scholar
• Ignjatović A, Stevanović L. Efficacy and limitations of ChatGPT as a biostatistical problem-solving tool in medical education in Serbia: A descriptive study. J Educ Eval Health Prof. 2023;20:28. doi:10.3352/jeehp.2023.20.28
   PubMed Web of Science ®Google Scholar
• Roos J, Kasapovic A, Jansen T, Kaczmarczyk R. Artificial Intelligence in medical education: Comparative analysis of ChatGPT, Bing, and medical students in Germany. JMIR Med Educ. 2023;9(e46482):e46482. doi:10.2196/46482
   PubMedGoogle Scholar
• Agarwal M, Goswami A, Sharma P. Evaluating ChatGPT-3.5 and Claude-2 in answering and explaining conceptual medical physiology multiple-choice questions. Cureus. 2023;15(9):e46222.
   PubMed Web of Science ®Google Scholar
• Khorshidi H, Mohammadi A, Yousem DM, et al. Application of ChatGPT in multilingual medical education: how does ChatGPT fare in 2023’s Iranian residency entrance examination. Inf Med Unlocked. 2023;41:101314.
   Google Scholar
• Cheung BHH, Lau GKK, Wong GTC, et al. ChatGPT versus human in generating medical graduate exam multiple choice questions-A multinational prospective study (Hong Kong S.A.R. Singapore, Ireland, and the United Kingdom). PLoS One. 2023;18(8):e0290691. doi:10.1371/journal.pone.0290691
   PubMed Web of Science ®Google Scholar
• Anderson, LW Krathwohl, DR. A Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives New York: Longman, 2021.
   Google Scholar
• Brin D, Sorin V, Vaid A, et al. Comparing ChatGPT and GPT-4 performance in USMLE soft skill assessments. Sci Rep. 2023;13(1):16492. doi:10.1038/s41598-023-43436-9
   PubMed Web of Science ®Google Scholar
• Huh S. Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study. J Educ Eval Health Prof. 2023;20:1. doi:10.3352/jeehp.2023.20.1
   PubMed Web of Science ®Google Scholar
• Buabbas AJ, Miskin B, Alnaqi AA, et al. Investigating students’ perceptions towards artificial intelligence in medical education. Healthcare. 2023;11(9):1298. doi:10.3390/healthcare11091298
   Web of Science ®Google Scholar
• Mir MM, Mir GM, Raina NT, et al. Application of artificial intelligence in medical education: Current scenario and future perspectives. J Adv Med Educ Prof. 2023;11(3):133–140. doi:10.30476/JAMP.2023.98655.1803
   PubMedGoogle Scholar
• Li Q, Qin Y. AI in medical education: medical student perception, curriculum recommendations and design suggestions. BMC Medical Education. 2023;23(1):852. doi:10.1186/s12909-023-04700-8
   PubMed Web of Science ®Google Scholar
• Kumar A, George C, Harry Campbell M, et al. Item analysis of multiple choice and extended matching questions in the final MBBS medicine and therapeutics examination. J Med Edu. 2022;21(1):e129450. doi:10.5812/jme-129450
   Google Scholar
• Epstein RM, Cox M, Irby DM. Assessment in medical education. N Engl J Med. 2007;356(4):387–396. doi:10.1056/NEJMra054784
   PubMed Web of Science ®Google Scholar
• van der Vleuten C. Validity of final examinations in undergraduate medical training. BMJ. 2000;321(7270):1217–1219. doi:10.1136/bmj.321.7270.1217
   PubMed Web of Science ®Google Scholar
• Sallam M, Salim N, Barakat M, Al-Tammemi A. ChatGPT applications in medical, dental, pharmacy, and public health education: a descriptive study highlighting the advantages and limitations. Narra J. 2023;3:e103.
   Google Scholar
• Ali R, Tang OY, Connolly ID, et al. Performance of ChatGPT and GPT-4 on neurosurgery written board examinations. Neurosurgery. 2023;93(6):1353–1365. doi:10.1227/neu.0000000000002632
   PubMed Web of Science ®Google Scholar
• Friederichs H, Friederichs WJ, März M. ChatGPT in medical school: how successful is AI in progress testing? Med Educ Online. 2023;28(1):2220920. doi:10.1080/10872981.2023.2220920
   PubMed Web of Science ®Google Scholar
• Lai UH, Wu KS, Hsu TY, Kan JKC. Evaluating the performance of ChatGPT-4 on the United Kingdom medical licensing assessment. Front Med Lausanne. 2023;10:1240915. doi:10.3389/fmed.2023.1240915
   PubMed Web of Science ®Google Scholar
• Kung TH, Cheatham M, Medenilla A, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2(2):e0000198. doi:10.1371/journal.pdig.0000198
   PubMedGoogle Scholar
• Park SH, Do KH, Kim S, Park JH, Lim YS. What should medical students know about artificial intelligence in medicine? J Educ Eval Health Prof. 2019;16:18. doi:10.3352/jeehp.2019.16.18
   PubMed Web of Science ®Google Scholar
• Lw A, Dr K, Pw A, et al. A taxonomy for learning, teaching, and assessing: A revision of bloom’s taxonomy of educational objectives;2001.
   Google Scholar
• Leupen SM, Kephart KL, Hodges LC, Knight J. factors influencing quality of team discussion: discourse analysis in an undergraduate team-based learning biology course. CBE Life Sci Educ. 2020;19(1):ar7. doi:10.1187/cbe.19-06-0112
   PubMed Web of Science ®Google Scholar
• Varma JR, Fernando S, Ting BY, Aamir S, Sivaprakasam R. The global use of artificial intelligence in the undergraduate medical curriculum: a systematic review. Cureus. 2023;15(5):e39701. doi:10.7759/cureus.39701
   PubMed Web of Science ®Google Scholar
• Ibrahim H, Liu F, Asim R, et al. Perception, performance, and detectability of conversational artificial intelligence across 32 university courses. Sci Rep. 2023;13(1):12187. doi:10.1038/s41598-023-38964-3
   PubMed Web of Science ®Google Scholar
• Takagi S, Watari T, Erabi A, Sakaguchi K. Performance of GPT-3.5 and GPT-4 on the Japanese Medical Licensing Examination: comparison Study. JMIR Med Educ. 2023;9(e48002):e48002. doi:10.2196/48002
   PubMedGoogle Scholar
• Johnson D, Goodman R, Patrinely J, et al. assessing the accuracy and reliability of AI-Generated medical responses: An evaluation of the Chat-GPT model. Res Sq. 2023.
   Google Scholar
• Sallam M, Al-Salahat K. Below average ChatGPT performance in medical microbiology exam compared to university students. Front Educ. 2023;8.
   PubMedGoogle Scholar
• Sinha RK, Deb Roy A, Kumar N, Mondal H. Applicability of ChatGPT in assisting to solve higher order problems in pathology. Cureus. 2023;15(2).
   Web of Science ®Google Scholar
• Yanagita Y, Yokokawa D, Uchida S, Tawara J, Ikusaka M. Accuracy of ChatGPT on medical questions in the national medical licensing examination in japan: Evaluation study. JMIR Form Res. 2023;7:e48023.
   PubMedGoogle Scholar
• Newton P, Xiromeriti M. ChatGPT performance on multiple choice question examinations in higher education. A pragmatic scoping review. Assess Eval High Educ. 2024;1–18. doi:10.1080/02602938.2023.2299059
   Web of Science ®Google Scholar
• Herrmann-Werner A, Festl-Wietek T, Holderried F, et al. Assessing ChatGPT’s Mastery of Bloom’s Taxonomy Using Psychosomatic Medicine Exam Questions: Mixed-Methods Study. J Med Internet Res. 2024;26:e52113.
   PubMed Web of Science ®Google Scholar
• Choi W. Assessment of the capacity of ChatGPT as a self-learning tool in medical pharmacology: a study using MCQs. BMC Med Educ. 2023;23(1):864. doi:10.1186/s12909-023-04832-x
   PubMed Web of Science ®Google Scholar
• Sallam M, Al-Salahat K, Al-Ajlouni E. ChatGPT performance in diagnostic clinical microbiology laboratory-oriented case scenarios. Cureus. 2023;15(12).
   Web of Science ®Google Scholar