Comparing the Performance of ChatGPT-4 and Medical Students on MCQs at Varied Levels of Bloom’s Taxonomy

Figures & data

Table 1 Overall Performance Metrics of ChatGPT-4

Performance	Frequency	Percentage (%)
Incorrect	80	26.3
Correct	224	73.7
Total	304	100.0

Table 2 Relationship of Program Levels, Courses, Bloom’s Taxonomy Levels and Performance of ChatGPT-4

Background information	Response of ChatGPT		χ^2
	Correct	Incorrect
Program levels
Preclinical	101 (77.7%)	29 (22.3%)	χ^2 (1, n=304) = 1.882, p=0.107
Clinical	123 (70.7%)	51 (29.3%)
Course titles
Cell Biology	41 (93.2%)	3 (6.8%)	χ^2 (5, n=304) = 14.865, p=0.011
Introduction to Molecular Medicine	36 (78.3%)	10 (21.7%)
Respiratory System	24 (60.0%)	16 (40.0%)
Junior Medicine Clerkship	16 (66.7%)	8 (33.3%)
Junior Surgery Clerkship	69 (69.0%)	31 (31.0%)
Aspects of Family Medicine	38 (76.0%)	12 (24.0%)
Bloom’s Taxonomy Levels
Remember	82 (75.2%)	27 (24.8%)	χ^2 (4, n=304) = 2.889, p=0.577
Understand	29 (74.4%)	10 (25.6%)
Apply	29 (65.9%)	15 (34.1%)
Analyse	18 (66.7%)	9 (33.3%)
Evaluate	66 (77.6%)	19 (22.4%)
Create	0	0

Table 3 Association Between Bloom’s Taxonomy and Program Levels for the Correct Answers from ChatGPT-4

Bloom’s Taxonomy Levels	Correct answers		χ^2
	Preclinical	Clinical
Remember	61 (74.4%)	21 (25.6%)	χ^2 (4, n=224) = 91.978, p<0.001
Understand	21 (72.4%)	8 (27.6%)
Apply	11 (37.9%)	18 (62.1%)
Analyse	8 (44.4%)	10 (55.6%)
Evaluate	0	66 (100%)
Create	0	0