A scoping review of artificial intelligence in medical education: BEME Guide No. 84

Abstract

Background

Artificial Intelligence (AI) is rapidly transforming healthcare, and there is a critical need for a nuanced understanding of how AI is reshaping teaching, learning, and educational practice in medical education. This review aimed to map the literature regarding AI applications in medical education, core areas of findings, potential candidates for formal systematic review and gaps for future research.

Methods

This rapid scoping review, conducted over 16 weeks, employed Arksey and O’Malley’s framework and adhered to STORIES and BEME guidelines. A systematic and comprehensive search across PubMed/MEDLINE, EMBASE, and MedEdPublish was conducted without date or language restrictions. Publications included in the review spanned undergraduate, graduate, and continuing medical education, encompassing both original studies and perspective pieces. Data were charted by multiple author pairs and synthesized into various thematic maps and charts, ensuring a broad and detailed representation of the current landscape.

Results

The review synthesized 278 publications, with a majority (68%) from North American and European regions. The studies covered diverse AI applications in medical education, such as AI for admissions, teaching, assessment, and clinical reasoning. The review highlighted AI's varied roles, from augmenting traditional educational methods to introducing innovative practices, and underscores the urgent need for ethical guidelines in AI's application in medical education.

Conclusion

The current literature has been charted. The findings underscore the need for ongoing research to explore uncharted areas and address potential risks associated with AI use in medical education. This work serves as a foundational resource for educators, policymakers, and researchers in navigating AI's evolving role in medical education. A framework to support future high utility reporting is proposed, the FACETS framework.

Background

The use of artificial intelligence (AI) in medicine is expected to have a significant impact on patient care, medical research, and health systems (Weidener and Fischer 2023). The integration of artificial intelligence (AI) and specific tools or techniques to access it, such as natural language processing (NLP), machine learning (ML) and generative pre-trained transformers (GPT), have the potential to transform both medical education content and processes. As technology continues to advance, it is essential to examine the impact of these developments on all aspects of medical education, from admissions to curricula, teaching, and assessment.

Practice points

• What is the extent of evidence on the use of AI (in all forms including NLP, ML and GPT) in medical education and what methods of study have been employed?

• What are the key content themes and areas of focus for these works?

• What, if any, areas are appropriate for more in-depth systematic review?

• What are the gaps in the evidence base currently that should be considered by future researchers? To aid consistent and transparent use of terms throughout this review, Table 1 gives some definitions we have employed throughout.

Whilst the field is not new, the recent application of AI based on machine learning and large language models (LLM), which form the core of AI based chat applications such as ChatGPT by OpenAI has made the technology more widely accessible and easy to use. Masters (2023a) characterised this monumental step as one that could lead to consideration of a pre and post ChatGPT world.

With the rapid expansion of AI, medical educators have become increasingly concerned about the associated risks. A recent piece from a physician defense organization (Graham 2023) warned of the risks of students employing these technologies for their assignments. Other works have cautioned the community concerning AI’s potential to undermine the integrity of academic publishing (Masters 2023b; Loh 2023). Educators have also become more excited by AI, as recent landmark works have highlighted the power of AI to enhance assessment (e.g. Schaye et al. 2022), and make educational processes such as admissions and selection more efficient (e.g. Burk-Rafel et al. 2021).

A review in 2021 by Lee J et al. found a diverse range of 22 published studies on what and how to teach AI within undergraduate medical education (UME), but the wider fields of graduate medical education (GME) and continuing professional development (CPD) were outside of scope. Publications and interest in AI has increased exponentially in the last 12 months in response to the launch of ChatGPT, indicating the need for a new review of the evidence.

This review aims to provide insights into the current state of AI applications and challenges within the full continuum of medical education. We chose a scoping review methodology, mapping out the landscape of existing literature, to delineate the evidence base, classify methodologies, and highlight thematic content areas ripe for systematic investigation. We aim to provide the path forward, guiding AI integration into medical education and ensuring that healthcare professionals are adeptly equipped for the evolving and increasingly complex healthcare milieu.

Methods

This scoping review was conducted in a rapid timeframe – completed within 16 weeks of its inception. Despite the expedited process, we ensured systematicity, and maintained integrity and methodological precision, in line with prior publications (Gordon et al. 2020; Daniel et al. 2021). Our methodology was guided by Arksey and O’Malley’s (2005) scoping review framework, which was refined by Levac et al. (2010). In terms of reporting, we adhered to the STORIES statement (STructured apprOach to the Reporting In healthcare education of Evidence Synthesis) (Gordon and Gibbs 2014) and BEME guidance (Hammick et al. 2010). We also conformed to the PRISMA-ScR reporting standards (Tricco et al. 2018), drawing on support from Peters et al. (2020, 2022) to offset the absence of specialized reporting standards for scoping reviews in health professions education. A protocol is available at the following link (https://clok.uclan.ac.uk/49900/).

The five stages of a scoping review were followed (Arksey and O’Malley 2005):

Stage 1: identifying the research aims/questions (as stated above)

Stage 2: identifying relevant studies

Stage 3: study selection

Stage 4: charting the data

Stage 5: collating, summarizing and reporting the results

The sixth stage recommended by Levac et al. (2010), expert consultation, was not performed, as AI (BK, JC, MS) and educational experts (CM, DD, JC, JH, MD, MG, MH, ST) on the author team fulfilled this role.

Stage 2: Search strategy

In conjunction with an experienced information specialist, we conducted a search of PubMed/MEDLINE, EMBASE, and MedEdPublish without restrictions on date or language. We used a comprehensive search strategy with both controlled vocabulary (e.g. MeSH terms) and keywords. The search terms were based on keywords related to AI, ML, NLP, GPT, and medical education. The search strategy evolved through iterative pilot searches to refine the use of complex terminology. The initial search in July 2023 yielded 1,598 publications. Reference list screening identified key publications not captured, prompting further refinement. The final, more inclusive search was executed in August 2023. The final complete search strategy is detailed in Supplementary Appendix 1.

We reviewed the bibliographies of included studies for any missed articles.

Stage 3: Study selection

The screening process was conducted by pairs of authors (AA, HU, MD, MG, RG) who independently reviewed titles, abstracts, and full texts. Discrepancies were resolved by consensus or with a third author’s input. A Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram was constructed to outline the publication selection steps.

Inclusion criteria

• Publications that described the use of AI, ML, NLP, or GPT in medical education

• Publications that discussed AI, its impact, attitude, knowledge or view, regardless of whether they deployed AI

• Publications in undergraduate medical education (UME), graduate medical education (GME) or continuing professional development

• Publications with medical students, residents, fellows, or physicians. If discussing multi-professional learner groups, medics comprised a significant proportion.

• Publications focused on all relevant areas, including curriculum, teaching and learning, or assessment

• Publications of all methodological forms

• Publications in any language

Exclusion criteria

• Publications that discussed the use of AI for diagnostic, clinical, healthcare organizational systems or governance roles in healthcare

• Publications that employed AI exclusively as a research tool in health professions education

• Publications employing AI to write their work without focusing on AI within their work

Stage 4: Charting the data

The approach taken was amended for each of two clear bodies of literature that were included.

The first category were articles and Innovations. These include scholarly reports of original studies of the proposed or actual employment of AI or related issues in medical education.

The second category were perspectives publications. These include scholarly pieces that present a point of view on AI issues in the field of medical education. They are not original study reports but rather informed opinions that draw on the authors’ expertise, experience, or review of the existing literature.

Articles and innovations

A team of eight authors (AA, BK, CJC, HU, MH, MS, NX, RB), working in pairs, charted the data. They used a predefined data extraction form, which was piloted and refined, with the following data extracted from each study:

• Study identifiers (authors, title, publication date, journal)

• Study characteristics (stage of education, country of origin, medical specialty, study aims, study design)

• AI application or focus of use (e.g. admissions and selection, curriculum, assessment, etc.)

• AI method (e.g. was AI treated as a general concept, or were specific methods such as DM, traditional ML, DL, and/or NLP used?)

• AI use cases

• Name of language model and software programs, if applicable

• Rationale for AI use

• Categorization per the SAMR technology integration framework (substitution, augmentation, modification, redefinition)

• Description of AI implementation

• Summary of results and Kirkpatrick’s outcomes, if applicable

• Implications for future practice, policy or research

Perspectives

A team of four authors (JC, CM, JH, ST), working in two independent pairs, charted not only similar metadata from publications in the Perspective category but also additional details such as the underlying rationale for using AI, its applications, frameworks, and recommended topics for curriculum and research. They further considered ethical issues and the constraints of these applications.

Stage 5: Collating, summarizing and reporting the results

Articles and innovations

Utilizing data from the extraction forms, all authors collated the data into a number of tables and figures for easy visualization, to provide a map of the current evidence base. After charting, we produced a narrative account of our findings that considers the extent and range of developments included in the review, as well as the outcomes assessed. We identified areas where a paucity of research exists. We also suggested areas for future primary and secondary studies (i.e. systematic reviews).

Perspectives

Upon familiarizing itself with the data, the team developed a systematic approach for synthesis. Publications were first organized into two main categories reflecting the central themes of their perspectives: 'Importance of Integrating AI into Medical Education’ and 'Potential Application into Medical Education.'

The sub-team then applied a utility-focused approach to the latter category, highlighting the practical value of each publication for readers, termed ‘Contributions.’ These contributions are pivotal for making informed decisions, designing educational programs, or evaluating AI applications in medical education, despite the fact that most have yet to be studied or evaluated. The Contributions are delineated as follows: 1. Suggested Educational Applications and Developments, 2. Educational Frameworks, 3. Recommendations for Curriculum Content, Competencies, Tools and Resources, and 4. Ethics Considerations.

At each step, team members independently categorized the publications, followed by discussions to reach a consensus. We produced a narrative synopsis of our findings as ‘Brief Summary’ and ‘Key Points’ to aid reader comprehension and application.

Results

Overview

The database search yielded 2,123 publications, with 2,019 remaining after duplicate removal. Title and abstract screening narrowed the pool to 310 publications, with a high inter-rater reliability (κ = 0.86). Full-text review further excluded 70 publications, primarily for using AI solely as a research tool or lacking focus on AI, leaving 241 for inclusion. Hand-searching references added 37 publications, with 278 publications ultimately included in the review. The PRISMA diagram is shown in Figure 1. Full appendix tables with extracted data for all publications have been uploaded to a repository and can be accessed following this link (https://clok.uclan.ac.uk/49919/). An infographic contained in Figure 2 gives much summary detail around the included studies.

Figure 1. Prisma flow diagram.

Figure 2. Infographic summarising results.

The geographic distribution of studies was predominantly Anglo-North American (n = 138, 49.6%), followed by European (n = 51, 18.4%), East Asian (n = 22, 7.9%), Middle Eastern (n = 22, 7.6%), and South Asian (n = 17, 6.1%) contributions. Other regions produced only a handful of studies and no studies were identified from Africa. The earliest publication on AI in medical education was in 1992. The temporal trend showed a surge in publications from 2018 onwards, with 11 in 2018, 14 in 2019, 18 in 2020, 49 in 2021, 57 in 2022, and 114 as of August 2023 (see Figure 2a,b and Supplementary Appendices 1 (columns E and I) and 2).

Most included publications fall under the category of articles or innovations (n = 191, 68.7%), as defined above, with the remainder being perspectives (n = 87, 31.3%). Undergraduate medical education (UME) was the focus of 48.6% (n = 135), graduate medical education (GME) 22.3% (n = 62), and continuing professional development (CPD) 2.5% (n = 7). The remaining publications spanned multiple levels (UME, GME, or CPD) (n = 74, 26.6%) (see Figure 2c,d and Supplementary Appendix 2 (column H) and 2).

Six basic science disciplines and 24 clinical specialties were represented in the included publications. Anatomy and Physiology were the most common basic science disciplines (n = 4, 1.4%, each), whereas radiology (n = 31, 11.2%) and surgery (n = 24, 8.7%) were the most common clinical specialties, followed by ophthalmology (n = 8, 2.9%) and neurosurgery (n = 6, 2.2%). One hundred fifty-seven studies (56.5%) did not specify a discipline or specialty (see Figure 2e and Supplementary Appendices 2 (column J) and 2).

Articles and innovations included in the review

This review encompasses a wide range of articles and innovations, systematically cataloged as A1–A191 in Supplementary Table 2, facilitating cross-reference throughout this review. Of these studies, seven (3.6%) used AI for admissions and selection (A1–A7), 34 (17.8%) focused on teaching about AI or AI-augmented instruction (A8–A41), 50 (26.2%) used AI in assessment (A42–A91), thirteen (6.8%) used AI to teach or assess clinical reasoning (A92–A104), and four (2.1%) used AI for automation of case and procedure logs (A105–A108). A substantial portion (n = 51, 26.7%) investigated knowledge or attitudes about AI in medicine (A109–A159), and the performance of large language models (LLMs) on medical questions or exams was evaluated in 32 (16.8%) of the studies (A160–A191. (see Figure 2f and Supplementary Appendix 2, column M).

The technical application of AI varied. In 65 (34%) of the studies, AI was treated as a general concept without actual computation. The remainder of studies engaged in various computations: data mining in 4 (2.1%), traditional ML in 41 (21.5%) and DL in 81 (42.4%). NLP was used in 78 articles (40.8%). (See Figure 2g,h, and Supplementary Appendix 2, columns N and O. Also refer to Table 1 for AI terminology.)

Table 1. AI terminology.

Artificial Intelligence (AI)	AI is a concept, the most general of the terms, that spans any process that involves a machine acting ‘intelligent.’ Intelligence is most often defined as ‘human-like’ in its ability to make decisions, learn from mistakes, generate insights, or understand language (Coppin 2004).
Machine Learning (ML)	ML is a method and discipline. ML involves the specific mathematical and computational structures which produce computer programs/algorithms that can make decisions given input data. ML is most frequently the way in which we achieve (a semblance of) AI (Gopinath and Churiwala 2019).
Traditional Machine Learning	Traditional ML involves extensive human-engineered preprocessing of input data to extract salient ‘features’ which are then piped into a mathematical model/equation to produce decisions/results. Examples: logistic regression, support vector machines, decision trees, K-nearest neighbors, gradient boosted machines (Gopinath and Churiwala 2019).
Data Mining (DM)	DM often borrows basic statistics or algorithms from traditional ML (for example, logistic regression or k-nearest neighbors), but instead of trying to teach a machine to make a decision, it instead attempts to sort through large volumes of data to allow humans to make insights and better understand the data itself. Whereas ML is concerned with assessing the ability of machines to make prospective decisions on heretofore-unseen data, DM instead tries to glean insights from full, known sets of previously collected data (Islam et al. 2018).
Deep Learning (DL)	Deep ML involves relatively little human-engineered preprocessing of input data, which is often input in raw or near-raw form. In DL, the model itself creates its own internal representations of features, as represented by the connection strength between mathematical ‘neurons’ within the artificial neural network. Deep neural networks require significant time/ energy/ money/ data investments to initially train. However, these ‘pre-trained’ models can be ‘customized’ to new domains, such as MedEd, cheaply (Naylor 2018).
Natural Language Processing (NLP)	NLP is any type of computational or mathematical approach that deals with natural human (written or spoken) language. NLP is almost always paired with one of the approaches above, which is often why you see it written with a slash (e.g. NLP/ML). ChatGPT, for example, is NLP/ML (specifically deep learning), because it is a deep machine-learning artificial neural network which processes natural language (Iroju and Olaleke 2015).

Employing the SAMR framework to evaluate four levels of technology integration, this review found that AI primarily served as a substitute in 36 articles (18.8%), enhanced existing processes through augmentation in 34 (17.8%), contributed to task redesign or modification in 10 (5.2%), and facilitated an entirely novel, previously unconceivable task or redefinition in 11 (5.8%). For a substantial number of articles (100, 52.4%) the SAMR framework did not apply (see Figure 2i and Supplementary Appendix 2).

The review also identified a spectrum of AI applications, described as ‘use cases,’ which included data analytics (n = 9), predictive models (n = 17), performance analytics (n = 19), sentiment analysis or bias evaluation (n = 7), virtual patient simulators (VPSs) (n = 11), personalized learning platforms or intelligent tutoring systems (n = 6), clinical guidance for trainees (n = 10), procedural guidance for trainees (n = 5), ChatBots (n = 2), data labeling (n = 2), content generation (n = 5), and summative assessment completion (n = 32) (see Figure 2j and Supplementary Appendix 2, column P).

AI use in admissions and selection for medical school and residency

In one study (A7), an AI Chatbot was used during recruitment to host a virtual question and answer session. In another study (A5), sentiment analysis was used to detect gender bias in letters of reference for a surgery residency over three decades. This study demonstrated that while gender bias has decreased over time, letters continue to be positively biased towards men, impacting selection. Another study (A3) compared faculty scoring of medical student performance evaluations (MSPEs) to a machine learning (ML) model’s scores. The ML model was unable to successfully rank MSPEs using commercially available sentiment analysis programs, largely because all MSPEs were highly positive. A bespoke model may be developed in the future that can better replicate faculty scores.

In one study (A4), ML was used to develop a predictive model to identify candidates likely to be ranked and matched. The predictive accuracy for ranking was outstanding, with an area under the receiver operating characteristic (AUROC) of 0.925. In three studies (2 in UME (A2, A6) and 1 in GME (A1)), ML was used to screen applications. The ML models were trained on retrospective data sets, then validated on retrospective or prospective data sets. The models demonstrated impressive accuracy with AUCs of 0.925–0.95, and hold promise in the future for partially automating holistic review of large numbers of applications. When applied in a restorative fashion that augments human review, ML models can be used to mitigate bias and identify applicants who may otherwise have been screened out and not offered an interview (A1). With further validation and refinement, ML models have the potential to make application screening more efficient and equitable, and the ranking process more accurate.

Teaching about AI and AI-augmented instruction

A total of thirty–three articles addressed teaching about or instruction with AI (A8–41). Of these, 12 focused on AI as a learning tool, 11 on curriculum development, three on AI use for personalized education, five on the use of AI in medical simulation, and two on AI educational competencies.

The majority of articles (7 out of 12) focused on AI as a learning tool, training users to identify findings on images, including histopathologic identification of glomerulopathies (A22), neuroanatomic structures (A35), general tissue types (A31), and melanomas (A25). A few articles emphasized identification of clinical pathology (e.g. diabetic retinopathy (A26, A30) and hip fractures on plain radiographs (A24)). One article described AI–powered, real–time instruction on acquiring optimal echocardiographic views with point–of–care ultrasound (A32) and another trained students to develop AI algorithms for sonographic identification of pulmonary structures (A33). Another set of articles used the interactive features of chatbot-style tools for anatomy instruction (A29), delivering difficult news (A34), and LLM-based summaries of surgical procedures (A28).

Of the eleven articles that addressed AI curriculum development, a majority (n = 6) described the development of instructional modules. Four emphasized fundamental concepts in AI and ML for UME learners (A11, A13, A16, and A19). The GME/CPD studies largely focused on radiologists and included AI workshops (A14, A17), a deep-learning curriculum (A18), a data-science elective (A20) and a medical DM course (A21). One article described an introductory AI course for practicing physicians (A12).

Three articles explored ML for personalized learning, including an NLP algorithm to support trainee evaluation of geriatric patients through just-in-time learning (A36), a ML-powered adaptive learning system for preclinical students (A37), and a DL model for patient triage in ophthalmology (A38).

Five articles examined AI in medical simulation, covering topics like the interview of standardized patients about the use of robotic standardized patients (A39), an AI-assessed cognitive load in simulation through ECG and galvanic skin response (A40), a DL model for simulation outcome evaluation of recorded simulation sessions (A41), a ML-based multimedia simulation of an individual nephron (A23), and generative AI for creating images of dermatologic lesions (A27).

Lastly, two articles delved into educational competencies in AI, one developing a tool for assessing medical student readiness for AI use in clinical settings (A9), and the other examining the prevalence of AI competencies in GME and CPD settings in Germany (A10).

AI use in assessment

Of the 50 articles related to AI use in assessment, a focus on psychomotor skills was prevalent, with 21 articles examining this area: 18 on surgical skills and 3 on procedural skills, with the latter group focusing exclusively on intubation techniques (A71–73). Surgical skills assessments predominantly focused on minimally invasive and robotic surgery within general surgery (A74–A83, A85, A87–A88, A91), with a handful addressing neurosurgical (A84, A86, A89) and pediatric surgery (A90). Fifteen articles focused on narrative text analysis, 10 on clinical skills, 3 on generation of multiple choice questions (MCQs), and 1 on predictive analytics.

A notable trend involved leveraging ML as an alternative to human observation and feedback (A74, A76–A78, A83, A86, A88, A90). Many utilized virtual trainers or simulators to facilitate assessment in simulated environments. A particularly innovative approach involved cloud-based, AI-powered video analytics to monitor and assess users’ performance in minimally invasive surgery (A81). A few studies described the development of algorithms capable of discerning expertise levels or predicting surgical skill (A74–A76, A85, A86) or learning curves (A80, A84, A87, A91) based on visual or kinesthetic inputs. One study developed a ML model, integrating EEG data analysis of participants, to distinguish expert from novice surgeons (A89). These studies underscored the potential of AI to mitigate the resource intensity and inherent bias of traditional assessments.

Narrative text assessment is another area where AI is gaining ground, with sentiment analysis and NLP being common tools for extracting insights from textual data. This application of AI promises an efficient and automated analysis of extensive narrative information for enriching and contextualizing quantitative assessments such as ratings. Faculty’s narrative evaluations of residents (A55, A57, A62-A64, A67-A68, A69) and clerkship students (A55, A56, A65) were areas where AI was deployed to inform clinical competency committees (A55), gain insights into competency development (A57, A68-A69), generate quality ratings of faculty feedback (A64, A67), facilitate objective interpretation of faculty evaluations (A65), identify learners at risk (A68), and identify biases within written student evaluations (A56, A66). NLP was further used to parse through reflective writings of students (A59), assess professionalism in short answer responses (A61), identify patterns suggestive of professionalism lapses in evaluative feedback among faculty (A60), and evaluate physician performance as it related to professionalism and patient safety (A58).

Among studies of clinical skills assessment, seven out of ten focused on UME applications. Here, VPSs were a mainstay, reducing logistical demands (e.g. time and resource) of traditional assessments, and enabling remote examinations (A42, A48–A49, A51). AI’s role extended to virtual Objective Structured Clinical Examinations (OSCEs) (A42, A49) and automated grading of OSCEs (A43–A44, A50) or both (A46, A48, A51). In GME, there were studies describing the use of a virtual provider simulator for resident training in communication skills, with automated assessment and feedback (A47) and NLP for assessment of critical care trainees’ oral case presentations (A45).

Three studies delved into AI-generated MCQs, harnessing DL and NLP to create assessment tools and study aids (A52–A54). These studies emphasized AI’s capability to streamline question development, substituting intense human efforts, though still requiring human verification to ensure quality. The benefits of AI included speed, and ChatGPT emerged as the preferred tool. Study results were mixed, however, with some skepticism of AI’s readiness to fully take over the MCQ creation process (A52–54).

The final point to note is that predictive analytics played a notable role in evaluating surgical skills across multiple studies. However, only one study (A70) primarily focused on this aspect. This particular study investigated the prediction of medical student performance on high-stakes exams through ML. This illustrates a nascent but promising application of AI in predicting academic success in high-stakes environments.

AI to teach and assess clinical reasoning

Thirteen articles examining AI‘s role in teaching and assessing clinical reasoning reveal a promising frontier. Eight studies focused on the fusion of AI-powered VPSs and intelligent tutoring systems (ITSs) (A92, A95–A98, A101–A102, A104). VPSs simulate authentic patient interactions, where learners ask questions and receive answers from an avatar or via text, offering dynamic environments for learners to hone their reasoning through varied clinical scenarios. This allows learners opportunities to practice various components of the clinical reasoning process, from information gathering and hypothesis generation, to problem representation and diagnostic justification.

The ITSs complemented this by delivering personalized learning and real-time feedback, guiding learning through correct and incorrect reasoning responses (A92, A95–A97, A101–A102), and suggesting further study resources(A 95). When combined with analytics (A92, A96, A101, A102), these systems could yield nuanced insights for educators and learners on performance, identifying specific areas for improvement. Six studies provided Kirkpatrick level 1 (A92, A98, A101) and/or 2b evidence supporting the use of these systems (A92, A95-A96, A101–A102).

The creation of VPSs and ITSs cases, a labor-intensive process, typically involved content experts. However, 2 studies (A101, A 104) demonstrated innovative use of NLP to automate the creation of extensive case libraries (413 and 2525 cases, respectively) from actual hospital records and the New England Journal of Medicine Clinical Case Competition series. Such expansive libraries enhanced the variety of cases available to learners, addressing common challenges in teaching and assessing clinical reasoning, like case variability and the unpredictable nature of clinical encounters in real-world settings.

Further exploration into AI's utility is found in three studies that apply ML to analyze clinical reasoning in written notes (A93–A94, A99). These studies assessed different forms of clinical documentation, including post-encounter OCSE notes (A93) and diagnostic justification essays (A94) by medical students, as well as residents’ admission notes from electronic health records using a standardized rubric (IDEA tool) for clinical notes (A99). The psychometric properties and inter-rater reliability of the ML algorithms were reported as acceptable. One study achieved a kappa statistic of 0.8, meeting the reliability standard for high-stakes assessment (A93), whereas others met the standard for lower-stakes assessments. The collective findings of these studies substantiate the concept of utilizing ML to streamline assessment of clinical reasoning, thereby enhancing efficiency of scoring and significantly enriching formative feedback through assessment of clinical notes–a largely underutilized source in clinical reasoning assessment.

The final two studies in this area highlighted AI's expanding role. The first study demonstrated the use of neural networks to recognize patterns of successful student problem-solving (A100), further highlighting AIs potential in clinical reasoning assessment. The second study evaluated the effectiveness of three different LLMs at providing clinical decision support for junior doctors (A103). The authors noted that more advanced LLMs trained on medical databases, rigorously scrutinized by medical experts, could result in valuable educational tools in the future.

AI use for automation of case and procedure logs

Four of the publications detailed efforts to utilize AI/ML to automate documentation of trainee clinical and procedural experiences. In one (A107), an NLP algorithm–based system identified clinical concepts in trainee clinical notes and mapped these onto expert–defined core clinical problems, with a precision of 92.3%. A similar system based on a commercial NLP product tracked case experiences for neurology residents, with a tripling of case experiences logged in a one–month timeframe (A105). A third system (termed ‘Trove’) (A106) used an NLP algorithm to tally classes of diagnoses based on resident–created imaging impressions and to populate a dashboard for residents with accuracies ranging from 93.2 to 97.0%. Finally, a system utilizing a deep neural network with a reinforcement learning model provided suggestions for data residents needed to manually log their surgical case experiences based on EHR data and surgical case schedules (A108).

Knowledge of AI

Fifty-one of the included articles (26.7%) explored knowledge, perceptions, attitudes, hopes, and concerns related to AI in medical education.

Three studies carried out objective assessments of AI knowledge. In Spain, a study of 281 medical students used true/false questions to assess general AI knowledge, with a 65% correct response rate (A117). In Saudi Arabia, another study of 476 medical students assessed knowledge of DL, and found that 54% failed to correctly answer any of five true/false questions (A118). Across Australia, New Zealand and the United States, a third study of 245 medical students evaluated ML knowledge through MCQs, resulting in an average score of 49.7% (A120).

Self-reported knowledge of AI varied. In Western Australia, 85% of medical students claimed basic understanding of AI (A151), and 91% of pediatric ophthalmologists reported familiarity with basic AI concepts (A154). In contrast, only 31% of German medical students reported basic knowledge of AI technologies (A145), and focus group discussion showed pediatric radiology trainees at a large academic hospital struggled with AI terminology and literature appraisal (A156). A South Korea survey indicated that medical students’ primary information sources were news publications and TV (82%), social media (41%), lectures (32%), or friends or family (29%), with academic publications and books at only 22% (A127). Similarly, in a US study, 72% of medical students and 59% of faculty learned about AI from media sources (A159).

AI-related course availability was inconsistent. In Germany, 71.8% of medical schools offered AI courses, mostly as electives or extra-curricular activities (A142). However, in a sample of Canadian medical students, 85% reported no formal AI education as part of the medical degree curriculum (A146).

Attitudes towards the application of AI in clinical medicine

The studies reviewed revealed mixed attitudes toward AI in clinical medicine. Most medical students and doctors acknowledged AI's potential benefits, such as enhancing clinical judgment, research, and auditing skills, and streamlining administrative processes (A111, A117, A141). However, concerns included the ethical implications, weakened physician-patient relationships, and AI’s limitations in unexpected scenarios (A114, A119, A154, A140).

Two studies investigated medical students’ intention to use AI. One used the Unified Theory of User Acceptance of Technology (UTAUT) to examine the intention to use AI for diagnosis support, finding social influence as the only key factor (A153). In the other study, intention to use AI was only observed when students had a strong belief in the role of AI in the future of the medical profession (A157).

Attitudes toward the application of AI in medical education

Two studies assessed attitudes on the use of AI in medical education. One study assessed Lebanese medical students’ views about using AI for assessment (A131). While 58% believed AI would be more objective than a human examiner, only 26.5% preferred AI assessments. Most (71%) preferred a combination of AI and humans. Another study in a Caribbean medical school found that 33% of faculty used ChatGPT, mainly for generating MCQs. The main concerns about its use were the potential for incorporation of misinformation into teaching materials and plagiarism.

Concerns and hopes for AI

Concerns about AI included its potential to reduce the demand for doctors and influence career choices, especially in radiology (A113, A115, A124, A135, A137,A147, A150). Insufficient AI training in medical curricula was another worry (A128, A143, A151). Three studies assessed aspects of AI which should be included in medical curricula, including clinical applications, algorithm development, algorithm appraisal, coding, and basics of AI, statistics, ethics, and privacy (A133, A156, A158).

Performance of LLMs on medical exams

Thirty-two studies assessed various LLMs’ abilities to perform on medical exams or on test questions drawn from large question banks (A160-A191). The LLMs tested included Microsoft’s Bing Chat, ChatGPT, GPT-3, GPT-3.5, and GPT-4, Google’s Bard, and several other pre-trained language models. They were tested on multiple choice (A160-A185, A188-A190), short answer and essay questions (A186-A187, A191), though questions with images and diagrams were typically excluded due to LLMs’ current limitations processing visuals. A few studies asked the LLM to justify its answers (A164, A168, A175-A176), which were then assessed for accuracy. LLM performance was compared to varying levels of human performance (i.e. novice to expert), as well as ‘gold standard’ answers (A160-A191). LLMs scored at or above passing thresholds on a number of local, university-derived, basic and clinical science exams (A178, A183, A184), as well as on national licensure exams (A160, A169, A180, A182, A187, A189) and specialty licensure exams (A169, A175, A181). Despite these successes, LLMs performed only modestly or poorly on other exams (A168, A170, A173-A174, A176, A190). Newer generation LLMs generally outperformed older generation models, though this was not universally true. LLMs typically performed better on general medical exams, compared to specialty or subspecialty exams, though this was also not universal. LLM performance was tested on exams from Australia (A177), Canada (A162, A174), China (A173, A190), France (A160, A170, A181), Germany (A165, A182), India (A160, A163, A183-A184, A187), Italy (A160), Japan (A189), Korea (A171, A180), Netherlands (A179), Saudi Arabia (A178), Spain (A160), Switzerland (A185), Taiwan (A173), Turkey (A166), the UK (A160, A167, A191), and the US (A160-A161, A164, A168-A169, A172-A173, A175-A176, A186, A188). In most cases, LLMs performed better on English language questions, or when questions in other languages were first translated into English. Their performance in other languages was highly variable (e.g. 22% accuracy in French vs. 73% in Italian (A160, A170). At this time, learners and educators should approach use of LLMs with caution due to variable accuracy (A188), though the notable improvements of newer LLMs may soon overcome this barrier. As capabilities of LLMs improve, educators must contend with concerns about the integrity of formal assessments, as well as their reliability and validity. Assessments may require redesign, and a paradigm shift towards open book tests that emphasize learners’ abilities to integrate information, rather than recall facts, may be warranted (A186).

Overview of perspectives publications included in the review

Eighty-seven perspective publications met inclusion criteria for this review, labeled P1–P87 in Supplementary Table 3. These perspectives, encompassing commentaries, editorials, and letters to the editor, offered insights on the use of AI in medical education, presenting opinions, recommendations, or examples of potential use cases of AI. We categorized these perspectives into five groups according to their primary content—general arguments for the importance of incorporating AI in medical education (n = 42); suggested educational applications or developments (n = 18); frameworks for curricular development (n = 8); recommendations for curricular content, AI competencies, or resources to support use of AI (n = 15); and ethics considerations (n = 4). Some publications had secondary content in an additional category (see Supplementary Table 3 and Supplementary Appendix 2 for additional details).

Importance of integrating AI into medical education

Many of the perspectives argued that incorporating AI in medical education is essential to prepare future physicians to apply AI to patient care (P50, P52–53, P60, P65–67, P70, P79). AI is already influencing diagnosis and treatment decisions in health care, and medical students and residents need to be ready to use AI tools effectively in the health care system. Additionally, several perspectives advocated for the use of AI to enhance teaching and learning (P55–56, P58–59, P61, P64, P69, P71, P84–86) or assessment (P48, P81). Two perspectives urged medical educators to incorporate teaching about AI’s influence and ethical and legal considerations (P57, P87). Some of the perspectives raise cautions about AI’s potential to compromise or limit clinical learning (P46, P49, P54, P63, P72–73, P82). The remaining perspectives contained general statements about the potential use of AI in medical education (P47, P49, P51, P54, P62, P68, P74–78, P80, P82–83).

Specific suggestions for educational applications and ideas for development

The 18 perspectives with specific educational applications included the following uses of AI tools in medical education:

• Intelligent Tutoring Systems – AI-powered platforms providing personalized learning experiences to improve the teaching of decision-making skills (P5)

• AI-assisted learner assessment – Latent Semantic Analysis to grade students’ clinical case summaries and provide feedback (P11); NLP to score clinical skills exams (P15)

• ChatBot (e.g. ChatGPT) – AI processing and understanding of medical literature to teach clinical management (P10), assist with USMLE (P12) and other exam (P17) preparation, promote critical thinking, creativity, and patient communication (P8, P14, P16)

• Personalized Learning Platforms – AI-driven platforms creating personalized learning paths for students (P1) and providing personalized feedback (P8)

• Robot-Assisted Surgery Simulations (e.g. Virtual Reality) – AI-powered simulators for surgical training and evaluation (P13)

• Enhanced anatomy education – enhancing teaching, learning, and assessment in anatomy education for deeper learning and long-term retention (P2)

• AI tools to prepare applications – medical school applications (with a call for guidelines to ensure fairness in admissions) (P6) and personal statements for residency applications (P18)

• AI-generated art – to enhance visual storytelling for patient encounters (P7)

• Machine learning and intra-operative video analysis to improve patient care – teaching competency-based patient assessments (P9)

Frameworks for AI in medical education

Several perspectives provided frameworks to guide the incorporation of AI into medical education. An AI literacy framework for medical school described aims, opportunities, and impact of AI for selection into medical school and training programs, learning, assessment, and research (P19). A framework for radiology education described the role of AI to promote personalized learning and decision support tools (P20). A framework that emphasized digital skills provided a structure for teaching data protection, legal aspects of digital tools, digital communication, data literacy and analytics (P21). A framework for integrating AI in UME used a case-based approach with a detailed delineation of what AI content could be taught through cases (P22). A framework for early and progressive exposure to AI for medical students provided an analytics hierarchy to scaffold learning to prepare students to adapt to evolving clinical settings (P23). A framework for utilizing the rapidly expanding medical knowledge base also incorporated content about how AI relates to health care economics, regulations, and patient care (P24). An embedded AI ethics education framework provided a curricular structure for teaching the ethical dimensions of medical AI in the context of existing bioethics curricula. (P25). Finally, a curriculum framework incorporated ML to build clinicians’ skills in research, applied statistics, and clinical care (P26).

Recommendations for AI curriculum content or competency, tools and resources

Fifteen perspectives offered explicit recommendations for curricular content about AI, AI competencies, and specific tools and resources that can support curriculum development for teaching about AI in medical education. Recommended content included what physicians will need to know to meet patients’ needs as AI changes the practice of medicine (P31–32), data science and concepts underlying AI (P33, P35), using AI to improve surgical outcomes (P41), surgical training for ophthalmologists (P40), AI model performance, financial implications, and generation of reports in radiology (P38–39, P37), data science to support evidence-based medicine (P36), using AI for diagnostics and prognosis in hematology (P28), concepts underlying machine learning (P34), and ethical uses of AI in medicine (P27). One publication proposed a curriculum for a new specialty, Doctor in Medical Data Sciences (P29). One publication explained six domains of competency for effective use of AI tools in primary care, in the context of the Quintuple Aim (P30).

Ethics considerations for AI education

Fourteen perspectives addressed ethics considerations and reflected a multifaceted view of the ethical landscape AI introduces to healthcare. Many of these publications expressed caution about the limitations of AI applications in medical education. These perspectives also stressed the importance of educational approaches that equip healthcare professionals with the necessary skills to navigate AI challenges, emphasizing the irreplaceable value of human judgment and ethical reasoning in the age of AI.

Recommended focus for ethics education

Many of the perspectives papers raised concerns about the ethics of using AI in healthcare and medical education, and discussed the responsibility of physicians to grapple with the ethical challenges that can compromise education and health care. These papers emphasize the importance of preparing current and future physicians to grapple with these issues, while acknowledging that we have neither fully grasped the ethical implications of AI nor learned how to adequately respond to the ethical challenges. When considered collectively, these papers contribute important insights about the topics that must be addressed in medical education. Referring to these papers can provide clear guidance to medical educators about ethical issues to include in curricula. The perspectives that addressed ethics related to AI recommended ethics education on the following topics, which are summarized in Figure 3:

Figure 3. Ethical considerations for AI in medical education.

• Algorithmic Bias and Equity: Future clinicians must understand biases in health AI, especially those caused by training on non-representative data sets, which might exacerbate healthcare disparities (P42). Through an AI curriculum, physicians can reflect critically on their assumptions and the biases in both AI and clinical practice. This could help them better interpret AI outputs and recognize the limitations of current medical knowledge (P22)​​.

• Resource Allocation: AI in healthcare brings to the fore issues of just resource allocation and equity, challenging students to consider how AI can be designed and used to promote justice in healthcare (P42).

• Safety and Quality Assurance: AI necessitates agreement among healthcare professionals to use AI tools responsibly and in alignment with professional obligations, such as practicing medicine safely and effectively (P25)​​. With AI's integration into clinical practice, students must understand regulatory efforts, gaps in AI safety, and the ethical implications of using such technologies (P42).

• Human Interaction and Compassionate Care: AI-based ethics training might be less likely to instill compassion in students compared to traditional methods. One publication argued that compassion can indeed be taught and that positive role models in medical education can deepen students’ understanding of compassionate practice (P44)​​.

• Data Privacy and Security: The use of AI in medicine necessitates a strong understanding of data privacy, the ethical handling of patient data, and the legal dimensions associated with AI (P42, P43). The large number of available data sets and increased power of AI capabilities make data de-anonymization (or reidentification) highly possible through cross-referencing with other data sets (P43).

• Automation Bias and Skill Preservation: Clinicians must be wary of automation bias, where undue reliance on AI could lead to errors, highlighting the need for balanced trust and skepticism (P25). Ethics education must ensure AI complements, not replaces, the clinician’s role in compassionate care (P25). Training should focus on preserving clinical skills and human interactions that AI cannot replicate (P42), leveraging ‘hybrid intelligence’ systems to strengthen therapeutic patient-provider relationship and empathy (P44).

• Transparency and Informed Consent: The deployment of health AI raises critical questions about informed consent and the physician’s duty to inform patients about AI's involvement in their care decisions, highlighting the need for medical students to engage in these discussions (P42). Proficiency in medical AI is intertwined with ethical competency, where ethical considerations significantly influence clinical decisions and vice versa. As future practitioners, students need to learn to identify AI’s potential consequences and how to discuss them with patients in light of ethical principles (P25). In medical education, students must be made aware of data collection, provide consent, and understand the extent and implications of their consent (P43).

Proposed educational approaches for ethics education

One recommended approach to teaching ethics about AI is the incorporation of real-life case studies in medical curricula to elucidate the multifaceted ethical issues presented by health AI, such as informed consent, bias, and transparency (P42). This case-based methodology not only enriches the theoretical understanding but also hones practical decision-making skills. An Embedded AI Ethics Education Framework, which is an incremental approach that integrates AI ethics within existing medical ethics courses, addresses the potential harms of technology misuse and equips students to conduct risk-benefit analyses fundamental to patient care (P25). A third approach uses AI scenarios currently employed in clinical practice as examples for case-based learning to prepare students to critically evaluate the technical aspects of AI alongside ethical implications (P22). This approach fosters critical reflection and encourages interdisciplinary dialogue essential for the application of AI in patient-centered care. Another approach emphasizes the limitations of AI through human-led ethics teaching, much like ethics teaching that has become commonplace in medical school curricula. This approach maintains an emphasis on human interaction and the cultivation of moral reasoning over reliance on technology, while remaining open to the potential for ‘hybrid’ intelligence systems, in which humans and machines learn from each other, resulting in superior performance in ethical decision-making. Collectively, these educational strategies underscore the necessity of equipping healthcare professionals with both technical skill and ethical discernment to navigate the complexities introduced by AI in healthcare and health professions education.

Discussion

This review embarked on a journey to map the explosion of interest in AI across the medical education landscape through the lens of published works. The detailed tables of articles and perspectives papers, summary infographics and associated narrative clarify the current state of the evidence. The first published study on AI in medical education dated back three decades to 1992, though AI has existed much longer than this. A steady flow of publications has been seen since, a testament to the enduring fascination and steady evolution of AI in medical education. The recent surge in publications may at first appear to be a growth spurt associated with maturation in the field, though our analysis suggests it is more likely a contemporaneous response to the public release of ChatGPT.

The recent advancements in AI technology have led to rapid, widespread and in many ways limitless distribution of accessible, cost-effective AI tools globally. This technological democratization is mirrored in the diverse origin of publications with almost a third from regions outside of the traditional North American or European academic hubs, more than found in most recently published BEME reviews (Daniel et al. 2022; Doyle et al. 2023; Hamza et al. 2023; Li S et al. 2023). This review reflects a shift from centralized to distributed perspectives in AI research and development, offering a more inclusive and comprehensive understanding of AI’s role in shaping medical education worldwide.

This scoping review, in charting the course of AI in medical education, reveals not just the peaks and troughs of evidence, but also an expansive change that has permeated every facet of the educational environment. When considering the impact at such a broad environmental level, our analysis, viewed through the conceptual lens of Oberg’s theory of ‘culture shock’ (Oberg 1960; Zhou et al. 2008), aligns with the theory’s four phases: Honeymoon, Frustration, Adaptation, and Acceptance. Each phase represents a distinct emotional, psychological, and behavioral response to AI integration manifesting variably, across different areas and individuals.

The initial ‘Honeymoon’ phase is characterized by a mixture of fascination and idealism, often reflected in perspective and innovation pieces. These works, often written by experts, provide valuable insights, useful resources and some directions or foundations to move the field forward. These sometimes portray an overtly optimistic view of AI’s potential, lacking detailed exploration of challenges and ethical considerations.

In contrast, the ‘Frustration’ phase has been seen in evidence of attitudes of feeling overwhelmed, fear related to lack of understanding, and feeling lost and out of place. These concerns are not only valid but a necessary step in progressing towards acceptance of such a shock in the system. The ethical considerations and challenges related to specific applications of AI are key to consider. Balancing these aspects is crucial for sustainable implementation before the next phase.

The ambivalence observed in a large group of studies examining NLP, mostly ChatGPT, for medical exams [A160–191], exemplifies the non-linear and individual nature of cultural adaptation. This straddling of both Honeymoon and Frustration phases reflects a duality – awe of AI's capabilities juxtaposed with apprehension about its implications, particularly in reducing the need for traditional medical education methods. We draw attention to this group of publications not just as an interesting example, but to highlight that it is clear that without an innovative or different perspective, no further studies of this form appear justified.

As the initial frustrations and challenges of AI integration begin to crystallize, we transition into the ‘Adaptation’ phase. Educators begin to find practical and effective ways to incorporate AI tools. This phase is marked by the pragmatic ‘use cases’ of AI, showing a shift from initial awe and skepticism towards AI to a more balanced and functional integration of these technologies. However, discussions around these use cases in perspective pieces often lack depth in building a longitudinal or iterative understanding, which is critical for effective integration.

Finally, in the ‘Acceptance’ phase, we witness a transformation in the educators’ relationship with AI, evolving from mere adaptation to a state of genuine integration and comfort. This phase signifies a shift where AI is no longer viewed as an external, disruptive force but as an integral part of the educational blueprint. The subset of publications exhibits a deep understanding and skillful utilization of AI, seamlessly incorporating these technologies into their wide-ranging educational developments, characterized by a blend of creativity, critical thinking, and a forward-looking perspective that acknowledges the potential of AI while remaining cognizant of its limitations and ethical considerations. This mature phase of acceptance is not merely about recovery from the initial 'shock’ of AI integration; it represents a proactive, thoughtful embrace of these technologies. These educators have become pioneers in a true sense, setting benchmarks for how AI can be harmoniously and innovatively integrated into medical education. Their experiences and practices offer valuable insights for the broader educational community, illustrating a roadmap from apprehension to acceptance, a journey that is as much about personal and professional growth as it is about technological adoption.

In the process of extracting and charting the data, we have identified critical gaps in how a study is reported – with a need for a ‘compass’ to navigate the roadmap in AI applications in medical education. Here, addressing gaps involves a shift from mere reporting of disparate items, such as educational contexts, stages, or AI technologies, to a comprehensive framework that supports dissemination, replication, and innovation through several elements. We propose the FACETS framework, presented in Figure 4, as a structured approach for future AI studies in medical education, focusing on key elements: form, use case, context, education form, technology, and SAMR framework for technology integration. Our review reveals that these elements are commonly present in the literature, but they are typically reported in isolation, not cohesively in a single publication. Bringing together these disparate elements in a more systematic and unified manner is valuable in explaining to both educators and education researchers the specific details of the AI being employed, their relevance to past practice, and how they can be integrated into the educational context. It is designed to not only guide authors in conceptualizing their works but also to help readers to grasp the full scope and impact of AI in medical education. This approach is essential for advancing understanding and innovation in the field, forming a core recommendation from the author team for future research.

Figure 4. FACETS framework to report future AI innovations in medical education.

This framework cannot be applied to perspective pieces or attitudes surveys, but the use of this framework by authors of works published regarding any AI use-case or broader innovation will be of significant benefit. The FACETS framework may aid in identifying converging areas of research and serve as a useful guide for more focused syntheses in the future.

As this review was rigorous, systematic, and wide ranging in true alignment with the scoping tradition, we do not believe further scoping reviews are indicated. However, given the velocity of output, more in-depth systematic reviews will soon be indicated in the areas of admissions, teaching, assessment, and clinical reasoning. Another area where a systematic review may be warranted was out of scope for this review, namely, AI use in medical education research. Given the huge untapped potential of big data in educational institutions, this is a key area for further work.

Limitations of this review include the limitations of the scoping tradition, which by definition, does not synthesise evidence (Levac et al. 2010). Further limitations include subjectivity in categorizing studies (e.g. some studies could be classified as both teaching and assessment), as well as issues with consistent terminology, which underscore the complexity of navigating a rapidly advancing field like AI in medical education. The review iterated the search on a number of occasions in the early phases, reflecting the vast and evolving scope of AI literature. In light of this, studies may have been missed, despite systematic and rigorous methods. The inclusion of perspective articles, while unconventional for BEME reviews, offers a diverse perspective on AI in medical education. However, this inclusion may be perceived as diluting the empirical strength of the review. Thus, a decision was made to present their synthesis transparently as a separate parallel data set within the results, as there was much value in clarifying what these articles offered (and did not offer). We did not perform the sixth stage of the Arksey and O’Malley scoping review framework. While we had AI and education experts on the author team who fulfilled this role, skipping the external expert consultation step could be viewed as a limitation.

Implications for practice and policy

This review underscores the transformative role of AI in medical education, necessitating an integration of AI into various aspects like curriculum development, teaching methodologies, and learner assessment to maintain alignment with technological advancements. It calls for updating competency frameworks and establishing standardized guidelines for AI usage in education. The need for collaborative efforts between educators, policymakers, and AI developers is highlighted to ensure effective integration and address equity and accessibility issues, ensuring AI tools are available to all without bias. Continuous investment in resources and time is essential to transition from initial fascination with AI to its profound adoption in educational settings. The review also stresses managing risks associated with AI, including ethical concerns and potential misuse. Crucially, while leveraging AI for new insights and efficiencies, the irreplaceable human element in healthcare must be preserved. Ethical education is vital, ensuring AI enhances rather than replaces the healthcare provider’s role, upholding patient-provider relationships, empathy, and respect for patient autonomy. This approach prevents the dehumanization of care and ensures AI's use bolsters rather than undermines the core human skills and values central to the medical profession.

Implications for future research

The review highlights several key areas for future research and educational development. It underscores the need for comprehensive studies on the long-term impacts of AI, both on the overall landscape of medical education and specific learning outcomes. Additionally, there is a call for research addressing potential risks associated with AI, such as automation bias, over- and under-skilling, exacerbating disparities, and the preservation of essential clinical skills. It also recommends employing the FACETS framework for future AI studies in medical education. This comprehensive approach to research and development will ensure that AI is integrated into medical education in a manner that is ethically sound, educationally effective, and cognizant of the potential risks and challenges.

Conclusions

The landscape of AI in medical education, as charted in this review, spans a wide array of stages, specialties, purposes, and use cases, primarily reflecting early adaptation phases–only a few describe more in-depth employment for longitudinal or deep change. The proposed FACETS framework is a key outcome, offering a structured approach for future research and practice. While no immediate areas for systematic review are identified, the dynamic nature of AI in education necessitates ongoing vigilance and adaptability.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.

Notes on contributors

Morris Gordon

Morris Gordon, MBChB, PHD, MMed, is Co-chair of the BEME Council, Coordinating Editor of the Cochrane Gut Group, Professor of Evidence Synthesis and Systematic Review and a Consultant Pediatrician in the North West of the UK.

Michelle Daniel

Michelle Daniel, MD, MHPE, is Co-chair of the BEME Council, Associate Editor for Medical Teacher, Vice Dean for Medical Education and Clinical Professor of Emergency Medicine at the University of California, San Diego School of Medicine, La Jolla, California, USA.

Aderonke Ajiboye

Aderonke Ajiboye, BDS, MPH, is a Researcher at the School of Medicine and Dentistry, University of Central Lancashire.

Hussein Uraiby

Hussein Uraiby, MBChB, MMedEd, FHEA, is a Specialty Trainee in Histopathology at the University Hospitals of Leicester NHS Trust, Leicester, UK.

Nicole Y. Xu

Nicole Y. Xu is a medical student at the University of California, San Diego School of Medicine, La Jolla, California, USA.

Rangana Bartlett

Rangana Bartlett is a recent graduate of the University of California, San Diego and current research assistant at the Icahn School of Medicine at Mount Sinai, New York, New York, USA.

Janice Hanson

Janice Hanson, PhD, EdS, MH, is Professor of Medicine, Co-Director of the Medical Education Research Unit, and Director of Education Scholarship Development at Washington University in Saint Louis, Missouri, USA.

Mary Haas

Mary Haas, MD, MHPE, is Assistant Residency Program Director and Assistant Professor, Department of Emergency Medicine, University of Michigan Medical School, Ann Arbor, Michigan, USA.

Maxwell Spadafore

Maxwell Spadafore, MD, has expertise in AI and informatics and is a Chief Resident, Department of Emergency Medicine, University of Michigan Medical School, Ann Arbor, Michigan, USA.

Ciaran Grafton-Clarke

Ciaran Grafton-Clarke, MBChB, is an NIHR Academic Clinical Fellow in Cardiology, University of East Anglia, Norwich, UK.

Rayhan Yousef Gasiea

Rayhan Yousef Gasiea, MBBS, is a junior clinical fellow in medical education at Blackpool Hospitals NHS Foundation Trust.

Colin Michie

Collin Michie, FRCPCH, FRSPH, FLS, RNutr, is Associate Dean for Research and Knowledge Exchange at the School of Medicine and Dentistry, University of Central Lancashire, Preston, UK.

Janet Corral

Janet Corral, EdD is Professor of Medicine and Associate Dean of Undergraduate Medical Education at the University of Nevada School of Medicine, Reno, Nevada, USA.

Brian Kwan

Brian Kwan, MD has advanced training in informatics, and is an Assistant Clinical Professor and Medical Director for Education Informatics at UC San Diego School of Medicine, USA.

Diana Dolmans

Diana Dolmans, PhD, is a Professor and Chair of the Department of Educational Development & Research, staff member of the School of Health Professions Education, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, NL.

Satid Thammasitboon

Satid Thammasitboon, MD, MHPE, is Director of the Center for Research, Innovation and Scholarship in Health Professions Education, Texas Children’s Hospital and Baylor College of Medicine, Houston, Texas, USA, and member of the BEME Council.