ChatGPT for assessment writing

Abstract

What is the educational challenge?

Medical schools invest significant resources into the creation of multiple-choice items for assessments. This process is costly and requires faculty training. Recently ChatGPT has been used in various areas to improve content creation efficiency, and it has otherwise been used to answer USMLE-style assessment items.

What are the proposed solutions?

We proposed the use of ChatGPT to create initial drafts of multiple-choice items.

What are the potential benefits to a wider global audience?

The use of ChatGPT to generate assessment items can decrease resources required, allowing for the creation of more items, and freeing-up faculty time to perform higher level assessment activities. ChatGPT is also able to consistently produce items using a standard format while adhering to item writing guidelines, which can be very challenging for faculty teams.

What are the next steps?

We plan to pilot ChatGPT drafted questions and compare item statistics for those written by ChatGPT with those written by our content experts. We also plan to further identify the types of questions that ChatGPT is most appropriate for, and incorporate media into assessment items (e.g. images, videos).

Keywords:

• Assessment
• artificial intelligence
• AI
• ChatGPT
• medical education

Acknowledgements

The authors thank the course and content directors of the School of Medicine Trek Curriculum for supporting this program, and the students of the Class of 2026.

Ethics statement

According to the policy activities that constitute research under COMIRB, this work met criteria for quality improvement activities exempt from ethics review.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.

Notes on contributors

Matthew Zuckerman

Matthew Zuckerman, MD, is Associate Professor and Pharmacology Content Director, an active Medical Toxicologist and Emergency Medicine physician, and a core mentor with Academic Life in Emergency Medicine faculty incubator. He has an interest in teaching with technology, assessment, and social media.

Ryan Flood

Ryan P. Flood, D.O., is an Assistant Professor with the University of Colorado School of Medicine, Department of Medicine and Division of Renal Disease and Hypertension. Dr. Flood works as a full-time clinical educator in addition to his roles as Assistant Course Director of the Renal and Urinary Systems Plains Curriculum and is an Assessment Developer with the Office of Assessment, Evaluation, and Outcomes.

Rachael J. B. Tan

Rachael J. B. Tan, PhD, is an Assistant Professor and Director of Assessment, Evaluation, and Outcomes at the University of Colorado, School of Medicine. She is a psychometrician with significant expertise in assessment design, analysis, and validation.

Nicole Kelp

Nicole Kelp, PhD, is the Chair of Medical Science Content for the University of Colorado School of Medicine as well as Assistant Professor of Microbiology, Immunology, and Pathology at Colorado State University. She directs pre-clerkship medical education courses in hematology, endocrinology, and reproduction and performs research in science education, including assessment methodologies.

David J. Ecker

David J. Ecker, MD, is an Associate Professor of Medicine in the Division of Hospital Medicine and the Assistant Dean of Medical Education, Director of the Pre-clerkship Curriculum, and Director of Standardized Patient Assessment at the University of Colorado School of Medicine in Aurora, Colorado.

Jonathan Menke

Jonathan Menke, M.Ed., is an Academic Services Senior Professional for the Office of Assessment, Evaluation, and Outcomes at the University of Colorado-Anschutz School of Medicine. He has extensive teaching and school administration experience on four continents.

Tai Lockspeiser

Tai Lockspeiser, MD, is a Professor of Pediatrics and the Assistant Dean of Assessment, Evaluation, and Outcomes at the University of Colorado, School of Medicine. She is a general pediatrician with significant expertise in medical education scholarship and leadership.