Leveraging evaluation of quality on medical education research with ChatGPT

Abstract

What is the educational challenge? The Medical Education Research Study Quality Instrument (MERSQI) is widely used to evaluate the quality of quantitative research in medical education. It has strong evidence of validity and is endorsed by guidelines. However, the manual appraisal process is time-consuming and resource-intensive, highlighting the need for more efficient methods. What are the proposed solutions? We propose to use ChatGPT to evaluate the quality of medical education research with the MERSQI and compare its scoring with those of human evaluators. What are the potential benefits to a broader global audience? Using ChatGPT to evaluate medical education research with the MERSQI can decrease the resources required for quality appraisal. This allows faster summaries of evidence, reducing the workload of researchers, editors, and educators. Furthermore, ChatGPTs’ capability to extract supporting excerpts provides transparency and may have the potential for data extraction and training new medical education researchers. What are the next steps? We plan to continue evaluating medical education research with ChatGPT using the MERSQI and other instruments to determine its feasibility in this realm. Moreover, we plan to investigate which types of studies ChatGPT performs best in.

Keywords:

• ChatGPT
• medical education research
• MERSQI
• quality

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Data are available upon reasonable request.

Geolocation information

Google Maps: −12.120978262980941, −77.02562395892912. Retrieved: May 20th, 2024

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.

Notes on contributors

Javier Alejandro Flores-Cohaila

Javier Alejandro Flores-Cohaila, MD, MEd candidate, is a researcher in medical education at Universidad Científica del Sur, where he works with the School of Medicine. He is also the academic director of the USAMEDIC institution for medical education.

Peter Garcia-Portocarrero

Peter Garcia-Portocarrero, MS, is a medical student at Universidad Nacional Pedro Ruiz Gallo and a member of SOCIMEP, an institution that promotes research for Peruvian medical students.

Deysi A. Saldaña-Amaya

Deysi A. Saldaña-Amaya, MS, is a medical student at Universidad Nacional Pedro Ruiz Gallo and a member of SOCIMEP, an institution that promotes research for Peruvian medical students.

Brayan Miranda-Chavez

Brayan Miranda-Chavez, MD. He is a professor of epidemiology and ethics at the School of Medicine at the Universidad Privada de Tacna.

Cesar Copaja-Corzo

Cesar Copaja-Corzo, MD, MsC candidate. He is a resident physician in infectious and tropical diseases at the Universidad Peruana Cayetano Heredia and an associate researcher at the Universidad San Ignacio de Loyola.