To what extent does ChatGPT understand genetics?

ABSTRACT

OpenAI’s ChatGPT, is a conversational chatbot that uses Generative Pre-trained Transformer or GPT language model to mimic human-like responses. Here we evaluated its performance in providing responses to genetics questions across five different tasks including solid genetic basics, identifying inheritance pattern based on described pedigrees, interpreting genetic mutation notations, solving genetic population problems, and taking a medical genetics Ph.D. entrance exam. Our results showed that ChatGPT was able to generate correct answers to approximately 70% of questions (n = 145). Its performance on descriptive and memorisation tasks showed more accuracy compared to analytical and critical thinking ones. Failure to capture human writing nuances in questions, and applying genetic basics to solve problems, alongside providing false information were the most notable drawbacks. However, overall results were promising suggesting that ChatGPT could be a well-prepared assistant for genetic educators and healthcare providers.

KEYWORDS:

• ChatGPT
• generative pre-trained transformer
• genetics
• artificial intelligence
• large language model

Acknowledgments

This study was supported by the Golestan University of Medical Sciences (Grant Number: 13350).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

No additional data were created or used in this study beyond what is presented in the manuscript.

Ethics

This study was ethically approved by Ethics Committee of Golestan University of Medical Sciences (Ethics Code: IR.GOUMS.REC.1401.522)

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/14703297.2023.2258842

Additional information

Funding

This work was supported by the Golestan University of Medical Sciences and Health Services [113350].

Notes on contributors

Teymoor Khosravi

Teymoor Khosravi is a postgraduate student at Golestan university of medical sciences studying human genetics.

Zainab M. Al Sudani

Zainab M. Al Sudani is a medical student at Golestan university of medical sciences.

Morteza Oladnabi

Morteza Oladnabi is an associate professor of medical genetics working at Gorgan congenital malformations research center.