A Systematic Review of the Limitations and Associated Opportunities of ChatGPT

References

• Alafnan, M. A., Dishari, S., Jovic, M., & Lomidze, K. (2023). ChatGPT as an educational tool: Opportunities, challenges, and recommendations for communication, business writing, and composition courses. Journal of Artificial Intelligence and Technology, 3(2), 60–68. https://doi.org/10.37965/jait.2023.0184
   Google Scholar
• Ali, M. J. (2023). ChatGPT and lacrimal drainage disorders: Performance and scope of improvement. Ophthalmic Plastic and Reconstructive Surgery, 39(5), 515–514. https://doi.org/10.1097/IOP.0000000000002418
   PubMed Web of Science ®Google Scholar
• Amin, M. M., Cambria, E., & Schuller, B. W. (2023). Will affective computing emerge from foundation models and general artificial intelligence? A first evaluation of ChatGPT. IEEE Intelligent Systems, 38(2), 15–23. https://doi.org/10.1109/MIS.2023.3254179
   Web of Science ®Google Scholar
• Ariyaratne, S., Iyengar, K. P., Nischal, N., Chitti Babu, N., & Botchu, R. (2023). A comparison of ChatGPT-generated articles with human-written articles. Skeletal Radiology, 52(9), 1755–1758. https://doi.org/10.1007/s00256-023-04340-5
   PubMed Web of Science ®Google Scholar
• Au Yeung, J., Kraljevic, Z., Luintel, A., Balston, A., Idowu, E., Dobson, R. J., & Teo, J. T. (2023). AI chatbots not yet ready for clinical use. Frontiers in Digital Health, 5, 1161098. https://doi.org/10.3389/fdgth.2023.1161098
   PubMedGoogle Scholar
• Aydin, Ö., & Karaarslan, E. (2023). Is ChatGPT leading generative AI? What is beyond expectations? Academic Platform Journal of Engineering and Smart Systems, 11(3), 118–134. https://doi.org/10.21541/apjess.1293702
   Google Scholar
• Bitzenbauer, P. (2023). ChatGPT in physics education: A pilot study on easy-to-implement activities. Contemporary Educational Technology, 15(3), ep430. https://doi.org/10.30935/cedtech/13176
   Google Scholar
• Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
   Google Scholar
• Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., Nori, H., Palangi, H., Ribeiro, M. T., & Zhang, Y. (2023). Sparks of Artificial General Intelligence: Early experiments with GPT-4. https://doi.org/10.48550/ARXIV.2303.12712
   Google Scholar
• Cadamuro, J., Cabitza, F., Debeljak, Z., De Bruyne, S., Frans, G., Perez, S. M., Ozdemir, H., Tolios, A., Carobene, A., & Padoan, A. (2023). Potentials and pitfalls of ChatGPT and natural-language artificial intelligence models for the understanding of laboratory medicine test results. An assessment by the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Working Group. Clinical Chemistry and Laboratory Medicine, 61(7), 1158–1166. https://doi.org/10.1515/cclm-2023-0355
   Web of Science ®Google Scholar
• Cascella, M., Montomoli, J., Bellini, V., & Bignami, E. (2023). Evaluating the feasibility of ChatGPT in healthcare: An analysis of multiple clinical and research scenarios. Journal of Medical Systems, 47(1), 33. https://doi.org/10.1007/s10916-023-01925-4
   PubMed Web of Science ®Google Scholar
• Clark, T. M. (2023). Investigating the use of an artificial intelligence chatbot with general chemistry exam questions. Journal of Chemical Education, 100(5), 1905–1916. https://doi.org/10.1021/acs.jchemed.3c00027
   Web of Science ®Google Scholar
• Day, T. (2023). A preliminary investigation of fake peer-reviewed citations and references generated by ChatGPT. The Professional Geographer, 75(6), 1024–1027. https://doi.org/10.1080/00330124.2023.2190373
   Web of Science ®Google Scholar
• Duong, D., & Solomon, B. D. (2023). Analysis of large-language model versus human performance for genetics questions. European Journal of Human Genetics, 32(4), 466–468. https://doi.org/10.1038/s41431-023-01396-8
   PubMed Web of Science ®Google Scholar
• Fergus, S., Botha, M., & Ostovar, M. (2023). Evaluating academic answers generated using ChatGPT. Journal of Chemical Education, 100(4), 1672–1675. https://doi.org/10.1021/acs.jchemed.3c00087
   Web of Science ®Google Scholar
• Giannos, P., & Delardas, O. (2023). Performance of ChatGPT on UK standardized admission tests: Insights from the BMAT, TMUA, LNAT, and TSA examinations. JMIR Medical Education, 9, e47737. https://doi.org/10.2196/47737
   PubMedGoogle Scholar
• Gregorcic, B., & Pendrill, A. M. (2023). ChatGPT and the frustrated Socrates. Physics Education, 58(3), 035021. https://doi.org/10.1088/1361-6552/acc299
   Google Scholar
• Hassani, H., & Silva, E. S. (2023). The role of ChatGPT in data science: How AI-assisted conversational interfaces are revolutionizing the field. Big Data and Cognitive Computing, 7(2), 62. https://doi.org/10.3390/bdcc7020062
   Google Scholar
• Hoch, C. C., Wollenberg, B., Lüers, J. C., Knoedler, S., Knoedler, L., Frank, K., Cotofana, S., & Alfertshofer, M. (2023). ChatGPT’s quiz skills in different otolaryngology subspecialties: An analysis of 2576 single-choice and multiple-choice board certification preparation questions. European Archives of Oto-Rhino-Laryngology, 280(9), 4271–4278. https://doi.org/10.1007/s00405-023-08051-4
   PubMed Web of Science ®Google Scholar
• Hunter, J. D. (2007). Matplotlib: A 2D graphics environment. Computing in Science & Engineering, 9(3), 90–95. https://doi.org/10.1109/MCSE.2007.55
   Web of Science ®Google Scholar
• Ibrahim, H., Asim, R., Zaffar, F., Rahwan, T., & Zaki, Y. (2023). Rethinking homework in the age of artificial intelligence. IEEE Intelligent Systems, 38(2), 24–27. https://doi.org/10.1109/MIS.2023.3255599
   Web of Science ®Google Scholar
• Kortemeyer, G. (2023). Could an artificial-intelligence agent pass an introductory physics course? Physical Review Physics Education Research, 19(1), 1–18. https://doi.org/10.1103/PhysRevPhysEducRes.19.010132
   Web of Science ®Google Scholar
• Kumar, H. A. (2023). Analysis of chatgpt tool to assess the potential of its utility for academic writing in biomedical domain. Biology, Engineering, Medicine and Science Reports, 9(1), 24–30. https://doi.org/10.5530/bems.9.1.5
   Google Scholar
• Lahat, A., Shachar, E., Avidan, B., Glicksberg, B., & Klang, E. (2023). Evaluating the utility of a large language model in answering common patients’ gastrointestinal health-related questions: Are we there yet? Diagnostics, 13(11), 1950. https://doi.org/10.3390/diagnostics13111950
   Web of Science ®Google Scholar
• Lai, K. (2023). How well does ChatGPT handle reference inquiries? An analysis based on question types and question complexities. College & Research Libraries, 84(6), 974–995. https://doi.org/10.5860/crl.84.6.974
   Web of Science ®Google Scholar
• Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), 410. https://doi.org/10.3390/educsci13040410
   Web of Science ®Google Scholar
• McIntosh, T. R., Liu, T., Susnjak, T., Watters, P., Ng, A., & Halgamuge, M. N. (2024). A culturally sensitive test to evaluate nuanced GPT hallucination. IEEE Transactions on Artificial Intelligence, 1–13. https://doi.org/10.1109/TAI.2023.3332837
   Google Scholar
• Nikolic, S., Daniel, S., Haque, R., Belkina, M., Hassan, G. M., Grundy, S., Lyden, S., Neal, P., & Sandison, C. (2023). ChatGPT versus engineering education assessment: A multidisciplinary and multi-institutional benchmarking and analysis of this generative artificial intelligence tool to investigate assessment integrity. European Journal of Engineering Education, 48(4), 559–614. https://doi.org/10.1080/03043797.2023.2213169
   Web of Science ®Google Scholar
• OpenAI. (2023). GPT-4 technical report. https://cdn.openai.com/papers/gpt-4.pdf
   Google Scholar
• Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. International Journal of Surgery, 88(2021), 105906. https://doi.org/10.1016/j.ijsu.2021.105906
   PubMed Web of Science ®Google Scholar
• Parsons, B., & Curry, J. H. (2024). Can ChatGPT pass graduate-level instructional design assignments? Potential implications of artificial intelligence in education and a call to action. TechTrends, 68(1), 67–78. https://doi.org/10.1007/s11528-023-00912-3
   Web of Science ®Google Scholar
• Poole, F. (2022). Using Chatgpt to design language material and exercises. https://fltmag.com/chatgpt-design-material-exercises/
   Google Scholar
• Prieto, S. A., Mengiste, E. T., & García de Soto, B. (2023). Investigating the use of ChatGPT for the scheduling of construction projects. Buildings, 13(4), 857. https://doi.org/10.3390/buildings13040857
   Web of Science ®Google Scholar
• Puthenpura, V., Nadkarni, S., DiLuna, M., Hieftje, K., & Marks, A. (2023). Personality changes and staring spells in a 12-year-old child: A case report incorporating ChatGPT, a natural language processing tool driven by artificial intelligence (AI). Cureus, 15(3), e36408. https://doi.org/10.7759/cureus.36408
   PubMed Web of Science ®Google Scholar
• Rahman, M. M., & Watanobe, Y. (2023). ChatGPT for education and research: Opportunities, threats, and strategies. Applied Sciences, 13(9), 5783. https://doi.org/10.3390/app13095783
   Google Scholar
• Rahman, M., Terano, H. J. R., Rahman, N., Salamzadeh, A., & Rahaman, S. (2023). Chatgpt and academic research: A review and recommendations based on practical examples. Journal of Education, Management and Development Studies, 3(1), 1–12. https://doi.org/10.52631/jemds.v3i1.175
   Google Scholar
• Ray, P. P. (2023). ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems, 3, 121–154. https://doi.org/10.1016/j.iotcps.2023.04.003
   Google Scholar
• Rozado, D. (2023). The political biases of ChatGPT. Social Sciences, 12(3), 148. https://doi.org/10.3390/socsci12030148
   Google Scholar
• Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning & Teaching, 6(1), 342–363. https://doi.org/10.37074/jalt.2023.6.1.9
   Google Scholar
• Sallam, M., Salim, N., Barakat, M., & Al-Tammemi, A. (2023). ChatGPT applications in medical, dental, pharmacy, and public health education: A descriptive study highlighting the advantages and limitations. Narra J, 3(1), e103. https://doi.org/10.52225/narra.v3i1.103
   PubMedGoogle Scholar
• Sanmarchi, F., Bucci, A., Nuzzolese, A. G., Carullo, G., Toscano, F., Nante, N., & Golinelli, D. (2023). A step-by-step researcher’s guide to the use of an AI-based transformer in epidemiology: An exploratory analysis of ChatGPT using the STROBE checklist for observational studies. Journal of Public Health, 1–36. https://doi.org/10.1007/s10389-023-01936-y
   Web of Science ®Google Scholar
• Segal, S., & Khanna, A. K. (2023). Anesthetic management of a patient with juvenile hyaline fibromatosis: A case report written with the assistance of the large language model chatgpt. Cureus, 15(3), e35946. https://doi.org/10.7759/cureus.35946
   PubMed Web of Science ®Google Scholar
• Seth, I., Sinkjær Kenney, P., Bulloch, G., Hunter-Smith, D. J., Bo Thomsen, J., & Rozen, W. M. (2023). Artificial or augmented authorship? A conversation with a Chatbot on base of thumb arthritis. Plastic and Reconstructive Surgery. Global Open, 11(5), e4999. https://doi.org/10.1097/GOX.0000000000004999
   PubMedGoogle Scholar
• Shoufan, A. (2023). Can students without prior knowledge use ChatGPT to answer test questions? An empirical study. ACM Transactions on Computing Education, 23(4), 1–29. https://doi.org/10.1145/3628162
   Web of Science ®Google Scholar
• Singh, H., & Singh, A. (2023). Chatgpt: Systematic review, applications, and agenda for multidisciplinary research. Journal of Chinese Economic and Business Studies, 21(2), 193–212. https://doi.org/10.1080/14765284.2023.2210482
   Web of Science ®Google Scholar
• Sok, S., & Heng, K. (2024). Opportunities, challenges, and strategies for using ChatGPT in higher education: A literature review. Journal of Digital Educational Technology, 4(1), ep2401. https://doi.org/10.30935/jdet/14027
   Google Scholar
• Stojanov, A. (2023). Learning with ChatGPT 3.5 as a more knowledgeable other: An autoethnographic study. International Journal of Educational Technology in Higher Education, 20(1), 35. https://doi.org/10.1186/s41239-023-00404-7
   Web of Science ®Google Scholar
• Su, J., & Yang, W. (2023). Unlocking the power of chatgpt: A framework for applying generative ai in education. ECNU Review of Education, 6(3), 355–366. https://doi.org/10.1177/20965311231168423
   Google Scholar
• Suchman, K., Garg, S., & Trindade, A. J. (2023). Chat generative pretrained transformer fails the multiple-choice American College of Gastroenterology Self-Assessment Test. The American Journal of Gastroenterology, 118(12), 2280–2282. https://doi.org/10.14309/ajg.0000000000002320
   PubMed Web of Science ®Google Scholar
• Tan, T. F., Thirunavukarasu, A. J., Campbell, J. P., Keane, P. A., Pasquale, L. R., Abramoff, M. D., Kalpathy-Cramer, J., Lum, F., Kim, J. E., Baxter, S. L., & Ting, D. S. W. (2023). Generative artificial intelligence through chatgpt and other large language models in ophthalmology: Clinical applications and challenges. Ophthalmology Science, 3(4), 100394. https://doi.org/10.1016/j.xops.2023.100394
   PubMedGoogle Scholar
• Thirunavukarasu, A. J., Hassan, R., Mahmood, S., Sanghera, R., Barzangi, K., El Mukashfi, M., & Shah, S. (2023). Trialling a large language model (ChatGPT) in General practice with the applied knowledge test: Observational study demonstrating opportunities and limitations in primary care. JMIR Medical Education, 9, e46599. https://doi.org/10.2196/46599
   PubMedGoogle Scholar
• Wagner, M. W., & Ertl-Wagner, B. B. (2023). Accuracy of information and references using ChatGPT-3 for retrieval of clinical radiological information. Canadian Association of Radiologists Journal, 75(1), 69–73. https://doi.org/10.1177/08465371231171125
   PubMed Web of Science ®Google Scholar