Big claims, low outcomes: fact checking ChatGPT’s efficacy in handling linguistic creativity and ambiguity

Abstract

Ambiguity has always been a pain in the neck of Natural Language Processing (NLP). Despite enormous AI tools for human language processing, it remains a key concern for Language Technology Researchers to develop a linguistically intelligent tool that could effectively understand linguistic ambiguity and creativity possessed by human language. In this regard, the newly designed AI tool ChatGPT has dramatically attracted human attention due to its remarkable ability to answer human questions from a wide range of domains, which needs a reality check. This article scrutinises ChatGPT’s ability to answer and interpret neologisms, codemixing, and linguistically ambiguous sentences. For this, we have tested lexically, syntactically, and semantically ambiguous expressions, codemixed words, as well as a few language game instances. The findings show that ChatGPT still fails to understand linguistically complex sentences, specifically those common in everyday discourse or not part of any standard textbook. More specifically, semantically ambiguous sentences and language games remain an uphill task for ChatGPT to understand. This has implications for further improving the output of ChatGPT.

Keywords:

• Linguistic ambiguity
• creativity
• ChatGPT
• AI tool development

REVIEWING EDITOR:

• Jeroen van de Weijer, College of International Studies, Shenzhen University, Shenzhen, China

SUBJECTS:

• Artificial Intelligence
• Language & Linguistics
• Language Teaching & Learning

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Md. Tauseef Qamar

Md. Tauseef Qamar is an Assistant Professor of English Linguistics at the School of Advanced Sciences and Languages, VIT Bhopal University, Sehore, Madhya Pradesh, India. He received his doctorate in Computational Linguistics at Aligarh Muslim University, Aligarh, India, in 2019. Currently, he teaches undergraduate and graduate engineering and management major students Computational linguistics, Cross-cultural communication, and Soft skills courses. His academic and research area focuses on Multiword Expressions, Foreign Language Teaching Methodologies, Technology-enhanced Learning, English in the Workplace, and Indic Machine Translation.

Juhi Yasmeen

Juhi Yasmeen is an Assistant Professor of English Linguistics at the School of Advanced Sciences and Languages, VIT Bhopal University, Sehore, Madhya Pradesh, India. She received PhD in Applied Linguistics at Aligarh Muslim University, Aligarh, India, in 2017. She currently teaches undergraduate and graduate engineering and management students Cross-cultural communication, Technical Communication, and Soft skills courses. Her research interests are Translation Studies, English Language Teaching, Education and Technology, Syllabus Design, and Media Linguistics.

Sanket Kumar Pathak

Sanket Kumar Pathak is a Language Manager at Mihup Communications, Kolkata, West Bengal, India. He received his doctorate in linguistics at Lucknow University, India, in 2010. Currently, he handles a large team of linguists and engineers. Besides this, he has developed several Indic machine translation systems. His research focuses on Artificial Intelligence Language Technology such as ASR, TTS, NLU/NLG, MTs, etc.

Shahab Saquib Sohail

Shahab Saquib Sohail holds a master’s and doctorate from Aligarh Muslim University, majoring in computer science. His research interests include Social Networks and Privacy, Recommender Systems, Personally Identifiable Information (PII), and Users’ Online Behavior. Notably, he has developed a book recommender system tailored for computer science graduates in India. With over 70 publications in reputed journals and top-tier conferences, he serves on the editorial boards of numerous scientific journals.

Dag Øivind Madsen

Dag Øivind Madsen obtained his bachelor’s degree from the University of Bergen, Norway, followed by an M.Sc. degree from the London School of Economics in 2002 and a Ph.D. from the Norwegian School of Economics in 2011. Presently, he serves as a Professor at the University of South-Eastern Norway. His research focuses on Industry 4.0/5.0 and the Utilization of emerging technologies, such as Big Data and Artificial Intelligence, in various business and organizational settings.

Mithila Rangarajan

Mithila Rangarajan, a postgraduate student in English at St. Joseph’s University in Bengaluru, India, centres her academic interests on Translation Studies, Aesthetics, and Artificial Intelligence.