A systematic review of research on speech-recognition chatbots for language learning: Implications for future directions in the era of large language models

References

• *Indicates the studies selected for review.
   Google Scholar
• *Ayedoun, E., Hayashi, Y., & Seta, K. (2020). Toward personalized scaffolding and fading of motivational support in L2 learner–dialogue agent interactions: An exploratory study. IEEE Transactions on Learning Technologies, 13(3), 604–616. https://doi.org/10.1109/TLT.2020.2989776
   Web of Science ®Google Scholar
• Bibauw, S., François, T., & Desmet, P. (2019). Discussing with a computer to practice a foreign language: Research synthesis and conceptual framework of dialogue-based CALL. Computer Assisted Language Learning, 32(8), 827–877. https://doi.org/10.1080/09588221.2018.1535508
   Web of Science ®Google Scholar
• Bibauw, S., François, T., Noortgate, V. d., & Desmet, W. (2022). Dialogue systems for language learning: A meta-analysis. Language Learning & Technology, 26(1), 1–24. https://doi.org/10.10125/73488
   Web of Science ®Google Scholar
• Bower, M., & Sturman, D. (2015). What are the educational affordances of wearable technologies? Computers & Education, 88, 343–353. https://doi.org/10.1016/j.compedu.2015.07.013
   Web of Science ®Google Scholar
• Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
   Google Scholar
• *Chen, H. H. J., Yang, C. T. Y., & Lai, K. K. W. (2020). Investigating college EFL learners’ perceptions toward the use of Google Assistant for foreign language learning. Interactive Learning Environments. Advance online publication. https://doi.org/10.1080/10494820.2020.1833043
   Web of Science ®Google Scholar
• *Chien, Y., Wu, T., Lai, C., Huang, Y. (2022). Investigation of the influence of artificial intelligence markup language-based LINE ChatBot in contextual English learning. Frontiers in Psychology, 13, 785752. https://doi.org/10.3389/fpsyg.2022.785752
   PubMedGoogle Scholar
• Coniam, D. (2014). The linguistic accuracy of chatbots: Usability from an ESL perspective. Text & Talk, 34(5), 545–567. https://doi.org/10.1515/text-2014-0018
   Web of Science ®Google Scholar
• Davis, R. O., Vincent, J., & Park, T. (2019). Reconsidering the voice principle with non-native language speakers. Computers & Education, 140, 103605. https://doi.org/10.1016/j.compedu.2019.103605
   Web of Science ®Google Scholar
• *Divekar, R. R., Drozdal, J., Chabot, S., Zhou, Y., Su, H., Chen, Y., Zhu, H., Hendler, J. A., & Braasch, J. (2021). Foreign language acquisition via artificial intelligence and extended reality: Design and evaluation. Computer Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2021.1879162
   Google Scholar
• *Dizon, G. (2020). Evaluating intelligent personal assistants for L2 listening and speaking development. Language Learning & Technology, 24(1), 16–26. https://doi.org/10.10125/44705
   Web of Science ®Google Scholar
• *Ebadi, S., & Amini, A. (2022). Examining the roles of social presence and human-likeness on Iranian EFL learners’ motivation using artificial intelligence technology: A case of CSIEC chatbot. Interactive Learning Environments. Advance online publication. https://doi.org/10.1080/10494820.2022.2096638
   Web of Science ®Google Scholar
• Elaish, M. M., Hussein, M. H., & Hwang, G. (2022). Critical research trends of mobile technology-supported English language learning: A review of the top 100 highly cited articles. Education and Information Technologies. Advance online publication. https://doi.org/10.1007/s10639-022-11352-6
   Google Scholar
• Fidan, M., & Gencel, N. (2022). Supporting the Instructional Videos With Chatbot and Peer Feedback Mechanisms in Online Learning: The Effects on Learning Performance and Intrinsic Motivation. Journal of Educational Computing Research, 60(7), 1716–1741. https://doi.org/10.1177/07356331221077901
   Web of Science ®Google Scholar
• *Forsyth, C. M., Luce, C., Zapata-Rivera, D., Jackson, G. T., Evanini, K., & So, Y. (2019). Evaluating English language learners’ conversations: Man vs. machine. Computer Assisted Language Learning, 32(4), 398–417. https://doi.org/10.1080/09588221.2018.1517126
   Web of Science ®Google Scholar
• *Fryer, L. K., Ainley, M., Thompson, A., Gibson, A., & Sherlock, Z. (2017). Stimulating and sustaining interest in a language course: An experimental comparison of Chatbot and Human task partners. Computers in Human Behavior, 75, 461–468. https://doi.org/10.1016/j.chb.2017.05.045
   Web of Science ®Google Scholar
• Fryer, L. K., Coniam, D., Carpenter, R., & Lăpușneanu, D. (2020). Bots for language learning now: Current and future directions. Language Learning & Technology, 24(2), 8–22. https://doi.org/10.10125/44719
   Web of Science ®Google Scholar
• *Fryer, L. K., Nakao, K., & Thompson, A. (2019). Chatbot learning partners: Connecting learning experiences, interest and competence. Computers in Human Behavior, 93, 279–289. https://doi.org/10.1016/j.chb.2018.12.023
   Web of Science ®Google Scholar
• Fu, Q., & Hwang, G. (2018). Trends in mobile technology-supported collaborative learning: A systematic review of journal publications from 2007 to 2016. Computers & Education, 119, 129–143. https://doi.org/10.1016/j.compedu.2018.01.004
   Web of Science ®Google Scholar
• *Gonulal, T. (2021). Investigating EFL learners’ humorous interactions with an intelligent personal assistant. Interactive Learning Environments. Advance online publication. https://doi.org/10.1080/10494820.2021.1974489
   Google Scholar
• *Hassani, K., Nahvi, A., & Ahmadi, A. (2016). Design and implementation of an intelligent virtual environment for improving speaking and listening skills. Interactive Learning Environments, 24(1), 252–271. https://doi.org/10.1080/10494820.2013.846265
   Web of Science ®Google Scholar
• Hew, K. F., Huang, W., Du, J., & Jia, C. (2023). Using chatbots to support student goal setting and social presence in fully online activities: Learner engagement and perceptions. Journal of Computing in Higher Education, 35(1), 40–68. Advance online publication. https://doi.org/10.1007/s12528-022-09338-x
   PubMed Web of Science ®Google Scholar
• *Hsu, H. L., Chen, H. H. J., & Todd, A. G. (2021). Investigating the impact of the Amazon Alexa on the development of L2 listening and speaking skills. Interactive Learning Environments. Advance online publication. https://doi.org/10.1080/10494820.2021.2016864
   Google Scholar
• Hsu, L. (2022). To CALL or not to CALL: Empirical evidence from neuroscience. Computer Assisted Language Learning, 35(4), 792–815. https://doi.org/10.1080/09588221.2020.1750429
   Web of Science ®Google Scholar
• *Hsu, M., Chen, P., & Yu, C. (2021). Proposing a task-oriented Chatbot system for EFL learners speaking practice. Interactive Learning Environments. Advance online publication. https://doi.org/10.1080/10494820.2021.1960864
   Web of Science ®Google Scholar
• Huang, W., Hew, K. F., & Fryer, L. K. (2022). Chatbots for language learning—Are they really useful? A systematic review of chatbot-supported language learning. Journal of Computer Assisted Learning, 38(1), 237–257. https://doi.org/10.1111/jcal.12610
   Web of Science ®Google Scholar
• Hwang, G.-J., & Chang, C.-Y. (2021). A review of opportunities and challenges of chatbots in education. Interactive Learning Environments. Advance online publication. https://doi.org/10.1080/10494820.2021.1952615
   Web of Science ®Google Scholar
• Hwang, G.-J., & Tsai, C.-C.. (2011). Research trends in mobile and ubiquitous learning: a review of publications in selected journals from 2001 to 2010. British Journal of Educational Technology, 42(4), E65–E70. https://doi.org/10.1111/bjet.2011.42.issue-4
   Web of Science ®Google Scholar
• *Hwang, W., Guo, B., Hoang, A., Chang, C., & Wu, N. (2022). Facilitating authentic contextual EFL speaking and conversation with smart mechanisms and investigating its influence on learning achievements. Computer Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2022.2095406
   Web of Science ®Google Scholar
• Jeon, J. (2021). Chatbot-assisted dynamic assessment (CA-DA) for L2 vocabulary learning and diagnosis. Computer Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2021.1987272
   Web of Science ®Google Scholar
• *Jeon, J. (2022a). Exploring a self-directed interactive app for informal EFL learning: A self-determination theory perspective. Education and Information Technologies, 27(4), 5767–5787. https://doi.org/10.1007/s10639-021-10839-y
   Web of Science ®Google Scholar
• *Jeon, J. (2022b). Exploring AI chatbot affordances in the EFL classroom: Young learners’ experiences and perspectives. Computer Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2021.2021241
   Web of Science ®Google Scholar
• Jeon, J, Lee, S, & Choe, H. (2022). Enhancing EFL pre-service teachers’ affordance noticing and utilizing with the Synthesis of Qualitative Evidence strategies: An exploratory study of a customizable virtual environment platform. Computers & Education, 190, 104620. https://doi.org/10.1016/j.compedu.2022.104620
   Web of Science ®Google Scholar
• Ji, H., Han, I., & Ko, Y. (2023). A systematic review of conversational AI in language education: Focusing on the collaboration with human teachers. Journal of Research on Technology in Education, 55(1), 48–63. Advance online publication. https://doi.org/10.1080/15391523.2022.2142873
   Web of Science ®Google Scholar
• Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., … Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274
   Web of Science ®Google Scholar
• *Lee, S., & Jeon, J. (2022). Visualizing a disembodied agent: Young EFL learners’ perceptions of voice-controlled conversational agents as language partners. Computer Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2022.2067182
   Web of Science ®Google Scholar
• Liang, J., Hwang, G., Chen, M. A., & Darmawansah, D. (2021). Roles and research foci of artificial intelligence in language education: An integrated bibliographic analysis and systematic review approach. Interactive Learning Environments. Advance online publication. https://doi.org/10.1080/10494820.2021.1958348
   Web of Science ®Google Scholar
• *Lin, C., & Mubarok, H. (2021). Learning analytics for investigating the mind map-guided AI chatbot approach in an EFL flipped speaking classroom. Educational Technology & Society, 24(4), 16–35.
   Web of Science ®Google Scholar
• Moher, D., Liberati, A., Terzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. BMJ, 339, b2535. https://doi.org/10.1136/bmj.b2535
   PubMed Web of Science ®Google Scholar
• *Morton, H., & Jack, M. (2010). Speech interactive computer-assisted language learning: A cross-cultural evaluation. Computer Assisted Language Learning, 23(4), 295–319. https://doi.org/10.1080/09588221.2010.493524
   Web of Science ®Google Scholar
• *Moussalli, S., & Cardoso, W. (2020). Intelligent personal assistants: Can they understand and be understood by accented L2 learners? Computer Assisted Language Learning, 33(8), 865–890. https://doi.org/10.1080/09588221.2019.1595664
   Web of Science ®Google Scholar
• O’Connor, C., & Joffe, H. (2020). Intercoder reliability in qualitative research: Debates and practical guidelines. International Journal of Qualitative Methods, 19, 1–13. https://doi.org/10.1177/1609406919899220
   Web of Science ®Google Scholar
• OpenAI. (2023, March 31). GPT-4. https://openai.com/research/gpt-4.
   Google Scholar
• Rospigliosi, P. A.. (2023). Artificial intelligence in teaching and learning: what questions should we ask of ChatGPT?. Interactive Learning Environments, 31(1), 1–3. https://doi.org/10.1080/10494820.2023.2180191
   Web of Science ®Google Scholar
• Shadiev, R., Hwang, W., & Huang, Y. (2017). Review of research on mobile language learning in authentic environments. Computer Assisted Language Learning, 30(3-4), 284–303. https://doi.org/10.1080/09588221.2017.1308383
   Web of Science ®Google Scholar
• Shadiev, R., & Liu, J. (2023). Review of research on applications of speech recognition technology to assist language learning. ReCALL, 35(1), 74–88. Advance online publication. https://doi.org/10.1017/S095834402200012X
   Web of Science ®Google Scholar
• *Tai, T. Y. (2022). Effects of intelligent personal assistants on EFL learners’ oral proficiency outside the classroom. Computer Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2022.2075013
   Web of Science ®Google Scholar
• *Tai, T. Y., & Chen, H. H. J. (2020). The impact of Google Assistant on adolescent EFL learners’ willingness to communicate. Interactive Learning Environments. Advance online publication. https://doi.org/10.1080/10494820.2020.1841801
   Web of Science ®Google Scholar
• *Tai, T. Y., & Chen, H. H. J. (2022a). The impact of intelligent personal assistants on adolescent EFL learners’ listening comprehension. Computer Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2022.2040536
   Web of Science ®Google Scholar
• *Tai, T. Y., & Chen, H. H. J. (2022b). The impact of intelligent personal assistants on adolescent EFL learners’ listening comprehension. Computer Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2022.2040536
   Web of Science ®Google Scholar
• *Timpe-Laughlin, V., & Dombi, J. (2020). Exploring L2 learners’ request behavior in a multi-turn conversation with a fully automated agent. Intercultural Pragmatics, 17(2), 221–257. https://doi.org/10.1515/ip-2020-0010
   Web of Science ®Google Scholar
• Timpe-Laughlin, V., Sydorenko, T., & Daurio, P. (2022). Using spoken dialogue technology for L2 speaking practice: What do teachers think? Computer Assisted Language Learning, 35(5-6), 1194–1217. https://doi.org/10.1080/09588221.2020.1774904
   Web of Science ®Google Scholar
• *Timpe-Laughlin, V., Sydorenko, T., & Dombi, J. (2022). Human versus machine: Investigating L2 learner output in face-to-face versus fully automated role-plays. Computer Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2022.2032184
   Google Scholar
• van Doremalen, J., Boves, L., Colpaert, J., Cucchiarini, C., & Strik, H. (2016). Evaluating automatic speech recognition-based language learning systems: A case study. Computer Assisted Language Learning, 29(4), 833–851. https://doi.org/10.1080/09588221.2016.1167090
   Web of Science ®Google Scholar
• *Wang, X., Liu, Q., Pang, H., Tan, S. C., Lei, J., Wallace, M. P., & Li, L. (2022). What matters in AI-supported learning: A study of human-AI interactions in language learning using cluster analysis and epistemic network analysis. Computers & Education. Advance online publication https://doi.org/10.1016/j.compedu.2022.104703.
   Google Scholar
• *Wang, X., Pang, H., Wallace, M. P., Wang, Q., & Chen, W. (2022). Learners’ perceived AI presences in AI-supported language learning: A study of AI as a humanized agent from community of inquiry. Computer Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2022.2056203
   Google Scholar
• Weizenbaum, J. (1966). ELIZA—A computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36–45. https://doi.org/10.1145/365153.365168
   Web of Science ®Google Scholar
• Xie, H., Chu, H., Hwang, G., & Wang, C. (2019). Trends and development in technology-enhanced adaptive/personalized learning: A systematic review of journal publications from 2007 to 2017. Computers & Education, 140, 103599. https://doi.org/10.1016/j.compedu.2019.103599
   Web of Science ®Google Scholar
• Xu, W., & Zammit, K. (2020). Applying thematic analysis to education: A hybrid approach to interpreting data in practitioner research. International Journal of Qualitative Methods, 19, 1–9. https://doi.org/10.1177/1609406920918810
   Web of Science ®Google Scholar
• *Xu, Y., Aubele, J., Vigil, V., Bustamante, A. S., Kim Y., & Warschauer, M. (2022). Dialogue with a conversational agent promotes children’s story comprehension via enhancing engagement. Child Development, 93(2), 149–167. https://doi.org/10.1111/cdev.13708
   PubMed Web of Science ®Google Scholar
• *Xu, Y., Wang, D., Collins, P., Lee, H., & Warschauer, M. (2021). Same benefits, different communication patterns: Comparing children’s reading with a conversational agent vs. a human partner. Computers & Education, 161, 104059. https://doi.org/10.1016/j.compedu.2020.104059
   Web of Science ®Google Scholar
• *Yang, C. T., Lai, S., & Chen, H. H. (2022). The impact of intelligent personal assistants on learners’ autonomous learning of second language listening and speaking. Interactive Learning Environments. Advance online publication. https://doi.org/10.1080/10494820.2022.2141266
   Web of Science ®Google Scholar
• *Yang, H., Kim, H., Lee, J., & Shin, D. (2022). Implementation of an AI chatbot as an English conversation partner in EFL speaking classes. ReCALL, 34(3), 327–343. https://doi.org/10.1017/S0958344022000039
   Web of Science ®Google Scholar
• Zhang, Y. (2023, March). Universal Speech Model (USM): State-of-the-art speech AI for 100+ languages. Google Research. https://ai.googleblog.com/2023/03/universal-speech-model-usm-state-of-art.html.
   Google Scholar