Talking Head-based L2 Pronunciation Training: Impact on Achievement Emotions, Cognitive Load, and Their Relationships with Learning Performance

ABSTRACT

Second language (L2) pronunciation training has been a worldwide task. Although computer technology makes it possible to develop a talking head to teach pronunciation like a real language teacher, little is known about how a talking head may act on L2 learners’ emotional and cognitive learning process. We investigate L2 learners’ achievement emotions, cognitive load, and pronunciation learning performance in a computer-assisted pronunciation training (CAPT) system embedded with four conditions: audio only (AU), a human face (HF), a 3D talking head with front view (3Df), and a 3D talking head with both front and profile views (3D). Results showed that, with learning time went on, participants’ perceived anxiety, boredom, and pride increased while shame and hopelessness decreased and enjoyment kept stable. With 3D, participants’ anxiety increased the most and boredom increased the least. Moreover, 3D group also perceived the highest germane load and got the highest pronunciation learning performance. Furthermore, anxiety and shame correlated with learning performance positively while boredom correlated with it negatively; enjoyment and pride correlated positively with performance on Mandarin tones. These findings significantly contribute to the efforts to design or select virtual characters for computer-aided language learning (CALL) and also provide a valuable reference to study achievement emotions in HCI systems.

Additional information

Funding

This work was supported by the National Key R\&D Program (2017YFB1002805), the National Natural Science Foundation of China (NSFC U1736202) and the ShenZhen Fundamental Research Program (JCYJ20170413161611534 and KQJSCX20170731163308665).

Notes on contributors

Xiaolan Peng

Xiaolan Peng received the BS and MS degrees in Human-Robot Interaction Lab from University of Science and Technology Beijing, P.R. China. She is currently an PhD student in Chinese Academy of Sciences, P.R. China. Her research interests include human-computer interaction and multimedia learning. Contact her at [email protected]

Hui Chen

Hui Chen received the PhD degree in computer science from the Chinese University of Hong Kong, Hong Kong. She is currently a Professor in Institute of Software Chinese Academy of Sciences, P.R. China. Her research interests include human-computer interaction and affective computing. Contact her at [email protected]

Lan Wang

Lan Wang obtained her PhD degree from the Machine Intelligence Lab of Cambridge University Engineering Department. She is currently a Professor of Shen-Zhen Institutes of Advanced Technology, Chinese Academy of Sciences, P.R. China. Her research interests are large vocabulary continuous speech recognition and speech visualization. Contact her at l[email protected]

Feng Tian

Feng Tian received the PhD degree from Institute of Software Chinese Academy of Sciences, P.R. China. He is currently a Professor in the Institute of Software, Chinese Academy of Sciences. He is interested in interaction techniques, pen-based user interface and ubiquitous computing. Contact him at [email protected]

Hongan Wang

Hongan Wang received the PhD degree from Institute of Software Chinese Academy of Sciences, P.R. China where he is currently a Research Professor and also the director of the Intelligence Engineering Lab. His research interests include real-time intelligence and human-computer interaction. Contact him at [email protected]