Evaluating the motivational impact of CALL systems: current practices and future directions

Abstract

A major aim of computer-assisted language learning (CALL) is to create computer environments that facilitate students’ second language (L2) acquisition. To achieve this aim, CALL employs technological innovations to create novel types of language practice. Evaluations of the new practice types serve the important role of distinguishing effective practice environments from less effective environments, while simultaneously informing educational practices and second language acquisition (SLA) theory. Accordingly, evaluations of CALL systems necessarily deal with multiple criteria. Most researchers would probably agree that motivation is an important criterion in CALL evaluations: a system can provide sufficient L2 input and opportunities for L2 output, yet fail to be pedagogically effective if learners are unwilling to participate. Furthermore, knowledge of the motivational impact of practice can provide valuable context linking individual language learners, practice effort and learning outcomes. From the perspective of recent theoretical developments in L2 motivation theory, this paper surveys a representative sample of CALL system evaluations that include motivational impact. Our analysis suggests not only that CALL needs to do more to align its treatment of motivation with recent L2 motivation theories, but also that it is well positioned to do so. We find that (1) few CALL studies treat motivation as it relates to practice as a dynamic variable, (2) behavioural practice logs are underexploited and (3) very few evaluations take into account learners’ individual interests and goals. Drawing on these and other findings, we suggest four new directions for developing the motivation dimension in CALL evaluations.

Keywords:

• CALL
• evaluation
• motivation
• review

Acknowledgements

We would like to thank our colleague Bart Penning de Vries for his comments regarding an earlier version of this manuscript. We would also like to thank the two anonymous reviewers whose helpful comments have improved the quality of this manuscript.

Funding

This work is part of the research program Feedback and the Acquisition of Syntax in Oral Proficiency (FASOP) NWO-Dossier Number 360-75-010, which is funded by the Netherlands Organisation for Scientific Research (NWO125D).

Notes

1. Previous literature has organised self-esteem (Brown, 1981) and later motivation (Brown, 2000; Ellis, 1985) into global, situational and task levels. In this review we organise the research using the categories of global and situated motivation to align with recent literature distinguishing between macro- and micro-perspectives on L2 motivation (e.g. Kissau, 2006; Ushioda, 2009b; Ushioda & Dörnyei, 2012).

2. Sources included primarily Computer Assisted Language Learning, ReCALL and CALICO Journal, but also Computers & Education, International Journal of Artificial Intelligence in Education and User Modeling and User-Adapted Interaction. We also considered a number of articles published in proceedings for conferences such as the International Conference on Artificial Intelligence in Education and the International Conference on Intelligent Tutoring Systems. Finally, during the peer evaluation process, two studies were suggested by the reviewers. Due to their relevance, we have also included these articles, published in Innovation in Language Learning and Teaching and The Canadian Journal of Applied Linguistics, in this review.

Additional information

Notes on contributors

Stephen Bodnar

Stephen Bodnar has an MSc degree in speech and language processing from Macquarie University and is currently a doctoral student at CLST. His research interests include L2 motivation, speech-interactive CALL systems and student modeling.

Catia Cucchiarini

Catia Cucchiarini is a senior researcher at CLST. Her research activities include speech processing, CALL and the application of automatic speech recognition (ASR) to language learning and testing.

Helmer Strik

Helmer Strik is an associate professor in Speech Science and Technology at the Radboud University Nijmegen. His fields of expertise include systems, e-Learning and e-Health.

Roeland van Hout

Roeland van Hout is a professor in applied linguistics and variationist linguistics at the Radboud University Nijmegen. He publishes in the fields of sociolinguitics, dialectology and second language acquisition and has a special interest in research methodology and statistics.