Evaluating the effectiveness of a computerised dynamic assessment of L2 English email requests

Abstract

Automated writing evaluation can be effective in providing support for L2 English learners. However, little research to date has investigated its use in the teaching of pragmatics in relation to L2 email writing, grounded in a sociocultural perspective on learning. We employ a quasi-experimental approach, investigating the effectiveness of a computerised dynamic assessment programme (C-DA) of L2 email writing, focusing on pragmatic development. The C-DA employs a developmentally sensitive approach, providing automated, immediate mediation to learners. The C-DA was administered twice – with a two-week delay between administrations – to a single group of 17 Japanese English L2 learner participants at a Japanese university; participants were 19–22 years of age with intermediate L2 proficiency levels. Text, identified pragmatic failure and mediation data were elicited from both administrations. Analysis of pragmatic failure frequency and explicitness of the mediation necessary for successful resolution of the identified pragmatic failure shows both frequency and explicitness decreasing not only within a round of administration, but also between rounds. Evidence of learner development was found across all types of pragmatic failure, including the requesting head act, email openings and closings. The study provides evidence to support a sociocultural approach to assessment and learning with regards to the pragmatics aspect of L2 email writing, in which mediation is sensitive to individual learners’ developmental needs. Further, findings support the use of a computerised approach to DA focusing on pragmatic competence, allowing for wider access to DA methodology among large learner group contexts.

Keywords:

• Automated mediation
• computerised dynamic assessment
• L2 email writing
• L2 pragmatics

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

Japan Society for the Promotion of Science 19K00826.

Notes on contributors

Allan Nicholas

Allan Nicholas is a senior associate professor at the Center for Language Research, University of Aizu, Japan. His research interests include sociocultural theory, dynamic assessment, computer-assisted language learning, and L2 pragmatics learning.

John Blake

John Blake is a senior associate professor at the Center for Language Research, University of Aizu, Japan. His research interests include computer-assisted language learning, corpus linguistics, and second language acquisition.

Jeremy Perkins

Jeremy Perkins is a senior associate professor at the Center for Language Research, University of Aizu, Japan. His research interests include phonetics, second language acquisition and language learning.

Maxim Mozgovoy

Maxim Mozgovoy is a senior associate professor in the Department of Computer Science and Engineering, University of Aizu, Japan. His research interests include computer-assisted language learning, natural language processing and artificial intelligence.