The effect of using machine translation on linguistic features in L2 writing across proficiency levels and text genres

Abstract

Many studies that have investigated the educational value of online machine translation (MT) in second language (L2) writing generally report significant improvements after MT use, but no study as of yet has comprehensively analyzed the effectiveness of MT use in terms of various measures in syntactic complexity, accuracy, lexical complexity, and fluency (CALF). The present study examined how learners’ use of MT affects CALF measures in L2 writing using evaluations by automated computational tools as well as human raters. In addition, the study investigated whether proficiency level and text genre affect the learners’ use of MT. A total of 91 Korean learners of English participated in the main task of the study in which they wrote on an assigned topic in English first without the help of any resource, and then on a different topic using only Google Translate a week later. Text analysis of students’ writing revealed major improvements in accuracy but unclear benefits in syntactic and lexical complexity. It was also found that MT use provided different advantages and disadvantages depending on the proficiency level (high vs. low) and text genre (narrative vs. argumentative). Survey responses strongly indicated that students are highly satisfied with MT and plan to use it again in the future despite being aware of its limitations. Overall, this study found that MT can be useful for improving accuracy but must be used with much discretion for it to benefit other aspects of L2 writing.

Keywords:

• CALF measures
• genre difference
• L2 writing
• machine translation
• proficiency effect

Disclosure statement

There is no conflict of interest for all authors.

Notes

1 Prior to the first writing task, the students were given a brief introduction to the purpose and characteristics of the genre (narrative vs argumentative) that they were assigned to and were given some basic information about MT (e.g., the use of MT around the world, different types of MT, examples of MT output). Since the present study examines how participants vary in their use of MT depending on their proficiency level and the genre of the writing task, the researchers did not try to define MT use or provide specific suggestions/guidelines so as not to bias the students towards certain types of behavior in the writing task.

2 Words are regarded as sophisticated when they are not on the list of the 2,000 most frequent words, as ranked by the American National Corpus.

Additional information

Funding

This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea under Grant NRF-2019S1A5B5A02041012.