In the face of fallible AWE feedback: how do students respond?

Abstract

Automated writing evaluation (AWE) systems can provide immediate computer-generated quantitative assessments and qualitative diagnostic feedback on an enormous number of submitted essays. However, limited research attention has been paid to locally designed AWE systems used in English as a foreign language (EFL) classroom contexts. This study explored the precision of feedback provided by a Chinese AWE programme Pigai and students’ uptake of such feedback. Data comprising Pigai feedback, students’ revised submissions, incorporated Pigai suggestions and proportions of ‘good’, ‘neutral’ and ‘bad’ revisions were collected from 210 drafts by 30 undergraduate students in an EFL writing class. Results show that (1) the precision rates of AWE feedback differed across categories of feedback (ie grammar, collocations and mechanics); (2) students were selective in their utilisation of AWE feedback and able to adjust their uptake of AWE suggestions according to the accuracy of the feedback. Survey and interview data demonstrated that the students were cognisant of the strengths and limitations of the AWE system as a source of feedback for improving their writing. These results indicate that AWE feedback can usefully supplement peer and instructor feedback in the EFL writing classroom but cannot replace the latter, given the low precision and accuracy rates of the former in identifying a range of errors.

Keywords:

• Automated writing evaluation (AWE)
• English as a foreign language writing instruction
• precision of AWE feedback
• feedback uptake
• student perceptions