Investigating the promise of automated writing evaluation for supporting formative writing assessment at scale

ABSTRACT

We investigated the promise of a novel approach to formative writing assessment at scale that involved an automated writing evaluation (AWE) system called MI Write. Specifically, we investigated elementary teachers’ perceptions and implementation of MI Write and changes in students’ writing performance in three genres from Fall to Spring associated with this implementation. Teachers in Grades 3–5 (n = 14) reported that MI Write was usable and acceptable, useful, and desirable; however, teachers tended to implement MI Write in a limited manner. Multilevel repeated measures analyses indicated that students in Grades 3–5 (n = 570) tended not to increase their performance from Fall to Spring except for third graders in all genres and fourth graders’ narrative writing. Findings illustrate the importance of educators utilising scalable formative assessments to evaluate and adjust core instruction.

KEYWORDS:

• Writing assessment
• elementary
• automated essay scoring
• automated writing evaluation

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed here

Additional information

Funding

This research was supported by Grant #201800044 from the Spencer Foundation to the University of Delaware. The opinions expressed are those of the authors and do not represent the views of the Foundation, and no official endorsement by this agency should be inferred. The authors declare no conflicts of interest relative to this research study.

Notes on contributors

Joshua Wilson

Andrew Potter is a Ph.D. student at the University of Delaware School of Education. His research interests include integrated reading and writing instruction and development for at-risk students using quantitative and mixed methods.

Matthew C. Myers

Matthew C. Myers is a Ph.D. student in Educational Statistics and Research Methods at the University of Delaware School of Education. His research interests include applications of machine learning for writing assessment as well as the evaluation and optimization of machine learning approaches.

Andrew Potter

Joshua Wilson, Ph.D., is an Associate Professor at the University of Delaware School of Education. His research focuses on ways that automated writing evaluation can help to transform the teaching and learning of writing. A former special education teacher, Dr. Wilson is particularly concerned with improving writing outcomes for those most at risk of learning difficulties.