Formative writing assessment for change – introduction to the special issue

This current special issue centres on formative writing assessment with children in the elementary grades. Participants in the investigations included in this special issue represent a span from the very youngest students just learning to write to students in fifth and sixth grades who generally have overcome the barriers of knowing how to encode writing, but who face increased demands for producing discursive, audience adapted texts. As editors, we limited papers in the special issue to include studies conducted with students in this grade span because it has been under-researched compared to other grade spans. That these grades have received less attention does not reflect on the importance of early writing instruction; becoming a skilled writer takes time, and the first writing instruction is essential.

Becoming a good writer is the result of many complex interactions–including but not limited to–interactions between a writer’s attitude towards writing, her cognitive capacity, the kind of writing instruction she is exposed to, as well as the writer’s perception of textual norms in relation to the reader’s perception of the same norms, and thereby the reader’s textual expectations (Graham, 2018a; Rijlaarsdam et al., 2012; Skar & Aasen, 2021). To help children progress as writers, then, there is a need for tools that can elicit information about students’ writing skills in different domains (e.g. affective, cognitive, textual) and tools that help teachers transform that information into instruction. Such tools are often described as tools for formative assessment.

Formative writing assessment has proven to be effective in increasing the writing skills of students. A review by (Graham, 2018b) reported positive effect sizes for text response (d = 0.36), adult feedback (d = 0.87), peer feedback (0.58), self-feedback (d = 0.62) and computerised feedback (d = 0.38). An earlier study by Graham et al. (2011) reported an effect size of d = 1.01 for feedback from adults or peers. So, formative writing assessment can work, and it can lead to positive change. But what is it?

Graham (2018b, pp. 145–147) suggested the following definition of formative writing assessment: ‘instructional feedback in writing as information provided by another person, group of people, agency, machine, self, or experience that allows a writer, one learning to write, or a writing teacher/mentor to compare some aspect of performance to an expected, desired, or idealized performance’ and that ‘Formative feedback is derived from assessments that involve collecting information or evidence about student learning, interpreting it in terms of learners’ needs, and using it to alter what happens.’ In other words, formative writing assessment concerns taking actions based on information about a writer’s skills in order to make that writer even more skilled. One might therefore say that formative writing assessment – in the end – is all about consequences.

The consequential aspect of formative writing assessment calls for frameworks or heuristics that can help us evaluate formative assessment from a perspective of consequences. In a series of papers and books, Bachman and colleagues (Bachman, 2005; Bachman & Palmer, 2010; Bachman et al., in preparation) have described and finetuned such a framework. The Bachman and colleague’s framework postulates that the chief concern of (formative) assessment is ‘beneficial consequences’, which in our instance is equivalent with students becoming better writers, thanks to the actions triggered by assessment.

To achieve beneficial writing consequences, for example, increased learning or increased motivation for writing, assessments need to fulfil three criteria (e.g. Bachman & Palmer, 2010). First, beneficial consequences are the result of good decisions based on the information provided by the assessment. If the assessment indicates a student employs sub-optimal writing strategies, an instructional decision can be made to work to change those strategies. Likewise, if students’ motivation for writing is low, decisions that can increase motivation need to be enacted. Second, it is of importance that decisions are based on meaningful interpretations of the skills assessed. For an interpretation to be meaningful, the assessment instrument needs to be designed to elicit relevant and sufficient information in an impartial manner. Traditionally, this component has been called ‘construct validity’, i.e. the measure assesses what it is supposed to assess. This second criterion relates not only to construct relevance and coverage but also to generalizability; we need to be sure that our interpretations about a student’s standing hold true across the ‘universe of generalization’, which is ‘the subdomain for which it is plausible to consider the observed performances to be a random or representative sample’ (Kane et al., 1999, p. 8). Third, the ‘assessment outcome’ needs to be consistent, or reliable.

In this special issue, we have collected papers that deal with formative writing assessment for change, and papers that relate to various aspects of the framework described above. Common to all papers are pursuits of assessment tools that can yield meaningful interpretations, and lead to decisions, and ultimately beneficial consequences for students. In the sections that follow, we describe the papers included in this issue.

Papers in this issue

The papers in the current special issue cover four themes: Scoring Writing, Predicting Writing Performance, Student Involvement in Formative Assessment, and Assessing Writing Motives and Engagement.

Scoring writing

Four papers focus on the scoring of writing and positive consequences that occurred. Kennedy and Shiel (this issue) present an investigation into the development and implementation of a writing assessment rubric targeting K–2 writing. The investigation indicated that the rubric could be used to reliably assess writing in grades K–2, and that ‘the rubric itself acted as a catalyst for teacher reflection […] prompting teachers to reconsider their perceptions of what writing could and should be for young writers’ (p. XX). The Kennedy and Shiel assessment rubric is a welcomed addition to the growing body of similar assessment rubrics (e.g. Skar et al., 2020), offering teachers tools for assessing and adapting instruction.

McNamara and Kendeou (this issue) present a framework for “early Automated

Writing Evaluation (e-AWE)”, which is a much-needed outline of requirements of automated feedback systems to help K–5 students to develop writing skills. Personal computers have been around for quite a while, and some of the promises of automated feedback have been instantiated in some applications, but as the authors point out further development is needed. McNamara and Kendeou conclude that automated feedback tools of the future need to support input using text-to-speech synthesis and automated speech recognition, integrate writing with reading tasks, enable students to undertake simple text-based writing tasks, and include a teacher interface that allows for progress monitoring and data-based decision-making.

Wilson et al. (this issue) investigated a year-long implementation of an AWE system, ‘MI Write’, in one school in the United States. They found that teachers in grades 3–5 were generally enthusiastic about MI Writing, but despite this enthusiasm, they were not utilising it to a large degree. Wilson and colleagues also found that the use of MI Write in general was not associated with writing skill development. The authors offer several suggestions as to why this might be the case, including the limited implementation of MI Write. This paper thus highlights the important fact that access to formative writing assessment tools in itself neither guarantees use of those tools nor writing growth.

Matta et al. (this issue) also studied the use of automated scoring, but in the context of Written expression curriculum-based measurement (WE-CBM). WE-CBM, which is probably better known in the United States than outside, collects short samples of students’ writing (3–5 minutes) and provides assessments of these texts with different levels of sophistication. This includes, for example, the total number of words or the number of instances of correct pairs of words. In this longitudinal study, Matta and colleagues undertook several investigations into the usefulness of WE-CMB, including the number of student texts needed to secure valid interpretations of students’ writing skills (more was better), if automated scoring was better than human scoring (or ‘hand-calculated scoring’; it was not) to what extent WE-CBM could predict scores on a standardised writing test (predictions were inaccurate), and if WE-CBM was associated with predictive or diagnostic bias (it was not).

Predicting writing performance

Skar and Huebner (this issue) studied to what extent writing performance was predictable using the same prompts at the start and at the end of first grade. Specifically, they investigated whether student performances at the end of first grade that were indicative of students running the risk of falling behind could be predicted at the start of first grade. To a large extent this was found to be true, meaning that those early measures of writing performance could have served as a basis for remedial action. However, the investigation also found a substantial number of false positives, indicating a need to employ several measures before taking such action.

Student involvement in formative assessment

Linda Allal (2021) prepared a paper which was accepted for this special issue, but it was mistakenly printed in an earlier issue (Allal, 2021). Her study presents an overview of research on formative assessment that focuses on involving primary school students in the co-construction of such assessments. Allal then drew on an earlier case study to exemplify how a teacher can include students in the making of both tasks and assessments. The case becomes an inspiring exemplar of how formative writing assessment, with student involvement, can be operationalised.

Assessing writing motives and engagement

Two papers report findings from investigation into development and use of scales for assessing writing motives and engagement. Graham et al. (this issue) presents an investigation of the validation of a survey instrument (the Writing Motivation Questionnaire, WMQ) for measuring students’ motives for writing. The researchers sampled the writing motives of 2,193 students in grades four and five, used the motive scores as predictors of performances on writing tests, and inquired into students’ perception of the usefulness of reflecting over motives for writing. The WMQ included items on intrinsic (curiosity and involvement), extrinsic (social recognition, grades, and competition), and self-regulatory (reduce boredom and emotional regulation) motives for writing. Students generally perceived the self-reflection practice as useful, and the researchers found that subscale scores from the WMQ could predict performance on standardised test, with for example, curiosity for writing showing a positive association with writing achievement.

Rogers et al. presents an investigation into the development of the Writing Engagement Scale (WES). The researchers sampled students from grades 3–5, and developed a tool with 12 items tapping into learner’s affective, behavioural, cognitive and social engagement. The WES was developed by researchers and teachers to quickly (5 minutes) yield insights into students’ engagement. Teachers of the students sampled found the WES scores to be informative and useful for making written instructional decisions. In an explorative part of the validation study, the researchers correlated the WES scores to writing performance scores for a sub-sample (n = 15) of participating students and found indication of positive relationships between cognitive engagement and the writing outcome.

Closing words

The papers in this special issue cover multiple aspects of writing skills. This includes, for example, the stability of writing performance and motivation for writing. The included articles further cover several modes of formative assessment – self assessment, automated assessment, to name two–noted by Graham (2018b), as well as to the core purpose of formative assessment, namely providing information that can be used for positive change. The papers also associated with criteria in the Bachman framework (Bachman & Palmer, 2010). They demonstrated relentless efforts into making assessments consistent, meaningful and actionable. The common thread in the investigations presented here is the intent to develop tools that can yield positive consequences, resulting in more skilled, motivated, and engaged young writers.

When the first author underwent teacher education in the early 2000s, writing assessment was conveyed as giving feedback to text. A model that was popular in Scandinavia (Hoel, 2001) postulated that this feedback could be provided on a ‘local level’ (e.g. mechanics) or on a ‘global level’ (e.g. if the text was a good instantiation of the genre it was attempting at). Essentially Hoel (2001) argued that good feedback would go beyond the surface level. The model extended writing assessment literature highlighting the what of text (e.g. Connors & Lunsford, 1993; Sommers, 1982), which was a reasonable theme given the power of text feedback to shape a writer’s understanding of what good writing is. We know, however, that learning to write requires far more than receiving response to texts that are already produced. While it is obviously important to not convey the message that writing skills equal spelling, there is a lot more ‘actionable’ information from students’ writing than just the text, and the papers in this special issue demonstrate just that. The papers in this special issue also show that more research is sorely needed. The promises of new technology, as suggested in the framework by McNamara and Kendeou, stand in contrast to its implementation. And all papers in this special issue point to important limitations in the research presented, including that the scales, rubrics, and other measures that have been constructed to this point need to be applied on a large scale. However, this issue also highlights that formative writing assessment in the elementary grades is a vivid field of research, which, if implemented, will have positive consequences for children around the globe as they learn to write.