A situative response to the conundrum of formative assessment

Abstract

While formative assessment is popular, it is difficult to evaluate and improve. In some settings, it may actually reduce disciplinary learning by competing with other more productive activities, making those activities less engaging, and narrowing curricular goals. Situative approaches to educational assessment offer a solution by (a) blurring the distinction between instruction and assessment, (b) moving beyond the intended purposes of assessment to focus on actual functions and (c) using the same assessment to accomplish multiple functions. Framing instruction, assessment and testing as primarily social practices and placing them on a continuum of assessment formality offers a coherent framework for aligning learning across different assessments and balancing functions within particular assessments. This paper introduces an approach called Participatory Assessment that has been used successfully to enhance (a) communal engagement, (b) individual knowledge and (c) aggregated achievement of standards, while (d) providing valid evidence of those refinements.

Keywords:

• assessment
• situativity
• formative assessment
• validity
• construct-irrelevant variance

Acknowledgements

Lorrie Shepard, Drew Gitomer, Phillip Piety, Patricia Murphy, James Willis and two anonymous reviewers provided invaluable feedback on earlier versions of this manuscript. Thanks also to Rebecca Itow, Sarah Mosier, Tara Kelly, Christine Chow, Katerina Schenke, Sam Stalion and Suraj Uttamchandani for feedback and proofreading versions of this manuscript. Many of the ideas here emerged while the author was supported by the MacArthur Foundation’s Twenty-First Century Assessment Project which was directed by James Gee, and included Pam Moss, Daniel Schwartz, Valerie Shute, Bob Mislevy and others. The author also wishes to acknowledge the specific contributions to these ideas from some of the many collaborators on these projects, particularly Ann Kindfield, Ed Wolfe, Nancy Schafer, Steven McGee, Steven Zuiker, Melissa Gresalfi, Sasha Barab, Michelle Honeyford, Henry Jenkins, Andrea Rehak and Rebecca Itow.

Notes

1. Construct-irrelevant easiness occurs when formative feedback and curricular revisions undermine the validity of scores on subsequent summative assessments. Most educational assessments are ‘indirect’ measures of some underlying proficiency. When this is the case, formative feedback on practice problems introduces some degree of construct-irrelevant easiness on subsequent similar problems. This means that some of the improvement results from learning how to solve the particular problems that the assessment uses to represent the domain, rather than learning the underlying knowledge, needed to solve the broader class of problems. This tends to exaggerate the impact of formative assessment in ways that are practically impossible to disentangle from the ‘construct-relevant’ learning that can be expected to transfer to new problems and new contexts.

2. This melding of curricular practices and theoretical advancement illustrates how knowledge of both advances side-by-side in design-based research.

3. Rebecca Rubert had completed advanced graduate-level coursework in English before embarking on a teaching career. She spent most of her career in the alternative school for students who had not succeeded in traditional schools where this research was carried out.

4. Suraj Uttamchandani helped derive these specific practices and uncovered this theoretical insight.

5. This is because that summative evaluation introduces construct-irrelevant easiness by calling for detailed guidelines and individualised formative feedback. This leads to what Bloome, Puro, and Theodorou (1989) labelled procedural display.

6. This qualification is a tentative one and this issue deserves focused theoretical and empirical consideration. Theoretically, the issue is that transitioning between the more concrete personalised informal assessment context and the more abstract semi-formal setting should reinforce both forms of knowledge. Even if the abstraction is indeed a special case of the more concrete cultural experiences, being able to represent one’s knowledge in this fashion is useful. Empirically, Goldstone and Son’s (2005) studies of ‘concreteness fading’ showed the greatest transfer of knowledge from simulations that went from concrete to abstract (progressive idealisation). However, the issue is quite challenging because any such experiments suffer from the same validity issues that motivate this paper. In most settings, it seems that the level of formative feedback needed to justify marks/grades to students is probably about as much learning as is likely to be gained in most settings from formative feedback on semi-formal (graded and open-ended) classroom assessments.

7. It seems that if these assessments reveal that individuals and/or topics are in need of remediation, that remediation likely should take place around the original personalised and meaningful contexts, rather than around the relatively abstract assessment context. While this is a very important question, little individual remediation has been carried out in the efforts so far, and the related issues have not been explored.

8. Arguably, construct-irrelevant easiness can be controlled for when using open-ended inquiry-oriented assessments as both formative and summative assessments, providing that careful attention is paid to varying the kinds of problems presented across the two. This principle assumes that such ‘interpretive’ responses to construct-irrelevant easiness are far more difficult than the ‘empirical’ response proposed here because the interpretive response requires extensive knowledge of both the discipline and assessment and careful implementation.

Additional information

Funding

This article was written while the author was supported by a grant from the MacArthur Foundation. The various projects mentioned in this paper were supported by the MacArthur Foundation, the US National Science Foundation, the US Department of Education, NASA, and Indiana University. This support is gratefully acknowledged.