Abstract
The construct of reading comprehension has changed significantly in the twenty-first century; however, some test designs have not evolved sufficiently to capture these changes. Specifically, the nature of literacy sources and skills required has changed (wrought primarily by widespread use of digital technologies). Modern theories of comprehension and discourse processes have been developed to accommodate these changes, and the learning sciences have followed suit. These influences have significant implications for how we think about the development of comprehension proficiency across grades. In this paper, we describe a theoretically driven, developmentally sensitive assessment system based on a scenario-based assessment paradigm, and present evidence for its feasibility and psychometric soundness.
ACKNOWLEDGEMENTS
We are grateful to the Institute of Education Sciences and Educational Testing Service for sponsoring and supporting this research. We would like to also like to thank Michael Kane and Robert Mislevy for their intellectual insights and thoughtful comments; Andre Rupp, Paul Deane and Heather Buzick for their review comments; Kelsey Dreier and Kim Fryer for their editorial assistance. We also wish to express our gratitude to the all of our RfU and Cognitively Based Assessment as, of, & for, Learning (CBALTM) colleagues, who have been and are contributing to this ongoing enterprise.
Notes
1 The CBAL initiative and Institute for Education Sciences grant had overlapping aims. It is beyond the scope of this article to review relevant CBAL publications. The interested reader is referred to the ETS website (https://www.ets.org/research/topics/) for a more complete bibliography.
2 Linguistic complexity is used here to refer to the linguistic demands such as text cohesion, syntactic complexity, and the level of vocabulary sophistication that may impact a reader’s ability to from a coherent model of the text (see McNamara, Graesser, & Louwerse, Citation2012). Linguistic complexity may contribute to item difficulty independent of the task and content demands.
3 Reading comprehension is not directly observable, but the quality of a student’s mental model can be inferred from the evidence gathered through a sample items and tasks.