Claims, Evidence, and Achievement-Level Descriptors as a Foundation for Item Design and Test Specifications

Abstract

Evidence-centered assessment design (ECD) explicates a transparent evidentiary argument to warrant the inferences we make from student test performance. This article describes how the vehicles for gathering student evidence—task models and test specifications—are developed. Task models, which are the basis for item development, flow directly from the claims we want to make about students and the observable evidence required to warrant those claims. The claims and evidence, and therefore, the task models, are directly connected to the underlying construct as defined by the achievement-level descriptors (ALDs). Test specifications must balance the claims and evidence that distinguish students at each achievement level with the psychometric characteristics necessary for reliability and comparability. The explicit relationships among claims, evidence, task models, items, achievement levels, and test specifications provide an evidentiary argument for making valid inferences about student test performance. The iterative nature, challenges, and resulting benefits of this evidence-centered approach to item design and test specifications are discussed.

Notes

¹The terms task and item are considered synonymous.

²All examples from the domain analyses and domain models are considered draft as of the time of this writing. The College Board is still in the iterative development process.

³The Standards for Foreign Language Learning in the 21st Century (National Foreign Language Standards Collaborative Project, 1999), the ACTFL Performance Guidelines for K–12 Learners (American Council on Teaching Foreign Language, 1998) and a survey of college and university practices were used as reference.

⁴All claims, evidence, content, and skills used in this article are considered draft and are used here for illustrative purposes only.