Relationships Among Item Cognitive Complexity, Contextual Demands, and Item Difficulty: Implications for Achievement-Level Descriptors

Abstract

A primary goal of standards-based statewide achievement tests is to classify students into achievement levels that enable valid inferences about student content area knowledge and skill. Explicating how knowledge and skills are expected to differ in complexity in achievement level descriptors, and how that complexity is related to empirical item difficulty—at the beginning of assessment design, rather than at the end—is critical to those inferences. The purpose of this study is to investigate whether sources of knowledge and skill descriptions that are commonly found in achievement-level descriptors relate to item complexity and difficulty. Implications of findings for using achievement-level descriptors as a method of interpreting test scores and for test design are discussed.