Evaluating the Instructional Sensitivity of Four States' Student Achievement Tests

Abstract

As state tests of student achievement are used for an increasingly wide array of high- and low-stakes purposes, evaluating their instructional sensitivity is essential. This article uses data from the Bill and Melinda Gates Foundation's Measures of Effective Project to examine the instructional sensitivity of 4 states' mathematics and English language arts assessments to 5 measures of pedagogical quality. The results indicate modest overall levels of sensitivity, with some state assessments insensitive to most or all measures of pedagogical quality. Furthermore, there are sometimes substantial differences across states in the sensitivity of their assessments. The article calls for more research on what makes state tests instructionally sensitive and cautions against the use of insensitive assessments for consequential decisions.

Acknowledgments

The generous support of the Bill and Melinda Gates Foundation is gratefully acknowledged. All errors are the author's.

Notes

¹ Due to confidentiality, each district was assigned a random number (1–6). These numbers are used for the remainder of the article. Therefore, all analyses are presented without district names.