Science Assessments and English Language Learners: Validity Evidence Based on Response Processes

Abstract

English language learners (ELLs) and their teachers, schools, and communities face increasingly high-stakes consequences due to test score gaps between ELLs and non-ELLs. It is essential that the field of educational assessment continue to investigate the meaning of these test score gaps. This article discusses the findings of an exploratory study of the response processes of grade 5 ELLs and non-ELLs on multiple-choice science test items from a high-stakes test. We found that the ELL students in our sample were more likely than the non-ELL students to answer incorrectly despite demonstrating knowledge of the science content targeted by the test items. Investigating the interview transcripts of ELL students who answered in this way revealed that ELL students’ interactions with specific linguistic features of test items often led to alternative interpretations of the items that resulted in incorrect answers. The implications of this work for the assessment of ELLs in science are discussed.

ACKNOWLEDGMENTS

The authors thank Josiane Hudicourt-Barnes, Rachel Kachchaf, and Curtis Killian of TERC; Christopher Wright of the University of Tennessee; Michael Russell of Measured Progress; and Rachel Kay of the Concord Consortium for their contributions to this work. In addition, the authors thank the editors of this special issue for their essential feedback on earlier versions of this article. The authors also thank the Massachusetts Department of Elementary and Secondary Education for making test items and performance data available so that research like ours can take place. We also thank Catherine Bowler, of the Massachusetts Department of Elementary and Secondary Education, whose interest in our work and willingness to help us to understand the MCAS has greatly facilitated this research. Finally, the authors thank all the students, parents, teachers, and administrators who made this work possible. This article is dedicated to them.