Extending Principles of Evidence-Centered Design for Diverse Populations: K–12 English Learners with the Most Significant Cognitive Disabilities

ABSTRACT

Designing an effective K–12 alternate English language proficiency assessment (ELP) for English learners with significant cognitive disabilities (ELSCDs) is a substantial undertaking, particularly given the absence of design models for such an assessment and limited empirical research to guide test design teams. These challenges emphasized the need to follow a rigorous and principled approach in order to establish an appropriate design. This paper describes how Evidence-Centered Design (ECD) – a comprehensive and flexible assessment design framework – was applied throughout the design process of such an assessment. By extending the principles of ECD, a maximally accessible alternate ELP assessment was designed and implemented, and evidence was collected to support validity. Implications for this assessment design effort include findings for the education and assessment of ELSCDs, the application of ECD to the assessment of diverse test takers, and to a general movement in the field toward more personalized assessments.

Acknowledgments

The authors wish to express sincere gratitude to the editor and reviewers at Educational Assessment for their valuable comments. The authors appreciate Heather Buzick, Cara Laitusis, and Caroline Wylie for reviewing previous versions of this paper. Any opinions expressed in this publication are those of the authors and not necessarily of Educational Testing Service or the California Department of Education.

Disclosure statement

We have no known conflicts of interest to disclose.

Notes

¹ We acknowledge that there is some debate regarding the use of an initialism, such as ELSCD, to refer to a group of students with some shared characteristics, such as English learners with significant cognitive disabilities. While the full form clearly indicates to readers (in person-first language) who is being discussed, we have chosen to use the initialism, ELSCD, in this paper because it allows for greater reading ease.

² The design process described in this paper was executed with meaningful consultation and support from a range of interest holders, including: a panel of nationally recognized experts convened specifically to support this assessment design effort, the Test Design Advisory Team; the state’s assessment Technical Advisory Group; panels of educators of ELSCDs, SSCDs, and ELs; a range of educational advocacy groups and, last but most importantly, K-12 ELSCDs themselves.

³ There is no federal definition that identifies students with the most significant cognitive disabilities and few states have criteria specific to ELs defining linguistic and sociocultural factors (Liu et al., 2021; M. L. Thurlow et al., 2017).

⁴ ESSA regulations have established that students identified with the most significant cognitive disabilities may not exceed 1% of a state’s population of students participating in large-scale assessments (ESSA, 2015; M. L. Thurlow et al., 2017); ELSCDs are, in turn, a small subset of the one percent population eligible for alternate assessment.

⁵ The abbreviation of Educational Testing Service, as ETS is used throughout the paper.

⁶ Important resources include the Council of Chief State School Officers’ (CCSSO) English Language Proficiency Standards for English Learners with Significant Cognitive Disabilities (CCSSO ELP Standards for ELSCDs; 2019), which are alternate ELP standards linked to the general CCSSO ELP Standards (2012), the ALTELLA Project (Christensen, Gholson, et al., 2018), Theory of Action (M. L. Gholson & Guzman-Orth, 2019), and the CA ELA/ELD Framework (California Department of Education, 2014).,

⁷ The ELD connectors were developed in 2019 (CDE, 2021).

⁸ The students background information questionnaire (BIQ) was designed from existing research and surveys on learners with significant cognitive disabilities and English language proficiency.

⁹ For all educator ratings (listening, speaking, reading, and writing) see ETS (2023).