Extending Principles of Evidence-Centered Design for Diverse Populations: K–12 English Learners with the Most Significant Cognitive Disabilities

References

• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for Educational and Psychological Testing. Washington, D.C.: American Educational Research Association 978-0-935302-25-6.
   Google Scholar
• American Speech-Language Hearing Association. (2019). Final report: Ad hoc committee on language proficiency. https://www.asha.org/siteassets/reports/ahc-language-proficiency.pdf
   Google Scholar
• Biggs, E. E., Carter, E. W., & Gilson, C. B. (2018). Systematic review of interventions involving aided AAC modeling for children with complex communication needs. American Journal on Intellectual and Developmental Disabilities, 123(5), 443–473. https://doi.org/10.1352/1944-7558-123.5.443
   PubMedGoogle Scholar
• Buzick, H. M., Casabianca, J. M., & Gholson, M. L. (2023). Personalizing large‐scale assessment in practice. Educational Measurement Issues & Practice, 42(2), 5–11. https://doi.org/10.1111/emip.12551
   Google Scholar
• California Department of Education. (2014). California English Language Development Standards. https://www.cde.ca.gov/sp/el/er/documents/eldstndspublication14.pdf
   Google Scholar
• California Department of Education. (2015). California alternate assessments of English language arts, math, and science: Item templates [unpublished manuscript].
   Google Scholar
• California Department of Education. (2021). English Language Development Connectors for the Alternate English Language proficiency assessments for California. https://www.cde.ca.gov/ta/tg/ep/documents/eldconnectorsaltelpac.docx
   Google Scholar
• Center for Literacy and Disability Studies. (2015). Project core. University of North Carolina at Chapel Hill. http://www.project-core.com/about-project-core/
   Google Scholar
• Christensen, L. L., Gholson, M. L., & Shyyan, V. V. (2018). Establishing a definition of English learners with significant cognitive disabilities (ALTELLA brief No. 1). University of Wisconsin–Madison, Wisconsin Center for Education Research, Alternate English Language Learning Assessment Project. https://altella.wceruw.org/pubs/ALTELLA_Brief-01_Definition_070218.pdf
   Google Scholar
• Christensen, L. L., & Mitchell, J. D. (2018, September). Classroom perspectives on English learners with significant cognitive disabilities. University of Wisconsin– Madison, Alternate English Language Learning Assessment. Retrieved from University of Wisconsin–Madison, Wisconsin Center for Education Research. https://altella.wceruw.org/pubs/Classroom-Perspectives-Report.pdf
   Google Scholar
• Christensen, L. L., Mitchell, J. D., Shyyan, V. V., & Ryan, S. (2018, September). Characteristics of English learners with significant cognitive disabilities: Findings from the individual characteristics questionnaire. University of Wisconsin–Madison, Alternate English Language Learning Assessment. University of Wisconsin–Madison, Wisconsin Center for Education Research. https://altella.wceruw.org/pubs/ICQ-Report.pdf
   Google Scholar
• Clark, A. K., Gholson, M. L., Christmus, J., Winter, P., & Christensen, L. (2018). What do we know about students with significant cognitive disabilities who are English learners? [conference presentation] National Conference on Student Assessment, San Diego, CA, June 28, 2018. http://altella.wceruw.org/pubs/WhatDoWeKnow_NCSA2018.pdf
   Google Scholar
• Clark, A. K., & Karvonen, M. (2020, June 1). Use of evidence-centered design to develop learning maps-based assessments. https://doi.org/10.35542/osf.io/wqn69
   Google Scholar
• Council of Chief State School Officers. (2012). Framework for English language proficiency development standards to the common core state standards and the next generation science standards. https://www.ccsso.org/sites/default/files/2017-11/ELPD%20Framework%20Booklet-Final%20for%20web.pdf
   Google Scholar
• Council of Chief State School Officers. (2019). English language proficiency standards for English learners with significant cognitive disabilities. https://ccsso.org/sites/default/files/2019-02/K-12%20ELP%20Standards%20for%20ELs%20with%20significant%20cognitive%20disabilities%202019%2002%2015%20.pdf
   Google Scholar
• Educational Testing Service. (2019a). Definitions of task types for the alternate English language proficiency assessments for California. California Department of Education. https://www.cde.ca.gov/ta/tg/ep/documents/altelpactasktypes.pdf
   Google Scholar
• Educational Testing Service. (2019b). Proposed high-level test design for the alternate English Language proficiency assessments for California. California Department of Education. https://www.cde.ca.gov/ta/tg/ep/documents/proposedhltdaltelpac.pdf
   Google Scholar
• Educational Testing Service. (2020a). Alternate English language proficiency assessments for California test blueprint. California Department of Education. https://www.cde.ca.gov/ta/tg/ep/documents/altelpacblueprint.pdf
   Google Scholar
• Educational Testing Service. (2020b). Final report for the alternate English language proficiency assessments for California Pilot using cognitive lab methodology study. California Department of Education. https://www.cde.ca.gov/be/pn/im/documents/dec20adad01a01.docx
   Google Scholar
• Educational Testing Service. (2022). Alternate ELPAC survey results: An interpretive report for educators. California Department of Education. https://www.cde.ca.gov/ta/tg/ep/documents/altelpacsurveyrpt.pdf
   Google Scholar
• Educational Testing Service. (2023). Alternate English language proficiency assessment for California 2021–2022 operational field test technical report. California Department of Education. https://www.cde.ca.gov/ta/tg/ep/documents/sumaltelpactechrpt22.docx
   Google Scholar
• English Language Proficiency Assessments for California. (2021). Alternate ELPAC Practice Tests. California Department of Education. https://www.elpac.org/resources/practicetests/#alternate
   Google Scholar
• Erickson, K. A., & Geist, L. A. (2016). The profiles of students with significant cognitive disabilities and complex communication needs. Augmentative and Alternative Communication, 32(3), 187–197. https://doi.org/10.1080/07434618.2016.1213312
   PubMed Web of Science ®Google Scholar
• Every Student Succeeds Act of 2015. (2015). Pub L No 114‐95 § 114 Stat 1177. https://www.congress.gov/114/plaws/publ95/PLAW-114publ95.pdf
   Google Scholar
• Gholson, M. L. (2018). English learners with significant cognitive disabilities: Observations and interviews. CCSSO Students with Disabilities Advisory Task Force Meeting, January 31 Dallas, TX.
   Google Scholar
• Gholson, M. L., & Guzman-Orth, D. (2019). Developing an alternate English language proficiency assessment (AELPA) system: A theory of action. ETS research report series 2019, 1–19. https://doi.org/10.1002/ets2.12262
   Google Scholar
• Gholson, M., Steinberg, J., & Albee, T. (2021, June 9). Methods for improving validity for cognitive labs using purposeful sampling procedures [conference presentation]. Annual Meeting for the National Council for Measurement in Education: Focus on Students with Disabilities, Virtual Meeting, June 8-11, 2021, Baltimore, Maryland.
   Google Scholar
• Houwen, S., Visser, L., van der Putten, A., & Vlaskamp, C. (2016). The interrelationships between motor, cognitive, and language development in children with and without intellectual and developmental disabilities. Research in Developmental Disabilities, 53, 19–31. https://doi.org/10.1016/j.ridd.2016.01.012
   PubMed Web of Science ®Google Scholar
• Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed. pp. 17–64). American Council on Education/Praeger.
   Google Scholar
• Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000
   Web of Science ®Google Scholar
• Karvonen, M., & Clark, A. K. (2019). Students with the most significant cognitive disabilities who are also English learners. Research and Practice for Persons with Severe Disabilities, 44(2), 71–86. https://doi.org/10.1177/1540796919835169
   Web of Science ®Google Scholar
• Karvonen, M., Clark, A. K., Carlson, C., Wells-Moreaux, S., & Burnes, J. (2021). Approaches to identification and instruction for students with significant cognitive disabilities who are English learners. Research and Practice for Persons with Severe Disabilities, 46(4), 223–239. https://doi.org/10.1177/15407969211040256
   Web of Science ®Google Scholar
• Kearns, J., Kleinert, H., Kleinert, J., & Towles-Reeves, E. (2006). Learner characteristics inventory. University of Kentucky, National Alternate Assessment Center.
   Google Scholar
• Light, J., & McNaughton, D. (2014). Communicative competence for individuals who require augmentative and alternative communication: A new definition for a new era of communication? Augmentative and Alternative Communication, 30(1), 1–18. https://doi.org/10.3109/07434618.2014.885080
   PubMed Web of Science ®Google Scholar
• Light, J., & McNaughton, D. (2015). Designing AAC research and intervention to improve outcomes for individuals with complex communication needs. Augmentative and Alternative Communication, 31(2), 85–96. https://doi.org/10.3109/07434618.2015.1036458
   PubMed Web of Science ®Google Scholar
• Liu, K. K., Thurlow, M. L., & Dosedel, M. J. (2020). A literature review of evidence-based literacy assessment and instruction practices for English learners with significant cognitive disabilities (NCEO report 422). National Center on Educational Outcomes. https://files.eric.ed.gov/fulltext/ED609750.pdf
   Google Scholar
• Liu, K. K., Wolforth, S., Thurlow, M. L., Jacques, C., Lazarus, S. S., & August, D. (2021). A framework for making decisions about participation in a State’s alternate ELP assessment (NCEO report 426). National Center on Educational Outcomes. https://files.eric.ed.gov/fulltext/ED616058.pdf
   Google Scholar
• Marion, S. F., & Pellegrino, J. W. (2007). A validity framework for evaluating the technical quality of alternate assessments. Educational Measurement Issues & Practice, 25(4), 47–57. https://doi.org/10.1111/j.1745-3992.2006.00078.x
   Google Scholar
• Mislevy, R. J. (2018). Sociocognitive foundations of educational measurement. Routledge.
   Google Scholar
• Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence-centered design. (Research report 03-16). Educational Testing Service. https://doi.org/10.1002/j.2333-8504.2003.tb01908.x
   Google Scholar
• Mislevy, R. J., Behrens, J. T., Dicerbo, K. E., & Levy, R. (2012). Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining. Journal of Educational Data Mining, 4(1), 11–48. https://doi.org/10.5281/zenodo.3554641
   Google Scholar
• Mislevy, R. J., & Haertel, G. D. (2006). Implications of evidence-centered design for educational testing. Educational Measurement Issues & Practice, 25(4), 6–20. https://doi.org/10.1111/j.1745-3992.2006.00075.x
   Google Scholar
• Mislevy, R. J., Haertel, G., Cheng, B. H., Ructtinger, L., DeBarger, A., Murray, E., Rose, D., Gravel, J., Colker, A. M., Rutstein, D., & Vendlinski, T. (2013). A “conditional” sense of fairness in assessment. Educational Research & Evaluation, 19(2–3), 121–140. https://doi.org/10.1080/13803611.2013.767614
   Google Scholar
• Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessment. Measurement: Interdisciplinary Research & Perspective, 1(1), 3–62. https://doi.org/10.1207/S15366359MEA0101_02
   Google Scholar
• Nash, B., Clark, A. K., & Karvonen, M. (2015). First contact: A census report on the characteristics of students eligible to take alternate assessments (Technical Report No. 16-01). University of Kansas, Center for Educational Testing and Evaluation. https://dynamiclearningmaps.org/sites/default/files/documents/publication/First_Contact_Census_2016.pdf
   Google Scholar
• Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 42(5), 533–544. https://doi.org/10.1007/s10488-013-0528-y
   PubMed Web of Science ®Google Scholar
• Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment. National Academy Press. https://doi.org/10.17226/10019
   Google Scholar
• Rowland, C., & Fried-Oken, M. (2010). Communication matrix: A clinical research assessment tool targeting children with severe communication disorders. Journal of Pediatric Rehabilitation Medicine: An Interdisciplinary Approach, 3(4), 319–329. https://doi.org/10.3233/prm-2010-0144
   PubMedGoogle Scholar
• Sánchez, M. T., Parker, C., Akbayin, B., & McTigue, A. (2010). Processes and challenges in identifying learning disabilities among students who are English language learners in three New York State districts (Issues & answers report, REL 2010–No. 085). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Northeast, and Islands. https://ies.ed.gov/ncee/edlabs/regions/northeast/pdf/REL_2010085.pdf
   Google Scholar
• Shyyan, V. V., Gholson, M. L., & Christensen, L. L. (2018). Considerations for educators serving English learners with significant cognitive disabilities (ALTELLA brief No. 2). University of Wisconsin–Madison, Wisconsin Center for Education Research. https://altella.wceruw.org/pubs/ALTELLA_Brief%2002_Considerations.pdf
   Google Scholar
• Sireci, S. G. (2020). Standardization and UNDERSTAND ardization in educational assessment. Educational Measurement Issues & Practice, 39(3), 100–105. https://doi.org/10.1111/emip.12377
   Google Scholar
• Thurlow, M., Christensen, L., & Shyyan, V. (2016). White paper on English language learners with significant cognitive disabilities. University of Minnesota, National Center on Educational Outcomes, English Language Proficiency Assessment for the 21st Century. https://nceo.umn.edu/docs/OnlinePubs/AAATMTWhite%20Paper.pdf
   Google Scholar
• Thurlow, M. L., Lazarus, S. S., Larson, E. D., Albus, D. A., Liu, K. K., & Kwong, E. (2017). Alternate assessments for students with significant cognitive disabilities: Participation guidelines and definitions (NCEO Report 406). University of Minnesota, National Center on Educational Outcomes https://nceo.umn.edu/docs/OnlinePubs/NCEOReport406.pdf
   Google Scholar
• Towles-Reeves, E., Kearns, J., Kleinert, H., & Kleinert, J. (2009). An analysis of the learning characteristics of students taking alternate assessments based on alternate achievement standards. The Journal of Special Education, 42(4), 241–254. https://doi.org/10.1177/0022466907313451
   Web of Science ®Google Scholar
• U.S. Department of Education. (2018). A state’s Guide to the U.S. Department of Education’s assessment peer review process. https://oese.ed.gov/files/2020/07/assessmentpeerreview.pdf
   Google Scholar
• U.S. Department of Education, Office of English Language Acquisition. (2017). English learner toolkit (2nd Rev. ed.). https://ncela.ed.gov/files/english_learner_toolkit/OELA_2017_ELsToolkit_508C.pdf
   Google Scholar
• Winter, P. C., Karvonen, M., & Christensen, L. L. (2018, August). Developing Item Templates for Alternate Assessments of English Language Proficiency. University of Wisconsin–Madison, Alternate English Language Learning Assessment (ALTELLA). Retrieved from University of Wisconsin–Madison, Wisconsin Center for Education Research: ALTELLA_Report_Item_083108.pdf (wceruw.org)
   Google Scholar
• Wolf, M. K., Everson, P., Lopez, A., Hauck, M., Pooler, E., & Wang, J. (2014). Building a Framework for a Next‐Generation English Language Proficiency Assessment System. (ETS Research Report Series No. RR-14-34). Educational Testing Service. https://doi.org/10.1002/ets2.12034
   Google Scholar
• Yorke, A. M., Light, J. C., Gosnell Caron, J., McNaughton, D. B., & Drager, K. D. (2018). The effects of explicit instruction in academic vocabulary during shared book reading on the receptive vocabulary of children with complex communication needs. Augmentative and Alternative Communication, 34(4), 288–300. https://doi.org/10.1080/07434618.2018.1506823
   PubMed Web of Science ®Google Scholar
• Zieky, M. J. (2014). An introduction to the use of evidence-centered design in test development. Psicología Educativa, 20(2), 79–87. https://doi.org/10.1016/j.pse.2014.11.003
   Google Scholar