Raising the Bar: How Revising an English Language Proficiency Assessment for Initial English Learner Classification Affects Students’ Later Academic Achievements

References

• Abedi, J. (2007). English language proficiency assessment in the nation: Current status and future practice. University of California.
   Google Scholar
• Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444–455. https://doi.org/10.1080/01621459.1996.10476902
   Web of Science ®Google Scholar
• Angrist, J. D., & Pischeke, J. (2009). Mostly harmless econometrics: An empiricist’s companion. Princeton University Press.
   Google Scholar
• Anstrom, K., DiCerbo, P., Butler, F. A., Katz, A., Millet, J., & Rivera, C. (2010). A review of the literature on academic English: Implications for K–12 English language learners. George Washington University Center for Equity and Excellence in Education.
   Google Scholar
• Bailey, A. L., & Butler, F. A. (2003). An evidentiary framework for operationalizing academic language for broad application to K–12 education: A design document (CSE tech. Report No. 611). University of California, National Center for Research on Evaluation, Standards, and Student Testing.
   Google Scholar
• Bailey, A. L., Butler, F. A., Stevens, R., & Lord, C. (2007). Further specifying the language demands of school. In A. L. Bailey (Ed.), Language demands of school: Putting academic language to the test (pp. 103–156). Yale University Press.
   Google Scholar
• Bailey, A. L., & Carroll, P. E. (2015). Assessment of English language learners in the era of new academic content standards. Review of Research in Education, 39(1), 253–294. https://doi.org/10.3102/0091732X14556074
   Web of Science ®Google Scholar
• Bailey, A. L., & Heritage, M. (2010). English language proficiency assessment foundations: External judgments of adequacy. Evaluating the validity of English language proficiency assessments. An Enhanced Assessment Grant).
   Google Scholar
• Bailey, A. L., & Huang, B. H. (2011). Do current English language development/proficiency standards reflect the English needed for success in school? Language Testing, 28(3), 343–365. https://doi.org/10.1177/0265532211404187
   Web of Science ®Google Scholar
• Bailey, A. L., & Kelly, K. R. (2013). Home language survey practices in the initial identification of English learners in the United States. Educational Policy, 27(5), 770–804. https://doi.org/10.1177/0895904811432137
   Web of Science ®Google Scholar
• Baron, P., Linquanti, R., & Huang, M. (2020). Validating threshold scores for English language proficiency assessment uses. In M. K. Wolf (Ed.), Assessing English language proficiency in U.S. K-12 schools (pp. 161–184). Routledge.
   Google Scholar
• Bedolla, G. L., & Rodriguez, R. (2011). Classifying California’s English learners: Is the CELDT too blunt an instrument?. UC Berkeley Center for Latino Policy Research. http://www.escholarship.org/uc/item/2m74v93d?query=lisagarcia bedolla.
   Google Scholar
• Bloom, H. S. (2009). Modern regression discontinuity analysis (MDRC working papers on research methodology). https://www.mdrc.org/sites/default/files/Regression_Discontinuity_embed.pdf
   Google Scholar
• Butler, F. A., Stevens, R., & Castellon, M. (2007). Ells and standardized assessments: The interaction between language proficiency and performance on standardized tests. In A. L. Bailey (Ed.), The language demands of school: Putting academic English to the test (pp. 27–49). Yale University Press.
   Google Scholar
• California Department of Education. (1999). English-language development standards for California public schools: Kindergarten through grade twelve.
   Google Scholar
• California Department of Education. (2002a). English-language development standards for California public schools: Kindergarten through grade twelve. https://www.cde.ca.gov/be/st/ss/documents/englangdevstnd.pdf
   Google Scholar
• California Department of Education. (2002b). Final minutes: State board of education. https://www.cde.ca.gov/be/mt/ms/documents/finmin062602.pdf
   Google Scholar
• California Department of Education. (2003). Technical report for the California English Language Development Test (CELDT), 2002-03 form B. https://www.cde.ca.gov/re/pr/documents/techrpt2.pdf
   Google Scholar
• California Department of Education. (2006). California English language development test: Including, but not limited to, update on California English language development test and review of possible modifications to guidelines for reclassification of English learners. https://www.cde.ca.gov/be/ag/ag/yr06/agenda0906.asp
   Google Scholar
• California Department of Education. (2007). Technical report for the California English Language Development Test (CELDT), 2006-07 edition (form F). https://www.cde.ca.gov/re/pr/documents/formftechreport.pdf
   Google Scholar
• California Department of Education. (2011). California English language development test (CELDT): Technical report 2009-10 edition. https://www.cde.ca.gov/re/pr/documents/celdt0910techrpt.pdf
   Google Scholar
• California Department of Education. (2013a). California common core state standards: English language arts & literacy in history/social studies, science, and technical subjects. https://www.cde.ca.gov/be/st/ss/documents/finalelaccssstandards.pdf
   Google Scholar
• California Department of Education. (2013b). California standards tests technical report: Spring 2012 administration. https://star.cde.ca.gov/techreports/CST/cst12techrpt.pdf
   Google Scholar
• California Department of Education. (2013c). Report to the legislature: Kindergarten and grade one early literacy assessment: Results and administrative process. https://www.cde.ca.gov/re/pr/documents/celdtlegrptk1earlylit0313.pdf
   Google Scholar
• California Department of Education. (2014). California English language development standards: Kindergarten through grade 12. https://www.cde.ca.gov/sp/el/er/documents/eldstndspublication14.pdf
   Google Scholar
• California Education Code § 313. (1999). https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?sectionNum=313&lawCode=EDC
   Google Scholar
• Carlson, D., & Knowles, J. (2016). The effect of English language learner reclassification on student ACT scores, high school graduation, and postsecondary enrollment: Regression discontinuity evidence from Wisconsin. Journal of Policy Analysis and Management, 35(3), 559–586. https://doi.org/10.1002/pam.21908
   Web of Science ®Google Scholar
• Carroll, P. E., & Bailey, A. L. (2016). Do decision rules matter? A descriptive study of English language proficiency assessment classifications for English-language learners and native English speakers in fifth grade. Language Testing, 33(1), 23–52. https://doi.org/10.1177/0265532215576380
   Web of Science ®Google Scholar
• Castañeda v. Pickard. (1981). 648 F.2d 989, 1006-07 (5th cir).
   Google Scholar
• Chamot, A. U., & O’Malley, J. M. (1994). The CALLA handbook: Implementing the cognitive academic language learning approach. Addison-Wesley.
   Google Scholar
• Chin, M. J. (2020). The effect of English learner reclassification on student achievement and noncognitive outcomes. Journal of Research on Educational Effectiveness, 14(1), 57–89. https://doi.org/10.1080/19345747.2020.1831116
   Web of Science ®Google Scholar
• Cimpian, J. R., Thompson, K. D., & Makowski, M. B. (2017). Evaluating English learner reclassification policy effects across districts. American Educational Research Journal, 54(1S), 255S–278S. https://doi.org/10.3102/0002831216635796
   Google Scholar
• Civil Rights Act of 1964. (1964). The civil rights act of 1964. Harvard Law Review, 78(3), 684. https://doi.org/10.2307/1339128
   Google Scholar
• Clogg, C. C., Petkova, E., & Haritou, A. (1995). Statistical methods for comparing regression coefficients between models. American Journal of Sociology, 100(5), 1261–1293. https://doi.org/10.1086/230638
   Web of Science ®Google Scholar
• Cook, H. G., Linquanti, R., Chinen, M., & Jung, H. (2012). National evaluation of title III implementation supplemental report: Exploring approaches to setting English language proficiency performance criteria and monitoring English learner progress. U.S. Department of Education, Office of Planning, Evaluation and Policy Development.
   Google Scholar
• Dabach, D. (2014). “I am not a shelter!”: Stigma and social boundaries in teachers’ accounts of students’ experience in separate “sheltered” English learner classrooms. Journal of Education for Students Placed at Risk (JESPAR), 19(2), 98–124. https://doi.org/10.1080/10824669.2014.954044
   Google Scholar
• Educational Testing Service. (2006). California standards tests technical report: Spring 2005 administration. https://star.cde.ca.gov/techreports/CST/cst05techrpt.pdf
   Google Scholar
• Estrada, P. (2014). English learner curricular streams in four middle schools: Triage in the trenches. The Urban Review, 46(4), 535–573. https://doi.org/10.1007/s11256-014-0276-7
   Google Scholar
• Every Student Succeeds Act (ESSA). (2015). Pub. L. No. 114-95 § 114 stat. 1177 (2015-2016).
   Google Scholar
• Flores, N. (2020). From academic language to language architecture: Challenging raciolinguistic ideologies in research and practice. Theory into Practice, 59(1), 22–31. https://doi.org/10.1080/00405841.2019.1665411
   Web of Science ®Google Scholar
• Flores, N., & Rosa, J. (2015). Undoing appropriateness: Raciolinguistic ideologies and language diversity in education. Harvard Educational Review, 85(2), 149–171. https://doi.org/10.17763/0017-8055.85.2.149
   Web of Science ®Google Scholar
• Florez, I. R. (2012). Examining the validity of the Arizona English language learners assessment cut scores. Language Policy, 11(1), 33–45. https://doi.org/10.1007/s10993-011-9225-4
   Web of Science ®Google Scholar
• Gándara, P., & Escamilla, K. (2017). Bilingual education in the United States. In O. García, A. Lin, & S. May (Eds.), Bilingual and multilingual education (pp. 439–452). Springer.
   Google Scholar
• Gelman, A., & Imbens, G. (2019). Why high-order polynomials should not be used in regression discontinuity designs. Journal of Business & Economic Statistics, 37(3), 447–456. https://doi.org/10.1080/07350015.2017.1366909
   Web of Science ®Google Scholar
• Grembi, V., Nannicini, T., & Troiano, U. (2016). Do fiscal rules matter? American Economic Journal: Applied Economics, 8(3), 1–30. https://doi.org/10.1257/app.20150076
   Web of Science ®Google Scholar
• Haas, E., Huang, M., Tran, L., & Yu, A. (2016). The achievement progress of English learner students in Nevada (REL 2016–154). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory West. http://ies.ed.gov/ncee/edlabs
   Google Scholar
• Hahn, J., Todd, P., & Van der Klaauw, W. (2001). Identification and estimation of treatment effects with a regression-discontinuity design. Econometrica, 69(1), 201–209. https://doi.org/10.1111/1468-0262.00183
   Web of Science ®Google Scholar
• Hayes, K., Salaz´ar, J., & Vuckovic, G. (2005). Evaluation of the structured English immersion program: Year 5 final report. https://files.eric.ed.gov/fulltext/ED462009.pdf
   Google Scholar
• Hill, L. E., Weston, M., & Hayes, J. M. (2014). Reclassification of English learner students in California. Public Policy Institute of California. https://www.ppic.org/wp-ontent/uploads/content/pubs/report/R_114LHR.pdf
   Google Scholar
• Imbens, G., & Kalyanaraman, K. (2012). Optimal bandwidth choice for the regression discontinuity estimator. The Review of Economic Studies, 79(3), 933–959. https://doi.org/10.1093/restud/rdr043
   Web of Science ®Google Scholar
• Imbens, G. W., & Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142(2), 615–635. https://doi.org/10.1016/j.jeconom.2007.05.001
   Web of Science ®Google Scholar
• Jacob, R., Zhu, P., Somers, M., & Bloom, H. (2012). A practical guide to regression discontinuity. MDRC.
   Google Scholar
• Johnson, A. (2019a). The effects of English learner classification on high school graduation and college attendance. American Educational Research Association Open, 5(2), 1–23. https://doi.org/10.1177/2332858419850801
   Google Scholar
• Johnson, A. (2019b). The impact of English learner reclassification on high school reading and academic progress. Educational Evaluation and Policy Analysis, 42(1), 46–65. https://doi.org/10.3102/0162373719877197
   Web of Science ®Google Scholar
• Kim, J., & Herman, J. (2008). Investigating ELL assessment and accommodation practices using state data. In providing validity evidence to improve the assessment of English language learners(CRESST tech. Rep. No. 738, pp. 81–138). University of California, National Center for Research on Evaluation, Standards and Student Testing.
   Google Scholar
• King, K., & Bigelow, M. (2018). The language policy of placement tests for newcomer English learners. Educational Policy, 32(7), 936–968. https://doi.org/10.1177/0895904816681527
   Web of Science ®Google Scholar
• Lau v. Nichols, 414 U.S. 563. (1974).
   Google Scholar
• Linquanti, R., & Bailey, A. L. (2014). Reprising the home language survey: Summary of a national working session on policies, practices, and tools for identifying potential English learners. Council of Chief State School Officers.
   Google Scholar
• Linquanti, R., Cook, H. G., Bailey, A. L., & MacDonald, R. (2016). Moving toward a more common definition of English learner: Collected guidance for states and multi-state assessment consortia. Council of Chief State School Officers.
   Google Scholar
• National Institute of Child Health and Human Development (NICHD). (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction (report of the national reading panel, NIH publication No. 00-4769). U.S. Government Printing Office.
   Google Scholar
• Nichols, A. (2011). Rd 2.0: Revised stata module for regression discontinuity estimation. http://ideas.repec.org/c/boc/bocode/s456888.html
   Google Scholar
• No Child Left Behind Act of 2001, Pub. L. No. 107-110, 115 stat. 1425. (2002).
   Google Scholar
• Office of Civil Rights (OCR). (2011). from https://www2.ed.gov/about/offices/list/ocr/docs/investigations/09105001-a.pdf
   Google Scholar
• Onda, M., & Seyler, E. (2020). English learners reclassification and academic achievement. Evidence from Minnesota. Economics of Education Review, 79, 102043. https://doi.org/10.1016/j.econedurev.2020.102043
   Web of Science ®Google Scholar
• Parker, C., Louie, J., & O’Dwyer, L. (2009). New measures of English language proficiency and their relationship to performance on large-scale content assessments (Issues and answers report No. REL 2009–no. 066). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Northeast and Islands. https://files.eric.ed.gov/fulltext/ED504060.pdf
   Google Scholar
• Parrish, T., Perez, M., Merickel, A., & Linquanti, R. (2006). Effects of the implementation of proposition 227 on the education of English learners, K-12: Findings from a five-year evaluation. American Institutes for Research.
   Google Scholar
• Porter, K. E., Reardon, S. F., Unlu, F., Bloom, H. S., & Cimpian, J. R. (2017). Estimating causal effects of education interventions using a two-rating regression discontinuity design: Lessons from a simulation study and an application. Journal of Research on Educational Effectiveness, 10(1), 138–167. https://doi.org/10.1080/19345747.2016.1219436
   Web of Science ®Google Scholar
• Reardon, S. F., & Robinson, J. P. (2012). Regression discontinuity designs with multiple rating- score variables. Journal of Research on Educational Effectiveness, 5(1), 83–104. https://doi.org/10.1080/19345747.2011.609583
   Web of Science ®Google Scholar
• Reyes, M., & Hwang, N. (2019). Middle school language classification effects on high school achievement and behavioral outcomes. Educational Policy, 35(4), 590–620. https://doi.org/10.1177/0895904818823747
   Web of Science ®Google Scholar
• Robinson, J. P. (2011). Evaluating criteria for English learner reclassification: A causal-effects approach using a binding-score regression discontinuity design with instrumental variables. Educational Evaluation and Policy Analysis, 33(3), 267–292. https://doi.org/10.3102/0162373711407912
   Web of Science ®Google Scholar
• Robinson-Cimpian, J. P., & Thompson, K. D. (2016). The effects of changing test-based policies for reclassifying English learners. Journal of Policy Analysis and Management, 35(2), 279–305. https://doi.org/10.1002/pam.21882
   Web of Science ®Google Scholar
• Sato, E., Lagunoff, R., Worth, P., Bailey, A. L., & Butler, F. A. (2005). ELD standards linkage and test alignment under title III: A pilot study of the CELDT and the California ELD and content standards. Final report (June) to the California department of education, Sacramento, CA.
   Google Scholar
• Saunders, W. M., & Marcelletti, D. J. (2013). The gap that can’t go away: The catch-22 of reclassification in monitoring the progress of English learners. Educational Evaluation and Policy Analysis, 35(2), 139–156. https://doi.org/10.3102/0162373712461849
   Web of Science ®Google Scholar
• Schleppegrell, M. J. (2012). Academic language in teaching and learning. The Elementary School Journal, 112(3), 409–418. https://doi.org/10.1086/663297
   Web of Science ®Google Scholar
• Shin, N. (2018). The effects of the initial English language learner classification on students’ later academic outcomes. Educational Evaluation and Policy Analysis, 40(2), 175–195. https://doi.org/10.3102/016237371773737
   Web of Science ®Google Scholar
• Shin, N. (2020). Stuck in the middle: Examination of long-term English learners. International Multilingual Research Journal, 14(3), 181–205. https://doi.org/10.1080/19313152.2019.1681614
   Web of Science ®Google Scholar
• Sireci, S. G., & Faulkner-Bond, M. (2015). Promoting validity in the assessment of English learners. Review of Research in Education, 39(1), 215–252. https://doi.org/10.3102/0091732X14557003
   Google Scholar
• Slama, R.B. (2014). Investigating whether and when English learners are reclassified into mainstream classrooms in the United States: A discrete-time survival analysis. American Educational Research Journal, 51, 220–252.
   Google Scholar
• Stokes-Guinan, K., & Goldenberg, C. (2011). Use with caution: What CELDT results can and cannot tell us. CATESOL Journal, 22(1), 189–202.
   Google Scholar
• Thompson, K. (2017). English learners’ time to reclassification: An analysis. Educational Policy, 31(3), 330–363. https://doi.org/10.1177/0895904815598394
   Web of Science ®Google Scholar
• Thompson, K. D. (2015). Questioning the long-term English learner label: How classification and categorization can blind us to students’ abilities. Teachers College Record, 117(12), 1–50. https://doi.org/10.1177/016146811511701203
   Web of Science ®Google Scholar
• Trochim, W. (1984). Research design for program evaluation: The regression-discontinuity approach. Sage Publications.
   Google Scholar
• Umansky, I., & Dumont, H. (2021). English learner labeling: How English learner status shapes teacher perceptions of student skills & the moderating role of bilingual instructional settings. American Educational Research Journal, 58(5), 993–1031. https://doi.org/10.3102/0002831221997571
   Web of Science ®Google Scholar
• Umansky, I. M. (2016a). Leveled and exclusionary tracking: English learners’ access to academic content in middle school. American Educational Research Journal, 53(6), 1792–1833. https://doi.org/10.3102/0002831216675404
   Web of Science ®Google Scholar
• Umansky, I. M. (2016b). To be or not to be EL: An examination of the impact of classifying students as English learners. Educational Evaluation and Policy Analysis, 38(4), 714–737. https://doi.org/10.3102/0162373716664802
   Web of Science ®Google Scholar
• Umansky, I. M. (2018). According to plan? Examining the intended and unintended treatment effects of EL classification in early elementary and the transition to middle school. Journal of Research on Educational Effectiveness, 11(4), 588–621. https://doi.org/10.1080/19345747.2018.1490470
   Web of Science ®Google Scholar
• Villegas, L., & Pompa, D. (2020). The patch landscape of state English learner policies under ESSA. Migration Policy Institute. https://www.migrationpolicy.org/sites/default/files/publications/ESSA-Compendium-Final.pdf
   Google Scholar
• Winke, P. (2011). Evaluating the validity of a high-stakes ESL test: Why teachers’ perceptions matter. TESOL Quarterly, 45(4), 628–660. https://doi.org/10.5054/tq.2011.268063
   Web of Science ®Google Scholar
• Wolf, M. K., Farnsworth, T., & Herman, J. L. (2008). Validity issues in assessing English language learners’ language proficiency. Educational Assessment, 13(2), 80–107. https://doi.org/10.1080/10627190802394222
   Google Scholar
• Wolf, M. K., & Faulkner-Bond, M. (2016). Validating English language proficiency assessment uses for English learners: Academic language proficiency and content assessment performance. Educational Measurement Issues & Practice, 35(2), 6–18. https://doi.org/10.1111/emip.12105
   Google Scholar
• Zakariya, S. B. (2015). Learning to read, reading to learn: Why third-grade is a pivotal year for mastering literacy. Center For Public Education. https://www.nsba.org/-/media/NSBA/File/cpe-learning-to-read-reading-to-learn-white-paper-2015.pdf?la=en&hash=8E0E470C3E263C66E4491EC035224DC9018C6D5F
   Google Scholar