Using Research to Improve College Readiness: A Research Partnership Between the Los Angeles Unified School District and the Los Angeles Education Research Institute

Abstract

The Los Angeles Unified School District (LAUSD) serves a large majority of socioeconomically disadvantaged students who are struggling academically and are underprepared for high school graduation and college. This article describes the partnership between LAUSD and the Los Angeles Education Research Institute, and how this collaboration endeavors to produce accessible and high-quality research to inform pressing problems of practice. The article also presents findings from an ongoing partnership research project analyzing a district policy focused on improving college readiness by aligning high school graduation and college-eligibility requirements. In a cohort that went through high school before the policy became mandatory for all students, less than 1/5 of all students (and 30% of graduates) met the college eligibility criteria. Our findings indicate that academic and behavioral indicators from 8th and 9th grade can help identify for possible intervention students who are not on track to meet these new graduation requirements.

ACKNOWLEDGMENTS

Maxwell Mansolf, Tim Chen, and Jordan Rickles provided excellent research assistance for this project. We are also very grateful to Julie Kane in LAUSD for her support of this work and inclusion of LAERI in key A–G and performance–management conversations at the district. Some aspects of this article are based on a paper presented at AERA (Yamashiro & Phillips, 2012).

Funding

This work was supported, in part, by grants from the California Community Foundation and the JPMorgan Chase Foundation. The partnership also made use of resources at the California Center for Population Research, UCLA, which is supported by infrastructure grant R24HD041022 from the Eunice Kennedy Shriver National Institute of Child Health and Human Development.

Notes

¹ Aligning high school curriculum with college eligibility has become increasingly popular in other districts across the state (see Betts, Zau, & Bachofer, 2013) and in other states around the country (see Achieve, Inc., 2007).

² Using data on first-time ninth graders in 2001–2002, Silver, Saunders, and Zarate (2008) found that course failures and Algebra I completion were especially important predictors of on-time high school graduation. Similarly, using data on first-time ninth graders in 2007–2008, researchers from the Strategic Data Project found that English language arts standardized test scores from eighth grade predicted high school graduation (Center for Education Policy Research, 2013).

³ Although the class of 2012 graduated prior to the new requirement being implemented, schools were still expected to provide all students access to A–G courses at this time; thus, this cohort shows the A–G completion rate before consequences (i.e., students not graduating) were phased in for A–G coursework.

⁴ In New York, Kemple, Segeritz, and Stephenson (2013) found that a somewhat similar indicator signifying the accumulation of at least 10 credits and one or more Regents exams passed by the end of ninth grade correctly predicted high school graduation in 4 years roughly 79% of the time.

⁵ Predicting college readiness well may also require measuring factors, such as college knowledge, that are less likely to exist in administrative data sets (Kless, Soland, & Santiago, 2013).

⁶ We defined first-time ninth graders as students who met two criteria: (a) They had a grade level of 9 in at least one of the following fall datasets in the 2008–2009 school year: norm fall demographics, end fall demographics, or end fall marks. When there were discrepancies in grade level among the datasets, we defined the student as a ninth grader if she was in grade 9 in two out of the three files. We also defined the student as a ninth grader if she was in grade 9 in one of the files but lacked data on grade level in the other two files. If the student had missing data in only one file, and conflicting grades in the other two files, we did not define the student as a ninth grader; and (b) They had a grade level lower than 9 in the 2007–2008 school year in the norm spring demographics file, the end spring demographics file, and the end spring marks file. If grade-level information was missing in any of these datasets, we still counted the student as a pre-ninth grader as long as the student's grade level was lower than 9 in the nonmissing data set. If the student was missing grade-level information in all these data sets but took an eighth-grade or lower California Standards tests in 2007–2008, we also defined the student as a pre-ninth grader in that year.

⁷ We excluded students who transferred to a public school outside the district, to a private school (including home schooling), or who left California. We also excluded students who did not appear in any course marks file during any semester between 2008–2009 and 2011–2012.

⁸ We excluded two subsets of students for whom the A–G requirements of the class of 2017 do not apply. These include EL 9th graders who scored at the lowest levels of English language proficiency (ELD 1 and 2) and students with disabilities who are on an alternative curriculum (we defined such students as those who were classified as special education and were enrolled in one of the alternate-curriculum ninth-grade courses listed in district policy documents; Aquino & Howell, 2013). Taken together, these excluded EL and disabled students constituted 3% of our initial all-students sample.

⁹ We defined graduates as those students with a leave code, assigned by the district, that indicated that they left because they graduated. For the subset of graduates for whom we also had specific reason codes (i.e., reasons why they left), we followed LAUSD's operational rules, excluding students who received a certificate of completion, passed the California proficiency exam, earned a GED, or were a special-education prior completer.

¹⁰ To limit the number of students dropped from the sample, the predictive models include missing data dummies for students who are missing eighth-grade English or math standardized test scores. The models also include missing data dummies for students who are missing any of the demographic variables. In total, we lost 1,383 students from the all-students sample when we estimated our predictive models. This remaining predictive sample had an A–G completion rate that is only 0.6 percentage points higher than that of the all-students sample.

¹¹ The results we present in this article are based on University of California A–G completion rules, but results based on California State University A–G completion rules hardly differ. Certain A–G designations allow for validation, in which students receive credit for an A–G course either by passing a test or by passing a more advanced course. The LAERI data archive does not account for validation through test taking or passage, but our A–G completion calculations do take into account validation through course work.

¹² We developed the models on a separate half of the data so that any overfitting would not bias predictive results based on the other half of the data. We retained variables that were statistically significant at the .10 level, with the standard errors adjusted for the clustering of students within high schools.

¹³ We estimated the predicted probability that each student in this sample would complete A–G based on the model, coded students who had at least a 50% estimated chance of completing A–G as predicted A–G completers, and examined the predictive accuracy of these models by comparing predicted completion rates to actual completion rates.

¹⁴ This approach to developing and combining flags has proved informative in early warning research on high school dropout (see, e.g., Balfanz, Herzog, & Mac Iver, 2007) and this index enabled us to incorporate a range of indicators transparently, which reflects and supports the district's desire to use empirically based, but simple, measures in their early warning system.

¹⁵ Using school-level data for the same timeframe (freshmen in 2007–2008), Saunders, Ventura, and Flores-Valmonte (2013) found that close to one in five graduates completed A-G.

¹⁶ The district's performance metric for 4-year cohort graduation rates was meant to increase each year and eventually get to 100%. In 2011–2012, the district reported a 64% 4-year cohort graduation rate, and set graduation rate targets for 2012–2013 and 2013–2014 at 68% and 70%, respectively (LAUSD, 2012).

¹⁷ More formally, these three percentages correspond to (a) classification accuracy; (b) 1-sensitivity, where sensitivity is defined as the proportion of A–G noncompleters identified by the indicator as noncompleters; and (c) 1-negative predictive value, where negative predictive value is defined as the proportion of students predicted to complete who actually complete. Recent literature on EWIs has stressed the importance of using various metrics of accuracy to understand the trade-offs associated with indicators and to more explicitly prioritize limited resources for intervention (Bowers et al., 2013; Knowles, 2014).

¹⁸ The district's current early-warning reporting system tracks changes in student performance over different time periods and across subject areas. The reports trigger a warning whenever a student's performance declines from one time period to the next. The at-risk report triggers an alert whenever student performance falls into a risk zone in one or more areas (LAUSD, 2014a).