The Educational and Behavioral Impacts of the Ewing Marion Kauffman Charter School

ABSTRACT

The Kauffman School is a public charter school serving students from low-income neighborhoods in Kansas City, Missouri. This paper used a matched comparison group design to estimate the impacts of the Kauffman School on student achievement, attendance, and suspensions. We found that the Kauffman School had large positive impacts on student achievement in mathematics, reading, and science. This paper also used qualitative data to explore possible mechanisms for the school's effects. We found evidence that the Kauffman School's hallmarks are being implemented faithfully, and that stakeholders believe the school's methods are having a positive influence on students' attitudes and performance

KEYWORDS:

• charter schools
• school choice
• school effectiveness

Acknowledgments

The authors are grateful to Melissa Bardwell, Leigh Ann Grant-Engle, Jason Young, Timothy Wittmann, and the Missouri Department of Elementary and Secondary Education for providing the data used in this article. We would like to thank the following Mathematica staff who contributed to the study: Brian Gill provided important feedback and comments on the report and Ira Nichols-Barrer gave valuable input on the study design. Cindy George and Jennifer Littel provided editing services and Colleen Fitts formatted the document.

Funding

This article is based on findings from an evaluation funded by the Ewing Marion Kauffman Foundation.

Notes

¹. The Kauffman School has more fifth-grade students than other public schools in Kansas City. The average Kansas City school enrolls 49 fifth graders each year, and the Kauffman School is the only school with more than 100 fifth-grade students.

². The attrition rate of fifth- and sixth-grade students from the Kauffman School is 13%, which is lower than the 30% average attrition rate of fifth- and sixth-grade students from other Kansas City schools. The Kansas City average attrition rate is calculated for schools where at least half the students remained in the school in the subsequent year, to remove from the sample any schools that closed or had a terminal grade of five or six. In Johnson et al. (2016), we discuss the issue of attrition in more detail and present 1-year attrition-adjusted impact estimates for students who remain enrolled in the Kauffman School.

³. Cohort I students who repeated fifth or sixth grade are also included in the Kauffman student group for the 3-year impact calculations. See Johnson et al. (2016) for details about how grade repeaters are treated in the analysis.

⁴. The official name of the reading MAP assessment is “Communication Arts.”

⁵. See the appendix of Johnson et al. (2016) for more details about our imputation procedure.

⁶. The statewide average (pooled across the years 2011–2012 through 2013–2014) fourth-grade MAP scaled score was 648 in mathematics and 661 in reading. The statewide standard deviation of fourth-grade MAP scores was 34 in mathematics and 38 in reading. Examples of test accommodations include extended test time, individual testing, and oral reading of test questions.

⁷. The covariates include all the variables listed in Table 1. We also include third-grade mathematics and reading MAP scores, second- and third-order polynomial terms for fourth-grade MAP scores, English language learner status, and indicator variables that equal one if a student has imputed prior test scores or imputed attendance or suspension data.

⁸. The statewide standard deviations of 7th-grade MAP scores were 41 in mathematics and 36 in reading. The statewide standard deviations of sixth-grade MAP scores were 40 in mathematics and 33 in reading. The statewide standard deviations of fifth-grade MAP scores were 43 in mathematics, 36 in reading, and 34 in science.

⁹. The effect sizes were estimated separately for each cohort of students. To calculate the impact estimates in Table 1, we averaged these effect sizes together, weighting by the number of Kauffman students in the analysis sample for each cohort.

¹⁰. The percentile ranks 3 years after enrollment at the Kauffman School were calculated by taking the average fourth-grade z-scores of Kauffman students and adding the 3-year effect size estimates. These calculations assume that the percentile rank of the average student in Kansas City does not change over time.

¹¹. The 3-year impact estimates reported in this section are generally obtained by tripling the average annual impact estimates reported by the authors. The exceptions to this are the KIPP study, the charter lottery study, and the CMO study. In the KIPP and CMO studies, the authors reported 3-year impact estimates separately from 1-year impact estimates. The 3-year impact estimates for the charter lottery study were obtained by increasing the 2-year impact estimates by 50%.

¹². The KIPP 1-year science impact estimate was obtained by dividing the 3- to 4-year impact estimate by 3.5.

¹³. In Grades 5 through 7 expulsions are very rare at both the Kauffman School and other Kansas City schools, so they are not analyzed as an outcome in this article.

¹⁴. Because the suspension outcome is a binary variable rather than a continuous one, we used a logit model in place of the linear regression to implement the analysis.

Additional information

Funding

This article is based on findings from an evaluation funded by the Ewing Marion Kauffman Foundation.