Summer School and Summer Learning: An Examination of the Short- and Longer Term Changes in Student Literacy

Abstract

Research Findings: Multiple student cohorts were longitudinally tracked and student participation in a summer program bridging the 1st- and 2nd-grade academic years was recorded to examine selection and efficacy issues related to a summer school implementation in the Pacific Northwest. The estimation of regression discontinuity models uncovered evidence of a local average treatment effect. At the cutscore for program admission, participating students had estimated summer oral reading fluency gains approximately 0.40 SD larger than those of nonparticipants. Further examination of the literacy outcomes among the sample of cutscore eligible students revealed that struggling readers who participated in the summer program increased their level of reading fluency relative to struggling readers who declined an invitation to participate. However, the advantage gained by cutscore eligible participants was not sustained over the subsequent academic year. Practice or Policy: These results suggest that supplemental summer instruction delivered to at-risk students may promote literacy gains during the otherwise challenging summer months and thereby serve as a useful intermediary tool for K–12 stakeholders seeking to keep struggling readers on track toward proficiency.

ACKNOWLEDGMENTS

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A090369 to the University of Oregon. The opinions expressed are those of the authors and do not represent the views of the Institute of Education Sciences or the U.S. Department of Education.

Notes

Note. The mean of each dichotomous variable represents the proportion of the sample with the identified characteristic (e.g., 4% of benchmark eligible participants were English language learners). TORF = Test of Oral Reading Fluency.

Note. Dummy codes were used to identify program participants in the TOT models and benchmark eligible students in the ITT model. IV = instrumental variable; TORF = Test of Oral Reading Fluency.

**p < .01. ***p < .001.

¹One might question whether some of the observed results could be due to the use of a gain score. The same models presented here were also run using the second-grade fall TORF score as the outcome. Results showed a statistically significant positive relationship between spring first-grade and fall second-grade TORF scores, but coefficients in the regression models were essentially identical to those reported in the text.

²In an RD model that included demographic covariates, a 6.7 words per minute discontinuity between participants and nonparticipants was observed (d = .47). Student age was positively related and Title I status was negatively related (non–Title I advantage) to summer changes in oral reading fluency (p < .05). The block of covariates accounted for an additional 2% of the variation in the outcome.

³As a further check of the functional form of the regression, the sample was limited to those with scores of 50 or less on the assignment variable (first-grade spring TORF). In this limited sample, the interaction between treatment status and the assignment score was not statistically significant. Instead, a statistically significant negative linear relationship between the assignment score and summer reading gains was observed. Nonetheless, the estimated treatment effect in the reduced sample was nearly identical to that observed in the full sample. The IV-adjusted TOT estimate revealed a statistically significant 5.6 words per minute discontinuity at the cutscore.

Note. A dummy code was used to identify program participants. TORF = Test of Oral Reading Fluency.

*p < .05. **p < .01.

Note. A dummy code was used to identify program participants. Standard errors are in parentheses. TORF = Test of Oral Reading Fluency.

*p < .05. **p < .01. ***p < .001.

⁴In a piecewise growth model that specified individual demographic covariates instead of propensity scores, the pattern of treatment group similarities and differences was analogous to that of the propensity adjusted model. Benchmark eligible participants were not statistically different from nonparticipants at the end of first grade (p > .05), gained 5.51 more words per minute over the summer, and grew at a 2.51 words per assessment deficit in Grade 2 (p < .05). First-grade winter TORF was positively related to first-grade exit status and summer oral reading fluency gain, whereas FRL status (non-FRL advantage) and ELL status (non-ELL advantage) were related to exit status and summer change, respectively (p < .05).