An Investigation of the Summer Learning Effect on Fourth Grade Students’ Reading Scores

Abstract

Students’ reading losses during the summer are a concern for schools in the United States. The purpose of this investigation was to examine fourth grade students’ (N = 5,113) reading development over the summer. The results indicated that students in the lowest two quartiles made achievement gains over summer while these same students evidenced limited reading growth throughout the academic year. In contrast, students in the upper two quartiles evidenced reading losses over summer, although evidencing continuous reading growth throughout the school year. The findings support the need for further investigation of the effects of summer on reading achievement.

Policy makers, school districts, and other stakeholders throughout the United States have debated formal year-round education for elementary school students (Heller & Bailey, 1976; Pedersen, 2015; Shields & Oberg, 2000). Some scholars have noted that continual year-round learning during elementary school would close the achievement gap and minimize summer reading losses, while enriching and extending students’ learning opportunities (Alexander, Entwisle, & Olson, 2001); Cooper, Nye, Charlton, Lindsay, & Greathouse, 1996; Dechenes, Malone, & Harvard Family Research Project, 2011). Nevertheless, other scholars and policy makers question the necessity of school-based year-round learning for elementary school students and discount the influence of summer school on students’ possible reading losses during the summer (Downey, von Hippel, & Broh, 2004; Heyns, 1987; McMullen & Rouse, 2012).Cooper et al. (1996) conducted a meta-analysis of 13 investigations examining summer learning losses between 1975 and 1995 and found that summer learning losses (reading and math) amounted to about one-tenth of a test score standard deviation, equivalent to about one month of schooling. In addition, Cooper et al. identified that students’ reading losses during the summer were strongly influences by their socioeconomic status (SES), where low SES students’ reading losses were larger than high-SES students and by grade level. In addition, the higher the students’ grade level the greater their reading loss.

“Examination of summer reading losses in the United States is important as elementary students lag behind in reading performance compared to their international counterparts.” The National Center of Education Statistics Reading Report Card for 2013 (Institute of Education Sciences (IES), 2014) measured reading comprehension among fourth grade students (N = 190,400) and identified no changes in the participants’ reading performance on the National Assessment of Educational Progress (NAEP) scales from 2007 to 2011, meaning students reading has not improved at the national level, leaving fourth grade students in America behind their international peers. “If America’s students are to remain competitive in a knowledge-based economy, our public schools must greatly accelerate the rate of progress of the last four years and do more to narrow America’s large achievement gaps” (US Department of Education [USDOE], 2013). Therefore, additional research is warranted that examines elementary school students’ reading changes during the summer. As a result, the purpose of the current study was to examine by quartile fourth grade students’ summer reading change scores utilizing a curriculum-based measure. Change scores were determined by changes in reading with a sample of fourth grade (end of the school year – May) to fifth grade (beginning of the school year – August) reading scores.

Students’ Reading

Reading achievement is often researched to determine changes in students’ learning during the summer (Cooper et al., 1996; Gershenson, 2013; Helf, Konrad, & Algozzine, 2008; Sandberg-Patton & Reschly, 2013). Prior research related to this study regarding students’ summer reading status may be classified into the following categories: (a) investigations examining correlations between external variables (e.g., SES) and changes in students’ learning, (b) investigations examining students’ personal characteristics (e.g., gender, grade level, and race) and changes in students’ learning, and (c) studies exploring a combination of students’ personal and systemic variables.

Changes in Students’ Learning During the Summer

Changes in students’ learning during the summer is defined as variability (gains or losses) in students’ academic knowledge and skills when students are not required to attend school. Summer learning losses are referred to as (a) summer reading regression (Cornelius & Semmel, 1982), (b) summer gap (Quinn, 2015), (c) summer slide (Slates, Alexander, Entwisle, & Olson, 2012), (d) summer learning effect (Jesson, McNaughton, & Kolose, 2014), (e) summer setback (Allington et al., 2010; Entwisle & Alexander, 1992; Helf et al., 2008), (f) summer learning gap (Alexander, Entwisle, & Olson, 2007), and (g) summer learning loss (Menard & Wilson, 2014; Sandberg-Patton & Reschly, 2013). For the purpose of this manuscript, summer learning loss and summer reading loss are used interchangeably and refer to the decline or unrealized potential gain of reading skills experienced by students after summer vacation from school. Further the terms, summer learning effect, summer reading changes or summer reading effect refers to the fluctuations (gains or loss) of students’ reading scores after their summer vacation from school.

BY QUARTILE

Students’ academic level (based on quartiles) may be a significant contributor to summer learning differences. Heyns (1987) examined elementary school students’ (N = 3000) reading achievement changes during the school year and during the summer based on the students’ quartiles. Those students who scored in the upper quartile were more likely to make greater academic gains and lose less in achievement than those students who scored in the lower quartile. While there were limited research investigating summer reading achievement by quartile, some studies examined students’ achievement level by Lexile scores (Allington et al., 2010). Wilkins et al. (2012) investigated the effects of a reading intervention (providing leveled-reading) on the reading comprehension achievement of students between third and fourth grade who scored below the 50th percentile (N = 1571). The results were not statistically significant for improved reading achievement during the summer months for students in a reading intervention. Further, there was no difference in the students’ reading comprehension scores when considering their Lexile distribution (top, middle, or bottom).

EXTERNAL VARIABLES

Socioeconomic status is a frequently investigated variable related to changes in students’ learning during the summer (Alexander et al., 2007; Allington & McGill-Franzen, 2003; Cooper et al., 1996; Kim, Quinn, & Society for Research on Educational Effectiveness [SREE], 2012; Sandberg-Patton & Reschly, 2013). Multiple researchers found significant differences in summer reading changes between students from low SES and higher SES backgrounds. For example, Cooper et al. (1996) identified that “Middle-class students showed a non-significant gain in grade-level equivalent reading scores, while lower-class students showed a significant loss” (p. 261). Therefore, students at a lower SES tend to have greater deficit in reading than peers from a higher SES following summer break (Allington & McGill-Franzen, 2003; Cooper, Charlton, Valentine, Muhlenbruck, & Borman, 2000; Heyns, 1978).

Additionally, Alexander et al. (2007) examined the long-term implications of students’ SES on reading and summer setback (students’ achievement difference if they had attended school during the summer months) and found a cumulative impact for students from a low SES background even though all these students had the same rate of growth during the school year. In addition, Alexander et al. concluded that the achievement gap between low and high SES students related to summer learning loss was indicative of students’ propensity to graduate from high school and attend college.

Nevertheless, not all students from low SES backgrounds experience summer reading loss. Specifically, Slates et al. (2012) identified family characteristics of students from low SES homes that did not experience summer reading loss (N = 44), including two parent households that provided reading experiences for their children such as visiting the library, checking out books, and reading for a longer period as compared to students who did experience summer loss. Furthermore, these two parent households checked their students’ homework completion and had other demonstrated behavioral qualities of parents of students from higher SES homes who did not experience summer reading loss.

Examination of SES in relation to student’s learning over the summer is of interest for Title I schools, which are schools where more than 40% of the students qualify for free and reduced lunch. Changes in students’ learning during the summer and Title I have an inconsistent relationship. Title I schools are designated as such when more than 40% of the students qualify for free and reduced lunch. Klibanoff, Haggert, RMC Research Corp., and System Development Corp. (1981) identified gains in Title I students’ reading scores as compared to the general population of students after the Title I students completed summer education in reading and math.

PERSONAL CHARACTERISTICS

In addition to external variables, scholars have also examined students’ personal characteristics and their mediating impact on changes in learning during the summer. Personal characteristics include studies examining students’ age and grade level, gender, ethnicity, and race. Reading for Kindergarten through second grade students is an activity where the students are learning to read; however, by third grade students often begin to read for learning (Chall & Jacobs, 2003; Chall, Jacobs, & Baldwin, 1990). The foundational years of learning to read are pivotal to reading achievement. Alexander et al. (2007) noted that the greatest achievement loss for students occurs in their early education years and during the summers of elementary school causing long-term implications for academic success. Conversely, Cooper et al. (1996) identified minimal non-significant summer learning gains in the lower grades and in the upper grades (grade 3 and above), representing significant losses. When investigating summer reading achievement based on grade levels or age, the results vary. However, when examining students’ reading achievement within the same grade by academic level, percentile, or quartile, limited information exists.

The personal characteristic of gender was considered in multiple studies related to summer reading achievement. Yet, in most cases, gender had no moderating effects on changes in students’ learning (Allington et al., 2010; Arnold, Fleming, DeAnda, Castleman, & Wartman, 2009; Bowers & Schwarz, 2018; Cooper et al., 1996). Conversely, other studies present differing results. Slates et al. (2012) indicated that gender was a factor in improved reading achievement over four summers among students that are socioeconomically disadvantaged, and English was their second language. Both genders mean reading achievement scores improved but females reading comprehension was greater. Downey et al. (2004) determined that a male gender gap in reading is evident as early as kindergarten in that females started about one and one-half months ahead of males. Further their investigation identified that females reading grew faster than males but declined in rate as the students moved through kindergarten, the subsequent summer, and into first grade.

Another personal characteristic considered in summer reading achievement is race and ethnicity. Scholars have examined ethnicity in relation to students’ summer learning changes, and results vary. Heyns (1978) indicated there was a racial gap in reading abilities between black and white students, attributing some of the discrepancy in reading ability to summer setbacks. In contrast, Cooper et al. (1996) found limited consistency in the relationship between changes in students’ learning during the summer and their gender, race, or cultural ethnicity. Quinn (2015) using the same data set, Early Childhood Longitudinal Study (ECLS) –K, as several previous summer gap studies (Benson & Borman, 2010; Downey et al., 2004), investigated the various models, methodologies, and assumptions regarding the summer achievement gap. Quinn identified that the types of questions asked, and the models and methodologies chosen may yield varying statistical outcomes and noted that caution should be observed when interpreting results. Quinn’s own analysis of the data concluded that Black and White students make statistically equivalent growth in reading during the summer months from kindergarten to first grade.

METRICS FOR MEASURING SUMMER LEARNING LOSS

The metrics and assessments employed in studies examining changes in students’ learning during the summer vary and often measure only one reading domain (e.g., oral fluency, spelling, comprehension, and grammar). Investigating one factor of students’ reading changes (e.g., spelling or vocabulary) may lead to misinterpretation of the results and limited practical significance of the findings based on the multidimensional nature of reading (National Reading Panel, 2000). For example, Helf et al. (2008) used a curriculum-based measurement, DIBELS (Deno, 1985), to measure students' learning changes during the summer and “found that students did not regress over the summer; in fact, their performance improved in four different areas of early reading skills” (p. 427). Helf et al. suggested a possible reason for the incongruence between their finding and other research (Cooper et al., 1996) is that most studies examining changes in reading during the summer assess students’ reading comprehension, which is different from measuring students’ levels of word recognition, decoding, or fluency.

In examining change in students’ learning during the summer in the areas of reading, a variety of assessment measures have been used, including: (a) silent reading (Elder, 1927), (b) spelling lists (Nelson, 1928), (c) achievement scores (Cooper et al., 1996; Entwisle, Alexander, & Olsen 1997; Heyns, 1987), (d) Metropolitan Achievement Test (Arnold,1968), (e) California Achievement Test (Parsley & Powell, 1962), (f) Iowa Test of Basic Skills, (g) Dynamic Indicators of Basic Early Literacy Skills (DIBELS; Sandberg-Patton & Reschly, 2013), (h) Lexile reading scores (Allington et al., 2010), and (i) curriculum-based measures (CBM; Allinder, Fuchs, Fuchs, & Hamlett, 1992; Helf et al., 2008). In addition, national data sets including the Beginning School Study (Entwisle & Alexander, 1992, 1994) and the Early Childhood Longitudinal Study (ECLS-K; Downey et al., 2004; Quinn, 2015) have determined summer reading loss in research. Consequentially, research investigating students’ changes in reading during the summer employ diverse assessment instruments to measure reading constructs and have limited sampling representation, mitigating inferences that may be made to interpret the results.

Standardized achievement test scores (e.g., Iowa Test of Basic Skills) as the metric for measuring changes in students’ learning lack the precise information required to ascertain students’ individual knowledge (Marston, Deno, & Tindal, 1983). Rather, standardized achievement instruments provide relative information of progress based on a normed sample. These summative assessments are unrelated to specific curriculum taught and are not meant to be taken repeatedly over a short period of time (May – August). For these reasons, in the current study, a curriculum-based measure, Istation’s Indicators of Progress: Advanced Reading (ISIP-AR; Mathes, 2014) was utilized to examine change in fourth grade students’ reading scores over the summer break (May–August). ISIP-AR is a computer adaptive curriculum-based measurement (CA-CBM) used in the classroom by teachers for continuous progress monitoring and differentiating instruction. Other usages of ISIP-AR range from benchmarking students’ reading progress for state assessments (Campbell et al., 2018; Patarapichayatham, 2018; Patarapichayatham, Fahle, & Roden, 2014) and as a screening instrument for student services (Hoelzle, 2012).

The Current Study

The purpose of the current study included: (a) examining a sample of fourth grade students’ changes in reading scores from the end of May (end of grade 4) to August (beginning of grade 5) during the summer months and (b) investigating the relationship between fourth grade students’ change in reading scores by reading achievement level (quartiles), gender, and Title I status. The research questions guiding this investigation were the following:

RQ1. What are the observed changes in fourth grade students’ reading after the conclusion of summer break (May–August)?

RQ2. What are the differences of fourth grade students’ reading changes after summer break (May–August) by quartile, gender, and Title I status?

Methods

Participants

Participants in this study included elementary school students from a state in the southeastern United States who had participated in a state appropriation for supplemental reading and had taken an assessment during the end of fourth grade (Spring of 2016) and the beginning of fifth grade (Fall of 2016; N = 31,634). Participants were included in this study if they had both spring (April and May) and fall (August and September) ISIP-AR assessment scores (n = 5530) above a threshold score of 1000 (typically the lowest score that can be earned on the assessment; n = 5513). Table 1 presents the demographic data for the overall sample and differentiated by students’ quartile level.

TABLE 1 Demographics of the overall sample and differentiated by students’ quartile level

Overall			By Quartiles
			<25% n = 1379 (25%)	50% n = 1378 (25%)	75% n = 1378 (25%)	>75% n = 1378 (25%)
Gender (11% not reported)
Female	2349	47.9%	482 (41.2%)	551 (45.8%)	634 (50.8%)	682 (53%)
Male	2559	52.1%	689 (58.8%)	651 (54.2%)	615 (49.2%)	604 (47%)
Title I (no missings)
Yes	4,388	79.6%	1,235 (89.6%)	1,151 (83.5%)	1,087 (78.9%)	915 (66.4%)
No	1,125	20.4%	144 (10.4%)	227 (16.5%)	291 (21.1%)	463 (33.6%)
Race (6.7% not reported)
African American	917	16.6%	345 (27.2%)	287 (22.2%)	184 (14.2%)	101 (7.8%)
Alaska Native/American Indian	49	0.9%	10 (.8%)	15 (1.2%)	13 (1%)	11 (.9%)
Asian	212	3.8%	30 (2.4%)	37 (2.9%)	59 (4.6%)	86 (6.7%)
Hispanic	46	0.8%	14 (1.1%)	13 (1%)	12 (.9%)	7 (.5%)
Mixed/Two or more	16	0.3%	2 (.2%)	6 (.5%)	2 (.2%)	6 (.5%)
Other	502	9.1%	133 (10.5%)	139 (10.8%)	126 (9.7%)	104 (8.1%)
Pacific Islander	34	0.6%	9 (.7%)	19 (1.5%)	6 (.5%)	0 (0%)
White	3,367	61.1%	724 (57.5%)	775 (60%)	893 (69%)	975 (75.6%)

Note. Quartiles determined by students’ May 2015/2016 (grade 4) score.

Measures

Reading measures in this study included the overall reading score (derived from the subscales reading comprehension, vocabulary, and spelling, score on the ISIP-AR computer adaptive testing system for continuous progress monitoring) and the Reading Ability overall score was computed using Bayes expected A posteriori (EAP) after all subtests were completed and is based on the entire response set from three subtests. The test-retest reliability for Reading Ability overall score was 0.910 with these data, indicating strong test-retest reliability (Crocker & Algina, 2006).

All assessments within the investigation were ISIP-AR computerized adaptive tests (CAT) with item difficulty adjusting to student ability level through the adaptive item algorithm using item response theory (IRT; de Ayala, 2009). Marginal reliability (IRT analog to internal consistency reliability) is approximately 0.90 (Mathes, 2012). The development of the ISIP-AR CAT began with a literature review to determine the theoretical approach to measuring each subtest in the ISIP-AR. Items for the item pool were constructed from considering theoretical perspectives, reviewing state reading standards, and by a predetermined framework. Evidence of item content validity was standardized under a two-parameter logistic item response theory (2PL-IRT) model. Items that did not statistically fit were removed to correctly reflect the domain measured (Mathes, 2012). In addition, correlations between the ISIP-AR subscales and norm referenced measures, including the Test of Preschool Early Literacy (Lonigan, Wagner, Torgesen, & Rashotte, 2007, [TOPEL]), Peabody Picture Vocabulary Test-IV (Dunn & Dunn, 1999, [PPVT]), and the Early Literacy Skills Assessment (ELSA; DeBruin-Parecki, 2005) provided evidence of concurrent validity with data demonstrating large to very large criterion validity..

Data Analyses

Data were analyzed to examine changes in participants’ reading achievement scores. To examine differences within and between different groups, paired sample t-tests and Cohen’s d were calculated for each quartile and risk factor (i.e., gender, Title I). To detect differences in the development of fourth grade students’ reading achievement between the quartiles, repeated measure analysis of variance (ANOVA) were computed with the assessments (May–August) as within-subject variables and quartile as between-subject factors. Histograms of the frequency distribution of the achievement scores for May and August were examined and they indicated normal distribution.

Academic Level by Quartiles

To determine students’ academic level, their May reading test score was classified by quartile to determine mean differences in achievement. Quartiles are often employed in statistics to better explore and explain data. The median represents the second quartile, meaning one-half of the data occurs below the median score and the other one-half occurs above the median. The first quartile is a data point that represent one-quarter of the data while the third quartile represents the data point that represent up to three-quarters of the data or 75%. In this study, initial quartiles were determined using SPSS based on the students’ May reading score.

Results

RQ1. What are the observed changes in fourth grade students’ reading after the conclusion of summer break (May–August)?

Achievement Development

On average, participants did not experience a summer learning loss as evidenced by positive May to August change scores from grade 4 to grade 5. In fact, students’ reading achievement scores significantly increased from 1925.85 (SD  = 189.34) in May (grade 4) to 1942.88 (SD  = 172.91) in August (grade 5), indicating an average increase of 17 points (t [5512] = −12.01, p < 0.001) over the summer.

Movement

Table 2 presents the stability/movement of students’ achievement quartile from grade 4 to grade 5. As noted, summer gap for this investigation was defined and measured as those months in which the students did not complete an ISIP-AR (i.e., the months after their last spring assessment and before their first fall assessment). There was movement of students between quartiles from August to May, suggesting that (a) some students increased their rank standing (i.e., percentile) from grade 4 to 5, (b) some students remained about the same, and (c) some students decreased in rank standing.

TABLE 2 May to August quartile movement (frequency and percent)

May (grade 4)	August (grade 5)
	<25%	50%	75%	>75%
<25%	943 (68.4%)	344 (25%)	81 (5.9%)	11 (0.8%)
50%	348 (25.2%)	660 (47.9%)	324 (23.5%)	46 (3.3%)
75%	76 (5.5%)	335 (24.3%)	694 (50.4%)	273 (19.8%)
>75%	12 (0.9%)	39 (2.8%)	279 (20.2)	1048 (76.1%)

RQ2. What are the differences of fourth grade students reading by quartile, by gender, and by Title I status?

Achievement by Quartiles

Table 3 presents the results of the paired sample t-test between the assessments (May–August), differentiated by quartiles. Whereas students below the second quartile (or the 50th percentile) made gains in terms of points from the May to the August assessment, students above the 50th percentile evidenced a decrease in their assessment scores. Students below the 25% quartile made the greatest amount of growth during the transition from fourth to fifth grade with an increase of 78 points (d = 0.58) from May (grade 4) to August (grade 5), followed by students between the 25% and 50% quartile (+ 17 points, d = 0.25). Over the summer, students in the top quartile lost, on average, 24 points from May to August (d = 0.25).

TABLE 3 Means (M) and standard deviations (SD) in the assessments May (grade 4) to August (grade 5), and dependent statistical values of t-test for paired samples (t, df, p) and Cohen’s d (d)

Quartile	May M (SD)	August M (SD)	Diff.	t (df)	p	D
<25%	1691.05 (135.66)	1769.53 (136.80)	−78.49	−78.49	0.000	0.58
25–50%	1874.27 (30.32)	1891.11 (88.74)	−16.84	−16.84	0.000	0.25
50–75%	1983.77 (34.78)	1980.22 (89.17)	3.55	3.55	0.117	0.05
>75%	2154.50 (97.00)	2130.77 (122.10)	23.73	23.73	0.000	0.25

A review of the development of students’ reading achievement throughout the school year provides a more detailed picture. Figure 1 presents the mean scores of the monthly assessments from the beginning of the participants’ fourth grade school year until the end of their fifth grade. It becomes evident that students in the top quartile (as measured by their May achievement) made the greatest gains throughout the school year, followed by a decline over summer (May–August). In contrast, the bottom quartile of students made comparatively slower gains in reading achievement during the academic year (grade 4), followed by significant gains over summer.

Figure 1 Reading scores over the course of grade 4 and 5 by Quartile (as determined by students’ May score).

Achievement by Gender

Table 4 presents the results of the paired sample t-test between the May and August assessments, differentiated by gender. Male students score lower than female students on both assessments. During the transition from fourth to fifth grade – as measured by students’ end of fourth grade scores in May and their beginning of fifth grade scores in August – boys made greater gains than females with average gains of 20 points (d = 0.11).

TABLE 4 Means (M) and standard deviations (SD) in the assessments May (grade 4) and August (grade 5) and dependent statistical values of t-test for paired samples (t, df, p) and Cohen’s d (d) by gender and Title I Status

Females (n = 2349)						Males (n = 2559)
MMay (SD)	MAugust (SD)	Diff.	t(df)	p	d	MMay (SD)	MAugust (SD)	Diff.	t(df)	p	d
1952.23 (174.89)	1966.86 (159.18)	−14.63	−7.30 (2348)	0.000	0.09	1914.51 (199.91)	1934.92 (181.60)	−20.41	−9.06 (2558)	0.000	0.11
Non-Title I students (n = 4388)						Title I students (n = 1125)
MMay (SD)	MAugust (SD)	Diff.	t(df)	p	d	MMay (SD)	MAugust (SD)	Diff.	t(df)	p	d
2004.38 (183.70)	2011.78 (169.46)	−7.40	−2.58 (4387)	0.010	0.04	1905.72 (185.50)	1925.21 (168.68)	−19.49	−12.04 (1124)	0.000	0.11
Females, non-Title I (n = 503)						Males, non-Title I (n = 499)
MMay (SD)	MAugust (SD)	Diff.	t(df)	p	d	MMay (SD)	MAugust (SD)	Diff.	t(df)	p	d
2029.16 (175.27)	2036.03 (160.19)	−7.77	−1.91 (502)	0.057	0.04	2000.79 (178.31)	2006.75 (166.28)	−5.96	−1.27 (498)	0.204	0.03
Females, Title I (n = 1846)						Males, Title I (n = 2060)
MMay (SD)	MAugust (SD)	Diff.	t(df)	p	d	MMay (SD)	MAugust (SD)	Diff.	t(df)	p	d
1931.27 (168.86)	1947.77 (153.50)	−16.50	−7.19 (1,845)	0.000	0.10	1893.61 (199.29)	1917.52 (180.78)	−23.91	−9.36 (2,059)	0.000	0.13

Achievement Development by Title I Status

Students from non-Title I schools score significantly higher on both assessments than students from Title I schools (see Table 3). However, during the transition from fourth to fifth grade – as measured by students’ end of fourth grade scores in May and their beginning of fifth grade scores in August – Title I students made considerably greater gains with an average increase of 19 points (d = 0.11), as compared to 7 points (d = 0.04) for students from non-Title I students. Despite students from Title I schools evidencing the greatest growth over summer, they are not able to catch up with students from non-Title I schools. In fact, participants at Title I schools ISIP-AR reading score at the beginning of fifth grade were lower than the ISIP-AR scores for students at non-Title I schools at the end of fourth grade.

Quartile, Gender, and Title I Status

Given that male students and students from Title I schools score lower than their counterparts, it stands to reason that male student and students from Title I schools are overrepresented in the bottom quartile (see Table 1). The bottom quartile is composed of almost 90% of Title I students (as compared to 66% in the upper quartile) and almost 60% male students (as compared to 47% in the upper quartile).

To explain the data further and examine whether there is an interaction between the students’ reading development from the May to the August score and students based on their quartiles, mixed analysis of variance (ANOVA) was computed with the assessments (May–August) as within-subject variables and quartile as between-subject factors. There was a significant interaction effect between the assessments and the groups (F [4, 5509] = 280.365, p = 0.000, partial η² = 0.132), suggesting that the predictors explain about 13.2% of the variance in overall reading summer change from May to August.

Discussion

Mitigating summer learning loss through summer reading remediation may help those students in the most need, students in the lowest quartiles, and those from low SES backgrounds (Mraz & Rasinski, 2007). Consequently, increasing attention has been devoted to the effects of summer vacation on the development of students’ reading achievement (Schaffner & Schiefele, 2016). In extending previous research on the effects of summer vacation on reading achievement, the present investigation not only included the overall trajectory of reading development over summer but also accounted for students’ academic achievement level, their Title I status, and gender.

Changes in Reading Achievement Over Summer

The results of the present study demonstrated that when considering the achievement scores of the sample, achievement gains over summer were made as measured by the ISIP-AR scores. These results contrast with some of the previous studies on the effects of summer vacation on students’ reading achievement (Cooper et al., 1996; Mraz & Rasinski, 2007), but concur with others (Helf et al., 2008). Potential reasons for participants’ increase in reading achievement may relate to access and opportunity. Students tend to practice reading over summer, given that reading activities take place in students’ leisure time and thus during summer vacation (Schaffner & Schiefele, 2016). Likewise, the increased access to technology-based reading applications and books may contribute to improved reading (Cheung & Slavin, 2011; Taylor & Parsons, 2011).

Students in the lowest quartile made less gains throughout the academic school year; however, they made greater gains over the summer months, contrasting previous findings that have demonstrated that high achieving students learned more during the summer than their average achieving counterparts (Rambo-Hernandez & Mccoach, 2015). One potential explanation for the greater reading growth of those who initially scored in the lowest quartile is that summer school may be mandated (Matsudaira, 2008). Higher achieving students are less likely to attend mandatory summer school programs than low-achieving students during summer vacation, and focused substantial emphasis on reading can support student progress (Bitter, O'Day, Gubbins, & Socias, 2009). In addition, Helf et al. (2008) agreed that summer reading gains were greater among students who scored lower than the students who scored higher at the end of the school year. Nevertheless, looking at the long-term development of students’ reading achievement over both school years, it seems that students’ in the lowest quartile benefited the most during the summer which is in line with prior research indicating that at-risk students evidence higher gains than those not at-risk (Munro, 2017). In summary, it appears that all students – even those that evidenced high reading achievement scores at the end of the school year – continue to participate in reading activities during the summer to sustain their reading skills.

Students from Title I schools not only evidenced reading achievement gains over summer, but the gains were greater than the achievement evidenced by students from non-Title I schools. As indicated in contrast to previous research, many of the studies have pointed to significant reading losses during the summer, for students from low SES (Downey et al., 2004; McCoach, O'Connell, Reis, & Levitt, 2006). Summer School programs are often funded through Title I program. Federal Title I funds provide additional instructional support (like summer reading programs) for students who are at most risk of failing academic standards (e.g., low-income and low-achieving students) thereby, providing students’ greater access to learning opportunities than their average to high achieving counterparts (https://nces.ed.gov/fastfacts/). Additional access and support for students’ reading development can lead to improvements in their reading skills (Allington & McGill-Franzen, 2003). Therefore, one potential explanation for the results is that these students continue to read and participate in reading activities over the summer months and/or participate in summer reading programs (for instance provided through Title I funds). Kim and Quinn’s (2013) meta-analysis results identified that summer reading interventions seem to be “particularly effective for low-income children” (p. 418). Similarly, two study from McDaniel, McLeod, Carter, and Robinson (2017) and Bowers and Schwarz (2018) demonstrated that a summer program could prevent summer reading loss among students from low income backgrounds.

Overall, a potential explanation for both the reading gains for low-achieving students and Title I students is that over the summer, these students experience increased motivation. Schaffner and Schiefele (2016) found that intrinsic reading motivation before summer vacation is positively associated with changes in reading comprehension over summer vacation, highlighting the crucial importance of motivation for the development of reading achievement. Students might have become engaged and motivated, initiating frequent reading activities during summer, which in turn, promoted reading achievement. In addition, the modality for reading may contribute to motivation for reading. Digital books accessible through digital devices including smartphones and tablets often motivate students to read (Hess, 2014).

Lastly, the exploration of gender revealed interesting results. Male students scored lower on all assessments than female students, however, during the transition from fourth to fifth grade, they made greater gains than females with average gains of 20 points (d = 0.11). Moreover, male students from Title I schools made the greatest gains from May to August. A potential explanation for these results is that these students started off with the lowest reading achievement scores by far and thus had the most room for improvement (Munro, 2017).

Limitations and Implications for Future Research

The study’s limitations included insufficient information regarding contributing variables such as: (a) students’ participation in summer school programs, or other community sponsored reading programs (library); (b) students access to personal reading materials; or (c) students use of online or mobile devices for reading. Another limitation of the study was that only fourth grade students were included in the investigation. While the results may be generalizable to other fourth grade students, it is unclear if the results would be similar for students in the formative years of learning to read, or those in middle and/or high school. Finally, the potential explanations noted in the discussion section are speculative and require further research.

In contrast to other studies (Klibanoff et al., 1981; Sandberg-Patton & Reschly, 2013; Slates et al., 2012), our findings identified that some students can make reading gains during the summer. Nevertheless, since information regarding students’ participation in formal, informal, and independent school-based or non-school-based reading programs and activities during the summer vacation (e.g., summer enrichment activities; summer school; community and day camp programs; and the use of software applications, computers, and mobile devices) was unknown, any hypothesis would be conjecture. Evaluating students’ involvement with reading during the summer may provide answers as to the efficacy of programs, activities, or personalized learning programs that could negate summer learning loss. The study considers one summer’s worth of data rather than examining several summers of data which would establish longitudinal patterns. Analyzing longitudinal data using a curriculum-based measure across several years may help to explain the extent learning loss may be recovered in subsequent school semesters or years, and the extent to which the pattern of decline continues, or if the learning loss or gain is cumulative (Cooper et al., 1996; Heyns, 1978; Sandberg-Patton & Reschly, 2013).

Implications from this study includes emphasizing the importance for students at all academic levels engaging in cultivating their reading skills during summer months, including engaging in informal reading activities. Programs such as summer school, school required summer reading, library and community-based reading programs and contests have been found to have varying effectiveness relative to negating summer reading loss (Allington & McGill-Franzen, 2013; Allington et al., 2010; Kim & Guryan, 2010; Kim et al., 2012; Kim & White, 2008; Slates et al., 2012). Continued research is needed to examine formal (summer school, required reading list, and reading camps) and informal reading activities (computer programs, student choice in reading, and reading competitions) to inform practice and policymakers. Based on our findings and previous research examining effectiveness of summer reading programs, stakeholders (e.g., educational policymakers, researchers, school administrators, educators, parents, and students) should consider being purposeful in seeking creative solutions (beyond what is typically done) to mitigate students’ reading loss during the summer (McDaniel et al., 2017). Access to reading resources (e.g., book, digital books, and digital applications or programs) need to be accessible to students during the summer from community organizations, schools, and libraries (Allington et al., 2010; Allington & McGill-Franzen, 2003; Heyns, 1978; Kim, 2004; Phillips & Chin, 2004). Research regarding the use of these reading support devices, computer programs, and mobile applications for reading in the summer is minimal and needs further investigation.

Conclusions

In our knowledge-based economy, understanding elementary school students’ development in reading achievement over summer as well as potential summer learning loss provides critical information for educators and others to consider as they seek to close a national and international achievement gap. Findings from the current study extend what is known about the summer learning effect related to reading. In addition, the role of students’ quartile level related to summer reading learning loss/gains adds new information to the literature base.