4,360
Views
2
CrossRef citations to date
0
Altmetric
Research Article

Virtual Kindergarten Readiness Programming for Preschool-Aged Children: Feasibility, Social Validity, and Preliminary Impacts

, , , &

ABSTRACT

Research Findings: The global COVID-19 pandemic prevented the implementation of in-person summer learning programs designed to improve school readiness for entering kindergartners. Thus, we conducted the current study examining the feasibility, social validity, and preliminary impacts of a virtual summer learning program. Ninety-one preschoolers and their caregivers participated in a 4-week program involving one weekly teacher-caregiver meeting, two weekly Watch Together home learning activities, two weekly Play Together home learning activities, one or two weekly Read Together home learning activities, and one or two weekly teacher-child video chat lessons. Recruitment and participation indicated high levels of interest. Caregivers reported high levels of satisfaction with the program and teachers reported moderate to high levels of child engagement during video chat lessons. Children showed significant gains on emotion knowledge, patterning, and alphabet knowledge. Practice or Policy: This study demonstrates the promise of a virtual kindergarten readiness program and suggests that the combination of educational media and video chat instruction may be effective for promoting children’s skills. A virtual intervention could be beneficial in other contexts, such as in rural areas where it would be challenging for families to participate in an in-person program or for immunocompromised or seriously ill children who are unable to attend in person.

The global COVID-19 pandemic prevented the implementation of in-person summer learning programs designed to improve school readiness for entering kindergartners. These types of programs are especially important for children with little to no early preschool experience because children’s skills at kindergarten entry predict academic achievement throughout their schooling. Prior research has established that these programs can be effective when conducted in-person and, with the availability of technology and media resources, it may be possible to implement a program that avoids face-to-face contact but also has the potential to improve outcomes for young children. However, no prior studies have investigated remote summer programming to improve kindergarten readiness. Thus, we conducted the current study examining the feasibility, social validity, and preliminary impacts of a virtual summer learning program. This investigation is significant given the potential for future pandemic-related closures and other circumstances necessitating social distancing and stay-at-home orders for all or certain members of the population.

Kindergarten Readiness

Kindergarten readiness is a term used to describe the desired age-appropriate language, literacy, math, and social-emotional skills for children to have acquired at kindergarten entry; with access to these skills, children arrive to kindergarten with the abilities to effectively communicate with peers and teachers and engage in structured tasks required in the kindergarten setting (Rimm-Kaufman & Pianta, Citation2000; Rimm-Kaufman et al., Citation2000). Research shows that children who enter kindergarten with higher levels of these readiness skills are well-positioned to continue to build their knowledge and skills at a steady rate, and subsequently experience higher academic achievement relative to children with lower levels of these skills (G. J. Duncan et al., Citation2007). Indeed, basic skills are the foundation upon which more complex skills can be acquired (Cunha & Heckman, Citation2007). Further, children with stronger initial skills are more likely to receive positive reinforcement and experience positive teacher-child relationships, which encourages further learning (G. J. Duncan et al., Citation2007). Importantly, these early academic skills are predictive of outcomes decades into the child’s future (Rabiner et al., Citation2016).

A large number of studies have investigated ways to enhance development of key readiness skills in pre-kindergarten settings (Bierman et al., Citation2008, Citation2014; Bleses et al., Citation2018; Hamre et al., Citation2012). Typically, these studies involve equipping teachers with explicit instructional strategies that target such readiness skills as self-regulation, vocabulary, and early math concepts, and many trials have shown positive impacts over the pre-kindergarten year (e.g., Bleses et al., Citation2018; Hamre et al., Citation2012), with some evidence of sustained impacts in kindergarten and beyond (e.g., Bierman et al., Citation2014). However, many children do not have access to quality pre-kindergarten programs in the year preceding kindergarten entry, thus presenting a barrier to such readiness-building experiences. This is particularly true for children in low-income families, with recent evidence indicating that only 40% of children in very-poor homes (annual household income of less than 10,000 USD) participate in center-based preschool programs (Friedman-Krauss et al., Citation2019).

Summer Programming to Improve Kindergarten Readiness

Some researchers have explored the potential for improving children’s kindergarten readiness during the summer prior to kindergarten start (e.g., R. J. Duncan et al., Citation2018; Khan et al., Citation2017). The extent to which such programming may boost children’s readiness skills in a relatively short period of time is an important area of investigation. Some evidence suggests that relatively short but intensive interventions directed toward preschool-aged children can significantly accelerate key readiness skills. For instance, Justice and Ezell (Citation2002) showed that an 8-week shared-reading program implemented in Head Start programs accelerated children’s growth in alphabet knowledge and other literacy skills compared to a control program. More recently, Leyva and colleagues examined impacts of a 4-week math-focused intervention for Latino kindergartners as implemented with their parents, finding a significant dose-response relationship between program exposure and children’s math gains: children with lower math skills at program start gained significantly from the intervention if their parents attended at higher rates (Leyva et al., Citation2018). Such work has occurred during the academic year but suggests the potential for summer-based programming focused on readiness skills to accelerate children’s skill development in a relatively short period of time.

To date, two studies of which we are aware have examined the impacts of kindergarten readiness programming conducted in the summer prior to kindergarten on children’s acquisition of key readiness skills. R. J. Duncan et al. (Citation2018) evaluated a 3-week summer program run by a school district targeting early academic skills, social-emotional behaviors, and self-regulation among entering kindergartners who had no prior preschool experience. Results showed that participation in the program was associated with increased growth in children’s math, literacy, and self-regulation skills during the fall of kindergarten compared to expected development. A similar program, Summer Success (Khan et al., Citation2017), targeted children from low-income backgrounds entering kindergarten with little-to-no formal early childhood education experience and was designed to improve children’s language, literacy, math, social-emotional, and motor skills. Offered over a 4-week program in the summer months, pilot findings for 24 participating children showed that children improved significantly in language and literacy, math, and social-emotional skills. In addition, effect sizes were medium to large in size, noteworthy given the relatively short duration of the program. Although the Khan et al. (Citation2017) Summer Success program was not evaluated with an experimental design, findings parallel those of R. J. Duncan et al. (Citation2018) to show that intensive summer programming can facilitate readiness skill development in children prior to kindergarten entry.

Challenge Due to Pandemic Circumstances

For children entering kindergarten in the fall of 2020, the potential for participation in site-based readiness programs in the summer of 2020 was preempted by the global COVID-19 pandemic, which resulted in school and childcare closures worldwide. Yet, for many children entering kindergarten in the forthcoming academic year, particularly those with little or no formal early-learning experiences, the need for readiness programming continued. Indeed, even children who were in preschool environments prior to the pandemic likely had the last several months of their pre-kindergarten year disrupted, and those who were not in preschool were likely to have experienced limited opportunities for out-of-home learning (e.g., museums, libraries), as well as potential disruptions at home due to caregivers’ job losses, stress about the pandemic and associated financial consequences, and struggles balancing work and childcare. The pandemic circumstances presented the possibility for assessing the feasibility, social validity, and potential impacts of summer readiness programming offered virtually, with design based on the emerging research base concerning the viability of using technology and media to enhance young children’s skill development (Dore et al., Citation2019; Gaudreau et al., Citation2020).

Video Chat and Educational Media

Research suggests that although very young children have trouble learning from video (Anderson & Pempek, Citation2005), the back-and-forth interaction that can occur over video chat can support children’s learning (Troseth et al., Citation2006). Indeed, recent research shows that by age 4, children learn equally well from a storybook read live and over video chat, with both conditions showing learning relative to a control condition (Gaudreau et al., Citation2020). Children can also gain some skills from educational media, with early research demonstrating the success of Sesame Street for reaching and improving the skills of children from disadvantaged backgrounds (Bogatz & Ball, Citation1971). More recent studies showed that viewing educational TV during early childhood was positively related to school readiness at age 3 (Anderson et al., Citation2001; Wright et al., Citation2001), and authors reported positive impacts of media interventions on both language and literacy (Hurwitz, Citation2019; Mares & Pan, Citation2013; Penuel et al., Citation2012) and math skills (Mares & Pan, Citation2013; McCarthy et al., Citation2018). Notably, research has shown that when adults use media with children, they can improve children’s comprehension and learning by discussing the material with them (e.g., Collins et al., Citation1981; Watkins et al., Citation1980). For example, when Strouse et al. (Citation2013) trained parents to ask questions while watching videos with their toddlers, children learned more vocabulary words (see also Strouse & Troseth, Citation2014). Other studies find that techniques such as pointing, repetition, attention elicitation, question-asking, and providing feedback may facilitate children’s learning from educational media (Neuman et al., Citation2019; Reiser et al., Citation1988; Samudra et al., Citation2019). Collectively, these studies suggest that combining video chat instruction with caregiver-supported educational media use in the home setting could be a viable way to deliver a summer readiness intervention for entering kindergartners under pandemic conditions.

Early Childhood Parenting Programs

Though a greater emphasis on using digital technology to support children in the home has arisen during a global pandemic, strategies to provide support to parents and young children in their home environment are not new. Following the assumption that parents and caregivers are children’s first and most influential teachers (e.g., Landry, Citation2020; Zigler et al., Citation2008), a wide range of programs have been developed to influence child outcomes by helping parents and caregivers in a variety of ways, including providing education on child development and supportive interactions, yielding positive results overall (e.g., Andrew et al., Citation2020; Baker et al., Citation1999; Lahti et al., Citation2019; Landry et al., Citation2012; Leung et al., Citation2020; Wagner et al., Citation2002; see Sweet & Appelbaum, Citation2004 for a meta-analysis). For example, the Play and Learning Strategies (PALS) program relies on a trained coach to facilitate weekly sessions with parents of children aged birth to 5 years geared toward developing a supportive and responsive relationship between parents and children (Landry, Citation2020). Similarly, the Parents as Teachers (PAT) program utilizes weekly, bi-weekly, or monthly home visits, depending on what the family needs and prefers, to allow PAT-certified parent educators to teach parents about appropriate developmental expectations for their child and demonstrate and practice strategies to help promote learning at home (Zigler et al., Citation2008). Notably, these programs require parents to practice the weekly strategies that are modeled for them so they can successfully implement them on their own (Landry, Citation2020; Zigler et al., Citation2008). Both of these interventions focus on working with the parents to alter their behavior and expectations when engaging with their young children, resulting in many positive effects on child outcomes. The PALS program has been associated with positive effects pertaining to children’s language and social skills (Landry et al., Citation2008), while children of PAT participants demonstrate greater school readiness and third grade achievement (Zigler et al., Citation2008) as well as improvements in reading, math, and ELL English proficiency (Lahti et al., Citation2019).

While many early childhood parenting programs have the general goal of improving caregivers’ responsive interactions with their children, others attempt to alter practices related to more specific child skills. In a recent study in which parents were provided and asked to follow a family guide focused on strategies to support children’s social and emotional skills, increases were observed in children’s social-emotional competence with simultaneous decreases in reported problem behaviors (Thomson & Carlson, Citation2017). After an initial home visit, parents worked through 8 weeks of lessons, reporting via phone or e-mail how much of the assigned reading they had done, how many written questions had been answered, and the number of times the strategies for that week and the prior weeks had been utilized, with a final in-person home visit to collect the workbooks that had been used (Thomson & Carlson, Citation2017). Encouragingly, while programs like the PALS and PAT often rely on frequent, face-to-face interactions with parents, Thomson and Carlson (Citation2017) study suggests that meaningful change can be obtained through virtual check-ins on program implementation as well. These findings are supported by results from Landry et al.’s study providing remote delivery of the PALS program, demonstrating that when coaches and parents used tablets for their interactions, effects were even stronger than the initial trial, particularly regarding children’s self-regulation skills (as cited in Landry, Citation2020, p. 35). This evidence supports the idea that, in pandemic circumstances, virtual intervention programming can be beneficial for children, influencing both academic and nonacademic skills.

Beyond the delivery format, other important components of these programs align with features of the Summer Success at Home program used within the current study. A meta-analysis of sixty such programs found that a majority were directed toward populations with some form of risk factor (Sweet & Appelbaum, Citation2004), largely families with backgrounds consistent with low socioeconomic status (Leung et al., Citation2020; Wagner et al., Citation2002). In terms of the intervention itself, 28.3% of studies in Sweet and Appelbaum (Citation2004) meta-analysis provided materials to families and 20% offered home-based early childhood education, both components of the current program. Particularly important to many programs is the provision of books and parents’ interactions with their children during their use (e.g., Baker et al., Citation1999; Landry et al., Citation2012; Zigler et al., Citation2008), another element that was included in the Summer Success at Home program. Overall, this program aligned with many foundational components of early childhood parenting programs. However, it expanded upon the format of such programs in that teachers, similar to coaches in many programs, not only interacted with parents to facilitate their understanding of learning strategies but also worked with the children directly. Thus, the current study serves to advance the literature in this area with a virtual version directly targeting both caregivers and children.

Theoretical Framework

We situate this study within the framework of the social systems approach to early intervention (Dunst, Citation2000). Early intervention is important to provide support to young children and their families to positively influence child and family functioning. Providing such supports during early childhood is ideal, given that later learning builds on the foundation laid in the early years (G. J. Duncan et al., Citation2007). The social systems approach to early intervention focuses on enhancing competence and positive functioning rather than focusing on remediation of a problem, follows a strength-based model to help people use their competencies to strengthen functioning, and encourages professionals to be responsive to families’ desires and concerns (Dunst, Citation2000). We follow this approach by working with caregivers to focus on what they can do to support their child’s learning at home and by engaging with families to support their child’s kindergarten readiness based on their identified needs and situation.

The Current Study

To summarize, based on the critical need to promote kindergarten readiness during a global pandemic that prevented in-person programming, coupled with research suggesting the potential value of learning over video chat and from educational media, we conducted the current study with three primary aims. We focus on a virtual kindergarten readiness program, which we defined as a program to support young children’s academic and socioemotional preparation for beginning the kindergarten year via activities that involve video chat, phone, and delivery of materials to the home rather than face-to-face instruction or interaction. To our knowledge, virtual kindergarten readiness programs have not been studied or reported on in the literature.

First, we aimed to assess the feasibility of a virtual summer kindergarten readiness program, Summer Success at Home. Feasibility focuses on understanding whether an intervention can work and includes assessing recruitment capability, sample characteristics, and data collection procedures (Orsmond & Cohn, Citation2015). In this case, we operationalized feasibility as assessing the extent to which families could effectively be recruited under pandemic conditions to enroll their entering kindergartners in a 4-week virtual summer readiness program and, in turn, whether a recruited sample could be maintained over the 4-week program of virtual summer learning. We viewed feasibility as an important aim to investigate, given that the pandemic conditions under which Summer Success at Home was conducted were unprecedented. Thus, we asked the research question: Is Summer Success at Home feasible under pandemic conditions? We hypothesized that the program would demonstrate adequate feasibility, with successful recruitment into the program and high levels of sample maintenance across 4 weeks.

Second, we aimed to assess the social validity of the program by examining parents’ and teachers’ satisfaction with the virtual program and children’s engagement during lessons. Social validity is defined as the social importance and acceptability of treatment goals, procedures, and outcomes (Foster & Mash, Citation1999) and was a key consideration of this study, given concerns about screen time and young children and the reliance on technology for implementing Summer Success at Home (Lissak, Citation2018). Thus, we asked the research question: Does Summer Success at Home have a high level of social validity? We hypothesized that the program would demonstrate a high level of social validity via parent and teacher satisfaction and child engagement.

Third, we aimed to assess preliminary impacts of the program; specifically, we examined the extent to which children showed improvements in targeted readiness skills over the 4-week program period. While prior summer readiness programs have shown positive effects in targeting skill areas, the impacts of virtual programming are unknown. Thus, we asked the research question: What are the preliminary impacts of Summer Success at Home on children’s readiness skills? We hypothesized that the program would demonstrate positive preliminary impacts on skills over the 4-week period.

The Summer Success at Home program was conceived of rapidly during the later-spring of 2020 as a means to virtually deliver an extant school-readiness program (Summer Success), which had been implemented and evaluated with four prior cohorts of entering kindergartners. Operating under a legislated stay-at-home order in the state of Ohio, our team needed to rapidly revise program materials for a virtual offering, develop partners for recruiting families to the program (given that our typical recruitment options, such as working with a home-visiting program, were not viable), and even develop strategies for ordering and organizing program materials, given that the university operating the program had all staff working from home and all non-essential purchases halted. Given these constraints, we acknowledge that this study does not feature a causally interpretable research design and that there are important ways in which the program can be improved (see Results and Discussion). Nonetheless, the present study offers an important opportunity to consider the potential for virtual learning opportunities to provide a means to accelerate readiness among entering kindergartners, which may be relevant to children without access to site-based experiences and for whom virtual experiences may be beneficial.

Thus, although this was not an experiment and therefore has no true independent variable, the outcome variables examined to address our research questions were (1) child recruitment, participation, and retention in the program (feasibility), (2) children’s engagement during the program (social validity), (3) teachers’ and caregivers’ satisfaction with the program (social validity), and (4) gains in children’s language, literacy, math, and social-emotional skills over the intervention period (preliminary impacts). In essence, this is a feasibility study, and we establish triangulation of evidence by using children, teachers, and caregivers as sources of data (Patton, Citation1999).

Materials and Method

Participants and Setting

This study was conducted in Franklin County, Ohio, the boundaries of which include Columbus, the state capital and the 14th largest city in the country. Recruitment activities were designed to target enrollment toward children entering kindergarten who were from lower-income homes, although a specific income-based criterion was not applied. To recruit families, our approach involved soliciting interest information from as many families as possible up to 3 weeks prior to program start, contacting families to gather information about level of interest and household income, and then prioritizing enrollment for lower-income families as we filled up to 100 slots until the program was fully subscribed at 1 week prior to program start.

Recruitment involved partnering with the lead pediatrician at a pediatric primary care center located in a neighborhood prioritized by the city for school-readiness investments based on kindergarten-readiness scores; one-third of slots were targeted to children served in this primary care center. In addition, recruitment also targeted families with entering kindergartners in one large local school district and local nonprofits serving lower-income families with children. In all cases, recruitment involved circulating a family-friendly flyer that invited contact with the study team via multiple means: texting, calling, providing information via a website, or filling out a brief form and giving it to the provider who shared the flyer.

In total, the enrollment goal of 100 families with entering kindergartners was reached 1 week prior to program start. At an initial enrollment meeting with caregivers, teachers obtained basic information on demographics and family characteristics, as well as information about what kinds of supports families needed in regard to the kindergarten transition. During the first week of the program, 91 families had direct contact with program staff and thus are considered the as-treated sample in this study, representing 91% of the 100 originally enrolled.

Institutional review board approval was not required for this activity, as it was considered program implementation and quality improvement (QI) rather than research. The review board at our institution considers analyses conducted on de-identified data that was originally collected for non-research purposes to not be human subjects research.

Caregivers and Children

The present study included 91 caregivers and their preschool-aged children (47 boys). Children averaged 63 months of age (SD = 4) and ranged from 53 to 72 months. A brief caregiver questionnaire was completed at enrollment, providing basic demographic information about the children and their families. Per caregiver report, 48 (52.7%) of families had two adults living in the home, whereas 11 (12.1%) of families had three or more adults in the home and 18 (19.8%) had one adult in the home (14 [15.4%] unreported). Sixteen families (17.6%) had one child in the home, 29 (31.9%) had two children in the home, and 38 (41.8%) had three or more children in the home (8 [8.8%] unreported). Three (3.3%) families reported speaking only Spanish in the home, two reported speaking only Arabic or Arabic and English (2.2%), one family reported speaking only Mandarin at home (1.1%), and one family reported speaking only Somali at home (1.1%), whereas the remainder reported speaking only English in the home (77 families [84.6%]; 7 [7.7%] unreported). In terms of race and ethnicity, 33 families (36.3%) identified as Black or African American, 29 families (31.9%) identified as White, 12 families (13.2%) identified as biracial or multiracial, seven families (7.7%) identified as Hispanic, and four families (4.4%) identified as Asian (6 families [6.6%] unreported). Total annual household income ranged from 0 USD to 110,000 USD and averaged 40,843 USD (Median = 33,120 USD).

Teachers

Summer Success at Home was staffed by a project supervisor, who also served as teacher to 10 children, and an additional six early educators who were on paid leave from a closed early-learning center, due to the pandemic. The six educators each were assigned to work with 15 children; assignments were generally random except that a Spanish-English bilingual teacher was assigned to work with Spanish-speaking families. All of the teachers and the project supervisor were female; three had Associate degrees in Early Childhood or Early Education and four had Bachelor’s degrees (three in Education, one with a focus on Early Intervention). The teachers had an average of 11.4 years of experience teaching (SD = 8.3 years, Range = 1.5 to 25 years).

Procedures

Materials

Caregivers were provided with a tablet, four children’s books (one for each week of the program), and materials for home learning activities, including one-page printouts describing each activity. Families were also provided with hot spots if they indicated that their home internet access was unreliable. The tablet was provided prior to the beginning of the program and packets of materials were provided weekly. No-contact material drops were made each week of the program by children’s teachers to homes.

Program Structure

The virtual summer readiness program consisted of five active ingredients which occurred over the span of 4 weeks: one weekly teacher-caregiver meeting, two weekly Watch Together home learning activities, two weekly Play Together home learning activities, one or two weekly Read Together home learning activities, and one or two weekly teacher-child video chat lessons (see Appendix A for an overview of the program). All active ingredients were explicitly designed to target one or more readiness skills in three areas: language and literacy skills (phonological awareness, letter knowledge, vocabulary, narrative), math (counting/cardinality, patterning, classification), and social-emotional (emotion regulation, conflict resolution).

The first ingredient was one weekly video chat or phone meeting between the teacher and the caregiver. These meetings included a review of the previous week’s lessons, an overview of the new materials delivered, and a discussion of the concepts being covered that week. Caregivers could use this time to discuss any challenges that arose during the previous week or ask any questions they had about their children’s kindergarten readiness.

The second ingredient was two weekly Watch Together educational media activities for the caregiver and child to engage in together (see Appendix B for an example). Watch Together activities consisted of a pre-viewing activity to familiarize children with the material, an episode of an educational television show for caregivers and children to watch together, suggestions for how caregivers can support children’s learning from the episode while watching, and post-viewing discussion points. Television shows included Super Why! (literacy), Daniel Tiger’s Neighborhood (social-emotional), and PEG + CAT (math).

The third ingredient was two weekly Play Together activities for the caregiver and child to engage in together (see Appendix C for an example). Play Together activities consisted of a game or playful learning activity for caregivers to do with children using provided materials. For example, learning activities included practicing patterns with toy dinosaurs or using music and songs to discuss different emotions the child might feel.

The fourth ingredient was one or two weekly Read Together activities (see Appendix D for an example). Read Together activities consisted of a pre-reading activity, suggestions for engaging children during reading, and a post-reading game or discussion points. For example, caregivers might talk to children about the title and author of the book, read a book about feelings, and then have children draw a picture of how they feel when they are angry and practice taking deep breaths to calm themselves down. Books included Chicka Chicka Boom Boom by Lois Ehlert, Ahn’s Anger by Gail Silver, Pitter Pattern by Joyce Hesselberth, The Doorbell Rang by Pat Hutchins, Ish by Peter M. Daniels, and Dr. Seuss’s ABC: An Amazing Alphabet Book! by Dr. Seuss.

The fifth ingredient was individual weekly video chat lessons between the teacher and the child (see Appendix E for an example). During the lessons, teachers engaged the child in a discussion about the educational television show they watched in the most recent Watch Together activity and then read a book and completed a learning activity supporting the same skill. During both the reading and the activity, teachers used strategies to adjust the lesson to the child’s level to provide more or less support or challenge as necessary. For example, in a literacy lesson, a teacher might focus on letter identification with fewer letters for a child that needs more support and expand to more letters and identifying what letter comes next for a child who needs more challenge. There was one lesson in Week 1 and Week 4 and two lessons in Week 2 and Week 3; see Appendix A.

Measures

Measures collected during this study were fourfold: (1) child recruitment, participation, and retention in the program, (2) children’s engagement during the program, (3) teachers’ and caregivers’ satisfaction with the program, and (3) children’s language, literacy, math, and social-emotional skill at program start and completion.

Child Participation

Participation in the video lessons was measured using attendance logs maintained by teachers during each session. Participation was measured as the percentage of individual video chat lessons each child attended (6 possible).

Child Engagement

To measure children’s engagement during the video chat lessons, teachers rated children’s engagement in activities on a 4-point scale, where 0 indicated Not at all engaged, 1 indicated Engaged for less than half of the lesson, 2 indicated Engaged for more than half of the lesson, and 3 indicated Engaged for all of the lesson. Teachers also rated children’s engagement in conversation during the lesson on a 4-point scale where 0 indicated Not at all engaged, 1 indicated Slightly engaged, 2 indicated Moderately engaged, and 3 indicated Very engaged.

Child Readiness Skills

To measure children’s readiness skills at the start and close of the 4-week Summer Success at Home program, children were administered three sets of measures focused on literacy, math, and social-emotional skills. Measures were administered in a one-on-one Zoom call involving the child and their teacher. Teachers followed an ordered script for implementing these measures, providing scripted language and guidance for prompting, and a PowerPoint presentation was used to present stimuli via screenshare. When responses required children to indicate the correct answer from multiple choices, the response options were placed on backgrounds of different colors and children were asked to indicate which color was correct. We describe here the measures in the order they were administered.

First, teachers administered a social-emotional measure involving 14 items of two types designed to assess children’s understanding of seven basic emotions (scared, happy, angry, sad, disgusted, surprise, and all right [neutral]), adapted from Pons et al. (Citation2004). The first seven items involved presenting a slide displaying children’s faces exhibiting four of the basic emotions. Children were asked to point to the picture indicating a specific emotion (e.g., “Which person feels sad?”). The next seven items involved presenting a brief scenario involving a child and an emotionally laden experience (e.g., “A girl can’t go outside because it is raining … ”), and then asking the child to identify which of the four basic emotions best represented how the child was feeling (e.g., “How is this girl feeling? Is she happy, alright, angry, or disgusted?”). For the 14 items, responses were scored as incorrect (0) or correct (1) for a total of 14 points possible.

Second, teachers administered a sequence of three math tasks to assess counting, cardinality, and patterning. Initially, each child was asked to count as high as they could. The last number the child counted correctly was recorded and children were stopped once they made a mistake or reached 100. To assess cardinality, or the number of items in a group, children were shown five different PowerPoint slides with dots displayed on them, asked to count the dots aloud, and then told to report how many dots were pictured. Children received one point (1) if they correctly counted and stated the number of dots shown, a half point (½) if they made a mistake while counting but still stated the correct number, and no points (0) if they incorrectly reported the total number of dots, regardless of the correctness of their counting, for a total of five possible points on this task. Third, to measure patterning ability, children were shown eight pattern images to complete on individual PowerPoint slides, adapted from Zippert et al. (Citation2019). Two problems required each child to complete the pattern by picking the missing final item from three displayed image options and answers were scored as correct (1) or incorrect (0). The next three questions had the child select the missing bead in the pattern from three displayed options, again scored as correct (1) or incorrect (0). The last three assessment items tested children’s abilities to extend patterns by asking them to select the final four smiley faces of the appropriate color to continue the pattern shown. Children picked the item for each slot individually from three options shown and points were earned only if all four choices in each pattern were correct (1), otherwise the answer was said to be incorrect (0), making up three of the eight total possible points on this patterning task.

Third, teachers administered an alphabet task in which each letter of the alphabet, first in uppercase and then in lowercase, was presented to the child who was asked to name it. Letters were presented in a random order. A total of 52 points were possible, with letters named correctly receiving one point.

Fourth, teachers administered the Get Ready to Read! (GRTR) screener (Whitehurst & Lonigan, Citation2001), a norm-referenced screening assessment of core emergent literacy skills. This task consists of 25 items assessing phonological awareness and print knowledge skills. Thus, a total of 25 points were possible.

Fifth, teachers administered the Head-Toes-Knees-Shoulders task (McClelland et al., Citation2014), a measure of behavioral self-regulation. In the task, children first respond to the prompts “touch your head,” “touch your toes,” “touch your shoulders,” and “touch your knees” naturally and then are asked to switch the rules, e.g., touching toes when the teacher says, “touch your head.” If children succeed in this version, the rules are switched again. Due to error in administration, nearly one-half of our sample was administered this task out-of-line with test protocols. Thus, we do not analyze these data further.

Caregiver Satisfaction

All caregivers were asked to complete a survey asking how they would rate (1) their child’s experience in Summer Success at Home, (2) their child’s interaction with the Summer Success at Home staff, (3) their experience with the Summer Success at Home staff, (4) the overall quality of the Summer Success at Home program, and (5) their overall experience in Summer Success at Home. Each of these items was rated on a 5-point scale from Poor to Excellent.

Results

Feasibility

Feasibility was conceptualized in this study in terms of recruiting and retaining caregivers and children in the 4-week program. In terms of recruitment, we noted previously that we successfully recruited 100 caregivers to participate by 1 week prior to program start, and 91% of caregivers started the program. Thus, it is feasible to recruit families with entering kindergartners into virtual programming.

In terms of retaining caregivers and children through the program duration, we examined the percentage of children who participated in each video chat lesson (six possible). Data were missing for 25 children due to teacher error in entering attendance data; for children who had data available, the average number of video chat lessons attended was three, representing 50% of target lessons. Thirty-six percent of children with data available participated in four or more video chat lessons. Despite variability in participation, a total of 77% of families were retained from recruitment to posttest.

Social Validity

To examine the social validity of virtual summer learning for entering kindergartners, we triangulated using data from both caregivers and teachers. First, we conducted a survey of caregivers in the week following program completion; the survey was distributed to all families and completed by 21 caregivers, such that these data represent about one-third of the children who were available at posttest. The survey contained 20 items, five of which captured overall child and caregiver experiences in Summer Success at Home (see ), rated on a 5-point scale with the following anchors: 1 = poor, 3 = average, 5 = excellent. Other items (not discussed further) asked caregivers about their favorite lessons, what they liked the most, whether their knowledge about readiness improved, and the like.

Table 1. Caregiver survey responses for program satisfaction (n = 21)

As can be seen in , social validity of Summer Success at Home was quite high based on caregiver survey responses. Across the five items, mean ratings exceeded 4.7 on a 5-point scale, for which 5 was equated to an excellent rating, and the majority of caregivers (> 85%) provided ratings of 4 or 5 for each of the five items. These data, while representing a minority of caregivers involved with Summer Success at Home, indicate that respondents viewed all features of the program favorably.

Second, we examined teacher ratings of children’s engagement during video chat lessons. On average, children’s engagement in the activities was rated 2.4 on a 0- to 3-point scale; in 53% of sessions, the child was rated as being engaged for the whole lesson and 90% of sessions were rated as the child being engaged for more than half of the lesson. On average, children’s engagement in conversations was rated 2.6 on a 0- to 3-point scale with 46% of sessions rated as very engaged and 82% of sessions rated as at least moderately engaged.

Preliminary Impacts

To assess preliminary impacts, we examined pretest to posttest change in children’s scores on the six measures of readiness: social-emotional skills (emotion knowledge), math (counting, cardinality, patterning), and literacy (alphabet knowledge, emergent literacy). Using these data, we conducted paired-samples t-tests to examine the extent to which scores significantly changed from pretest to posttest. We also report Cohen’s d effect sizes, where .2 is considered a small effect, .5 is considered a medium effect, and .8 is considered a large effect (Cohen, Citation1988). Depending on the measure, pretest and posttest measures were available for 61 to 66 children. In line with research showing that attrition is higher among lower-income samples (Fitzgerald, Citation1998), children who were missing posttest data had a significantly lower family income (M = 18,447 USD) than children who contributed posttest data (M = 49,970 USD), p < .001.

Descriptive data for these measures appear in . As can be seen, there was an upward trend on all measures from the first to second time-point, albeit the difference was very small for cardinality. Results of the paired-samples t-tests showed that children demonstrated small but significant gains on emotion knowledge, t(65) = 2.37, p < .05, d = 0.26; patterning, t(65) = 3.27, p < .01, d = 0.38; and alphabet knowledge, t(60) = 4.02, p < .001, d = 0.20. Gains from pretest to posttest for the remaining two measures of math, namely counting and cardinality, and on the emergent literacy measure were not significant (ps > .05). We acknowledge the non-experimental nature of these findings, and thus cannot draw causal claims linking program participation to gains in key readiness skills transcending social-emotional, math, and literacy domains; however, prior to conducting causally interpretable research studies using experimental methods, it is important to conduct preliminary assessment of intervention outcomes.

Table 2. Child scores on pretest and posttest readiness measures

We also examined whether there were associations between dosage and gains in skills across the intervention period. Dosage was coded as the number of video chat lessons each child participated in. Regression models were conducted for each outcome predicting posttest scores from number of video chat sessions, controlling for pretest scores, gender, and income. These analyses showed that participating in more video chat lessons significantly predicted children’s gains for counting (B = 4.38, p < .0001), emergent literacy (B = 0.63, p = .0004), alphabet knowledge (B = 0.86, p = .009), and emotion knowledge (B = 0.20, p = .03). Although dosage was positively associated with gains for cardinality and patterning, these did not reach the level of statistical significance, ps > .43.

Discussion

Evidence from prior studies has shown that intensive summer programming to promote school readiness prior to kindergarten entry may be a promising avenue for improving skills among children from low-income backgrounds. The current study investigated an adaptation of such a program during a particularly challenging time for families – the summer of 2020, during school and childcare closures due to the COVID-19 pandemic. Results using triangulation of evidence showed that the Summer Success at Home program is feasible for families to participate in, has high social validity as reported by teachers, and may have impacts on several important school-readiness skills for children.

It was vital to assess the feasibility of an entirely contactless, virtual kindergarten readiness summer program. Our findings showed that there was ample interest in this type of program, despite its virtual nature and its reliance on caregivers to implement many of the intervention activities. Further, although there was substantial variability in how many video chat sessions families attended, the level of participation overall was moderately high given the many obstacles to engagement, including scheduling around caregivers’ work commitments and challenges with using the technology.

It was also important to capture data on the social validity of the program, given negative views about children’s screen time and skeptical attitudes toward screen-based learning. Among a subsample of caregivers who responded to a post-program survey, satisfaction with the program was quite high. Teachers’ ratings of child engagement can also shed light on social validity. Overall, teachers rated children’s engagement in both the activities and the conversations as quite high, especially given concerns about children’s ability to engage over video chat.

Although this study did not have a causally interpretable design, we assessed preliminary impacts by examining the extent to which children showed improvement in the targeted readiness skills over the 4-week program period. Children showed gains on several measures of readiness, including emotion knowledge, patterning, and alphabet knowledge. These were all aspects of the curriculum that were emphasized in both video chat lessons with teachers and in home learning activities. Patterning and alphabet knowledge are both important foundational skills for the more advanced learning that takes place in kindergarten (G. J. Duncan et al., Citation2007) and previous research investigating effects of in-person summer learning programs have shown effects on similar measures (R. J. Duncan et al., Citation2018; Khan et al., Citation2017). However, these prior studies have not investigated effects on emotion knowledge, a skill that is also likely to be important in kindergarten, as being able to understand and express one’s emotions can facilitate social interactions with peers and teachers. The current results suggest that emotion knowledge, in addition to math and early literacy skills, may be a valuable and malleable target for future summer learning programs. Though effect sizes in the current study were small to medium in size, that gains emerged in this short timeframe and with a less intensive intervention than prior kindergarten readiness programs is evidence of promise. Although children did not demonstrate gains on all assessments given, this may be due to the limited time frame of the intervention (4 weeks) and the low dosage across this time frame (only six 30-minute lessons with teachers plus home learning activities).

Notably, results showed that dosage was associated with stronger gains on four of six measures, including counting and emergent literacy which did not emerge as overall gains. This finding suggests that children who participated in more program activities (namely video chat lessons) gained more on the tested skills, supporting the idea that the intervention was a causal factor in children’s learning across this time period. It is important to note that although we only measured dosage of video chat lessons, participation in the lessons is likely to be correlated with participation in other aspects of the program (i.e., caregivers who schedule and attend video chat lessons may also be more likely to complete the home activities), meaning that this finding may reflect a broader impact of intervention dosage on children’s outcomes.

Limitations and Future Directions

Although overall we found that the Summer Success at Home program was feasible, on average, children participated in only half of the available video chat lessons, potentially missing out on opportunities for further learning. Future research should investigate methods for removing barriers to participating in the full suite of video chat lessons and encouraging participation. Despite less-than-perfect participation in video chat lessons, we determined that the program was feasible based on our recruitment success and maintenance of the majority of families across the 4-week program, both benchmarks that have not previously been established in a virtual kindergarten readiness intervention under pandemic conditions.

Further, our caregiver social validity data came from a survey distributed at the end of the program that obtained a relatively low response rate. It is possible that families with positive views of the program were more likely to participate and that we may be missing more critical viewpoints. However, given the preliminary nature of this work, we see the positive responses we did receive as encouraging evidence that the program was seen as beneficial for at least a subsample of our target population. Relatedly, our preliminary impacts data comes from a comparison of children’s skills at pretest and posttest, but not all children participated in the posttest session, leading to a smaller sample size for this analysis. Further, the subsample with posttest data had higher family income than children who were missing this data, suggesting that the preliminary impact findings reported here may not be generalizable to the lowest income families we served with this program. This is an important limitation of the current data and these findings should be replicated in future work.

One aspect that may have contributed to the program’s success was the high level of teacher education and experience. These experienced teachers were seemingly able to adapt to the virtual format and work effectively with children in this online context despite the challenges. It is possible that implementation with less experienced teachers would not be as successful or that novice teachers would require additional training; future research should examine the role of teacher experience in implementing virtual programs.

Another potential limitation to generalizability of our results is that children scored relatively high on several of our kindergarten readiness measures at pretest, suggesting that perhaps there was selection bias in our sample. It would be useful for additional studies to determine if this type of program could be beneficial for children with lower levels of preexisting skills. Notably, however, that we saw increases in these skills despite high starting points may speak to the robustness of these findings and measures with room for more variability may demonstrate even stronger gains.

It is important to note that Summer Success at Home was a multifaceted program and these data do not allow us to determine which aspect(s) of the intervention might have driven children’s learning. Although we have data on children’s participation and engagement in the individual video chat lessons with teachers, because this was primarily a programmatic offering for local families rather than a research project, we were unable to obtain systematic information about the frequency with which caregivers delivered the home activities. However, future research should examine the contribution of each component to determine which aspects may be driving any impacts. Relatedly, given the multifaceted nature of this program and the reliance on caregiver delivery of several components, our results may not be generalizable to remote or online schooling in general and more research is needed on the effectiveness of these modes of instruction for young children.

Implications

Although our findings should be seen as preliminary given these limitations, they nonetheless have important implications, both in relation to and outside of the pandemic context. During the summer of 2020, many aspects of families’ lives were disrupted by the pandemic, including both many preschool programs as well as out-of-home learning opportunities. Compared to Khan et al. (Citation2017) and other similar programs, Summer Success at Home was much less intensive in terms of the dosage of instruction provided to children directly. However, given the ongoing nature of the COVID-19 pandemic in the U.S. and the possibility of future surges of COVID-19 or other viruses, these data provide promising evidence that a contactless, virtual intervention may be able to be successful in promoting children’s kindergarten readiness skills even when they cannot be in preschool or in an in-person summer program.

Although more research is needed, in the future a virtual intervention could be beneficial in other contexts, such as in rural areas where it would be challenging for families to participate in an in-person program or for immunocompromised or seriously ill children who are unable to attend in person. A modified program could also be added as a supplement to existing kindergarten readiness programs to support caregivers in providing positive home learning experiences for their children.

The current study also adds to the existing literature supporting the idea that intensive summer programming prior to the kindergarten year can help improve children’s skills across domains. This is vital as children who are from low-income families and who do not have access to quality pre-kindergarten programs are at risk for having lower levels of school readiness skills. Having age-appropriate language, literacy, math, and social-emotional skills prior to kindergarten entry is a strong predictor of outcomes in the elementary years and beyond (Rabiner et al., Citation2016).

Furthermore, the current data provides an important contribution to the literature base on children’s learning from video. The Summer Success at Home program relied on prior studies demonstrating that by age 4, children can learn from educational media (e.g., Mares & Pan, Citation2013) and from the back-and-forth interaction that occurs over video chat (Gaudreau et al., Citation2020; Troseth et al., Citation2006). However, no studies to our knowledge have systematically examined video chat instruction occurring with teachers as part of a school readiness intervention. Despite negative attitudes toward children’s increasing use of screens, these data show that in combination with other caregiver-led activities, educational media and video chat instruction may be effective for promoting children’s skills.

Challenges

We also must point out challenges we experienced in conducting virtual school-readiness programming for young children. Largely, identification of challenges emerged from open-ended items on survey instruments and a focus group with Summer Success at Home teachers conducted at program close. The most prominent challenge concerned the duration of the program, as both teachers and caregivers indicated that they would have liked the program to have been longer. Although teachers had about one month to prepare for the program, they felt that this was not enough time to craft a meaningful virtual curriculum, and that the 4-week duration of programming limited teachers’ work with children. Teachers also explained that they spent much of their time assisting caregivers with transitioning their child to virtual learning and had to spend time helping caregivers navigate the local school district and community organizations’ regulations and policies, especially in regard to the forthcoming kindergarten year. Although helping caregivers with these activities was viewed as an important component of the program, given the limited time and resources available, teachers felt that this took away some valuable time that they could have used for implementing lessons with the children.

From the caregiver perspective, when asked what they would change about Summer Success at Home, many caregivers mentioned wanting “more classes,” “making the program longer,” and “having a longer session.” Interestingly, in prior site-based variations of Summer Success that involved 4 weeks of programming, few if any caregivers made requests that the program be longer in duration. Perhaps the lack of alternative activities for children during the summer of 2020, with libraries, pools, recreation centers, and camps largely closed, contributed to caregivers’ perspectives. Nonetheless, future research involving virtual programming for young children should experiment with program duration, as it is likely that longer duration may result in more robust impacts on children’s readiness skills, as well as increased caregiver satisfaction.

Conclusion

In sum, the current study provides preliminary evidence for the feasibility, social validity, and potential impacts of Summer Success at Home, a virtual summer kindergarten readiness program. The program attracted many interested families, was feasible to implement, and retained most families across the intervention period with moderate participation in key program activities. Further, a subsample of caregivers who responded to a post-program survey reported positive views of their participation and teachers reported high levels of child engagement during video chat lessons. Finally, there was evidence of preliminary impacts on several targeted school readiness skills. Although more research is needed to replicate and extend these findings, the current study demonstrates the promise of a virtual kindergarten readiness program that could be beneficial both under pandemic conditions, as well as other contexts.

Acknowledgments

This research was supported by a grant from the City of Columbus. We are grateful to the many teachers, families, and children who participated in this work. We are also grateful for partnerships with local organizations, including Nationwide Pediatric Primary Care Network and South-Western City School district, for their help in recruiting families to Summer Success at Home. Copies of the Summer Success teaching manual are available for free download at https://earlychildhood.ehe.osu.edu/community/programs/summer-success-program/.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

References

Appendix A

Activity and Lesson Schedule By Week

Appendix B

Sample Watch Together Lesson

Watch Together! – Letter Recognition

Super Why! Naila and the Magic Map Season 5 Ep. 10

https://www.youtube.com/watch?v=88yZY1JEY1k

Materials Provided: Dry erase boards, markers, and foam letters

Before Viewing:

  •  ● Take out your foam letters and build strings of letters together. Help them with any letters they forget! Try saying the letters in order without singing the “ABC” song.

  •  ● If your child gets all the letters, try mixing up letters, asking your child to fix the mistakes.

While Watching:

  •  ● Pause the video to write down a few words you see or hear in the episode (ex. map, Sphinx, fish, dish, wish, swish, open, decorates, solves, scratches, sees, and secret code)

  •  ● Look for the Super Letters, especially D, F and G.

  •  ● Ask your child to name them and write the Super Letters. Model writing the letter, then have your child try writing it themselves, or let them trace the letter you wrote.

After Watching:

  •  ● Talk about the favorite part of the episode.

  •  ● How did the Super Friends solve the problem?

Appendix C

Sample Play Together Lesson

Play Together! – Letter Recognition

Provided Materials: Fish cutouts, fishing pole, magnets, and paper clips

Instructions:

  •  ● Have the children attach the paper clip to the fishing pole.

  •  ● Have children take turns fishing for “ish” words. Have them read each word as they catch a fish.

  •  ● If it is an “ish” word, they keep it, if not, they throw it back in.

  •  ● Help the child point to each letter.

Too Easy:

  •  ● You can challenge your child to come up with their own “ish” words and write them on the cutouts (nonsensical words welcome) provided.

Too Hard:

  •  ● You can continue to read the words together and work to come up with new words. If all “ish” words are exhausted, come up with other rhyming word families (at, ing, ug, eep, an, etc.).

  •  ● Allow time for your child to respond. Tell them the correct answer if they are having trouble and ask them to repeat it with you. Repeat this activity with other rhyming word families listed above.

Appendix D

Sample Read Together Lesson

Read Together! – Letter Recognition

Chicka Chicka Boom Boom by Bill Martin, Jr. and John Archambault

Before Reading:

  •  ● Sing “Eyes are watching” song.

  •  ● Ask your child, “Look at the cover and tell me, what do you think the book is going to be about?”

  •  ● Tell your child, “You will see all the letters in the alphabet in this book, can you help me identify them as we read along?”

While Reading:

  •  ● Ask your child, “Why are some letters tall and some letters short? Which letters are lowercase? Which are uppercase? What does that mean?”

  •  ● Ask your child, “Can you tell me something that starts with the letter _____?”

  •  ● Ask your child to find the letter of their name throughout the pages.

After Reading:

  •  ● Ask your child to retell the events of the story; help them by revisiting some of the pages.

  •  ● Play an Eye Spy Letter Game (Ex. “I spy something that is yellow that starts with the letter F.” “That’s right, flowers. Flowers start with the letter F. Now it’s your turn.”).

Appendix E

Sample Teacher Lesson

Materials: White board, dry erase marker, vocabulary check list (self-made), highlighters, white paper, white crayons, liquid watercolors, paint brushes

Synopsis of the Watch Together episode for this week:

  •  ● Pig and his friends are playing when they find what looks like an Egyptian treasure map. To search for some answers, the Super Readers soar into the book, Naila and the Magic Map, and explore the world of Ancient Egypt. Will they be able to help Naila unravel the mysterious code that will unlock where the treasure lies?

Learning Objectives:

  •  ● Recognize and “read” familiar words or environmental print.

  •  ● Recognize and name some upper- and lower-case letters in addition to those in their name.

  •  ● Use a combination of drawings, dictation, and emergent writing for a variety of purposes.

  •  ● “Read” what they have written.

Before Activity

  •  ● Sing “Eyes are Watching” song before starting.

  •  ● Revisit the episode. Ask the child what their favorite part of the episode was. Ask them what letters they recognized in the episode.

  •  ● Say, “We are going to work on reading letters today! Let’s practice reading some letters of the alphabet.”

During Activity

  •  ● High Support

  •   ● Students will recognize the correct order of the alphabet and identify lowercase letters of the alphabet, specifically D, F, G. Slowly draw each letter, talking about your marker strokes as you make them. Ask the child to identify each letter. Then mix up the order of the letters and have the child help you put them in alphabetical order.

  •  ● Low Support

  •   ● Write a series of letters from the alphabet, omitting a letter. First ask the child to identify the series of letters. Ask the child to identify what letter is missing or comes next (Ex. n, o, p, ____, r, s, t). Choose to change the location of the omitted letter with the same series of letters for a challenge.

  •  ● Choose 5 words from the vocabulary list (map, Sphinx, fish, dish, wish, swish, open, decorates, solves, scratches, sees, secret code), talk about the meaning of the word, then have the child identify the first and last letter (if child needs more support, provide the letter name and practice writing it with the child).

After Activity

  •  ● Using the materials provided (white paper, white crayons, liquid watercolors, paint brushes), the student will write a message using the “invisible ink.”

  •  ● Have the child write a secret message with the white crayon on white paper.

  •  ● Mix a small amount (5–10 drops) of liquid watercolor in a small amount of water

  •  ● Have the child run the paint over the words they wrote, revealing the message!

Ask the child to point to their letters and ask them to read it to you.