Using a Factorial Design to Maximize the Effectiveness of a Parental Text Messaging Intervention

Abstract

 Parental text messaging interventions are growing in popularity to encourage at-home reading, school-attendance, and other educational behaviors. These interventions, which often combine multiple components, frequently demonstrate varying amounts of effectiveness, and researchers often cannot determine how individual components work alone or in combination with one another. Using a 2x2x3 factorial experiment, we investigate the effects of individual and interacted components from three behavioral levers to support summer reading: providing updated, personalized information; emphasizing different reading views; and goal setting. We find that the personalized information condition scored on average 0.03 SD higher on fall reading assessments. Texting effects on test scores were enhanced by messages that emphasized reading being useful for both entertainment and building skills compared to skill building alone or entertainment alone. These results continue to build our understanding that while text message can be an effective tool for parent engagement, the specific content of the message can lead to meaningful differences in the magnitude of the effects.

Keywords:

• Factorial design
• literacy
• parental text messaging
• experimental design

Open Scholarship

This article has earned the Center for Open Science badges for Open Data, Open Materials, and Preregistered through Open Practices Disclosure. The data and materials are openly accessible at https://doi.org/10.7910/DVN/XUEU90, https://doi.org/10.7910/DVN/XUEU90 and the preregistration is accessible at https://sreereg.icpsr.umich.edu/sreereg/subEntry/2365/pdf.

Acknowledgments

The authors are thankful for feedback and advice from Nicole Pashley, Carly Robinson, Luke Miratrix, Monnica Chan, Eddie Kim, Thomas Kelley-Kemple, participants in the READS Lab, CARES Lab, and conference participants at AEFP and APPAM.

Notes

1 Other studies have used informational messaging campaigns administered through a non-texting medium to reduce absenteeism among students (Robinson et al., 2018; Rogers et al., 2017; Rogers & Feller, 2018).

2 Besides English and Spanish, no other home language represented more than (1%) of students. Families whose home language was neither English nor Spanish received English messages due to resource constraints.

3 Though not displayed in Table 1, we also conducted balance checks between our message-receiving conditions and the pure control. We found no significant differences on any pre-treatment characteristic.

4 The differences in effects between the two outcomes are not due to differences in sample; in sensitivity analyses we find similar effects on the MAP when we limit the sample to rising third graders who also took the BOG.

5 As described previously, all students received access to 6 electronic books and activities via the MORE@Home app, but half also received a set of 10 hardcopy books and additional app activities. Thus, for one half of the sample, the percent of available books read is a proportion of the 16 total books they had access to via the program.

Additional information

Funding

This work is supported by the Chan Zuckerberg Initiative and the Institute of Education Sciences, U.S. Department of Education through grant R305B150010. The opinions expressed are those of the authors and do not represent the views of the Chan Zuckerberg Initiative, the Institute, or the U.S. Department of Education.