Persistence and Fadeout in the Impacts of Child and Adolescent Interventions

ABSTRACT

Many interventions targeting cognitive skills or socioemotional skills and behaviors demonstrate initially promising but then quickly disappearing impacts. Our article seeks to identify the key features of interventions, as well as the characteristics and environments of the children and adolescents who participate in them, that can be expected to sustain persistently beneficial program impacts. We describe three such processes: skill-building, foot-in-the-door and sustaining environments. We argue that skill-building interventions should target “trifecta” skills—ones that are malleable, fundamental, and would not have developed eventually in the absence of the intervention. Successful foot-in-the-door interventions equip a child with the right skills or capacities at the right time to avoid imminent risks (e.g., grade failure or teen drinking) or seize emerging opportunities (e.g., entry into honors classes). The sustaining environments perspective views high quality of environments subsequent to the completion of the intervention as crucial for sustaining earlier skill gains. These three perspectives generate both complementary and competing hypotheses regarding the nature, timing, and targeting of interventions that generate enduring impacts.

KEYWORDS:

• interventions
• fadeout
• methodology

Acknowledgments

We would like to extend thanks to Ruben Arslan, Peg Burchinal, Pamela Davis-Kean, Thad Domina, Ken Dodge, Dorothy Duncan, Dale Farran, Jarrod Ellingson, George Farkas, David Grissmer, Paul Hanselman, Jade Jenkins, Daniel Keating, Ron Haskins, Josh Lawrence, Andrew Littlefield, Susanna Loeb, Dan McAdams, Andrew Penner, Emily Penner, John Protzko, Maria Rosales-Rueda, Robert Siegler, Judy Singer, Jeff Smith, Aaron Sojourner, Larry Schweinhart, Deborah Vandell, Tyler Watts, Russ Whitehurst, and Hirokazu Yoshikawa; to seminar participants at the University of Michigan, the University of Bath, and the University Brisbane; and to many of those who attended a keynote address on this topic at the 2015 SREE spring conference for helpful comments on prior drafts.

Funding

We are grateful to the Eunice Kennedy Shriver National Institute of Child Health & Human Development of the National Institutes of Health under award number P01-HD065704. Candice L. Odgers is supported by a Jacobs Foundation Advanced Research Fellowship.

Notes

¹ We do not concentrate on adult health outcomes because the physiological processes linking experiences in childhood, particularly early childhood, to adult health are only just beginning to be understood (Center on the Developing Child, 2010).

² The meta-analytic database is the product of the National Forum on Early Childhood Policy and Programs (http://developingchild.harvard.edu/initiatives/forum/) based on a comprehensive search of the literature from 1960 to 2007, when the coding project began. Studies had to have a treatment and control/comparison group, rather than simply assessing the growth of one group of children over time. Early childhood education programs were defined as structured, center-based early childhood education classes, day care with some educational component, or center-based child care. These include full preschool programs such as Head Start and other interventions conducted by researchers. Programs included were required to have provided services to children, their families, or staff at the program sites, and assessed program impacts on children's cognitive and achievement outcomes. About one third of the ECE studies used random assignment, with the remainder following quasi-experimental designs such as change models, individual or family fixed-effects models, regression discontinuity, difference in difference, propensity score matching, interrupted time series, instrumental variables, and some other types of matching. Studies that used quasi-experimental designs must have had pretest and posttest information on the outcome or established baseline equivalence of groups on demographic characteristics determined by a joint test.

³ In the meta-analytic database used in Figure 2, only about one third of the studies followed subjects beyond the end of treatment. In a meta-analysis of adolescent alcohol using RCT designs, only 3 of 18 studies reported on long-term effects (>48 months; Smit et al. (2008). In the case of prevention of depressive symptoms, only 12 of 30 studies collected data past six months depression (Horowitz & Garber, 2006).

⁴ We ignore the lower-right quadrant containing skills that are neither fundamental nor malleable. An example might be ambidexterity. It can be trained, but only with great difficulty (and is thus less malleable). It can offer an advantage (e.g., for NBA players), but for most individuals in the population would not confer a significant advantage in life (and is thus peripheral). Lacking both malleability and fundamentality, it exemplifies the least promising kinds of characteristics to target with interventions.

⁵ Conscientiousness is highly correlated with the lower level construct “grit,” which has been found in some studies (but not others) to predict achievement outcomes above and beyond conscientiousness (Duckworth, Peterson, Matthews, & Kelly, 2007; Duckworth & Quinn, 2009; Ivcevic & Brackett, 2014). Other “Big Five” personality traits are openness to experience, extraversion, agreeableness, and neuroticism.

⁶ Heckman, Pinto, and Savelyev (2012) show that Perry is a possible exception; see also Krasner et al. (2009) for recent evidence of intervention-driven change in conscientiousness among adults. Heckman and Kautz (2013) argue that behavioral outcomes, such as substance abuse and crime, are better operationalizations of “character” than self-reported measures. Although we agree with their concerns about measuring changes in conscientiousness using self-reported measures, using positive behavioral outcomes as effects of personality changes evidenced by these very outcomes is worrisome, especially given the possible changes in other skills and environments that might plausibly affect such outcomes (Benda, 2005).

⁷ Cascio and Staiger (2012) proposed that perhaps fadeout patterns and larger treatment effects of interventions earlier in development could be explained by a smaller variance of test scores earlier in development. However, their findings suggested that little of the fadeout following early interventions could be explained by increasing variance in test scores, especially in the period immediately following an intervention, during which the most pronounced fadeout occurs.

⁸ Spoth et al. (2011) summarizes this view as follows: “The extant literature on universal interventions emphasizes the importance of timing program implementation to occur during the developmental window when adolescents are just beginning to initiate substance use. Epidemiological research suggests that well-timed interventions could accrue substantial public health and economic benefits, should they delay onset of substance use or delay transition to more serious use” (pp. 621–622).

⁹ Another possible source of impacts is that the extended instructional time in the Double Dose condition enabled teachers to use instructional activities such as working in small groups and on boards and engaging in more probing and open-ended questions.

¹⁰ Although residential mobility enabled some Perry children to eventually attend higher resourced and more integrated middle and high schools, almost all spent at least their first few school years in Ypsilanti schools (L. Schweinhart, personal communication, August 15, 2015).

¹¹ Of course, it is also possible that enriched environments might boost the achievement of control-group children even more than treatment-group children. That pattern best fits the data on preschool impacts observed in Magnuson, Ruhm, and Waldfogel (2007).

¹² The idea that children's skills built during the intervention can lead to more positive subsequent environments experienced by the child overlaps with the Cunha and Heckman (2007) hypothesis that early skills increase the productivity of subsequent investments. If the definition of “investments” includes, say, school- or community-based opportunities for establishing positive peer relations, then the right kind of early parent–child interventions may increase the chance that these opportunities are taken up.

¹³ Chetty's analyses of classroom quality impacts in the Project STAR study also showed persistent effects on children's noncognitive skills (Chetty et al., 2010). In Abecedarian, impacts on the study's teacher-reported index of child hostility in the early grades were perversely positive (Haskins, 1985).

¹⁴ Conversation with Tomoko Wakabayashi, Cheryl Polk, and Mary Delcamp on March 10, 2016.