Moving Education Science Forward by Leaps and Bounds: The Need for Interdisciplinary Approaches to Improving Children's Educational Trajectories

As we announced last year, we have added a new section to the Journal of Research on Educational Effectiveness focused on “Theory, Contexts, and Mechanisms.” This new section of the journal augments its focus to include rich, theory-based reflection and evidence regarding the conditions, contexts, and processes that shape the effects of educational practices and policies. This section builds on the journal's solid commitment to publishing both rigorous studies of educational practices and policies and advances in the research designs and analytic methods needed to test such practices.

We are pleased to debut this section with two articles focused on a set of theoretical and methodological challenges facing the field of education science. These articles are based on plenary addresses given at the 2015 conference of the Society for Research on Educational Effectiveness. They embody the type of work that we hope to publish more of in the coming years.

Long-Term Benefits and the Limits of Education Science

The theme of the Spring 2015 conference of the Society for Research on Educational Effectiveness, “Creating and Sustaining Gains From Early Childhood Through Adulthood,” highlighted the role of research designed to improve our understanding of children's learning trajectories from preschool to college. This theme grew out of a concern that the field of education science faces a challenge of developmental and contextual overspecialization. That is, most educational research typically focuses on a single period of development and a primary context with strong influence on that period. In many cases it also focuses on a relatively narrow population of children. So, for example, education science is often split into small areas of research by child age (e.g., preschool, middle childhood, adolescence, young adulthood), by subpopulation (e.g., low-income students, English learners, community college students), by learning domain (e.g., social-emotional development, literacy, math education, science education), and by context of influence (e.g., home, school, neighborhood). Although this approach facilitates rigorous research on the effects of specific interventions on short-term learning and educational outcomes, the absence of a broad understanding of processes as they unfold across developmental time, across educational transitions, and across contexts of influence, may yield fragmented—and thus less effective—science, policy, and education practice. In short, it is unlikely that we can succeed at improving children's educational trajectories if we only focus on a single domain, a single time period, a single context in which children learn and develop.

Greg Duncan (University of California at Irvine) and Mark Greenberg (Pennsylvania State University) gave the two plenary addresses at the 2015 SREE conference. Both addresses focused on long-term developmental processes and the need for interdisciplinary research and improved research designs to address the challenge of understanding, and thus improving, developmental processes and trajectories in education. We asked Duncan and Greenberg to elaborate on the ideas they discussed in their 2015 talks; the articles that follow in this special section are a result of that effort.

Challenges to Altering Long-Term Learning Trajectories

One puzzle in research on altering long-term learning trajectories is evidence of the “fade-out” of early childhood program effects, a phenomenon found in many studies. Though early childhood programs are typically designed to facilitate the transition to elementary school and are thought to have the potential for cascading effects over subsequent school transitions, they often lead to gains in early childhood that fade immediately following the intervention, or over the elementary years. As shown in Bailey, Duncan, Odgers, and Yu (this issue), meta-analytic estimates of early childhood program impacts on achievement and cognitive outcomes decline by a little more than half in the year following the intervention, and by half again in years 2–4 postintervention.

A second puzzle in this same line of research is the evidence that some of the most well-known early childhood education programs demonstrate long-run effects on high school completion, earnings, and crime, despite the apparent early fade-out of their effects on cognitive skills (Currie & Thomas, 1998; Deming, 2009; Garces, Thomas, & Currie, 2002; Ludwig & Miller, 2005). These so-called “sleeper effects” may result from the interplay of unmeasured developmental and contextual processes.

In addition, one notable challenge in research on the long-term effects of educational interventions is that these effects are often heterogeneous. A focus on average effects in a targeted subpopulation may obscure important differences in effects across populations or may lead to misleading conclusions about the total effects of universal interventions. Small population average effects may be difficult to detect but may nonetheless be cost-effective if small long-term benefits are experienced broadly.

The two puzzles—and the challenge of impact heterogeneity—are, at best, poorly understood in most areas of education science. Moreover, they pose a challenge not only with respect to early childhood programs, but in fact are relevant across all domains and developmental periods. They are particularly salient in early childhood programs, however, because of the interest in cost–benefit analyses in a field where many of the cost savings accrue long after the intervention occurred and where the benefits are likely heterogeneous among the population.

A focus on educational transitions (from preschool to kindergarten, from elementary to middle school, from middle to high school, and from high school to college) and learning across time may be critical in understanding how to support children's long-term educational trajectories. Likewise, a focus on cross-domain influences may help us to understand to what extent focusing on one outcome in early childhood can improve outcomes in other domains (e.g., whether early math skills can support children's long-term executive function skills). Finally, comprehensively assessing contexts may help us to understand how improvements in one sphere of a child's life may be supported in another sphere (e.g., not simply examining classrooms, but the hallways, recess yards, and neighborhoods children pass through to get to those classrooms) and how contexts are sequenced across a child's educational career. Such an understanding of context and contextual trajectories is particularly important for understanding how learning trajectories vary among student subpopulations.

Strategies for Improving Education Research and Interventions

The two articles included in this special section of the journal address these challenges in complementary ways. Bailey et al. take program effect “fade-out” as evidence that we have not yet developed an optimal set of early childhood interventions. They recommend interventions that target different skills, are timed to key developmental transitions, and take into account the role of postintervention contexts on long-term outcomes. Greenberg and Abenavoli (this issue) suggest that a solution to better addressing long-term program impacts is to consider universal interventions and to reexamine our metrics for assessing the size, and thus significance, of any effects observed. Together the articles provide a useful, albeit ambitious, guide for the ways in which we need to revise education science if we are to make progress on the goal of understanding and improving long-term educational outcomes. Both articles draw extensively on perspectives and insights from multiple disciplines; in doing so, they implicitly highlight the extent to which education science may benefit from other fields. The need for multidisciplinarity is one that the Institute of Education Sciences has long recognized through support for interdisciplinary predoctoral training programs, but one that perhaps has not progressed far enough (yet) in our scholarly work.

Bailey et al.: Outcomes, Timing, and Environmental Trajectories

Bailey et al. propose three ways that interventions should be improved in order to create lasting positive effects. First, drawing on insights from prevention science, developmental psychology, and education and policy research, they argue that interventions must target skills that are malleable, fundamental, and would not have developed in the absence of the intervention. The notion that some skills can be central for long-term consequences draws on developmental cascades theory that highlights cumulative consequences of single behaviors that “alter development” (Masten & Cicchetti, 2010, p. 491). Cascade theory suggests, like Bailey and colleagues, that some domains of development have long-term implications while others do not.

Second, Bailey et al. argue for the importance of considering the timing of outcomes relative to key life course or contextual transitions. This “foot in the door” perspective is compelling, and it suggests a different perspective on timing that would be typical coming from a developmental psychology or even education science perspective. Indeed, their argument here is that particular life course transitions (in life course terms, “socially-defined, age-graded events and roles”; Elder & Shanahan, 2006, p. 667) are key to understanding the influence of intervention effects. In effect, life course theory provides a framework for the ideas presented here—in that events in one's life have cultural and societal meaning and it is these meanings that make events “matter” more at some stages than others.

Third, Bailey et al. argue for the importance of understanding environments as dynamic systems across time, such that differences in micro-setting experiences for an individual can play a role in the long-term effects of educational interventions. This perspective is often underappreciated in assessing the effects of education interventions. That is, many analyses are designed without considering that (a) there is variation in individuals' post-intervention experiences and environments; (b) some environments may sustain intervention effects more than others; and (c) interventions may affect individuals' selection into subsequent environments. Drawing from Bronfenbrenner's “ecology of human development,” the field has long recognized that human behavior and the environment are inextricably entwined components of one system (Bronfenbrenner, 1979). Education science has built on this theoretical tradition with interventions that targeted the classroom context as a means to improve outcomes for students (in Bronfenbrenner's terms, targeting the microsystem). But what Bailey and colleagues' ideas highlight are the ways in which our consideration of context has been too static. In order to understand how to support children's trajectories of learning, we need to bring “time” (a later addition to ecological theory; Bronfenbrenner & Morris, 1998) into our models of education science, by considering linked microcontexts across individual time.

Greenberg and Abenavoli: Interventions and Analytics

Greenberg and Abenavoli's take on the challenge is quite different. Rather than focusing on the outcomes that are targeted or their timing in the life course, their central argument is that we undervalue both universal interventions and their potential to induce small but widespread impacts in our quest to find effective approaches. Moreover, they argue, even when we do evaluate universal strategies, our conventional analytic approaches and methods for quantifying their effects are ill-designed to fully assess their impact. Their work draws from medical research, public health, and prevention science as a means to highlight blind spots in education science research.

First, Greenberg and Abenavoli argue that we have overlooked the “prevention paradox” in focusing our interventions. The result has been a focus on high-risk populations primarily, and thus we miss key populations where positive intervention effects may occur. They describe Rose's 1985 insight from epidemiology that many high-risk outcomes can be prevented among “low-risk” portions of the population and that, in fact, the majority of prevention benefits may come from this low-risk group (Rose, 1985). Part of the challenge, as they acknowledge, is our inability to effectively screen for poor outcomes; as a result, targeting interventions to narrow portions of the population is unwise. But what is notable about their argument is that whereas Bailey et al. argue for targeting different outcomes, Greenberg and Abenavoli challenge us to examine who we are targeting and thus, in effect, substantially rethink the approaches to intervention we should consider.

Second, Greenberg and Abenavoli argue that the standard effect size metric that is used to assess intervention effects is not appropriate for communicating the relative benefits of intervention effects. Again, the prevention science perspective guides their recommendation: given a focus on high-risk, and thus low-base-rate, outcomes, effect size metrics may not provide an understanding of the significance of events or the difference they make in terms of impact to society. They offer a number of alternative metrics that expand our tool kit for assessing program impact.

Third, Greenberg and Abenavoli indicate the ways in which our focus on main effects on outcomes, rather than impact heterogeneity, limit the ways in which we characterize impacts from our intervention studies. Although they do not mention this, we are reminded of Bronfenbrenner urging psychological research to consider components of the ecological environment as interdependent and interactive, given that, “in ecological research, the principal main effects are likely to be interactions” (Bronfenbrenner, 2005, p. 518). In this case, though, they argue for interactions between person characteristics and interventions, rather than environments. Greenberg and Abenavoli highlight a number of methodological approaches to assessing such impact heterogeneity in order to understand the ways in which treatment effects might vary across groups as well as the way in which, from a population perspective, the proportion of the population at risk may be altered by treatment. In intersecting this with varying “types” of effects—treatment, prevention, and promotion—they challenge the field to think in more nuanced ways about treatment effects. Again, their perspective derives largely from the prevention science field, where the focus on high-risk populations and population-level effects makes clear these gaps in the field of education science.

Conclusion

As these articles together point out, improving the educational trajectories of children requires a major revision of our approach to education science. Both articles highlight how we need a stronger “longitudinal” focus—whether on settings or on outcomes. They also highlight the need to reconsider a number of aspects of our inquiry, from the outcomes we target, the population we study, the settings we assess, and the metrics and methods we employ. Our studies must examine development across time and multiple contexts, and must be designed to test theory and to illuminate educational processes. If our ultimate goal is to alter children's learning trajectories throughout their academic careers and into adulthood, then the center of gravity of our field—addressing specific outcomes for individual groups of children in single classroom or school contexts—seems ill-equipped to get us there. This is perhaps in part a failure of the overly narrow methodological and domain-specific training that most education scientists receive. Such specialization facilitates rigor, but often at the cost of broad, multidisciplinary theoretical perspectives on education. This is not to say that such perspectives are not to be found anywhere in our field, but they are not as common as is necessary to substantially move the needle on improving educational trajectories.

One major takeaway from this set of articles is that interventions and designs that are currently commonplace in our field could be adapted by looking to other disciplines. It is perhaps instructive that Bailey et al.'s suggestions might be thought of as coming from developmental theory, life course theory, and an ecological perspective, whereas Greenberg and Abenavoli's are informed by prevention science, public health, and medicine. By drawing on the most relevant aspects of theories in related fields, we may be able to build a new science of education that is well-suited to the challenges ahead of us. Indeed, IES's commitment to interdisciplinary predoctoral training can provide a strong foundation for such work. But what is new here is the fact that we may need to reach beyond those disciplines most closely aligned with the field of education (e.g., psychology, sociology) to additional disciplines (e.g., public health, medicine) that are typically less central to the field if we are going to make the kind of progress we need.

These articles suggest a number of areas where we need to strengthen our training and scholarship as we approach the next decade of research in education science. First, we need to reconsider the interventions we study—ensuring that they are focusing on “trifecta skills” with the largest potential for “cascade” effects and aimed at populations where such interventions have the opportunity to produce population-level impact. Second, we need better evaluation designs—designs that allow us to examine treatment heterogeneity, cross-time environmental influences, and the long-term cost benefit of shorter term programs. And, finally, we need our training and scholarship to draw from a wide range of fields outside of education science—including developmental science, ecological theory, prevention science, public health, and medicine.

This is an ambitious agenda, but one that is critical if we are to succeed at changing the education (and life) trajectories of children. Having built a foundation over the last decade in rigorously assessing educational interventions, education science is now well-positioned to take on this next challenge. We plan to use JREE, and in particular its new section on Theory, Contexts, and Mechanisms, as a space for the field to begin to do this work.