Getting ‘What Works’ working: building blocks for the integration of experimental and improvement science

ABSTRACT

As a systemic approach to improving educational practice through research, ‘What Works’ has come under repeated challenge from alternative approaches, most recently that of improvement science. While ‘What Works’ remains a dominant paradigm for centralized knowledge-building efforts, there is need to understand why this alternative has gained support, and what it can contribute. I set out how the core elements of experimental and improvement science can be combined into a strategy to raise educational achievement with the support of evidence from randomized experiments. Central to this combined effort is a focus on identifying and testing mechanisms for improving teaching and learning, as applications of principles from the learning sciences. This article builds on current efforts to strengthen approaches to evidence-based practice and policy in a range of international contexts. It provides a foundation for those who aim to avoid another paradigm war and to accelerate international discussions on the design of systemic education research infrastructure and funding.

KEYWORDS:

• ‘What Works’
• evaluation
• RCTs
• improvement science
• implementation
• evidence-based policy
• knowledge mobilization
• education
• learner achievement

Acknowledgements

I would like to thank Martin West, Jal Mehta, Stephanie Jones, Catherine Darrow, Ruth Deakin Crick, Andrew Volkert, Andy de Barros, Justin Reich, Calum Davey, Ben White, Rachel Glennerster and Nancy Cartwright for conversations contributing to thinking in this article.

Disclosure statement

No potential conflict of interest was reported by the author.

Notes

1. Internationally in the education field, ‘What Works’ is predominantly associated with RCTs; in the U.K., the ‘What Works Network’, recently established by the UK government and funded by the Economic and Social Research Council, uses the term more broadly refers to the sharing and use of ‘high quality evidence for decision making’. See www.gov.uk/guidance/what-works-network.

2. http://ies.ed.gov/ncee/wwc/.

3. https://educationendowmentfoundation.org.uk/about/.

4. Not all mediator variables are mechanisms (Vancouver and Carlson 2015), but these designs are intended to isolate mediators that act as mechanisms, as in, have a considerable causal impact on the intended outcome.