Conducting a Cost-Effectiveness Analysis of an Early Numeracy Intervention

Abstract

This study examined the cost-effectiveness of a 50-lesson mathematics intervention program focused on whole number concepts for at-risk kindergarten students, ROOTS. The study utilized a randomized block design with 1,251 at-risk students within 138 classrooms randomly assigned to one of two active treatment conditions (small groups of either two or five students) or control condition. Proximal and distal measures were collected in the fall (pretest) and spring (posttest). The costs per group per effect size unit change varied from $216 to $736 depending on differing district scenarios and group size. The cost-effectiveness ratios per student varied from $267 to $3,201 depending on district scenario, group size, and the measure. Implications for conducting cost-effectiveness evaluations and public policy are discussed.

Keywords:

• mathematics
• learning disabilities
• cost-effectiveness

DISCLOSURE

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Scott Baker, Ben Clarke, Chris Doabler, and Hank Fien are eligible to receive a portion of royalties from the University of Oregon’s distribution and licensing of certain ROOTS-based works. Potential conflicts of interest are managed through the University of Oregon’s Research Compliance Services. An independent external evaluator and coauthor of this publication completed the research analysis described in the article.

Additional information

Funding

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant number R324A120304 to the Center on Teaching and Learning. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

Notes on contributors

Ben Clarke

Ben Clarke, PhD, is an associate professor in the School Psychology program at the University of Oregon and associate director of the Center on Teaching and Learning. His work is focused on early screening for mathematics risk and the development and efficacy testing of mathematics intervention programs spanning the K–sixth grade spectrum in both traditional and technology-based formats.

Gulcan Cil

Gulcan Cil, PhD, is an environmental and public health economist with a particular interest in policy evaluation, non-market valuation, and estimation of the effects of environmental and behavioral factors on human health. She has completed a PhD in economics at the University of Oregon and a post-doc at the U.S. Environmental Protection Agency. She continues pursuing empirical research in various domains of health, including maternal and infant health, geriatric health, substance use, childhood obesity, and environmental health, as an associate research scientist at the Oregon Research Institute (ORI). Gulcan also works on research projects that involve population-level evaluation of program implementation and cost analyses for school-based programs. Finally, she participates in research projects as a member of the Mikesell Environmental and Resource Economics Lab in the Department of Economics in the University of Oregon.

Keith Smolkowski

Keith Smolkowski, PhD, a senior research scientist at Oregon Research Institute, has 30 years of experience in education, social science, and public health research. He has become an expert in the design and analysis of complex efficacy and effectiveness trials. His research interests include reading and mathematics instruction, child social behavior interventions, adolescent problem behavior and substance use, and the assessment of teachers and students. Dr. Smolkowski has served as principle investigator, co-investigator, or lead methodologist on over 47 grants funded by the U.S. Department of Education, National Institutes of Health, National Science Foundation, and Centers for Disease Control and Prevention. His work has led to over 100 peer-reviewed articles and book chapters.

Marah Sutherland

Marah Sutherland, MS, is a doctoral candidate in the School Psychology program at the University of Oregon and is a graduate employee at the Center on Teaching and Learning. She received her bachelor’s degree in psychology from the University of Wisconsin–Madison and her master’s degree in special education from the University of Oregon. Her research interests include early numeracy intervention and assessment, the effect of teacher coaching and professional development in mathematics, and the integration of academic and behavioral supports within multitiered systems of support.

Jessica Turtura

Jessica Turtura, PhD, is a research associate at the Center on Teaching and Learning at the University of Oregon. Her research interests include effective instructional practices to support all learners, academic and behavioral interventions, and implementation of schoolwide, coordinated instructional and assessment practices. Her school and district experience involves providing professional development at the local and state levels to support implementation of effective positive behavior and literacy practices.

Christian T. Doabler

Christian T. Doabler, PhD, is an assistant professor in the Department of Special Education at The University of Texas at Austin and a Research Fellow of the Mathematics and Science Institute for Students with Special Needs at The Meadows Center for Preventing Educational Risk. Dr. Doabler’s research focuses on designing and testing intensive, early mathematics and science interventions for students who are with or at risk for learning disabilities. Currently, Dr. Doabler serves as principal investigator on two DRK-12 Design and Development projects funded by the National Science Foundation and as a co-principal investigator on two Goal-3 Efficacy Trials funded by the Institute of Education Sciences (IES). He has published 40 peer-reviewed publications and led the design and development of four IES-sponsored Tier 2 mathematics interventions.

Hank Fien

Hank Fien, PhD, is principal and co-principal investigator on 10 IES grants in the areas of early reading, adolescent reading, and early mathematics interventions for diverse learners in school settings. His research interests include reading and mathematics development in young children, instructional design, and empirically validated interventions aimed at preventing or ameliorating student academic problems. Dr. Fien’s most recent work is focused on extending the learning environment from the physical classroom to include virtual environments (e.g., gaming platforms) and leveraging gaming technology to maximize instructional design and delivery principles.

Scott K. Baker

Scott K. Baker, PhD, joined the Center on Research and Evaluation (CORE) at Southern Methodist University (SMU) in 2013. Dr. Baker is interested in the role that scientific research can play in improving policies and practices associated with child outcomes. His specific research interests have been shaped by his role as principal investigator on numerous education grants from the Institute of Education Sciences and other federal agencies. Currently, Dr. Baker is interested in the effect of interventions on child outcomes, mechanisms that underlie effective interventions, and how intervention effect varies by factors intrinsic and extrinsic to the child.

Prior to joining SMU, Dr. Baker was the associate director of the Center on Teaching and Learning (CTL) at the University of Oregon and he was CTL’s first director of research from 2006 to 2012. He was the founder of Pacific Institutes for Research and was its executive director and president from 2003 to 2010. Dr. Baker has been a principal investigator on 30 federally funded grants totaling over $57 million. Much of this work has focused on the development and evaluation of assessment and instructional interventions for different groups of students, including English learners and students with learning difficulties, and how to ensure that these practices and interventions are implemented effectively in real school settings. Dr. Baker has also conducted several evaluations of bilingual programs in public school settings and he was a lead investigator on an evaluation of a volunteer tutoring program being implemented throughout the state of Oregon with children in the early elementary grades.