Learning Rates: A Correction of Gain Scores to Assess Math Learning Interventions

Abstract

Traditional analyses of pretest-posttest designs draw inferences about learning using linear models that predict posttest performance by accounting for pretest scores. These analyses allow inferring student growth effects but prior research has pointed out some conceptual problems, such as its meaningful interpretation as learning. By drawing on research about physics teaching, this study introduces the Learning Rate Model (LR-Model) to describe learning as a non-linear process of knowledge acquisition. Analysis of data gathered from two real-world math learning interventions indicates that the model can provide adequate interpretations of math learning phenomena observed in pretest-posttest study designs. The model seems well suited to assess math learning interventions that teach unfamiliar knowledge to learners, where the pretest and posttest measures are designed to evaluate the content delivered during the intervention. Under these conditions, the LR-Model shows advantages to alternative linear models as (1) it describes learning more accurately, (2) it delivers more meaningful learning estimators, and (3) it provides higher statistical power for hypothesis testing aimed at determining learning effects.

Keywords:

• Gain scores
• math learning assessment
• pretest-posttest design
• student growth

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Parallel measurements have identical true scores and linearly experimentally independent errors that have equal variances (Lord & Novick, 1968).

Additional information

Funding

This research was supported by Agencia Nacional de Investigación y Desarrollo del Gobierno de Chile, by funding of Program ANID/FONDEF/IT23I0012 granted to J.A.N-U.