Rerandomization to Balance Tiers of Covariates

Abstract

When conducting a randomized experiment, if an allocation yields treatment groups that differ meaningfully with respect to relevant covariates, groups should be rerandomized. The process involves specifying an explicit criterion for whether an allocation is acceptable, based on a measure of covariate balance, and rerandomizing units until an acceptable allocation is obtained. Here, we illustrate how rerandomization could have improved the design of an already conducted randomized experiment on vocabulary and mathematics training programs, then provide a rerandomization procedure for covariates that vary in importance, and finally offer other extensions for rerandomization, including methods addressing computational efficiency. When covariates vary in a priori importance, better balance should be required for more important covariates. Rerandomization based on Mahalanobis distance preserves the joint distribution of covariates, but balances all covariates equally. Here, we propose rerandomizing based on Mahalanobis distance within tiers of covariate importance. Because balancing covariates in one tier will in general also partially balance covariates in other tiers, for each subsequent tier we explicitly balance only the components orthogonal to covariates in more important tiers.

• Causal inference
• Covariate balance
• Experimental design
• Mahalanobis distance
• Randomization
• Treatment allocation

Notes

The second author of the current article was provided the data as a discussant of Shadish, Clark, and Steiner (2008).

Additional information

Notes on contributors

Kari Lock Morgan

Kari Lock Morgan is Assistant Professor, Department of Statistics, Penn State University, University Park, PA 16802 (E-mail: [email protected]). Donald B. Rubin is John L. Loeb Professor, Department of Statistics, Harvard University, Cambridge, MA 02138 (E-mail: [email protected]). This work was partially supported by the National Science Foundation (NSF SES-0550887 and NSF IIS-1017967) and the National Institutes of Health (NIH-R01DA023879). The authors are grateful for helpful comments from Peng Ding.

Donald B. Rubin

Kari Lock Morgan is Assistant Professor, Department of Statistics, Penn State University, University Park, PA 16802 (E-mail: [email protected]). Donald B. Rubin is John L. Loeb Professor, Department of Statistics, Harvard University, Cambridge, MA 02138 (E-mail: [email protected]). This work was partially supported by the National Science Foundation (NSF SES-0550887 and NSF IIS-1017967) and the National Institutes of Health (NIH-R01DA023879). The authors are grateful for helpful comments from Peng Ding.