The Use of Randomisation in Educational Research and Evaluation: A Critical Analysis of Underlying Assumptions

Abstract

This article considers the underlying assumptions related to the use of random assignment in educational research and evaluation; more specifically, the ability of random assignment to create similar comparison groups for the purpose of determining the effectiveness of educational programmes. In theory, randomly assigning individuals to comparison groups is considered to be the best method available for creating similar groups; however, in educational research designed to identify proven best practices, random assignment of individuals is rarely possible; other methods including random assignment of intact units and nonrandom selection techniques are often used. Using a database simulation, this study explored the degree to which various methods might be effective at creating comparable groups. Given the complex dynamics of the teaching and learning process and the abundance of potentially confounding variables, it seems likely that comparison groups will always be dissimilar to some degree. While random assignment of individuals performed as expected when controlling for a single extraneous factor, the likelihood that comparison groups created in this manner will differ is extremely likely when multiple confounding variable are present. Based on the results of this study, random assignment of intact units is not an acceptable alternative to random assignment of individuals.

Keywords:

• research
• Experiments
• Evaluation
• Effectiveness
• Education policy

Notes

1. P(A or B) = 1–(1–α)^k, k = number of potentially confounding independent factors; α = 0.05.