Accuracy and Precision of an Effect Size and Its Variance From a Multilevel Model for Cluster Randomized Trials: A Simulation Study

Abstract

This article investigates an effect size (MLM ES) and its variance for cluster randomized trials based on parameter estimates from multilevel modeling analysis. Accuracy and precision of MLM ES were evaluated using Monte Carlo simulation methods and compared with the performance of an effect size, computed from summary statistics, proposed by Hedges (2007; Hedges' d_B ). Simulation results indicated that MLM ES had acceptable accuracy in all conditions, also demonstrating efficiency and consistency. With small sample sizes, MLM ES did not suffer from the same negative bias as Hedges' d_B due to overestimation of between-cluster variance. With large sample sizes, MLM ES and Hedges' d_B were comparable for accuracy and efficiency. Both MLM ES and Hedges' d_B showed considerable bias in some conditions when cluster sizes were unequal. An illustrative example using real data was provided.

Notes

¹The term Hedges' d_B was chosen to reflect notation in Hedges (2007). This is not to be confused with Hedges' g (Hedges, 1981), a bias-corrected standardized mean difference effect size.

²Some notation has been changed from Hedges (2007) for consistency.