Power Computations for Polynomial Change Models in Block-Randomized Designs

Abstract

Education experiments frequently assign students to treatment or control conditions within schools. Longitudinal components added in these studies (e.g., students followed over time) allow researchers to assess treatment effects in average rates of change (e.g., linear or quadratic). We provide methods for a priori power analysis in three-level polynomial change models for block-randomized designs. We discuss unconditional models and models with covariates at the second and third level. We illustrate how power is influenced by the number of measurement occasions, the sample sizes at the second and third levels, and the covariates at the second and third levels.

KEYWORDS:

• HLM
• growth curve analysis
• sampling
• effect size
• univariate (classical)
• design study

Notes

Notes

1 It is noteworthy that when ${\omega }_{Tpp}^2+{\tau }_{pp}^2=1$, ${\sigma }_p^2$ needs to be rescaled to keep the ICC and the reliability coefficient unchanged. For example, in the linear model when ${\omega }_{T11}^2=1$, ${\tau }_{11}^2=4$, and ${\sigma }_1^2=5$, the reliability coefficient is equal to 4/9. If we were to set ${\omega }_{T11}^2+{\tau }_{11}^2=1$ and ${\omega }_{T11}^2=0.20$, ${\tau }_{11}^2=0.80$ then ${\sigma }_1^2$ has to be rescaled accordingly (i.e., ${\sigma }_1^2=1)$ in order for the reliability coefficient to remain the same (i.e., 4/9).

2 The variance ${\sigma }_e^2$ is kept constant and ${\sigma }_p^2$ is allowed to vary as a function of G. In particular, in the linear model the percentage of the sum of the variances (${\sigma }_e^2$, ${\tau }_{11}^2$, and ${\omega }_{T11}^2$) that is accounted for by ${\sigma }_e^2$ is kept constant. Similarly, in the quadratic model the percentage of the sum of the variances (${\sigma }_e^2$, ${\tau }_{22}^2$, and ${\omega }_{T22}^2$) captured by ${\sigma }_e^2$ is kept constant. For example, using data from Project STAR, ${\sigma }_e^2$ accounts for 90% of the sum of the variances in the linear model and 75% of the sum of the variances in the quadratic model.

3 In Tables 1, 2, and 4 (and in Tables 5, and 6), we varied the ratio ${\tau }_{pp}^2/{\omega }_{Tpp}^2$ and kept constant the percentage of the sum of the variances captured by ${\sigma }_e^2$. In the linear model, ${\sigma }_e^2$ accounts for 90% of the sum of the variances (${\sigma }_e^2$, ${\tau }_{11}^2$, and ${\omega }_{T11}^2$) and the remaining 10% is captured by ${\tau }_{11}^2$ and ${\omega }_{T11}^2$. In the quadratic model, ${\sigma }_e^2$ accounts for 75% of the sum of the variances (${\sigma }_e^2$, ${\tau }_{22}^2$, and ${\omega }_{T22}^2$) and the remaining 25% is captured by ${\tau }_{22}^2$ and ${\omega }_{T22}^2$.

4 In Table 3, we varied the ratio ${\tau }_{pp}^2/{\sigma }_e^2$ and kept constant the percentage of the sum of the variances captured by ${\omega }_{Tpp}^2$. Using data from Project STAR, the variance ${\omega }_{T11}^2$ captures 6.30% of the sum of the variances (${\sigma }_e^2$, ${\tau }_{11}^2$, and ${\omega }_{T11}^2$) in the linear model. In the quadratic model, the variance ${\omega }_{T22}^2$ accounts for 23% of the sum of the variances (${\sigma }_e^2$, ${\tau }_{22}^2$, and ${\omega }_{T22}^2$). In the linear model the remaining 93.70% of the sum of the variances is accounted for by ${\sigma }_e^2$ and ${\tau }_{11}^2$. Similarly, in the quadratic model, the remaining 77% of the sum of the variances is accounted for by ${\sigma }_e^2$ and ${\tau }_{22}^2$.