Analyzing Small Samples of Repeated Measures Data with the Mixed-Model Adjusted F Test

Abstract

This research examines the Type I error rates obtained when using the mixed model with the Kenward-Roger correction and compares them with the Between–Within and Satterthwaite approaches in split-plot designs. A simulated study was conducted to generate repeated measures data with small samples under normal distribution conditions. The data were obtained via three covariance matrices (unstructured, heterogeneous first-order auto-regressive, and random coefficients), the one with the best fit being selected according to the Akaike criterion. The results of the simulation study showed the Kenward-Roger test to be more robust, particularly when the population covariance matrices were unstructured or heterogeneous first-order auto-regressive. The main contribution of this study lies in showing that the Kenward–Roger method corrects the liberal Type I error rates obtained with the Between–Within and Satterthwaite approaches, especially with positive pairings between group sizes and covariance matrices.

Keywords:

• Kenward–Roger method
• Linear mixed model
• Repeated measures
• Simulation
• Type I error rate

Mathematics Subject Classification:

• 62F35
• 62H10
• 62H12
• 62H15
• 62C05

Acknowledgments

This research was supported by Grants MEC-SEJ2005-01923 and MEC-SEJ2005-01883 from Spain's Ministry of Education and Science.

Notes

Note: UN=unstructured model; ARH=heterogeneous first-order autroregressive model; RC=random coefficients model; ϵ = sphericity index.

Note: J: groups, K: number of repeated measurements, ϵ: sphericity, N: total sample size, n ₁, n ₂, and n ₃: group sizes, Δ n _j : variance coefficient of the group size, =/ ≠ : homogeneity/heterogeneity of covariance matrices between groups, null/+/−: null/positive/negative pairing of group sizes and covariance matrices.

Note: The structure with the highest selection percentage is in bold.

Note: The structure with the highest selection percentage is in bold.

Note: The structure with the highest selection percentage is in bold.

Note: BW, SW, and KR=methods for calculating the degrees of freedom (Between–Within, Satterthwaite, and Kenward–Roger). In bold=liberal.

Note: BW, SW, and KR=methods for calculating the degrees of freedom (Between–Within, Satterthwaite, and Kenward–Roger). In bold=liberal.

Note: BW, SW, and KR=methods for calculating the degrees of freedom (Between–Within, Satterthwaite, and Kenward–Roger). In bold=liberal; in italics=conservative.

Note: BW, SW, and KR=methods for calculating the degrees of freedom (Between–Within, Satterthwaite, and Kenward–Roger). In bold=liberal.

Note: BW, SW, and KR=methods for calculating the degrees of freedom (Between–Within, Satterthwaite, and Kenward–Roger). In bold=liberal; in italics=conservative.

Note: BW, SW, and KR=methods for calculating the degrees of freedom (Between–Within, Satterthwaite, and Kenward–Roger). In bold=liberal; in italics=conservative.