Analysis of Type I Error Rates of Univariate and Multivariate Procedures in Repeated Measures Designs

Abstract

We compared the robustness of univariate and multivariate statistical procedures to control Type I error rates when the normality and homocedasticity assumptions were not fulfilled. The procedures we evaluated are the mixed model adjusted by means of the SAS Proc Mixed module, and Bootstrap-F approach, Brown–Forsythe multivariate approach, Welch–James multivariate approach, and Welch–James multivariate approach with robust estimators. The results suggest that the Kenward Roger, Brown–Forsythe, Welch–James, and Improved Generalized Aprroximate procedures satisfactorily kept Type I error rates within the nominal levels for both the main and interaction effects under most of the conditions assessed.

Keywords:

• Assumption non fulfillment
• Repeated measure designs
• Type I error

Acknowledgment

Financial support of this work by the Ministerio de Ciencia e Innovación de España (Ref.: PSI-2008-03624/PSIC) is gratefully acknowledged. Moreover, the authors thank the referees for their comments, which improved the article.

Notes

Abbreviations: N=Group size; D=Distribution; NS=Normal and biased distribution; CE=Covariance structure; R=Relationship dispersion matrices and the group sizes; RC=Random coefficients matrix; UN=Unstructured matrix; ARH=First order autoregressive heterogeneous matrix; F=F procedure without corrected degreees of freedom; S=Fai and Cornelius procedure; KR=Kenward–Roger procedure; BF=Brown–Forsythe procedure; WJ=Welch–James procedure; BT=Trimmed nonparametric bootstrap; TM=Trimmed means WJ procedure; IGA=Improved Generalized Approximate procedure; Bold type=Error rates.

Abbreviations: N=Group size; D=Distribution; NS=Normal and biased distribution; CE=Covariance structure; R= Relationship dispersion matrices and the group sizes; RC=Random coefficients matrix; UN=Unstructured matrix; ARH=First order autoregressive heterogeneous matrix; F=F procedure without corrected degreees of freedom; S=Fai and Cornelius procedure; KR=Kenward–Roger procedure; BF=Brown–Forsythe procedure; WJ=Welch–James procedure; BT=Trimmed nonparametric bootstrap; TM=Trimmed means WJ procedure; IGA=Improved Generalized Approximate procedure; Bold type=Error rates.