Pairwise comparisons of means under realistic nonnormality, unequal variances, outliers and equal sample sizes

Abstract

A Monte Carlo simulation evaluated six pairwise multiple comparison procedures for controlling Type I error rates, any-pair power, and all-pairs power. Realistic conditions of nonnormality were based on a previous survey, and the effects of outliers were investigated. Variance ratios varied from 1:1 to 8:1. Evaluated procedures included Tukey's honestly significant difference (HSD) preceded by an F-test, the Hayter–Fisher, the Games–Howell procedure tested at 0.9α, the Peritz with F-tests, the Peritz with Brown–Forsythe tests, and the Peritz with Alexander–Govern tests. Peritz with Brown–Forsythe procedure shows the greatest robustness in Type I error control. Any-pair power is generally best with the Hayter–Fisher, whereas all-pairs power is best with the Peritz F-test procedure. Even though the Hayter–Fisher shows only slightly lower all-pair power rates than the Peritz, it is still much easier to perform.

Keywords:

• multiple hypothesis testing
• all pairwise comparisons
• ANOVA
• Tukey test