Statistical Significance Levels of Nonparametric Tests Biased by Heterogeneous Variances of Treatment Groups

Abstract

The statistical significance levels of the Wilcoxon-Mann-Whitney test and the Kruskal-Wallis test are substantially biased by heterogeneous variances of treatment groups—even when sample sizes are equal. Under these conditions, the Type I error probabilities of the nonparametric tests, performed at the .01, .05, and .10 significance levels, increase by as much as 40%-50% in many cases and sometimes as much as 300%. The bias increases systematically as the ratio of standard deviations of treatment groups increases and remains fairly constant for various sample sizes. There is no indication that Type I error probabilities approach the significance level asymptotically as sample size increases.

Key Words:

• homogeneity of variance
• Kruskal-Wallis test
• nonparametric tests
• rank-randomization
• significance level
• statistical significance tests
• Student t test
• Type I error
• Wilcoxon-Mann-Whitney test