Use of a preliminary test in comparing two sample means

Abstract

Given independent samples from two distributions it is desired to test equality of means. It is well known that unequal variances can bias the t-test, particularly when the sample sizes differ. Welch's test has been shown to ameliorate this bias. Practitioners often test first for equality of variances, using an alternate test such as Welch's only if equality is rejected. By simulation it is shown that the use of the preliminary test does not remove the bias fully. This is true for normal, uniform, and exponential distributions.

Keywords:

• simulation
• variance heterogeneity
• pooling
• Behrens-Fisher problem
• two-sample tests
• two-sample t-test
• Welch's test
• non-normal distribution