Assessment of a Preliminary F-Test Solution to the Behrens-Fisher Problem

Abstract

A detailed analysis of the effect of preliminary statistical (i.e., data-based) model selection on confidence intervals (based on the same data) constructed on the (incorrect) assumption that the selected model had been specified a priori has been carried out only recently; see Arabatzis et al. (1989), Regal and Hook (1991), Chiou and Han (1995a,b), Kabaila (1995), and Han (1998). Kabaila (1998) describes a technique of confidence interval (CI) construction that properly accounts for preliminary statistical model selection (i.e., the resulting confidence interval has the desired minimum coverage probability) and applies this technique to a linear regression problem. In this article we apply this technique to CI construction for the difference of two normal population means after a preliminary F-test that their population variances are equal. This preliminary test can be motivated by the prior belief (though not a certainty) that the population variances are equal. It is therefore of interest to examine the expected length of the adjusted CI found using the technique of Kabaila (1998) to see the extent to which it reflects this prior belief. It is plausible, because of the way it is constructed, that for small and medium sample sizes this CI has the property that its expected length is relatively small when these variances are equal. We use an extensive numerical investigation to determine the sample sizes for which this CI has this property.

Key Words:

• Preliminary test
• Welch confidence interval