Assessing Equality of Means Using the Overlap of Range-Preserving Confidence Intervals

ABSTRACT

Hypothesis testing procedures where equality of means is assessed at a prespecified level based on the (non-)overlap of confidence intervals are discussed. Assessing statistical significance via the (non-)overlap of two confidence intervals with an appropriate confidence level provides a simple and effective way of visually understanding statistical results. This article extends previous approaches by considering range-preserving confidence intervals where the values in such intervals are in the allowable range of the parameter of interest. To obtain reliable procedures, appropriate effective degrees of freedom are suggested by considering the Welch-Satterthwaite equation for both independent two-sample and paired-sample cases. The proposed procedures also allow users to express results in terms of commonly used scale-free effect sizes, which are highly useful for interpreting parameters of interest. Simulation results suggest that the proposed procedures may be robust to unequal or small sample sizes, nonnormal distributions, heterogeneous variances, and various degrees of correlation. A real-life application from a study in cognitive psychology illustrates the effectiveness of the proposed procedures.

KEYWORDS:

• Effect size
• Two-sample problem
• Visualization
• Welch-Satterthwaite equation