Performance of some multiple comparison tests under heteroscedasticity and dependency

Abstract

In this article, 18 multiple comparison tests are compared according to powers and type I error measures under some violations of analysis of variance assumptions with a Monte Carlo simulation study. Considered violations of assumptions are heterogeneity in subgroup variances and dependency between subgroups. Various numbers of subgroups and subgroup sizes are considered simultaneously with the violations of assumptions. Simulation results are analysed by using visual inspection, graphical representations, decision-tree and correspondence analyses. Wide inferences are drawn on the behaviour of considered tests with respect to measures used. Some general suggestions are given on which tests should be used or avoided under violations of assumptions.

Keywords:

• type I error
• statistical power
• correlated data
• variance heterogeneity
• statistical simulation

AMS Subject Classification :

• 62J14
• 62J15
• 65C05
• 37M05

Acknowledgements

We would like to thank the referee whose suggestions and questions improved the clarity and quality of the paper.