A simple data-driven fallback procedure for multiple comparisons

Abstract

We propose a data-driven fallback procedure to combine the positive aspects of both data-driven multiple comparison procedures and those relying on predetermined strategies. The proposed procedure tests the hypotheses based on the ordering of p-values, but the significance level of the test at each sequential step is accumulated in the manner of the fallback procedure. It is proven that the new procedure strongly controls the familywise error rate and is uniformly more powerful than the weighted Holm procedure for more than two hypotheses. The simulation study shows that the new procedure is more powerful than the fallback in most cases.

Keywords:

• Familywise error rate
• Holm procedure
• multiple hypotheses testing
• strong and weak control
• fallback procedure

Acknowledgment

We sincerely thank the editor and the referee for their valuable comments in the improvement of the quality of our manuscript.