The False Discovery Rate for Multiple Testing in Factorial Experiments

Abstract

Identifying the important factors and effects is an important goal in the analysis of two-level fractional factorial experiments. In moderate and large experiments, the large number of potential effects presents problems of multiple statistical inference. In this work we apply the false discovery rate (FDR) idea in multiple inference to effect identification in factorial experiments. The FDR is the expected proportion of inert effects among those declared active, which we believe is a more relevant quantity to control than the probability of a single erroneous declaration, the criterion that has been adopted in previous studies. We show how to combine the control of FDR with popular methods for estimating contrast standard error in unreplicated experiments. We present simulations of unreplicated 16- and 32-run experiments that illustrate the improvements in power that can be obtained by controlling the FDR. We also analyze data from an actual 128-run experiment with replication, in which the FDR identifies many more active effects than other methods.

KEY WORDS :

• Active effects
• Dong's method
• Experimentwise error rate
• Half-normal plot
• Lenth's method
• Multiple inference