Analysis of Optimization Experiments

Abstract

The typical practice for analyzing industrial experiments is to identify statistically significant effects with a 5% level of significance and then to optimize the model containing only those effects. In this article, we illustrate the danger in utilizing this approach. We propose methodology using the practical significance level, which is a quantity that a practitioner can easily specify. We also propose utilizing empirical Bayes estimation, which gives shrinkage estimates of the effects. Interestingly, the mechanics of statistical testing can be viewed as an approximation to empirical Bayes estimation, but with a significance level in the range of 15–40%. We also establish the connections that our approach has with a less known but intriguing technique proposed by Taguchi, known as the beta coefficient method. A real example and simulations are used to demonstrate the advantages of the proposed methodology.

• Empirical Bayes Method
• Practical Significance Level
• Shrinkage Estimation
• Variable Selection

Additional information

Notes on contributors

V. Roshan Joseph

Dr. Joseph is an Assistant Professor in the H. Milton Stewart School of Industrial and Systems Engineering at Georgia Tech. He is a Member of ASQ. His email address is [email protected].

James Dillon Delaney

Dr. Delaney is a Visiting Assistant Professor in the Department of Statistics at Carnegie Mellon University. His email address is [email protected].