Design and analysis of confirmation experiments

Abstract

The statistical literature and practitioners have long advocated the use of confirmation experiments as the final stage of a sequence of designed experiments to verify that the optimal operating conditions identified as part of a response surface methodology strategy are attainable and able to achieve the value of the response desired. However, until recently there has been a gap between this recommendation and details about how to perform an analysis to quantitatively assess whether the confirmation runs are adequate. Similarly, there has been little in the way of specific recommendations for the number and nature of the confirmation runs that should be performed. In this article, we propose analysis methods to assess agreement between the mean response from previous experiments and the confirmation experiment, and suggest a strategy for the design of confirmation experiments that more fully explores the region around the optimum.

Keywords:

• commensurate probability
• confirmation runs
• equiradial design
• probability of agreement
• response surface methodology
• sequential experimentation
• validation

Acknowledgments

The authors acknowledge the work of Ockuly et al. (2017) that provides an excellent and comprehensive review of response surface methodology in practice. Furthermore, they thank Dr. Maria Weese for generously sharing references from their literature review that mentioned confirmation runs.

Additional information

Notes on contributors

Nathaniel T. Stevens

Dr. Stevens is an Assistant Professor of Statistics in the Statistics and Actuarial Science Department at the University of Waterloo. At the time of this research, Dr. Stevens was affiliated with the Mathematics and Statistics Department at the University of San Francisco. He is a member of the ASQ. Email: [email protected].

Christine M. Anderson-Cook

Dr. Anderson-Cook is a research scientist at the Los Alamos National Laboratory. She is a fellow of the ASA and ASQ. Her email address is [email protected].