Optimal designs for two-stage inference

Abstract

The analysis of screening experiments is often done in two stages, starting with factor selection via an analysis under a main effects model. The success of this first stage is influenced by three components: (1) main effect estimators’ variances and (2) bias, and (3) the estimate of the noise variance. Component (3) has only recently been given attention with design techniques that ensure an unbiased estimate of the noise variance. In this paper, we propose a design criterion based on expected confidence intervals of the first stage analysis that balances all three components. To address model misspecification, we propose a computationally-efficient all-subsets analysis and a corresponding constrained design criterion based on lack-of-fit. Scenarios found in existing design literature are revisited with our criteria and new designs are provided that improve upon existing methods.

Keywords:

• all-subsets analysis
• definitive screening design
• expected confidence interval criterion
• pre-selection variance estimator
• pure error

Acknowledgments

We would like to thank the editor and two anonymous referees for their valuable feedback that improved this article.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Jonathan Stallrich

Jonathan Stallrich is an Associate Professor in the Department of Statistics at North Carolina State University. He earned his Ph.D. in Statistics from Virginia Tech in 2014. His research interests include design and analysis of screening experiments, computer experiments, functional data analysis, and variable selection. In 2021, he and his coauthors were awarded the American Statistical Association’s Statistics in Physical Engineering Sciences Award for the paper, “Optimal EMG placement for a robotic prosthesis controller with sequential, adaptive functional estimation.” He is currently chair of the American Statistical Association’s Section on Physical and Engineering Sciences.

Michael McKibben

Michael McKibben is a Senior Statistician at the Johns Hopkins University Applied Physics Lab. He earned his PhD in Statistics from North Carolina State University in 2022. His primary research interest is the design and analysis of experiments. Current projects include applying design techniques for experiments in both industrial and machine learning applications. Michael previously worked at Google as a Data Scientist on projects such as estimating bias in bidding models and estimating treatment effects for privacy impacted data.