Using Definitive Screening Designs to Identify Active First- and Second-Order Factor Effects

Abstract

Definitive screening designs (DSDs) were recently introduced by Jones and Nachtsheim (2011b). The use of three-level factors and the desirable aliasing structure of the DSDs make them potentially suitable for identifying main effects and second-order terms in one stage of experimentation. However, as the number of active effects approaches the number of runs, the performance of standard model-selection routines will inevitably degrade. In this paper, we characterize the ability of DSDs to correctly identify first- and second-order model terms as a function of the level of sparsity, the number of factors in the design, the signal-to-noise ratio, the model type (unrestricted or following strong heredity), the model-selection technique, and the number of augmented runs. We find that minimum-run-size DSDs can be used to identify active terms with high probability as long as the number of effects is less than or equal to about half the number of runs and the signal-to-noise ratios for the active effects are above about 2.0. We also find that if minimum-run-size designs are augmented with four or more runs, the number of model terms that can be identified with high probability increases substantially. Among the model-selection methods investigated, we found that both Lasso and the Gauss–Dantzig selector (both based on AICc) can be used to effectively identify active model terms in the presence of unrestricted models. For models following strong heredity, the SHIM method developed by Choi et al. (2010) was the best among methods tested that were designed for the strong-heredity case.

• Dantzig Selector
• Foldover Design
• Lasso
• Model Selection
• Supersaturated Design

Additional information

Notes on contributors

Anna Errore

Dr. Errore is a doctoral student in the Supply Chain and Operations Department at the University of Minnesota. Her email address is [email protected].

Bradley Jones

Dr. Jones is Principal Research Fellow for the JMP Division of SAS. His email address is [email protected].

William Li

Dr. Li is the Xian Dong Eric Jing Professor for Business Teaching and Research in the Supply Chain and Operations Department at the University of Minnesota. His email address is [email protected].

Christopher J. Nachtsheim

Dr. Nachtsheim is Professor and Frank A. Donaldson Chair of Operations Management in the Supply Chain and Operations Department at the University of Minnesota. His email address is [email protected].