21st century screening experiments: What, why, and how

ABSTRACT

The primary aim of screening experiments is to identify the active factors; that is, those having the largest effects on the response of interest. Large factor effects can be either main effects, two-factor interactions (2FIs), or even strong curvature effects. Because the number of runs in a screening experiment is generally on the order of the number of factors, the designs rely heavily on the factor or effect sparsity assumption. That is, practitioners performing such experiments must be willing to assume that only a small fraction of the factors or effects are active.

Traditional screening designs such as regular fractional factorial and Plackett-Burman designs employ factors at two levels only. Though they have orthogonal linear main effects, such designs cannot uniquely identify factors with strong curvature effects.

Definitive screening designs (DSDs) have many desirable properties that make them appealing alternatives to other screening design methods. They are orthogonal for the main effects. In addition, main effects are orthogonal to all second-order effects and second-order effects are not confounded with each other. In addition, quadratic effects of every factor are estimable. For more than five factors, a DSD projects onto any three factors so that a full quadratic model in those three factors is estimable with reasonable efficiency. As a result, when three or fewer factors turn out to be important, follow-up optimization experiments may not be necessary.

All this begs the question, “Are DSDs really as good as they are advertised to be?”  This article addresses this question with an even-handed comparison of the various screening approaches. It also considers the sparsity assumption common to all screening designs and provides some guidance for quantifying what effect sparsity means for both traditional screening designs and DSDs.

KEYWORDS:

• definitive screening design
• orthogonal
• sparsity assumption

Additional information

Notes on contributors

Bradley Jones

Bradley Jones is the Principal Research Fellow in the JMP Division of SAS where he is responsible for implementing the DOE tools in JMP. His primary field of research is the design and analysis of experiments. With Chris Nachtsheim he is the coinventor of Definitive Screening Designs. He served as Editor of the Journal of Quality Technology and is an Associate Editor of Technometrics. He is a Fellow of the American Statistical Association and twice a winner of the Brumbaugh and Lloyd S. Nelson awards. He is also winner of the Jack Youden prize and the Ziegel Prize.