A comparison of methods for identifying location effects in unreplicated fractional factorials in the presence of dispersion effects

Abstract

Most methods for identifying location effects in unreplicated fractional factorial designs assume homoscedasticity of the response values. However, dispersion effects in the underlying process may create heteroscedasticity in the response values. This heteroscedasticity may go undetected when identification of location effects is pursued. Indeed, methods for identifying dispersion effects typically require first modeling location effects. Therefore, it is imperative to understand how methods for identifying location effects function in the presence of undetected dispersion effects. We used simulation studies to examine the robustness of four different methods for identifying location effects—Box and Meyer (1986), Lenth (1989), Berk and Picard (1991), and Loughin and Noble (1997)—under models with one, two, or three dispersion effects of varying sizes. We found that the first three methods usually performed acceptably with respect to error rates and power, but the Loughin–Noble method lost control of the individual error rate when moderate-to-large dispersion effects were present.

Keywords:

• Berk–Picard
• Box–Meyer
• confounding
• Lenth
• Loughin–Noble

About the authors

Dr. Loughin is Chair and Professor in the Department of Statistics and Actuarial Science. E-mail: [email protected].

Ms. Zhang is Manager in the Risk Assurance Department. E-mail: [email protected]

Acknowledgments

We gratefully acknowledge two reviewers whose comments led to improvements in the content and presentation of this work.

Funding

This work was supported by the National Science and Engineering Research Council of Canada.

Notes

1 It is somewhat easier to measure heteroscedasticity in real data with continuous explanatory variables than in unreplicated $2^k$ factorials. In separate research, Gelfand (2015) studied 25 such data sets exhibiting “significant” heteroscedasticity, as determined by a statistically significant positive slope in a plot of the absolute values of residuals vs. fitted values. Estimated standard deviation ratios between regions of low and high variance were found to range from 1.1 to 64.4 (reduced to 15.6 with a possible outlier removed), with a median of 3.2 and a third quartile of 5.9. Furthermore, Henrey and Loughin (2017) performed joint analysis of location and dispersion effects on three often-studied experiments and found that two have apparent active dispersion effects. In both of these, the effect estimate corresponds to a standard deviation ratio of roughly 5. Thus, our chosen values represent a range of realistic dispersion effect sizes, although the upper two levels might be somewhat extreme.