Efficient, nearly orthogonal-and-balanced, mixed designs: an effective way to conduct trade-off analyses via simulation

Abstract

Designed experiments are powerful methodologies for gaining insights into the behaviour of complex simulation models. In recent years, many new designs have been created to address the large number of factors and complex response surfaces that often arise in simulation studies, but handling discrete-valued or qualitative factors remains problematic. We propose a framework for generating a design, of specified size, that is nearly orthogonal and nearly balanced for any mix of factor types (categorical, numerical discrete, and numerical continuous) and mix of factor levels. These new designs allow decision makers structured methods for trade-off analyses in situations that are not necessarily amenable to other methods for choosing alternatives, such as simulation optimization or ranking and selection approaches. These new designs also compare well to existing approaches for constructing custom designs for smaller experiments, and may also be of interest for exploring computer models in domains where fewer factors are involved.

Keywords:

• simulation
• design of experiments
• integer programming
• statistics
• defense studies

Acknowledgements

This work was supported in part by grants from the Office of Naval Research and the U.S. Army Survivability and Lethality Analysis Directorate, and a grant of computer time from the DOD High Performance Computing Modernization Program at the Navy DSRC at the Stennis Space Center.