Optimal budget allocation for stochastic simulation with importance sampling: Exploration vs. replication

Abstract

This article investigates a budget allocation problem for optimally running stochastic simulation models with importance sampling in computer experiments. In particular, we consider a two-level (or nested) simulation to estimate the expectation of the simulation output, where the first-level draws random input samples and the second-level obtains the output given the input from the first-level. The two-level simulation faces the trade-off in allocating the computational budgets: exploring more inputs (exploration) or exploiting the stochastic response surface at a sampled point in more detail (replication). We study an appropriate computational budget allocation strategy that strikes a balance between exploration and replication to minimize the variance of the estimator when importance sampling is employed at the first-level simulation. Our analysis suggests that exploration can be beneficial than replication in many practical situations. We also conduct numerical experiments in a wide range of settings and wind turbine case study to investigate the trade-off.

Keywords:

• Computer experiment
• Monte Carlo sampling
• reliability
• variance reduction

Acknowledgments

We would like to thank the Editor, Department Editor, Associate Editor and anonymous reviewers for their constructive comments on various aspects of this work.

Additional information

Funding

This work was partially supported by the Basic Science Research Program through National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2016R1D1A1B04933453) and the U.S. National Science Foundation (Grant/Award number: IIS-1741166).

Notes on contributors

Young Myoung Ko

Young Myoung Ko received BS and MS degrees in Industrial Engineering from Seoul National University, Seoul, South Korea, and a PhD degree in industrial engineering from Texas A&M University, College Station, TX, USA, in 1998, 2000, and 2011 respectively. He is currently an associate professor with the Department of Industrial and Management Engineering, Pohang University of Science and Technology (POSTECH), Pohang, South Korea, where he focuses on simulation and optimization of stochastic systems, such as telecommunication networks, ICT infrastructure, and renewable energy systems.

Eunshin Byon

Eunshin Byon is an Associate Professor in the Department of Industrial and Operations Engineering at the University of Michigan. She received her BS and MS in industrial and systems engineering from the Korea Advanced Institute of Science and Technology (KAIST) and a PhD in industrial and systems engineering from Texas A&M University. Her research interests include optimizing operations of renewable systems, data science, quality and reliability engineering, and sustainability. She is a member of IISE, INFORMS, and IEEE.