Meaningful sensitivities: A new family of simulation sensitivity measures

Abstract

Sensitivity analysis quantifies how a model output responds to variations in its inputs. However, the following sensitivity question has never been rigorously answered: How sensitive is the mean or variance of a stochastic simulation output to the mean or variance of a stochastic input distribution? This question does not have a simple answer because there is often more than one way of changing the mean or variance of an input distribution, which leads to correspondingly different impacts on the simulation outputs. In this article we propose a new family of output-property-with-respect-to-input-property sensitivity measures for stochastic simulation. We focus on four useful members of this general family: sensitivity of output mean or variance with respect to input-distribution mean or variance. Based on problem-specific characteristics of the simulation we identify appropriate point and error estimators for these sensitivities that require no additional simulation effort beyond the nominal experiment. Two representative examples are provided to illustrate the family, estimators and interpretation of results.

Keywords:

• Local sensitivity analysis
• stochastic simulation

Acknowledgments

We thank the Department Editor, Associate Editor and two referees for helpful improvements to the paper. Some background material in this paper was previously published in Jiang et al. (2019).

Additional information

Funding

This research was partially supported by National Science Foundation Grant No. CMMI-1634982.

Notes on contributors

Xi Jiang

Xi Jiang is an Operations Research Specialist at SAS Institute. She received her Ph.D. in Industrial Engineering and Management Sciences at Northwestern University, where she majored in applied statistics and statistical learning, with minors in analytics and optimization. Her research focus is on stochastic simulation methodology and uncertainty and sensitivity analysis.

Barry L. Nelson

Barry L. Nelson is the Walter P. Murphy Professor of the Department of Industrial Engineering and Management Sciences at Northwestern and a Visiting Scholar at Lancaster University. His research is on the design and analysis of computer simulation experiments on models of discrete-event, stochastic systems, including methodology for simulation optimization, quantifying and reducing model risk, variance reduction, output analysis, metamodeling and multivariate input modeling. He has published numerous papers and three books, including Foundations and Methods of Stochastic Simulation: A First Course (Springer, 2013). Nelson is a Fellow of INFORMS and IISE.

L. Jeff Hong

Jeff Hong is Fudan Distinguished Professor, Hongyi Chair Professor, Head of Department of Management Science and Associate Dean of School of Data Science at Fudan University. His research interests include stochastic simulation, stochastic optimization, financial risk management and supply chain management. He is currently the Associate Editor-in-Chief of the Journal of Operations Research Society of China, the Simulation Area Editor of Operations Research, an Associate Editor of Management Science, and the President of INFORMS Simulation Society.