Simulation-optimization via Kriging and bootstrapping: a survey

Abstract

This article surveys optimization of simulated systems. The simulation may be either deterministic or random. The survey reflects the author’s extensive experience with simulation-optimization through Kriging (or Gaussian process) metamodels, analysed through parametric bootstrapping for deterministic and random simulation and distribution-free bootstrapping (or resampling) for random simulation. The survey covers: (1) simulation-optimization through ‘efficient global optimization’ using ‘expected improvement’ (EI); this EI uses the Kriging predictor variance, which can be estimated through bootstrapping accounting for the estimation of the Kriging parameters; (2) optimization with constraints for multiple random simulation outputs and deterministic inputs through mathematical programming applied to Kriging metamodels validated through bootstrapping; (3) Taguchian robust optimization for uncertain environments, using mathematical programming—applied to Kriging metamodels—and bootstrapping to estimate the variability of the Kriging metamodels and the resulting robust solution; (4) bootstrapping for improving convexity or preserving monotonicity of the Kriging metamodel.

Keywords:

• simulation
• optimization
• stochastic process
• non-linear programming
• risk

Acknowledgements

This paper is based on a seminar I presented at the conference ‘Stochastic and noisy simulators’ organized by the French Research Group on Stochastic Analysis Methods for COdes and NUMerical treatments called ‘GDR MASCOT-NUM’ in Paris on 17 May 2011. I also presented a summary of this paper at the workshop ‘Accelerating industrial productivity via deterministic computer experiments and stochastic simulation experiments’ organized at the Isaac Newton Institute for Mathematical Sciences in Cambridge on 5–9 September 2011. And I presented a summary at the ‘First Workshop On Applied Meta-Modeling’, Cologne University of Applied Sciences, Gummersbach, Germany, 16 November 2012. I like to thank the co-authors with whom I wrote the various articles that are summarized in this survey. I also thank two anonymous referees for their comments on the original version of this article.