The use of search experimentation in discrete-event simulation practice

Abstract

The practice of DES experimentation has not been rigorously assessed in over a decade. Past studies of DES practice report little transfer of experimentation theory into real-world application. We conducted an international survey of over 300 modellers to investigate the extent to which simulation optimisation, meta-modelling and design of experiments are used in practice. Over the last decade there has been substantial growth in the use of optimisation and to a lesser extent design of experiments to tackle practical problems. However, users rarely make use of optimisers bundled with commercial software, opting instead for custom or third-party solutions. Outside of academia, the use of methods is hampered by a lack of application knowledge and a persisting view that such techniques are not necessary. It is clear that academics must not become complacent regarding the dissemination of theory into common practice and continue to reach out to industry users.

Keywords:

• simulation
• practice
• experimentation
• simulation optimisation
• meta-model
• design of experiment

Electronic supplementary material

The online version of this article (doi:10.1057/jors.2014.79) contains supplementary material, which is available to authorized users.

Supplementary information accompanies this article on the Journal of the Operational Research Society website (www.palgrave-journals.com/jors)

Electronic supplementary material

The online version of this article (doi:10.1057/jors.2014.79) contains supplementary material, which is available to authorized users.

Supplementary information accompanies this article on the Journal of the Operational Research Society website (www.palgrave-journals.com/jors)

Acknowledgements

A working paper (Hoad et al, 2012) discussing the first pilot study of eight UK DES modellers was previously published in the Proceedings of the Operational Research Society Simulation Workshop 2012 (SW12). We also acknowledge the invaluable assistance of Ashwin Rajaskanda and Shijie Qian, who as part of their MSc dissertations produced a draft version of the survey used in this study (Rajaskanda, 2011) as well as relevant literature reviews (Qian, 2011; Rajaskanda, 2011). Thomas Monks is funded by the National Institute for Health Research (NIHR) Collaborations for Leadership in Applied Health Research and Care (CLAHRC). The views expressed in this publication are those of the author and not necessarily those of the National Health Service, the NIHR, or the Department of Health.