Are “wrong” models useful? A qualitative study of discrete event simulation modeller stories

References

• Balci, O., Nance, R. E., & Arthur, J. D. (2002). Expanding our horizons in verification, validation, and accreditation research and practice. In E. Yücesan, C.-H. Chen, J. L. Snowdon, and J. M. Charnes, eds. Proceedings of the 2002 Winter Simulation Conference, (pp.653–663).
   Google Scholar
• Bankes, S. (1993). Exploratory Modeling for policy analysis. Operations Research, 41(3), 435–449. https://doi.org/10.1287/opre.41.3.435
   Web of Science ®Google Scholar
• Bankes, S. (1998). Policy analysis for complex and uncertain systems through computational experiments. In IEEE Aerospace Conference, vol.1, (pp.9–21).
   Google Scholar
• Bazeley, P. (2013). Qualitative data analysis: Practical strategies. SAGE Publications.
   Google Scholar
• Box, G. E. P., & Draper, N. R. (1987). Empirical model-building and response surfaces. John Wiley & Sons.
   Google Scholar
• Brinkmann, S., & Kvale, S. (2015). Interviews: Learning the craft of qualitative research interviewing (3rd ed.). Sage Publications.
   Google Scholar
• Brooks, R. J., & Tobias, A. M. (1996). Choosing the best model: Level of detail, complexity, and model performance. Mathematical and Computer Modelling, 24(4), 1–14. https://doi.org/10.1016/0895-7177(96)00103-3
   Web of Science ®Google Scholar
• Bryman, A. (2012). Social research methods (4th ed.). Oxford University Press.
   Google Scholar
• Castaño, Y. Á. (1999). Dynamic Behavior of NPD. Proceedings of the 17th International Conference of the System Dynamics Society and 5th Australian and New Zealand Systems Conference, (p.10).
   Google Scholar
• Chussil, M. (2005). With all this intelligence, why don’t we have better strategies? Journal of Business Strategy, 26(1), 26–33. https://doi.org/10.1108/02756660510575023
   Google Scholar
• Eskinasi, M., & Fokkema, E. (2006). Lessons learned from unsuccessful modelling interventions. Systems Research and Behavioral Science, 23(4) , 483–492. https://doi.org/10.1002/sres.774
   Web of Science ®Google Scholar
• Gass, S. I. (1983). Decision-aiding models: Validation, assessment, and related issues for policy analysis. Operations Research, 31(4), 603–631. https://doi.org/10.1287/opre.31.4.603
   Web of Science ®Google Scholar
• Gelfert, A. (2019). Assessing the credibility of conceptual models. In C. Beisbart & N. Saam. (Eds.), Computer Simulation Validation. Simulation Foundations, Methods and Applications. (pp. 249–269). Springer. https://doi.org/10.1007/978-3-319-70766-2_10
   Google Scholar
• Gogi, A. (2016). Insight generation in simulation studies : an empirical exploration. [ Thesis]. School of Business and Economics, Loughborough University.
   Google Scholar
• Goldberg, J., Districh, R., Chen, J. M., Mitwasi, M., Valenzuela, T., & Criss, E. (1990). A simulation model for evaluating a set of emergency vehicle base locations: Development, validation, and usage. Socio-Economic Planning Sciences, 24(2), 125–141. https://doi.org/10.1016/0038-0121(90)90.017-2
   PubMed Web of Science ®Google Scholar
• Harper, A., Mustafee, N., & Yearworth, M. (2021). Facets of trust in simulation studies. European Journal of Operational Research, 289(1), 197–213. https://doi.org/10.1016/j.ejor.2020.06.043
   Web of Science ®Google Scholar
• Hinkkanen, A., Lang, K. R., & Whinston, A. B. (1995). On the usage of qualitative reasoning as an approach towards enterprise modelling. Annals of Operations Research, 55(1), 101–137. https://doi.org/10.1007/BF02031718
   Google Scholar
• Hodges, J. S. (1991). Six (Or So) things you can do with a bad model. Operations Research, 39(3), 355–365. https://doi.org/10.1287/opre.39.3.355
   Web of Science ®Google Scholar
• Hodges, J., & Dewar, J. (1992). Is it you or your model talking? A framework for model validation. RAND Publication Series.
   Google Scholar
• Hooker, J. N. (2007). Good and bad futures for constraint programming (and operations research). Constraint Programming Letters, 1, 21–32. http://public.tepper.cmu.edu/jnh/cpfutures.pdf
   Google Scholar
• Jessop, A. (2002). Exploring structure: A block model approach. Civil Engineering and Environmental Systems, 19(4), 263–284. https://doi.org/10.1080/10.286.600.215.048
   Web of Science ®Google Scholar
• Kasaie, P., Kelton, W. D., Ancona, R., Ward, M. J., Froehle, C. M., & Lyons, M. S. (2017). Lessons learned from the development and parameterization of a computer simulation model to evaluate task modification for health care providers. In Consensus conference, (pp.1–12).
   Google Scholar
• Kleijnen, J. P. C. (1995). Statistical validation of simulation models. European Journal of Operational Research, 87(1), 21–34. https://doi.org/10.1016/0377-2217(95)00132-A
   Web of Science ®Google Scholar
• Kolkman, D. A., Campo, P., Balke-Visser, T., & Gilbert, N. (2016). How to build models for government: Criteria driving model acceptance in policymaking. Policy Sciences, 49(4), 489–504. https://doi.org/10.1007/s11077-016-9250-4
   Web of Science ®Google Scholar
• Kotiadis, K., & Mingers, J. (2014). Combining PSMs with hard OR methods: The philosophical and practical challenges. In Sally Brailsford, Leonid Churilov, Brian Dangerfield (eds). Discrete‐Event Simulation and System Dynamics for Management Decision Making. (pp. 52–75). John Wiley & Sons, Ltd.
   Google Scholar
• Kotiadis, K., Tako, A. A., & Vasilakis, C. (2014). A participative and facilitative conceptual modelling framework for discrete event simulation studies in healthcare. Journal of the Operational Research Society, 65(2), 197–213. https://doi.org/10.1057/jors.2012.176
   Web of Science ®Google Scholar
• Kotiadis, K., & Tako, A. A. (2018). Facilitated post-model coding in discrete event simulation (DES): A case study in healthcare. European Journal of Operational Research, 266(3), 1120–1133. https://doi.org/10.1016/j.ejor.2017.10.047
   Web of Science ®Google Scholar
• Lance, C. B., Woehr, D. J., & Meade, A. W. (2007). Case study : A Monte Carlo investigation of assessment center construct validity models. Organizational Research Methods, 10(3), 430–448. https://doi.org/10.1177/1.094.428.106.289.395
   Web of Science ®Google Scholar
• Law, A. M., & McComas, M. G. (1991).Secrets of successful simulation studies. Proceedings of the 1991 Winter Simulation Conference, (pp.21–27).
   Google Scholar
• Lombaers, H. J. M. (1979). Operational research in practice. Engineering and Process Economics, 4(2-3), 245–248. doi:10.1016/0377-841X(79)90036-6.
   Google Scholar
• Mens, T., & Van Gorp, P. (2006). A taxonomy of model transformation. Electronic Notes in Theoretical Computer Science, 152(1–2), 125–142. https://doi.org/10.1016/j.entcs.2005.10.021
   Google Scholar
• Mingers, J. (2003). A classification of the philosophical assumptions of management sciencemethods. Journal of the Operational Research Society, 54(6), 559–570. https://doi.org/10.1057/palgrave.jors.2601436
   Web of Science ®Google Scholar
• Ormerod, J. (2018). The logic and methods of OR consulting practice: Towards a foundational view. Journal of the Operational Research Society, 69(9), 1357–1378. https://doi.org/10.1080/01605682.2017.1392407
   Web of Science ®Google Scholar
• Phillips, L. D. (1984). A theory of requisite decision models. Acta Psychologica, 56 (1-3), 29–48. https://doi.org/10.1016/0001-6918(84)90,005-2
   Web of Science ®Google Scholar
• Pidd, M. (2004). Systems modelling theory and practice. John Wiley & Sons.
   Google Scholar
• Pidd, M. (2009). Tools for thinking: Modelling in management science (3rd ed.). John Wiley and Sons Ltd.
   Google Scholar
• Prieto, D. M., Das, T. K., Savachkin, A. A., Uribe, A., Izurieta, R., & Malavade, S. (2012). A systematic review to identify areas of enhancements of pandemic simulation models for operational use at provincial and local levels. BMC Public Health, 12(1), 1–13. https://doi.org/10.1186/1471-2458-12-251
   PubMedGoogle Scholar
• Robinson, S., & Pidd, M. (1998). Provider and customer expectations of successful simulation projects. Journal of the Operational Research Society, 49(3), 200–209. https://doi.org/10.1057/palgrave.jors.2600516
   Web of Science ®Google Scholar
• Robinson, S. (1999). Three sources of simulation inaccuracy (and how to overcome them). In P. A. Farrington, H. B. Nembhard, D. T. Sturrock, and G. W. Evans (Eds), Proceedings of the 1999 Winter Simulation Conference. (pp.1701–1708).
   Google Scholar
• Robinson, S. (2002a). General concepts of quality for discrete-event simulation. European Journal of Operational Research, 138(1), 103–117. https://doi.org/10.1016/S0377-2217(01)00127-8
   Web of Science ®Google Scholar
• Robinson, S. (2002b). Modes of simulation practice: Approaches to business and military simulation. Simulation Modelling Practice and Theory, 10(8), 513–523. https://doi.org/10.1016/S1569-190X(02)00117-X
   Web of Science ®Google Scholar
• Robinson, S. (2014). Simulation - the practice of model development and use (2nd ed.). Macmillan education, Palgrave.
   Google Scholar
• Robinson, S., Worthington, C., Burgess, N., & Radnor, Z. J. (2014). Facilitated modelling with discrete-event simulation: Reality or myth? European Journal of Operational Research, 234(1), 231–240. https://doi.org/10.1016/j.ejor.2012.12.024
   Web of Science ®Google Scholar
• Robinson, S. (2015). A tutorial on conceptual modeling for simulation. In L. Yilmaz, W. K. V. Chan, I. Moon, T. M. K. Roeder, C. Macal, and M. D. Rossetti (Eds), Proceedings of the 2015 Winter Simulation Conference, (pp.1820–1834).
   Google Scholar
• Rouwette, E. A. J. A., Vennix, J. A. M., & Van Mullekom, T. (2002). Group model building effectiveness: A review of assessment studies. System Dynamics Review, 18(1), 5–45. https://doi.org/10.1002/sdr.229
   Web of Science ®Google Scholar
• Salt, J. D. (2008). The seven habits of highly defective simulation projects. Journal of Simulation, 2(3), 155–161. https://doi.org/10.1057/jos.2008.7
   Web of Science ®Google Scholar
• Sargent, R. G. (1991). Simulation model verification and validation. In B. L. Nelson, W. D. Kelton, & G. M. Clark, eds., Proceedings of the 23rd conference on Winter simulation, (pp.37–47).
   Google Scholar
• Tako, A. A., Kotiadis, K., & Vasilakis, C. (2010). A participative modelling framework for developing conceptual models in healthcare simulation studies. In Proceedings of the 2010 Winter Simulation Conference, (pp. 500–512).
   Google Scholar
• Tako, A., & Kotiadis, K. (2015). PartiSim: A multi-methodology framework to support facilitated simulation modelling in healthcare. European Journal of Operational Research, 244(2), 555–564. https://doi.org/10.1016/j.ejor.2015.01.046
   Web of Science ®Google Scholar
• Tako, A. A. (2015). Exploring the model development process in discrete-event simulation: Insights from six expert modellers. Journal of the Operational Research Society, 66(5), 747–760. https://doi.org/10.1057/jors.2014.52
   Web of Science ®Google Scholar
• Tako, A., Tsioptsias, N., & Robinson, S. (2020). Can we learning from simplified simulation models? An experimental study on user learning. Journal Of Simulation, 14(2), 130–144. https://doi.org/10.1080/17.477.778.2019.1704636
   Web of Science ®Google Scholar
• Tolk, A. (2015). Learning something right from models that are wrong: Epistemology of simulation. In L. InYilmaz (Ed.), Concepts and Methodologies for Modeling and Simulation. Simulation Foundations, Methods andApplications. Springer. (pp. 87–106). https://doi.org/10.1007/978-3-319-15096-3_5
   Google Scholar
• Tsioptsias, N., Tako, A., & Robinson, S. (2021). An exploratory study on the uses of “wrong” discrete event simulation models in practice. In Proceedings of the Operational Research Society Simulation Workshop 2021 (SW21), (pp.168–177).
   Google Scholar
• Vennix, J. A. (1999). Group model-building: Tackling messy problems. System Dynamics Review, 15(4), 379–401. https://doi.org/10.1002/(SICI)1099-1727(199.924)15:4<379::AID-SDR179>3.0.CO;2-E
   Web of Science ®Google Scholar
• Williams, H. P. (1989). How important are models to operational research? IMA Journal of Management Mathematics, 2(2), 189–195. https://doi.org/10.1093/imaman/2.2.189
   Google Scholar
• Yilmaz, L., & Liu, B. (2020). Model credibility revisited: Concepts and considerations for appropriate trust. Journal of Simulation. https://doi.org/10.1080/17477778.2020.1821587
   Web of Science ®Google Scholar