Queueing inspired feature engineering to improve and simplify patient flow simulation metamodels

References

• Altiok, T., & Melamed, B. (2010). Simulation modeling and analysis with arena. Elsevier. https://www.sciencedirect.com/book/9780123705235/simulation-modeling-and-analysis-with-arena
   Google Scholar
• Amorim, M., Antunes, F., Ferreira, S., & Couto, A. (2019). An integrated approach for strategic and tactical decisions for the emergency medical service: Exploring optimization and metamodel-based simulation for vehicle location. Computers & Industrial Engineering, 137, 106057. https://doi.org/10.1016/j.cie.2019.106057
   Web of Science ®Google Scholar
• Ankenman, B., Nelson, B. L., & Staum, J. (2010). Stochastic kriging for simulation metamodeling. Operations Research, 58(2), 371–382. https://doi.org/10.1287/opre.1090.0754
   Web of Science ®Google Scholar
• Banerjee, S., Chattopadhyay, T., Pal, A., & Garain, U. (2018). Automation of feature engineering for iot analytics. ACM Sigbed Review, 15(2), 24–30. https://doi.org/10.1145/3231535.3231538
   Google Scholar
• Barton, R. R. (2009). Simulation optimization using metamodels. In Proceedings of the 2009 winter simulation conference (wsc) (pp. 230–238). https://doi.org/10.1109/WSC.2009.5429328
   Google Scholar
• Bitran, G. R., & Dasu, S. (1992). A review of open queueing network models of manufacturing systems. Queueing Systems, 12(1–2), 95–133. https://doi.org/10.1007/BF01158637
   Google Scholar
• Bojer, C. S., & Meldgaard, J. P. (2021). Kaggle forecasting competitions: An overlooked learning opportunity. International Journal of Forecasting, 37(2), 587–603. https://doi.org/10.1016/j.ijforecast.2020.07.007
   Web of Science ®Google Scholar
• Boots, N. K., & Tijms, H. (1999). A multiserver queueing system with impatient customers. Management Science, 45(3), 444–448. https://doi.org/10.1287/mnsc.45.3.444
   Web of Science ®Google Scholar
• Brailsford, S. C., Eldabi, T., Kunc, M., Mustafee, N., & Osorio, A. F. (2019). Hybrid simulation modelling in operational research: A state-of-the-art review. European Journal of Operational Research, 278(3), 721–737. https://doi.org/10.1016/j.ejor.2018.10.025
   Web of Science ®Google Scholar
• Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324
   Web of Science ®Google Scholar
• Butcher, B., & Smith, B. J. (2020). Feature engineering and selection: A practical approach for predictive models. The American Statistician, 74(3), 308–309. https://doi.org/10.1080/00031305.2020.1790217
   Web of Science ®Google Scholar
• Carmen, R., Defraeye, M., & Van Nieuwenhuyse, I. (2015). A decision support system for capacity planning in emergency departments. International Journal of Simulation Modelling, 14(2), 299–312. https://doi.org/10.2507/IJSIMM14(2)10.308
   Web of Science ®Google Scholar
• Chandy, K. M., & Sauer, C. H. (1978). Approximate methods for analyzing queueing network models of computing systems. ACM Computing Surveys (CSUR), 10(3), 281–317. https://doi.org/10.1145/356733.356737
   Google Scholar
• Chen, J., Song, L., Wainwright, M., & Jordan, M. (2018). Learning to explain: An information-theoretic perspective on model interpretation. In International conference on machine learning (pp. 883–892). http://proceedings.mlr.press/v80/chen18j/chen18j.pdf
   Google Scholar
• Coelho, G. F., & Pinto, L. R. (2018). Kriging-based simulation optimization: An emergency medical system application. Journal of the Operational Research Society, 69(12), 2006–2020. https://doi.org/10.1080/01605682.2017.1418149
   Web of Science ®Google Scholar
• Covert, I., Lundberg, S. M., & Lee, S. -I. (2021). Explaining by removing: A unified framework for model explanation. Journal of Machine Learning Research: JMLR, 22, 209–1. https://www.jmlr.org/papers/volume22/20-1316/20-1316.pdf
   Web of Science ®Google Scholar
• Dhurandhar, A., Chen, P. -Y., Luss, R., Tu, C. -C., Ting, P., Shanmugam, K., & Das, P. (2018). Explanations based on the missing: Towards contrastive explanations with pertinent negatives. Advances in Neural Information Processing Systems, 31. https://proceedings.neurips.cc/paper/2018/file/c5ff2543b53f4cc0ad3819a36752467b-Paper.pdf
   Google Scholar
• Do Amaral, J. V. S., Montevechi, J. A. B., de Carvalho Miranda, R., & de Sousa Junior, W. T. (2022). Metamodel-based simulation optimization: A systematic literature review. Simulation Modelling Practice and Theory, 114, 102403. https://doi.org/10.1016/j.simpat.2021.102403
   Web of Science ®Google Scholar
• Došilović, F. K., Brčić, M., & Hlupić, N. (2018). Explainable artificial intelligence: A survey. In 2018 41st international convention on information and communication technology, electronics and microelectronics (mipro) (pp. 0210–0215). https://doi.org/10.23919/MIPRO.2018.8400040
   Google Scholar
• dos Santos, V. H., Kotiadis, K., & Scaparra, M. P. (2020). A review of hybrid simulation in healthcare. In 2020 winter simulation conference (wsc) (pp. 1004–1015). https://doi.org/10.1109/WSC48552.2020.9383913
   Google Scholar
• Fan, C., Sun, Y., Zhao, Y., Song, M., & Wang, J. (2019). Deep learning-based feature engineering methods for improved building energy prediction. Applied Energy, 240, 35–45. https://doi.org/10.1016/j.apenergy.2019.02.052
   Web of Science ®Google Scholar
• Fatma, N., Mohd, S., Ramamohan, V., & Mustafee, N. (2020). Primary healthcare delivery network simulation using stochastic metamodels. In 2020 winter simulation conference (wsc) (pp. 818–829). https://doi.org/10.1109/WSC48552.2020.9384069
   Google Scholar
• Ferraro, N. M., Reamer, C. B., Reynolds, T. A., Howell, L. J., Moldenhauer, J. S., & Day, T. E. (2015). Capacity planning for maternal–fetal medicine using discrete event simulation. American Journal of Perinatology, 32(08), 761–770. https://doi.org/10.1055/s-0034-1396074
   Google Scholar
• Friedman, L. W. (2012). The simulation metamodel. Springer Science & Business Media. https://doi.org/10.1007/978-1-4613-1299-4
   Google Scholar
• Green, L. V., & Liu, N. (2015). A study of new york city obstetrics units demonstrates the potential for reducing hospital inpatient capacity. Medical Care Research and Review, 72(2), 168–186. https://doi.org/10.1177/1077558715572388
   Web of Science ®Google Scholar
• Griffin, J., Xia, S., Peng, S., & Keskinocak, P. (2012). Improving patient flow in an obstetric unit. Health Care Management Science, 15(1), 1–14. https://doi.org/10.1007/s10729-011-9175-6
   PubMed Web of Science ®Google Scholar
• Hamza, N., Majid, M. A., Adam, K., & Bakar, N. A. A. (2019). A review on simulation and modelling for patient flow in emergency department. In Iop conference series: Materials science and engineering (Vol. 551, pp. 012037). https://doi.org/10.1088/1757-899X/551/1/012037
   Google Scholar
• Jansson, B. (1966). Choosing a good appointment system—a study of queues of the type (d, m, 1). Operations Research, 14(2), 292–312. https://doi.org/10.1287/opre.14.2.292
   Web of Science ®Google Scholar
• Keys, A. C., Rees, L. P., & Greenwood, A. G. (2002). Performance measures for selection of metamodels to be used in simulation optimization. Decision Sciences, 33(1), 31–58. https://doi.org/10.1111/j.1540-5915.2002.tb01635.x
   Web of Science ®Google Scholar
• Khurana, U., Samulowitz, H., & Turaga, D. (2018). Feature engineering for predictive modeling using reinforcement learning. In Proceedings of the aaai conference on artificial intelligence (Vol. 32). https://doi.org/10.1609/aaai.v32i1.11678
   Google Scholar
• Khurana, U., Turaga, D., Samulowitz, H., & Parthasrathy, S. (2016). Cognito: Automated feature engineering for supervised learning. In 2016 ieee 16th international conference on data mining workshops (icdmw) (pp. 1304–1307). https://doi.org/10.1109/ICDMW.2016.0190
   Google Scholar
• Kilmer, R. A., Smith, A. E., & Shuman, L. J. (1997). An emergency department simulation and a neural network metamodel. Journal of the Society for Health Systems, 5(3), 63–79. https://www.eng.auburn.edu/~smithae/files/bobhosp.pdf
   PubMedGoogle Scholar
• Kimura, T. (1994). Approximations for multi-server queues: System interpolations. Queueing Systems, 17(3), 347–382. https://doi.org/10.1007/BF01158699
   Web of Science ®Google Scholar
• Kleijnen, J. P. (2008). Response surface methodology for constrained simulation optimization: An overview. Simulation Modelling Practice and Theory, 16(1), 50–64. https://doi.org/10.1016/j.simpat.2007.10.001
   Web of Science ®Google Scholar
• Kleijnen, J., & van Beers, W. (2005). Robustness of kriging when interpolating in random simulation with heterogeneous variances: Some experiments. European Journal of Operational Research, 165(3), 826–834. https://doi.org/10.1016/j.ejor.2003.09.037
   Web of Science ®Google Scholar
• Kuhn, M., & Wickham, H. (2020). Tidymodels: A collection of packages for modeling and machine learning using tidyverse principles. [Computer software manual]. https://www.tidymodels.org
   Google Scholar
• Lam, H. T., Thiebaut, J. -M., Sinn, M., Chen, B., Mai, T., & Alkan, O. (2017). One button machine for automating feature engineering in relational databases. arXiv preprint arXiv:1706.00327. https://arxiv.org/pdf/1706.00327.pdf
   Google Scholar
• Lam, H., & Zhang, H. (2021). Neural predictive intervals for simulation metamodeling. In 2021 winter simulation conference (wsc) (pp. 1–12). https://doi.org/10.1109/WSC52266.2021.9715494
   Google Scholar
• Li, T., Kou, G., & Peng, Y. (2020). Improving malicious urls detection via feature engineering: Linear and nonlinear space transformation methods. Information Systems, 91, 101494. https://doi.org/10.1016/j.is.2020.101494
   Web of Science ®Google Scholar
• Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2021). Explainable ai: A review of machine learning interpretability methods. Entropy, 23(1), 18. https://doi.org/10.3390/e23010018
   Web of Science ®Google Scholar
• Li, Y. F., Ng, S. H., Xie, M., & Goh, T. (2010). A systematic comparison of metamodeling techniques for simulation optimization in decision support systems. Applied Soft Computing, 10(4), 1257–1273. https://doi.org/10.1016/j.asoc.2009.11.034
   Web of Science ®Google Scholar
• Lucas, Y., Portier, P. -E., Laporte, L., He-Guelton, L., Caelen, O., Granitzer, M., & Calabretto, S. (2020). Towards automated feature engineering for credit card fraud detection using multi-perspective hmms. Future Generation Computer Systems, 102, 393–402. https://doi.org/10.1016/j.future.2019.08.029
   Web of Science ®Google Scholar
• Lundberg, S. M., & Lee, S. -I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30. https://proceedings.neurips.cc/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf?xgtab&
   Google Scholar
• Moreira, C., Chou, Y. -L., Velmurugan, M., Ouyang, C., Sindhgatta, R., & Bruza, P. (2021). Linda-bn: An interpretable probabilistic approach for demystifying black-box predictive models. Decision Support Systems, 150, 113561. https://doi.org/10.1016/j.dss.2021.113561
   Web of Science ®Google Scholar
• Nargesian, F., Samulowitz, H., Khurana, U., Khalil, E. B., & Turaga, D. (2017). Learning feature engineering for classification. In Proceedings of the 26th international joint conference on artificial intelligence (pp. 2529–2535). https://dl.acm.org/doi/abs/10.5555/3172077.3172240
   Google Scholar
• Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, E. (2011). Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12, 2825–2830. https://www.jmlr.org/papers/volume12/pedregosa11a/pedregosa11a.pdf?ref=https:/
   Web of Science ®Google Scholar
• Petering, M. E., Aydas, O. T., Kuzu, K., & Ross, A. (2015). Simulation analysis of hospital intensive care unit reimbursement policies from the triple bottom line perspective. Journal of Simulation, 9(2), 86–98. https://doi.org/10.1057/jos.2014.24
   Web of Science ®Google Scholar
• Petsiuk, V., Das, A., & Saenko, K. (2018). Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421. https://arxiv.org/pdf/1806.07421.pdf%20/
   Google Scholar
• Powell, J. H., & Mustafee, N. (2017). Widening requirements capture with soft methods: An investigation of hybrid m&s studies in health care. Journal of the Operational Research Society, 68(10), 1211–1222. https://doi.org/10.1057/s41274-016-0147-6
   Web of Science ®Google Scholar
• Qiu, Y., Song, J., & Liu, Z. (2016). A simulation optimisation on the hierarchical health care delivery system patient flow based on multi-fidelity models. International Journal of Production Research, 54(21), 6478–6493. https://doi.org/10.1080/00207543.2016.1197437
   Web of Science ®Google Scholar
• Rabe, M., Deininger, M., & Juan, A. A. (2020). Speeding up computational times in simheuristics combining genetic algorithms with discrete-event simulation. Simulation Modelling Practice and Theory, 103, 102089. https://doi.org/10.1016/j.simpat.2020.102089
   Web of Science ®Google Scholar
• Rasheed, F., & Lee, Y. H. (2014). Enhancing performance of healthcare facility via novel simulation metamodeling based decision support framework. International Journal of Industrial Engineering, 21, 5. https://doi.org/10.23055/ijietap.2014.21.5.946
   Google Scholar
• R Core Team. (2022). R: A language and environment for statistical computing [Computer software manual]. https://www.R-project.org/
   Google Scholar
• Rudin, C., & Radin, J. (2019). Why are we using black box models in ai when we don’t need to? A lesson from an explainable ai competition. Harvard Data Science Review, 1(2), 10–1162. https://doi.org/10.1162/99608f92.5a8a3a3d
   Google Scholar
• Sanchez, S. M. (2000). Robust design: Seeking the best of all possible worlds. In 2000 winter simulation conference proceedings (cat. no. 00ch37165) (Vol. 1, pp. 69–76). https://doi.org/10.1109/WSC.2000.899700
   Google Scholar
• Santos, P. R., & Santos, I. R. (2010). Reinsch’s smoothing spline simulation metamodels. In Proceedings of the 2010 winter simulation conference (pp. 925–934). https://doi.org/10.1109/WSC.2010.5679097
   Google Scholar
• Semenova, L., Rudin, C., & Parr, R. (2019). A study in Rashomon curves and volumes: A new perspective on generalization and model simplicity in machine learning. arXiv preprint arXiv:1908.01755. https://arxiv.org/pdf/1908.01755.pdf
   Google Scholar
• Shortle, J. F., Thompson, J. M., Gross, D., & Harris, C. M. (2018). Fundamentals of queueing theory (Vol. 399). John Wiley & Sons. https://doi.org/10.1002/9781119453765
   Google Scholar
• Shrikumar, A., Greenside, P., & Kundaje, A. (2017). Learning important features through propagating activation differences. In International conference on machine learning (pp. 3145–3153). http://proceedings.mlr.press/v70/shrikumar17a
   Google Scholar
• Snyder, M., & Smucker, B. J. (2022). Metamodel optimization of a complex, rural–urban emergency medical services system. Simulation Modelling Practice and Theory, 118, 102526. https://doi.org/10.1016/j.simpat.2022.102526
   Web of Science ®Google Scholar
• Stiglic, G., Kocbek, P., Fijacko, N., Zitnik, M., Verbert, K., & Cilar, L. (2020). Interpretability of machine learning-based prediction models in healthcare. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 10(5), e1379. https://doi.org/10.1002/widm.1379
   Web of Science ®Google Scholar
• Sundararajan, M., Taly, A., & Yan, Q. (2017). Axiomatic attribution for deep networks. In International conference on machine learning (pp. 3319–3328). http://proceedings.mlr.press/v70/sundararajan17a/sundararajan17a.pdf
   Google Scholar
• Team simpy. (2022). https://simpy.readthedocs.io/en/latest/contents.html
   Google Scholar
• Vellido, A. (2020). The importance of interpretability and visualization in machine learning for applications in medicine and health care. Neural Computing & Applications, 32(24), 18069–18083. https://doi.org/10.1007/s00521-019-04051-w
   Web of Science ®Google Scholar
• Wang, X., Gong, X., Geng, N., Jiang, Z., & Zhou, L. (2020). Metamodel-based simulation optimisation for bed allocation. International Journal of Production Research, 58(20), 6315–6335. https://doi.org/10.1080/00207543.2019.1677962
   Web of Science ®Google Scholar
• Wang, H. E., Landers, M., Adams, R., Subbaswamy, A., Kharrazi, H., Gaskin, D. J., & Saria, S. (2022). A bias evaluation checklist for predictive models and its pilot application for 30-day hospital readmission models. Journal of the American Medical Informatics Association, 29(8), 1323–1333. https://doi.org/10.1093/jamia/ocac065
   Web of Science ®Google Scholar
• Wei, D., Dash, S., Gao, T., & Gunluk, O. (2019). Generalized linear rule models. In International conference on machine learning (pp. 6687–6696). https://proceedings.mlr.press/v97/wei19a.html
   Google Scholar
• Whitt, W. (1993). Approximations for the gi/g/m queue. Production and Operations Management, 2(2), 114–161. https://doi.org/10.1111/j.1937-5956.1993.tb00094.x
   Google Scholar
• Whitt, W. (1999). Predicting queueing delays. Management Science, 45(6), 870–888. https://doi.org/10.1287/mnsc.45.6.870
   Web of Science ®Google Scholar
• Whitt, W. (2007). What you should know about queueing models to set staffing requirements in service systems. Naval Research Logistics (NRL), 54(5), 476–484. https://doi.org/10.1002/nav.20243
   Web of Science ®Google Scholar
• Wu, M., Hughes, M., Parbhoo, S., Zazzi, M., Roth, V., & Doshi-Velez, F. (2018). Beyond sparsity: Tree regularization of deep models for interpretability. In Proceedings of the aaai conference on artificial intelligence (Vol. 32). https://doi.org/10.1609/aaai.v32i1.11501
   Google Scholar
• Yildiz, G., & Eski, O. (2006). An artificial neural network based simulation metamodeling approach for dual resource constrained assembly line. In International conference on artificial neural networks (pp. 1002–1011). https://doi.org/10.1007/11840930104
   Google Scholar
• Yin, J., & Ng, K. (2011). Kriging metamodel with modified nugget-effect: The heteroscedastic variance case. Computers & Industrial Engineering, 61(3), 760–777. https://doi.org/10.1016/j.cie.2011.05.008
   Web of Science ®Google Scholar
• Yousefi, M., & Yousefi, M. (2019). Human resource allocation in an emergency department: A metamodel-based simulation optimization. Kybernetes, 49(3), 779–796. https://doi.org/10.1108/K-12-2018-0675
   Web of Science ®Google Scholar
• Yousefi, M., Yousefi, M., Ferreira, R. P. M., Kim, J. H., & Fogliatto, F. S. (2018). Chaotic genetic algorithm and adaboost ensemble metamodeling approach for optimum resource planning in emergency departments. Artificial Intelligence in Medicine, 84, 23–33. https://doi.org/10.1016/j.artmed.2017.10.002
   PubMed Web of Science ®Google Scholar
• Zeinali, F., Mahootchi, M., & Sepehri, M. M. (2015). Resource planning in the emergency departments: A simulation-based metamodeling approach. Simulation Modelling Practice and Theory, 53, 123–138. https://doi.org/10.1016/j.simpat.2015.02.002
   Web of Science ®Google Scholar
• Zhu, Z., Hen, B. H., & Teow, K. L. (2012). Estimating ICU bed capacity using discrete event simulation. International Journal of Health Care Quality Assurance, 25(2), 134–144. https://doi.org/10.1108/09526861211198290
   PubMedGoogle Scholar