Multi-task Gaussian process upper confidence bound for hyperparameter tuning and its application for simulation studies of additive manufacturing

References

• Álvarez, M.A. and Lawrence, N.D. (2011) Computationally efficient convolved multiple output Gaussian processes. Journal of Machine Learning Research, 12 , 1459–1500.
   Web of Science ®Google Scholar
• Bengio, Y. and Grandvalet, Y. (2004) No unbiased estimator of the variance of k-fold cross-validation. Journal of Machine Learning Research , 5 , 1089–1105.
   Web of Science ®Google Scholar
• Bergstra, J. , Bardenet, R. , Bengio,Y. and Kégl, B. (2011) Algorithms for hyper-parameter optimization. Advances in Neural Information Processing Systems, 24 , 2546–2554.
   Google Scholar
• Brochu, E. , Cora, V.M. and De Freitas, N. (2010) A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599.
   Google Scholar
• Chaloner, K. and Verdinelli, I. (1995) Bayesian experimental design: A review. Statistical Science, 10 (3), 273–304.
   Web of Science ®Google Scholar
• Chiumenti, M. , Lin, X. , Cervera, M. , Lei, W. , Zheng, Y. and Huang, W. (2017) Numerical simulation and experimental calibration of additive manufacturing by blown powder technology. Part i: Thermal analysis. Rapid Prototyping Journal, 23 (2), 448–463.
   Web of Science ®Google Scholar
• Cover, T.M. and Thomas, J.A. (1991) Information theory and statistics. Elements of Information Theory , 1 , 279–335.
   Google Scholar
• Dai, S. , Song, J. and Yue, Y. (2020) Multi-task Bayesian optimization via Gaussian process upper confidence bound, in ICML 2020 Workshop on Real World Experiment Design and Active Learning , PMLR, Vienna, Austria, pp. 1–12.
   Google Scholar
• Flow Science Inc (2019) FLOW-3D, Version 12.0 . Flow Science, Santa Fe, NM.
   Google Scholar
• Frazier, P.I. and Wang, J. (2016) Bayesian optimization for materials design, in Information Science for Materials Discovery and Design , Springer, NY, USA, pp. 45–75.
   Google Scholar
• Garrido-Merchán, E.C. and Hernández-Lobato, D. (2020) Dealing with categorical and integer-valued variables in Bayesian optimization with Gaussian processes. Neurocomputing, 380 , 20–35.
   Web of Science ®Google Scholar
• Genton, M.G. (2001) Classes of kernels for machine learning: A statistics perspective. Journal of Machine Learning Research, 2 , 299–312.
   Web of Science ®Google Scholar
• Ghosal, S. and Roy, A. (2006) Posterior consistency of Gaussian process prior for nonparametric binary regression. The Annals of Statistics, 34 (5), 2413–2429.
   Web of Science ®Google Scholar
• Gokuldoss, P.K. , Kolla, S. and Eckert, J. (2017) Additive manufacturing processes: Selective laser melting, electron beam melting and binder jetting—selection guidelines. Materials, 10 (6), 672.
   PubMed Web of Science ®Google Scholar
• Guyon, I. , Gunn, S. , Ben-Hur, A. and Dror, G. (2005) Result analysis of the NIPS 2003 feature selection challenge. Advances in Neural Information Processing Systems, 17 , 545–552.
   Google Scholar
• Han, S. , Qubo, C. and Meng, H. (2012) Parameter selection in SVM with RBF kernel function , in World Automation Congress 2012 , IEEE Press, Piscataway, NJ, p. 1–4.
   Google Scholar
• Hastie, T. , Tibshirani, R. and Friedman, J. (2009) The Elements of Statistical Learning: Data Mining, Inference, and Prediction . Springer Science & Business Media, NY.
   Google Scholar
• Hutter, F. , Hoos, H.H. and Leyton-Brown, K. (2011) Sequential model-based optimization for general algorithm configuration, in International Conference on Learning and Intelligent Optimization , Springer, NY, USA, pp. 507–523.
   Google Scholar
• Janusevskis, J. and Le Riche,R. (2013) Simultaneous kriging-based estimation and optimization of mean response. Journal of Global Optimization, 55 (2), 313–336.
   Web of Science ®Google Scholar
• Jones, D.R. , Schonlau, M. and Welch, W.J. (1998) Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13 (4), 455–492.
   Web of Science ®Google Scholar
• Joseph, V.R. and Hung,Y. (2008) Orthogonal-maximin Latin hypercube designs. Statistica Sinica, 18 (1), 171–186.
   Web of Science ®Google Scholar
• Kandasamy, K. , Dasarathy, G. , Oliva, J. , Schneider, J. and Poczos, B. (2019) Multi-fidelity Gaussian process bandit optimisation. Journal of Artificial Intelligence Research, 66 , 151–196.
   Web of Science ®Google Scholar
• Kandasamy, K. , Dasarathy, G. , Schneider, J. and Póczos, B. (2017) Multi-fidelity Bayesian optimisation with continuous approximations, in 34th International Conference on Machine Learning, ICML 2017 , PMLR, Sydney, Australia, pp. 2861–2878.
   Google Scholar
• Kandasamy, K. , Schneider, J. and Póczos, B. (2015) High dimensional Bayesian optimisation and bandits via additive models, in International Conference on Machine Learning , PMLR, Lille, France, pp. 295–304.
   Google Scholar
• Krause, A. and Ong, C.S. (2011) Contextual Gaussian process bandit optimization. Advances in Neural Information Processing Systems, 24 , 2447–2455.
   Google Scholar
• Lawrence, S. , Giles, C.L. , Tsoi, A.C. and Back, A.D. (1997) Face recognition: A convolutional neural-network approach. IEEE Transactions on Neural Networks, 8 (1), 98–113.
   PubMed Web of Science ®Google Scholar
• LeCun, Y. , Bottou, L. , Bengio, Y. and Haffner, P. (1998) Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 (11), 2278–2324.
   Web of Science ®Google Scholar
• Maftouni, M. , Law, A.C.C. , Shen, B. , Kong, Z.J. , Zhou,Y. and Yazdi, N.A. (2021) A robust ensemble-deep learning model for Covid-19 diagnosis based on an integrated CT scan images database, in Proceeding of the IISE Annual Conference , ProQuest, Norcross, GA, pp. 632–637.
   Google Scholar
• Mahmoudi, M. , Tapia, G. , Karayagiz, K. , Franco, B. , Ma, J. , Arroyave, R. , Karaman, I. and Elwany, A. (2018) Multivariate calibration and experimental validation of a 3D finite element thermal model for laser powder bed fusion metal additive manufacturing. Integrating Materials and Manufacturing Innovation, 7 (3), 116–135.
   Web of Science ®Google Scholar
• Malkomes, G. and Garnett, R. (2018) Automating Bayesian optimization with Bayesian optimization. Advances in Neural Information Processing Systems, 31 , 5984–5994.
   Google Scholar
• Moss, H.B. , Leslie, D.S. and Rayson, P. (2020) Mumbo: Multi-task max-value Bayesian optimization, in Joint European Conference on Machine Learning and Knowledge Discovery in Databases , Springer, NY, USA, pp. 447–462.
   Google Scholar
• Motaman, S.A.H. , Kies, F. , Köhnen, P. , Létang, M. , Lin, M. , Molotnikov, A. and Haase, C. (2020) Optimal design for metal additive manufacturing: An integrated computational materials engineering (ICME) approach. JOM, 72 (3), 1092–1104.
   Web of Science ®Google Scholar
• Ridolfi, M.R. , Folgarait, P. , Battaglia, V. , Vela, T. , Corapi, D. and Di Schino, A. (2019) Development and calibration of a CFD-based model of the bed fusion SLM additive manufacturing process aimed at optimising laser parameters. Procedia Structural Integrity, 24 , 370–380.
   Google Scholar
• Shahriari, B. , Swersky, K. , Wang, Z. , Adams, R.P. and De Freitas, N. (2015) Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE, 104 (1), 148–175.
   Web of Science ®Google Scholar
• Shen, B. , Wang, R. , Law, A.C.C. , Kamath, R. , Choo, H. and Kong, Z. (2021) Super resolution for multi-sources image stream data using smooth and sparse tensor completion and its applications in data acquisition of additive manufacturing. Technometrics, 64 (1), 2–17.
   Web of Science ®Google Scholar
• Shen, B. , Xie, W. and Kong, Z.J. (2020) Clustered discriminant regression for high-dimensional data feature extraction and its applications in healthcare and additive manufacturing. IEEE Transactions on Automation Science and Engineering, 18 (4), 1998–2010.
   Web of Science ®Google Scholar
• Snoek, J. , Larochelle, H. and Adams, R.P. (2012) Practical Bayesian optimization of machine learning algorithms. Advances in Neural Information Processing Systems, 25 , 2951–2959.
   Google Scholar
• Srinivas, N. , Krause, A. , Kakade, S.M. and Seeger, M.W. (2012) Information-theoretic regret bounds for Gaussian process optimization in the bandit setting. IEEE Transactions on Information Theory, 58 (5), 3250–3265.
   Web of Science ®Google Scholar
• Surjanovic, S. and Bingham, D. (2020) Virtual library of simulation experiments: Test functions and datasets. Retrieved December 22, 2020, from http://www.sfu.ca/ssurjano.
   Google Scholar
• Sutskever, I. , Martens, J., Dahl, G. and Hinton, G. (2013) On the importance of initialization and momentum in deep learning, in International Conference on Machine Learning, PMLR, Atlanta, GA, pp. 1139–1147.
   Google Scholar
• Swersky, K. , Snoek, J. and Adams, R.P. (2013) Multi-task Bayesian optimization. Advances in Neural Information Processing Systems, 26 , 2004–2012.
   Google Scholar
• Tan, M.H. (2020) Bayesian optimization of expected quadratic loss for multiresponse computer experiments with internal noise. SIAM/ASA Journal on Uncertainty Quantification, 8 (3), 891–925.
   Web of Science ®Google Scholar
• Toscano-Palmerin, S. and Frazier, P.I. (2018) Bayesian optimization with expensive integrands. arXiv preprint arXiv:1803.08661.
   Google Scholar
• Wahba, G. (1990) Spline Models for Observational Data . SIAM Philadelphia, PA.
   Google Scholar
• Wang, Z. , Jiang, C. , Liu, P. , Yang, W. , Zhao, Y. , Horstemeyer, M.F. , Chen, L.-Q., Hu, Z. and Chen, L. (2020) Uncertainty quantification and reduction in metal additive manufacturing. NPJ Computational Materials, 6 (1), 1–10.
   Web of Science ®Google Scholar
• Williams, B.J. , Santner, T.J. and Notz, W.I. (2000) Sequential design of computer experiments to minimize integrated response functions. Statistica Sinica, 10 (4), 1133–1152.
   Web of Science ®Google Scholar
• Williams, C. , Bonilla, E.V. and Chai, K.M. (2007) Multi-task Gaussian process prediction. Advances in Neural Information Processing Systems, 20 , 153–160.
   Google Scholar
• Williams, C.K. and Rasmussen, C.E. (2006) Gaussian Processes for Machine Learning . MIT Press Cambridge, MA.
   Google Scholar
• Wilson, J. , Hutter, F. and Deisenroth, M. (2018) Maximizing acquisition functions for Bayesian optimization. Advances in Neural Information Processing Systems, 31 , 9884–9895.
   Google Scholar
• Yan, X. , Xia, Y. , Blum, H.B. and Gernay, T. (2020) Elevated temperature material properties of advanced high strength steel alloys. Journal of Constructional Steel Research, 174 , 106299.
   Web of Science ®Google Scholar
• Ye, J. , Mahmoudi, M. , Karayagiz, K. , Johnson, L. , Seede, R. , Karaman, I. , Arroyave, R. and Elwany, A. (2021) Bayesian calibration of multiple coupled simulation models for metal additive manufacturing: A Bayesian network approach. ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems, Part B: Mechanical Engineering, 8 (1), 011111.
   Web of Science ®Google Scholar
• Zhao, C. , Fezzaa, K. , Cunningham, R.W. , Wen, H. , De Carlo, F. , Chen, L. , Rollett, A.D. and Sun, T. (2017) Real-time monitoring of laser powder bed fusion process using high-speed X-ray imaging and diffraction. Scientific Reports, 7 (1), 1–11.
   PubMed Web of Science ®Google Scholar