Multi-task Gaussian process upper confidence bound for hyperparameter tuning and its application for simulation studies of additive manufacturing

Abstract

In many scientific and engineering applications, Bayesian Optimization (BO) is a powerful tool for hyperparameter tuning of a machine learning model, materials design and discovery, etc. Multi-task BO is a general method to efficiently optimize multiple different, but correlated, “black-box” functions. The objective of this work is to develop an algorithm for multi-task BO with automatic task selection so that only one task evaluation is needed per query round. Specifically, a new algorithm, namely, Multi-Task Gaussian Process Upper Confidence Bound (MT-GPUCB), is proposed to achieve this objective. The MT-GPUCB is a two-step algorithm, where the first step chooses which query point to evaluate, and the second step automatically selects the most informative task to evaluate. Under the bandit setting, a theoretical analysis is provided to show that our proposed MT-GPUCB is no-regret under some mild conditions. Our proposed algorithm is verified experimentally on a range of synthetic functions. In addition, our algorithm is applied to Additive Manufacturing simulation software, namely, Flow-3D Weld, to determine material property values, ensuring the quality of simulation output. The results clearly show the advantages of our query strategy for both design point and task.

KEYWORDS:

• Multi-task Gaussian process upper confidence bound
• automatic task selection
• no-regret
• hyperparameter tuning

Additional information

Notes on contributors

Bo Shen

Bo Shen (M’18) received a BS degree in statistics from the University of Science and Technology of China, Hefei, China, in 2017. He is currently pursuing a PhD degree in industrial and systems engineering at Virginia Tech, Blacksburg, VA, USA. His research interests include optimization in machine learning and data analytics in smart manufacturing.

Raghav Gnanasambandam

Raghav Gnanasambandam (M’21) received his combined B.Tech and M.Tech degree in mechanical engineering from the Indian Institute of Technology Madras, Chennai, India, in 2019. He is currently pursuing PhD degree in industrial and systems engineering at Virginia Tech, Blacksburg, VA, USA. His research interests include machine learning and optimization for advanced manufacturing.

Rongxuan Wang

Rongxuan Wang is a PhD student at the Grado Department of Industrial and Systems Engineering at the Virginia Polytechnic Institute and State University in Blacksburg, VA, specializing in sensing techniques such as three-dimensional scanning, digital image correlation, radiology, and thermography. His research focuses on process monitoring and quality control in smart manufacturing.

Zhenyu James Kong

Zhenyu (James) Kong (M’02) received his BS and MS degrees in mechanical engineering from the Harbin Institute of Technology, China, in 1993 and 1995, respectively, and his PhD degree from the Department of Industrial and System Engineering, University of Wisconsin–Madison, Madison, WI, USA, in 2004. He is currently a professor with the Grado Department of Industrial and Systems Engineering, Virginia Tech, Blacksburg, VA, USA. His research interests include sensing and analytics for smart manufacturing, and modeling, synthesis, and diagnosis for large and complex manufacturing systems. He is a fellow of IISE and ASME.