Adaptive confidence bound based Bayesian optimization via potentially optimal Lipschitz conditions

Abstract

The confidence bound (CB) is one of the most popular acquisition functions for Bayesian optimization (BO). It realizes the balance between local exploitation and global exploration through an explicit trade-off coefficient. In practice, researchers tend to employ fixed or random trade-offs for CB, which however is inflexible in tackling challenging optimization scenarios. Therefore, this article presents an adaptive CB acquisition function, called ACB, to address the issue. Specifically, this article uses Lipschitz conditions to identify a set of potentially optimal trade-off coefficients dynamically by the estimated prediction mean and the leave-one-out cross-validation variance. Thereby, the proposed ACB achieves adaptive trade-off between exploitation and exploration by cycling through the set of dynamically updated coefficients. This article verifies the superiority of the proposed ACB by comparing it against current CB-based BOs using different trade-off strategies on nine numerical examples and the design optimization of a supercritical carbon dioxide centrifugal compressor.

Keywords:

• Bayesian optimization
• Lipschitz conditions
• exploitation and exploration
• adaptive confidence bound

Disclosure statement

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this article.

Data availability statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Notes

1 Using all the identified trade-off coefficients in a single iteration raises the batch Bayesian optimization paradigm (Joy et al. 2020), which is left for further exploration.

2 The proposed ACB algorithm is showcased by solving the minimization problem using the acquisition function ALCB.

3 The absolute error is used for test functions with a global minimum of zero.

4 The time assumption of a single simulation of the S-CO${}_2$ centrifugal compressor is around 35 seconds on a personal computer.

5 Considering the dynamic balance in ACB, the specifically related constraint handling strategy needs further investigation, which is left for the authors' future work.

Additional information

Funding

This work was supported by the National Key Research and Development Program of China [2020YFA0714403]; the National Natural Science Foundation of China [52005074 and 52005073]; the Fundamental Research Funds for the Central Universities [DUT19RC(3)070]. Additionally, it was partially supported by the Research and Innovation in Science and Technology Major Project of Liaoning Province [2019JH1-10100024]; and the MIIT Marine Welfare Project [Z135060009002].