Purposeful cross-validation: a novel cross-validation strategy for improved surrogate optimizability

ABSTRACT

Parameter selection during the construction of surrogates is often conducted by minimizing the Mean Squared Cross-Validation Error (MSE-CV). Surrogates constructed using MSE are poorly optimized using gradient-based optimizers. Hence, Nelder–Mead like optimizers are often favoured, which is unfortunate as surrogates make analytical gradients freely available and gradient-based optimizers scale better with increasing dimension. To address this shortcoming, this article proposes a new Cross-Validation (CV) approach, by optimizing the surrogate and computing the Mean Optimizer Distance (MOD-CV) to the best design in the surrogate. Four experimental CV measures are compared on seven test problems and it is demonstrated that the performance of gradient-based optimizers can be significantly enhanced, with a possible 97% improvement in MOD-CV over MSE-CV using Sequential Least-Squares Quadratic Programming (SLSQP). Additionally, surrogates constructed using MOD-CV outperform surrogates constructed with MSE-CV, 80% of the time when optimized with SLSQP and 68% when optimized with Nelder-Mead.

Keywords:

• Surrogate-based optimization
• cross validation
• radial basis function
• design optimization

Disclosure statement

No potential conflict of interest was reported by the authors.