Coordinate search algorithms in multilevel optimization

Abstract

Many optimization problems of practical interest arise from the discretization of continuous problems. Classical examples can be found in calculus of variations, optimal control and image processing. In the recent years a number of strategies have been proposed for the solution of such problems, broadly known as multilevel methods. Inspired by classical multigrid schemes for linear systems, they exploit the possibility of solving the problem on coarser discretization levels to accelerate the computation of a finest-level solution. In this paper, we study the applicability of coordinate search algorithms in a multilevel optimization paradigm. We develop a multilevel derivative-free coordinate search method, where coarse-level objective functions are defined by suitable surrogate models. We employ a recursive v-cycle correction scheme, which exhibits multigrid-like error smoothing properties. On a practical level, the algorithm is implemented in tandem with a full-multilevel initialization. A suitable strategy to manage the coordinate search stepsize on different levels is also proposed, which gives a substantial contribution to the overall speed of the algorithm. Numerical experiments on several examples show promising results. The presented algorithm can solve large problems in a reasonable time, thus overcoming size and convergence speed limitations typical of coordinate search methods.

Keywords:

• multilevel optimization
• multigrid methods
• coordinate search
• derivative-free optimization
• discretized problems

AMS Subject Classifications:

• 90C360
• 90C56
• 65N55

Acknowledgements

We thank Philippe Toint for very fruitful discussions, and the two anonymous referees for their valuable comments and suggestions, which greatly helped to improve the overall quality of the first manuscript.

Notes

1. As discussed before, however, if we do not check for (5) and perform a full correction step, convergence is not guaranteed in general.

2. It must be noted that results in Table 2 are representative of the behaviour of FM/CS on all the considered test problems.