Improving particle swarm optimization performance with local search for high-dimensional function optimization

Abstract

Particle swarm optimization (PSO) is one recently proposed population-based stochastic optimization technique, and gradient-based descent methods are efficient local optimization techniques that are often used as a necessary ingredient of hybrid algorithms for global optimization problems (GOPs). By examining the properties of the two methods, a two-stage hybrid algorithm for global optimization is proposed. In the present algorithm, the gradient descent technique is used to find a local minimum of the objective function efficiently, and a PSO method with latent parallel search capability is employed to help the algorithm to escape from the previously converged local minima to a better point which is then used as a starting point for the gradient methods to restart a new local search. The above search procedure is applied repeatedly until a global minimum is found (when a global minimum is known in advance) or the maximum number of function evaluations is reached. In addition, a repulsion technique and partially initializing population method are incorporated in the new algorithm to increase its global jumping ability. Simulation results on 15 test problems including five large-scale ones with dimensions up to 1000 demonstrate that the proposed method is more stable and efficient than several other existing methods.

Keywords:

• global optimization
• gradient descent method
• particle swarm optimization (PSO)
• repulsion technique

AMS Subject Classification :

• 90C30
• 90C52
• 90C26
• 65K05
• 49M37

Acknowledgements

The author thanks the anonymous reviewers and the editor for their constructive remarks and valuable comments that greatly improved the paper quality.