Nonlinear stepsize control, trust regions and regularizations for unconstrained optimization

Abstract

A class of algorithms for unconstrained optimization is introduced, which subsumes the classical trust-region algorithm and two of its newer variants, as well as the cubic and quadratic regularization methods. A unified theory of global convergence to first-order critical points is then described for this class.

Keywords:

• nonlinear optimization
• unconstrained problems
• global convergence

Acknowledgements

The author is indebted to B. Morini, D. Tomanos and M. Porcelli for their suggestions on this paper and to two anonymous referees whose comments on a first draft of this paper led to the present (tardy) revision.

Notes

The update described here is that proposed in 25. Fan 12 used a very similar update, which is closer to that of 15.

When considering Cauchy decrease only.