Variational Methods for Non-Linear Least-Squares

Abstract

We consider Newton-like line search descent methods for solving non-linear least-squares problems. The basis of our approach is to choose a method, or parameters within a method, by minimizing a variational measure which estimates the error in an inverse Hessian approximation. In one approach we consider sizing methods and choose sizing parameters in an optimal way. In another approach we consider various possibilities for hybrid Gauss-Newton/BFGS methods. We conclude that a simple Gauss-Newton/BFGS hybrid is both efficient and robust and we illustrate this by a range of comparative tests with other methods. These experiments include not only many well known test problems but also some new classes of large residual problem.

Keywords:

• non-linear least-squares
• Gauss-Newton method
• BFGS method
• variational method
• sizing
• hybrid method
• large residual problem

An early version of this work (with some omissions) was presented at the XI International Symposium on Mathematical Programming, Bonn, 1982, under the title ‘Optimally scaled methods for non-linear least-squares’.

An early version of this work (with some omissions) was presented at the XI International Symposium on Mathematical Programming, Bonn, 1982, under the title ‘Optimally scaled methods for non-linear least-squares’.