Duality in quasi-Newton methods and new variational characterizations of the DFP and BFGS updates

Abstract

It is known that quasi-Newton updates can be characterized by variational means, sometimes in more than one way. This paper has two main goals. We first formulate variational problems appearing in quasi-Newton methods within the vector space of symmetric matrices. This simplifies both their formulations and subsequent solutions. This part of the paper may be viewed as an efficient, modern survey of the variational problems occurring in quasi-Newton methods. We then construct, for the first time, duals of the variational problems for the DFP and BFGS updates and discover the remarkable fact that the solution to a dual problem is either the same as the corresponding primal solution or the solutions are inverses of each other. Consequently, we obtain six new variational characterizations for the DFP and BFGS updates, three for each one. Finally, we extend some of our results to an infinite dimensional setting.

Keywords:

• quasi-Newton
• DFP
• BFGS
• variational problems
• duality

AMS (MOS) Subject Classification :

• Primary: 90C53
• 90C30
• 90C46
• 65K10
• 49N15
• Secondary: 65K05
• 49M37
• 52A41

Acknowledgements

This research was partially supported by the National Science Foundation under grant DMS-0411955.