Adaptive augmented Lagrangian methods: algorithms and practical numerical experience

References

• AMPL Home Page. Available at http://www.ampl.com.
   Google Scholar
• R. Andreani, E.G. Birgin, J.M. Martínez, and M.L. Schuverdt, Augmented Lagrangian methods under the constant positive linear dependence constraint qualification, Math. Program. 111 (2008), pp. 5–32. Available at http://dx.doi.org/10.1007/s10107-006-0077-1. doi: 10.1007/s10107-006-0077-1
   Web of Science ®Google Scholar
• E.G. Birgin and J.M. Martínez, Improving ultimate convergence of an augmented Lagrangian method, Optim. Methods Softw. 23 (2008), pp. 177–195. doi: 10.1080/10556780701577730
   Web of Science ®Google Scholar
• E.G. Birgin and J.M. Martínez, Augmented Lagrangian method with nonmonotone penalty parameters for constrained optimization, Comput. Optim. Appl. 51 (2012), pp. 941–965. Available at http://dx.doi.org/10.1007/s10589-011-9396-0. doi: 10.1007/s10589-011-9396-0
   Web of Science ®Google Scholar
• E.G. Birgin and J.M. Martínez, Practical Augmented Lagrangian Methods for Constrained Optimization, Fundamentals of Algorithms, SIAM, Philadelphia, PA, 2014.
   Google Scholar
• A. Bondarenko, D. Bortz, and J.J. Moré, COPS: Large-scale nonlinearly constrained optimization problems, Technical Report ANL/MCS-TM-237, Mathematics and Computer Science division, Argonne National Laboratory, Argonne, IL, 1998, revised October 1999.
   Google Scholar
• S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Found. Trends Mach. Learn. 3 (2011), pp. 1–122. doi: 10.1561/2200000016
   Google Scholar
• R.H. Byrd, G. Lopez-Calva, and J. Nocedal, A line search exact penalty method using steering rules, Math. Program. 133 (2012), pp. 39–73. doi: 10.1007/s10107-010-0408-0
   Web of Science ®Google Scholar
• R.H. Byrd, J. Nocedal, and R.A. Waltz, Steering exact penalty methods for nonlinear programming, Optim. Methods Softw. 23 (2008), pp. 197–213. Available at http://dx.doi.org/10.1080/10556780701394169. doi: 10.1080/10556780701394169
   Web of Science ®Google Scholar
• A.R. Conn, N.I.M. Gould, and Ph.L. Toint, Global convergence of a class of trust region algorithms for optimization with simple bounds, SIAM J. Numer. Anal. 25 (1988), pp. 433–460. doi: 10.1137/0725029
   Web of Science ®Google Scholar
• A.R. Conn, N.I.M. Gould, and Ph.L. Toint, Testing a class of methods for solving minimization problems with simple bounds on the variables, Math. Comput. 50 (1988), pp. 399–430. doi: 10.1090/S0025-5718-1988-0929544-3
   Web of Science ®Google Scholar
• A.R. Conn, N.I.M. Gould, and Ph.L. Toint, A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds, SIAM J. Numer. Anal. 28 (1991), pp. 545–572. doi: 10.1137/0728030
   Web of Science ®Google Scholar
• A.R. Conn, N.I.M. Gould, and Ph.L. Toint, Lancelot: A Fortran package for large-scale nonlinear optimization (Release A), Lecture Notes in Computation Mathematics 17, Springer Verlag, Berlin, Heidelberg, New York, London, Paris and Tokyo, 1992.
   Google Scholar
• A.R. Conn, N.I.M. Gould, and Ph.L. Toint, Numerical experiments with the LANCELOT package (Release A) for large-scale nonlinear optimization, Math. Program. 73 (1996), pp. 73–110.
   Web of Science ®Google Scholar
• A.R. Conn, N.I.M. Gould, and Ph.L. Toint, Trust-Region Methods, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2000.
   Google Scholar
• F.E. Curtis, H. Jiang, and D.P. Robinson, An adaptive augmented Lagrangian method for large-scale constrained optimization, Math. Program. 152 (2015), pp. 201–245. doi: 10.1007/s10107-014-0784-y
   Web of Science ®Google Scholar
• F.E. Curtis, N.I.M. Gould, H. Jiang, and D.P. Robinson, Adaptive augmented Lagrangian methods: Algorithms and practical numerical experience. Available at http://xxx.tau.ac.il/abs/1408.4500, arXiv:1408.4500.
   Google Scholar
• K.R. Davidson and A.P. Donsig, Real Analysis and Applications, Undergraduate Texts in Mathematics, Springer, New York, 2010. Available at http://dx.doi.org/10.1007/978-0-387-98098-0.
   Google Scholar
• E.D. Dolan and J.J. Moré, Benchmarking optimization software with performance profiles, Math. Program. 91 (2002), pp. 201–213. doi: 10.1007/s101070100263
   Web of Science ®Google Scholar
• D. Fernández and M.V. Solodov, Local convergence of exact and inexact augmented Lagrangian methods under the second-order sufficiency condition, SIAM J. Optim. 22 (2012), pp. 384–407. doi: 10.1137/10081085X
   Web of Science ®Google Scholar
• R. Fourer, D.M. Gay, and B.W. Kernighan, AMPL: A Modeling Language for Mathematical Programming, Brooks/Cole—Thomson Learning, Pacific Grove, 2003.
   Google Scholar
• D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite element approximations, Comput. Math. Appl. 2 (1976), pp. 17–40. doi: 10.1016/0898-1221(76)90003-1
   Google Scholar
• R. Glowinski and A. Marroco, Sur l'Approximation, par Elements Finis d'Ordre Un, el la Resolution, par Penalisation-Dualité, d'une Classe de Problèmes de Dirichlet Nonlineares, Revue Française d'Automatique, Informatique et Recherche Opérationelle 9 (1975), pp. 41–76.
   Google Scholar
• F.A. Gomes, M.C. Maciel, and J.M. Martínez, Nonlinear programming algorithms using trust regions and augmented lagrangians with nonmonotone penalty parameters, Math. Program. 84 (1999), pp. 161–200.
   Web of Science ®Google Scholar
• N.I.M. Gould, D. Orban, and Ph.L. Toint, CUTEr and sifdec: A constrained and unconstrained testing environment, revisited, ACM Trans. Math. Softw. 29 (2003), pp. 373–394. doi: 10.1145/962437.962439
   Web of Science ®Google Scholar
• N.I.M. Gould, D. Orban, and Ph.L. Toint, Galahad—a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization, ACM Trans. Math. Softw. 29 (2003), pp. 353–372. doi: 10.1145/962437.962438
   Web of Science ®Google Scholar
• N.I.M. Gould, D. Orban, and Ph.L. Toint, CUTEst: A constrained and unconstrained testing environment with safe threads for mathematical optimization, Comput. Optim. Appl. 60 (2015), pp. 545–557. doi: 10.1007/s10589-014-9687-3
   Web of Science ®Google Scholar
• M.R. Hestenes, Multiplier and gradient methods, J. Optim. Theory Appl. 4 (1969), pp. 303–320. doi: 10.1007/BF00927673
   Google Scholar
• A.F. Izmailov and M.V. Solodov, On attraction of linearly constrained Lagrangian methods and of stabilized and quasi-Newton SQP methods to critical multipliers, Math. Program. 126 (2011), pp. 231–257. Available at http://dx.doi.org/10.1007/s10107-009-0279-4. doi: 10.1007/s10107-009-0279-4
   Web of Science ®Google Scholar
• M. Kočvara and M. Stingl, PENNON: A code for convex nonlinear and semidefinite programming, Optim. Methods Softw. 18 (2003), pp. 317–333. doi: 10.1080/1055678031000098773
   Web of Science ®Google Scholar
• M. Kočvara and M. Stingl, PENNON: A generalized augmented Lagrangian method for semidefinite programming, in High Performance Algorithms and Software for Nonlinear Optimization (Erice, 2001), Applied Optimization, Vol. 82, Kluwer Academic Publishing, Norwell, MA, 2003, pp. 303–321. Available at http://dx.doi.org/10.1007/978-1-4613-0241-4_14.
   Google Scholar
• M. Mongeau and A. Sartenaer, Automatic decrease of the penalty parameter in exact penalty function methods, European J. Oper. Res. 83 (1995), pp. 686–699. doi: 10.1016/0377-2217(93)E0339-Y
   Web of Science ®Google Scholar
• J.L. Morales, A numerical study of limited memory BFGS methods, Appl. Math. Lett. 15 (2002), pp. 481–487. doi: 10.1016/S0893-9659(01)00162-8
   Web of Science ®Google Scholar
• J.J. Moré, Trust regions and projected gradients, in System Modelling and Optimization, Vol. 113, Lecture Notes in Control and Information Sciences, Masao Iri and Keiji Yajima eds., Springer Verlag, Heidelberg, Berlin, New York, 1988, pp. 1–13.
   Google Scholar
• M.J.D. Powell, A method for nonlinear constraints in minimization problems, in Optimization, Roger Fletcher ed., Academic Press, London and New York, 1969, pp. 283–298.
   Google Scholar
• Z. Qin, D. Goldfarb, and S. Ma, An alternating direction method for total variation denoising, arXiv preprint (2011). Available at arXiv:1108.1587.
   Google Scholar
• Ph.L. Toint, Nonlinear stepsize control, trust regions and regularizations for unconstrained optimization, Optim. Methods Softw. 28 (2013), pp. 82–95. Available at http://www.tandfonline.com/doi/abs/10.1080/10556788.2011.610458. doi: 10.1080/10556788.2011.610458
   Web of Science ®Google Scholar
• J. Yang, Y. Zhang, and W. Yin, A fast alternating direction method for TVL1-L2 signal reconstruction from partial Fourier data, IEEE J. Sel. Top. Signal Process. 4 (2010), pp. 288–297. doi: 10.1109/JSTSP.2010.2042333
   Web of Science ®Google Scholar
• R.D. Zimmerman, C.E. Murillo-Sánchez, and R.J. Thomas, Matpower: Steady-state operations, planning, and analysis tools for power systems research and education, IEEE Trans. Power Syst. 26 (2011), pp. 12–19. doi: 10.1109/TPWRS.2010.2051168
   Web of Science ®Google Scholar