Gradient methods exploiting spectral properties

References

• H. Akaike, On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method, Ann. Inst. Stat. Math. 11 (1959), pp. 1–16. doi: 10.1007/BF01831719
   Web of Science ®Google Scholar
• J. Barzilai and J.M. Borwein, Two-point step size gradient methods, IMA J. Numer. Anal. 8 (1988), pp. 141–148. doi: 10.1093/imanum/8.1.141
   Web of Science ®Google Scholar
• E.G. Birgin, J.M. Martínez, and M. Raydan, Nonmonotone spectral projected gradient methods on convex sets, SIAM J. Optim. 10 (2000), pp. 1196–1211. doi: 10.1137/S1052623497330963
   Web of Science ®Google Scholar
• E.G. Birgin, J.M. Martínez, and M. Raydan, Spectral projected gradient methods: Review and perspectives, J. Stat. Softw. 60 (2014), pp. 539–559. doi: 10.18637/jss.v060.i03
   Web of Science ®Google Scholar
• A. Cauchy, Méthode générale pour la résolution des systemes d'équations simultanées, Comp. Rend. Sci. Paris 25 (1847), pp. 536–538.
   Google Scholar
• C. Cortes and V. Vapnik, Support-vector networks, Mach. Learn. 20 (1995), pp. 273–297.
   Web of Science ®Google Scholar
• Y.H. Dai, Alternate step gradient method, Optimization 52 (2003), pp. 395–415. doi: 10.1080/02331930310001611547
   Web of Science ®Google Scholar
• Y.H. Dai, M. Al-Baali, and X. Yang, A positive Barzilai–Borwein-like stepsize and an extension for symmetric linear systems, in Numerical Analysis and Optimization, Springer, 2015, pp. 59–75.
   Google Scholar
• Y.H. Dai and R. Fletcher, Projected Barzilai–Borwein methods for large-scale box-constrained quadratic programming, Numer. Math. 100 (2005), pp. 21–47. doi: 10.1007/s00211-004-0569-y
   Web of Science ®Google Scholar
• Y.H. Dai, Y. Huang, and X.W. Liu, A family of spectral gradient methods for optimization, Comp. Optim. Appl. 74 (2019), pp. 43–65. doi: 10.1007/s10589-019-00107-8
   Web of Science ®Google Scholar
• Y.H. Dai and L.Z. Liao, R-linear convergence of the Barzilai and Borwein gradient method, IMA J. Numer. Anal. 22 (2002), pp. 1–10. doi: 10.1093/imanum/22.1.1
   Web of Science ®Google Scholar
• Y.H. Dai and X. Yang, A new gradient method with an optimal stepsize property, Comp. Optim. Appl. 33 (2006), pp. 73–88. doi: 10.1007/s10589-005-5959-2
   Web of Science ®Google Scholar
• Y.H. Dai and Y.X. Yuan, Alternate minimization gradient method, IMA J. Numer. Anal. 23 (2003), pp. 377–393. doi: 10.1093/imanum/23.3.377
   Web of Science ®Google Scholar
• Y.H. Dai and Y.X. Yuan, Analysis of monotone gradient methods, J. Ind. Mang. Optim. 1 (2005), pp. 181–192.
   Web of Science ®Google Scholar
• Y.H. Dai and H. Zhang, Adaptive two-point stepsize gradient algorithm, Numer. Algor. 27 (2001), pp. 377–385. doi: 10.1023/A:1013844413130
   Web of Science ®Google Scholar
• R. De Asmundis, D. Di Serafino, W.W. Hager, G. Toraldo, and H. Zhang, An efficient gradient method using the Yuan steplength, Comp. Optim. Appl. 59 (2014), pp. 541–563. doi: 10.1007/s10589-014-9669-5
   Web of Science ®Google Scholar
• D. Di Serafino, V. Ruggiero, G. Toraldo, and L. Zanni, On the steplength selection in gradient methods for unconstrained optimization, Appl. Math. Comput. 318 (2018), pp. 176–195.
   Web of Science ®Google Scholar
• E.D. Dolan and J.J. Moré, Benchmarking optimization software with performance profiles, Math. Program. 91 (2002), pp. 201–213. doi: 10.1007/s101070100263
   Web of Science ®Google Scholar
• H.C. Elman and G.H. Golub, Inexact and preconditioned Uzawa algorithms for saddle point problems, SIAM J. Numer. Anal. 31 (1994), pp. 1645–1661. doi: 10.1137/0731085
   Web of Science ®Google Scholar
• M.A. Figueiredo, R.D. Nowak, and S.J. Wright, Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems, IEEE J. Sel. Top. Signal Process. 1 (2007), pp. 586–597. doi: 10.1109/JSTSP.2007.910281
   Web of Science ®Google Scholar
• R. Fletcher, On the Barzilai–Borwein method, in Optimization and Control with Applications, L. Qi, K. Teo, X. Yang, P. M. Pardalos and D. Hearn, eds., Springer, 2005, pp. 235–256.
   Google Scholar
• G.E. Forsythe, On the asymptotic directions of the s-dimensional optimum gradient method, Numer. Math. 11 (1968), pp. 57–76. doi: 10.1007/BF02165472
   Web of Science ®Google Scholar
• G. Frassoldati, L. Zanni, and G. Zanghirati, New adaptive stepsize selections in gradient methods, J. Ind. Mang. Optim. 4 (2008), pp. 299–312.
   Web of Science ®Google Scholar
• A. Friedlander, J.M. Martínez, B. Molina, and M. Raydan, Gradient method with retards and generalizations, SIAM J. Numer. Anal. 36 (1998), pp. 275–289. doi: 10.1137/S003614299427315X
   Web of Science ®Google Scholar
• C.C. Gonzaga and R.M. Schneider, On the steepest descent algorithm for quadratic functions, Comp. Optim. Appl. 63 (2016), pp. 523–542. doi: 10.1007/s10589-015-9775-z
   Web of Science ®Google Scholar
• N.I. Gould, D. Orban, and P.L. Toint, CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization, Comp. Optim. Appl. 60 (2015), pp. 545–557. doi: 10.1007/s10589-014-9687-3
   Web of Science ®Google Scholar
• L. Grippo, F. Lampariello, and S. Lucidi, A nonmonotone line search technique for Newton's method, SIAM J. Numer. Anal. 23 (1986), pp. 707–716. doi: 10.1137/0723046
   Web of Science ®Google Scholar
• W.W. Hager and H. Zhang, A new active set algorithm for box constrained optimization, SIAM J. Optim. 17 (2006), pp. 526–557. doi: 10.1137/050635225
   Web of Science ®Google Scholar
• Y. Huang and H. Liu, Smoothing projected Barzilai–Borwein method for constrained non-Lipschitz optimization, Comp. Optim. Appl. 65 (2016), pp. 671–698. doi: 10.1007/s10589-016-9854-9
   Web of Science ®Google Scholar
• Y. Huang, H. Liu, and S. Zhou, Quadratic regularization projected Barzilai–Borwein method for nonnegative matrix factorization, Data Min. Knowl. Disc. 29 (2015), pp. 1665–1684. doi: 10.1007/s10618-014-0390-x
   Web of Science ®Google Scholar
• B. Jiang and Y.H. Dai, Feasible Barzilai–Borwein-like methods for extreme symmetric eigenvalue problems, Optim. Method Softw. 28 (2013), pp. 756–784. doi: 10.1080/10556788.2012.656115
   Web of Science ®Google Scholar
• D.D. Lee and H.S. Seung, Learning the parts of objects by non-negative matrix factorization, Nature 401 (1999), pp. 788–791. doi: 10.1038/44565
   PubMed Web of Science ®Google Scholar
• Y.F. Liu, Y.H. Dai, and Z.Q. Luo, Coordinated beamforming for MISO interference channel: Complexity analysis and efficient algorithms, IEEE Trans. Signal Process. 59 (2011), pp. 1142–1157. doi: 10.1109/TSP.2010.2092772
   Web of Science ®Google Scholar
• J. Nocedal, A. Sartenaer, and C. Zhu, On the behavior of the gradient norm in the steepest descent method, Comp. Optim. Appl. 22 (2002), pp. 5–35. doi: 10.1023/A:1014897230089
   Web of Science ®Google Scholar
• M. Raydan, On the Barzilai and Borwein choice of steplength for the gradient method, IMA J. Numer. Anal. 13 (1993), pp. 321–326. doi: 10.1093/imanum/13.3.321
   Web of Science ®Google Scholar
• M. Raydan, The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM J. Optim. 7 (1997), pp. 26–33. doi: 10.1137/S1052623494266365
   Web of Science ®Google Scholar
• Y.X. Yuan, A new stepsize for the steepest descent method, J. Comput. Math. 24 (2006), pp. 149–156.
   Web of Science ®Google Scholar
• Y.X. Yuan, Step-sizes for the gradient method, AMS/IP Stud. Adv. Math. 42 (2008), pp. 785–796.
   Google Scholar
• H. Zhang and W.W. Hager, A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim. 14 (2004), pp. 1043–1056. doi: 10.1137/S1052623403428208
   Web of Science ®Google Scholar
• B. Zhou, L. Gao, and Y.H. Dai, Gradient methods with adaptive step-sizes, Comp. Optim. Appl. 35 (2006), pp. 69–86. doi: 10.1007/s10589-006-6446-0
   Web of Science ®Google Scholar