An inertial proximal alternating direction method of multipliers for nonconvex optimization

References

• F. Alvarez, On the minimizing property of a second order dissipative system in Hilbert spaces, SIAM J. Control Optim. 38 (2000), pp. 1102–1119. doi: 10.1137/S0363012998335802
   Web of Science ®Google Scholar
• F. Alvarez, Weak convergence of a relaxed and inertial hybrid projection-proximal point algorithm for maximal monotone operators in Hilbert space, SIAM J. Optim. 14 (2004), pp. 773–782. doi: 10.1137/S1052623403427859
   Web of Science ®Google Scholar
• F. Alvarez and H. Attouch, An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping, Set-Valued Anal. 9 (2001), pp. 3–11. doi: 10.1023/A:1011253113155
   Google Scholar
• H. Attouch and J. Bolte, On the convergence of the proximal algorithm for nonsmooth functions involving analytic features, Math. Program. Ser. B 116 (2009), pp. 5–16. doi: 10.1007/s10107-007-0133-5
   Web of Science ®Google Scholar
• H. Attouch, J. Bolte, P. Redont, and A. Soubeyran, A proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka–Łojasiewicz inequality, Math. Oper. Res. 35 (2010), pp. 438–457. doi: 10.1287/moor.1100.0449
   Web of Science ®Google Scholar
• H. Attouch, J. Bolte, and B.F. Svaiter, Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods, Math. Program. 137 (2013), pp. 91–129. doi: 10.1007/s10107-011-0484-9
   Web of Science ®Google Scholar
• A. Moudafi, E. Elissabeth, Approximate inertial proximal methods using the enlargement of maximal monotone operators. Int. J. Pure Appl. Math. 5(3) (2003), pp. 283–299.
   Google Scholar
• H. Attouch, J. Peypouquet, and P. Redont, A dynamical approach to an inertial forward-backward algorithm for convex minimization, Siam J. Optim. 24 (2014), pp. 232–256. doi: 10.1137/130910294
   Web of Science ®Google Scholar
• R.I. Bot and E.R. Csetnek, An inertial tsengs type proximal algorithm for nonsmooth and nonconvex optimization problems, J. Optim. Theory Appl. 171 (2016), pp. 1–17. doi: 10.1007/s10957-015-0730-z
   Web of Science ®Google Scholar
• R.I. Bot, E.R. Csetnek, and C. Hendrich, Inertial Douglas–Rachford splitting for monotone inclusion problems, Appl. Math. Comput. 256 (2015), pp. 472–487.
   Web of Science ®Google Scholar
• R.I. Bot, E.R. Csetnek, and S.C. Laszlo, An inertial forward–backward algorithm for the minimization of the sum of two nonconvex functions, Eur. J. Comput. Optim. 4 (2016), pp. 3–25. doi: 10.1007/s13675-015-0045-8
   Web of Science ®Google Scholar
• J. Bolte, A. Daniilidis, O. Ley, and L. Mazet, Characterizations of Lojasiewicz inequalities and applications, Trans. Amer. Math. Soc. 362 (2008), pp. 3319–3363. doi: 10.1090/S0002-9947-09-05048-X
   Web of Science ®Google Scholar
• D.P. Bertsekas and J.N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods, Prentice-Hall, New Jersey, 1989.
   Google Scholar
• C.H. Chen, R.H. Chan, S.Q. Ma, and J.F. Yang, Inertial proximal ADMM for linearly constrained separable convex optimization, SIAM J. Imaging Sci. 8 (2015), pp. 2239–2267. doi: 10.1137/15100463X
   Web of Science ®Google Scholar
• D.L. Donoho, Compressed sensing, IEEE Trans. Inf. Theory. 52 (2006), pp. 1289C–1306. doi: 10.1109/TIT.2006.871582
   Web of Science ®Google Scholar
• M. Elad, J.L. Starck, P. Querre, and D.L. Donoho, Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA), Appl. Comput. Harmon. Anal. 19 (2005), pp. 340C–358. doi: 10.1016/j.acha.2005.03.005
   Web of Science ®Google Scholar
• J. Eckstein, Splitting methods for monotone operators with applications to parallel optimization, Ph.D. diss. Operations Research Center, MIT, 1989.
   Google Scholar
• J. Fan, Y. Liao, and H. Liu, An overview on the estimation of large covariance and precision matrices, The Econometrics J. 19 (2016), pp. 1–32. doi: 10.1111/ectj.12061
   Web of Science ®Google Scholar
• T. Goldstein, B. Donoghue, S. Setzer, and R. Baraniuk, Fast alternating direction optimization methods, SIAM J. Imaging Sci. 7 (2014), pp. 1588–1623. doi: 10.1137/120896219
   Web of Science ®Google Scholar
• K. Guo, D.R. Han, and T.T. Wu, Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints, Int. J. Comput. Math. 94 (2017), pp. 1653–1669. doi: 10.1080/00207160.2016.1227432
   Web of Science ®Google Scholar
• K. Guo, D.R. Han, D.Z.W. Wang, and T.T. Wu, Convergence of ADMM for multi-block nonconvex separable optimization models, Front. Math. China 12 (2017), pp. 1139–1162. doi: 10.1007/s11464-017-0631-6
   Web of Science ®Google Scholar
• D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite element approximations, Comput. Math. Appl. 2 (1976), pp. 17–40. doi: 10.1016/0898-1221(76)90003-1
   Google Scholar
• M.L.N. Goncalves, J.G. Melo, and R.D.C. Monteiro, Convergence rate bounds for a proximal ADMM with over-relaxation stepsize parameter for solving nonconvex linearly constrained problems, 2017.
   Google Scholar
• M.Y. Hong, Z.Q. Luo, and M. Razaviyayn, Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems, SIAM J. Optim. 26 (2014), pp. 3836–3840.
   Web of Science ®Google Scholar
• M. Hong, Z.Q. Luo, and M. Razaviyayn, Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems, SIAM J. Optim. 26 (2016), pp. 337–364. doi: 10.1137/140990309
   Web of Science ®Google Scholar
• P. Jain and P. Kar, Non-convex optimization for machine learning, Found. Trends Machine Learn. 10 (2017), pp. 142–336. doi: 10.1561/2200000058
   Web of Science ®Google Scholar
• K. Kurdyka, On gradients of functions definable in o-minimal structures, Ann. L. Fourier 48 (1998), pp. 769–783. doi: 10.5802/aif.1638
   Web of Science ®Google Scholar
• S. Lojasiewicz, Une propri été topologique des sous-ensembles analytiques reels, Les Éq. Aux Dériv. Part. 117 (1963), pp. 87–89.
   Google Scholar
• P.L. Lions and B. Mercier, Splitting algorithms for the sum of two nonlinear operators, SIAM J. Numer. Anal. 16 (1979), pp. 964–979. doi: 10.1137/0716071
   Web of Science ®Google Scholar
• G.Y. Li and T.K. Pong, Global convergence of splitting methods for nonconvex composite optimization, SIAM J. Optim. 25 (2015), pp. 2434–2460. doi: 10.1137/140998135
   Web of Science ®Google Scholar
• Dirk A. Lorenz and P. Thomas, An inertial forward–backward algorithm for monotone inclusions, J. Math. Imaging Vision 51 (2)(2015), pp. 311–325. doi: 10.1007/s10851-014-0523-2
   Web of Science ®Google Scholar
• B. Mordukhovich Variational Analysis and Generalized Differentiation I: Basic Theory, Grundlehren der Mathematischen Wissenschaften, Vol. 330, Springer, Berlin, 2006.
   Google Scholar
• Y. Nesterov, Introduction Lectures on Convex Optimization: A Basic Course, Springer, New York, 2004. 87.
   Google Scholar
• P. Ochs, Y.J. Chen, T. Brox, and T. Pock, iPiano: inertial proximal algorithm for non-convex optimization, Siam J. Imaging Sci. 7 (2014), pp. 1388–1419. doi: 10.1137/130942954
   Web of Science ®Google Scholar
• P. Ochs, T. Brox, and T. Pock, iPiasco: inertial proximal algorithm for strongly convex optimization, J. Math. Imaging Vision 53 (2015), pp. 171–181. doi: 10.1007/s10851-015-0565-0
   Web of Science ®Google Scholar
• R.T. Rockafellar, Monotone operators and the proximal point algorithm, SIAM J. Control Optim. 14 (1976), pp. 877-–898. doi: 10.1137/0314056
   Web of Science ®Google Scholar
• R.T. Rockafellar, R.J.B. Wets, Variational Analysis, Fundamental Principles of Mathematical Sciences, Vol. 317, Springer-Verlag, Berlin, 1998.
   Google Scholar
• T. Pock and S. Sabach, Inertial proximal alternating linearized minimization (iPALM) for nonconvex and nonsmooth problems, Siam J. Imaging Sci. 9 (2016), pp. 1756-–1787. doi: 10.1137/16M1064064
   Web of Science ®Google Scholar
• F.H. Wang, W.F. Cao, and Z.B. Xu, Convergence of multi-block Bregman ADMM for nonconvex composite problems, Sci. China Inf. Sci. 61 (2018), pp. 122101. Available at https://doi.org/10.1007/s11432-017-9367-6.
   Web of Science ®Google Scholar
• F.H. Wang, Z.B. Xu, and H.K. Xu, Convergence of alternating direction method with multipliers for non-convex composite problems, preprint (2014). arXiv:1410.8625.
   Google Scholar
• Y. Wang, W.T. Yin, and J.S. Zeng, Global convergence of ADMM in nonconvex nonsmooth optimization, J. Sci. Comput. (2018) Available at https://doi.org/10.1007/s10915-018-0757-z.
   Web of Science ®Google Scholar
• F. Wen, L. Chu, P. Liu, and R.C. Qiu, A survey on nonconvex regularization based sparse and low-rank recovery in signal processing, statistics, and machine learning, IEEE Access. 6 (2018), pp. 69883–69906. doi: 10.1109/ACCESS.2018.2880454
   Web of Science ®Google Scholar
• Z. Wu and M. Li, General inertial proximal gradient method for a class of nonconvex nonsmooth optimization problems, Comput. Optim. Appl. 73 (2019), pp. 129–158. doi: 10.1007/s10589-019-00073-1
   Web of Science ®Google Scholar
• Z.B. Xu, X.Y. Chang, F.M. Xu, and H. Zhang, l12 regularization: a thresholding representation theory and a fast solver, IEEE T. Neur. Net. Lear. 23 (2012), pp. 1013–1027. doi: 10.1109/TNNLS.2012.2197412
   PubMed Web of Science ®Google Scholar
• L.F. Yang, J.Y. Luo, Y. Xuet al., A distributed dual consensus ADMM based on partition for DC-DOPF with carbon emission trading. IEEE T. Ind. Inform. 16 (2020), pp. 1858–1872. doi: 10.1109/TII.2019.2937513
   Web of Science ®Google Scholar
• L. Yang, T.K. Pong, and X.J. Chen, Alternating direction method of multipliers for a class of nonconvex and nonsmooth problems with applications to background/foreground extraction, SIAM J. Imaging Sci. 10 (2017), pp. 74–110. doi: 10.1137/15M1027528
   Web of Science ®Google Scholar