A stochastic inertial forward–backward splitting algorithm for multivariate monotone inclusions

References

• Sibony M. Méthodes itératives pour les équations et in équations aux dérivées partielles non linéaires de type monotone [Iterative methods for monotone nonlinear partial differential equations and inequalities]. Calcolo. 1970;7:65–183.
   Google Scholar
• Attouch H, Briceño-Arias LM, Combettes PL. A parallel splitting method for coupled monotone inclusions. SIAM J. Control Optim. 2010;48:3246–3270.
   Web of Science ®Google Scholar
• Glowinski R, Le Tallec P. Augmented Lagrangian and operator-splitting methods in nonlinear mechanics. Philadelphia: SIAM; 1989.
   Google Scholar
• Haraux A. Nonlinear evolution equations: global behavior of solutions. Vol. 841, Lecture notes in mathematics. New York (NY): Springer-Verlag; 1981.
   Google Scholar
• Mercier B. Topics in finite element solution of elliptic problems. Vol. 63, Lectures on mathematics. Bombay: Tata Institute of Fundamental Research; 1979.
   Google Scholar
• Mercier B. Inéquations Variationnelles de la Mécanique [Variational inequalities in mechanics] (Publications Mathématiques d’Orsay, no. 80.01). Orsay: Université de Paris-XI; 1980.
   Google Scholar
• Combettes PL, Wajs VR. Signal recovery by proximal forward--backward splitting. Multiscale Model. Simul. 2005;4:1168–1200.
   Web of Science ®Google Scholar
• Daubechies I, Defrise M, De Mol C. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Comm. Pure Appl. Math. 2004;57:1413–1457.
   Web of Science ®Google Scholar
• De Vito E, Umanità V, Villa S. A consistent algorithm to solve Lasso, elastic-net and Tikhonov regularization. J. Complexity. 2011;27:188–200.
   Web of Science ®Google Scholar
• Duchi J, Singer Y. Efficient online and batch learning using forward backward splitting. J. Mach. Learn. Res. 2009;10:2899–2934.
   Web of Science ®Google Scholar
• Mosci S, Rosasco L, Santoro M, et al. Solving structured sparsity regularization with proximal methods. In: Machine learning and knowledge discovery in databases European conference; Barcelona; 2010. p. 418–433.
   Google Scholar
• Rosasco L, Villa S, Mosci S, et al. Nonparametric sparsity and regularization. J. Mach. Learn. Res. 2013;14:1665–1714.
   Web of Science ®Google Scholar
• Villa S, Salzo S, Baldassarre L, et al. Accelerated and inexact forward-backward algorithms. SIAM J. Optim. 2013;23:1607–1633.
   Web of Science ®Google Scholar
• Villa S, Rosasco L, Mosci S, et al. Proximal methods for the latent group lasso penalty. Comput. Anal. Optim. 2014;58:381–407.
   Web of Science ®Google Scholar
• Briceño-Arias LM, Combettes PL. Monotone operator methods for Nash equilibria in non-potential games. In: Bailey D, Bauschke HH, Borwein P, Garvan F, Théra M, Vanderwerff J, Wolkowicz H, editors. Computational and analytical mathematics. New York (NY): Springer; 2013.
   Google Scholar
• Facchinei F, Pang J-S. Finite-dimensional variational inequalities and complementarity problems. New York (NY): Springer-Verlag; 2003.
   Google Scholar
• Tseng P. Further applications of a splitting algorithm to decomposition in variational inequalities and convex programming. Math. Program. 1990;48:249–263.
   Web of Science ®Google Scholar
• Tseng P. Applications of a splitting algorithm to decomposition in convex programming and variational inequalities. SIAM J. Control Optim. 1991;29:119–138.
   Web of Science ®Google Scholar
• Zhu DL, Marcotte P. Co-coercivity and its role in the convergence of iterative schemes for solving variational inequalities. SIAM J. Optim. 1996;6:714–726.
   Web of Science ®Google Scholar
• Barty K, Roy J-S, Strugarek C. Hilbert-valued perturbed subgradient algorithms. Math. Oper. Res. 2007;32:551–562.
   Web of Science ®Google Scholar
• Bennar A, Monnez J-M. Almost sure convergence of a stochastic approximation process in a convex set. Int. J. Appl. Math. 2007;20:713–722.
   Google Scholar
• Nemirovski A, Juditsky A, Lan G, et al. Robust stochastic approximation approach to stochastic programming. SIAM J. Optim. 2008;19:1574–1609.
   Web of Science ®Google Scholar
• Combettes PL. Solving monotone inclusions via compositions of nonexpansive averaged operators. Optimization. 2004;53:475–504.
   Web of Science ®Google Scholar
• Chen GH-G, Rockafellar T. Convergence rates in forward--backward splitting. SIAM J. Optim. 1997;7:421–444.
   Web of Science ®Google Scholar
• Combettes PL, Vũ BC. Variable metric forward--backward splitting with applications to monotone inclusions in duality. Optimization. 2013;63:1289–1318.
   Web of Science ®Google Scholar
• Condat L. A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 2013;158:460–479.
   Web of Science ®Google Scholar
• Combettes PL, Condat L, Pesquet J-C, et al. A forward--backward view of some primal-dual optimization methods in image recovery. In: Proceedings of international conference on image processing; October. Paris, France; 2014. p. 27–30.
   Google Scholar
• Vũ BC. A splitting algorithm for coupled system of primal-dual monotone inclusions. J. Optim. Theory Appl. 2015;164:993–1025.
   Web of Science ®Google Scholar
• Nesterov Y. A method for unconstrained convex minimization problem with the rate of convergence O(1/k2). Doklady AN SSSR. 1983;269:543–547.
   Web of Science ®Google Scholar
• Lorenz DA, Pock T. An inertial forward--backward method for monotone inclusions. J. Math. Imaging Vis. 2015;51:311–325.
   Web of Science ®Google Scholar
• Alvarez F, Attouch H. An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping. Set-valued Anal. 2001;9:3–11.
   Google Scholar
• Moudafi A, Oliny M. Convergence of a splitting inertial proximal method for monotone operators. J. Comput. Appl. Math. 2003;155:447–454.
   Web of Science ®Google Scholar
• Pesquet JC, Pustelnik N. A parallel inertial proximal optimization method. Pac. J. Optim. 2012;8:273–306.
   Web of Science ®Google Scholar
• Polyak BT. Some methods of speeding up the convergence of iteration methods, U.S.S.R. Comput. Math. Math. Phys. 1964;4:1–17.
   Google Scholar
• Combettes PL, Pesquet J-C. Stochastic quasi-Fejér block-coordinate fixed point iterations with random sweeping. SIAM J. Optim. 2015;25:1221–1248.
   Web of Science ®Google Scholar
• Rosasco L, Villa S, Vũ BC. A stochastic forward--backward splitting method for solving monotone inclusions in Hilbert spaces. arXiv:1403.7999. 2014.Available from: http://arxiv.org/abs/1403.7999.
   Google Scholar
• Bianchi P, Hachem W, Iutzeler F. A stochastic coordinate descent primal-dual algorithm and applications to large-scale composite optimization. Available from: http://arxiv.org/abs/1407.0898.
   Google Scholar
• Pesquet J-C, Repetti A. A class of randomized primal-dual algorithms for distributed optimization. J. Nonlinear Convex Anal. 2015;16. arXiv:1406.6404.
   Web of Science ®Google Scholar
• Rosasco L, Villa S, Vũ BC. Convergence of stochastic proximal gradient. arXiv:1403.5074. 2014. Available from: http://arxiv.org/abs/1403.5074.
   Google Scholar
• Polyak BT. Introduction to optimization. New York (NY): Optimization Software Inc.; 1987.
   Google Scholar
• Moreau JJ. Fonctions convexes duales et points proximaux dans un espace hilbertien [Dual convex functions and proximal points in a Hilbert space]. C. R. Acad. Sci. Paris Sér. A. 1962;255:2897–2899.
   Google Scholar
• Bauschke HH, Combettes PL. Convex analysis and monotone operator theory in Hilbert spaces. New York (NY): Springer; 2011.
   Google Scholar
• Ledoux M, Talagrand M. Probability in Banach spaces: isoperimetry and processes. New York (NY): Springer; 1991.
   Google Scholar
• Robbins H, Siegmund D. A convergence theorem for non negative almost supermartingales and some applications. In: Rustagi JS, editor. Optimizing methods in statistic. New York (NY): Academic Press; 1971. p. 233–257.
   Google Scholar
• Raguet H, Fadili J, Peyré G. Generalized forward-backward splitting. SIAM J. Imaging Sci. 2013;6:1199–1226.
   Web of Science ®Google Scholar
• Chambolle A, Pock T. On the ergodic convergence rates of a first-order primal-dual algorithm. Math. Program. A. Forthcoming 2014. Available from: http://www.optimization-online.org/DBFILE/2014/09/4532.pdf.
   Web of Science ®Google Scholar
• Combettes PL. Systems of structured monotone inclusions: duality, algorithms, and applications. SIAM J. Optim. 2013;23:2420–2447.
   Web of Science ®Google Scholar
• Davis D. Convergence rate analysis of primal-dual splitting schemes. Forthcoming 2014. Available from: http://arxiv.org/abs/1408.4419.
   Google Scholar
• Chen P, Huang J, Zhang X. A primal-dual fixed point algorithm for convex separable minimization with applications to image restoration. Inverse Probl. 2013;29. doi:10.1088/0266-5611/29/2/025011.
   Web of Science ®Google Scholar
• Loris I, Verhoeven C. On a generalization of the iterative soft-thresholding algorithm for the case of non-separable penalty. Inverse Probl. 2011;27:125007.
   Web of Science ®Google Scholar