Some extensions of the operator splitting schemes based on Lagrangian and primal–dual: a unified proximal point analysis

References

• Shefi R, Teboulle M. Rate of convergence analysis of decomposition methods based on the proximal method of multipliers for convex minimization. SIAM J Optim. 2014;24(1):269–297.
   Web of Science ®Google Scholar
• Bredies K, Sun H. Preconditioned alternating direction method of multipliers for the minimization of quadratic plus non-smooth convex functionals. Universität Graz, 2015. (SFB Report 2015-006).
   Google Scholar
• Banert S, Boţ RI, Csetnek ER. Fixing and extending some recent results on the ADMM algorithm. Numer Algorithms. 2021;86:1303–1325.
   PubMed Web of Science ®Google Scholar
• Douglas J, Rachford H. On the numerical solution of heat conduction problems in two and three space variables. Trans Amer Math Soc. 1956;82:421–439.
   Google Scholar
• Zhu M, Chan T. An efficient primal-dual hybrid gradient algorithm for total variation image restoration. UCLA; 2008. (CAM Report 08-34).
   Google Scholar
• Chambolle A, Pock T. A first-order primal-dual algorithm for convex problems with applications to imaging. J Math Imaging Vis. 2011;40(1):120–145.
   Web of Science ®Google Scholar
• Boyd S, Parikh N, Chu E, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations Trends Mach Learning. 2011;3(1):1–122.
   Google Scholar
• Osher S, Burger M, Goldfarb D, et al. An iterative regularization method for total variation-based image restoration. Multiscale Model Simul. 2005;4(2):460–489.
   Web of Science ®Google Scholar
• Yin W, Osher S, Goldfarb D, et al. Bregman iterative algorithms for ℓ1-minimization with applications to compressed sensing. SIAM J Imaging Sci. 2008;1(1):143–168.
   Web of Science ®Google Scholar
• Drori Y, Sabach S, Teboulle M. A simple algorithm for a class of nonsmooth convex–concave saddle-point problems. Oper Res Lett. 2015;43(2):209–214.
   Web of Science ®Google Scholar
• Boţ RI, Csetnek E. ADMM for monotone operators: convergence analysis and rates. Adv Comput Math. 2019;45:327–359.
   Web of Science ®Google Scholar
• Vũ B. A splitting algorithm for coupled system of primal–dual monotone inclusions. J Optim Theory Appl. 2015;164:993–1025.
   Web of Science ®Google Scholar
• Chouzenoux E, Pesquet JC, Repetti A. A block coordinate variable metric forward–backward algorithm. J Global Optim. 2016;66:457–485.
   Web of Science ®Google Scholar
• Bauschke HH, Combettes PL. Convex analysis and monotone operator theory in Hilbert spaces. New York: Springer; 2011. (CMS books in mathematics).
   Google Scholar
• Combettes PL, Wajs VR. Signal recovery by proximal forward-backward splitting. Multiscale Model Simul. 2005;4(4):1168–1200.
   Web of Science ®Google Scholar
• Condat L. A primal-dual splitting method for convex optimization involving Lipschitzian, proximable, and linear composite terms. J Optim Theory Appl. 2013;158(2):460–479.
   Web of Science ®Google Scholar
• Combettes PL, Pesquet JC. Primal-dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators. Set-Valued Var Anal. 2012;20(2):307–330.
   Web of Science ®Google Scholar
• Vũ B. A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv Comput Math. 2013;38(3):667–681.
   Web of Science ®Google Scholar
• Esser E, Zhang X, Chan T. A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science. SIAM J Imaging Sci. 2010;3(4):1015–1046.
   Web of Science ®Google Scholar
• Boţ RI, Hendrich C. Convergence analysis for a primal-dual monotone+skew splitting algorithm with applications to total variation minimization. J Math Imaging Vis. 2014;49:551–568.
   Web of Science ®Google Scholar
• Boţ R, Hendrich C. A Douglas-Rachford type primal-dual method for solving inclusions with mixtures of composite and parallel-sum type monotone operators. SIAM J Optim. 2013;23(4):2541–2565.
   Web of Science ®Google Scholar
• Parikh N, Boyd S. Proximal algorithms. Found Trends Optim. 2014;1(3):123–231.
   Google Scholar
• Moreau JJ. Proximité et dualité dans un espace hilbertien. Bull Soc Math France. 1965;93(2):273–299.
   Google Scholar
• Martinet B. Régularisation d'inéquations variationnelles par approximations successives. Rev Fr Inform Rech Opér. 1970;4:154–158.
   Google Scholar
• Rochafellar R. Monotone operators and the proximal point algorithm. SIAM J Control Optim. 1976;14(5):877–898.
   Google Scholar
• He B, Yuan X. On the O(1/n) convergence rate of the Douglas-Rachford alternating direction method. SIAM J Numer Anal. 2012;50(2):700–709.
   Web of Science ®Google Scholar
• He B, Yuan X. Convergence analysis of primal-dual algorithms for a saddle-point problem: from contraction perspective. SIAM J Imaging Sci. 2012;5(1):119–149.
   Web of Science ®Google Scholar
• He B, Ma F, Yuan X. An algorithmic framework of generalized primal-dual hybrid gradient methods for saddle point problems. J Math Imaging Vis. 2017;58(2):279–293.
   Web of Science ®Google Scholar
• Ma F, Ni M. A class of customized proximal point algorithms for linearly constrained convex optimization. Comput Appl Math. 2018;37:896–911.
   Google Scholar
• Fang EX, He B, Liu H, et al. Generalized alternating direction method of multipliers: new theoretical insights and applications. Math Program Comput. 2015;7(2):149–187.
   PubMedGoogle Scholar
• He B, Yuan X. A class of ADMM-based algorithms for three-block separable convex programming. Comput Optim Appl. 2018;70:791–826.
   Web of Science ®Google Scholar
• Chambolle A, Pock T. On the ergodic convergence rates of a first-order primal–dual algorithm. Math Program Ser A. 2016;159(1–2):253–287.
   Web of Science ®Google Scholar
• Deng W, Lai M, Peng Z, et al. Parallel multi-block ADMM with O(1/k) convergence. J Sci Comput. 2017;71(2):712–736.
   Web of Science ®Google Scholar
• Bai J, Zhang H, Li J. A parameterized proximal point algorithm for separable convex optimization. Optim Lett. 2018;12:1589–1608.
   Web of Science ®Google Scholar
• He B, Xu M, Yuan X. Block-wise ADMM with a relaxation factor for multiple-block convex programming. J Oper Res Soc China. 2018;6:485–505.
   Web of Science ®Google Scholar
• Bùi MN, Combettes PL. Warped proximal iterations for monotone inclusions. J Math Anal Appl. 2020;491(1):124315.
   Google Scholar
• Teboulle M. A simplified view of first order methods for optimization. Math Program Ser B. 2018;170:67–96.
   Web of Science ®Google Scholar
• Nguyen Q. Variable quasi-Bregman monotone sequences. Numer Algorithms. 2016;73(4):1107–1130.
   Web of Science ®Google Scholar
• Bùi MN, Combettes PL. Bregman forward-backward operator splitting. Set-Valued Var Anal. 2021;29:583–603.
   Web of Science ®Google Scholar
• Rockafellar RT. Convex analysis. Princeton, NJ: Princeton University Press; 1996. (Princeton landmarks in mathematics and physics).
   Google Scholar
• Rockafellar RT, Wets RJB. Variational analysis. Berlin: Springer-Verlag; 2004. (Grundlehren der Mathematischen Wissenschaft); 317.
   Google Scholar
• Beck A. First-order methods in optimization. Philadelphia: Society for Industrial and Applied Mathematics, MOS-SIAM series on optimization; 2017.
   Google Scholar
• He B, Yuan X. On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers. Numer Math. 2015;130(3):567–577.
   Web of Science ®Google Scholar
• Opial Z. Weak convergence of the sequence of successive approximations for nonexpansive mappings. Bull Amer Math Soc. 1967;73:591–597.
   Web of Science ®Google Scholar
• He B, Xu H, Yuan X. On the proximal Jacobian decomposition of ALM for multiple-block separable convex minimization problems and its relationship to ADMM. J Sci Comput. 2016;66(3):1204–1217.
   Web of Science ®Google Scholar
• Tao M, Yuan X. Convergence analysis of the direct extension of ADMM for multiple-block separable convex minimization. Adv Comput Math. 2018;44:773–813.
   Web of Science ®Google Scholar
• Gonçalves MLN, Marques AM, Melo JG. Pointwise and ergodic convergence rates of a variable metric proximal alternating direction method of multipliers. J Optim Theory Appl. 2018;177:448–478.
   Web of Science ®Google Scholar
• Zhang X, Burger M, Osher S. A unified primal-dual algorithm framework based on Bregman iteration. J Sci Comput. 2011;46(1):20–46.
   Web of Science ®Google Scholar
• Cai XJ, Gu GY, He BS, et al. A proximal point algorithm revisit on the alternating direction method of multipliers. Sci China (Math). 2013;10:2179–2186.
   Google Scholar
• Bredies K, Sun H. A proximal point analysis of the preconditioned alternating direction method of multipliers. J Optim Theory Appl. 2017;173:878–907.
   Web of Science ®Google Scholar
• Afonso M, Bioucas-Dias J, MAT F. An augmented Lagrangian approach to the constrained optimization formulation of imaging inverse problems. IEEE Trans Image Process. 2011;20(3):681–695.
   PubMed Web of Science ®Google Scholar
• Chambolle A. An algorithm for total variation minimization and applications. J Math Imaging Vis. 2004;20:89–97.
   Web of Science ®Google Scholar
• Xue F, Luisier F, Blu T. Multi-Wiener SURE-LET deconvolution. IEEE Trans Image Process. 2013;22(5):1954–1968.
   PubMed Web of Science ®Google Scholar
• O'Connor D, Vandenberghe L. Primal-dual decomposition by operator splitting and applications to image deblurring. SIAM J Imaging Sci. 2014;7(3):1724–1754.
   Web of Science ®Google Scholar
• Combettes PL, Pesquet JC. Proximal splitting methods in signal processing. New York (NY): Springer-Verlag; 2011. p. 185–212.
   Google Scholar
• Beck A, Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sci. 2009;2(1):183–202.
   Web of Science ®Google Scholar