Convergence rates of mixed primal-dual dynamical systems with Hessian driven damping

References

• Goldstein T, O'Donoghue B, Setzer S, et al. Fast alternating direction optimization methods. SIAM J Imaging Sci. 2014;7(3):1588–1623. doi: 10.1137/120896219
   Web of Science ®Google Scholar
• He B, Yuan X, Zhang W. A customized proximal point algorithm for convex minimization with linear constraints. Comput Optim Appl. 2013;56(3):559–572. doi: 10.1007/s10589-013-9564-5
   Web of Science ®Google Scholar
• Boyd S, Parikh N, Chu E, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn. 2011;3(1):1–122. doi: 10.1561/2200000016
   Google Scholar
• Lin Z, Li H, Fang C. Accelerated optimization for machine learning. Nature Singapore: Springer; 2020.
   Google Scholar
• Lin Z, Li H, Fang C. Alternating direction method of multipliers for machine learning. Nature Singapore: Springer; 2022.
   Google Scholar
• Liu Q, Yang S, Hong Y. Constrained consensus algorithms with fixed step size for distributed convex optimization over multiagent networks. IEEE Trans Autom Control. 2017;62(8):4259–4265. doi: 10.1109/TAC.2017.2681200
   Web of Science ®Google Scholar
• Shi G, Johansson KH. Randomized optimal consensus of multi-agent systems. Automatica. 2012;48(12):3018–3030. doi: 10.1016/j.automatica.2012.08.018
   Web of Science ®Google Scholar
• Candès EJ, Wakin MB. An introduction to compressive sampling. IEEE Trans Signal Process. 2008;25(2):21–30. doi: 10.1109/MSP.2007.914731
   Google Scholar
• He X, Hu R, Fang YP. Inertial accelerated primal-dual methods for linear equality constrained convex optimization problems. Numer Algorithms. 2022;90:1669–1690. doi: 10.1007/s11075-021-01246-y
   Web of Science ®Google Scholar
• Su W, Boyd S, Candès EJ. A differential equation for modeling Nesterov's accelerated gradient method: theory and insights. J Mach Learn Res. 2016;17:5312–5354.
   Web of Science ®Google Scholar
• Nesterov Y. A method of solving a convex programming problem with convergence rate O(1/k2). InSov Math Dokl. 1983;27(2):372–376.
   Google Scholar
• Beck A, Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sci. 2009;2(1):183–202. doi: 10.1137/080716542
   Web of Science ®Google Scholar
• Attouch H, Chbani Z, Peypouquet J, et al. Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity. Math Program. 2018;168(1–2):123–175. doi: 10.1007/s10107-016-0992-8
   Web of Science ®Google Scholar
• Wibisono A, Wilson AC, Jordan MI. A variational perspective on accelerated methods in optimization. Proc Natl Acad Sci USA. 2016;113(47):E7351–E7358. doi: 10.1073/pnas.1614734113
   PubMed Web of Science ®Google Scholar
• Wilson AC, Recht B, Jordan MI. A Lyapunov analysis of accelerated methods in optimization. J Mach Learn Res. 2021;22(113):1–34.
   Google Scholar
• Attouch H, Chbani Z, Riahi H. Fast proximal methods via time scaling of damped inertial dynamics. SIAM J Optim. 2019;29(3):2227–2256. doi: 10.1137/18M1230207
   Web of Science ®Google Scholar
• Attouch H, Peypouquet J. The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than 1/k2. SIAM J Optim. 2016;26(3):1824–1834. doi: 10.1137/15M1046095
   Web of Science ®Google Scholar
• Fazlyab M, Koppel A, Preciado VM, et al. A variational approach to dual methods for constrained convex optimization. In: 2017 American Control Conference. 2017. p. 5269–5275.
   Google Scholar
• Zeng X, Lei J, Chen J. Dynamical primal-dual accelerated method with applications to network optimization. IEEE Trans Autom Control. 2023;68(3):1760–1767. doi: 10.1109/TAC.2022.3152720
   Web of Science ®Google Scholar
• Boct RI, Nguyen DK. Improved convergence rates and trajectory convergence for primal-dual dynamical systems with vanishing damping. J Differ Equ. 2021;303:369–406. doi: 10.1016/j.jde.2021.09.021
   Web of Science ®Google Scholar
• Boct RI, Csetnek ER, Nguyen DK. Fast augmented Lagrangian method in the convex regime with convergence guarantees for the iterates. Math Program. 2023;200:147–197. doi: 10.1007/s10107-022-01879-4
   PubMed Web of Science ®Google Scholar
• He X, Hu R, Fang YP. Inertial primal-dual dynamics with damping and scaling for linearly constrained convex optimization problems. Appl Anal. 2022. doi: 10.1080/00036811.2022.2104260
   PubMed Web of Science ®Google Scholar
• He X, Hu R, Fang YP. Convergence rates of inertial primal-dual dynamical methods for separable convex optimization problems. SIAM J Control Optim. 2021;59(5):3278–3301. doi: 10.1137/20M1355379
   Web of Science ®Google Scholar
• Attouch H, Chbani Z, Fadili J, et al. Fast convergence of dynamical ADMM via time scaling of damped inertial dynamics. J Optim Theory Appl. 2022;193(1–3):704–736. doi: 10.1007/s10957-021-01859-2
   Web of Science ®Google Scholar
• Hassan-Moghaddam S, Jovanović MR. Proximal gradient flow and Douglas-Rachford splitting dynamics: global exponential stability via integral quadratic constraints. Automatica. 2021;123:109311. doi: 10.1016/j.automatica.2020.109311
   Google Scholar
• Bitterlich S, Csetnek ER, Wanka G. A dynamical approach to two-block separable convex optimization problems with linear constraints. Numer Funct Anal Optim. 2020;42(1):1–38. doi: 10.1080/01630563.2020.1845730
   Web of Science ®Google Scholar
• Boct RI, Csetnek ER, László SC. A primal-dual dynamical approach to structured convex minimization problems. J Differ Equ. 2020;269(12):10717–10757. doi: 10.1016/j.jde.2020.07.039
   Web of Science ®Google Scholar
• Liang S, Yin G. Exponential convergence of distributed primal–dual convex optimization algorithm without strong convexity. Automatica. 2019;105:298–306. doi: 10.1016/j.automatica.2019.04.004
   Web of Science ®Google Scholar
• Luo H. A primal-dual flow for affine constrained convex optimization. ESAIM Control Optim Calc Var. 2022;28:33. doi: 10.1051/cocv/2022032
   Web of Science ®Google Scholar
• Tang Y, Qu G, Li N. Semi-global exponential stability of augmented primal-dual gradient dynamics for constrained convex optimization. Syst Control Lett. 2020;144:104754. doi: 10.1016/j.sysconle.2020.104754
   Google Scholar
• He X, Hu R, Fang YP. Fast primal-dual algorithm via dynamical system for a linearly constrained convex optimization problem. Automatica. 2022;146:110547. doi: 10.1016/j.automatica.2022.110547
   Google Scholar
• He X, Hu R, Fang YP. ‘Second-order primal’ + ‘first-order dual’ dynamical systems with time scaling for linear equality constrained convex optimization problems. IEEE Trans Autom Control. 2022;67(8):4377–4383. doi: 10.1109/TAC.2022.3176527
   Web of Science ®Google Scholar
• Alvarez F, Attouch H, Bolte J, et al. A second-order gradient-like dissipative dynamical system with Hessian-driven damping: application to optimization and mechanics. J Math Pures Appl. 2002;81(8):747–779. doi: 10.1016/S0021-7824(01)01253-3
   Web of Science ®Google Scholar
• Attouch H, Peypouquet J, Redont P. Fast convex optimization via inertial dynamics with Hessian driven damping. J Differ Equ. 2016;261(10):5734–5783. doi: 10.1016/j.jde.2016.08.020
   Web of Science ®Google Scholar
• Shi B, Du SS, Jordan MI, et al. Understanding the acceleration phenomenon via high-resolution differential equations. Math Program. 2022;195:79–148. doi: 10.1007/s10107-021-01681-8
   Web of Science ®Google Scholar
• Shi B, Du SS, Su WJ, et al. Acceleration via symplectic discretization of high-resolution differential equations. Adv Neural Inf Process Syst. 2019:5745–5753.
   Google Scholar
• Boct RI, Csetnek ER, László SC. Tikhonov regularization of a second order dynamical system with Hessian driven damping. Math Program. 2021;189(1–2):151–186.
   PubMed Web of Science ®Google Scholar
• Attouch H, Chbani Z, Fadili J, et al. First-order optimization algorithms via inertial systems with Hessian driven damping. Math Program. 2022;193:113–155. doi: 10.1007/s10107-020-01591-1
   Web of Science ®Google Scholar
• Attouch H, Boct RI, Csetnek ER. Fast optimization via inertial dynamics with closed-loop damping. J Eur Math Soc. 2023;25(5):1985–2056. doi: 10.4171/JEMS/1231
   Web of Science ®Google Scholar
• Attouch H, Maingé PE, Redont P. A second-order differential system with Hessian-driven damping; application to non-elastic shock laws. Differ Equ Appl. 2012;4(1):27–65.
   Google Scholar
• Attouch H, Fadili JM, Kungurtsev V. On the effect of perturbations in first-order optimization methods with inertia and Hessian driven damping. Evol Equ Control Theory. 2023;12(1):71–117. doi: 10.3934/eect.2022022
   Google Scholar
• Chen L, Luo H. First order optimization methods based on Hessian-driven Nesterov accelerated gradient flow. 2019. arXiv:1912.09276
   Google Scholar
• Nesterov Y. Introductory lectures on convex optimization: a basic course. New York: Springer Science & Business Media; 2004.
   Google Scholar
• Attouch H, Boct RI, Nguyen DK. Fast convex optimization via closed-loop time scaling of gradient dynamics. 2023. arXiv:2301.00701
   Google Scholar
• Xu B, Wen B. On the convergence of a class of inertial dynamical systems with Tikhonov regularization. Optim Lett. 2021;15:2025–2052. doi: 10.1007/s11590-020-01663-3
   Web of Science ®Google Scholar