A proximal forward-backward splitting based algorithmic framework for Wasserstein logistic regression using heavy ball strategy

References

• Arrigo, A., Ordoudis, C., Kazempour, J., De Grève, Z., Toubeau, J. F., & Vallée, F. (2022). Wasserstein distributionally robust chance-constrained optimization for energy and reserve dispatch: An exact and physically-bounded formulation. European Journal of Operational Research, 296(1), 304–322. https://doi.org/10.1016/j.ejor.2021.04.015
   Web of Science ®Google Scholar
• Beck, A., & Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1), 183–202. https://doi.org/10.1137/080716542
   Web of Science ®Google Scholar
• Belli, E., & Vantini, S. (2022). Ridge regression with adaptive additive rectangles and other piecewise functional templates. Neurocomputing, 488, 328–344. https://doi.org/10.1016/j.neucom.2022.03.003
   Web of Science ®Google Scholar
• Bertsimas, D., Gupta, V., & Kallus, N. (2018). Data-driven robust optimization. Mathematical Programming, 167(2), 235–292. https://doi.org/10.1007/s10107-017-1125-8
   Web of Science ®Google Scholar
• Briceño-Arias, L. M., Chierchia, G., Chouzenoux, E., & Pesquet, J. C. (2019). Efficient learning using forward-backward splitting. Computational Optimization and Applications, 72(3), 707–726. https://doi.org/10.1007/s10589-019-00060-6
   Google Scholar
• Chen, R. D., & Paschalidis, I. C. (2018). A robust learning approach for regression models based on distributionally robust optimization. Journal of Machine Learning Research, 19(1), 517–564. https://doi.org/10.5555/3291125.3291138
   PubMedGoogle Scholar
• Combettes, P.L., & Wajs, V.R. (2005). Signal recovery by proximal forward-backward splitting. Multiscale modeling & simulation, 4(4), 1168–1200. https://doi.org/10.1137/050626090
   Web of Science ®Google Scholar
• Duchi, J. C., Glynn, P. W., & Namkoong, H. (2021). Statistics of robust optimization: A generalized empirical likelihood approach. Mathematics of Operations Research, 46(3), 946–969. https://doi.org/10.1287/moor.2020.1085
   Web of Science ®Google Scholar
• Ghadimi, E., Feyzmahdavian, H. R., & Johansson, M. (2015). Global convergence of the heavy-ball method for convex optimization. In European control conference (ECC) (pp. 310–315). IEEE. https://doi.org/10.1109/ecc.2015.7330562
   Google Scholar
• Giselsson, P. (2021). Nonlinear forward-backward splitting with projection correction. SIAM Journal on Optimization, 31(3), 2199–2226. https://doi.org/10.1137/20M1345062
   Web of Science ®Google Scholar
• Heuts, Y., Cazar, P. P., & Donkers, T. (2021). An adaptive restart heavy-ball projected primal-dual method for solving constrained linear quadratic optimal control problems. IEEE Conference on Decision and Control (CDC), 6722–6727.
   Google Scholar
• Holland, M. J., & Ikeda, K. (2019). Efficient learning with robust gradient descent. Machine Learning, 108(8-9), 1523–1560. https://doi.org/10.1007/s10994-019-05802-5
   Web of Science ®Google Scholar
• Ito, S. (2021). On optimal robustness to adversarial corruption in online decision problems. Advances in Neural Information Processing Systems, 7409–7420.
   Google Scholar
• Jayawardena, S., Epps, J., & Ambikairajah, E. (2023). Ordinal logistic regression with partial proportional odds for depression prediction. IEEE Transactions on Affective Computing, 14(1), 563–577. https://doi.org/10.1109/TAFFC.2020.3031300
   Web of Science ®Google Scholar
• Li, H. Q., Cheng, H. Q., Wang, Z., & Wu, G. C. (2020). Distributed Nesterov gradient and heavy-ball double accelerated asynchronous optimization. IEEE Transactions on Neural Networks and Learning Systems, 32(12), 5723–5737. https://doi.org/10.1109/TNNLS.2020.3027381
   Web of Science ®Google Scholar
• Li, J., Huang, S., & So, A.M. (2019). A first-order algorithmic framework for distributionally robust logistic regression. Advances in Neural Information Processing Systems, 3937–3947.
   Google Scholar
• Liu, Y. C., Yuan, X. M., Zeng, S. Z., & Zhang, J. (2017). Primal-dual hybrid gradient method for distributionally robust optimization problems. Operations Research Letters, 45(6), 625–630. https://doi.org/10.1016/j.orl.2017.10.001
   Web of Science ®Google Scholar
• Lofberg, J. (2004). YALMIP: A toolbox for modeling and optimization in MATLAB. In IEEE International Conference on Robotics and Automation, Taipei, Taiwan (pp. 284–289). IEEE. https://doi.org/10.1109/CACSD.2004.1393890
   Google Scholar
• Luo, F. Q., & Mehrotra, S. (2019). Decomposition algorithm for distributionally robust optimization using Wasserstein metric with an application to a class of regression models. European Journal of Operational Research, 278(1), 20–35. https://doi.org/10.1016/j.ejor.2019.03.008
   Web of Science ®Google Scholar
• Mohajerin, E. P., & Kuhn, D. (2018). Data-driven distributionally robust optimization using the Wasserstein metric: Performance guarantees and tractable reformulations. Mathematical Programming, 171(1-2), 115–166. https://doi.org/10.1007/s10107-017-1172-1
   Web of Science ®Google Scholar
• Mohammadi, M., Atashin, A. A., & Tamburri, D. A. (2023). From ℓ1-subgradient to projection: A compact neural network for ℓ1-regularized logistic regression. Neurocomputing, 526, 30–38. https://doi.org/10.1016/j.neucom.2023.01.021
   Web of Science ®Google Scholar
• Namkoong, H., & Duchi, J. C. (2016). Stochastic gradient methods for distributionally robust optimization with f-divergences. Advances in Neural Information Processing Systems, 2208–2216.
   Google Scholar
• Nesterov, Y. (2003). Introductory lectures on convex optimization: A basic course (Vol. 87). Springer Science & Business Media.
   Google Scholar
• Nesterov, Y. E. (1983). A method for solving the convex programming problem with convergence rate O(1/k2). Doklady Akademii Nauk, 269(3), 543–547.
   Web of Science ®Google Scholar
• Polyak, B. T. (1964). Some methods of speeding up the convergence of iteration methods. Doklady Akademii Nauk, 4(5), 1–17.
   Google Scholar
• Rockafellar, R. T. (1976). Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Mathematics of Operations Research, 1(2), 97–116. https://doi.org/10.1287/moor.1.2.97
   Google Scholar
• Shafieezadeh Abadeh, S., Mohajerin Esfahani, P. M., & Kuhn, D. (2015). Distributionally robust logistic regression. Advances in Neural Information Processing Systems, 1576–1584.
   Google Scholar
• Singer, Y., & Duchi, J. C. (2009). Efficient learning using forward-backward splitting. Advances in Neural Information Processing Systems, 493–503.
   Google Scholar
• Ugrinovskii, V., Petersen, I. R., & Shames, I. (2023). A robust control approach to asymptotic optimality of the heavy ball method for optimization of quadratic functions. Automatica, 155, 111129. https://doi.org/10.1016/j.automatica.2023.111129
   Web of Science ®Google Scholar
• Wächter, A., & Biegler, L. T. (2006). On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical Programming, 106(1), 25–57. https://doi.org/10.1007/s10107-004-0559-y
   Web of Science ®Google Scholar
• Wang, J. Y., Wang, H. M., Nie, F. P., & Li, X. L. (2023). Feature selection with multi-class logistic regression. Neurocomputing, 543, 126268. https://doi.org/10.1016/j.neucom.2023.126268
   Web of Science ®Google Scholar
• Wiesemann, W., Kuhn, D., & Sim, M. (2014). Forward-backward quasi-Newton methods for non-smooth optimization problems. Operations Research, 62(6), 1358–1376. https://doi.org/10.1287/opre.2014.1314
   Web of Science ®Google Scholar
• Wu, Y., & Xiong, S. F. (2023). Kernel reconstruction learning. Neurocomputing, 522, 1–10. https://doi.org/10.1016/j.neucom.2022.12.015
   Web of Science ®Google Scholar
• Yuan, Y. F., Song, Q. K., & Zhou, B. (2022). A Wasserstein distributionally robust chance constrained programming approach for emergency medical system planning problem. International Journal of Systems Science, 53(10), 2136–2148. https://doi.org/10.1080/00207721.2022.2040641
   Web of Science ®Google Scholar
• Yuan, Y. F., Song, Q. K., & Zhou, B. (2023). Emergency medical service location problem based on physical bounds using chance constrained programming approach. International Journal of Systems Science, 54(3), 704–716. https://doi.org/10.1080/00207721.2022.2141593
   Web of Science ®Google Scholar
• Yuan, Y. F., Song, Q. K., & Zhou, B. (2023). A multi-period emergency medical service location problem based on Wasserstein-metric approach using generalised benders decomposition method. International Journal of Systems Science, 54(6), 1173–1185. https://doi.org/10.1080/00207721.2023.2168144
   Web of Science ®Google Scholar
• Zhang, Y. H., Zou, L., Liu, Y., Ding, D. R., & Hu, J. (2023). A brief survey on nonlinear control using adaptive dynamic programming under engineering-oriented complexities. International Journal of Systems Science, 54(8), 1855–1872. https://doi.org/10.1080/00207721.2023.2209846
   Web of Science ®Google Scholar
• Zhou, B., Chen, G., Huang, T. W., Song, Q. K, & Yuan, Y. F. (2020a). Planning PEV fast-charging stations using data-driven distributionally robust optimization approach based on Φ-divergence. IEEE Transactions on Transportation Electrification, 6(1), 170–180. https://doi.org/10.1109/TTE.2020.2971825
   Web of Science ®Google Scholar
• Zhou, B., Chen, G., Song, Q. K, & Dong, Z. Y. (2020b). Robust chance-constrained programming approach for the planning of fast-charging stations in electrified transportation networks. Applied Energy, 262, 114480. https://doi.org/10.1016/j.apenergy.2019.114480
   Web of Science ®Google Scholar