Computation of second-order directional stationary points for group sparse optimization

References

• M. Ahn, J.-S. Pang, and J. Xin, Difference-of-convex learning: Directional stationarity, optimality, and sparsity, SIAM J. Optim. 27 (2017), pp. 1637–1665. doi: 10.1137/16M1084754
   Web of Science ®Google Scholar
• A. Ben-Tal and J. Zowe, Necessary and sufficient optimality conditions for a class of nonsmooth minimization problems, Math. Program. 24 (1982), pp. 70–91. doi: 10.1007/BF01585095
   Web of Science ®Google Scholar
• W. Bian and X. Chen, Optimality and complexity for constrained optimization problems with nonconvex regularization, Math. Oper. Res. 42 (2017), pp. 1063–1084. doi: 10.1287/moor.2016.0837
   Web of Science ®Google Scholar
• W. Bian, X. Chen, and Y. Ye, Complexity analysis of interior point algorithms for non-Lipschitz and nonconvex minimization, Math. Program. 149 (2015), pp. 301–327. doi: 10.1007/s10107-014-0753-5
   Web of Science ®Google Scholar
• P. Breheny and J. Huang, Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors, Stat. Comput. 25 (2015), pp. 173–187. doi: 10.1007/s11222-013-9424-2
   PubMed Web of Science ®Google Scholar
• A.M. Bruckstein, D.L. Donoho, and M. Elad, From sparse solutions of systems of equations to sparse modeling of signals and images, SIAM Rev. 51 (2009), pp. 34–81. doi: 10.1137/060657704
   Web of Science ®Google Scholar
• T.H. Chang, M. Hong, and J.-S. Pang, Local minimizers and second-order conditions in composite piecewise programming via directional derivatives, (2017). Available at arxiv.org/abs/1709.05758v1.
   Google Scholar
• X. Chen, Smoothing methods for nonsmooth, novonvex minimization, Math. Program. 134 (2012), pp. 71–99. doi: 10.1007/s10107-012-0569-0
   Web of Science ®Google Scholar
• X. Chen, F. Xu, and Y. Ye, Lower bound theory of nonzero entries in solutions of ℓ2−ℓp minimization, SIAM J. Sci. Comput. 32 (2010), pp. 2832–2852. doi: 10.1137/090761471
   Web of Science ®Google Scholar
• X. Chen, L. Niu, and Y. Yuan, Optimality conditions and a smoothing trust region Newton method for non-Lipschitz optimization, SIAM J. Optim. 23 (2013), pp. 1528–1552. doi: 10.1137/120871390
   Web of Science ®Google Scholar
• M. Eren Ahsen and M. Vidyasagar, Error bounds for compressed sensing algorithms with group sparsity: A unified approach, Appl. Comput. Harmon. Anal. 43 (2017), pp. 212–232. doi: 10.1016/j.acha.2015.11.006
   Web of Science ®Google Scholar
• J. Fan and R. Li, Variable selection via nonconcave penalized likelihood and its oracle properties, J. Amer. Stat. Assoc. 96 (2001), pp. 1348–1360. doi: 10.1198/016214501753382273
   Web of Science ®Google Scholar
• J. Fan and R. Li, Statistical challenges with high dimensionality: Feature selection in knowledge discovery, Proceedings of the International Congress of Mathematicians, Vol. 3, Madrid, Spain, 2006, pp. 595–622.
   Google Scholar
• J. Fan, L. Xue, and H. Zou, Strong oracle optimality of folded concave penalized estimation, Ann. Stat. 42 (2014), pp. 819–849. doi: 10.1214/13-AOS1198
   PubMed Web of Science ®Google Scholar
• Y. Hu, C. Li, K. Meng, J. Qin, and X. Yang, Group sparse optimization via ℓp,q regularization, J. Mach. Learn. Res. 18 (2017), pp. 1–52.
   Web of Science ®Google Scholar
• J. Huang and T. Zhang, The benefit of group sparsity, Ann. Stat. 38 (2010), pp. 1978–2004. doi: 10.1214/09-AOS778
   Web of Science ®Google Scholar
• J. Huang, S. Ma, H. Xue, and C.H. Zhang, A group bridge approach for variable selection, Biometrika 96 (2009), pp. 339–355. doi: 10.1093/biomet/asp020
   PubMed Web of Science ®Google Scholar
• L. Jacob, G. Obozinski, and J.P. Vert, Group lasso with overlap and graph lasso, Proceedings of the 26th International Conference on Machine Learning, Montreal, Canada, 2009, pp. 433–440.
   Google Scholar
• Y. Jiao, B. Jin, and X. Lu, Group sparse recovery via the ℓ0(ℓ2) penalty: Theory and algorithm, IEEE Trans. Signal Process. 65 (2017), pp. 998–1012. doi: 10.1109/TSP.2016.2630028
   Google Scholar
• K. Knight and W.J. Fu, Asymptotics for lasso-type estimators, Ann. Stat. 28 (2000), pp. 1356–1378. doi: 10.1214/aos/1015957397
   Web of Science ®Google Scholar
• H.A. Le Thi, T. Pham Dinh, H.M. Le, and X.T. Vo, DC approximation approaches for sparse optimization, Eur. J. Oper. Res. 244 (2015), pp. 26–46. doi: 10.1016/j.ejor.2014.11.031
   Web of Science ®Google Scholar
• S. Lee, M. Oh, and Y. Kim, Sparse optimization for nonconvex group penalized estimation, J. Stat. Comput. Simul. 86 (2016), pp. 597–610. doi: 10.1080/00949655.2015.1026822
   Web of Science ®Google Scholar
• H. Liu, T. Yao, R. Li, and Y. Ye, Folded concave penalized sparse linear regression: Sparsity, statistical performance, and algorithmic theory for local solutions, Math. Program. 166 (2017), pp. 207–240. doi: 10.1007/s10107-017-1114-y
   PubMed Web of Science ®Google Scholar
• L. Meier, S. Van De Geer, and P. Bühlmann, The group Lasso for logistic regression, J. R. Stat. Soc. B 70 (2008), pp. 53–71. doi: 10.1111/j.1467-9868.2007.00627.x
   Google Scholar
• M. Nikolova, M.K. Ng, S. Zhang, and W. Ching, Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization, SIAM J. Imag. Sci. 1 (2008), pp. 2–25. doi: 10.1137/070692285
   Web of Science ®Google Scholar
• J.-S. Pang, M. Razaviyayn, and A. Alvarado, Computing B-stationary points of nonsmooth DC programs, Math. Oper. Res. 42 (2017), pp. 95–118. doi: 10.1287/moor.2016.0795
   Web of Science ®Google Scholar
• R.T. Rochafellar and R.J.-B. Wets, Variational Analysis, 3rd ed., Springer-Verlag, Berlin, 2009.
   Google Scholar
• R. Tibshirani, Regression shrinkage and selection via the LASSO, J. R. Stat. Soc. B 58 (1996), pp. 267–288.
   Google Scholar
• F. Wei and H. Zhu, Group coordinate descent algorithms for nonconvex penalized regression, Comput. Stat. Data Anal. 56 (2012), pp. 316–326. doi: 10.1016/j.csda.2011.08.007
   Web of Science ®Google Scholar
• X.Q. Yang, On second-order directional derivatives, Nonlin. Anal. TMA 26 (1996), pp. 55–66. doi: 10.1016/0362-546X(94)00209-Z
   Web of Science ®Google Scholar
• E. Yang and A.C. Lozano, Sparse + group-sparse dirty models: Statistical guarantees without unreasonable conditions and a case for non-convexity, Proceedings of the 34th International Conference on Machine Learning, PMLR 70, Sydney, Australia, 2017, pp. 3911–3920.
   Google Scholar
• Y. Yang and H. Zou, A fast unified algorithm for solving group-lasso penalize learning problems, Stat. Comput. 25 (2015), pp. 1129–1141. doi: 10.1007/s11222-014-9498-5
   Web of Science ®Google Scholar
• Y. Yuan, Conditions for convergence of trust region algorithms for nonsmooth optimization, Math. Program. 31 (1985), pp. 220–228. doi: 10.1007/BF02591750
   Web of Science ®Google Scholar
• M. Yuan and Y. Lin, Model selection and estimation in regression with grouped variables, J. R. Stat. Soc. B 68 (2006), pp. 49–67. doi: 10.1111/j.1467-9868.2005.00532.x
   Google Scholar
• C.H. Zhang, Nearly unbiased variable selection under minimax concave penalty, Ann. Stat. 38 (2010), pp. 894–942. doi: 10.1214/09-AOS729
   Web of Science ®Google Scholar
• P. Zhao, G. Rocha, and B. Yu, The composite absolute penalties family for grouped and hierarchical variable selection, Ann. Stat. 37 (2009), pp. 3468–3497. doi: 10.1214/07-AOS584
   Web of Science ®Google Scholar
• T. Zhang, Analysis of multi-stage convex relaxation for sparse regularization, J. Mach. Learn. Res. 11 (2010), pp. 1081–1107.
   Web of Science ®Google Scholar