An Interactive Greedy Approach to Group Sparsity in High Dimensions

References

• Beck, A. and Teboulle, M. (2009), ‘A fast iterative shrinkage-thresholding algorithm for linear inverse problems’, SIAM Journal on Imaging Sciences 2(1), 183–202.
   Google Scholar
• Ben-Haim, Z. and Eldar, Y. C. (2011), ‘Near-oracle performance of greedy block-sparse estimation techniques from noisy measurements’, IEEE Journal of Selected Topics in Signal Processing 5(5), 1032–1047.
   Google Scholar
• Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009), ‘Simultaneous analysis of lasso and Dantzig selector’, The Annals of Statistics 37(4), 1705–1732.
   Google Scholar
• Candes, E. J. and Tao, T. (2005), ‘Decoding by linear programming’, IEEE Transactions on Information Theory 51(12), 4203–4215.
   Google Scholar
• Candes, E. J. and Tao, T. (2007), ‘The Dantzig selector: Statistical estimation when p is much larger than n’, The Annals of Statistics 35(6), 2313–2351.
   Google Scholar
• Culp, M. V., Ryan, K. J., Banerjee, P. and Morehead, M. (2018), ‘On data integration problems with manifolds’, Technometrics, to appear.
   Google Scholar
• Ding, S. and Cook, R. D. (2018), ‘Matrix variate regressions and envelope models’, Journal of the Royal Statistical Society: Series B (Statistical Methodology) 80(2), 387–408.
   Google Scholar
• Fan, J. and Li, R. (2001), ‘Variable selection via nonconcave penalized likelihood and its oracle properties’, Journal of the American Statistical Association 96(456), 1348–1360.
   Google Scholar
• Fan, J., Xue, L. and Zou, H. (2014), ‘Strong oracle optimality of folded concave penalized estimation’, Annals of Statistics 42(3), 819.
   Google Scholar
• Friedman, J., Hastie, T. and Tibshirani, R. (2010), ‘Regularization paths for generalized linear models via coordinate descent’, Journal of Statistical Software 33(1), 1.
   Google Scholar
• Gu, Y., Fan, J., Kong, L., Ma, S. and Zou, H. (2018), ‘ADMM for high-dimensional sparse penalized quantile regression’, Technometrics 60(3), 319–331.
   Google Scholar
• Hsu, D., Kakade, S. and Zhang, T. (2012), ‘A tail inequality for quadratic forms of subgaussian random vectors’, Electronic Communications in Probability 17(52), 1–6.
   Google Scholar
• Huang, J., Breheny, P. and Ma, S. (2012), ‘A selective review of group selection in high-dimensional models’, Statistical Science 27(4).
   Google Scholar
• Huang, J. and Zhang, T. (2010), ‘The benefit of group sparsity’, The Annals of Statistics 38(4), 1978–2004.
   Google Scholar
• Huang, Y. and Liu, J. (2018), ‘Exclusive sparsity norm minimization with random groups via cone projection’, IEEE Transactions on Neural Networks and Learning Systems.
   Google Scholar
• Ing, C.-K. and Lai, T. L. (2011), ‘A stepwise regression method and consistent model selection for high-dimensional sparse linear models’, Statistica Sinica 21(4), 1473–1513.
   Google Scholar
• Jenatton, R., Gramfort, A., Michel, V., Obozinski, G., Eger, E., Bach, F. and Thirion, B. (2012), ‘Multiscale mining of fmri data with hierarchical structured sparsity’, SIAM Journal on Imaging Sciences 5(3), 835–856.
   Google Scholar
• Jiao, Y., Jin, B. and Lu, X. (2017), ‘Group sparse recovery via the l0 (l2) penalty: Theory and algorithm’, IEEE Transactions on Signal Processing 65(4), 998–1012.
   Google Scholar
• Kim, Y., Kim, J. and Kim, Y. (2006), ‘Blockwise sparse regression’, Statistica Sinica 16(2), 375.
   Google Scholar
• Kong, D., Fujimaki, R., Liu, J., Nie, F. and Ding, C. (2014), ‘Exclusive feature learning on arbitrary structures via l12-norm’, Advances in Neural Information Processing Systems pp. 1655–1663.
   Google Scholar
• Lafferty, J., McCallum, A., Pereira, F. et al. (2001), Conditional random fields: Probabilistic models for segmenting and labeling sequence data, in ‘International Conference on Machine Learning’, pp. 282–289.
   Google Scholar
• Liu, J., Fujimaki, R. and Ye, J. (2013), ‘Forward-backward greedy algorithms for general convex smooth functions over a cardinality constraint’, International Conference on Machine Learning.
   Google Scholar
• Liu, J., Wonka, P. and Ye, J. (2012), ‘A multi-stage framework for Dantzig selector and lasso’, Journal of Machine Learning Research 13(1), 1189–1219.
   Google Scholar
• Lounici, K., Pontil, M., Van De Geer, S. and Tsybakov, A. B. (2011), ‘Oracle inequalities and optimal inference under group sparsity’, The Annals of Statistics 39(4), 2164–2204.
   Google Scholar
• Lozano, A. C., Swirszcz, G. and Abe, N. (2011), Group orthogonal matching pursuit for logistic regression, in ‘International Conference on Artificial Intelligence and Statistics’, pp. 452–460.
   Google Scholar
• Mallat, S. G. and Zhang, Z. (1993), ‘Matching pursuits with time-frequency dictionaries’, IEEE Transactions on Signal Processing 41(12), 3397–3415.
   Google Scholar
• McCullagh, P. and Nelder, J. A. (1989), Generalized Linear Models, Chapman and Hall.
   Google Scholar
• Meier, L., Van De Geer, S. and Bühlmann, P. (2008), ‘The group lasso for logistic regression’, Journal of the Royal Statistical Society, Series B 70(1), 53–71.
   Google Scholar
• Mitra, R., Zhang, C.-H. et al. (2016), ‘The benefit of group sparsity in group inference with de-biased scaled group lasso’, Electronic Journal of Statistics 10(2), 1829–1873.
   Google Scholar
• Nardi, Y. and Rinaldo, A. (2008), ‘On the asymptotic properties of the group lasso estimator for linear models’, Electronic Journal of Statistics 2, 605–633.
   Google Scholar
• Qian, W., Ding, S. and Cook, R. D. (2018), ‘Sparse minimum discrepancy approach to sufficient dimension reduction with simultaneous variable selection in ultrahigh dimension’, Journal of the American Statistical Association, to appear.
   Google Scholar
• Qian, W. and Yang, Y. (2013), ‘Model selection via standard error adjusted adaptive lasso’, Annals of the Institute of Statistical Mathematics 65(2), 295–318.
   Google Scholar
• Qian, W. and Yang, Y. (2016), ‘Kernel estimation and model combination in a bandit problem with covariates’, Journal of Machine Learning Research 17(1), 5181–5217.
   Google Scholar
• Qian, W., Yang, Y. and Zou, H. (2016), ‘Tweedie's compound Poisson model with grouped elastic net’, Journal of Computational and Graphical Statistics 25(2), 606–625.
   Google Scholar
• Sutton, C., McCallum, A. et al. (2012), ‘An introduction to conditional random fields’, Foundations and Trends[textregistered] in Machine Learning 4(4), 267–373.
   Google Scholar
• Swirszcz, G., Abe, N. and Lozano, A. C. (2009), Grouped orthogonal matching pursuit for variable selection and prediction, in ‘Advances in Neural Information Processing Systems’, pp. 1150–1158.
   Google Scholar
• Tibshirani, R. (1996), ‘Regression shrinkage and selection via the lasso’, Journal of the Royal Statistical Society: Series B (Methodological) 58(1), 267–288.
   Google Scholar
• Tropp, J. A. (2004), ‘Greed is good: Algorithmic results for sparse approximation’, IEEE Transactions on Information Theory 50(10), 2231–2242.
   Google Scholar
• van de Geer, S., Bühlmann, P. and Zhou, S. (2011), ‘The adaptive and the thresholded lasso for potentially misspecified models (and a lower bound for the lasso)’, Electronic Journal of Statistics 5, 688–749.
   Google Scholar
• Vershynin, R. (2010), ‘Introduction to the non-asymptotic analysis of random matrices’, arXiv preprint:1011.3027.
   Google Scholar
• Wei, F. and Huang, J. (2010), ‘Consistent group selection in high-dimensional linear regression’, Bernoulli 16(4), 1369.
   Google Scholar
• Yang, H., Huang, Y., Tran, L., Liu, J. and Huang, S. (2016), ‘On benefits of selection diversity via bilevel exclusive sparsity’, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 5945–5954.
   Google Scholar
• Yuan, M. and Lin, Y. (2006), ‘Model selection and estimation in regression with grouped variables’, Journal of the Royal Statistical Society: Series B (Statistical Methodology) 68(1), 49–67.
   Google Scholar
• Zhang, C.-H. et al. (2010), ‘Nearly unbiased variable selection under minimax concave penalty’, The Annals of Statistics 38(2), 894–942.
   Google Scholar
• Zhang, T. (2009), ‘On the consistency of feature selection using greedy least squares regression’, Journal of Machine Learning Research 10, 555–568.
   Google Scholar
• Zhang, T. (2011 a), ‘Adaptive forward-backward greedy algorithm for learning sparse representations’, IEEE Transactions on Information Theory 57(7), 4689–4708.
   Google Scholar
• Zhang, T. (2011 b), ‘Sparse recovery with orthogonal matching pursuit under RIP’, IEEE Transactions on Information Theory 57(9), 6215–6221.
   Google Scholar
• Zhao, T., Liu, H. and Zhang, T. (2017), ‘Pathwise coordinate optimization for sparse learning: algorithm and theory’, The Annals of Statistics 46(1), 180–218.
   Google Scholar
• Zhou, H., Sehl, M. E., Sinsheimer, J. S. and Lange, K. (2010), ‘Association screening of common and rare genetic variants by penalized regression’, Bioinformatics 26(19), 2375–2382.
   Google Scholar
• Zou, H. (2006), ‘The adaptive lasso and its oracle properties’, Journal of the American Statistical Association 101(476), 1418–1429.
   Google Scholar
• Zou, H. and Zhang, H. H. (2009), ‘On the adaptive elastic-net with a diverging number of parameters’, The Annals of Statistics 37(4), 1733.
   Google Scholar