Robust Variable and Interaction Selection for Logistic Regression and General Index Models

References

• Anderson, T. W. (1958), An Introduction to Multivariate Statistical Analysis, New York: Wiley.
   Google Scholar
• Beer, D. G., Kardia, S. L., Huang, C.-C., Giordano, T. J., Levin, A. M., Misek, D. E., Lin, L., Chen, G., Gharib, T. G., Thomas, D. G., Lizyness, M. L., Kuick, R., Hayasaka, S., Taylor, J. M. G., Iannettoni, M. D., Orringer, M. B., and Hanash, S. (2002), “Gene-Expression Profiles Predict Survival of Patients With Lung Adenocarcinoma,” Nature Medicine, 8, 816–824.
   PubMed Web of Science ®Google Scholar
• Bien, J., Taylor, J., and Tibshirani, R. (2013), “A Lasso for Hierarchical Interactions,” The Annals of Statistics, 41, 1111–1141.
   PubMed Web of Science ®Google Scholar
• Boser, B. E., Guyon, I. M., and Vapnik, V. N. (1992), “A Training Algorithm for Optimal Margin Classifiers,” in Proceedings of the Fifth Annual Workshop on Computational Learning Theory, ACM, pp. 144–152.
   Google Scholar
• Breiman, L. (2001), “Random Forests,” Machine Learning, 45, 5–32.
   Web of Science ®Google Scholar
• Broman, K. W., and Speed, T. P. (2002), “A Model Selection Approach for the Identification of Quantitative Trait Loci in Experimental Crosses,” Journal of the Royal Statistical Society, Series B, 64, 641–656.
   Google Scholar
• Cai, T., and Liu, W. (2011), “A Direct Estimation Approach to Sparse Linear Discriminant Analysis,” Journal of the American Statistical Association, 106, 1566–1577.
   Web of Science ®Google Scholar
• Chen, J., and Chen, Z. (2008), “Extended Bayesian Information Criteria for Model Selection With Large Model Spaces,” Biometrika, 95, 759–771.
   Web of Science ®Google Scholar
• ——— (2012), “Extended BIC for Small-n-Large-P Sparse GLM,” Statistica Sinica, 22, 555–574.
   Web of Science ®Google Scholar
• Chowdhary, R., Zhang, J., and Liu, J. S. (2009), “Bayesian Inference of Protein–Protein Interactions From Biological Literature,” Bioinformatics, 25, 1536–1542.
   PubMed Web of Science ®Google Scholar
• Clemmensen, L., Hastie, T., Witten, D., and Ersbøll, B. (2011), “Sparse Discriminant Analysis,” Technometrics, 53, 406–413.
   Web of Science ®Google Scholar
• Cook, R. D. (2007), “Fisher Lecture: Dimension Reduction in Regression,” Statistical Science, 22, 1–26.
   Web of Science ®Google Scholar
• Efron, B. (2009), “Empirical Bayes Estimates for Large-Scale Prediction Problems,” Journal of the American Statistical Association, 104, 1015–1028.
   PubMed Web of Science ®Google Scholar
• ——— (2010), Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction (Vol. 1), Cambridge: Cambridge University Press.
   Google Scholar
• Fan, J., and Fan, Y. (2008), “High Dimensional Classification Using Features Annealed Independence Rules,” Annals of Statistics, 36, 2605–2637.
   PubMed Web of Science ®Google Scholar
• Fan, Y., Jin, J., and Yao, Z. (2013), “Optimal Classification in Sparse Gaussian Graphic Model,” The Annals of Statistics, 41, 2537–2571.
   Web of Science ®Google Scholar
• Fan, Y., Kong, Y., Li, D., and Zheng, Z. (2015), “Innovated Interaction Screening for High-Dimensional Nonlinear Classification,” The Annals of Statistics, 43, 1243–1272.
   Web of Science ®Google Scholar
• Foygel, R., and Drton, M. (2011), “Bayesian Model Choice and Information Criteria in Sparse Generalized Linear Models,” arXiv preprint arXiv:1112.5635.
   Google Scholar
• Guo, Y., Hastie, T., and Tibshirani, R. (2007), “Regularized Linear Discriminant Analysis and Its Application in Microarrays,” Biostatistics, 8, 86–100.
   PubMed Web of Science ®Google Scholar
• Han, F., Zhao, T., and Liu, H. (2013), “Coda: High Dimensional Copula Discriminant Analysis,” The Journal of Machine Learning Research, 14, 629–671.
   Web of Science ®Google Scholar
• Jia, J., and Yu, B. (2010), “On Model Selection Consistency of the Elastic Net When p≫n,” Statistica Sinica, 20, 595–611.
   Web of Science ®Google Scholar
• Jiang, B., and Liu, J. S. (2014), “Variable Selection for General Index Models via Sliced Inverse Regression,” The Annals of Statistics, 42, 1751–1786.
   Web of Science ®Google Scholar
• Joachims, T. (1998), Text Categorization With Support Vector Machines: Learning With Many Relevant Features, New York: Springer.
   Google Scholar
• Li, K.-C. (1991), “Sliced Inverse Regression for Dimension Reduction,” Journal of the American Statistical Association, 86, 316–327.
   Web of Science ®Google Scholar
• Li, L. (2007), “Sparse Sufficient Dimension Reduction,” Biometrika, 94, 603–613.
   Web of Science ®Google Scholar
• Li, L., Dennis Cook, R., and Nachtsheim, C. J. (2005), “Model-Free Variable Selection,” Journal of the Royal Statistical Society, Series B, 67, 285–299.
   Google Scholar
• Li, R., Zhong, W., and Zhu, L. (2012), “Feature Screening via Distance Correlation Learning,” Journal of the American Statistical Association, 107, 1129–1139.
   PubMed Web of Science ®Google Scholar
• Lin, Q., Zhao, Z., and Liu, J. S. (2018a), “On Consistency and Sparsity for Sliced Inverse Regression in High Dimensions,” Annals of Statistics, 46, 580–610.
   Web of Science ®Google Scholar
• ——— (2018b), “Sparse Sliced Inverse Regression for High Dimensional Data,” Journal of the American Statistical Association, under revision.
   Web of Science ®Google Scholar
• Mai, Q., Zou, H., and Yuan, M. (2012), “A Direct Approach to Sparse Discriminant Analysis in Ultra-High Dimensions,” Biometrika, 99, 29–42.
   Web of Science ®Google Scholar
• Maugis, C., Celeux, G., and Martin-Magniette, M.-L. (2011), “Variable Selection in Model-Based Discriminant Analysis,” Journal of Multivariate Analysis, 102, 1374–1387.
   Web of Science ®Google Scholar
• Murphy, T. B., Dean, N., and Raftery, A. E. (2010), “Variable Selection and Updating in Model-Based Discriminant Analysis for High Dimensional Data With Food Authenticity Applications,” The Annals of Applied Statistics, 4, 396–421.
   PubMed Web of Science ®Google Scholar
• Phillips, P. J. (1998), Support Vector Machines Applied to Face Recognition (Vol. 285), Gaithersburg, MD: NIST.
   Google Scholar
• Ravikumar, P., Wainwright, M. J., and Lafferty, J. D. (2010), “High-Dimensional Ising Model Selection Using l1-Regularized Logistic Regression,” The Annals of Statistics, 38, 1287–1319.
   Web of Science ®Google Scholar
• Schwarz, G. (1978), “Estimating the Dimension of a Model,” The Annals of Statistics, 6, 461–464.
   Web of Science ®Google Scholar
• Shao, J., Wang, Y., Deng, X., and Wang, S. (2011), “Sparse Linear Discriminant Analysis by Thresholding for High Dimensional Data,” The Annals of Statistics, 39, 1241–1265.
   Web of Science ®Google Scholar
• Simon, N., and Tibshirani, R. (2016), “A Permutation Approach to Testing Interactions in Many Dimensions,” Journal of the American Statistical Association, 110, 1707–1716.
   Web of Science ®Google Scholar
• Singh, D., Febbo, P. G., Ross, K., Jackson, D. G., Manola, J., Ladd, C., Tamayo, P., Renshaw, A. A., D’Amico, A. V., Richie, J. P., Lander, E. S., Loda, M., Kantoff, P. W., Golub, T. R., and Sellers, W. R. (2002), “Gene Expression Correlates of Clinical Prostate Cancer Behavior,” Cancer Cell, 1, 203–209.
   PubMed Web of Science ®Google Scholar
• Szretter, M. E., and Yohai, V. J. (2009), “The Sliced Inverse Regression Algorithm as a Maximum Likelihood Procedure,” Journal of Statistical Planning and Inference, 139, 3570–3578.
   Web of Science ®Google Scholar
• Tibshirani, R. (1996), “Regression Shrinkage and Selection via the Lasso,” Journal of the Royal Statistical Society, Series B, 58, 267–288.
   Google Scholar
• Tibshirani, R., Hastie, T., Narasimhan, B., and Chu, G. (2002), “Diagnosis of Multiple Cancer Types by Shrunken Centroids of Gene Expression,” Proceedings of the National Academy of Sciences, 99, 6567–6572.
   PubMed Web of Science ®Google Scholar
• Vershynin, R. (2012), “Introduction to the Non-Asymptotic Analysis of Random Matrices,” in Compressed Sensing: Theory and Applications, eds. Y. Eldar and G. Kutyniok, Cambridge: Cambridge University Press, pp. 210–268.
   Google Scholar
• Waldmann, P., Mészáros, G., Gredler, B., Fuerst, C., and Sölkner, J. (2013), “Evaluation of the Lasso and the Elastic Net in Genome-Wide Association Studies,” Frontiers in Genetics, 4, 270.
   PubMedGoogle Scholar
• Wang, H. (2009), “Forward Regression for Ultra-High Dimensional Variable Screening,” Journal of the American Statistical Association, 104, 1512–1524.
   Web of Science ®Google Scholar
• Wasserman, L., and Roeder, K. (2009), “High Dimensional Variable Selection,” Annals of Statistics, 37, 2178–2201.
   PubMed Web of Science ®Google Scholar
• Witten, D. M., and Tibshirani, R. (2011), “Penalized Classification Using Fisher’s Linear Discriminant,” Journal of the Royal Statistical Society, Series B, 73, 753–772.
   Google Scholar
• Yu, Y., and Feng, Y. (2014), “Modified Cross-Validation for Penalized High-Dimensional Linear Regression Models,” Journal of Computational and Graphical Statistics, 23, 1009–1027.
   Web of Science ®Google Scholar
• Zhang, Q., and Wang, H. (2011), “On BIC’s Selection Consistency for Discriminant Analysis,” Statistica Sinica, 21, 731–740.
   Web of Science ®Google Scholar
• Zhao, P., and Yu, B. (2006), “On Model Selection Consistency of Lasso,” The Journal of Machine Learning Research, 7, 2541–2563.
   Web of Science ®Google Scholar
• Zhong, W., Zhang, T., Zhu, Y., and Liu, J. S. (2012), “Correlation Pursuit: Forward Stepwise Variable Selection for Index Models,” Journal of the Royal Statistical Society, Series B, 74, 849–870.
   Google Scholar
• Zou, H., and Hastie, T. (2005), “Regularization and Variable Selection via the Elastic Net,” Journal of the Royal Statistical Society, Series B, 67, 301–320.
   Google Scholar