A New Basis for Sparse Principal Component Analysis

References

• Amini, A. A., and Wainwright, M. J. (2009), “High-Dimensional Analysis of Semidefinite Relaxations for Sparse Principal Components,” The Annals of Statistics, 37, 2877–2921. DOI: 10.1214/08-AOS664.
   Web of Science ®Google Scholar
• Baglama, J., and Reichel, L. (2005), “Augmented Implicitly Restarted Lanczos Bidiagonalization Methods,” SIAM Journal on Scientific Computing, 27, 19–42. DOI: 10.1137/04060593X.
   Web of Science ®Google Scholar
• Baron, M., Veres, A., Wolock, S. L., Faust, A. L., Gaujoux, R., Vetere, A., Ryu, J. H., Wagner, B. K., Shen-Orr, S. S., Klein, A. M., et al. (2016), “A Single-Cell Transcriptomic Map of the Human and Mouse Pancreas Reveals Inter- and Intra-Cell Population Structure,” Cell Systems, 3, 346–360. DOI: 10.1016/j.cels.2016.08.011.
   PubMed Web of Science ®Google Scholar
• Bell, A. J., and Sejnowski, T. J. (1997), “The “Independent Components” of Natural Scenes are Edge Filters,” Vision Research, 37, 3327–3338. DOI: 10.1016/s0042-6989(97)00121-1.
   PubMed Web of Science ®Google Scholar
• Bernaards, C. A., and Jennrich, R. I. (2005), “Gradient Projection Algorithms and Software for Arbitrary Rotation Criteria in Factor Analysis,” Educational and Psychological Measurement, 65, 676–696. DOI: 10.1177/0013164404272507.
   Web of Science ®Google Scholar
• Berthet, Q., and Rigollet, P. (2013), “Optimal Detection of Sparse Principal Components in High Dimension,” The Annals of Statistics, 41, 1780–1815. DOI: 10.1214/13-AOS1127.
   Web of Science ®Google Scholar
• Cai, T. T., Ma, Z., and Wu, Y. (2013), “Sparse PCA: Optimal Rates and Adaptive Estimation,” The Annals of Statistics, 41, 3074–3110. DOI: 10.1214/13-AOS1178.
   Web of Science ®Google Scholar
• Carroll, J. B. (1953), “An Analytical Solution for Approximating Simple Structure in Factor Analysis,” Psychometrika, 18, 23–38. DOI: 10.1007/BF02289025.
   Google Scholar
• Chen, F., Zhang, Y., and Rohe, K. (2020), “Targeted Sampling from Massive Block Model Graphs with Personalized PageRank,” Journal of the Royal Statistical Society, Series B, 82, 99–126. DOI: 10.1111/rssb.12349.
   Google Scholar
• Chen, F., Roch, S., Rohe, K., and Yu, S. (2021), “Estimating Graph Dimension with Cross-Validated Eigenvalues,” arXiv preprint arXiv:2108.03336.
   Google Scholar
• Comon, P. (1994), “Independent Component Analysis, A New Concept?” Signal Processing, 36, 287–314. DOI: 10.1016/0165-1684(94)90029-9.
   Web of Science ®Google Scholar
• d’Aspremont, A., Ghaoui, L. E., Jordan, M. I., and Lanckriet, G. R. G. (2007), “A Direct Formulation for Sparse PCA Using Semidefinite Programming,” SIAM Review, 49, 434–448. DOI: 10.1137/050645506.
   Web of Science ®Google Scholar
• Donoho, D. L. (1995), “De-Noising by Soft-Thresholding,” IEEE Transactions on Information Theory, 41, 613–627. DOI: 10.1109/18.382009.
   Web of Science ®Google Scholar
• Field, D. J. (1987), “Relations between the Statistics of Natural Images and the Response Properties of Cortical Cells,” Journal of the Optical Society of America A, 4, 2379–2394. DOI: 10.1364/josaa.4.002379.
   PubMedGoogle Scholar
• Field, D. J. (1994), “What is the Goal of Sensory Coding?” Neural Computation, 6, 559–601. DOI: 10.1162/neco.1994.6.4.559.
   Web of Science ®Google Scholar
• Fortunato, S. (2010), “Community Detection in Graphs,” Physics Reports, 486, 75–174. DOI: 10.1016/j.physrep.2009.11.002.
   Web of Science ®Google Scholar
• Gallivan, K. A., and Absil, P. A. (2010), “Note on the Convex Hull of the Stiefel Manifold,” Florida State University.
   Google Scholar
• Gataric, M., Wang, T., and Samworth, R. J. (2020), “Sparse Principal Component Analysis via Axis-Aligned Random Projections,” Journal of the Royal Statistical Society, Series B, 82, 329–359. DOI: 10.1111/rssb.12360.
   Google Scholar
• Gregor, K., and LeCun, Y. (2010), “Learning Fast Approximations of Sparse Coding,” in Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML’10, pp. 399–406, Madison, WI, USA, Omnipress.
   Google Scholar
• Holland, P. W., Laskey, K. B., and Leinhardt, S. (1983), “Stochastic Blockmodels: First Steps,” Social Networks, 5, 109–137. DOI: 10.1016/0378-8733(83)90021-7.
   Web of Science ®Google Scholar
• Horn, R. A., and Johnson, C. R. (1985), Matrix Analysis, Cambridge, UK: Cambridge University Press.
   Google Scholar
• Hotelling, H. (1933), “Analysis of a Complex of Statistical Variables Into Principal Components,” Journal of Educational Psychology, 24, 417–441. DOI: 10.1037/h0071325.
   Google Scholar
• Hu, Z., Pan, G., Wang, Y., and Wu, Z. (2016), “Sparse Principal Component Analysis via Rotation and Truncation,” IEEE Transactions on Neural Networks and Learning Systems, 27, 875–890. DOI: 10.1109/TNNLS.2015.2427451.
   PubMed Web of Science ®Google Scholar
• Jeffers, J. N. R. (1967), “Two Case Studies in the Application of Principal Component Analysis,” Journal of the Royal Statistical Society, Series C, 16, 225–236. DOI: 10.2307/2985919.
   Web of Science ®Google Scholar
• Jennrich, R. I. (2001), “A Simple General Procedure for Orthogonal Rotation,” Psychometrika, 66, 289–306. DOI: 10.1007/BF02294840.
   Web of Science ®Google Scholar
• Johnstone, I. M., and Lu, A. Y. (2009), “On Consistency and Sparsity for Principal Components Analysis in High Dimensions,” Journal of the American Statistical Association, 104, 682–693. DOI: 10.1198/jasa.2009.0121.
   PubMed Web of Science ®Google Scholar
• Jolliffe, I. T. (1995), “Rotation of Principal Components: Choice of Normalization Constraints,” Journal of Applied Statistics, 22, 29–35. DOI: 10.1080/757584395.
   Web of Science ®Google Scholar
• Jolliffe, I. T., Trendafilov, N. T., and Uddin, M. (2003), “A Modified Principal Component Technique based on the LASSO,” Journal of Computational and Graphical Statistics, 12, 531–547. DOI: 10.1198/1061860032148.
   Web of Science ®Google Scholar
• Journée, M., Nesterov, Y., Richtárik, P., and Sepulchre, R. (2010), “Generalized Power Method for Sparse Principal Component Analysis,” Journal of Machine Learning Research, 11, 517–553.
   Web of Science ®Google Scholar
• Kaiser, H. F. (1958), “The Varimax Criterion for Analytic Rotation in Factor Analysis,” Psychometrika, 23, 187–200. DOI: 10.1007/BF02289233.
   Web of Science ®Google Scholar
• Kaiser, H. F. (1960), “The Application of Electronic Computers to Factor Analysis,” Educational and Psychological Measurement, 20, 141–151.
   Web of Science ®Google Scholar
• Karrer, B., and Newman, M. E. J. (2011), “Stochastic Blockmodels and Community Structure in Networks,” Physical Review E, 83, 016107. DOI: 10.1103/PhysRevE.83.016107.
   PubMed Web of Science ®Google Scholar
• Lee, H., Battle, A., Raina, R., and Ng, A. Y. (2006), “Efficient Sparse Coding Algorithms,” in Proceedings of the 19th International Conference on Neural Information Processing Systems, NIPS’06, pp. 801–808, Cambridge, MA, MIT Press.
   Google Scholar
• Mackey, L. (2008), “Deflation Methods for Sparse PCA,” in Proceedings of the 21st International Conference on Neural Information Processing Systems, NIPS’08, pp. 1017–1024, Red Hook, NY: Curran Associates Inc.
   Google Scholar
• Moghaddam, B., Weiss, Y., and Avidan, S. (2006), “Generalized Spectral Bounds for Sparse LDA,” in Proceedings of the 23rd International Conference on Machine Learning, ICML ’06, pp. 641–648, New York: Association for Computing Machinery. DOI: 10.1145/1143844.1143925.
   Google Scholar
• Montoro, D. T., Haber, A. L., Biton, M., Vinarsky, V., Lin, B., Birket, S. E., Yuan, F., Chen, S., Leung, H. M., Villoria, J, et al. (2018), “A Revised Airway Epithelial Hierarchy Includes CFTR-Expressing Ionocytes,” Nature, 560, 319–324. DOI: 10.1038/s41586-018-0393-7.
   PubMed Web of Science ®Google Scholar
• Nocedal, J., and Wright, S. (2006), Numerical Optimization (2nd ed.), New York: Springer.
   Google Scholar
• Olshausen, B. A., and Field, D. J. (1996), “Emergence of Simple-Cell Receptive Field Properties by Learning a Sparse Code for Natural Images,” Nature, 381, 607. DOI: 10.1038/381607a0.
   PubMed Web of Science ®Google Scholar
• Pearson, K. (1901), “On Lines and Planes of Closest Fit to Systems of Points in Space,” The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2, 559–572. DOI: 10.1080/14786440109462720.
   Google Scholar
• Plasschaert, L. W., Žilionis, R., Choo-Wing, R., Savova, V., Knehr, J., Roma, G., Klein, A. M., and Jaffe, A. B. (2018), “A Single-Cell Atlas of the Airway Epithelium Reveals the CFTR-Rich Pulmonary Ionocyte,” Nature, 560, 377–381. DOI: 10.1038/s41586-018-0394-6.
   PubMed Web of Science ®Google Scholar
• Rohe, K., and Zeng, M. (2020), “Vintage Factor Analysis with Varimax Performs Statistical Inference,” arXiv:2004.05387.
   Google Scholar
• Rohe, K., Qin, T., and Yu, B. (2016), “Co-clustering Directed Graphs to Discover Asymmetries and Directional Communities,” Proceedings of the National Academy of Sciences, 113, 12679–12684. DOI: 10.1073/pnas.1525793113.
   PubMed Web of Science ®Google Scholar
• Shen, D., Shen, H., and Marron, J. S. (2013), “Consistency of Sparse PCA in High Dimension, Low Sample Size Contexts,” Journal of Multivariate Analysis, 115, 317–333. DOI: 10.1016/j.jmva.2012.10.007.
   Web of Science ®Google Scholar
• Shen, H., and Huang, J. Z. (2008), “Sparse Principal Component Analysis via Regularized Low Rank Matrix Approximation,” Journal of Multivariate Analysis, 99, 1015–1034. DOI: 10.1016/j.jmva.2007.06.007.
   Web of Science ®Google Scholar
• Thurstone, L. L. (1931), “Multiple Factor Analysis,” Psychological Review, 38, 406. DOI: 10.1037/h0069792.
   Google Scholar
• Tibshirani, R. (1996), “Regression Shrinkage and Selection via the Lasso,” Journal of the Royal Statistical Society, Series B, 58, 267–288. DOI: 10.1111/j.2517-6161.1996.tb02080.x.
   Google Scholar
• Tillmann, A. M., and Pfetsch, M. E. (2014), “The Computational Complexity of the Restricted Isometry Property, the Nullspace Property, and Related Concepts in Compressed Sensing,” IEEE Transactions on Information Theory, 60, 1248–1259. DOI: 10.1109/TIT.2013.2290112.
   Google Scholar
• Vu, V. Q., and Lei, J. (2013), “Minimax Sparse Principal Subspace Estimation in High Dimensions,” The Annals of Statistics, 41, 2905–2947. DOI: 10.1214/13-AOS1151.
   Web of Science ®Google Scholar
• Vu, V. Q., Cho, J., Lei, J., and Rohe, K. (2013), “Fantope Projection and Selection: A Near-Optimal Convex Relaxation of Sparse PCA,” in Proceedings of the 26th International Conference on Neural Information Processing Systems, NIPS’13, pp. 2670–2678, Red Hook, NY, USA, 2013. Curran Associates Inc.
   Google Scholar
• Wang, T., Berthet, Q., and Samworth, R. J. (2016), “Statistical and Computational Trade-Offs in Estimation of Sparse Principal Components,” The Annals of Statistics, 44, 1896–1930. DOI: 10.1214/15-AOS1369.
   Web of Science ®Google Scholar
• Witten, D. M., Tibshirani, R., and Hastie, T. (2009), “A Penalized Matrix Decomposition, with Applications to Sparse Principal Components and Canonical Correlation Analysis,” Biostatistics, 10, 515–534. DOI: 10.1093/biostatistics/kxp008.
   PubMed Web of Science ®Google Scholar
• Yuan, X.-T., and Zhang, T. (2013), “Truncated Power Method for Sparse Eigenvalue Problems,” Journal of Machine Learning Research, 14, 899–925.
   Web of Science ®Google Scholar
• Zhang, Y., and Rohe, K. (2018), “Understanding Regularized Spectral Clustering via Graph Conductance,” in Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18, pp. 10654–10663, Red Hook, NY, USA, 2018. Curran Associates Inc.
   Google Scholar
• Zhang, Y., Chen, F., and Lukito, J. (2022), “Network Amplification of Politicized Information and Misinformation about Covid-19 by Conservative Media and Partisan Influencers on Twitter,” Political Communication, 40, 24–47. DOI: 10.1080/10584609.2022.2113844.
   Web of Science ®Google Scholar
• Zhang, Y., Chen, F., and Rohe, K. (2022), “Social Media Public Opinion as Flocks in a Murmuration: Conceptualizing and Measuring Opinion Expression on Social Media,” Journal of Computer-Mediated Communication, 27, zmab021.
   Web of Science ®Google Scholar
• Zhang, Y., Yue, Z., Yang, X., Chen, F., and Kwak, N. (2022), “How a Peripheral Ideology Becomes Mainstream: Strategic Performance, Audience Reaction, and News Media Amplification in the Case of Qanon Twitter Accounts,” New Media & Society, 14614448221137324. DOI: 10.1177/14614448221137324.
   Web of Science ®Google Scholar
• Zou, H., and Xue, L. (2018), “A Selective Overview of Sparse Principal Component Analysis,” Proceedings of the IEEE, 106, 1311–1320. DOI: 10.1109/JPROC.2018.2846588.
   Web of Science ®Google Scholar
• Zou, H., Hastie, T., and Tibshirani, R. (2006), “Sparse Principal Component Analysis,” Journal of Computational and Graphical Statistics, 15, 265–286. DOI: 10.1198/106186006X113430.
   Web of Science ®Google Scholar