Robust Low-Rank Tensor Decomposition with the L₂ Criterion

References

• Acar, E., Dunlavy, D. M., and Kolda, T. G. (2011), “A Scalable Optimization Approach for Fitting Canonical Tensor Decompositions,” Journal of Chemometrics, 25, 67–86. DOI: 10.1002/cem.1335.
   Web of Science ®Google Scholar
• Acar, E., Dunlavy, D. M., Kolda, T. G., and Mørup, M. (2011), “Scalable Tensor Factorizations for Incomplete Data,” Chemometrics and Intelligent Laboratory Systems, 106, 41–56. DOI: 10.1016/j.chemolab.2010.08.004.
   Web of Science ®Google Scholar
• Anandkumar, A., Jain, P., Shi, Y., and Niranjan, U. N. (2016), “Tensor vs. Matrix Methods: Robust Tensor Decomposition Under Block Sparse Perturbations,” in Artificial Intelligence and Statistics, PMLR, pp. 268–276.
   Google Scholar
• Bader, B. W., and Kolda, T. G. (2006), “Algorithm 862: MATLAB Tensor Classes for Fast Algorithm Prototyping,” ACM Transactions on Mathematical Software (TOMS), 32, 635–653. DOI: 10.1145/1186785.1186794.
   Web of Science ®Google Scholar
• Bader, B. W., and Kolda, T. G. (2008), “Efficient MATLAB Computations with Sparse and Factored Tensors,” SIAM Journal on Scientific Computing, 30, 205–231.
   Web of Science ®Google Scholar
• Baunsgaard, D. (1999), “Factors Affecting 3-way Modelling (PARAFAC) of Fluorescence Landscapes,” Internal Report, Department of Dairy and Food Science, The Royal Veterinary and Agricultural University Denmark.
   Google Scholar
• Becker, S. (2015), “L-BFGS-B-C,” GitHub, available at https://github.com/stephenbeckr/L-BFGS-B-C.
   Google Scholar
• Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J., et al. (2011), “Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers,” Foundations and Trends[textregistered] in Machine Learning, 3, 1–122. DOI: 10.1561/2200000016.
   Google Scholar
• Byrd, R. H., Lu, P., Nocedal, J., and Zhu, C. (1995), “A Limited Memory Algorithm for Bound Constrained Optimization,” SIAM Journal on Scientific Computing, 16, 1190–1208. DOI: 10.1137/0916069.
   Web of Science ®Google Scholar
• Cai, C., Li, G., Poor, H. V., and Chen, Y. (2019), “Nonconvex Low-Rank Tensor Completion from Noisy Data,” in Advances in Neural Information Processing Systems (Vol. 32).
   Google Scholar
• Cai, C., Poor, H. V., and Chen, Y. (2023), “Uncertainty Quantification for Nonconvex Tensor Completion: Confidence Intervals, Heteroscedasticity and Optimality,” IEEE Transactions on Information Theory, 69, 407–452. DOI: 10.1109/TIT.2022.3205781.
   Web of Science ®Google Scholar
• Cai, J.-F., Li, J., and Xia, D. (2022), “Generalized Low-Rank Plus Sparse Tensor Estimation by Fast Riemannian Optimization,” Journal of the American Statistical Association, DOI: 10.1080/01621459.2022.2063131.
   PubMed Web of Science ®Google Scholar
• Candès, E. J., Li, X., Ma, Y., and Wright, J. (2011), “Robust Principal Component Analysis?” Journal of the ACM, 58, 1–37. DOI: 10.1145/1970392.1970395.
   Web of Science ®Google Scholar
• Carroll, J. D., and Chang, J.-J. (1970), “Analysis of Individual Differences in Multidimensional Scaling via an N-way Generalization of “Eckart-Young” Decomposition,” Psychometrika, 35, 283–319. DOI: 10.1007/BF02310791.
   Web of Science ®Google Scholar
• Chachlakis, D. G., Prater-Bennette, A., and Markopoulos, P. P. (2019), “L1-norm Tucker Tensor Decomposition,” IEEE Access, 7, 178454–178465. DOI: 10.1109/ACCESS.2019.2955134.
   Web of Science ®Google Scholar
• Chi, E. C., and Scott, D. W. (2014), “Robust Parametric Classification and Variable Selection by a Minimum Distance Criterion,” Journal of Computational and Graphical Statistics, 23, 111–128. DOI: 10.1080/10618600.2012.737296.
   Web of Science ®Google Scholar
• Chi, J. T., and Chi, E. C. (2022), “A User-Friendly Computational Framework for Robust Structured Regression with the L2 Criterion,” Journal of Computational and Graphical Statistics, 31, 1051–1062. DOI: 10.1080/10618600.2022.2035232.
   PubMed Web of Science ®Google Scholar
• De Lathauwer, L., De Moor, B., and Vandewalle, J. (2000a), “A Multilinear Singular Value Decomposition,” SIAM Journal on Matrix Analysis and Applications, 21, 1253–1278. DOI: 10.1137/S0895479896305696.
   Web of Science ®Google Scholar
• De Lathauwer, L., De Moor, B., and Vandewalle, J. (2000b), “On the Best Rank-1 and Rank-(r1,r2,…,rn) Approximation of Higher-Order Tensors,” SIAM Journal on Matrix Analysis and Applications, 21, 1324–1342.
   Web of Science ®Google Scholar
• Deng, L. (2012), “The MNIST Database of Handwritten Digit Images for Machine Learning Research,” IEEE Signal Processing Magazine, 29, 141–142.
   Google Scholar
• Donoho, D. L., and Liu, R. C. (1988), “The “Automatic” Robustness of Minimum Distance Functionals,” The Annals of Statistics, 16, 552–586. DOI: 10.1214/aos/1176350820.
   Web of Science ®Google Scholar
• Filipović, M., and Jukić, A. (2015), “Tucker Factorization with Missing Data with Application to Low-n-Rank Tensor Completion,” Multidimensional Systems and Signal Processing, 26, 677–692. DOI: 10.1007/s11045-013-0269-9.
   Web of Science ®Google Scholar
• Gandy, S., Recht, B., and Yamada, I. (2011), “Tensor Completion and Low-n-Rank Tensor Recovery via Convex Optimization,” Inverse Problems, 27, 025010. DOI: 10.1088/0266-5611/27/2/025010.
   Web of Science ®Google Scholar
• Goldfarb, D., and Qin, Z. (2014), “Robust Low-Rank Tensor Recovery: Models and Algorithms,” SIAM Journal on Matrix Analysis and Applications, 35, 225–253. DOI: 10.1137/130905010.
   Web of Science ®Google Scholar
• Gu, Q., Gui, H., and Han, J. (2014), “Robust Tensor Decomposition with Gross Corruption,” in Advances in Neural Information Processing Systems (Vol. 27).
   Google Scholar
• Han, R., Willett, R., and Zhang, A. R. (2022), “An Optimal Statistical and Computational Framework for Generalized Tensor Estimation,” The Annals of Statistics, 50, 1–29. DOI: 10.1214/21-AOS2061.
   Web of Science ®Google Scholar
• Harshman, R. A. (1970), “Foundations of the PARAFAC Procedure: Models and Conditions for an “Explanatory” Multimodal Factor Analysis,” UCLA Working Papers in Phonetics, 16, 1–84.
   Google Scholar
• Hjort, N. (1994), “Minimum L2 and Robust Kullback-Leibler Estimation,” in Proceedings of the 12th Prague Conference, pp. 102–105.
   Google Scholar
• Hong, D., Kolda, T. G., and Duersch, J. A. (2020), “Generalized Canonical Polyadic Tensor Decomposition,” SIAM Review, 62, 133–163. DOI: 10.1137/18M1203626.
   Web of Science ®Google Scholar
• Kapteyn, A., Neudecker, H., and Wansbeek, T. (1986), “An Approach to n-mode Components Analysis,” Psychometrika, 51, 269–275. DOI: 10.1007/BF02293984.
   Web of Science ®Google Scholar
• Kilmer, M. E., and Martin, C. D. (2011), “Factorization Strategies for Third-Order Tensors,” Linear Algebra and its Applications, 435, 641–658. DOI: 10.1016/j.laa.2010.09.020.
   Web of Science ®Google Scholar
• Kolda, T. G. (2006), “Multilinear Operators for Higher-Order Decompositions,” Technical Report SAND2006-2081, Sandia National Laboratories, Albuquerque, NM, Livermore, CA.
   Google Scholar
• Kolda, T. G., and Bader, B. W. (2009), “Tensor Decompositions and Applications,” SIAM Review, 51, 455–500. DOI: 10.1137/07070111X.
   Web of Science ®Google Scholar
• Kroonenberg, P. M., and De Leeuw, J. (1980), “Principal Component Analysis of Three-Mode Data by Means of Alternating Least Squares Algorithms,” Psychometrika, 45, 69–97. DOI: 10.1007/BF02293599.
   Web of Science ®Google Scholar
• Lane, J. W. (2012), “Robust Quantile Regression using L2E,” Ph.D. thesis.
   Google Scholar
• Lee, J. (2010), “L2E Estimation for Finite Mixture of Regression Models with Applications and L2E with Penalty and Non-normal Mixtures,” Ph.D. thesis.
   Google Scholar
• Liu, D. C., and Nocedal, J. (1989), “On the Limited Memory BFGS Method for Large Scale Optimization,” Mathematical Programming, 45, 503–528. DOI: 10.1007/BF01589116.
   Web of Science ®Google Scholar
• Liu, X., Chi, E. C., and Lange, K. (in press), “A Sharper Computational Tool for L2E Regression,” Technometrics.
   Web of Science ®Google Scholar
• Lozano, A. C., Meinshausen, N., and Yang, E. (2016), “Minimum Distance Lasso for Robust High-Dimensional Regression,” Electronic Journal of Statistics, 10, 1296–1340. DOI: 10.1214/16-EJS1136.
   Web of Science ®Google Scholar
• Lu, C., Feng, J., Chen, Y., Liu, W., Lin, Z., and Yan, S. (2019), “Tensor Robust Principal Component Analysis with a New Tensor Nuclear Norm,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 42, 925–938. DOI: 10.1109/TPAMI.2019.2891760.
   PubMed Web of Science ®Google Scholar
• Ma, J., Qiu, W., Zhao, J., Ma, Y., Yuille, A. L., and Tu, Z. (2015), “Robust L2E Estimation of Transformation for Non-rigid Registration,” IEEE Transactions on Signal Processing, 63, 1115–1129.
   Web of Science ®Google Scholar
• Ma, J., Zhao, J., Tian, J., Tu, Z., and Yuille, A. L. (2013), “Robust Estimation of Nonrigid Transformation for Point Set Registration,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2147–2154.
   Google Scholar
• Murphy, K. R., Stedmon, C. A., Graeber, D., and Bro, R. (2013), “Fluorescence Spectroscopy and Multi-Way Techniques. PARAFAC,” Analytical Methods, 5, 6557–6566. DOI: 10.1039/c3ay41160e.
   Web of Science ®Google Scholar
• Nene, S. A., Nayar, S. K., and Murase, H. (1996), “Columbia Object Image Library (COIL-20)”.
   Google Scholar
• Oseledets, I. V. (2011), “Tensor-Train Decomposition,” SIAM Journal on Scientific Computing, 33, 2295–2317. DOI: 10.1137/090752286.
   Web of Science ®Google Scholar
• Phan, A. H., and Cichocki, A. (2010), “Tensor Decompositions for Feature Extraction and Classification of High Dimensional Datasets,” Nonlinear Theory and its Applications, IEICE, 1, 37–68. DOI: 10.1587/nolta.1.37.
   Google Scholar
• Ramos, J. J. (2014), “Robust Methods for Forecast Aggregation,” Ph.D. thesis.
   Google Scholar
• Riu, J., and Bro, R. (2003), “Jack-Knife Technique for Outlier Detection and Estimation of Standard Errors in PARAFAC Models,” Chemometrics and Intelligent Laboratory Systems, 65, 35–49. DOI: 10.1016/S0169-7439(02)00090-4.
   Web of Science ®Google Scholar
• Scott, A. I. (2006), “Denoising by Wavelet Thresholding Using Multivariate Minimum Distance Partial Density Estimation,” Ph.D. thesis.
   Google Scholar
• Scott, D. W. (2001), “Parametric Statistical Modeling by Minimum Integrated Square Error,” Technometrics, 43, 274–285. DOI: 10.1198/004017001316975880.
   Web of Science ®Google Scholar
• Scott, D. W. (2009), “The L2E Method,” Wiley Interdisciplinary Reviews: Computational Statistics, 1, 45–51.
   Google Scholar
• Terrell, G. R. (1990), Linear Density Estimates, Department of Statistics, Virginia Polytechnic Institute and State University.
   Google Scholar
• Tomioka, R., Suzuki, T., Hayashi, K., and Kashima, H. (2011), “Statistical Performance of Convex Tensor Decomposition,” in Advances in Neural Information Processing Systems (Vol. 24), pp. 972–980.
   Google Scholar
• Tong, T., Ma, C., Prater-Bennette, A., Tripp, E., and Chi, Y. (2022), “Scaling and Scalability: Provable Nonconvex Low-Rank Tensor Estimation from Incomplete Measurements,” Journal of Machine Learning Research, 23, 1–77.
   Web of Science ®Google Scholar
• Tucker, L. R. (1966), “Some Mathematical Notes on Three-Mode Factor Analysis,” Psychometrika, 31, 279–311. DOI: 10.1007/BF02289464.
   PubMed Web of Science ®Google Scholar
• Wu, Y., Tan, H., Li, Y., Li, F., and He, H. (2017), “Robust Tensor Decomposition based on Cauchy Distribution and its Applications,” Neurocomputing, 223, 107–117. DOI: 10.1016/j.neucom.2016.10.030.
   Web of Science ®Google Scholar
• Xia, D., and Yuan, M. (2019), “On Polynomial Time Methods for Exact Low-Rank Tensor Completion,” Foundations of Computational Mathematics, 19, 1265–1313. DOI: 10.1007/s10208-018-09408-6.
   Web of Science ®Google Scholar
• Xia, D., Yuan, M., and Zhang, C.-H. (2021), “Statistically Optimal and Computationally Efficient Low Rank Tensor Completion from Noisy Entries,” The Annals of Statistics, 49, 76–99. DOI: 10.1214/20-AOS1942.
   Web of Science ®Google Scholar
• Yang, J., and Scott, D. W. (2013), “Robust Fitting of a Weibull Model with Optional Censoring,” Computational Statistics & Data Analysis, 67, 149–161. DOI: 10.1016/j.csda.2013.05.009.
   PubMed Web of Science ®Google Scholar
• Yang, K., Pan, A., Yang, Y., Zhang, S., Ong, S. H., and Tang, H. (2017), “Remote Sensing Image Registration Using Multiple Image Features,” Remote Sensing, 9, 581. DOI: 10.3390/rs9060581.
   Web of Science ®Google Scholar
• Yang, Y., Feng, Y., and Suykens, J. A. (2015), “Robust Low-Rank Tensor Recovery with Regularized Redescending M-estimator,” IEEE Transactions on Neural Networks and Learning Systems, 27, 1933–1946. DOI: 10.1109/TNNLS.2015.2465178.
   PubMed Web of Science ®Google Scholar
• Zhang, A. (2019), “Cross: Efficient Low-Rank Tensor Completion,” The Annals of Statistics, 47, 936–964. DOI: 10.1214/18-AOS1694.
   Web of Science ®Google Scholar
• Zhang, A., and Xia, D. (2018), “Tensor SVD: Statistical and Computational Limits,” IEEE Transactions on Information Theory, 64, 7311–7338. DOI: 10.1109/TIT.2018.2841377.
   Web of Science ®Google Scholar
• Zhao, Q., Zhou, G., Xie, S., Zhang, L., and Cichocki, A. (2016), “Tensor Ring Decomposition,” arXiv preprint arXiv:1606.05535.
   Google Scholar
• Zhao, Q., Zhou, G., Zhang, L., Cichocki, A., and Amari, S.-I. (2015), “Bayesian Robust Tensor Factorization for Incomplete Multiway Data,” IEEE Transactions on Neural Networks and Learning Systems, 27, 736–748. DOI: 10.1109/TNNLS.2015.2423694.
   PubMed Web of Science ®Google Scholar
• Zhu, C., Byrd, R. H., Lu, P., and Nocedal, J. (1997), “Algorithm 778: L-BFGS-B: Fortran Subroutines for Large-Scale Bound-Constrained Optimization,” ACM Transactions on Mathematical Software (TOMS), 23, 550–560. DOI: 10.1145/279232.279236.
   Web of Science ®Google Scholar