Orthogonal canonical correlation analysis and applications

Abstract

Canonical correlation analysis (CCA) is a cornerstone of linear dimensionality reduction techniques that jointly maps two datasets to achieve maximal correlation. CCA has been widely used in applications for capturing data features of interest. In this paper, we establish a range constrained orthogonal CCA (OCCA) model and its variant and apply them for three data analysis tasks of datasets in real-life applications, namely unsupervised feature fusion, multi-target regression and multi-label classification. Numerical experiments show that the OCCA and its variant produce superior accuracy compared to the traditional CCA.

Keywords:

• Canonical correlation analysis (CCA)
• orthogonal CCA
• singular value decomposition
• unsupervised feature fusion
• multi-target regression
• multi-label classification

AMS Subject Classifications:

• 15A18
• 15A21
• 62H20
• 62H25
• 65F15
• 65F30

Acknowledgements

The authors wish to thank anonymous referees for their constructive comments and suggestions that improved the presentation.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 Private communications, 2019.

2 https://archive.ics.uci.edu/ml/datasets/Multiple+Features.

3 https://www.csie.ntu.edu.tw/∼cjlin/liblinear/.

4 http://mulan.sourceforge.net/datasets-mtr.html.

5 https://www.csie.ntu.edu.tw/∼cjlin/liblinear/.

6 http://mulan.sourceforge.net/datasets-mlc.html.

7 https://www.csie.ntu.edu.tw/∼cjlin/libsvmtools/datasets/multilabel.html.

8 http://manikvarma.org/downloads/XC/XMLRepository.html.

Additional information

Funding

The research of Zhang was supported in part by the National Natural Science Foundations of China grant numbers NSFC-11671246 and NSFC-91730303. Bai was supported in part by NSF grants CCF-1527091 and DMS-1913364. Li was supported in part by NSF grants 1527104 and DMS-1719620.

Notes on contributors

Li Wang

Li Wang is an assistant professor with Department of Mathematics and Department of Computer Science and Engineering, University of Texas at Arlington, Texas, USA. She received her PhD degree from Department of Mathematics, University of California, San Diego, USA, in 2014. Her research interests include large scale optimization, polynomial optimization and machine learning.

Lei-hong Zhang

Lei-hong Zhang is a professor with the School of Mathematical Sciences, Soochow University, Suzhou, China. He received the PhD degree in mathematics from Hong Kong Baptist University, China in 2008. His research interest includes numerical optimization, eigenvalue problems and machine learning.

Zhaojun Bai

Zhaojun Bai is a professor in the Department of Computer Science and Department of Mathematics, University of California, Davis. He obtained his PhD from Fudan University, China in 1988. His main research interests include linear algebra algorithm design and analysis, mathematical software engineering and applications in computational science and engineering, and data science.

Ren-Cang Li

Ren-cang Li is a professor with the Department of Mathematics, University of Texas at Arlington, Texas, USA. He received his PhD degree in applied mathematics from the University of California at Berkeley in 1995. His research interest includes floating-point support for scientific computing, large and sparse linear systems, eigenvalue problems, and model reduction, machine learning and unconventional schemes for differential equations.