Dynamic Principal Component Analysis in High Dimensions

Abstract

Principal component analysis is a versatile tool to reduce dimensionality which has wide applications in statistics and machine learning. It is particularly useful for modeling data in high-dimensional scenarios where the number of variables p is comparable to, or much larger than the sample size n. Despite an extensive literature on this topic, researchers have focused on modeling static principal eigenvectors, which are not suitable for stochastic processes that are dynamic in nature. To characterize the change in the entire course of high-dimensional data collection, we propose a unified framework to directly estimate dynamic eigenvectors of covariance matrices. Specifically, we formulate an optimization problem by combining the local linear smoothing and regularization penalty together with the orthogonality constraint, which can be effectively solved by manifold optimization algorithms. We show that our method is suitable for high-dimensional data observed under both common and irregular designs, and theoretical properties of the estimators are investigated under $l_q(0\le q\le 1)$ sparsity. Extensive experiments demonstrate the effectiveness of the proposed method in both simulated and real data examples.

Keywords:

• Dimension reduction
• Local linear smoothing
• Manifold optimization
• Sparsity

Supplementary Materials

The supplementary material contains the algorithm details, additional simulation and technical proofs.

Acknowledgments

Fang Yao is the corresponding author. Xiaoyu Hu is the first author, and conducted this research when she was a PhD student at Peking University. The authors thank the Editor, Associate Editor and two anonymous referees for their many helpful comments that have resulted in significant improvements in the article.

Additional information

Funding

This research is supported by National Key R&D Program of China (No. 2020YFE0204200, 2022YFA1003801), National Natural Science Foundation of China (No. 11931001, 12292981, 11871080), the LMAM, and the LMEQF.