Correlation structure regularization via entropy loss function for high-dimension and low-sample-size data

Abstract

Estimating structured covariance or correlation matrix has been paid more and more attentions in recent years. A recent method based on the entropy loss function was proposed to regularize the covariance structure for a given covariance matrix whose underlying structure may be blurred due to random noises from different sources. However, the entropy loss function considered is very likely to be unavailable in covariance regularization for high-dimension and low-sample-size (HDLSS) data. In this paper, a new discrepancy is proposed for regularizing correlation structure, in which the given correlation matrix (e.g., sample correlation matrix) and the candidate structure in the entropy loss function are both added by the identity matrix multiplied by a constant, so that the problem owing to likely singularity of sample correlation matrix for HDLSS data can be overcome. The candidate correlation structures considered in this paper include tri-diagonal Toeplitz, compound symmetry, AR(1) and banded Toeplitz. The regularized correlation estimates for the first three structures can be obtained by solving one-dimensional optimization problems, while the regularized one for the fourth structure can be computed efficiently using Newton’s iteration method. Simulation studies show that the proposed new approach works well, providing a reliable method to regularize the correlation structure for HDLSS data.

Keywords:

• Bregman divergence
• Correlation matrix estimation
• Entropy loss function
• High-dimensional correlation matrix
• Regularization

Additional information

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 61374027 and the Program for Changjiang Scholars and Innovative Research Team in University under Grant IRT_16R53.