Tuning-parameter selection in regularized estimations of large covariance matrices

Abstract

Recently many regularized estimators of large covariance matrices have been proposed, and the tuning parameters in these estimators are usually selected via cross-validation. However, there is a lack of consensus on the number of folds for conducting cross-validation. One round of cross-validation involves partitioning a sample of data into two complementary subsets, a training set and a validation set. In this manuscript, we demonstrate that if the estimation accuracy is measured in the Frobenius norm, the training set should consist of majority of the data; whereas if the estimation accuracy is measured in the operator norm, the validation set should consist of majority of the data. We also develop methods for selecting tuning parameters based on the bootstrap and compare them with their cross-validation counterparts. We demonstrate that the cross-validation methods with ‘optimal’ choices of folds are more appropriate than their bootstrap counterparts.

Keywords:

• banding
• bootstrap
• covariance matrix
• cross-validation
• Frobenius norm
• operator norm
• thresholding

AMS Subject Classification:

• 62H12

Acknowledgments

We would like to thank the editor, the associate editor and two referees for their valuable comments which led to substantial improvements in this paper.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Yang Feng http://orcid.org/0000-0001-7746-7598

Additional information

Funding

Dr Yang Feng's work is supported in part by US NSF grant [DMS-1308566].