Eigengap Sparsity for Covariance Parsimony

Abstract

Covariance estimation is a central problem in statistics. An important issue is that there are rarely enough samples $n$ to accurately estimate the $p(p+1)/2$ coefficients in dimension $p$. Parsimonious covariance models are therefore preferred, but the discrete nature of model selection makes inference computationally challenging. In this paper, we propose a relaxation of covariance parsimony termed "eigengap sparsity" and motivated by the good accuracy-parsimony tradeoffs of eigenvalue-equalization in covariance matrices. This penalty can be included in a penalized-likelihood framework that we propose to solve with a projected gradient descent on a monotone cone. The algorithm turns out to resemble an isotonic regression of mutually-attracted sample eigenvalues, drawing an interesting link between covariance parsimony and shrinkage.

References in corpus (8)

• Regularized estimation of large covariance matrices
• High-Dimensional Data Clustering
• Robust Shrinkage Estimation of High-dimensional Covariance Matrices
• Covariance Estimation: The GLM and Regularization Perspectives
• Projected gradient descent accumulates at Bouligand stationary points
• The curse of isotropy: from principal components to principal subspaces
• Nested subspace learning with flags
• Parsimonious Gaussian mixture models with piecewise-constant eigenvalue profiles