Estimating covariance and precision matrices along subspaces

Abstract

We study the accuracy of estimating the covariance and the precision matrix of a $D$-variate sub-Gaussian distribution along a prescribed subspace or direction using the finite sample covariance. Our results show that the estimation accuracy depends almost exclusively on the components of the distribution that correspond to desired subspaces or directions. This is relevant and important for problems where the behavior of data along a lower-dimensional space is of specific interest, such as dimension reduction or structured regression problems. We also show that estimation of precision matrices is almost independent of the condition number of the covariance matrix. The presented applications include direction-sensitive eigenspace perturbation bounds, relative bounds for the smallest eigenvalue, and the estimation of the single-index model. For the latter, a new estimator, derived from the analysis, with strong theoretical guarantees and superior numerical performance is proposed.

arXiv comment25 pages, 9 figures

References in corpus (14)

• High-dimensional graphs and variable selection with the Lasso
• Regularized estimation of large covariance matrices
• Covariance regularization by thresholding
• Quantitative estimates of the convergence of the empirical covariance matrix in Log-concave Ensembles
• Fast and Adaptive Sparse Precision Matrix Estimation in High Dimensions
• Covariance estimation for distributions with $2+\epsilon$ moments
• Normal approximation and concentration of spectral projectors of sample covariance
• Efficient Learning of Generalized Linear and Single Index Models with Isotonic Regression
• Concentration in unbounded metric spaces and algorithmic stability
• A new algorithm for estimating the effective dimension-reduction subspace
• Score estimation in the monotone single index model
• Lower bounds on the smallest eigenvalue of a sample covariance matrix
• On strict sub-Gaussianity, optimal proxy variance and symmetry for bounded random variables
• On the perturbation of the Moore-Penrose inverse of a matrix

Cited by in corpus (1)

• Asymmetric canonical correlation analysis of Riemannian and high-dimensional data