Euclidean Representation of Low-Rank Matrices and Its Statistical Applications
arXiv:2103.04220
Abstract
Low-rank matrices are pervasive throughout statistics, machine learning, signal processing, optimization, and applied mathematics. In this paper, we propose a novel and user-friendly Euclidean representation framework for low-rank matrices. Correspondingly, we establish a collection of technical and theoretical tools for analyzing the intrinsic perturbation of low-rank matrices in which the underlying referential matrix and the perturbed matrix both live on the same low-rank matrix manifold. Our analyses show that, locally around the referential matrix, the sine-theta distance between subspaces is equivalent to the Euclidean distance between two appropriately selected orthonormal basis, circumventing the orthogonal Procrustes analysis. We also establish the regularity of the proposed Euclidean representation function, which has a profound statistical impact and a meaningful geometric interpretation. These technical devices are applicable to a broad range of statistical problems. Specific applications considered in detail include Bayesian sparse spiked covariance model with non-intrinsic loss, efficient estimation in stochastic block models where the block probability matrix may be degenerate, and least-squares estimation in biclustering problems. Both the intrinsic perturbation analysis of low-rank matrices and the regularity theorem may be of independent interest.
References in corpus (16)
- Stochastic blockmodels and community structure in networks
- Matrix estimation by Universal Singular Value Thresholding
- Sparse principal component analysis and iterative thresholding
- Sparse PCA: Optimal rates and adaptive estimation
- Optimal detection of sparse principal components in high dimension
- Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels
- Sharp nonasymptotic bounds on the norm of random matrices with independent entries
- Covariate-assisted spectral clustering
- Minimax risk of matrix denoising by singular value thresholding
- Unitary orbits of Hermitian operators with convex or concave functions
- Optimal Estimation and Completion of Matrices with Biclustering Structures
- Co-clustering separately exchangeable network data
- Asymptotically efficient estimators for stochastic blockmodels: the naive MLE, the rank-constrained MLE, and the spectral
- Bayesian Estimation of Sparse Spiked Covariance Matrices in High Dimensions
- Spike and slab Bayesian sparse principal component analysis
- Efficient Estimation for Random Dot Product Graphs via a One-step Procedure