Faster Low-rank Approximation using Adaptive Gap-based Preconditioning

Abstract

We propose a method for rank $k$ approximation to a given input matrix $X\in {\mathbb{R}}^{d\times n}$ which runs in time \[ \tilde{O} \left(d ~\cdot~ \min\left\{n + \tilde{sr}(X) \,G^{-2}_{k,p+1}\ ,\ n^{3/4}\, \tilde{sr}(X)^{1/4} \,G^{-1/2}_{k,p+1} \right\} ~\cdot~ \text{poly}(p)\right) ~, \] where $p>k$, $\widetilde{sr}(X)$ is related to stable rank of $X$, and $G_{k,p+1}=\frac{{\sigma }_k-{\sigma }_p}{{\sigma }_k}$ is the multiplicative gap between the $k$-th and the $(p+1)$-th singular values of $X$. In particular, this yields a linear time algorithm if the gap is at least $1/\sqrt{n}$ and $k,p,\widetilde{sr}(X)$ are constants.