Dimensionality Reduction of Massive Sparse Datasets Using Coresets

Abstract

In this paper we present a practical solution with performance guarantees to the problem of dimensionality reduction for very large scale sparse matrices. We show applications of our approach to computing the low rank approximation (reduced SVD) of such matrices. Our solution uses coresets, which is a subset of $O(k/\eps^2)$ scaled rows from the $n\times d$ input matrix, that approximates the sub of squared distances from its rows to every $k$-dimensional subspace in $\REAL^d$, up to a factor of $1\pm\eps$. An open theoretical problem has been whether we can compute such a coreset that is independent of the input matrix and also a weighted subset of its rows. %An open practical problem has been whether we can compute a non-trivial approximation to the reduced SVD of very large databases such as the Wikipedia document-term matrix in a reasonable time. We answer this question affirmatively. % and demonstrate an algorithm that efficiently computes a low rank approximation of the entire English Wikipedia. Our main technical result is a novel technique for deterministic coreset construction that is based on a reduction to the problem of ${\ell }_2$ approximation for item frequencies.

References in corpus (2)

• Generalized power method for sparse principal component analysis
• A Note on Element-wise Matrix Sparsification via a Matrix-valued Bernstein Inequality

Cited by in corpus (9)

• Real-Time EEG Classification via Coresets for BCI Applications
• Fixed-Rank Approximation of a Positive-Semidefinite Matrix from Streaming Data
• Compressed Deep Networks: Goodbye SVD, Hello Robust Low-Rank Approximation
• Robust Coreset Construction for Distributed Machine Learning
• Online Coresets for Clustering with Bregman Divergences
• Introduction to Coresets: Approximated Mean
• Joint Coreset Construction and Quantization for Distributed Machine Learning
• Sparse Coresets for SVD on Infinite Streams
• Faster Projective Clustering Approximation of Big Data