Random Projections for $k$-means Clustering

Abstract

This paper discusses the topic of dimensionality reduction for $k$-means clustering. We prove that any set of $n$ points in $d$ dimensions (rows in a matrix $A \in \RR^{n \times d}$) can be projected into $t = Ω(k / \eps^2)$ dimensions, for any $\eps \in (0,1/3)$, in $O(n d \lceil \eps^{-2} k/ \log(d) \rceil )$ time, such that with constant probability the optimal $k$-partition of the point set is preserved within a factor of $2+\eps$. The projection is done by post-multiplying $A$ with a $d\times t$ random matrix $R$ having entries $+1/\sqrt{t}$ or $-1/\sqrt{t}$ with equal probability. A numerical implementation of our technique and experiments on a large face images dataset verify the speed and the accuracy of our theoretical results.

Neural Information Processing Systems (NIPS) 2010

Cited by in corpus (23)

• Scalable Kernel K-Means Clustering with Nystrom Approximation: Relative-Error Bounds
• Random projections for Bayesian regression
• Fair Coresets and Streaming Algorithms for Fair k-Means Clustering
• Quantized Compressive K-Means
• Winner-Take-All Computation in Spiking Neural Networks
• An Analysis of Random Projections in Cancelable Biometrics
• Improved Guarantees for k-means++ and k-means++ Parallel
• Improved Coresets for Euclidean $k$-Means
• Using the Johnson-Lindenstrauss lemma in linear and integer programming
• Approximation and Streaming Algorithms for Projective Clustering via Random Projections
• Probabilistic methods for approximate archetypal analysis
• Recovery guarantees for exemplar-based clustering
• Tensorized Random Projections
• Subspace Embeddings Under Nonlinear Transformations
• Sketching Datasets for Large-Scale Learning (long version)
• Dimensionality Reduction for $k$-means Clustering
• Scalable Initialization Methods for Large-Scale Clustering
• VLSI Extreme Learning Machine: A Design Space Exploration
• On Generalization Bounds for Projective Clustering
• Efficient Clustering on Riemannian Manifolds: A Kernelised Random Projection Approach
• Convexification with bounded gap for randomly projected quadratic optimization
• Stable Sparse Subspace Embedding for Dimensionality Reduction
• Effective and Sparse Count-Sketch via k-means clustering