Quantized Compressive K-Means

Abstract

The recent framework of compressive statistical learning aims at designing tractable learning algorithms that use only a heavily compressed representation-or sketch-of massive datasets. Compressive K-Means (CKM) is such a method: it estimates the centroids of data clusters from pooled, non-linear, random signatures of the learning examples. While this approach significantly reduces computational time on very large datasets, its digital implementation wastes acquisition resources because the learning examples are compressed only after the sensing stage. The present work generalizes the sketching procedure initially defined in Compressive K-Means to a large class of periodic nonlinearities including hardware-friendly implementations that compressively acquire entire datasets. This idea is exemplified in a Quantized Compressive K-Means procedure, a variant of CKM that leverages 1-bit universal quantization (i.e. retaining the least significant bit of a standard uniform quantizer) as the periodic sketch nonlinearity. Trading for this resource-efficient signature (standard in most acquisition schemes) has almost no impact on the clustering performances, as illustrated by numerical experiments.

References in corpus (5)

• Hilbert space embeddings and metrics on probability measures
• Random Projections through multiple optical scattering: Approximating kernels at the speed of light
• Random Projections for $k$-means Clustering
• Orthogonal Matching Pursuit with Replacement
• Simple Classification using Binary Data

Cited by in corpus (6)

• Making AI Forget You: Data Deletion in Machine Learning
• Breaking the waves: asymmetric random periodic features for low-bitrate kernel machines
• Sketching Datasets for Large-Scale Learning (long version)
• Compressive Classification (Machine Learning without learning)
• Compressive Meta-Learning
• Asymmetric compressive learning guarantees with applications to quantized sketches