Tradeoffs for Space, Time, Data and Risk in Unsupervised Learning

Abstract

Faced with massive data, is it possible to trade off (statistical) risk, and (computational) space and time? This challenge lies at the heart of large-scale machine learning. Using k-means clustering as a prototypical unsupervised learning problem, we show how we can strategically summarize the data (control space) in order to trade off risk and time when data is generated by a probabilistic model. Our summarization is based on coreset constructions from computational geometry. We also develop an algorithm, TRAM, to navigate the space/time/data/risk tradeoff in practice. In particular, we show that for a fixed risk (or data size), as the data size increases (resp. risk increases) the running time of TRAM decreases. Our extensive experiments on real data sets demonstrate the existence and practical utility of such tradeoffs, not only for k-means but also for Gaussian Mixture Models.

Conference version

References in corpus (3)

• Computational and Statistical Tradeoffs via Convex Relaxation
• Distributed k-Means and k-Median Clustering on General Topologies
• Moment-based Uniform Deviation Bounds for $k$-means and Friends

Cited by in corpus (6)

• Real-Time EEG Classification via Coresets for BCI Applications
• Training Gaussian Mixture Models at Scale via Coresets
• Strong Coresets for Hard and Soft Bregman Clustering with Applications to Exponential Family Mixtures
• Starting Small -- Learning with Adaptive Sample Sizes
• Computational and Statistical Tradeoffs in Learning to Rank
• Coresets for Kinematic Data: From Theorems to Real-Time Systems