Coresets for Near-Convex Functions

Abstract

Coreset is usually a small weighted subset of $n$ input points in ${\mathbb{R}}^d$, that provably approximates their loss function for a given set of queries (models, classifiers, etc.). Coresets become increasingly common in machine learning since existing heuristics or inefficient algorithms may be improved by running them possibly many times on the small coreset that can be maintained for streaming distributed data. Coresets can be obtained by sensitivity (importance) sampling, where its size is proportional to the total sum of sensitivities. Unfortunately, computing the sensitivity of each point is problem dependent and may be harder to compute than the original optimization problem at hand. We suggest a generic framework for computing sensitivities (and thus coresets) for wide family of loss functions which we call near-convex functions. This is by suggesting the $f$-SVD factorization that generalizes the SVD factorization of matrices to functions. Example applications include coresets that are either new or significantly improves previous results, such as SVM, Logistic regression, M-estimators, and ${\ell }_z$-regression. Experimental results and open source are also provided.

References in corpus (9)

• Scikit-learn: Machine Learning in Python
• One-Shot Coresets: The Case of k-Clustering
• Fast and Accurate Least-Mean-Squares Solvers
• Discrepancy, Coresets, and Sketches in Machine Learning
• Generic Coreset for Scalable Learning of Monotonic Kernels: Logistic Regression, Sigmoid and more
• Strong Coresets for k-Median and Subspace Approximation: Goodbye Dimension
• On Coresets for Regularized Loss Minimization
• Coresets for Ordered Weighted Clustering
• Tight Sensitivity Bounds For Smaller Coresets

Cited by in corpus (4)

• Improved Coresets for Euclidean $k$-Means
• Compressed Deep Networks: Goodbye SVD, Hello Robust Low-Rank Approximation
• Persistent Reductions in Regularized Loss Minimization for Variable Selection
• Deep Learning Meets Projective Clustering