Large Scale Kernel Learning using Block Coordinate Descent
arXiv:1602.05310
Abstract
We demonstrate that distributed block coordinate descent can quickly solve kernel regression and classification problems with millions of data points. Armed with this capability, we conduct a thorough comparison between the full kernel, the Nyström method, and random features on three large classification tasks from various domains. Our results suggest that the Nyström method generally achieves better statistical accuracy than random features, but can require significantly more iterations of optimization. Lastly, we derive new rates for block coordinate descent which support our experimental findings when specialized to kernel methods.
References in corpus (6)
- HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
- Distributed Coordinate Descent Method for Learning with Big Data
- Communication-Efficient Distributed Dual Coordinate Ascent
- Fast Randomized Kernel Methods With Statistical Guarantees
- Least Squares Revisited: Scalable Approaches for Multi-class Prediction
- Rows vs. Columns: Randomized Kaczmarz or Gauss-Seidel for Ridge Regression
Cited by in corpus (10)
- Deep Learning with Differential Privacy
- Recursive Sampling for the Nyström Method
- Kernel methods through the roof: handling billions of points efficiently
- Efficient Data-Driven Geologic Feature Detection from Pre-stack Seismic Measurements using Randomized Machine-Learning Algorithm
- Breaking Locality Accelerates Block Gauss-Seidel
- Faster Kernel Ridge Regression Using Sketching and Preconditioning
- High-Dimensional Optimization in Adaptive Random Subspaces
- Sigma-Delta and Distributed Noise-Shaping Quantization Methods for Random Fourier Features
- A convex method for classification of groups of examples
- Communication-Efficient Parallel Block Minimization for Kernel Machines