Distributed Mini-Batch SDCA
arXiv:1507.08322
Abstract
We present an improved analysis of mini-batched stochastic dual coordinate ascent for regularized empirical loss minimization (i.e. SVM and SVM-type objectives). Our analysis allows for flexible sampling schemes, including where data is distribute across machines, and combines a dependence on the smoothness of the loss and/or the data spread (measured through the spectral norm).
References in corpus (10)
- Optimal Distributed Online Prediction using Mini-Batches
- Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting
- Stochastic Dual Coordinate Ascent Methods for Regularized Loss Minimization
- Parallel Coordinate Descent for L1-Regularized Loss Minimization
- Distributed Coordinate Descent Method for Learning with Big Data
- Parallel Coordinate Descent Methods for Big Data Optimization
- Distributed Block Coordinate Descent for Minimizing Partially Separable Functions
- Accelerated, Parallel and Proximal Coordinate Descent
- Coordinate Descent with Arbitrary Sampling I: Algorithms and Complexity
- Analysis of Distributed Stochastic Dual Coordinate Ascent
Cited by in corpus (14)
- Federated Optimization: Distributed Machine Learning for On-Device Intelligence
- Stochastic, Distributed and Federated Optimization for Machine Learning
- Gradient Diversity: a Key Ingredient for Scalable Distributed Learning
- DAve-QN: A Distributed Averaged Quasi-Newton Method with Local Superlinear Convergence Rate
- Primal-Dual Rates and Certificates
- Distributed Inexact Damped Newton Method: Data Partitioning and Load-Balancing
- Partitioning Data on Features or Samples in Communication-Efficient Distributed Optimization?
- Privileged Multi-label Learning
- Distributed Asynchronous Dual Free Stochastic Dual Coordinate Ascent
- Decoupled Asynchronous Proximal Stochastic Gradient Descent with Variance Reduction
- Improving SAGA via a Probabilistic Interpolation with Gradient Descent
- Dual Free Adaptive Mini-batch SDCA for Empirical Risk Minimization
- Learning Under Delayed Feedback: Implicitly Adapting to Gradient Delays
- Large-Scale Stochastic Learning using GPUs