Dual Free Adaptive Mini-batch SDCA for Empirical Risk Minimization
arXiv:1510.06684
Abstract
In this paper we develop dual free mini-batch SDCA with adaptive probabilities for regularized empirical risk minimization. This work is motivated by recent work of Shai Shalev-Shwartz on dual free SDCA method, however, we allow a non-uniform selection of "dual" coordinates in SDCA. Moreover, the probability can change over time, making it more efficient than fix uniform or non-uniform selection. We also propose an efficient procedure to generate a random non-uniform mini-batch through iterative process. The work is concluded with multiple numerical experiments to show the efficiency of proposed algorithms.
References in corpus (6)
- SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
- A Stochastic Gradient Method with an Exponential Convergence Rate for Finite Training Sets
- Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting
- SDCA without Duality
- Distributed Mini-Batch SDCA
- Primal Method for ERM with Flexible Mini-batching Schemes and Non-convex Losses