Linear Convergence of Stochastic Frank Wolfe Variants
arXiv:1703.07269
Abstract
In this paper, we show that the Away-step Stochastic Frank-Wolfe Algorithm (ASFW) and Pairwise Stochastic Frank-Wolfe algorithm (PSFW) converge linearly in expectation. We also show that if an algorithm convergences linearly in expectation then it converges linearly almost surely. In order to prove these results, we develop a novel proof technique based on concepts of empirical processes and concentration inequalities. Such a technique has rarely been used to derive the convergence rates of stochastic optimization algorithms. In large-scale numerical experiments, ASFW and PSFW perform as well as or better than their stochastic competitors in actual CPU time.
arXiv commentAISTAT 2017
Cited by in corpus (6)
- Stochastic L-BFGS: Improved Convergence Rates and Practical Acceleration Strategies
- Stochastic Frank-Wolfe for Constrained Finite-Sum Minimization
- Revisiting Projection-free Online Learning: the Strongly Convex Case
- Revisiting Frank-Wolfe for Polytopes: Strict Complementarity and Sparsity
- Sparse Optimization on General Atomic Sets: Greedy and Forward-Backward Algorithms
- MEMOIR: Multi-class Extreme Classification with Inexact Margin