Locally Accelerated Conditional Gradients

Abstract

Conditional gradients constitute a class of projection-free first-order algorithms for smooth convex optimization. As such, they are frequently used in solving smooth convex optimization problems over polytopes, for which the computational cost of orthogonal projections would be prohibitive. However, they do not enjoy the optimal convergence rates achieved by projection-based accelerated methods; moreover, achieving such globally-accelerated rates is information-theoretically impossible for these methods. To address this issue, we present Locally Accelerated Conditional Gradients -- an algorithmic framework that couples accelerated steps with conditional gradient steps to achieve local acceleration on smooth strongly convex problems. Our approach does not require projections onto the feasible set, but only on (typically low-dimensional) simplices, thus keeping the computational cost of projections at bay. Further, it achieves the optimal accelerated local convergence. Our theoretical results are supported by numerical experiments, which demonstrate significant speedups of our framework over state of the art methods in both per-iteration progress and wall-clock time.

References in corpus (11)

• Convergence Rates of Inexact Proximal-Gradient Methods for Convex Optimization
• A Universal Catalyst for First-Order Optimization
• A geometric alternative to Nesterov's accelerated gradient descent
• The Complexity of Large-scale Convex Programming under a Linear Optimization Oracle
• The Approximate Duality Gap Technique: A Unified Theory of First-Order Methods
• On Acceleration with Noise-Corrupted Gradients
• On Symplectic Optimization
• Linear-memory and Decomposition-invariant Linearly Convergent Conditional Gradient Algorithm for Structured Polytopes
• Conditional Accelerated Lazy Stochastic Gradient Descent
• Blended Conditional Gradients: the unconditioning of conditional gradients
• Improved Complexities of Conditional Gradient-Type Methods with Applications to Robust Matrix Recovery Problems