Efficient Projection-Free Algorithms for Saddle Point Problems

Abstract

The Frank-Wolfe algorithm is a classic method for constrained optimization problems. It has recently been popular in many machine learning applications because its projection-free property leads to more efficient iterations. In this paper, we study projection-free algorithms for convex-strongly-concave saddle point problems with complicated constraints. Our method combines Conditional Gradient Sliding with Mirror-Prox and shows that it only requires $\tilde{O}(1/\sqrt{\epsilon })$ gradient evaluations and $\tilde{O}(1/{\epsilon }^2)$ linear optimizations in the batch setting. We also extend our method to the stochastic setting and propose first stochastic projection-free algorithms for saddle point problems. Experimental results demonstrate the effectiveness of our algorithms and verify our theoretical guarantees.

References in corpus (12)

• On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems
• Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methods
• The Complexity of Large-scale Convex Programming under a Linear Optimization Oracle
• On the Computational Efficiency of Training Neural Networks
• Variance-Reduced and Projection-Free Stochastic Optimization
• Projection-free Online Learning
• Reducing Noise in GAN Training with Variance Reduced Extragradient
• An Affine Invariant Linear Convergence Analysis for Frank-Wolfe Algorithms
• Efficient Algorithms for Smooth Minimax Optimization
• Frank-Wolfe Algorithms for Saddle Point Problems
• Stochastic Conditional Gradient++
• Online and Bandit Algorithms for Nonstationary Stochastic Saddle-Point Optimization