Stochastic Projective Splitting: Solving Saddle-Point Problems with Multiple Regularizers
arXiv:2106.13067
Abstract
We present a new, stochastic variant of the projective splitting (PS) family of algorithms for monotone inclusion problems. It can solve min-max and noncooperative game formulations arising in applications such as robust ML without the convergence issues associated with gradient descent-ascent, the current de facto standard approach in such situations. Our proposal is the first version of PS able to use stochastic (as opposed to deterministic) gradient oracles. It is also the first stochastic method that can solve min-max games while easily handling multiple constraints and nonsmooth regularizers via projection and proximal operators. We close with numerical experiments on a distributionally robust sparse logistic regression problem.
References in corpus (18)
- Searching for Exotic Particles in High-Energy Physics with Deep Learning
- Which Training Methods for GANs do actually Converge?
- The Numerics of GANs
- Distributionally Robust Logistic Regression
- Achieving Fairness through Adversarial Learning: an Application to Recidivism Prediction
- Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile
- Estimation of Simultaneously Sparse and Low Rank Matrices
- Censoring Representations with an Adversary
- Rare Feature Selection in High Dimensions
- Improved Adversarial Learning for Fair Classification
- Explore Aggressively, Update Conservatively: Stochastic Extragradient Methods with Variable Stepsize Scaling
- On the convergence of single-call stochastic extra-gradient methods
- Convergence Rates for Projective Splitting
- Forward-backward-forward methods with variance reduction for stochastic variational inequalities
- Stochastic Three-Composite Convex Minimization
- Generative Minimization Networks: Training GANs Without Competition
- Fast Distributionally Robust Learning with Variance Reduced Min-Max Optimization
- Proximal Splitting Meets Variance Reduction