Finding Mixed Nash Equilibria of Generative Adversarial Networks
arXiv:1811.02002
Abstract
We reconsider the training objective of Generative Adversarial Networks (GANs) from the mixed Nash Equilibria (NE) perspective. Inspired by the classical prox methods, we develop a novel algorithmic framework for GANs via an infinite-dimensional two-player game and prove rigorous convergence rates to the mixed NE, resolving the longstanding problem that no provably convergent algorithm exists for general GANs. We then propose a principled procedure to reduce our novel prox methods to simple sampling routines, leading to practically efficient algorithms. Finally, we provide experimental evidence that our approach outperforms methods that seek pure strategy equilibria, such as SGD, Adam, and RMSProp, both in speed and quality.
References in corpus (10)
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Improved Training of Wasserstein GANs
- LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
- A Note on the Inception Score
- A Variational Inequality Perspective on Generative Adversarial Networks
- Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks
- Amortised MAP Inference for Image Super-resolution
- Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile
- Global Convergence to the Equilibrium of GANs using Variational Inequalities
- Negative Momentum for Improved Game Dynamics
Cited by in corpus (12)
- What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization?
- Stabilizing Generative Adversarial Networks: A Survey
- Invariant Risk Minimization Games
- GANs May Have No Nash Equilibria
- Fast and Provable ADMM for Learning with Generative Priors
- Zeroth-Order Algorithms for Nonconvex Minimax Problems with Improved Complexities
- Generalization Properties of Optimal Transport GANs with Latent Distribution Learning
- Infinite-dimensional gradient-based descent for alpha-divergence minimisation
- Investigating Under and Overfitting in Wasserstein Generative Adversarial Networks
- Online Sinkhorn: Optimal Transport distances from sample streams
- An Exponentially Converging Particle Method for the Mixed Nash Equilibrium of Continuous Games
- Towards Optimal Randomized Strategies in Adversarial Example Game