SGD Learns One-Layer Networks in WGANs
arXiv:1910.07030
Abstract
Generative adversarial networks (GANs) are a widely used framework for learning generative models. Wasserstein GANs (WGANs), one of the most successful variants of GANs, require solving a minmax optimization problem to global optimality, but are in practice successfully trained using stochastic gradient descent-ascent. In this paper, we show that, when the generator is a one-layer network, stochastic gradient descent-ascent converges to a global solution with polynomial time and sample complexity.
24 pages, 4 figures, ICML2020
References in corpus (18)
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Progressive Growing of GANs for Improved Quality, Stability, and Variation
- Which Training Methods for GANs do actually Converge?
- Adversarially Learned Inference
- Compressed Sensing using Generative Models
- Low-rank optimization for semidefinite convex problems
- Generalization and Equilibrium in Generative Adversarial Nets (GANs)
- Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity
- On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems
- The Numerics of GANs
- Learning One-hidden-layer Neural Networks with Landscape Design
- What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization?
- On the Discrimination-Generalization Tradeoff in GANs
- Understanding GANs: the LQG Setting
- Approximability of Discriminators Implies Diversity in GANs
- How Well Generative Adversarial Networks Learn Distributions
- Last-Iterate Convergence: Zero-Sum Games and Constrained Min-Max Optimization
- Last iterate convergence in no-regret learning: constrained min-max optimization for convex-concave landscapes
Cited by in corpus (8)
- How Well Generative Adversarial Networks Learn Distributions
- GANs May Have No Nash Equilibria
- Minimax Estimation of Conditional Moment Models
- Understanding Overparameterization in Generative Adversarial Networks
- A mean-field analysis of two-player zero-sum games
- Forward Super-Resolution: How Can GANs Learn Hierarchical Generative Models for Real-World Distributions
- Making Method of Moments Great Again? -- How can GANs learn distributions
- Solving Min-Max Optimization with Hidden Structure via Gradient Descent Ascent