An Online Learning Approach to Generative Adversarial Networks

Abstract

We consider the problem of training generative models with a Generative Adversarial Network (GAN). Although GANs can accurately model complex distributions, they are known to be difficult to train due to instabilities caused by a difficult minimax optimization problem. In this paper, we view the problem of training GANs as finding a mixed strategy in a zero-sum game. Building on ideas from online learning we propose a novel training method named Chekhov GAN 1 . On the theory side, we show that our method provably converges to an equilibrium for semi-shallow GAN architectures, i.e. architectures where the discriminator is a one layer network and the generator is arbitrary. On the practical side, we develop an efficient heuristic guided by our theoretical results, which we apply to commonly used deep GAN architectures. On several real world tasks our approach exhibits improved stability and performance compared to standard GAN training.

References in corpus (2)

• Adam: A Method for Stochastic Optimization
• NIPS 2016 Tutorial: Generative Adversarial Networks

Cited by in corpus (8)

• How Generative Adversarial Networks and Their Variants Work: An Overview
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Competitive Gradient Descent
• A Decentralized Parallel Algorithm for Training Generative Adversarial Nets
• Skill Rating for Generative Models
• CMB-GAN: Fast Simulations of Cosmic Microwave background anisotropy maps using Deep Learning
• Regret minimization in stochastic non-convex learning via a proximal-gradient approach
• Game of GANs: Game-Theoretical Models for Generative Adversarial Networks