Generative Minimization Networks: Training GANs Without Competition

Abstract

Many applications in machine learning can be framed as minimization problems and solved efficiently using gradient-based techniques. However, recent applications of generative models, particularly GANs, have triggered interest in solving min-max games for which standard optimization techniques are often not suitable. Among known problems experienced by practitioners is the lack of convergence guarantees or convergence to a non-optimum cycle. At the heart of these problems is the min-max structure of the GAN objective which creates non-trivial dependencies between the players. We propose to address this problem by optimizing a different objective that circumvents the min-max structure using the notion of duality gap from game theory. We provide novel convergence guarantees on this objective and demonstrate why the obtained limit point solves the problem better than known techniques.

References in corpus (19)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Adversarial Feature Learning
• Adversarially Learned Inference
• Unrolled Generative Adversarial Networks
• On Convergence and Stability of GANs
• A Variational Inequality Perspective on Generative Adversarial Networks
• Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methods
• Non-convex Min-Max Optimization: Applications, Challenges, and Recent Theoretical Advances
• On Finding Local Nash Equilibria (and Only Local Nash Equilibria) in Zero-Sum Games
• What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization?
• Last-iterate convergence rates for min-max optimization
• A domain agnostic measure for monitoring and evaluating GANs
• Implicit competitive regularization in GANs
• Online to Offline Conversions, Universality and Adaptive Minibatch Sizes
• Primal-Dual Wasserstein GAN
• Investigating Under and Overfitting in Wasserstein Generative Adversarial Networks
• Training Generative Adversarial Networks via Primal-Dual Subgradient Methods: A Lagrangian Perspective on GAN

Cited by in corpus (3)

• Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
• Finding mixed-strategy equilibria of continuous-action games without gradients using randomized policy networks
• Stochastic Projective Splitting: Solving Saddle-Point Problems with Multiple Regularizers