LOGAN: Latent Optimisation for Generative Adversarial Networks

Abstract

Training generative adversarial networks requires balancing of delicate adversarial dynamics. Even with careful tuning, training may diverge or end up in a bad equilibrium with dropped modes. In this work, we improve CS-GAN with natural gradient-based latent optimisation and show that it improves adversarial dynamics by enhancing interactions between the discriminator and the generator. Our experiments demonstrate that latent optimisation can significantly improve GAN training, obtaining state-of-the-art performance for the ImageNet ($128\times 128$) dataset. Our model achieves an Inception Score (IS) of $148$ and an Fréchet Inception Distance (FID) of $3.4$, an improvement of $17\%$ and $32\%$ in IS and FID respectively, compared with the baseline BigGAN-deep model with the same architecture and number of parameters.

arXiv commentImproved writing, added new analysis and evaluation

References in corpus (22)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Self-Attention Generative Adversarial Networks
• Which Training Methods for GANs do actually Converge?
• MINE: Mutual Information Neural Estimation
• Unrolled Generative Adversarial Networks
• Large Scale Adversarial Representation Learning
• New insights and perspectives on the natural gradient method
• The Numerics of GANs
• Adversarial Video Generation on Complex Datasets
• Classification Accuracy Score for Conditional Generative Models
• The Mechanics of n-Player Differentiable Games
• Implicit Generation and Generalization in Energy-Based Models
• Geometric GAN
• Maximum Entropy Generators for Energy-Based Models
• Deep Compressed Sensing
• Prescribed Generative Adversarial Networks
• Differentiable Game Mechanics
• Global Convergence to the Equilibrium of GANs using Variational Inequalities
• Discriminator optimal transport

Cited by in corpus (21)

• Diffusion Models Beat GANs on Image Synthesis
• Cascaded Diffusion Models for High Fidelity Image Generation
• ContraGAN: Contrastive Learning for Conditional Image Generation
• Is Attention Better Than Matrix Decomposition?
• Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling
• Improved Consistency Regularization for GANs
• Top-k Training of GANs: Improving GAN Performance by Throwing Away Bad Samples
• Self-Diagnosing GAN: Diagnosing Underrepresented Samples in Generative Adversarial Networks
• Improving the Speed and Quality of GAN by Adversarial Training
• Group Equivariant Generative Adversarial Networks
• Latent-Optimized Adversarial Neural Transfer for Sarcasm Detection
• Generalized Energy Based Models
• Refining Deep Generative Models via Discriminator Gradient Flow
• Instance Selection for GANs
• Lessons Learned from the Training of GANs on Artificial Datasets
• Neural Function Modules with Sparse Arguments: A Dynamic Approach to Integrating Information across Layers
• A Limited-Capacity Minimax Theorem for Non-Convex Games or: How I Learned to Stop Worrying about Mixed-Nash and Love Neural Nets
• Unrealistic Feature Suppression for Generative Adversarial Networks
• Enhancing Stability in Training Conditional Generative Adversarial Networks via Selective Data Matching
• Class Balancing GAN with a Classifier in the Loop
• Complex Momentum for Optimization in Games