Which Training Methods for GANs do actually Converge?
arXiv:1801.04406
Abstract
Recent work has shown local convergence of GAN training for absolutely continuous data and generator distributions. In this paper, we show that the requirement of absolute continuity is necessary: we describe a simple yet prototypical counterexample showing that in the more realistic case of distributions that are not absolutely continuous, unregularized GAN training is not always convergent. Furthermore, we discuss regularization strategies that were recently proposed to stabilize GAN training. Our analysis shows that GAN training with instance noise or zero-centered gradient penalties converges. On the other hand, we show that Wasserstein-GANs and WGAN-GP with a finite number of discriminator updates per generator update do not always converge to the equilibrium point. We discuss these results, leading us to a new explanation for the stability problems of GAN training. Based on our analysis, we extend our convergence results to more general GANs and prove local convergence for simplified gradient penalties even if the generator and data distribution lie on lower dimensional manifolds. We find these penalties to work well in practice and use them to learn high-resolution generative image models for a variety of datasets with little hyperparameter tuning.
arXiv commentconference
Cited by in corpus (241)
- Training Generative Adversarial Networks with Limited Data
- AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control
- GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
- Improved Precision and Recall Metric for Assessing Generative Models
- Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review
- TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up
- Semantic Image Synthesis with Spatially-Adaptive Normalization
- Adaptive Diffusion Priors for Accelerated MRI Reconstruction
- StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis
- How to GAN LHC Events
- A Variational Inequality Perspective on Generative Adversarial Networks
- Swapping Autoencoder for Deep Image Manipulation
- Game-Theoretic Multiagent Reinforcement Learning
- Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow
- Contrastive Learning for Unpaired Image-to-Image Translation
- Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GAN
- Deep unsupervised learning of turbulence for inflow generation at various Reynolds numbers
- Review: Deep Learning in Electron Microscopy
- From Variational to Deterministic Autoencoders
- The Unusual Effectiveness of Averaging in GAN Training
- Differentiable Augmentation for Data-Efficient GAN Training
- ContraGAN: Contrastive Learning for Conditional Image Generation
- StarGAN v2: Diverse Image Synthesis for Multiple Domains
- Joint Discriminative and Generative Learning for Person Re-identification
- Few-Shot Unsupervised Image-to-Image Translation
- ViTGAN: Training GANs with Vision Transformers
- CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis
- Generative Steganography Network
- Stabilizing Generative Adversarial Networks: A Survey
- SofGAN: A Portrait Image Generator with Dynamic Styling
- A Systematic Survey of Regularization and Normalization in GANs
- On Leveraging Pretrained GANs for Generation with Limited Data
- LOGAN: Latent Optimisation for Generative Adversarial Networks
- On gradient regularizers for MMD GANs
- StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation
- Machine Learning for Physics-Informed Generation of Dispersed Multiphase Flow Using Generative Adversarial Networks
- DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregation
- medigan: a Python library of pretrained generative models for medical image synthesis
- COutfitGAN: Learning to Synthesize Compatible Outfits Supervised by Silhouette Masks and Fashion Styles
- Generating Multi-Categorical Samples with Generative Adversarial Networks
- Convergence of Learning Dynamics in Stackelberg Games
- Lipschitz Generative Adversarial Nets
- Lower Dimensional Kernels for Video Discriminators
- Last-iterate convergence rates for min-max optimization
- Active Domain Randomization
- Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
- Synthetic flow-based cryptomining attack generation through Generative Adversarial Networks
- A Closer Look at the Optimization Landscapes of Generative Adversarial Networks
- A Large-Scale Study on Regularization and Normalization in GANs
- PConv: Simple yet Effective Convolutional Layer for Generative Adversarial Network
- How Well Generative Adversarial Networks Learn Distributions
- Focal Frequency Loss for Image Reconstruction and Synthesis
- Feature Quantization Improves GAN Training
- Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data
- Arbitrary Talking Face Generation via Attentional Audio-Visual Coherence Learning
- Explore Aggressively, Update Conservatively: Stochastic Extragradient Methods with Variable Stepsize Scaling
- High-resolution Deep Convolutional Generative Adversarial Networks
- Global Convergence to the Equilibrium of GANs using Variational Inequalities
- On Catastrophic Forgetting and Mode Collapse in Generative Adversarial Networks
- MSG-GAN: Multi-Scale Gradients for Generative Adversarial Networks
- Can AI be enabled to dynamical downscaling? A Latent Diffusion Model to mimic km-scale COSMO5.0\_CLM9 simulations
- Mode Collapse and Regularity of Optimal Transportation Maps
- Image Outpainting and Harmonization using Generative Adversarial Networks
- Fast and Scalable Earth Texture Synthesis using Spatially Assembled Generative Adversarial Neural Networks
- Latent Denoising Diffusion GAN: Faster sampling, Higher image quality
- Fast and Provable ADMM for Learning with Generative Priors
- InfinityGAN: Towards Infinite-Pixel Image Synthesis
- Encoding Invariances in Deep Generative Models
- Accelerating Smooth Games by Manipulating Spectral Shapes
- GANeRF: Leveraging Discriminators to Optimize Neural Radiance Fields
- Smoothness and Stability in GANs
- Adversarial Variational Optimization of Non-Differentiable Simulators
- Towards a Better Understanding and Regularization of GAN Training Dynamics
- Unsupervised Discovery of Object Radiance Fields
- COVID-19 CT Image Synthesis with a Conditional Generative Adversarial Network
- A Generative Model for Texture Synthesis based on Optimal Transport between Feature Distributions
- CariGAN: Caricature Generation through Weakly Paired Adversarial Learning
- A Spectral Regularizer for Unsupervised Disentanglement
- Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learning
- InMoDeGAN: Interpretable Motion Decomposition Generative Adversarial Network for Video Generation
- Interactive Sketch & Fill: Multiclass Sketch-to-Image Translation
- Compressing PDF sets using generative adversarial networks
- Human Motion Prediction via Spatio-Temporal Inpainting
- Negative Momentum for Improved Game Dynamics
- An Imitation from Observation Approach to Transfer Learning with Dynamics Mismatch
- RGBD-GAN: Unsupervised 3D Representation Learning From Natural Image Datasets via RGBD Image Synthesis
- CGOF++: Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields
- Generative Adversarial Networks (GANs): What it can generate and What it cannot?
- Efficient Generation of Structured Objects with Constrained Adversarial Networks
- Self-Diagnosing GAN: Diagnosing Underrepresented Samples in Generative Adversarial Networks
- From Inference to Generation: End-to-end Fully Self-supervised Generation of Human Face from Speech
- Scalable Unbalanced Optimal Transport using Generative Adversarial Networks
- Manifold regularization with GANs for semi-supervised learning
- Generative Models as Distributions of Functions
- Content and Colour Distillation for Learning Image Translations with the Spatial Profile Loss
- SGD Learns One-Layer Networks in WGANs
- Training Generative Adversarial Networks by Solving Ordinary Differential Equations
- Training Deep Energy-Based Models with f-Divergence Minimization
- LatteGAN: Visually Guided Language Attention for Multi-Turn Text-Conditioned Image Manipulation
- Regularizing Generative Adversarial Networks under Limited Data
- GAN-powered Deep Distributional Reinforcement Learning for Resource Management in Network Slicing
- Text-only domain adaptation for end-to-end ASR using integrated text-to-mel-spectrogram generator
- The Spatially-Correlative Loss for Various Image Translation Tasks
- GraN-GAN: Piecewise Gradient Normalization for Generative Adversarial Networks
- COCO-GAN: Generation by Parts via Conditional Coordinating
- How does Lipschitz Regularization Influence GAN Training?
- Image-to-image Translation via Hierarchical Style Disentanglement
- An Improved Self-supervised GAN via Adversarial Training
- Task-Aware Feature Generation for Zero-Shot Compositional Learning
- MRI Banding Removal via Adversarial Training
- COCO-FUNIT: Few-Shot Unsupervised Image Translation with a Content Conditioned Style Encoder
- Dynamics of Fourier Modes in Torus Generative Adversarial Networks
- Understanding the Effectiveness of Lipschitz-Continuity in Generative Adversarial Nets
- Self-Supervised GAN to Counter Forgetting
- HyperInverter: Improving StyleGAN Inversion via Hypernetwork
- Group Equivariant Generative Adversarial Networks
- Non-saturating GAN training as divergence minimization
- Training Federated GANs with Theoretical Guarantees: A Universal Aggregation Approach
- Gradient Descent-Ascent Provably Converges to Strict Local Minmax Equilibria with a Finite Timescale Separation
- AniGAN: Style-Guided Generative Adversarial Networks for Unsupervised Anime Face Generation
- Neural Architecture Search for Generative Adversarial Networks: A Comprehensive Review and Critical Analysis
- Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective
- Self Sparse Generative Adversarial Networks
- Understanding and Stabilizing GANs' Training Dynamics with Control Theory
- Label-Noise Robust Generative Adversarial Networks
- Self-Supervised GANs via Auxiliary Rotation Loss
- Tdcgan: Temporal Dilated Convolutional Generative Adversarial Network for End-to-end Speech Enhancement
- PAGAN: Portfolio Analysis with Generative Adversarial Networks
- How Does GAN-based Semi-supervised Learning Work?
- Robust Imitation Learning from Noisy Demonstrations
- Train simultaneously, generalize better: Stability of gradient-based minimax learners
- Discriminator Feature-based Inference by Recycling the Discriminator of GANs
- Site-Specific Ground Motion Generative Model for Crustal Earthquakes in Japan Based on Generative Adversarial Networks
- PE-GAN: Prior Embedding GAN for PXD images at Belle II
- Benchmarks, Algorithms, and Metrics for Hierarchical Disentanglement
- Follow-the-Regularized-Leader Routes to Chaos in Routing Games
- Meta Learning for Few-Shot One-class Classification
- GAN Cocktail: mixing GANs without dataset access
- Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesis
- Poincaré Recurrence, Cycles and Spurious Equilibria in Gradient-Descent-Ascent for Non-Convex Non-Concave Zero-Sum Games
- Properties of f-divergences and f-GAN training
- ComicGAN: Text-to-Comic Generative Adversarial Network
- High Resolution Face Editing with Masked GAN Latent Code Optimization
- Generating artificial digital image correlation data using physics-guided adversarial networks
- First Order Generative Adversarial Networks
- An Acceleration Framework for High Resolution Image Synthesis
- Towards Principled Disentanglement for Domain Generalization
- SPG-VTON: Semantic Prediction Guidance for Multi-pose Virtual Try-on
- A Pragmatic Look at Deep Imitation Learning
- Unsupervised Eyeglasses Removal in the Wild
- Taming GANs with Lookahead-Minmax
- Disentangling representations of retinal images with generative models
- DeltaGAN: Towards Diverse Few-shot Image Generation with Sample-Specific Delta
- MatchingGAN: Matching-based Few-shot Image Generation
- Reciprocal Adversarial Learning via Characteristic Functions
- BalaGAN: Image Translation Between Imbalanced Domains via Cross-Modal Transfer
- CAM-GAN: Continual Adaptation Modules for Generative Adversarial Networks
- Efficient Learning of Generative Models via Finite-Difference Score Matching
- Cross-Sensor Adversarial Domain Adaptation of Landsat-8 and Proba-V images for Cloud Detection
- Improved Transformer for High-Resolution GANs
- GAN Memory with No Forgetting
- Hidden Convexity of Wasserstein GANs: Interpretable Generative Models with Closed-Form Solutions
- Where is the Fake? Patch-Wise Supervised GANs for Texture Inpainting
- Multiqubit state learning with entangling quantum generative adversarial networks
- A Mathematical Framework for Learning Probability Distributions
- Face Reconstruction with Variational Autoencoder and Face Masks
- Image-to-Image Translation with Low Resolution Conditioning
- Class-Distinct and Class-Mutual Image Generation with GANs
- Learning Neural Light Transport
- From Shadow Segmentation to Shadow Removal
- Adaptive Weighted Discriminator for Training Generative Adversarial Networks
- MPG: A Multi-ingredient Pizza Image Generator with Conditional StyleGANs
- Diverse Image Generation via Self-Conditioned GANs
- FaceDet3D: Facial Expressions with 3D Geometric Detail Prediction
- SLGAN: Style- and Latent-guided Generative Adversarial Network for Desirable Makeup Transfer and Removal
- Double Descent and Other Interpolation Phenomena in GANs
- Lessons Learned from the Training of GANs on Artificial Datasets
- Instance Selection for GANs
- The use of Generative Adversarial Networks to characterise new physics in multi-lepton final states at the LHC
- Curriculum Learning for Deep Generative Models with Clustering
- Generative Adversarial Networks and Adversarial Autoencoders: Tutorial and Survey
- Perceptual Image Super-Resolution with Progressive Adversarial Network
- Blur, Noise, and Compression Robust Generative Adversarial Networks
- Gradient penalty from a maximum margin perspective
- A Benchmarking Initiative for Audio-Domain Music Generation Using the Freesound Loop Dataset
- A Novel Framework for Selection of GANs for an Application
- not-so-BigGAN: Generating High-Fidelity Images on Small Compute with Wavelet-based Super-Resolution
- Alleviation of Gradient Exploding in GANs: Fake Can Be Real
- Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence
- Accelerated WGAN update strategy with loss change rate balancing
- Omni-GAN: On the Secrets of cGANs and Beyond
- Efficient Heuristic Generation for Robot Path Planning with Recurrent Generative Model
- Lipschitz Constrained GANs via Boundedness and Continuity
- Optimal 1-Wasserstein Distance for WGANs
- Semi-Anchored Multi-Step Gradient Descent Ascent Method for Structured Nonconvex-Nonconcave Composite Minimax Problems
- Unsupervised Object Learning via Common Fate
- Reparameterized Variational Divergence Minimization for Stable Imitation
- Reliable Estimation of KL Divergence using a Discriminator in Reproducing Kernel Hilbert Space
- SyMetric: Measuring the Quality of Learnt Hamiltonian Dynamics Inferred from Vision
- GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation
- Conditioning Trick for Training Stable GANs
- Minimax Optimization with Smooth Algorithmic Adversaries
- Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing
- Domain Partitioning Network
- Kernel Mean Matching for Content Addressability of GANs
- Semisupervised Adversarial Neural Networks for Cyber Security Transfer Learning
- Adaptive Divergence for Rapid Adversarial Optimization
- LatentKeypointGAN: Controlling Images via Latent Keypoints
- Amortised Learning by Wake-Sleep
- Understanding Entropic Regularization in GANs
- A Kernel Test for Causal Association via Noise Contrastive Backdoor Adjustment
- MixSyn: Learning Composition and Style for Multi-Source Image Synthesis
- Noise Robust Generative Adversarial Networks
- Convolutional generative adversarial imputation networks for spatio-temporal missing data in storm surge simulations
- Label Geometry Aware Discriminator for Conditional Generative Networks
- Topics to Avoid: Demoting Latent Confounds in Text Classification
- The Geometric Occam's Razor Implicit in Deep Learning
- Creation of Synthetic Networked PMU Data: A Generative Adversarial Network Approach
- Complex Momentum for Optimization in Games
- Capturing Evolution Genes for Time Series Data
- Generative Adversarial Network: Some Analytical Perspectives
- Convergence of GANs Training: A Game and Stochastic Control Methodology
- Synthesizing Photorealistic Images with Deep Generative Learning
- Efficient training for future video generation based on hierarchical disentangled representation of latent variables
- Stochastic Projective Splitting: Solving Saddle-Point Problems with Multiple Regularizers
- Generative Adversarial Learning via Kernel Density Discrimination
- WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points
- Generation and Simulation of Yeast Microscopy Imagery with Deep Learning
- Towards GANs' Approximation Ability
- NeurInt : Learning to Interpolate through Neural ODEs
- Local Stability and Performance of Simple Gradient Penalty mu-Wasserstein GAN
- Enhanced 3DMM Attribute Control via Synthetic Dataset Creation Pipeline
- Training End-to-end Single Image Generators without GANs
- The Benefits of Pairwise Discriminators for Adversarial Training
- Toward Zero-Shot Unsupervised Image-to-Image Translation
- Translate the Facial Regions You Like Using Region-Wise Normalization
- Toward a Generalization Metric for Deep Generative Models
- When Relation Networks meet GANs: Relation GANs with Triplet Loss
- FusedProp: Towards Efficient Training of Generative Adversarial Networks
- Analysis of Discriminator in RKHS Function Space for Kullback-Leibler Divergence Estimation
- Dual Projection Generative Adversarial Networks for Conditional Image Generation