Is Generator Conditioning Causally Related to GAN Performance?

Abstract

Recent work (Pennington et al, 2017) suggests that controlling the entire distribution of Jacobian singular values is an important design consideration in deep learning. Motivated by this, we study the distribution of singular values of the Jacobian of the generator in Generative Adversarial Networks (GANs). We find that this Jacobian generally becomes ill-conditioned at the beginning of training. Moreover, we find that the average (with z from p(z)) conditioning of the generator is highly predictive of two other ad-hoc metrics for measuring the 'quality' of trained GANs: the Inception Score and the Frechet Inception Distance (FID). We test the hypothesis that this relationship is causal by proposing a 'regularization' technique (called Jacobian Clamping) that softly penalizes the condition number of the generator Jacobian. Jacobian Clamping improves the mean Inception Score and the mean FID for GANs trained on several datasets. It also greatly reduces inter-run variance of the aforementioned scores, addressing (at least partially) one of the main criticisms of GANs.

References in corpus (16)

• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Spectral Normalization for Generative Adversarial Networks
• Self-Attention Generative Adversarial Networks
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Improved Training of Wasserstein GANs
• NICE: Non-linear Independent Components Estimation
• Density estimation using Real NVP
• cGANs with Projection Discriminator
• Unrolled Generative Adversarial Networks
• Sensitivity and Generalization in Neural Networks: an Empirical Study
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• Do GANs actually learn the distribution? An empirical study
• Gradient descent GAN optimization is locally stable
• Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
• Comparison of Maximum Likelihood and GAN-based training of Real NVPs

Cited by in corpus (17)

• EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning
• Large Scale Image Completion via Co-Modulated Generative Adversarial Networks
• TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
• On Self Modulation for Generative Adversarial Networks
• A Systematic Survey of Regularization and Normalization in GANs
• Lipschitz Generative Adversarial Nets
• Improved Consistency Regularization for GANs
• Negative Momentum for Improved Game Dynamics
• Globally Injective ReLU Networks
• A Brief Introduction to Generative Models
• Regularized Autoencoders via Relaxed Injective Probability Flow
• Understanding the Effectiveness of Lipschitz-Continuity in Generative Adversarial Nets
• Convolutional Normalization: Improving Deep Convolutional Network Robustness and Training
• MR-GAN: Manifold Regularized Generative Adversarial Networks
• Causal Adversarial Network for Learning Conditional and Interventional Distributions
• Adversarial sampling of unknown and high-dimensional conditional distributions
• Inducing Optimal Attribute Representations for Conditional GANs