Quantitatively Evaluating GANs With Divergences Proposed for Training

Abstract

Generative adversarial networks (GANs) have been extremely effective in approximating complex distributions of high-dimensional, input data samples, and substantial progress has been made in understanding and improving GAN performance in terms of both theory and application. However, we currently lack quantitative methods for model assessment. Because of this, while many GAN variants are being proposed, we have relatively little understanding of their relative abilities. In this paper, we evaluate the performance of various types of GANs using divergence and distance functions typically used only for training. We observe consistency across the various proposed metrics and, interestingly, the test-time metrics do not favour networks that use the same training-time criterion. We also compare the proposed metrics to human perceptual scores.

arXiv commentICLR 2018

References in corpus (17)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Improved Training of Wasserstein GANs
• NICE: Non-linear Independent Components Estimation
• Improved Techniques for Training GANs
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Energy-based Generative Adversarial Network
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Towards Principled Methods for Training Generative Adversarial Networks
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• On Convergence and Stability of GANs
• Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy
• Comparison of Maximum Likelihood and GAN-based training of Real NVPs
• An empirical analysis of the optimization of deep network loss surfaces

Cited by in corpus (3)

• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models
• Fair Generative Modeling via Weak Supervision