Comparison of Maximum Likelihood and GAN-based training of Real NVPs

Abstract

We train a generator by maximum likelihood and we also train the same generator architecture by Wasserstein GAN. We then compare the generated samples, exact log-probability densities and approximate Wasserstein distances. We show that an independent critic trained to approximate Wasserstein distance between the validation set and the generator distribution helps detect overfitting. Finally, we use ideas from the one-shot learning literature to develop a novel fast learning critic.

References in corpus (11)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Improved Training of Wasserstein GANs
• Matching Networks for One Shot Learning
• Density estimation using Real NVP
• Kernel Mean Embedding of Distributions: A Review and Beyond
• A note on the evaluation of generative models
• Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks
• Towards Principled Methods for Training Generative Adversarial Networks
• One-shot Learning with Memory-Augmented Neural Networks
• Towards Conceptual Compression

Cited by in corpus (27)

• How Generative Adversarial Networks and Their Variants Work: An Overview
• Normalizing Flows for Probabilistic Modeling and Inference
• Guided Image Generation with Conditional Invertible Neural Networks
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• Variational Approaches for Auto-Encoding Generative Adversarial Networks
• Demystifying MMD GANs
• Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
• PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows
• Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting
• Is Generator Conditioning Causally Related to GAN Performance?
• Distribution Matching in Variational Inference
• Transferring GANs: generating images from limited data
• Justifying Diagnosis Decisions by Deep Neural Networks
• Fair Generative Modeling via Weak Supervision
• Quantitatively Evaluating GANs With Divergences Proposed for Training
• Training Invertible Neural Networks as Autoencoders
• Training Generative Reversible Networks
• Recursive Chaining of Reversible Image-to-image Translators For Face Aging
• Neural Density Estimation and Likelihood-free Inference
• A Method to Model Conditional Distributions with Normalizing Flows
• Filtering Variational Objectives
• Generative Adversarial Data Programming
• Lessons Learned from the Training of GANs on Artificial Datasets
• Restricting Greed in Training of Generative Adversarial Network
• GILBO: One Metric to Measure Them All
• Testing for Typicality with Respect to an Ensemble of Learned Distributions