Implicit Maximum Likelihood Estimation

Abstract

Implicit probabilistic models are models defined naturally in terms of a sampling procedure and often induces a likelihood function that cannot be expressed explicitly. We develop a simple method for estimating parameters in implicit models that does not require knowledge of the form of the likelihood function or any derived quantities, but can be shown to be equivalent to maximizing likelihood under some conditions. Our result holds in the non-asymptotic parametric setting, where both the capacity of the model and the number of data examples are finite. We also demonstrate encouraging experimental results.

arXiv comment21 pages, 4 figures. In the interest of promoting discussion, we make the reviews available at https://people.eecs.berkeley.edu/~ke.li/papers/imle_reviews.pdf

References in corpus (21)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Pixel Recurrent Neural Networks
• Improved Techniques for Training GANs
• Energy-based Generative Adversarial Network
• Adversarial Feature Learning
• Adversarially Learned Inference
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• A note on the evaluation of generative models
• Generative Moment Matching Networks
• Towards Principled Methods for Training Generative Adversarial Networks
• Deep Generative Stochastic Networks Trainable by Backprop
• A Kernel Method for the Two-Sample Problem
• Better Mixing via Deep Representations
• How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?
• Learning in Implicit Generative Models
• Do GANs actually learn the distribution? An empirical study
• Boundary-Seeking Generative Adversarial Networks
• Likelihood-free inference via classification
• Fast k-Nearest Neighbour Search via Prioritized DCI
• Non-parametric estimation of Jensen-Shannon Divergence in Generative Adversarial Network training

Cited by in corpus (21)

• Deep Learning for Single Image Super-Resolution: A Brief Review
• You Only Need Adversarial Supervision for Semantic Image Synthesis
• Responsible Disclosure of Generative Models Using Scalable Fingerprinting
• Generative Latent Flow
• On the Implicit Assumptions of GANs
• Diverse Image Synthesis from Semantic Layouts via Conditional IMLE
• FakePolisher: Making DeepFakes More Detection-Evasive by Shallow Reconstruction
• Diversifying Semantic Image Synthesis and Editing via Class- and Layer-wise VAEs
• Dodging DeepFake Detection via Implicit Spatial-Domain Notch Filtering
• Inclusive GAN: Improving Data and Minority Coverage in Generative Models
• Super-Resolution via Conditional Implicit Maximum Likelihood Estimation
• Batch norm with entropic regularization turns deterministic autoencoders into generative models
• A Forest from the Trees: Generation through Neighborhoods
• Towards A Multi-agent System for Online Hate Speech Detection
• Generating Unobserved Alternatives
• Contrastive Mixture of Posteriors for Counterfactual Inference, Data Integration and Fairness
• Exploiting the Hidden Tasks of GANs: Making Implicit Subproblems Explicit
• Training End-to-end Single Image Generators without GANs
• Testing for Typicality with Respect to an Ensemble of Learned Distributions
• PixelTransformer: Sample Conditioned Signal Generation
• Cascading Modular Network (CAM-Net) for Multimodal Image Synthesis