Learning Deep Energy Models: Contrastive Divergence vs. Amortized MLE
arXiv:1707.00797
Abstract
We propose a number of new algorithms for learning deep energy models and demonstrate their properties. We show that our SteinCD performs well in term of test likelihood, while SteinGAN performs well in terms of generating realistic looking images. Our results suggest promising directions for learning better models by combining GAN-style methods with traditional energy-based learning.
References in corpus (10)
- Adam: A Method for Stochastic Optimization
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Improved Techniques for Training GANs
- LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
- Energy-based Generative Adversarial Network
- f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
- Learning to learn by gradient descent by gradient descent
- Deep Directed Generative Models with Energy-Based Probability Estimation
- Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning
- Generative Adversarial Networks as Variational Training of Energy Based Models
Cited by in corpus (10)
- Deep Energy Estimator Networks
- Improved Contrastive Divergence Training of Energy Based Models
- Minimum Stein Discrepancy Estimators
- Exponential Family Estimation via Adversarial Dynamics Embedding
- To Relieve Your Headache of Training an MRF, Take AdVIL
- Contrastive Divergence Learning is a Time Reversal Adversarial Game
- General Probabilistic Surface Optimization and Log Density Estimation
- Deep PDF: Probabilistic Surface Optimization and Density Estimation
- Bridging Explicit and Implicit Deep Generative Models via Neural Stein Estimators
- Learning Energy-Based Models as Generative ConvNets via Multi-grid Modeling and Sampling