A Review of Learning with Deep Generative Models from Perspective of Graphical Modeling
arXiv:1808.01630
Abstract
This document aims to provide a review on learning with deep generative models (DGMs), which is an highly-active area in machine learning and more generally, artificial intelligence. This review is not meant to be a tutorial, but when necessary, we provide self-contained derivations for completeness. This review has two features. First, though there are different perspectives to classify DGMs, we choose to organize this review from the perspective of graphical modeling, because the learning methods for directed DGMs and undirected DGMs are fundamentally different. Second, we differentiate model definitions from model learning algorithms, since different learning algorithms can be applied to solve the learning problem on the same model, and an algorithm can be applied to learn different models. We thus separate model definition and model learning, with more emphasis on reviewing, differentiating and connecting different learning algorithms. We also discuss promising future research directions.
arXiv commentadd SN-GANs, SA-GANs, conditional generation (cGANs, AC-GANs). arXiv admin note: text overlap with arXiv:1606.00709, arXiv:1801.03558 by other authors
References in corpus (29)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Auto-Encoding Variational Bayes
- Conditional Generative Adversarial Nets
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Spectral Normalization for Generative Adversarial Networks
- Categorical Reparameterization with Gumbel-Softmax
- Neural Discrete Representation Learning
- Self-Attention Generative Adversarial Networks
- Conditional Image Synthesis With Auxiliary Classifier GANs
- Conditional Image Generation with PixelCNN Decoders
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- NICE: Non-linear Independent Components Estimation
- Variational Inference with Normalizing Flows
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- Adversarial Feature Learning
- Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
- Auxiliary Deep Generative Models
- Variational Lossy Autoencoder
- GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
- Neural Variational Inference and Learning in Belief Networks
- Learning in Implicit Generative Models
- Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
- Variational inference for Monte Carlo objectives
- Deep AutoRegressive Networks
- On Unifying Deep Generative Models
- Deep Directed Generative Models with Energy-Based Probability Estimation
- Auto-Encoding Sequential Monte Carlo
- Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference
- Joint Stochastic Approximation learning of Helmholtz Machines
Cited by in corpus (6)
- Generative models for sampling and phase transition indication in spin systems
- Concept-Oriented Deep Learning: Generative Concept Representations
- Latent Variable Modeling for Generative Concept Representations and Deep Generative Models
- Energy-Based Models with Applications to Speech and Language Processing
- Neural Network Statistical Mechanics
- Predictive Synthesis of Quantum Materials by Probabilistic Reinforcement Learning