A Review of Learning with Deep Generative Models from Perspective of Graphical Modeling

Abstract

This document aims to provide a review on learning with deep generative models (DGMs), which is an highly-active area in machine learning and more generally, artificial intelligence. This review is not meant to be a tutorial, but when necessary, we provide self-contained derivations for completeness. This review has two features. First, though there are different perspectives to classify DGMs, we choose to organize this review from the perspective of graphical modeling, because the learning methods for directed DGMs and undirected DGMs are fundamentally different. Second, we differentiate model definitions from model learning algorithms, since different learning algorithms can be applied to solve the learning problem on the same model, and an algorithm can be applied to learn different models. We thus separate model definition and model learning, with more emphasis on reviewing, differentiating and connecting different learning algorithms. We also discuss promising future research directions.

add SN-GANs, SA-GANs, conditional generation (cGANs, AC-GANs). arXiv admin note: text overlap with arXiv:1606.00709, arXiv:1801.03558 by other authors

References in corpus (10)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Conditional Generative Adversarial Nets
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• NICE: Non-linear Independent Components Estimation
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• Variational Lossy Autoencoder
• GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
• Learning in Implicit Generative Models
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks

Cited by in corpus (6)

• Generative models for sampling and phase transition indication in spin systems
• Concept-Oriented Deep Learning: Generative Concept Representations
• Latent Variable Modeling for Generative Concept Representations and Deep Generative Models
• Energy-Based Models with Applications to Speech and Language Processing
• Neural Network Statistical Mechanics
• Predictive Synthesis of Quantum Materials by Probabilistic Reinforcement Learning