Joint Training of Variational Auto-Encoder and Latent Energy-Based Model

Abstract

This paper proposes a joint training method to learn both the variational auto-encoder (VAE) and the latent energy-based model (EBM). The joint training of VAE and latent EBM are based on an objective function that consists of three Kullback-Leibler divergences between three joint distributions on the latent vector and the image, and the objective function is of an elegant symmetric and anti-symmetric form of divergence triangle that seamlessly integrates variational and adversarial learning. In this joint training scheme, the latent EBM serves as a critic of the generator model, while the generator model and the inference model in VAE serve as the approximate synthesis sampler and inference sampler of the latent EBM. Our experiments show that the joint training greatly improves the synthesis quality of the VAE. It also enables learning of an energy function that is capable of detecting out of sample examples for anomaly detection.

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Spectral Normalization for Generative Adversarial Networks
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• DRAW: A Recurrent Neural Network For Image Generation
• Energy-based Generative Adversarial Network
• Adversarial Feature Learning
• Adversarially Learned Inference
• Efficient GAN-Based Anomaly Detection
• Learning in Implicit Generative Models
• Implicit Generation and Generalization in Energy-Based Models
• Deep Directed Generative Models with Energy-Based Probability Estimation
• Stochastic Variational Video Prediction
• Maximum Entropy Generators for Energy-Based Models