Variational Approaches for Auto-Encoding Generative Adversarial Networks

Abstract

Auto-encoding generative adversarial networks (GANs) combine the standard GAN algorithm, which discriminates between real and model-generated data, with a reconstruction loss given by an auto-encoder. Such models aim to prevent mode collapse in the learned generative model by ensuring that it is grounded in all the available training data. In this paper, we develop a principle upon which auto-encoders can be combined with generative adversarial networks by exploiting the hierarchical structure of the generative model. The underlying principle shows that variational inference can be used a basic tool for learning, but with the in- tractable likelihood replaced by a synthetic likelihood, and the unknown posterior distribution replaced by an implicit distribution; both synthetic likelihoods and implicit posterior distributions can be learned using discriminators. This allows us to develop a natural fusion of variational auto-encoders and generative adversarial networks, combining the best of both these methods. We describe a unified objective for optimization, discuss the constraints needed to guide learning, connect to the wide range of existing work, and use a battery of tests to systematically and quantitatively assess the performance of our method.

References in corpus (20)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Improved Training of Wasserstein GANs
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Adversarial Feature Learning
• Adversarially Learned Inference
• A note on the evaluation of generative models
• Mode Regularized Generative Adversarial Networks
• Learning in Implicit Generative Models
• Hierarchical Implicit Models and Likelihood-Free Variational Inference
• On distinguishability criteria for estimating generative models
• Variational Inference using Implicit Distributions
• Generative Adversarial Nets from a Density Ratio Estimation Perspective
• Comparison of Maximum Likelihood and GAN-based training of Real NVPs
• Likelihood-free inference via classification
• Adversarial Message Passing For Graphical Models

Cited by in corpus (82)

• Generating Natural Adversarial Examples
• Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images
• How Generative Adversarial Networks and Their Variants Work: An Overview
• Large Scale Adversarial Representation Learning
• Recent Advances in Autoencoder-Based Representation Learning
• A Review on Deep Learning Techniques for Video Prediction
• Multimodal Unsupervised Image-to-Image Translation
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• A Note on the Inception Score
• RSGAN: Face Swapping and Editing using Face and Hair Representation in Latent Spaces
• Hierarchical Amortized Training for Memory-efficient High Resolution 3D GAN
• Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
• Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
• Learning Data Augmentation for Brain Tumor Segmentation with Coarse-to-Fine Generative Adversarial Networks
• FSNet: An Identity-Aware Generative Model for Image-based Face Swapping
• Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting
• Stabilizing Generative Adversarial Networks: A Survey
• Recent Developments Combining Ensemble Smoother and Deep Generative Networks for Facies History Matching
• Conditional Flow Variational Autoencoders for Structured Sequence Prediction
• Large-Scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55
• Yes, we GAN: Applying Adversarial Techniques for Autonomous Driving
• Distribution Matching in Variational Inference
• Prescribed Generative Adversarial Networks
• An advanced hybrid deep adversarial autoencoder for parameterized nonlinear fluid flow modelling
• FakeNews: GAN-based generation of realistic 3D volumetric data -- A systematic review and taxonomy
• Deep Generative Models in the Real-World: An Open Challenge from Medical Imaging
• Autoregressive Quantile Networks for Generative Modeling
• MGGAN: Solving Mode Collapse using Manifold Guided Training
• Adversarial Distillation of Bayesian Neural Network Posteriors
• Visual Intelligence through Human Interaction
• Sketch-based Creativity Support Tools using Deep Learning
• Improved Training of Generative Adversarial Networks Using Representative Features
• Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies
• Semi-supervised Image Attribute Editing using Generative Adversarial Networks
• Information bottleneck through variational glasses
• Adversarial Variational Optimization of Non-Differentiable Simulators
• Learning Disentangled Representations with Reference-Based Variational Autoencoders
• Hierarchical Adversarially Learned Inference
• O-GAN: Extremely Concise Approach for Auto-Encoding Generative Adversarial Networks
• Generative Adversarial Networks (GANs): What it can generate and What it cannot?
• Fair Generative Modeling via Weak Supervision
• Deep Generative Models for Distribution-Preserving Lossy Compression
• Self-Diagnosing GAN: Diagnosing Underrepresented Samples in Generative Adversarial Networks
• Lip-to-Speech Synthesis for Arbitrary Speakers in the Wild
• 3D Brain and Heart Volume Generative Models: A Survey
• Limb-Aware Virtual Try-On Network with Progressive Clothing Warping
• Parser-Free Virtual Try-on via Distilling Appearance Flows
• Generation of 3D Brain MRI Using Auto-Encoding Generative Adversarial Networks
• Unsupervised Anomaly Detection for X-Ray Images
• T2Net: Synthetic-to-Realistic Translation for Solving Single-Image Depth Estimation Tasks
• Manifold Modeling in Embedded Space: A Perspective for Interpreting Deep Image Prior
• Unsupervised Anomaly Detection with Adversarial Mirrored AutoEncoders
• Constraint-Based Visual Generation
• CAGAN: Text-To-Image Generation with Combined Attention GANs
• Data Augmentation For Medical MR Image Using Generative Adversarial Networks
• Detecting Overfitting via Adversarial Examples
• Generate High Resolution Images With Generative Variational Autoencoder
• Learning Implicit Generative Models with the Method of Learned Moments
• Statistical Parametric Speech Synthesis Using Generative Adversarial Networks Under A Multi-task Learning Framework
• Image Quality Assessment Techniques Show Improved Training and Evaluation of Autoencoder Generative Adversarial Networks
• Coconditional Autoencoding Adversarial Networks for Chinese Font Feature Learning
• Unpaired Image-to-Image Translation using Adversarial Consistency Loss
• DualDis: Dual-Branch Disentangling with Adversarial Learning
• DISSECT: Disentangled Simultaneous Explanations via Concept Traversals
• A General Framework for Revealing Human Mind with auto-encoding GANs
• Generative adversarial network based on chaotic time series
• Resembled Generative Adversarial Networks: Two Domains with Similar Attributes
• Inclusive GAN: Improving Data and Minority Coverage in Generative Models
• Training Generative Adversarial Networks in One Stage
• Inferential Wasserstein Generative Adversarial Networks
• Pairwise Augmented GANs with Adversarial Reconstruction Loss
• Auto-Embedding Generative Adversarial Networks for High Resolution Image Synthesis
• IID-GAN: an IID Sampling Perspective for Regularizing Mode Collapse
• Capturing the diversity of biological tuning curves using generative adversarial networks
• Learning a Representation Map for Robot Navigation using Deep Variational Autoencoder
• Sampling Using Neural Networks for colorizing the grayscale images
• Efficient Deep Representation Learning by Adaptive Latent Space Sampling
• MixPoet: Diverse Poetry Generation via Learning Controllable Mixed Latent Space
• Traversing Latent Space using Decision Ferns
• DINO: A Conditional Energy-Based GAN for Domain Translation
• WAYLA - Generating Images from Eye Movements
• Synthesizing Photorealistic Images with Deep Generative Learning