Tutorial on Variational Autoencoders

Abstract

In just three years, Variational Autoencoders (VAEs) have emerged as one of the most popular approaches to unsupervised learning of complicated distributions. VAEs are appealing because they are built on top of standard function approximators (neural networks), and can be trained with stochastic gradient descent. VAEs have already shown promise in generating many kinds of complicated data, including handwritten digits, faces, house numbers, CIFAR images, physical models of scenes, segmentation, and predicting the future from static images. This tutorial introduces the intuitions behind VAEs, explains the mathematics behind them, and describes some empirical behavior. No prior knowledge of variational Bayesian methods is assumed.

References in corpus (4)

• Semi-Supervised Learning with Deep Generative Models
• DRAW: A Recurrent Neural Network For Image Generation
• Deep Convolutional Inverse Graphics Network
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap

Cited by in corpus (276)

• Time-Series Anomaly Detection Service at Microsoft
• Three scenarios for continual learning
• How Generative Adversarial Networks and Their Variants Work: An Overview
• Recent Advances in Autoencoder-Based Representation Learning
• An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series
• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• Deep Representation Learning of Patient Data from Electronic Health Records (EHR): A Systematic Review
• An Overview of Deep Semi-Supervised Learning
• Artificial Intelligence for Satellite Communication: A Review
• Generative replay with feedback connections as a general strategy for continual learning
• Towards a Robust Parameterization for Conditioning Facies Models Using Deep Variational Autoencoders and Ensemble Smoother
• High-throughput discovery of novel cubic crystal materials using deep generative neural networks
• AtomAI: A Deep Learning Framework for Analysis of Image and Spectroscopy Data in (Scanning) Transmission Electron Microscopy and Beyond
• Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases
• A learning-based method for solving ill-posed nonlinear inverse problems: a simulation study of Lung EIT
• Recent Advances in Deep Learning: An Overview
• Machine learning and AI-based approaches for bioactive ligand discovery and GPCR-ligand recognition
• TAC-GAN - Text Conditioned Auxiliary Classifier Generative Adversarial Network
• Review: Deep Learning in Electron Microscopy
• Scalable Population Synthesis with Deep Generative Modeling
• Machine learning with neural networks
• Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems
• Wireframe-Based UI Design Search Through Image Autoencoder
• Diagnosing and Enhancing VAE Models
• On the Origin of Deep Learning
• Interference Suppression Using Deep Learning: Current Approaches and Open Challenges
• Artificial Neural Networks for Photonic Applications: From Algorithms to Implementation
• Unsupervised machine learning of topological phase transitions from experimental data
• Neural Abstractive Text Summarization with Sequence-to-Sequence Models
• Transfer Learning via Contextual Invariants for One-to-Many Cross-Domain Recommendation
• Variational Autoencoders for Collaborative Filtering
• Recent Developments Combining Ensemble Smoother and Deep Generative Networks for Facies History Matching
• Learning Robust Recommenders through Cross-Model Agreement
• A Point Set Generation Network for 3D Object Reconstruction from a Single Image
• Adversarial-Residual-Coarse-Graining: Applying machine learning theory to systematic molecular coarse-graining
• Deep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesis
• A Novel Framework for Brain Tumor Detection Based on Convolutional Variational Generative Models
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• The continuous Bernoulli: fixing a pervasive error in variational autoencoders
• Yes, we GAN: Applying Adversarial Techniques for Autonomous Driving
• Attention-based Convolutional Autoencoders for 3D-Variational Data Assimilation
• Hybrid Classical-Quantum Autoencoder for Anomaly Detection
• Adversarial Examples - A Complete Characterisation of the Phenomenon
• Contrastive Variational Autoencoder Enhances Salient Features
• Physical Layer Authentication Based on Hierarchical Variational Auto-Encoder for Industrial Internet of Things
• Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences
• Generating Multivariate Load States Using a Conditional Variational Autoencoder
• ReCoG: A Deep Learning Framework with Heterogeneous Graph for Interaction-Aware Trajectory Prediction
• A physics-aware, probabilistic machine learning framework for coarse-graining high-dimensional systems in the Small Data regime
• Physics-aware Reduced-order Modeling of Transonic Flow via $\beta$-Variational Autoencoder
• A Survey on Radar-Based Fall Detection
• Learning from Few Samples: A Survey
• Recovering compressed images for automatic crack segmentation using generative models
• Deep Generative Models in the Real-World: An Open Challenge from Medical Imaging
• Deep Learning for Neuroimaging-based Diagnosis and Rehabilitation of Autism Spectrum Disorder: A Review
• Assume, Augment and Learn: Unsupervised Few-Shot Meta-Learning via Random Labels and Data Augmentation
• Generative data-driven approaches for stochastic subgrid parameterizations in an idealized ocean model
• 3D-LMNet: Latent Embedding Matching for Accurate and Diverse 3D Point Cloud Reconstruction from a Single Image
• Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks
• Generating Sentences by Editing Prototypes
• Deep Music Analogy Via Latent Representation Disentanglement
• Entity Abstraction in Visual Model-Based Reinforcement Learning
• Exploring particle dynamics during self-organization processes via rotationally invariant latent representations
• Decoding Molecular Graph Embeddings with Reinforcement Learning
• Pedestrian Attribute Recognition: A Survey
• Model-free Deep Reinforcement Learning for Urban Autonomous Driving
• Networked Time Series Imputation via Position-aware Graph Enhanced Variational Autoencoders
• Generative models for sampling and phase transition indication in spin systems
• Dynamical Hyperspectral Unmixing with Variational Recurrent Neural Networks
• Opportunities and Challenges of Deep Learning Methods for Electrocardiogram Data: A Systematic Review
• A Classification Supervised Auto-Encoder Based on Predefined Evenly-Distributed Class Centroids
• Revisiting Bayesian Autoencoders with MCMC
• Fast mesh denoising with data driven normal filtering using deep variational autoencoders
• Neural Language Generation: Formulation, Methods, and Evaluation
• Merger-Ringdown Consistency: A New Test of Strong Gravity using Deep Learning
• Electrocardiogram Generation and Feature Extraction Using a Variational Autoencoder
• In-Bed Human Pose Estimation from Unseen and Privacy-Preserving Image Domains
• Learning latent state representation for speeding up exploration
• What You See is Not What the Network Infers: Detecting Adversarial Examples Based on Semantic Contradiction
• Machines Learn to Infer Stellar Parameters Just by Looking at a Large Number of Spectra
• Particle Filter Networks with Application to Visual Localization
• Fundamental problems in statistical physics XIV: Lecture on Machine Learning
• xGEMs: Generating Examplars to Explain Black-Box Models
• Towards meaningful physics from generative models
• Deep Learning for Learning Graph Representations
• Neural Net Models for Open-Domain Discourse Coherence
• SCORES: Shape Composition with Recursive Substructure Priors
• Deep Predictive Policy Training using Reinforcement Learning
• Robust Variational Autoencoder
• Gaussian Mixture Variational Autoencoder with Contrastive Learning for Multi-Label Classification
• SRGAN: Training Dataset Matters
• Accelerating Bayesian microseismic event location with deep learning
• Deep Bayesian Reward Learning from Preferences
• Learning about learning by many-body systems
• Approximate Query Processing using Deep Generative Models
• Better accuracy with quantified privacy: representations learned via reconstructive adversarial network
• Latent space conditioning for improved classification and anomaly detection
• Multi-Modal Anomaly Detection for Unstructured and Uncertain Environments
• Deep Representation Learning for Social Network Analysis
• Modeling Continuous Time Sequences with Intermittent Observations using Marked Temporal Point Processes
• Training Invertible Neural Networks as Autoencoders
• Conditional out-of-sample generation for unpaired data using trVAE
• Text to Image Synthesis using Stacked Conditional Variational Autoencoders and Conditional Generative Adversarial Networks
• MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing
• Unsupervised Dialog Structure Learning
• Cloth-Changing Person Re-identification from A Single Image with Gait Prediction and Regularization
• Finding quadruply imaged quasars with machine learning. I. Methods
• Estimating the Euclidean quantum propagator with deep generative modeling of Feynman paths
• Signal retrieval with measurement system knowledge using variational generative model
• Deep Generative Model Driven Protein Folding Simulation
• Efficient Cyber Attacks Detection in Industrial Control Systems Using Lightweight Neural Networks and PCA
• Sparsity in Variational Autoencoders
• Machine Learning for Estimation and Control of Quantum Systems
• Relational Data Synthesis using Generative Adversarial Networks: A Design Space Exploration
• PuVAE: A Variational Autoencoder to Purify Adversarial Examples
• Detecting, Localising and Classifying Polyps from Colonoscopy Videos using Deep Learning
• ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech
• VaPar Synth -- A Variational Parametric Model for Audio Synthesis
• A Classifying Variational Autoencoder with Application to Polyphonic Music Generation
• Fork or Fail: Cycle-Consistent Training with Many-to-One Mappings
• Variational Prototype Replays for Continual Learning
• Autoencoder Image Interpolation by Shaping the Latent Space
• Natural Language Generation with Neural Variational Models
• MIDI-Sandwich2: RNN-based Hierarchical Multi-modal Fusion Generation VAE networks for multi-track symbolic music generation
• DeepPBM: Deep Probabilistic Background Model Estimation from Video Sequences
• Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction
• Describe What to Change: A Text-guided Unsupervised Image-to-Image Translation Approach
• Accelerated Discovery of Sustainable Building Materials
• Representation Learning with Autoencoders for Electronic Health Records: A Comparative Study
• Concept-Oriented Deep Learning: Generative Concept Representations
• Estimating densities with nonlinear support using Fisher-Gaussian kernels
• Learning a Multi-Modal Policy via Imitating Demonstrations with Mixed Behaviors
• Learning Visually Guided Latent Actions for Assistive Teleoperation
• Domain Embedded Multi-model Generative Adversarial Networks for Image-based Face Inpainting
• Deciphering the Scattering of Mechanically Driven Polymers using Deep Learning
• Intact-VAE: Estimating Treatment Effects under Unobserved Confounding
• Variational Deep Semantic Hashing for Text Documents
• Latent Variables on Spheres for Autoencoders in High Dimensions
• Guided Incremental Local Densification for Accelerated Sampling-based Motion Planning
• Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoders
• Machine Learning for Relaying Topology: Optimization of IoT Network with Energy Harvesting
• Factor Analysis, Probabilistic Principal Component Analysis, Variational Inference, and Variational Autoencoder: Tutorial and Survey
• Diffusion-Weighted Magnetic Resonance Brain Images Generation with Generative Adversarial Networks and Variational Autoencoders: A Comparison Study
• The Labeling Distribution Matrix (LDM): A Tool for Estimating Machine Learning Algorithm Capacity
• Searching a Database of Source Codes Using Contextualized Code Search
• Few-Shot Anomaly Detection for Polyp Frames from Colonoscopy
• Bi-level Feature Alignment for Versatile Image Translation and Manipulation
• Deep learning in bioinformatics: introduction, application, and perspective in big data era
• Combinational Q-Learning for Dou Di Zhu
• Learning latent representations for style control and transfer in end-to-end speech synthesis
• A Survey of Deep Facial Attribute Analysis
• Deciphering quantum fingerprints in electric conductance
• SHOT-VAE: Semi-supervised Deep Generative Models With Label-aware ELBO Approximations
• Reconstruction of 3D Porous Media From 2D Slices
• Millimeter Wave Channel Modeling via Generative Neural Networks
• Multi-Task Variational Information Bottleneck
• Generative Zero-shot Network Quantization
• On the Compressive Power of Boolean Threshold Autoencoders
• Variational Optimization for Quantum Problems using Deep Generative Networks
• Straight-Through Estimator as Projected Wasserstein Gradient Flow
• Automated Model Selection for Time-Series Anomaly Detection
• GIQA: Generated Image Quality Assessment
• Probabilistic Software Modeling: A Data-driven Paradigm for Software Analysis
• IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization
• MRI Pulse Sequence Integration for Deep-Learning Based Brain Metastasis Segmentation
• Representation Learning for Heterogeneous Information Networks via Embedding Events
• Data Efficient Voice Cloning from Noisy Samples with Domain Adversarial Training
• Unsupervised Dual Adversarial Learning for Anomaly Detection in Colonoscopy Video Frames
• An Uncertain Future: Forecasting from Static Images using Variational Autoencoders
• What Colour is Neural Noise?
• Hyperprior Induced Unsupervised Disentanglement of Latent Representations
• Segment-Based Credit Scoring Using Latent Clusters in the Variational Autoencoder
• Learning Global and Local Features of Normal Brain Anatomy for Unsupervised Abnormality Detection
• Robotic Motion Planning using Learned Critical Sources and Local Sampling
• Tutorial and Survey on Probabilistic Graphical Model and Variational Inference in Deep Reinforcement Learning
• An In-depth Summary of Recent Artificial Intelligence Applications in Drug Design
• Retrieve-Then-Adapt: Example-based Automatic Generation for Proportion-related Infographics
• Depth Structure Preserving Scene Image Generation
• On the Size and Width of the Decoder of a Boolean Threshold Autoencoder
• Group Anomaly Detection using Deep Generative Models
• Fully Unsupervised Diversity Denoising with Convolutional Variational Autoencoders
• LHC Hadronic Jet Generation Using Convolutional Variational Autoencoders with Normalizing Flows
• Automatic Differentiation Variational Inference with Mixtures
• Automated Label Generation for Time Series Classification with Representation Learning: Reduction of Label Cost for Training
• A Framework for Joint Unsupervised Learning of Cluster-Aware Embedding for Heterogeneous Networks
• RNN-based Generative Model for Fine-Grained Sketching
• A witness function based construction of discriminative models using Hermite polynomials
• Better sampling in explanation methods can prevent dieselgate-like deception
• HpRNet : Incorporating Residual Noise Modeling for Violin in a Variational Parametric Synthesizer
• Variational Encoders and Autoencoders : Information-theoretic Inference and Closed-form Solutions
• Variational Autoencoding the Lagrangian Trajectories of Particles in a Combustion System
• Laplacian Pyramid-like Autoencoder
• The ELBO of Variational Autoencoders Converges to a Sum of Three Entropies
• Learning a Hierarchical Latent-Variable Model of 3D Shapes
• Unsupervised Star Galaxy Classification with Cascade Variational Auto-Encoder
• An Alarm System For Segmentation Algorithm Based On Shape Model
• Human Pose Forecasting via Deep Markov Models
• Deep learning based inverse method for layout design
• Merging Deterministic Policy Gradient Estimations with Varied Bias-Variance Tradeoff for Effective Deep Reinforcement Learning
• Learning to Play by Imitating Humans
• Emulation with uncertainty quantification of regional sea-level change caused by the Antarctic Ice Sheet
• Discriminative Clustering for Robust Unsupervised Domain Adaptation
• Maximum Entropy Auto-Encoding
• Stable Object Reorientation using Contact Plane Registration
• A probabilistic database approach to autoencoder-based data cleaning
• An Introduction to Variational Inference
• Unsupervised Doppler Radar-Based Activity Recognition for e-Healthcare
• Effect of backdoor attacks over the complexity of the latent space distribution
• Deep generative demixing: Recovering Lipschitz signals from noisy subgaussian mixtures
• A Quaternion-Valued Variational Autoencoder
• Flow-based Generative Models for Learning Manifold to Manifold Mappings
• Convex Smoothed Autoencoder-Optimal Transport model
• MCENET: Multi-Context Encoder Network for Homogeneous Agent Trajectory Prediction in Mixed Traffic
• Off-the-shelf deep learning is not enough: parsimony, Bayes and causality
• Database development and exploration of microstructure versus process relationships using variational autoencoders
• Quantifying many-body learning far from equilibrium with representation learning
• Learning and Inference in Imaginary Noise Models
• Scalable Modeling of Spatiotemporal Data using the Variational Autoencoder: an Application in Glaucoma
• Image De-Quantization Using Generative Models as Priors
• Generative networks as inverse problems with fractional wavelet scattering networks
• Mosaic: A Sample-Based Database System for Open World Query Processing
• Partially-Shared Variational Auto-encoders for Unsupervised Domain Adaptation with Target Shift
• Performance Evaluation of Deep Generative Models for Generating Hand-Written Character Images
• Lie Group Auto-Encoder
• Automatic dimensionality selection for principal component analysis models with the ignorance score
• Learning a Representation Map for Robot Navigation using Deep Variational Autoencoder
• Variational Auto-Encoder Based Approximate Bayesian Computation Uncertian Inverse Method for Sheet Metal Forming Problem
• Self-Net: Lifelong Learning via Continual Self-Modeling
• Unsupervised Representation Adversarial Learning Network: from Reconstruction to Generation
• Enhanced Image Classification With Data Augmentation Using Position Coordinates
• Generative-Discriminative Variational Model for Visual Recognition
• Capturing the diversity of biological tuning curves using generative adversarial networks
• Data augmentation using generative networks to identify dementia
• Understanding in Artificial Intelligence
• Variational Embeddings for Community Detection and Node Representation
• Neural Embedding for Physical Manipulations
• A Tutorial on the Mathematical Model of Single Cell Variational Inference
• Learn Proportional Derivative Controllable Latent Space from Pixels
• Contributions to Representation Learning with Graph Autoencoders and Applications to Music Recommendation
• Prior Flow Variational Autoencoder: A density estimation model for Non-Intrusive Load Monitoring
• Authoring image decompositions with generative models
• Unstructured Primary Outcome in Randomized Controlled Trials
• End-To-End Dilated Variational Autoencoder with Bottleneck Discriminative Loss for Sound Morphing -- A Preliminary Study
• DSBERT:Unsupervised Dialogue Structure learning with BERT
• Dynamic Variational Autoencoders for Visual Process Modeling
• $\beta$-Intact-VAE: Identifying and Estimating Causal Effects under Limited Overlap
• Generating the support with extreme value losses
• MeronymNet: A Hierarchical Approach for Unified and Controllable Multi-Category Object Generation
• Neural network based order parameter for phase transitions and its applications in high-entropy alloys
• Label-Removed Generative Adversarial Networks Incorporating with K-Means
• Learning from Multiple Noisy Augmented Data Sets for Better Cross-Lingual Spoken Language Understanding
• Subitizing with Variational Autoencoders
• Structure-Aware Feature Generation for Zero-Shot Learning
• Deep Generative Models: Deterministic Prediction with an Application in Inverse Rendering
• Introducing a Generative Adversarial Network Model for Lagrangian Trajectory Simulation
• Cascade Decoders-Based Autoencoders for Image Reconstruction
• Tool- and Domain-Agnostic Parameterization of Style Transfer Effects Leveraging Pretrained Perceptual Metrics
• To each route its own ETA: A generative modeling framework for ETA prediction
• Learning from imperfections: constructing phase diagrams from atomic imaging of fluctuations
• Counterfactual Explanations via Latent Space Projection and Interpolation
• Be More Active! Understanding the Differences between Mean and Sampled Representations of Variational Autoencoders
• On the Use of Time Series Kernel and Dimensionality Reduction to Identify the Acquisition of Antimicrobial Multidrug Resistance in the Intensive Care Unit
• Learning to Conceal: A Deep Learning Based Method for Preserving Privacy and Avoiding Prejudice
• OPAL-Net: A Generative Model for Part-based Object Layout Generation
• Deep Fiducial Inference
• Image-to-image Transformation with Auxiliary Condition
• Network representation learning: A macro and micro view
• Multiple Imputation for Biomedical Data using Monte Carlo Dropout Autoencoders
• Maximum likelihood estimation of the Fisher-Bingham distribution via efficient calculation of its normalizing constant
• A Deeper Look at the Unsupervised Learning of Disentangled Representations in $\beta$-VAE from the Perspective of Core Object Recognition
• A Generative Learning Approach for Spatio-temporal Modeling in Connected Vehicular Network
• Decoupled Variational Embedding for Signed Directed Networks
• Learning more expressive joint distributions in multimodal variational methods
• Deep Generative Variational Autoencoding for Replay Spoof Detection in Automatic Speaker Verification
• Applications of Generative Adversarial Models in Visual Search Reformulation
• GIM: Gaussian Isolation Machines