Wasserstein Auto-Encoders

Abstract

We propose the Wasserstein Auto-Encoder (WAE)---a new algorithm for building a generative model of the data distribution. WAE minimizes a penalized form of the Wasserstein distance between the model distribution and the target distribution, which leads to a different regularizer than the one used by the Variational Auto-Encoder (VAE). This regularizer encourages the encoded training distribution to match the prior. We compare our algorithm with several other techniques and show that it is a generalization of adversarial auto-encoders (AAE). Our experiments show that WAE shares many of the properties of VAEs (stable training, encoder-decoder architecture, nice latent manifold structure) while generating samples of better quality, as measured by the FID score.

Published at ICLR 2018.. Included much wider hyperparameter sweep: in significant improvements in FIDs on CelebA

References in corpus (4)

• Energy-based Generative Adversarial Network
• Generative Moment Matching Networks
• Training generative neural networks via Maximum Mean Discrepancy optimization
• From optimal transport to generative modeling: the VEGAN cookbook

Cited by in corpus (114)

• Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Implicit Quantile Networks for Distributional Reinforcement Learning
• Learning Factorized Multimodal Representations
• Learning Model-Agnostic Counterfactual Explanations for Tabular Data
• DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder
• From Variational to Deterministic Autoencoders
• ODE${}^2$VAE: Deep generative second order ODEs with Bayesian neural networks
• Generative Models for Automatic Chemical Design
• Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
• Hypernetwork functional image representation
• Conditional Flow Variational Autoencoders for Structured Sequence Prediction
• Sliced-Wasserstein Autoencoder: An Embarrassingly Simple Generative Model
• Social-WaGDAT: Interaction-aware Trajectory Prediction via Wasserstein Graph Double-Attention Network
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• Prescribed Generative Adversarial Networks
• The Information Autoencoding Family: A Lagrangian Perspective on Latent Variable Generative Models
• Educating Text Autoencoders: Latent Representation Guidance via Denoising
• Revisiting Bayesian Autoencoders with MCMC
• Poincaré Wasserstein Autoencoder
• The Deep Kernelized Autoencoder
• Uncertainty Quantification with Generative Models
• Generative Latent Flow
• Deep Learning for Learning Graph Representations
• Out-of-distribution Prediction with Invariant Risk Minimization: The Limitation and An Effective Fix
• Cramer-Wold AutoEncoder
• Structured Variational Inference for Simulating Populations of Radio Galaxies
• Intrinsically Motivated Discovery of Diverse Patterns in Self-Organizing Systems
• Puzzle-AE: Novelty Detection in Images through Solving Puzzles
• Multi-Domain Translation by Learning Uncoupled Autoencoders
• Gaussian Word Embedding with a Wasserstein Distance Loss
• Deep Extrapolation for Attribute-Enhanced Generation
• Towards Interpretable Sparse Graph Representation Learning with Laplacian Pooling
• X-ray Study of Spatial Structures in Tycho's Supernova Remnant Using Unsupervised Deep Learning
• Consistency Regularization for Variational Auto-Encoders
• Latent Variable Modelling with Hyperbolic Normalizing Flows
• Distributional Sliced-Wasserstein and Applications to Generative Modeling
• (q,p)-Wasserstein GANs: Comparing Ground Metrics for Wasserstein GANs
• Accelerating Monte Carlo Bayesian Inference via Approximating Predictive Uncertainty over Simplex
• Regularized Autoencoders via Relaxed Injective Probability Flow
• Asymptotic Guarantees for Learning Generative Models with the Sliced-Wasserstein Distance
• The Benchmark Lottery
• Assisted Sound Sample Generation with Musical Conditioning in Adversarial Auto-Encoders
• Sliced-Wasserstein Flows: Nonparametric Generative Modeling via Optimal Transport and Diffusions
• DT-LET: Deep Transfer Learning by Exploring where to Transfer
• Variational Autoencoder with Implicit Optimal Priors
• Stochastic Adversarial Koopman Model for Dynamical Systems
• HyperFlow: Representing 3D Objects as Surfaces
• Learning to Synthesize Fashion Textures
• Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems
• Optimal Transport Classifier: Defending Against Adversarial Attacks by Regularized Deep Embedding
• Anomaly scores for generative models
• Double Backpropagation for Training Autoencoders against Adversarial Attack
• A gradual, semi-discrete approach to generative network training via explicit Wasserstein minimization
• The Level Weighted Structural Similarity Loss: A Step Away from the MSE
• MRI Pulse Sequence Integration for Deep-Learning Based Brain Metastasis Segmentation
• Sliced Iterative Normalizing Flows
• Learning Latent Space Energy-Based Prior Model
• Curriculum By Smoothing
• Image Hashing by Minimizing Discrete Component-wise Wasserstein Distance
• Neural Drum Machine : An Interactive System for Real-time Synthesis of Drum Sounds
• Continual Learning from the Perspective of Compression
• Uncertainty-guided Model Generalization to Unseen Domains
• Neural Monge Map estimation and its applications
• An In-depth Summary of Recent Artificial Intelligence Applications in Drug Design
• Generating various airfoil shapes with required lift coefficient using conditional variational autoencoders
• Efficient Learning of Generative Models via Finite-Difference Score Matching
• Wasserstein Autoencoders for Collaborative Filtering
• Generated Loss, Augmented Training, and Multiscale VAE
• Batch norm with entropic regularization turns deterministic autoencoders into generative models
• Scalable Topological Data Analysis and Visualization for Evaluating Data-Driven Models in Scientific Applications
• A New Framework for Query Efficient Active Imitation Learning
• Exploring Generative Physics Models with Scientific Priors in Inertial Confinement Fusion
• Uniform Priors for Data-Efficient Transfer
• Double cycle-consistent generative adversarial network for unsupervised conditional generation
• Radon Sobolev Variational Auto-Encoders
• Ranking Neural Checkpoints
• Adversarial Networks and Autoencoders: The Primal-Dual Relationship and Generalization Bounds
• A Generative Approach Towards Improved Robotic Detection of Marine Litter
• D2C: Diffusion-Denoising Models for Few-shot Conditional Generation
• Modeling 3D Surface Manifolds with a Locally Conditioned Atlas
• A Generative Map for Image-based Camera Localization
• Tessellated Wasserstein Auto-Encoders
• SimpleChrome: Encoding of Combinatorial Effects for Predicting Gene Expression
• MaskAAE: Latent space optimization for Adversarial Auto-Encoders
• Cauchy-Schwarz Regularized Autoencoder
• Topic Modeling with Wasserstein Autoencoders
• Variational Mutual Information Maximization Framework for VAE Latent Codes with Continuous and Discrete Priors
• Nonparametric Score Estimators
• Neural Topic Modeling by Incorporating Document Relationship Graph
• Low-Complexity Data-Parallel Earth Mover's Distance Approximations
• LDC-VAE: A Latent Distribution Consistency Approach to Variational AutoEncoders
• AriEL: volume coding for sentence generation
• Multi-view Alignment and Generation in CCA via Consistent Latent Encoding
• Zero-shot Singing Technique Conversion
• Learning Robust Decision Policies from Observational Data
• One-element Batch Training by Moving Window
• Polyline Generative Navigable Space Segmentation for Autonomous Visual Navigation
• Momentum Contrastive Autoencoder: Using Contrastive Learning for Latent Space Distribution Matching in WAE
• CWAE-IRL: Formulating a supervised approach to Inverse Reinforcement Learning problem
• WiSE-ALE: Wide Sample Estimator for Approximate Latent Embedding
• Multi-task deep-learning optimization of trade-off properties for superior-performance Fe-based soft magnetic alloys
• Scalable Personalised Item Ranking through Parametric Density Estimation
• Wavelets to the Rescue: Improving Sample Quality of Latent Variable Deep Generative Models
• Stylized Text Generation Using Wasserstein Autoencoders with a Mixture of Gaussian Prior
• A general framework for defining and optimizing robustness
• Neuron Coverage-Guided Domain Generalization
• Learned Interpolation for 3D Generation
• Robust W-GAN-Based Estimation Under Wasserstein Contamination
• Knowledge Generation -- Variational Bayes on Knowledge Graphs
• A Wasserstein Minimum Velocity Approach to Learning Unnormalized Models
• Quantile Propagation for Wasserstein-Approximate Gaussian Processes
• Dual Adversarial Variational Embedding for Robust Recommendation
• PIE: Pseudo-Invertible Encoder
• Neighbor Embedding Variational Autoencoder