Sliced-Wasserstein Autoencoder: An Embarrassingly Simple Generative Model

Abstract

In this paper we study generative modeling via autoencoders while using the elegant geometric properties of the optimal transport (OT) problem and the Wasserstein distances. We introduce Sliced-Wasserstein Autoencoders (SWAE), which are generative models that enable one to shape the distribution of the latent space into any samplable probability distribution without the need for training an adversarial network or defining a closed-form for the distribution. In short, we regularize the autoencoder loss with the sliced-Wasserstein distance between the distribution of the encoded training samples and a predefined samplable distribution. We show that the proposed formulation has an efficient numerical solution that provides similar capabilities to Wasserstein Autoencoders (WAE) and Variational Autoencoders (VAE), while benefiting from an embarrassingly simple implementation.

References in corpus (8)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Improved Training of Wasserstein GANs
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Learning with a Wasserstein Loss
• Wasserstein Auto-Encoders
• From optimal transport to generative modeling: the VEGAN cookbook
• An efficient linear programming method for Optimal Transportation
• Image to Image Translation for Domain Adaptation

Cited by in corpus (24)

• Learning to Simulate High Energy Particle Collisions from Unlabeled Data
• Deep Generative Models for Detector Signature Simulation: A Taxonomic Review
• A survey of machine learning-based physics event generation
• Learning Autoencoders with Relational Regularization
• Cramer-Wold AutoEncoder
• Distributional Sliced-Wasserstein and Applications to Generative Modeling
• Training Generative Reversible Networks
• Sliced-Wasserstein Flows: Nonparametric Generative Modeling via Optimal Transport and Diffusions
• Secure Domain Adaptation with Multiple Sources
• Learning Implicit Generative Models with Theoretical Guarantees
• Sliced Iterative Normalizing Flows
• Augmented Sliced Wasserstein Distances
• Unified Shapley Framework to Explain Prediction Drift
• Generative Learning With Euler Particle Transport
• Sliced Multi-Marginal Optimal Transport
• Sound2Sight: Generating Visual Dynamics from Sound and Context
• Tessellated Wasserstein Auto-Encoders
• Learning Graphon Autoencoders for Generative Graph Modeling
• One-element Batch Training by Moving Window
• Hierarchical Reinforcement Learning with Optimal Level Synchronization Based on Flow-Based Deep Generative Model
• Efficient estimates of optimal transport via low-dimensional embeddings
• Wasserstein Neural Processes
• Learning to Detect Fortified Areas
• Hawkes Processes on Graphons