Distributional Sliced-Wasserstein and Applications to Generative Modeling

Abstract

Sliced-Wasserstein distance (SW) and its variant, Max Sliced-Wasserstein distance (Max-SW), have been used widely in the recent years due to their fast computation and scalability even when the probability measures lie in a very high dimensional space. However, SW requires many unnecessary projection samples to approximate its value while Max-SW only uses the most important projection, which ignores the information of other useful directions. In order to account for these weaknesses, we propose a novel distance, named Distributional Sliced-Wasserstein distance (DSW), that finds an optimal distribution over projections that can balance between exploring distinctive projecting directions and the informativeness of projections themselves. We show that the DSW is a generalization of Max-SW, and it can be computed efficiently by searching for the optimal push-forward measure over a set of probability measures over the unit sphere satisfying certain regularizing constraints that favor distinct directions. Finally, we conduct extensive experiments with large-scale datasets to demonstrate the favorable performances of the proposed distances over the previous sliced-based distances in generative modeling applications.

References in corpus (13)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Adversarially Learned Inference
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration
• Wasserstein Auto-Encoders
• Scaling Algorithms for Unbalanced Transport Problems
• Sliced-Wasserstein Autoencoder: An Embarrassingly Simple Generative Model
• On Efficient Optimal Transport: An Analysis of Greedy and Accelerated Mirror Descent Algorithms
• Subspace Robust Wasserstein Distances
• Cramer-Wold AutoEncoder
• Orthogonal Estimation of Wasserstein Distances
• Strong equivalence between metrics of Wasserstein type
• Augmented Sliced Wasserstein Distances

Cited by in corpus (4)

• Minimax Confidence Intervals for the Sliced Wasserstein Distance
• Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey
• Projection Robust Wasserstein Distance and Riemannian Optimization
• Encoded Prior Sliced Wasserstein AutoEncoder for learning latent manifold representations