Interpolating between Optimal Transport and MMD using Sinkhorn Divergences

Abstract

Comparing probability distributions is a fundamental problem in data sciences. Simple norms and divergences such as the total variation and the relative entropy only compare densities in a point-wise manner and fail to capture the geometric nature of the problem. In sharp contrast, Maximum Mean Discrepancies (MMD) and Optimal Transport distances (OT) are two classes of distances between measures that take into account the geometry of the underlying space and metrize the convergence in law. This paper studies the Sinkhorn divergences, a family of geometric divergences that interpolates between MMD and OT. Relying on a new notion of geometric entropy, we provide theoretical guarantees for these divergences: positivity, convexity and metrization of the convergence in law. On the practical side, we detail a numerical scheme that enables the large scale application of these divergences for machine learning: on the GPU, gradients of the Sinkhorn loss can be computed for batches of a million samples.

arXiv comment15 pages, 5 figures

Cited by in corpus (63)

• The Born Supremacy: Quantum Advantage and Training of an Ising Born Machine
• Faster Wasserstein Distance Estimation with the Sinkhorn Divergence
• Sinkhorn Divergences for Unbalanced Optimal Transport
• Entropic Optimal Transport between Unbalanced Gaussian Measures has a Closed Form
• Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem
• Flows for simultaneous manifold learning and density estimation
• The Unbalanced Gromov Wasserstein Distance: Conic Formulation and Relaxation
• Solving Schrödinger Bridges via Maximum Likelihood
• Discovering Conservation Laws using Optimal Transport and Manifold Learning
• Differentiable Divergences Between Time Series
• Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey
• Learning with minibatch Wasserstein : asymptotic and gradient properties
• Minibatch optimal transport distances; analysis and applications
• How Well Do WGANs Estimate the Wasserstein Metric?
• Linear Time Sinkhorn Divergences using Positive Features
• Debiased Sinkhorn barycenters
• Controlling diverse robots by inferring Jacobian fields with deep networks
• Generalization Properties of Optimal Transport GANs with Latent Distribution Learning
• Spectral convergence of diffusion maps: improved error bounds and an alternative normalisation
• Contrastive Graph Neural Network Explanation
• Differentiable Particle Filtering via Entropy-Regularized Optimal Transport
• Optimal Transport losses and Sinkhorn algorithm with general convex regularization
• Stochastic Optimization for Regularized Wasserstein Estimators
• Meta-PU: An Arbitrary-Scale Upsampling Network for Point Cloud
• Spatio-Temporal Alignments: Optimal transport through space and time
• Entropic estimation of optimal transport maps
• Unbalanced Sobolev Descent
• A contribution to Optimal Transport on incomparable spaces
• HNPE: Leveraging Global Parameters for Neural Posterior Estimation
• Sinkhorn Natural Gradient for Generative Models
• Sinkhorn Barycenter via Functional Gradient Descent
• Improving Federated Relational Data Modeling via Basis Alignment and Weight Penalty
• Geometric Losses for Distributional Learning
• KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Support
• Online Sinkhorn: Optimal Transport distances from sample streams
• Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic Spaces
• On the Existence of Optimal Transport Gradient for Learning Generative Models
• Super-efficiency of automatic differentiation for functions defined as a minimum
• Utility/Privacy Trade-off through the lens of Optimal Transport
• Equitable and Optimal Transport with Multiple Agents
• Discrepancy-based Inference for Intractable Generative Models using Quasi-Monte Carlo
• Alignment Attention by Matching Key and Query Distributions
• Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence
• Order Constraints in Optimal Transport
• Deep Ordinal Regression using Optimal Transport Loss and Unimodal Output Probabilities
• Regularized Policies are Reward Robust
• Estimating 2-Sinkhorn Divergence between Gaussian Processes from Finite-Dimensional Marginals
• Convergence and finite sample approximations of entropic regularized Wasserstein distances in Gaussian and RKHS settings
• Convergence of Gaussian-smoothed optimal transport distance with sub-gamma distributions and dependent samples
• Soft and subspace robust multivariate rank tests based on entropy regularized optimal transport
• Direct Measure Matching for Crowd Counting
• Top-N: Equivariant set and graph generation without exchangeability
• Optimal transport weights for causal inference
• Informative GANs via Structured Regularization of Optimal Transport
• Generation of tubular and membranous shape textures with curvature functionals
• Optimal transport with $f$-divergence regularization and generalized Sinkhorn algorithm
• Efficient estimates of optimal transport via low-dimensional embeddings
• Bundle Networks: Fiber Bundles, Local Trivializations, and a Generative Approach to Exploring Many-to-one Maps
• Unsupervised Diffeomorphic Surface Registration and Non-Linear Modelling
• Fast and Scalable Optimal Transport for Brain Tractograms
• Understanding Entropic Regularization in GANs
• Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddings
• Aligning Time Series on Incomparable Spaces