Differential Properties of Sinkhorn Approximation for Learning with Wasserstein Distance

Abstract

Applications of optimal transport have recently gained remarkable attention thanks to the computational advantages of entropic regularization. However, in most situations the Sinkhorn approximation of the Wasserstein distance is replaced by a regularized version that is less accurate but easy to differentiate. In this work we characterize the differential properties of the original Sinkhorn distance, proving that it enjoys the same smoothness as its regularized version and we explicitly provide an efficient algorithm to compute its gradient. We show that this result benefits both theory and applications: on one hand, high order smoothness confers statistical guarantees to learning with Wasserstein approximations. On the other hand, the gradient formula allows us to efficiently solve learning and optimization problems in practice. Promising preliminary experiments complement our analysis.

26 pages, 4 figures

References in corpus (1)

• Wasserstein Discriminant Analysis

Cited by in corpus (36)

• GOT: An Optimal Transport framework for Graph comparison
• An Entropic Optimal Transport Loss for Learning Deep Neural Networks under Label Noise in Remote Sensing Images
• Rates of Estimation of Optimal Transport Maps using Plug-in Estimators via Barycentric Projections
• Sinkhorn AutoEncoders
• Improving Sequence-to-Sequence Learning via Optimal Transport
• Kernel Wasserstein Distance
• Differentiable Divergences Between Time Series
• A mathematical theory of cooperative communication
• Differentiable Top-k Operator with Optimal Transport
• Sinkformers: Transformers with Doubly Stochastic Attention
• Debiased Sinkhorn barycenters
• Optimal Transport losses and Sinkhorn algorithm with general convex regularization
• Stochastic Optimization for Regularized Wasserstein Estimators
• Estimating Barycenters of Measures in High Dimensions
• Structured Prediction for Conditional Meta-Learning
• A General Framework for Consistent Structured Prediction with Implicit Loss Embeddings
• ST-MTL: Spatio-Temporal Multitask Learning Model to Predict Scanpath While Tracking Instruments in Robotic Surgery
• Sinkhorn Distributionally Robust Optimization
• Bi-level Feature Alignment for Versatile Image Translation and Manipulation
• Optimal Transport for Stationary Markov Chains via Policy Iteration
• Implementation of batched Sinkhorn iterations for entropy-regularized Wasserstein loss
• An Exploration of Learnt Representations of W Jets
• Learning transport cost from subset correspondence
• Geometric Losses for Distributional Learning
• Improving Text Generation with Student-Forcing Optimal Transport
• Adaptive Softassign via Hadamard-Equipped Sinkhorn
• Nested Sinkhorn Divergence To Compute The Nested Distance
• Supervised Quantile Normalization for Low-rank Matrix Approximation
• Utility/Privacy Trade-off through the lens of Optimal Transport
• Convergence of Gaussian-smoothed optimal transport distance with sub-gamma distributions and dependent samples
• One-step differentiation of iterative algorithms
• Stylized Neural Painting
• Optimal transport with $f$-divergence regularization and generalized Sinkhorn algorithm
• A Hypergradient Approach to Robust Regression without Correspondence
• Bayesian Distributional Policy Gradients
• Learning Graphon Autoencoders for Generative Graph Modeling