Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey
arXiv:2305.05080 · doi:10.1109/TPAMI.2024.3379571
Abstract
Optimal Transport (OT) is a mathematical framework that first emerged in the eighteenth century and has led to a plethora of methods for answering many theoretical and applied questions. The last decade has been a witness to the remarkable contributions of this classical optimization problem to machine learning. This paper is about where and how optimal transport is used in machine learning with a focus on the question of scalable optimal transport. We provide a comprehensive survey of optimal transport while ensuring an accessible presentation as permitted by the nature of the topic and the context. First, we explain the optimal transport background and introduce different flavors (i.e., mathematical formulations), properties, and notable applications. We then address the fundamental question of how to scale optimal transport to cope with the current demands of big and high dimensional data. We conduct a systematic analysis of the methods used in the literature for scaling OT and present the findings in a unified taxonomy. We conclude with presenting some open challenges and discussing potential future research directions. A live repository of related OT research papers is maintained in https://github.com/abdelwahed/OT_for_big_data.git
arXiv commentAccepted @ TPAMI 24
References in corpus (115)
- Fast Computation of Wasserstein Barycenters
- Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration
- Deep Equilibrium Models
- Learning Latent Permutations with Gumbel-Sinkhorn Networks
- Stochastic Optimization for Large-scale Optimal Transport
- Optimal Transport for structured data with application on graphs
- Sample Complexity of Sinkhorn divergences
- Interpolating between Optimal Transport and MMD using Sinkhorn Divergences
- Self-labelling via simultaneous clustering and representation learning
- On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport
- Convergence and Concentration of Empirical Measures under Wasserstein Distance in Unbounded Functional Spaces
- Learning to Optimize: A Primer and A Benchmark
- Gromov-Wasserstein Learning for Graph Matching and Node Embedding
- Ranking via Sinkhorn Propagation
- A Sparse Multi-Scale Algorithm for Dense Optimal Transport
- A Fast Proximal Point Method for Computing Exact Wasserstein Distance
- LoFTR: Detector-Free Local Feature Matching with Transformers
- Large-Scale Optimal Transport and Mapping Estimation
- Geometric Dataset Distances via Optimal Transport
- Faster Wasserstein Distance Estimation with the Sinkhorn Divergence
- Wasserstein Adversarial Examples via Projected Sinkhorn Iterations
- GOT: An Optimal Transport framework for Graph comparison
- Obtaining fairness using optimal transport theory
- Subspace Robust Wasserstein Distances
- Hierarchical Optimal Transport for Document Representation
- Stochastic Wasserstein Barycenters
- Sinkhorn Divergences for Unbalanced Optimal Transport
- DOTmark - A Benchmark for Discrete Optimal Transport
- Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem
- AttGAN: Facial Attribute Editing by Only Changing What You Want
- Optimal transport mapping via input convex neural networks
- Efficient and Modular Implicit Differentiation
- Scalable Gromov-Wasserstein Learning for Graph Partitioning and Matching
- Quantum Optimal Transport for Tensor Field Processing
- Unbalanced minibatch Optimal Transport; applications to Domain Adaptation
- Gradient descent algorithms for Bures-Wasserstein barycenters
- The Unbalanced Gromov Wasserstein Distance: Conic Formulation and Relaxation
- On the Complexity of Approximating Multimarginal Optimal Transport
- Optimal Control Via Neural Networks: A Convex Approach
- Accurate Point Cloud Registration with Robust Optimal Transport
- Learning Generative Models across Incomparable Spaces
- On Unbalanced Optimal Transport: An Analysis of Sinkhorn Algorithm
- Sliced Gromov-Wasserstein
- Statistical and Topological Properties of Sliced Probability Divergences
- Optimal Transport Tools (OTT): A JAX Toolbox for all things Wasserstein
- Scalable Computations of Wasserstein Barycenter via Input Convex Neural Networks
- Learning with minibatch Wasserstein : asymptotic and gradient properties
- A Survey on Optimal Transport for Machine Learning: Theory and Applications
- Continuous Regularized Wasserstein Barycenters
- Hardness results for Multimarginal Optimal Transport problems
- Proximal Optimal Transport Modeling of Population Dynamics
- Supervised Training of Conditional Monge Maps
- A fixed-point approach to barycenters in Wasserstein space
- Sinkformers: Transformers with Doubly Stochastic Attention
- Wasserstein GANs Work Because They Fail (to Approximate the Wasserstein Distance)
- Optimal Transport: Fast Probabilistic Approximation with Exact Solvers
- Debiased Sinkhorn barycenters
- Linear Time Sinkhorn Divergences using Positive Features
- Large-scale optimal transport map estimation using projection pursuit
- Outlier-Robust Optimal Transport
- Speeding up Word Mover's Distance and its variants via properties of distances between embeddings
- Do Neural Optimal Transport Solvers Work? A Continuous Wasserstein-2 Benchmark
- A Study of Performance of Optimal Transport
- Learning 3D-3D Correspondences for One-shot Partial-to-partial Registration
- Subspace Detours: Building Transport Plans that are Optimal on Subspace Projections
- Learning Embeddings into Entropic Wasserstein Spaces
- Distributional Sliced-Wasserstein and Applications to Generative Modeling
- Regularized Optimal Transport is Ground Cost Adversarial
- Watch and Match: Supercharging Imitation with Regularized Optimal Transport
- Asymptotic Guarantees for Learning Generative Models with the Sliced-Wasserstein Distance
- Linear-Time Gromov Wasserstein Distances using Low Rank Couplings and Costs
- Strong equivalence between metrics of Wasserstein type
- Asymptotic Guarantees for Generative Modeling Based on the Smooth Wasserstein Distance
- Personalised Federated Learning On Heterogeneous Feature Spaces
- Low-rank Optimal Transport: Approximation, Statistics and Debiasing
- Overrelaxed Sinkhorn-Knopp Algorithm for Regularized Optimal Transport
- Dimensionality Reduction for Wasserstein Barycenter
- Point-set Distances for Learning Representations of 3D Point Clouds
- Set Representation Learning with Generalized Sliced-Wasserstein Embeddings
- Measuring Generalization with Optimal Transport
- MREC: a fast and versatile framework for aligning and matching point clouds with applications to single cell molecular data
- Averaging on the Bures-Wasserstein manifold: dimension-free convergence of gradient descent
- Differentially Private Sliced Wasserstein Distance
- Feature Robust Optimal Transport for High-dimensional Data
- Making transport more robust and interpretable by moving data through a small number of anchor points
- Projection Robust Wasserstein Barycenters
- Measure-valued spline curves: an optimal transport viewpoint
- Sketching and Clustering Metric Measure Spaces
- Wasserstein Iterative Networks for Barycenter Estimation
- Kantorovich Strikes Back! Wasserstein GANs are not Optimal Transport?
- Approximating 1-Wasserstein Distance with Trees
- Unsupervised Action Segmentation by Joint Representation Learning and Online Clustering
- Tutorial on amortized optimization
- Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic Spaces
- Cross-Domain Imitation Learning via Optimal Transport
- Entropic Gromov-Wasserstein between Gaussian Distributions
- Improving Approximate Optimal Transport Distances using Quantization
- Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization
- Rethinking Initialization of the Sinkhorn Algorithm
- Transport with Support: Data-Conditional Diffusion Bridges
- Exploiting Problem Structure in Deep Declarative Networks: Two Case Studies
- Fixed Support Tree-Sliced Wasserstein Barycenter
- Gromov-Wasserstein Autoencoders
- Functional optimal transport: map estimation and domain adaptation for functional data
- A Review on Modern Computational Optimal Transport Methods with Applications in Biomedical Research
- Efficient Optimal Transport Algorithm by Accelerated Gradient descent
- A Unified Framework for Implicit Sinkhorn Differentiation
- Optimal Transport of Classifiers to Fairness
- Quantized Wasserstein Procrustes Alignment of Word Embedding Spaces
- Scalable Optimal Transport in High Dimensions for Graph Distances, Embedding Alignment, and More
- Budget-Constrained Bounds for Mini-Batch Estimation of Optimal Transport
- MMD-Regularized Unbalanced Optimal Transport
- Asymptotics of smoothed Wasserstein distances in the small noise regime
- Approximating Optimal Transport via Low-rank and Sparse Factorization
- Order Constraints in Optimal Transport