Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey
arXiv:2305.05080 · doi:10.1109/TPAMI.2024.3379571
Abstract
Optimal Transport (OT) is a mathematical framework that first emerged in the eighteenth century and has led to a plethora of methods for answering many theoretical and applied questions. The last decade has been a witness to the remarkable contributions of this classical optimization problem to machine learning. This paper is about where and how optimal transport is used in machine learning with a focus on the question of scalable optimal transport. We provide a comprehensive survey of optimal transport while ensuring an accessible presentation as permitted by the nature of the topic and the context. First, we explain the optimal transport background and introduce different flavors (i.e., mathematical formulations), properties, and notable applications. We then address the fundamental question of how to scale optimal transport to cope with the current demands of big and high dimensional data. We conduct a systematic analysis of the methods used in the literature for scaling OT and present the findings in a unified taxonomy. We conclude with presenting some open challenges and discussing potential future research directions. A live repository of related OT research papers is maintained in https://github.com/abdelwahed/OT_for_big_data.git
Accepted @ TPAMI 24
References in corpus (39)
- Deep Equilibrium Models
- Self-labelling via simultaneous clustering and representation learning
- Gromov-Wasserstein Learning for Graph Matching and Node Embedding
- Ranking via Sinkhorn Propagation
- LoFTR: Detector-Free Local Feature Matching with Transformers
- Geometric Dataset Distances via Optimal Transport
- Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem
- Unbalanced minibatch Optimal Transport; applications to Domain Adaptation
- Accurate Point Cloud Registration with Robust Optimal Transport
- Learning Generative Models across Incomparable Spaces
- Optimal Transport Tools (OTT): A JAX Toolbox for all things Wasserstein
- A Survey on Optimal Transport for Machine Learning: Theory and Applications
- Supervised Training of Conditional Monge Maps
- Large-scale optimal transport map estimation using projection pursuit
- Debiased Sinkhorn barycenters
- Do Neural Optimal Transport Solvers Work? A Continuous Wasserstein-2 Benchmark
- Learning 3D-3D Correspondences for One-shot Partial-to-partial Registration
- A Study of Performance of Optimal Transport
- Learning Embeddings into Entropic Wasserstein Spaces
- Watch and Match: Supercharging Imitation with Regularized Optimal Transport
- Personalised Federated Learning On Heterogeneous Feature Spaces
- Low-rank Optimal Transport: Approximation, Statistics and Debiasing
- Measuring Generalization with Optimal Transport
- MREC: a fast and versatile framework for aligning and matching point clouds with applications to single cell molecular data
- Set Representation Learning with Generalized Sliced-Wasserstein Embeddings
- Dimensionality Reduction for Wasserstein Barycenter
- Approximating 1-Wasserstein Distance with Trees
- Feature Robust Optimal Transport for High-dimensional Data
- Differentially Private Sliced Wasserstein Distance
- Kantorovich Strikes Back! Wasserstein GANs are not Optimal Transport?
- Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization
- Gromov-Wasserstein Autoencoders
- Rethinking Initialization of the Sinkhorn Algorithm
- Exploiting Problem Structure in Deep Declarative Networks: Two Case Studies
- Transport with Support: Data-Conditional Diffusion Bridges
- A Unified Framework for Implicit Sinkhorn Differentiation
- Quantized Wasserstein Procrustes Alignment of Word Embedding Spaces
- Budget-Constrained Bounds for Mini-Batch Estimation of Optimal Transport
- Asymptotics of smoothed Wasserstein distances in the small noise regime