On Efficient Optimal Transport: An Analysis of Greedy and Accelerated Mirror Descent Algorithms

Abstract

We provide theoretical analyses for two algorithms that solve the regularized optimal transport (OT) problem between two discrete probability measures with at most $n$ atoms. We show that a greedy variant of the classical Sinkhorn algorithm, known as the \emph{Greenkhorn algorithm}, can be improved to $\tilde{\mathcal{O}}(n^2{\epsilon }^{-2})$, improving on the best known complexity bound of $\tilde{\mathcal{O}}(n^2{\epsilon }^{-3})$. Notably, this matches the best known complexity bound for the Sinkhorn algorithm and helps explain why the Greenkhorn algorithm can outperform the Sinkhorn algorithm in practice. Our proof technique, which is based on a primal-dual formulation and a novel upper bound for the dual solution, also leads to a new class of algorithms that we refer to as \emph{adaptive primal-dual accelerated mirror descent} (APDAMD) algorithms. We prove that the complexity of these algorithms is $\tilde{\mathcal{O}}(n^2\sqrt{\delta }{\epsilon }^{-1})$, where $\delta >0$ refers to the inverse of the strong convexity module of Bregman divergence with respect to $\Vert \cdot {\Vert }_{\infty }$. This implies that the APDAMD algorithm is faster than the Sinkhorn and Greenkhorn algorithms in terms of $\epsilon$. Experimental results on synthetic and real datasets demonstrate the favorable performance of the Greenkhorn and APDAMD algorithms in practice.

Derive the explicit dual objective function for APDAMD (Remark 4.2) which satisfies Lemma~4.1; Accepted by ICML 2019; The longer version is available here: arXiv:1906.01437

References in corpus (6)

• Fast Computation of Wasserstein Barycenters
• Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration
• Sliced Wasserstein Kernel for Persistence Diagrams
• Smooth and Sparse Optimal Transport
• Greedy stochastic algorithms for entropy-regularized optimal transport problems
• Probabilistic Multilevel Clustering via Composite Transportation Distance

Cited by in corpus (27)

• Density-aware Chamfer Distance as a Comprehensive Metric for Point Cloud Completion
• On the Complexity of Approximating Multimarginal Optimal Transport
• On the Complexity of Approximating Wasserstein Barycenter
• On Unbalanced Optimal Transport: An Analysis of Sinkhorn Algorithm
• Screening Sinkhorn Algorithm for Regularized Optimal Transport
• Fixed-Support Wasserstein Barycenters: Computational Hardness and Fast Algorithm
• Optimal Transport: Fast Probabilistic Approximation with Exact Solvers
• Linear Time Sinkhorn Divergences using Positive Features
• Bipartite Matching in Nearly-linear Time on Moderately Dense Graphs
• Evaluating the Disentanglement of Deep Generative Models through Manifold Topology
• Distributional Sliced-Wasserstein and Applications to Generative Modeling
• A Riemannian Block Coordinate Descent Method for Computing the Projection Robust Wasserstein Distance
• Interior-Point Methods Strike Back: Solving the Wasserstein Barycenter Problem
• Point-set Distances for Learning Representations of 3D Point Clouds
• Projection Robust Wasserstein Distance and Riemannian Optimization
• On Transportation of Mini-batches: A Hierarchical Approach
• On Robust Optimal Transport: Computational Complexity and Barycenter Computation
• Augmented Sliced Wasserstein Distances
• Optimal Transport for Stationary Markov Chains via Policy Iteration
• Flow-based Alignment Approaches for Probability Measures in Different Spaces
• Improving Mini-batch Optimal Transport via Partial Transportation
• Sinkhorn Algorithm as a Special Case of Stochastic Mirror Descent
• Entropic Gromov-Wasserstein between Gaussian Distributions
• Stochastic Saddle-Point Optimization for Wasserstein Barycenters
• Tessellated Wasserstein Auto-Encoders
• On Efficient Multilevel Clustering via Wasserstein Distances
• Order Constraints in Optimal Transport