On The Chain Rule Optimal Transport Distance

Abstract

We define a novel class of distances between statistical multivariate distributions by modeling an optimal transport problem on their marginals with respect to a ground distance defined on their conditionals. These new distances are metrics whenever the ground distance between the marginals is a metric, generalize both the Wasserstein distances between discrete measures and a recently introduced metric distance between statistical mixtures, and provide an upper bound for jointly convex distances between statistical mixtures. By entropic regularization of the optimal transport, we obtain a fast differentiable Sinkhorn-type distance. We experimentally evaluate our new family of distances by quantifying the upper bounds of several jointly convex distances between statistical mixtures, and by proposing a novel efficient method to learn Gaussian mixture models (GMMs) by simplifying kernel density estimators with respect to our distance. Our GMM learning technique experimentally improves significantly over the EM implementation of {\tt sklearn} on the {\tt MNIST} and {\tt Fashion MNIST} datasets.

arXiv comment23 pages, 6 figures

References in corpus (9)

• Scikit-learn: Machine Learning in Python
• Auto-Encoding Variational Bayes
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Rényi Divergence and Kullback-Leibler Divergence
• Statistical exponential families: A digest with flash cards
• A family of statistical symmetric divergences based on Jensen's inequality
• Generalized Bhattacharyya and Chernoff upper bounds on Bayes error using quasi-arithmetic means
• On the joint convexity of the Bregman divergence of matrices
• On $w$-mixtures: Finite convex combinations of prescribed component distributions