Statistical and Topological Properties of Sliced Probability Divergences

Abstract

The idea of slicing divergences has been proven to be successful when comparing two probability measures in various machine learning applications including generative modeling, and consists in computing the expected value of a `base divergence' between one-dimensional random projections of the two measures. However, the topological, statistical, and computational consequences of this technique have not yet been well-established. In this paper, we aim at bridging this gap and derive various theoretical properties of sliced probability divergences. First, we show that slicing preserves the metric axioms and the weak continuity of the divergence, implying that the sliced divergence will share similar topological properties. We then precise the results in the case where the base divergence belongs to the class of integral probability metrics. On the other hand, we establish that, under mild conditions, the sample complexity of a sliced divergence does not depend on the problem dimension. We finally apply our general results to several base divergences, and illustrate our theory on both synthetic and real data experiments.

Published at NeurIPS 2020 (Spotlight)

References in corpus (6)

• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• From optimal transport to generative modeling: the VEGAN cookbook
• On integral probability metrics, ϕ-divergences and binary classification
• Minimax Confidence Intervals for the Sliced Wasserstein Distance
• Generalized Sliced Wasserstein Distances
• Strong equivalence between metrics of Wasserstein type

Cited by in corpus (9)

• Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey
• Smooth $p$-Wasserstein Distance: Structure, Empirical Approximation, and Statistical Applications
• Limit Distribution Theory for the Smooth 1-Wasserstein Distance with Applications
• On Projection Robust Optimal Transport: Sample Complexity and Model Misspecification
• Fast Approximation of the Sliced-Wasserstein Distance Using Concentration of Random Projections
• Sliced Iterative Normalizing Flows
• Sliced Multi-Marginal Optimal Transport
• Spherical Sliced-Wasserstein
• Active Slices for Sliced Stein Discrepancy