A Pseudo-Metric between Probability Distributions based on Depth-Trimmed Regions

Abstract

The design of a metric between probability distributions is a longstanding problem motivated by numerous applications in Machine Learning. Focusing on continuous probability distributions on the Euclidean space ${\mathbb{R}}^d$, we introduce a novel pseudo-metric between probability distributions by leveraging the extension of univariate quantiles to multivariate spaces. Data depth is a nonparametric statistical tool that measures the centrality of any element $x\in {\mathbb{R}}^d$ with respect to (w.r.t.) a probability distribution or a data set. It is a natural median-oriented extension of the cumulative distribution function (cdf) to the multivariate case. Thus, its upper-level sets -- the depth-trimmed regions -- give rise to a definition of multivariate quantiles. The new pseudo-metric relies on the average of the Hausdorff distance between the depth-based quantile regions w.r.t. each distribution. Its good behavior w.r.t. major transformation groups, as well as its ability to factor out translations, are depicted. Robustness, an appealing feature of this pseudo-metric, is studied through the finite sample breakdown point. Moreover, we propose an efficient approximation method with linear time complexity w.r.t. the size of the data set and its dimension. The quality of this approximation as well as the performance of the proposed approach are illustrated in numerical experiments.

References in corpus (19)

• Scikit-learn: Machine Learning in Python
• Efficient Estimation of Word Representations in Vector Space
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• BERTScore: Evaluating Text Generation with BERT
• Statistical Aspects of Wasserstein Distances
• Multivariate quantiles and multiple-output regression quantiles: From $L_1$ optimization to halfspace depth
• On Bregman Distances and Divergences of Probability Measures
• Subspace Robust Wasserstein Distances
• Statistical Optimal Transport via Factored Couplings
• Robust Optimal Transport with Applications in Generative Modeling and Domain Adaptation
• Learning by Semantic Similarity Makes Abstractive Summarization Better
• Convergence of depths and depth-trimmed regions
• Outlier-Robust Optimal Transport
• Code-switched inspired losses for generic spoken dialog representations
• Beam Search with Bidirectional Strategies for Neural Response Generation
• Distributionally robust halfspace depth
• Affine-Invariant Integrated Rank-Weighted Depth: Definition, Properties and Finite Sample Analysis
• Generalization Bounds in the Presence of Outliers: a Median-of-Means Study