Equivalence of distance-based and RKHS-based statistics in hypothesis testing

Abstract

We provide a unifying framework linking two classes of statistics used in two-sample and independence testing: on the one hand, the energy distances and distance covariances from the statistics literature; on the other, maximum mean discrepancies (MMD), that is, distances between embeddings of distributions to reproducing kernel Hilbert spaces (RKHS), as established in machine learning. In the case where the energy distance is computed with a semimetric of negative type, a positive definite kernel, termed distance kernel, may be defined such that the MMD corresponds exactly to the energy distance. Conversely, for any positive definite kernel, we can interpret the MMD as energy distance with respect to some negative-type semimetric. This equivalence readily extends to distance covariance using kernels on the product space. We determine the class of probability distributions for which the test statistics are consistent against all alternatives. Finally, we investigate the performance of the family of distance kernels in two-sample and independence tests: we show in particular that the energy distance most commonly employed in statistics is just one member of a parametric family of kernels, and that other choices from this family can yield more powerful tests.

Published in at http://dx.doi.org/10.1214/13-AOS1140 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (6)

• Measuring and testing dependence by correlation of distances
• Brownian distance covariance
• Kernel-based Conditional Independence Test and Application in Causal Discovery
• A Kernel Method for the Two-Sample Problem
• Universality, Characteristic Kernels and RKHS Embedding of Measures
• Discussion of: Brownian distance covariance

Cited by in corpus (145)

• Learning Transferable Features with Deep Adaptation Networks
• VisDA: The Visual Domain Adaptation Challenge
• Applications of Unsupervised Deep Transfer Learning to Intelligent Fault Diagnosis: A Survey and Comparative Study
• Kernel Mean Embedding of Distributions: A Review and Beyond
• Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks
• Wasserstein Distance based Deep Adversarial Transfer Learning for Intelligent Fault Diagnosis
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• A Survey of Learning Causality with Data: Problems and Methods
• Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences
• Demystifying MMD GANs
• A Kernel Multiple Change-point Algorithm via Model Selection
• Deep Transfer Learning with Joint Adaptation Networks
• Large-Scale Kernel Methods for Independence Testing
• Contrastive Adaptation Network for Unsupervised Domain Adaptation
• A Kernel Test of Goodness of Fit
• The Chi-Square Test of Distance Correlation
• A fast algorithm for computing distance correlation
• DeepMDP: Learning Continuous Latent Space Models for Representation Learning
• Semi-supervised Multi-modal Emotion Recognition with Cross-Modal Distribution Matching
• Multi-modal Deep Analysis for Multimedia
• Expected Similarity Estimation for Large-Scale Batch and Streaming Anomaly Detection
• Distance Correlation Methods for Discovering Associations in Large Astrophysical Databases
• Consistent distribution-free $K$-sample and independence tests for univariate random variables
• From Distance Correlation to Multiscale Graph Correlation
• Syn2Real: A New Benchmark forSynthetic-to-Real Visual Domain Adaptation
• Improving GANs Using Optimal Transport
• FastMMD: Ensemble of Circular Discrepancy for Efficient Two-Sample Test
• On the optimal estimation of probability measures in weak and strong topologies
• BET on Independence
• Discovering and Deciphering Relationships Across Disparate Data Modalities
• Distance multivariance: New dependence measures for random vectors
• The Exact Equivalence of Distance and Kernel Methods for Hypothesis Testing
• Independence weights for causal inference with continuous treatments
• A Kernel Two-Sample Test for Functional Data
• Measuring Association on Topological Spaces Using Kernels and Geometric Graphs
• A nonparametric two-sample hypothesis testing problem for random dot product graphs
• K2-ABC: Approximate Bayesian Computation with Kernel Embeddings
• Algorithmic Decision Making with Conditional Fairness
• Sampling Permutations for Shapley Value Estimation
• Generative machine learning methods for multivariate ensemble post-processing
• A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings
• Stochastic Runge-Kutta Accelerates Langevin Monte Carlo and Beyond
• A Kernel Test for Three-Variable Interactions
• A Wild Bootstrap for Degenerate Kernel Tests
• Estimating Feature-Label Dependence Using Gini Distance Statistics
• Adaptivity and Computation-Statistics Tradeoffs for Kernel and Distance based High Dimensional Two Sample Testing
• A Spectral Energy Distance for Parallel Speech Synthesis
• Review of end-to-end speech synthesis technology based on deep learning
• A survey on domain adaptation theory: learning bounds and theoretical guarantees
• Multivariate tests of association based on univariate tests
• Interpretable Distribution Features with Maximum Testing Power
• Measures of dependence between random vectors and tests of independence. Literature review
• Synthetic to Real Adaptation with Generative Correlation Alignment Networks
• Kernel k-Groups via Hartigan's Method
• Antithetic and Monte Carlo kernel estimators for partial rankings
• MONK -- Outlier-Robust Mean Embedding Estimation by Median-of-Means
• Distance Correlation Coefficients for Lancaster Distributions
• Methods for Quantifying Dataset Similarity: a Review, Taxonomy and Comparison
• Target and Conditional Sensitivity Analysis with Emphasis on Dependence Measures
• Fast and Robust Comparison of Probability Measures in Heterogeneous Spaces
• A simpler condition for consistency of a kernel independence test
• Wasserstein Steepest Descent Flows of Discrepancies with Riesz Kernels
• Distance-based and RKHS-based Dependence Metrics in High Dimension
• hyppo: A Multivariate Hypothesis Testing Python Package
• Explaining the data or explaining a model? Shapley values that uncover non-linear dependencies
• Estimating and Evaluating Regression Predictive Uncertainty in Deep Object Detectors
• Probabilistic multivariate electricity price forecasting using implicit generative ensemble post-processing
• The energy distance for ensemble and scenario reduction
• Robust Multivariate Nonparametric Tests via Projection-Averaging
• PETGEN: Personalized Text Generation Attack on Deep Sequence Embedding-based Classification Models
• Universally Consistent K-Sample Tests via Dependence Measures
• Higher Order Kernel Mean Embeddings to Capture Filtrations of Stochastic Processes
• Multimodal Multi-User Surface Recognition with the Kernel Two-Sample Test
• On the High-dimensional Power of Linear-time Kernel Two-Sample Testing under Mean-difference Alternatives
• Applications of Distance Correlation to Time Series
• Optimal quantisation of probability measures using maximum mean discrepancy
• Nonparametric data segmentation in multivariate time series via joint characteristic functions
• Generalized Kernel Thinning
• Cross-Domain Sentiment Classification with In-Domain Contrastive Learning
• Generalized kernel distance covariance in high dimensions: non-null CLTs and power universality
• Data-Space Validation of High-Dimensional Models by Comparing Sample Quantiles
• Disentangling representations of retinal images with generative models
• Cycle Consistent Probability Divergences Across Different Spaces
• Network Dependence Testing via Diffusion Maps and Distance-Based Correlations
• Comparing a Large Number of Multivariate Distributions
• A low variance consistent test of relative dependency
• A uniform kernel trick for high-dimensional two-sample problems
• Improving Disentangled Text Representation Learning with Information-Theoretic Guidance
• Model predictivity assessment: incremental test-set selection and accuracy evaluation
• Intelligent Icing Detection Model of Wind Turbine Blades Based on SCADA data
• Minimax Estimation of Kernel Mean Embeddings
• Using Kernel-Based Statistical Distance to Study the Dynamics of Charged Particle Beams in Particle-Based Simulation Codes
• Learning Interpretable Characteristic Kernels via Decision Forests
• Generalized Sliced Distances for Probability Distributions
• High-Dimensional Independence Testing via Maximum and Average Distance Correlations
• Dependence and dependence structures: estimation and visualization using the unifying concept of distance multivariance
• A nonparametric test for elliptical distribution based on kernel embedding of probabilities
• BEAUTY Powered BEAST
• Robust Distance Covariance
• Energy distance and kernel mean embeddings for two-sample survival testing
• Testing mutual independence in high dimension via distance covariance
• Biased bootstrap sampling for efficient two-sample testing
• Distance Metrics for Measuring Joint Dependence with Application to Causal Inference
• Graph Random Neural Features for Distance-Preserving Graph Representations
• On Testing Independence and Goodness-of-fit in Linear Models
• Radon Sobolev Variational Auto-Encoders
• A Basic Treatment of the Distance Covariance
• Cost-Effective Incentive Allocation via Structured Counterfactual Inference
• Huber-energy measure quantization
• Robust Classification of High-Dimensional Data using Data-Adaptive Energy Distance
• Independent Component Analysis via Energy-based and Kernel-based Mutual Dependence Measures
• Fast Kernel Summation in High Dimensions via Slicing and Fourier Transforms
• A Note on Optimizing Distributions using Kernel Mean Embeddings
• Pareto GAN: Extending the Representational Power of GANs to Heavy-Tailed Distributions
• Consistency of permutation tests for HSIC and dHSIC
• Information Condensing Active Learning
• Equivalence of Kernel Machine Regression and Kernel Distance Covariance for Multidimensional Trait Association Studies
• What can the millions of random treatments in nonexperimental data reveal about causes?
• The GaussianSketch for Almost Relative Error Kernel Distance
• New efficient algorithms for multiple change-point detection with kernels
• On the Robustness of Domain Adaption to Adversarial Attacks
• Asymptotically Optimal One- and Two-Sample Testing with Kernels
• Meta Two-Sample Testing: Learning Kernels for Testing with Limited Data
• Generalized R-squared for Detecting Dependence
• Weighted-Graph-Based Change Point Detection
• Soft and subspace robust multivariate rank tests based on entropy regularized optimal transport
• Optimal Kernel Combination for Test of Independence against Local Alternatives
• Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation
• Bregman divergences based on optimal design criteria and simplicial measures of dispersion
• Adaptive Geo-Topological Independence Criterion
• Change-Point Detection for Object-valued Time Series
• Interpretation and visualization of distance covariance through additive decomposition of correlations formula
• Level Sets Based Distances for Probability Measures and Ensembles with Applications
• Algorithms that get old : the case of generative deep neural networks
• Discussion of "Functional Models for Time-Varying Random Objects'' by Dubey and Müller
• Equitability of Dependence Measure
• Decorrelated Variable Importance
• Implicit Generative Copulas
• Estimating Rényi's $\alpha$-Cross-Entropies in a Matrix-Based Way
• Towards Fair Cross-Domain Adaptation via Generative Learning
• Sufficient Dimension Reduction via Inverse Conditional Mean or Variance Independence
• Bayesian Kernelised Test of (In)dependence with Mixed-type Variables
• A nonparametric distribution-free test of independence among continuous random vectors based on \texorpdfstring{$L_1$}{}-norm
• A Kernel Two-sample Test for Dynamical Systems
• Unsupervised Domain Adaptation via Discriminative Manifold Embedding and Alignment