Consistent distribution-free $K$-sample and independence tests for univariate random variables

Abstract

A popular approach for testing if two univariate random variables are statistically independent consists of partitioning the sample space into bins, and evaluating a test statistic on the binned data. The partition size matters, and the optimal partition size is data dependent. While for detecting simple relationships coarse partitions may be best, for detecting complex relationships a great gain in power can be achieved by considering finer partitions. We suggest novel consistent distribution-free tests that are based on summation or maximization aggregation of scores over all partitions of a fixed size. We show that our test statistics based on summation can serve as good estimators of the mutual information. Moreover, we suggest regularized tests that aggregate over all partition sizes, and prove those are consistent too. We provide polynomial-time algorithms, which are critical for computing the suggested test statistics efficiently. We show that the power of the regularized tests is excellent compared to existing tests, and almost as powerful as the tests based on the optimal (yet unknown in practice) partition size, in simulations as well as on a real data example.

arXiv admin note: substantial text overlap with arXiv:1308.1559

References in corpus (4)

• Brownian distance covariance
• Comment on "Detecting Novel Associations In Large Data Sets" by Reshef Et Al, Science Dec 16, 2011
• Testing for Homogeneity with Kernel Fisher Discriminant Analysis
• A Wild Bootstrap for Degenerate Kernel Tests

Cited by in corpus (18)

• Measuring dependence powerfully and equitably
• BET on Independence
• The Exact Equivalence of Distance and Kernel Methods for Hypothesis Testing
• Multivariate tests of association based on univariate tests
• Counting Small Permutation Patterns
• Wasserstein-based fairness interpretability framework for machine learning models
• An Empirical Study of Leading Measures of Dependence
• Data-Driven Representations for Testing Independence: Modeling, Analysis and Connection with Mutual Information Estimation
• Exchangeability, Conformal Prediction, and Rank Tests
• copent: Estimating Copula Entropy and Transfer Entropy in R
• Network Dependence Testing via Diffusion Maps and Distance-Based Correlations
• Testing mutual independence in high dimension via distance covariance
• BEAUTY Powered BEAST
• Learning Interpretable Characteristic Kernels via Decision Forests
• High-Dimensional Independence Testing via Maximum and Average Distance Correlations
• The Binary Expansion Randomized Ensemble Test (BERET)
• independence: Fast Rank Tests
• Generalized R-squared for Detecting Dependence