Estimating Feature-Label Dependence Using Gini Distance Statistics
arXiv:1906.02171 · doi:10.1109/TPAMI.2019.2960358
Abstract
Identifying statistical dependence between the features and the label is a fundamental problem in supervised learning. This paper presents a framework for estimating dependence between numerical features and a categorical label using generalized Gini distance, an energy distance in reproducing kernel Hilbert spaces (RKHS). Two Gini distance based dependence measures are explored: Gini distance covariance and Gini distance correlation. Unlike Pearson covariance and correlation, which do not characterize independence, the above Gini distance based measures define dependence as well as independence of random variables. The test statistics are simple to calculate and do not require probability density estimation. Uniform convergence bounds and asymptotic bounds are derived for the test statistics. Comparisons with distance covariance statistics are provided. It is shown that Gini distance statistics converge faster than distance covariance statistics in the uniform convergence bounds, hence tighter upper bounds on both Type I and Type II errors. Moreover, the probability of Gini distance covariance statistic under-performing the distance covariance statistic in Type II error decreases to 0 exponentially with the increase of the sample size. Extensive experimental results are presented to demonstrate the performance of the proposed method.
References in corpus (11)
- Measuring and testing dependence by correlation of distances
- Brownian distance covariance
- Equivalence of distance-based and RKHS-based statistics in hypothesis testing
- Distance covariance in metric spaces
- Feature Selection with Annealing for Computer Vision and Big Data Learning
- Screening Tests for Lasso Problems
- A Feature Selection Method for Multivariate Performance Measures
- Variable selection in functional data classification: a maxima-hunting proposal
- Functional learning through kernels
- Estimating Feature-Label Dependence Using Gini Distance Statistics
- A new Gini correlation between quantitative and qualitative variables