Measuring the Discrepancy between Conditional Distributions: Methods, Properties and Applications
arXiv:2005.02196 · doi:10.24963/ijcai.2020/385
Abstract
We propose a simple yet powerful test statistic to quantify the discrepancy between two conditional distributions. The new statistic avoids the explicit estimation of the underlying distributions in highdimensional space and it operates on the cone of symmetric positive semidefinite (SPS) matrix using the Bregman matrix divergence. Moreover, it inherits the merits of the correntropy function to explicitly incorporate high-order statistics in the data. We present the properties of our new statistic and illustrate its connections to prior art. We finally show the applications of our new statistic on three different machine learning problems, namely the multi-task learning over graphs, the concept drift detection, and the information-theoretic feature selection, to demonstrate its utility and advantage. Code of our statistic is available at https://bit.ly/BregmanCorrentropy.
arXiv commentmanuscript accepted at IJCAI 20; added additional notes on computational complexity and auto-differentiable property; code is available at https://github.com/SJYuCNEL/Bregman-Correntropy-Conditional-Divergence
References in corpus (8)
- Auto-Encoding Variational Bayes
- Overcoming catastrophic forgetting in neural networks
- Recent Advances in Autoencoder-Based Representation Learning
- Kernel Risk-Sensitive Loss: Definition, Properties and Application to Robust Adaptive Filtering
- Evading the curse of dimensionality in nonparametric density estimation with simplified vine copulas
- Resampling methods for parameter-free and robust feature selection with mutual information
- Conditional Generative Moment-Matching Networks
- Fast Estimation of Information Theoretic Learning Descriptors using Explicit Inner Product Spaces