co-BPM: a Bayesian Model for Divergence Estimation
arXiv:1410.0726
Abstract
Divergence is not only an important mathematical concept in information theory, but also applied to machine learning problems such as low-dimensional embedding, manifold learning, clustering, classification, and anomaly detection. We proposed a bayesian model---co-BPM---to characterize the discrepancy of two sample sets, i.e., to estimate the divergence of their underlying distributions. In order to avoid the pitfalls of plug-in methods that estimate each density independently, our bayesian model attempts to learn a coupled binary partition of the sample space that best captures the landscapes of both distributions, then make direct inference on their divergences. The prior is constructed by leveraging the sequential buildup of the coupled binary partitions and the posterior is sampled via our specialized MCMC. Our model provides a unified way to estimate various types of divergences and enjoys convincing accuracy. We demonstrate its effectiveness through simulations, comparisons with the \emph{state-of-the-art} and a real data example.
arXiv commentKey Words: coupled binary partition, divergence, MCMC, clustering, classification
References in corpus (6)
- Estimating divergence functionals and the likelihood ratio by convex risk minimization
- Consistency of spectral clustering
- A class of Rényi information estimators for multidimensional densities
- Nonparametric Divergence Estimation with Applications to Machine Learning on Distributions
- Empirical estimation of entropy functionals with confidence
- Density Estimation via Discrepancy Based Adaptive Sequential Partition