Relative Density-Ratio Estimation for Robust Distribution Comparison

Abstract

Divergence estimators based on direct approximation of density-ratios without going through separate approximation of numerator and denominator densities have been successfully applied to machine learning tasks that involve distribution comparison such as outlier detection, transfer learning, and two-sample homogeneity test. However, since density-ratio functions often possess high fluctuation, divergence estimation is still a challenging task in practice. In this paper, we propose to use relative divergences for distribution comparison, which involves approximation of relative density-ratios. Since relative density-ratios are always smoother than corresponding ordinary density-ratios, our proposed method is favorable in terms of the non-parametric convergence speed. Furthermore, we show that the proposed divergence estimator has asymptotic variance independent of the model complexity under a parametric setup, implying that the proposed estimator hardly overfits even with complex models. Through experiments, we demonstrate the usefulness of the proposed approach.

Cited by in corpus (11)

• Learning from Positive and Unlabeled Data with Arbitrary Positive Shift
• B-tests: Low Variance Kernel Two-Sample Tests
• Proximal Policy Optimization with Relative Pearson Divergence
• Support Consistency of Direct Sparse-Change Learning in Markov Networks
• Meta-Learning for Relative Density-Ratio Estimation
• Self-supervised Representation Learning with Relative Predictive Coding
• Transfer String Kernel for Cross-Context DNA-Protein Binding Prediction
• Density Ratio Estimation via Infinitesimal Classification
• F-measure Maximizing Logistic Regression
• Uplift Modeling from Separate Labels
• Gradient of Probability Density Functions based Contrasts for Blind Source Separation (BSS)