Non-Negative Bregman Divergence Minimization for Deep Direct Density Ratio Estimation
arXiv:2006.06979
Abstract
Density ratio estimation (DRE) is at the core of various machine learning tasks such as anomaly detection and domain adaptation. In existing studies on DRE, methods based on Bregman divergence (BD) minimization have been extensively studied. However, BD minimization when applied with highly flexible models, such as deep neural networks, tends to suffer from what we call train-loss hacking, which is a source of overfitting caused by a typical characteristic of empirical BD estimators. In this paper, to mitigate train-loss hacking, we propose a non-negative correction for empirical BD estimators. Theoretically, we confirm the soundness of the proposed method through a generalization error bound. Through our experiments, the proposed methods show a favorable performance in inlier-based outlier detection.
References in corpus (17)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Deep Residual Learning for Image Recognition
- A Contextual-Bandit Approach to Personalized News Article Recommendation
- Classification with Noisy Labels by Importance Reweighting
- Estimating divergence functionals and the likelihood ratio by convex risk minimization
- Bayesian Online Changepoint Detection
- Marginalized Denoising Autoencoders for Domain Adaptation
- Deep Anomaly Detection Using Geometric Transformations
- Doubly Robust Policy Evaluation and Learning
- Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning
- Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models
- Mitigating Overfitting in Supervised Classification from Two Unlabeled Datasets: A Consistent Risk Correction Approach
- More Efficient Off-Policy Evaluation through Regularized Targeted Learning
- Off-Policy Evaluation and Learning for External Validity under a Covariate Shift
- Identifying Different Definitions of Future in the Assessment of Future Economic Conditions: Application of PU Learning and Text Mining