Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning

Abstract

In PU learning, a binary classifier is trained from positive (P) and unlabeled (U) data without negative (N) data. Although N data is missing, it sometimes outperforms PN learning (i.e., ordinary supervised learning). Hitherto, neither theoretical nor experimental analysis has been given to explain this phenomenon. In this paper, we theoretically compare PU (and NU) learning against PN learning based on the upper bounds on estimation errors. We find simple conditions when PU and NU learning are likely to outperform PN learning, and we prove that, in terms of the upper bounds, either PU or NU learning (depending on the class-prior probability and the sizes of P and N data) given infinite U data will improve on PN learning. Our theoretical findings well agree with the experimental results on artificial and benchmark data even when the experimental setup does not match the theoretical assumptions exactly.

arXiv commentNIPS 2016 camera-ready version

References in corpus (1)

• Mixture Proportion Estimation via Kernel Embedding of Distributions

Cited by in corpus (29)

• Positive-Unlabeled Learning with Non-Negative Risk Estimator
• Efficient Training for Positive Unlabeled Learning
• Combating noisy labels by agreement: A joint training method with co-regularization
• On Symmetric Losses for Learning from Corrupted Labels
• Semi-Supervised AUC Optimization based on Positive-Unlabeled Learning
• Provably Consistent Partial-Label Learning
• Compete to Win: Enhancing Pseudo Labels for Barely-supervised Medical Image Segmentation
• On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data
• Semi-Supervised Classification Based on Classification from Positive and Unlabeled Data
• Mitigating Overfitting in Supervised Classification from Two Unlabeled Datasets: A Consistent Risk Correction Approach
• Learning from Positive and Unlabeled Data with Arbitrary Positive Shift
• Self-supervised learning for autonomous vehicles perception: A conciliation between analytical and learning methods
• Classification from Pairwise Similarity and Unlabeled Data
• Positive-Unlabeled Reward Learning
• Learning from a Complementary-label Source Domain: Theory and Algorithms
• Principled analytic classifier for positive-unlabeled learning via weighted integral probability metric
• Multi-Complementary and Unlabeled Learning for Arbitrary Losses and Models
• Non-Negative Bregman Divergence Minimization for Deep Direct Density Ratio Estimation
• GAN-based Recommendation with Positive-Unlabeled Sampling
• Uncoupled Regression from Pairwise Comparison Data
• Revisiting Sample Selection Approach to Positive-Unlabeled Learning: Turning Unlabeled Data into Positive rather than Negative
• Pointwise Binary Classification with Pairwise Confidence Comparisons
• Binary Matrix Completion Using Unobserved Entries
• A method on selecting reliable samples based on fuzziness in positive and unlabeled learning
• Positive-Unlabeled Classification under Class Prior Shift and Asymmetric Error
• Probabilistic Decoupling of Labels in Classification
• A Symmetric Loss Perspective of Reliable Machine Learning
• A deep multiple instance learning approach based on coarse labels for high-resolution land-cover mapping
• Active Refinement for Multi-Label Learning: A Pseudo-Label Approach