Positive-Unlabeled Learning with Non-Negative Risk Estimator

Abstract

From only positive (P) and unlabeled (U) data, a binary classifier could be trained with PU learning, in which the state of the art is unbiased PU learning. However, if its model is very flexible, empirical risks on training data will go negative, and we will suffer from serious overfitting. In this paper, we propose a non-negative risk estimator for PU learning: when getting minimized, it is more robust against overfitting, and thus we are able to use very flexible models (such as deep neural networks) given limited P data. Moreover, we analyze the bias, consistency, and mean-squared-error reduction of the proposed risk estimator, and bound the estimation error of the resulting empirical risk minimizer. Experiments demonstrate that our risk estimator fixes the overfitting problem of its unbiased counterparts.

arXiv commentNIPS 2017 camera-ready version (this paper was selected for oral presentation)

References in corpus (7)

• PU Learning for Matrix Completion
• Mixture Proportion Estimation via Kernel Embedding of Distributions
• Estimating the class prior and posterior from noisy positives and unlabeled data
• Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning
• Efficient Training for Positive Unlabeled Learning
• Loss factorization, weakly supervised learning and label noise robustness
• Semi-Supervised Classification Based on Classification from Positive and Unlabeled Data

Cited by in corpus (23)

• Learning from positive and unlabeled data: a survey
• Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels
• A Survey of Label-noise Representation Learning: Past, Present and Future
• Semi-Supervised Classification Based on Classification from Positive and Unlabeled Data
• Positive-Unlabeled Learning with Adversarial Data Augmentation for Knowledge Graph Completion
• Learning Graph Neural Networks with Positive and Unlabeled Nodes
• Classification from Pairwise Similarity and Unlabeled Data
• Binary classification with ambiguous training data
• Principled analytic classifier for positive-unlabeled learning via weighted integral probability metric
• Butterfly: One-step Approach towards Wildly Unsupervised Domain Adaptation
• Federated Learning with Positive and Unlabeled Data
• ScarceGAN: Discriminative Classification Framework for Rare Class Identification for Longitudinal Data with Weak Prior
• Information-Theoretic Representation Learning for Positive-Unlabeled Classification
• Lower-Bounded Proper Losses for Weakly Supervised Classification
• Learning Classifiers on Positive and Unlabeled Data with Policy Gradient
• Stochastic Optimization for Non-convex Inf-Projection Problems
• Generative-Discriminative Complementary Learning
• Stochastic Optimization for DC Functions and Non-smooth Non-convex Regularizers with Non-asymptotic Convergence
• Box-Adapt: Domain-Adaptive Medical Image Segmentation using Bounding BoxSupervision
• Density-Ratio Based Personalised Ranking from Implicit Feedback
• Binary matrix completion with nonconvex regularizers
• Towards Explainable Fact Checking
• Active Refinement for Multi-Label Learning: A Pseudo-Label Approach