On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data

Abstract

Empirical risk minimization (ERM), with proper loss function and regularization, is the common practice of supervised classification. In this paper, we study training arbitrary (from linear to deep) binary classifier from only unlabeled (U) data by ERM. We prove that it is impossible to estimate the risk of an arbitrary binary classifier in an unbiased manner given a single set of U data, but it becomes possible given two sets of U data with different class priors. These two facts answer a fundamental question---what the minimal supervision is for training any binary classifier from only U data. Following these findings, we propose an ERM-based learning method from two sets of U data, and then prove it is consistent. Experiments demonstrate the proposed method could train deep models and outperform state-of-the-art methods for learning from two sets of U data.

References in corpus (12)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Learning to Reweight Examples for Robust Deep Learning
• Classification with Noisy Labels by Importance Reweighting
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• Distributional Smoothing with Virtual Adversarial Training
• Learning Discrete Representations via Information Maximizing Self-Augmented Training
• Masking: A New Perspective of Noisy Supervision
• Composite Binary Losses
• Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning
• Semi-Supervised Learning via Compact Latent Space Clustering

Cited by in corpus (18)

• Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning
• Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates
• On Symmetric Losses for Learning from Corrupted Labels
• Provably Consistent Partial-Label Learning
• Progressive Identification of True Labels for Partial-Label Learning
• Mitigating Overfitting in Supervised Classification from Two Unlabeled Datasets: A Consistent Risk Correction Approach
• Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization
• Learning with Multiple Complementary Labels
• Learning from Positive and Unlabeled Data with Arbitrary Positive Shift
• Complementary-Label Learning for Arbitrary Losses and Models
• Rethinking Importance Weighting for Deep Learning under Distribution Shift
• Multi-Complementary and Unlabeled Learning for Arbitrary Losses and Models
• Robust Imitation Learning from Noisy Demonstrations
• When Optimizing $f$-divergence is Robust with Label Noise
• Pointwise Binary Classification with Pairwise Confidence Comparisons
• Positive-Unlabeled Classification under Class Prior Shift and Asymmetric Error
• Binary Classification from Multiple Unlabeled Datasets via Surrogate Set Classification
• A Symmetric Loss Perspective of Reliable Machine Learning