One-bit Supervision for Image Classification

Abstract

This paper presents one-bit supervision, a novel setting of learning from incomplete annotations, in the scenario of image classification. Instead of training a model upon the accurate label of each sample, our setting requires the model to query with a predicted label of each sample and learn from the answer whether the guess is correct. This provides one bit (yes or no) of information, and more importantly, annotating each sample becomes much easier than finding the accurate label from many candidate classes. There are two keys to training a model upon one-bit supervision: improving the guess accuracy and making use of incorrect guesses. For these purposes, we propose a multi-stage training paradigm which incorporates negative label suppression into an off-the-shelf semi-supervised learning algorithm. In three popular image classification benchmarks, our approach claims higher efficiency in utilizing the limited amount of annotations.

References in corpus (9)

• Distilling the Knowledge in a Neural Network
• FitNets: Hints for Thin Deep Nets
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Temporal Ensembling for Semi-Supervised Learning
• Cost-Effective Active Learning for Deep Image Classification
• Born Again Neural Networks
• Shake-Shake regularization
• Knowledge Distillation in Generations: More Tolerant Teachers Educate Better Students
• Bayesian Batch Active Learning as Sparse Subset Approximation

Cited by in corpus (2)

• Overcoming Failures of Imagination in AI Infused System Development and Deployment
• One-bit Supervision for Image Classification: Problem, Solution, and Beyond