Noisy Batch Active Learning with Deterministic Annealing
arXiv:1909.12473
Abstract
We study the problem of training machine learning models incrementally with batches of samples annotated with noisy oracles. We select each batch of samples that are important and also diverse via clustering and importance sampling. More importantly, we incorporate model uncertainty into the sampling probability to compensate for poor estimation of the importance scores when the training data is too small to build a meaningful model. Experiments on benchmark image classification datasets (MNIST, SVHN, CIFAR10, and EMNIST) show improvement over existing active learning strategies. We introduce an extra denoising layer to deep networks to make active learning robust to label noises and show significant improvements.
References in corpus (9)
- Scikit-learn: Machine Learning in Python
- What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
- Bayesian Active Learning for Classification and Preference Learning
- EMNIST: an extension of MNIST to handwritten letters
- BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning
- Bayesian Uncertainty Estimation for Batch Normalized Deep Networks
- Learning From Noisy Singly-labeled Data
- UPAL: Unbiased Pool Based Active Learning
- Noisy Bayesian Active Learning