Bayesian active learning for production, a systematic study and a reusable library
arXiv:2006.09916
Abstract
Active learning is able to reduce the amount of labelling effort by using a machine learning model to query the user for specific inputs. While there are many papers on new active learning techniques, these techniques rarely satisfy the constraints of a real-world project. In this paper, we analyse the main drawbacks of current active learning techniques and we present approaches to alleviate them. We do a systematic study on the effects of the most common issues of real-world datasets on the deep active learning process: model convergence, annotation error, and dataset imbalance. We derive two techniques that can speed up the active learning loop such as partial uncertainty sampling and larger query size. Finally, we present our open-source Bayesian active learning library, BaaL.
10 pages, 6 figures
References in corpus (8)
- What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
- Weight Uncertainty in Neural Networks
- YouTube-8M: A Large-Scale Video Classification Benchmark
- Bayesian Active Learning for Classification and Preference Learning
- Bayesian Deep Learning and a Probabilistic Perspective of Generalization
- A Simple Baseline for Bayesian Uncertainty in Deep Learning
- Deeper Connections between Neural Networks and Gaussian Processes Speed-up Active Learning
- Evaluating Scalable Uncertainty Estimation Methods for DNN-Based Molecular Property Prediction