Parting with Illusions about Deep Active Learning

Abstract

Active learning aims to reduce the high labeling cost involved in training machine learning models on large datasets by efficiently labeling only the most informative samples. Recently, deep active learning has shown success on various tasks. However, the conventional evaluation scheme used for deep active learning is below par. Current methods disregard some apparent parallel work in the closely related fields. Active learning methods are quite sensitive w.r.t. changes in the training procedure like data augmentation. They improve by a large-margin when integrated with semi-supervised learning, but barely perform better than the random baseline. We re-implement various latest active learning approaches for image classification and evaluate them under more realistic settings. We further validate our findings for semantic segmentation. Based on our observations, we realistically assess the current state of the field and propose a more suitable evaluation protocol.

References in corpus (8)

• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Improved Regularization of Convolutional Neural Networks with Cutout
• Unsupervised Data Augmentation for Consistency Training
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Cost-Effective Active Learning for Deep Image Classification
• Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
• There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average
• Active Learning for Deep Detection Neural Networks

Cited by in corpus (3)

• Inf-Net: Automatic COVID-19 Lung Infection Segmentation from CT Images
• Active Self-Semi-Supervised Learning for Few Labeled Samples
• Exploring UMAP in hybrid models of entropy-based and representativeness sampling for active learning in biomedical segmentation