Diffusion-based Deep Active Learning

Abstract

The remarkable performance of deep neural networks depends on the availability of massive labeled data. To alleviate the load of data annotation, active deep learning aims to select a minimal set of training points to be labelled which yields maximal model accuracy. Most existing approaches implement either an `exploration'-type selection criterion, which aims at exploring the joint distribution of data and labels, or a `refinement'-type criterion which aims at localizing the detected decision boundaries. We propose a versatile and efficient criterion that automatically switches from exploration to refinement when the distribution has been sufficiently mapped. Our criterion relies on a process of diffusing the existing label information over a graph constructed from the hidden representation of the data set as provided by the neural network. This graph representation captures the intrinsic geometry of the approximated labeling function. The diffusion-based criterion is shown to be advantageous as it outperforms existing criteria for deep active learning.

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Scikit-learn: Machine Learning in Python
• Cost-Effective Active Learning for Deep Image Classification
• Adversarial Active Learning for Deep Networks: a Margin Based Approach
• Active Self-Paced Learning for Cost-Effective and Progressive Face Identification
• Discriminative Active Learning
• Deep Active Learning over the Long Tail

Cited by in corpus (4)

• Novel Batch Active Learning Approach and Its Application to Synthetic Aperture Radar Datasets
• Model-Change Active Learning in Graph-Based Semi-Supervised Learning
• Efficient Graph-Based Active Learning with Probit Likelihood via Gaussian Approximations
• An efficient scheme based on graph centrality to select nodes for training for effective learning