Deep Active Learning over the Long Tail

Abstract

This paper is concerned with pool-based active learning for deep neural networks. Motivated by coreset dataset compression ideas, we present a novel active learning algorithm that queries consecutive points from the pool using farthest-first traversals in the space of neural activation over a representation layer. We show consistent and overwhelming improvement in sample complexity over passive learning (random sampling) for three datasets: MNIST, CIFAR-10, and CIFAR-100. In addition, our algorithm outperforms the traditional uncertainty sampling technique (obtained using softmax activations), and we identify cases where uncertainty sampling is only slightly better than random sampling.

References in corpus (3)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Bayesian Active Learning with Image Data
• Efficient and Parsimonious Agnostic Active Learning

Cited by in corpus (24)

• Deep Anomaly Detection Using Geometric Transformations
• Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds
• Class-Balanced Loss Based on Effective Number of Samples
• Discriminative Active Learning
• Federated Active Learning (F-AL): an Efficient Annotation Strategy for Federated Learning
• Deep Active Learning for Computer Vision: Past and Future
• ScatterShot: Interactive In-context Example Curation for Text Transformation
• Active Learning for Deep Detection Neural Networks
• Gone Fishing: Neural Active Learning with Fisher Embeddings
• Deep Active Learning: Unified and Principled Method for Query and Training
• Feature Space Augmentation for Long-Tailed Data
• On the Relationship between Data Efficiency and Error for Uncertainty Sampling
• Active Learning with TensorBoard Projector
• Diffusion-based Deep Active Learning
• Deep Multi-Fidelity Active Learning of High-dimensional Outputs
• FLANNEL: Focal Loss Based Neural Network Ensemble for COVID-19 Detection
• Multi-Level Analyzation of Imbalance to Resolve Non-IID-Ness in Federated Learning
• Synthetic Information towards Maximum Posterior Ratio for deep learning on Imbalanced Data
• Feature Space Exploration For Planning Initial Benthic AUV Surveys
• Crowd Sourcing based Active Learning Approach for Parking Sign Recognition
• Information Condensing Active Learning
• Ask-n-Learn: Active Learning via Reliable Gradient Representations for Image Classification
• Low-Regret Active learning
• Active Learning of Neural Collision Handler for Complex 3D Mesh Deformations