Learning Active Learning from Data

Abstract

In this paper, we suggest a novel data-driven approach to active learning (AL). The key idea is to train a regressor that predicts the expected error reduction for a candidate sample in a particular learning state. By formulating the query selection procedure as a regression problem we are not restricted to working with existing AL heuristics; instead, we learn strategies based on experience from previous AL outcomes. We show that a strategy can be learnt either from simple synthetic 2D datasets or from a subset of domain-specific data. Our method yields strategies that work well on real data from a wide range of domains.

Cited by in corpus (43)

• A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis
• TDACNN: Target-domain-free Domain Adaptation Convolutional Neural Network for Drift Compensation in Gas Sensors
• Active, Continual Fine Tuning of Convolutional Neural Networks for Reducing Annotation Efforts
• A Survey of Active Learning for Text Classification using Deep Neural Networks
• Artificial Intelligence in Glioma Imaging: Challenges and Advances
• Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data
• ALiPy: Active Learning in Python
• Adaptive Self-training for Few-shot Neural Sequence Labeling
• Active Learning: Problem Settings and Recent Developments
• Reinforced active learning for image segmentation
• An Empirical Analysis of Deep Learning for Cardinality Estimation
• Stopping Active Learning based on Predicted Change of F Measure for Text Classification
• Using Error Decay Prediction to Overcome Practical Issues of Deep Active Learning for Named Entity Recognition
• Adaptive LiDAR Sampling and Depth Completion using Ensemble Variance
• Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Cost
• Meta-Learning for Contextual Bandit Exploration
• Adapting Behaviour via Intrinsic Reward: A Survey and Empirical Study
• Active Feature Acquisition with Generative Surrogate Models
• Learning Intelligent Dialogs for Bounding Box Annotation
• Stopping Criterion for Active Learning Based on Error Stability
• Learning to Sample the Most Useful Training Patches from Images
• Multiple-criteria Based Active Learning with Fixed-size Determinantal Point Processes
• Which Strategies Matter for Noisy Label Classification? Insight into Loss and Uncertainty
• Information-Guided Sampling for Low-Rank Matrix Completion
• Enabling On-Device CNN Training by Self-Supervised Instance Filtering and Error Map Pruning
• Improved Synthetic Training for Reading Comprehension
• Towards more Reliable Transfer Learning
• Interpret-able feedback for AutoML systems
• Ask-n-Learn: Active Learning via Reliable Gradient Representations for Image Classification
• Distribution Aware Active Learning
• Not Your Grandfathers Test Set: Reducing Labeling Effort for Testing
• Learning active learning at the crossroads? evaluation and discussion
• Learning to Sample: an Active Learning Framework
• Towards Understanding the Behaviors of Optimal Deep Active Learning Algorithms
• Robust Active Learning for Electrocardiographic Signal Classification
• Learning to Actively Learn: A Robust Approach
• ALdataset: a benchmark for pool-based active learning
• On Training Instance Selection for Few-Shot Neural Text Generation
• ImitAL: Learning Active Learning Strategies from Synthetic Data
• SoQal: Selective Oracle Questioning in Active Learning
• Batch Decorrelation for Active Metric Learning
• Bias-Aware Heapified Policy for Active Learning
• Meta-Active Learning for Node Response Prediction in Graphs