Learning how to Active Learn: A Deep Reinforcement Learning Approach

Abstract

Active learning aims to select a small subset of data for annotation such that a classifier learned on the data is highly accurate. This is usually done using heuristic selection methods, however the effectiveness of such methods is limited and moreover, the performance of heuristics varies between datasets. To address these shortcomings, we introduce a novel formulation by reframing the active learning as a reinforcement learning problem and explicitly learning a data selection policy, where the policy takes the role of the active learning heuristic. Importantly, our method allows the selection policy learned using simulation on one language to be transferred to other languages. We demonstrate our method using cross-lingual named entity recognition, observing uniform improvements over traditional active learning.

To appear in EMNLP 2017

References in corpus (4)

• Massively Parallel Methods for Deep Reinforcement Learning
• Massively Multilingual Word Embeddings
• Move Evaluation in Go Using Deep Convolutional Neural Networks
• Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning

Cited by in corpus (20)

• Deep Reinforcement Learning: An Overview
• Diminishing Uncertainty within the Training Pool: Active Learning for Medical Image Segmentation
• Robust Distant Supervision Relation Extraction via Deep Reinforcement Learning
• A non-cooperative meta-modeling game for automated third-party calibrating, validating, and falsifying constitutive laws with parallelized adversarial attacks
• Large-Scale Visual Active Learning with Deep Probabilistic Ensembles
• Using Error Decay Prediction to Overcome Practical Issues of Deep Active Learning for Named Entity Recognition
• Improving Neural Relation Extraction with Positive and Unlabeled Learning
• Capturing Label Distribution: A Case Study in NLI
• Multiple-criteria Based Active Learning with Fixed-size Determinantal Point Processes
• Few Is Enough: Task-Augmented Active Meta-Learning for Brain Cell Classification
• Visual Curiosity: Learning to Ask Questions to Learn Visual Recognition
• Ask-n-Learn: Active Learning via Reliable Gradient Representations for Image Classification
• Learning How to Self-Learn: Enhancing Self-Training Using Neural Reinforcement Learning
• Deep Reinforcement Learning for Fog Computing-based Vehicular System with Multi-operator Support
• On the Robustness of Active Learning
• Deep Reinforcement Learning for Dynamic Spectrum Sensing and Aggregation in Multi-Channel Wireless Networks
• Learning to Expand: Reinforced Pseudo-relevance Feedback Selection for Information-seeking Conversations
• Learning to Actively Learn: A Robust Approach
• Bias-Aware Heapified Policy for Active Learning
• Leveraging Motion Priors in Videos for Improving Human Segmentation