Active and Continuous Exploration with Deep Neural Networks and Expected Model Output Changes

Abstract

The demands on visual recognition systems do not end with the complexity offered by current large-scale image datasets, such as ImageNet. In consequence, we need curious and continuously learning algorithms that actively acquire knowledge about semantic concepts which are present in available unlabeled data. As a step towards this goal, we show how to perform continuous active learning and exploration, where an algorithm actively selects relevant batches of unlabeled examples for annotation. These examples could either belong to already known or to yet undiscovered classes. Our algorithm is based on a new generalization of the Expected Model Output Change principle for deep architectures and is especially tailored to deep neural networks. Furthermore, we show easy-to-implement approximations that yield efficient techniques for active selection. Empirical experiments show that our method outperforms currently used heuristics.

accepted contribution at NIPS 2016 Workshop on Continual Learning and Deep Networks

References in corpus (4)

• Bird Species Categorization Using Pose Normalized Deep Convolutional Nets
• Analyzing the Performance of Multilayer Neural Networks for Object Recognition
• The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition
• Active Sampler: Light-weight Accelerator for Complex Data Analytics at Scale

Cited by in corpus (7)

• Deep Ensemble Bayesian Active Learning : Addressing the Mode Collapse issue in Monte Carlo dropout via Ensembles
• Active and Incremental Learning with Weak Supervision
• Temporal Output Discrepancy for Loss Estimation-based Active Learning
• Multiple instance active learning for object detection
• Downstream-Pretext Domain Knowledge Traceback for Active Learning
• Domain Adaptation and Active Learning for Fine-Grained Recognition in the Field of Biodiversity
• State-Relabeling Adversarial Active Learning