BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning

Abstract

We develop BatchBALD, a tractable approximation to the mutual information between a batch of points and model parameters, which we use as an acquisition function to select multiple informative points jointly for the task of deep Bayesian active learning. BatchBALD is a greedy linear-time $1-\frac{1}{e}$-approximate algorithm amenable to dynamic programming and efficient caching. We compare BatchBALD to the commonly used approach for batch data acquisition and find that the current approach acquires similar and redundant points, sometimes performing worse than randomly acquiring data. We finish by showing that, using BatchBALD to consider dependencies within an acquisition batch, we achieve new state of the art performance on standard benchmarks, providing substantial data efficiency improvements in batch acquisition.

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Practical Bayesian Optimization of Machine Learning Algorithms
• Semi-Supervised Learning with Deep Generative Models
• Cost-Effective Active Learning for Deep Image Classification
• Semi-Supervised Learning with Ladder Networks
• Bayesian Active Learning for Classification and Preference Learning
• Batch Bayesian Optimization via Local Penalization
• Predictive Entropy Search for Multi-objective Bayesian Optimization
• Actively Learning what makes a Discrete Sequence Valid

Cited by in corpus (51)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis
• Directions in Abusive Language Training Data: Garbage In, Garbage Out
• Computer-Aided Multi-Objective Optimization in Small Molecule Discovery
• Selection via Proxy: Efficient Data Selection for Deep Learning
• Coresets via Bilevel Optimization for Continual Learning and Streaming
• On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty
• Uncertainty in Gradient Boosting via Ensembles
• On Statistical Bias In Active Learning: How and When To Fix It
• Uncertainty Estimation in Autoregressive Structured Prediction
• Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
• Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation
• Deep Indexed Active Learning for Matching Heterogeneous Entity Representations
• MUS-CDB: Mixed Uncertainty Sampling with Class Distribution Balancing for Active Annotation in Aerial Object Detection
• Active Learning of Driving Scenario Trajectories
• Better, Not Just More: Data-Centric Machine Learning for Earth Observation
• Task-Aware Variational Adversarial Active Learning
• A Survey on Cost Types, Interaction Schemes, and Annotator Performance Models in Selection Algorithms for Active Learning in Classification
• Bayesian Active Meta-Learning for Few Pilot Demodulation and Equalization
• Regression Prior Networks
• Model-Centric and Data-Centric Aspects of Active Learning for Deep Neural Networks
• Gone Fishing: Neural Active Learning with Fisher Embeddings
• Active Testing: Sample-Efficient Model Evaluation
• Semi-supervised Open-World Object Detection
• Unpacking Information Bottlenecks: Unifying Information-Theoretic Objectives in Deep Learning
• Composite Bayesian Optimization In Function Spaces Using NEON -- Neural Epistemic Operator Networks
• Scaling Ensemble Distribution Distillation to Many Classes with Proxy Targets
• Deep Active Learning by Leveraging Training Dynamics
• Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Data
• Test Distribution-Aware Active Learning: A Principled Approach Against Distribution Shift and Outliers
• Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks
• Deep Multi-Fidelity Active Learning of High-dimensional Outputs
• Sequential Graph Convolutional Network for Active Learning
• Cold-start Active Learning through Self-supervised Language Modeling
• Noisy Batch Active Learning with Deterministic Annealing
• Multi-class Text Classification using BERT-based Active Learning
• ReDAL: Region-based and Diversity-aware Active Learning for Point Cloud Semantic Segmentation
• One-Round Active Learning
• CLOPS: Continual Learning of Physiological Signals
• Bayesian Active Learning by Disagreements: A Geometric Perspective
• One-bit Supervision for Image Classification: Problem, Solution, and Beyond
• Looking at the posterior: accuracy and uncertainty of neural-network predictions
• Sample Noise Impact on Active Learning
• PowerEvaluationBALD: Efficient Evaluation-Oriented Deep (Bayesian) Active Learning with Stochastic Acquisition Functions
• VirAAL: Virtual Adversarial Active Learning For NLU
• Low-Regret Active learning
• Diversity-Aware Batch Active Learning for Dependency Parsing
• A Practical & Unified Notation for Information-Theoretic Quantities in ML
• Knowledge-driven Active Learning
• Reducing the Long Tail Losses in Scientific Emulations with Active Learning
• Neural Active Learning with Performance Guarantees