Bayesian Batch Active Learning as Sparse Subset Approximation

Abstract

Leveraging the wealth of unlabeled data produced in recent years provides great potential for improving supervised models. When the cost of acquiring labels is high, probabilistic active learning methods can be used to greedily select the most informative data points to be labeled. However, for many large-scale problems standard greedy procedures become computationally infeasible and suffer from negligible model change. In this paper, we introduce a novel Bayesian batch active learning approach that mitigates these issues. Our approach is motivated by approximating the complete data posterior of the model parameters. While naive batch construction methods result in correlated queries, our algorithm produces diverse batches that enable efficient active learning at scale. We derive interpretable closed-form solutions akin to existing active learning procedures for linear models, and generalize to arbitrary models using random projections. We demonstrate the benefits of our approach on several large-scale regression and classification tasks.

arXiv commentNeurIPS 2019

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Weight Uncertainty in Neural Networks
• Deep Bayesian Active Learning with Image Data
• Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
• Bayesian Active Learning for Classification and Preference Learning
• Variational Dropout and the Local Reparameterization Trick
• Batch Bayesian Optimization via Local Penalization
• Coresets for Scalable Bayesian Logistic Regression
• Automated Scalable Bayesian Inference via Hilbert Coresets
• Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functions
• Leveraged volume sampling for linear regression

Cited by in corpus (12)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Task-Aware Variational Adversarial Active Learning
• Adversarial Representation Active Learning
• Benchmarking the Neural Linear Model for Regression
• Bayesian Deep Learning via Subnetwork Inference
• Deep Bayesian Active Learning, A Brief Survey on Recent Advances
• Sequential Graph Convolutional Network for Active Learning
• One-bit Supervision for Image Classification
• One-bit Supervision for Image Classification: Problem, Solution, and Beyond
• One Backward from Ten Forward, Subsampling for Large-Scale Deep Learning
• Bayesian Coresets: Revisiting the Nonconvex Optimization Perspective
• Active Learning for Bayesian 3D Hand Pose Estimation