Beyond Disagreement-based Agnostic Active Learning

Abstract

We study agnostic active learning, where the goal is to learn a classifier in a pre-specified hypothesis class interactively with as few label queries as possible, while making no assumptions on the true function generating the labels. The main algorithms for this problem are {\em{disagreement-based active learning}}, which has a high label requirement, and {\em{margin-based active learning}}, which only applies to fairly restricted settings. A major challenge is to find an algorithm which achieves better label complexity, is consistent in an agnostic setting, and applies to general classification problems. In this paper, we provide such an algorithm. Our solution is based on two novel contributions -- a reduction from consistent active learning to confidence-rated prediction with guaranteed error, and a novel confidence-rated predictor.

References in corpus (3)

• A tutorial on conformal prediction
• Surrogate Losses in Passive and Active Learning
• Noisy Bayesian Active Learning

Cited by in corpus (20)

• Active Learning from Weak and Strong Labelers
• Exploring Connections Between Active Learning and Model Extraction
• Efficient and Parsimonious Agnostic Active Learning
• Asymptotic Analysis of Objectives based on Fisher Information in Active Learning
• Noise-adaptive Margin-based Active Learning and Lower Bounds under Tsybakov Noise Condition
• The Extended Littlestone's Dimension for Learning with Mistakes and Abstentions
• Exponential Savings in Agnostic Active Learning through Abstention
• Online Active Model Selection for Pre-trained Classifiers
• Exploiting Context for Robustness to Label Noise in Active Learning
• Search Improves Label for Active Learning
• Convergence Rates of Active Learning for Maximum Likelihood Estimation
• Refined Error Bounds for Several Learning Algorithms
• Revisiting Model-Agnostic Private Learning: Faster Rates and Active Learning
• When are epsilon-nets small?
• On the Power of Localized Perceptron for Label-Optimal Learning of Halfspaces with Adversarial Noise
• Efficient active learning of sparse halfspaces
• Noise-Tolerant Interactive Learning from Pairwise Comparisons
• Bias-Aware Heapified Policy for Active Learning
• Improved Algorithms for Efficient Active Learning Halfspaces with Massart and Tsybakov noise
• Chernoff Sampling for Active Testing and Extension to Active Regression