Surrogate Losses in Passive and Active Learning
arXiv:1207.3772 · doi:10.1214/19-EJS1635
Abstract
Active learning is a type of sequential design for supervised machine learning, in which the learning algorithm sequentially requests the labels of selected instances from a large pool of unlabeled data points. The objective is to produce a classifier of relatively low risk, as measured under the 0-1 loss, ideally using fewer label requests than the number of random labeled data points sufficient to achieve the same. This work investigates the potential uses of surrogate loss functions in the context of active learning. Specifically, it presents an active learning algorithm based on an arbitrary classification-calibrated surrogate loss function, along with an analysis of the number of label requests sufficient for the classifier returned by the algorithm to achieve a given risk under the 0-1 loss. Interestingly, these results cannot be obtained by simply optimizing the surrogate risk via active learning to an extent sufficient to provide a guarantee on the 0-1 loss, as is common practice in the analysis of surrogate losses for passive learning. Some of the results have additional implications for the use of surrogate losses in passive learning.
References in corpus (9)
- 2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization
- Local Rademacher complexities
- Fast learning rates for plug-in classifiers
- Concentration inequalities and asymptotic results for ratio type empirical processes
- Activized Learning: Transforming Passive to Active with Improved Label Complexity
- Rates of convergence in active learning
- Plug-in Approach to Active Learning
- A note on active learning for smooth problems
- Adaptivity to Noise Parameters in Nonparametric Active Learning
Cited by in corpus (11)
- Beyond Disagreement-based Agnostic Active Learning
- Building Bridges: Viewing Active Learning from the Multi-Armed Bandit Lens
- Exponential Savings in Agnostic Active Learning through Abstention
- Efficient Learning of Linear Separators under Bounded Noise
- Refined Error Bounds for Several Learning Algorithms
- A Compression Technique for Analyzing Disagreement-Based Active Learning
- Minimax Analysis of Active Learning
- Active Robust Learning
- Amortized Synthesis of Constrained Configurations Using a Differentiable Surrogate
- Noise-Tolerant Interactive Learning from Pairwise Comparisons
- Beating the Minimax Rate of Active Learning with Prior Knowledge