A Benchmark and Comparison of Active Learning for Logistic Regression

Abstract

Logistic regression is by far the most widely used classifier in real-world applications. In this paper, we benchmark the state-of-the-art active learning methods for logistic regression and discuss and illustrate their underlying characteristics. Experiments are carried out on three synthetic datasets and 44 real-world datasets, providing insight into the behaviors of these active learning methods with respect to the area of the learning curve (which plots classification accuracy as a function of the number of queried examples) and their computational costs. Surprisingly, one of the earliest and simplest suggested active learning methods, i.e., uncertainty sampling, performs exceptionally well overall. Another remarkable finding is that random sampling, which is the rudimentary baseline to improve upon, is not overwhelmed by individual active learning techniques in many cases.

accepted by Pattern Recognition

References in corpus (7)

• Cost-Effective Active Learning for Deep Image Classification
• Generative Adversarial Active Learning
• Agnostic Active Learning Without Constraints
• A Variance Maximization Criterion for Active Learning
• Bag-Level Aggregation for Multiple Instance Active Learning in Instance Classification Problems
• On Measuring and Quantifying Performance: Error Rates, Surrogate Loss, and an Example in SSL
• Active Decision Boundary Annotation with Deep Generative Models

Cited by in corpus (23)

• Discriminative Active Learning
• Active emulation of computer codes with Gaussian processes -- Application to remote sensing
• Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience
• Active Learning to Classify Macromolecular Structures in situ for Less Supervision in Cryo-Electron Tomography
• Active Reward Learning for Co-Robotic Vision Based Exploration in Bandwidth Limited Environments
• Hitting the Target: Stopping Active Learning at the Cost-Based Optimum
• From Weakly Supervised Learning to Biquality Learning: an Introduction
• On the Relationship between Data Efficiency and Error for Uncertainty Sampling
• Incorporating Unlabeled Data into Distributionally Robust Learning
• Active Learning for Computationally Efficient Distribution of Binary Evolution Simulations
• Relaxed Softmax for learning from Positive and Unlabeled data
• Neural Networks Are More Productive Teachers Than Human Raters: Active Mixup for Data-Efficient Knowledge Distillation from a Blackbox Model
• Understanding Goal-Oriented Active Learning via Influence Functions
• Active Deep Learning on Entity Resolution by Risk Sampling
• Predictive case control designs for modification learning
• Learning active learning at the crossroads? evaluation and discussion
• On the Importance of Adaptive Data Collection for Extremely Imbalanced Pairwise Tasks
• Predicting the Results of LTL Model Checking using Multiple Machine Learning Algorithms
• FrameScope: Temporal Data Valuation for Stream Active Learning in Autonomous Vehicle Systems
• Active Learning for Argument Mining: A Practical Approach
• Learning to Rank for Active Learning: A Listwise Approach
• Feedback Coding for Active Learning
• Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering