Impact of Batch Size on Stopping Active Learning for Text Classification

Abstract

When using active learning, smaller batch sizes are typically more efficient from a learning efficiency perspective. However, in practice due to speed and human annotator considerations, the use of larger batch sizes is necessary. While past work has shown that larger batch sizes decrease learning efficiency from a learning curve perspective, it remains an open question how batch size impacts methods for stopping active learning. We find that large batch sizes degrade the performance of a leading stopping method over and above the degradation that results from reduced learning efficiency. We analyze this degradation and find that it can be mitigated by changing the window size parameter of how many past iterations of learning are taken into account when making the stopping decision. We find that when using larger batch sizes, stopping methods are more effective when smaller window sizes are used.

arXiv comment2 pages, 1 table; published in Proceedings of the IEEE 12th International Conference on Semantic Computing (ICSC 2018), Laguna Hills, CA, USA, pages 306-307, January 2018

References in corpus (3)

• Support Vector Machine Active Learning Algorithms with Query-by-Committee versus Closest-to-Hyperplane Selection
• Analysis of Stopping Active Learning based on Stabilizing Predictions
• Filtering Tweets for Social Unrest

Cited by in corpus (5)

• Support Vector Machine Active Learning Algorithms with Query-by-Committee versus Closest-to-Hyperplane Selection
• Stopping Active Learning based on Predicted Change of F Measure for Text Classification
• Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience
• The Use of Unlabeled Data versus Labeled Data for Stopping Active Learning for Text Classification
• Early Forecasting of Text Classification Accuracy and F-Measure with Active Learning