The Use of Unlabeled Data versus Labeled Data for Stopping Active Learning for Text Classification

Abstract

Annotation of training data is the major bottleneck in the creation of text classification systems. Active learning is a commonly used technique to reduce the amount of training data one needs to label. A crucial aspect of active learning is determining when to stop labeling data. Three potential sources for informing when to stop active learning are an additional labeled set of data, an unlabeled set of data, and the training data that is labeled during the process of active learning. To date, no one has compared and contrasted the advantages and disadvantages of stopping methods based on these three information sources. We find that stopping methods that use unlabeled data are more effective than methods that use labeled data.

arXiv comment8 pages, 4 figures, 3 tables; published in Proceedings of the IEEE 13th International Conference on Semantic Computing (ICSC), Newport Beach, CA, USA, pages 287-294, January 2019

References in corpus (6)

• Bucking the Trend: Large-Scale Cost-Focused Active Learning for Statistical Machine Translation
• Support Vector Machine Active Learning Algorithms with Query-by-Committee versus Closest-to-Hyperplane Selection
• Analysis of Stopping Active Learning based on Stabilizing Predictions
• Stopping Active Learning based on Predicted Change of F Measure for Text Classification
• Impact of Batch Size on Stopping Active Learning for Text Classification
• Filtering Tweets for Social Unrest

Cited by in corpus (4)

• Discovery of Self-Assembling $\pi$-Conjugated Peptides by Active Learning-Directed Coarse-Grained Molecular Simulation
• Stopping Active Learning based on Predicted Change of F Measure for Text Classification
• Hitting the Target: Stopping Active Learning at the Cost-Based Optimum
• Early Forecasting of Text Classification Accuracy and F-Measure with Active Learning