Stopping Active Learning based on Predicted Change of F Measure for Text Classification

Abstract

During active learning, an effective stopping method allows users to limit the number of annotations, which is cost effective. In this paper, a new stopping method called Predicted Change of F Measure will be introduced that attempts to provide the users an estimate of how much performance of the model is changing at each iteration. This stopping method can be applied with any base learner. This method is useful for reducing the data annotation bottleneck encountered when building text classification systems.

8 pages, 12 tables; published in Proceedings of the 2019 IEEE 13th International Conference on Semantic Computing (ICSC), Newport Beach, CA, USA, pages 47-54, January 2019

References in corpus (3)

• Bucking the Trend: Large-Scale Cost-Focused Active Learning for Statistical Machine Translation
• Analysis of Stopping Active Learning based on Stabilizing Predictions
• The Use of Unlabeled Data versus Labeled Data for Stopping Active Learning for Text Classification

Cited by in corpus (5)

• Active Learning: Problem Settings and Recent Developments
• The Use of Unlabeled Data versus Labeled Data for Stopping Active Learning for Text Classification
• Computational catalyst discovery: Active classification through myopic multiscale sampling
• Stopping criterion for active learning based on deterministic generalization bounds
• Stopping Criterion for Active Learning Based on Error Stability