DEUCE: Dual-diversity Enhancement and Uncertainty-awareness for Cold-start Active Learning

Abstract

Cold-start active learning (CSAL) selects valuable instances from an unlabeled dataset for manual annotation. It provides high-quality data at a low annotation cost for label-scarce text classification. However, existing CSAL methods overlook weak classes and hard representative examples, resulting in biased learning. To address these issues, this paper proposes a novel dual-diversity enhancing and uncertainty-aware (DEUCE) framework for CSAL. Specifically, DEUCE leverages a pretrained language model (PLM) to efficiently extract textual representations, class predictions, and predictive uncertainty. Then, it constructs a Dual-Neighbor Graph (DNG) to combine information on both textual diversity and class diversity, ensuring a balanced data distribution. It further propagates uncertainty information via density-based clustering to select hard representative instances. DEUCE performs well in selecting class-balanced and hard representative data by dual-diversity and informativeness. Experiments on six NLP datasets demonstrate the superiority and efficiency of DEUCE.

18 pages, 3 figures, 12 tables. Accepted manuscript by TACL. For published version by MIT Press, see https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00731/125950

References in corpus (10)

• Llama 2: Open Foundation and Fine-Tuned Chat Models
• LoRA: Low-Rank Adaptation of Large Language Models
• Unifying Large Language Models and Knowledge Graphs: A Roadmap
• Green AI: A Preliminary Empirical Study on Energy Consumption in DL Models Across Different Runtime Infrastructures
• Calibration in Deep Learning: A Survey of the State-of-the-Art
• DebCSE: Rethinking Unsupervised Contrastive Sentence Embedding Learning in the Debiasing Perspective
• Utilising a Large Language Model to Annotate Subject Metadata: A Case Study in an Australian National Research Data Catalogue
• Addressing practical challenges in Active Learning via a hybrid query strategy
• Human Still Wins over LLM: An Empirical Study of Active Learning on Domain-Specific Annotation Tasks
• Cold Start Active Learning Strategies in the Context of Imbalanced Classification