Deep Active Learning for Text Classification with Diverse Interpretations
arXiv:2108.10687 · doi:10.1145/3459637.3482080
Abstract
Recently, Deep Neural Networks (DNNs) have made remarkable progress for text classification, which, however, still require a large number of labeled data. To train high-performing models with the minimal annotation cost, active learning is proposed to select and label the most informative samples, yet it is still challenging to measure informativeness of samples used in DNNs. In this paper, inspired by piece-wise linear interpretability of DNNs, we propose a novel Active Learning with DivErse iNterpretations (ALDEN) approach. With local interpretations in DNNs, ALDEN identifies linearly separable regions of samples. Then, it selects samples according to their diversity of local interpretations and queries their labels. To tackle the text classification problem, we choose the word with the most diverse interpretations to represent the whole sentence. Extensive experiments demonstrate that ALDEN consistently outperforms several state-of-the-art deep active learning methods.
arXiv commentAccepted to CIKM 2021. Authors' version
References in corpus (14)
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
- Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
- On the Number of Linear Regions of Deep Neural Networks
- SmoothGrad: removing noise by adding noise
- Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
- Active Learning for Convolutional Neural Networks: A Core-Set Approach
- A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts
- Deep Bayesian Active Learning with Image Data
- Bayesian Active Learning for Classification and Preference Learning
- Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds
- Visualizing and Understanding Neural Models in NLP
- Active Learning for Wireless IoT Intrusion Detection
- Active Learning for Speech Recognition: the Power of Gradients
- Exact and Consistent Interpretation for Piecewise Linear Neural Networks: A Closed Form Solution