Denoising Multi-Source Weak Supervision for Neural Text Classification

Abstract

We study the problem of learning neural text classifiers without using any labeled data, but only easy-to-provide rules as multiple weak supervision sources. This problem is challenging because rule-induced weak labels are often noisy and incomplete. To address these two challenges, we design a label denoiser, which estimates the source reliability using a conditional soft attention mechanism and then reduces label noise by aggregating rule-annotated weak labels. The denoised pseudo labels then supervise a neural classifier to predicts soft labels for unmatched samples, which address the rule coverage issue. We evaluate our model on five benchmarks for sentiment, topic, and relation classifications. The results show that our model outperforms state-of-the-art weakly-supervised and semi-supervised methods consistently, and achieves comparable performance with fully-supervised methods even without any labeled data. Our code can be found at https://github.com/weakrules/Denoise-multi-weak-sources.

arXiv comment16 pages, 7 figures

References in corpus (16)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Character-level Convolutional Networks for Text Classification
• On Calibration of Modern Neural Networks
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Temporal Ensembling for Semi-Supervised Learning
• Learning to Reweight Examples for Robust Deep Learning
• Deep Learning for Hate Speech Detection in Tweets
• Snorkel: Rapid Training Data Creation with Weak Supervision
• Identifying Mislabeled Training Data
• Training Convolutional Networks with Noisy Labels
• Data Programming: Creating Large Training Sets, Quickly
• Weakly-Supervised Neural Text Classification
• WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations
• Avoiding Your Teacher's Mistakes: Training Neural Networks with Controlled Weak Supervision
• Approval Voting and Incentives in Crowdsourcing
• Learning from Rules Generalizing Labeled Exemplars

Cited by in corpus (10)

• Challenges in Deploying Machine Learning: a Survey of Case Studies
• WRENCH: A Comprehensive Benchmark for Weak Supervision
• Sparse Conditional Hidden Markov Model for Weakly Supervised Named Entity Recognition
• SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup
• Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach
• Local Boosting for Weakly-Supervised Learning
• Learning with Noisy Labels by Adaptive Gradient-Based Outlier Removal
• Customer Sentiment Analysis using Weak Supervision for Customer-Agent Chat
• BERTifying the Hidden Markov Model for Multi-Source Weakly Supervised Named Entity Recognition
• Learning to Selectively Learn for Weakly-supervised Paraphrase Generation