Reducing and Exploiting Data Augmentation Noise through Meta Reweighting Contrastive Learning for Text Classification

Abstract

Data augmentation has shown its effectiveness in resolving the data-hungry problem and improving model's generalization ability. However, the quality of augmented data can be varied, especially compared with the raw/original data. To boost deep learning models' performance given augmented data/samples in text classification tasks, we propose a novel framework, which leverages both meta learning and contrastive learning techniques as parts of our design for reweighting the augmented samples and refining their feature representations based on their quality. As part of the framework, we propose novel weight-dependent enqueue and dequeue algorithms to utilize augmented samples' weight/quality information effectively. Through experiments, we show that our framework can reasonably cooperate with existing deep learning models (e.g., RoBERTa-base and Text-CNN) and augmentation techniques (e.g., Wordnet and Easydata) for specific supervised learning tasks. Experiment results show that our framework achieves an average of 1.6%, up to 4.3% absolute improvement on Text-CNN encoders and an average of 1.4%, up to 4.4% absolute improvement on RoBERTa-base encoders on seven GLUE benchmark datasets compared with the best baseline. We present an indepth analysis of our framework design, revealing the non-trivial contributions of our network components. Our code is publicly available for better reproducibility.

IEEE BigData 2021

References in corpus (12)

• The Effectiveness of Data Augmentation in Image Classification using Deep Learning
• Unsupervised Data Augmentation for Consistency Training
• Learning to Reweight Examples for Robust Deep Learning
• A Survey on Data Augmentation for Text Classification
• Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting
• TextBugger: Generating Adversarial Text Against Real-world Applications
• Word-level Textual Adversarial Attacking as Combinatorial Optimization
• Data Noising as Smoothing in Neural Network Language Models
• Augmenting Data with Mixup for Sentence Classification: An Empirical Study
• A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation
• Meta-Learning Requires Meta-Augmentation
• Data Augmentation for Meta-Learning