Augmenting Data with Mixup for Sentence Classification: An Empirical Study

Abstract

Mixup, a recent proposed data augmentation method through linearly interpolating inputs and modeling targets of random samples, has demonstrated its capability of significantly improving the predictive accuracy of the state-of-the-art networks for image classification. However, how this technique can be applied to and what is its effectiveness on natural language processing (NLP) tasks have not been investigated. In this paper, we propose two strategies for the adaption of Mixup on sentence classification: one performs interpolation on word embeddings and another on sentence embeddings. We conduct experiments to evaluate our methods using several benchmark datasets. Our studies show that such interpolation strategies serve as an effective, domain independent data augmentation approach for sentence classification, and can result in significant accuracy improvement for both CNN and LSTM models.

7 pages

References in corpus (2)

• Sequence to Sequence Learning with Neural Networks
• mixup: Beyond Empirical Risk Minimization

Cited by in corpus (34)

• A Survey on Data Augmentation for Text Classification
• Data Augmentation Approaches in Natural Language Processing: A Survey
• A Comprehensive Overview and Comparative Analysis on Deep Learning Models: CNN, RNN, LSTM, GRU
• FedMix: Approximation of Mixup under Mean Augmented Federated Learning
• Do We Need Zero Training Loss After Achieving Zero Training Error?
• i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning
• MixKD: Towards Efficient Distillation of Large-scale Language Models
• CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding
• Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning
• Influence-guided Data Augmentation for Neural Tensor Completion
• Effects of Word-frequency based Pre- and Post- Processings for Audio Captioning
• Neural Semi-supervised Learning for Text Classification Under Large-Scale Pretraining
• Towards Domain-Agnostic Contrastive Learning
• Combining Ensembles and Data Augmentation can Harm your Calibration
• Overlapping Word Removal is All You Need: Revisiting Data Imbalance in Hope Speech Detection
• LearnDA: Learnable Knowledge-Guided Data Augmentation for Event Causality Identification
• MixUp Training Leads to Reduced Overfitting and Improved Calibration for the Transformer Architecture
• MixSpeech: Data Augmentation for Low-resource Automatic Speech Recognition
• SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup
• Mixup Regularization for Region Proposal based Object Detectors
• Reducing and Exploiting Data Augmentation Noise through Meta Reweighting Contrastive Learning for Text Classification
• Substructure Substitution: Structured Data Augmentation for NLP
• AUGNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation
• MixDec Sampling: A Soft Link-based Sampling Method of Graph Neural Network for Recommendation
• RockNER: A Simple Method to Create Adversarial Examples for Evaluating the Robustness of Named Entity Recognition Models
• Reweighting Augmented Samples by Minimizing the Maximal Expected Loss
• Data Augmentation for Text Generation Without Any Augmented Data
• AMPLIFY:Attention-based Mixup for Performance Improvement and Label Smoothing in Transformer
• Distiller: A Systematic Study of Model Distillation Methods in Natural Language Processing
• Regularization via Adaptive Pairwise Label Smoothing
• Constructing Contrastive samples via Summarization for Text Classification with limited annotations
• Out-of-Manifold Regularization in Contextual Embedding Space for Text Classification
• Adversarial Mixing Policy for Relaxing Locally Linear Constraints in Mixup
• Midpoint Regularization: from High Uncertainty Training to Conservative Classification