Augmenting Data with Mixup for Sentence Classification: An Empirical Study

Abstract

Mixup, a recent proposed data augmentation method through linearly interpolating inputs and modeling targets of random samples, has demonstrated its capability of significantly improving the predictive accuracy of the state-of-the-art networks for image classification. However, how this technique can be applied to and what is its effectiveness on natural language processing (NLP) tasks have not been investigated. In this paper, we propose two strategies for the adaption of Mixup on sentence classification: one performs interpolation on word embeddings and another on sentence embeddings. We conduct experiments to evaluate our methods using several benchmark datasets. Our studies show that such interpolation strategies serve as an effective, domain independent data augmentation approach for sentence classification, and can result in significant accuracy improvement for both CNN and LSTM models.

7 pages

References in corpus (1)

• Sequence to Sequence Learning with Neural Networks

Cited by in corpus (19)

• FedMix: Approximation of Mixup under Mean Augmented Federated Learning
• MixKD: Towards Efficient Distillation of Large-scale Language Models
• Influence-guided Data Augmentation for Neural Tensor Completion
• Effects of Word-frequency based Pre- and Post- Processings for Audio Captioning
• Neural Semi-supervised Learning for Text Classification Under Large-Scale Pretraining
• MixUp Training Leads to Reduced Overfitting and Improved Calibration for the Transformer Architecture
• LearnDA: Learnable Knowledge-Guided Data Augmentation for Event Causality Identification
• MixSpeech: Data Augmentation for Low-resource Automatic Speech Recognition
• SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup
• Mixup Regularization for Region Proposal based Object Detectors
• Substructure Substitution: Structured Data Augmentation for NLP
• Reweighting Augmented Samples by Minimizing the Maximal Expected Loss
• RockNER: A Simple Method to Create Adversarial Examples for Evaluating the Robustness of Named Entity Recognition Models
• AUGNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation
• Data Augmentation for Text Generation Without Any Augmented Data
• Midpoint Regularization: from High Uncertainty Training to Conservative Classification
• Regularization via Adaptive Pairwise Label Smoothing
• Adversarial Mixing Policy for Relaxing Locally Linear Constraints in Mixup
• Distiller: A Systematic Study of Model Distillation Methods in Natural Language Processing