An Empirical Survey of Data Augmentation for Limited Data Learning in NLP

Abstract

NLP has achieved great progress in the past decade through the use of neural models and large labeled datasets. The dependence on abundant data prevents NLP models from being applied to low-resource settings or novel tasks where significant time, money, or expertise is required to label massive amounts of textual data. Recently, data augmentation methods have been explored as a means of improving data efficiency in NLP. To date, there has been no systematic empirical overview of data augmentation for NLP in the limited labeled data setting, making it difficult to understand which methods work in which settings. In this paper, we provide an empirical survey of recent progress on data augmentation for NLP in the limited labeled data setting, summarizing the landscape of methods (including token-level augmentations, sentence-level augmentations, adversarial augmentations, and hidden-space augmentations) and carrying out experiments on 11 datasets covering topics/news classification, inference tasks, paraphrasing tasks, and single-sentence tasks. Based on the results, we draw several conclusions to help practitioners choose appropriate augmentations in different settings and discuss the current challenges and future directions for limited data learning in NLP.

References in corpus (19)

• mixup: Beyond Empirical Risk Minimization
• Improved Regularization of Convolutional Neural Networks with Cutout
• Generating Natural Adversarial Examples
• QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
• FreeLB: Enhanced Adversarial Training for Natural Language Understanding
• On Adversarial Examples for Character-Level Neural Machine Translation
• A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation
• Improved Variational Autoencoders for Text Modeling using Dilated Convolutions
• It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations
• Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures
• Robust Neural Machine Translation with Doubly Adversarial Inputs
• Learning Compositional Rules via Neural Program Synthesis
• Robust Neural Abstractive Summarization Systems and Evaluation against Adversarial Information
• When Low Resource NLP Meets Unsupervised Language Model: Meta-pretraining Then Meta-learning for Few-shot Text Classification
• Semi-Supervised Models via Data Augmentationfor Classifying Interactive Affective Responses
• Compositional Generalization via Neural-Symbolic Stack Machines
• Controllable Paraphrase Generation with a Syntactic Exemplar
• HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalization
• Sequence-Level Mixed Sample Data Augmentation

Cited by in corpus (4)

• Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
• DaCy: A Unified Framework for Danish NLP
• Identifying Introductions in Podcast Episodes from Automatically Generated Transcripts
• Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification