Data Augmentation Approaches in Natural Language Processing: A Survey

Abstract

As an effective strategy, data augmentation (DA) alleviates data scarcity scenarios where deep learning techniques may fail. It is widely applied in computer vision then introduced to natural language processing and achieves improvements in many tasks. One of the main focuses of the DA methods is to improve the diversity of training data, thereby helping the model to better generalize to unseen testing data. In this survey, we frame DA methods into three categories based on the diversity of augmented data, including paraphrasing, noising, and sampling. Our paper sets out to analyze DA methods in detail according to the above categories. Further, we also introduce their applications in NLP tasks as well as the challenges. Some helpful resources are provided in the appendix.

arXiv commentaccepted by AI Open (2022) at https://www.sciencedirect.com/science/article/pii/S2666651022000080 ; 55 pages, 12 figures, 3 tables

References in corpus (34)

• mixup: Beyond Empirical Risk Minimization
• Generative Adversarial Networks
• Character-level Convolutional Networks for Text Classification
• Unsupervised Data Augmentation for Consistency Training
• A Survey on Data Augmentation for Text Classification
• Improving language models by retrieving from trillions of tokens
• Regularization for Deep Learning: A Taxonomy
• Data Noising as Smoothing in Neural Network Language Models
• Improved Relation Classification by Deep Recurrent Neural Networks with Data Augmentation
• Augmenting Data with Mixup for Sentence Classification: An Empirical Study
• Data Augmentation using Pre-trained Transformer Models
• Learning Data Manipulation for Augmentation and Weighting
• Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generation
• TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP
• Soft Contextual Data Augmentation for Neural Machine Translation
• A Multi-cascaded Model with Data Augmentation for Enhanced Paraphrase Detection in Short Texts
• Text Data Augmentation Made Simple By Leveraging NLP Cloud APIs
• Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering
• Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning
• On Data Augmentation for Extreme Multi-label Classification
• Dictionary-based Data Augmentation for Cross-Domain Neural Machine Translation
• Fast Cross-domain Data Augmentation through Neural Sentence Editing
• Simple is Better! Lightweight Data Augmentation for Low Resource Slot Filling and Intent Classification
• Atalaya at TASS 2019: Data Augmentation and Robust Embeddings for Sentiment Analysis
• Denoising Pre-Training and Data Augmentation Strategies for Enhanced RDF Verbalization with Transformers
• Reverse Operation based Data Augmentation for Solving Math Word Problems
• Multimodal Dialogue State Tracking By QA Approach with Data Augmentation
• Text Data Augmentation: Towards better detection of spear-phishing emails
• Can We Achieve More with Less? Exploring Data Augmentation for Toxic Comment Classification
• Multilingual Transfer Learning for QA Using Translation as Data Augmentation
• Deep Transformer based Data Augmentation with Subword Units for Morphologically Rich Online ASR
• Pattern-aware Data Augmentation for Query Rewriting in Voice Assistant Systems
• On the Effects of Knowledge-Augmented Data in Word Embeddings
• Improving Commonsense Causal Reasoning by Adversarial Training and Data Augmentation

Cited by in corpus (6)

• Survey of Hallucination in Natural Language Generation
• Channel Reflection: Knowledge-Driven Data Augmentation for EEG-Based Brain-Computer Interfaces
• Explainable AI: XAI-Guided Context-Aware Data Augmentation
• Face4RAG: Factual Consistency Evaluation for Retrieval Augmented Generation in Chinese
• AMPLIFY:Attention-based Mixup for Performance Improvement and Label Smoothing in Transformer
• Cross-Domain Data Selection and Augmentation for Automatic Compliance Detection