Data Augmentation Approaches in Natural Language Processing: A Survey

Abstract

As an effective strategy, data augmentation (DA) alleviates data scarcity scenarios where deep learning techniques may fail. It is widely applied in computer vision then introduced to natural language processing and achieves improvements in many tasks. One of the main focuses of the DA methods is to improve the diversity of training data, thereby helping the model to better generalize to unseen testing data. In this survey, we frame DA methods into three categories based on the diversity of augmented data, including paraphrasing, noising, and sampling. Our paper sets out to analyze DA methods in detail according to the above categories. Further, we also introduce their applications in NLP tasks as well as the challenges. Some helpful resources are provided in the appendix.

accepted by AI Open (2022) at https://www.sciencedirect.com/science/article/pii/S2666651022000080 ; 55 pages, 12 figures, 3 tables

References in corpus (25)

• Generative Adversarial Networks
• A Survey on Data Augmentation for Text Classification
• Improving language models by retrieving from trillions of tokens
• Regularization for Deep Learning: A Taxonomy
• Data Noising as Smoothing in Neural Network Language Models
• Augmenting Data with Mixup for Sentence Classification: An Empirical Study
• Learning Data Manipulation for Augmentation and Weighting
• Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generation
• TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP
• Soft Contextual Data Augmentation for Neural Machine Translation
• A Multi-cascaded Model with Data Augmentation for Enhanced Paraphrase Detection in Short Texts
• Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering
• Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning
• On Data Augmentation for Extreme Multi-label Classification
• Dictionary-based Data Augmentation for Cross-Domain Neural Machine Translation
• Simple is Better! Lightweight Data Augmentation for Low Resource Slot Filling and Intent Classification
• Atalaya at TASS 2019: Data Augmentation and Robust Embeddings for Sentiment Analysis
• Reverse Operation based Data Augmentation for Solving Math Word Problems
• Denoising Pre-Training and Data Augmentation Strategies for Enhanced RDF Verbalization with Transformers
• Multimodal Dialogue State Tracking By QA Approach with Data Augmentation
• Can We Achieve More with Less? Exploring Data Augmentation for Toxic Comment Classification
• Improving Commonsense Causal Reasoning by Adversarial Training and Data Augmentation
• Pattern-aware Data Augmentation for Query Rewriting in Voice Assistant Systems
• Multilingual Transfer Learning for QA Using Translation as Data Augmentation
• On the Effects of Knowledge-Augmented Data in Word Embeddings

Cited by in corpus (6)

• Survey of Hallucination in Natural Language Generation
• Channel Reflection: Knowledge-Driven Data Augmentation for EEG-Based Brain-Computer Interfaces
• Explainable AI: XAI-Guided Context-Aware Data Augmentation
• Face4RAG: Factual Consistency Evaluation for Retrieval Augmented Generation in Chinese
• AMPLIFY:Attention-based Mixup for Performance Improvement and Label Smoothing in Transformer
• Cross-Domain Data Selection and Augmentation for Automatic Compliance Detection