Neural Data Augmentation via Example Extrapolation

Abstract

In many applications of machine learning, certain categories of examples may be underrepresented in the training data, causing systems to underperform on such "few-shot" cases at test time. A common remedy is to perform data augmentation, such as by duplicating underrepresented examples, or heuristically synthesizing new examples. But these remedies often fail to cover the full diversity and complexity of real examples. We propose a data augmentation approach that performs neural Example Extrapolation (Ex2). Given a handful of exemplars sampled from some distribution, Ex2 synthesizes new examples that also belong to the same distribution. The Ex2 model is learned by simulating the example generation procedure on data-rich slices of the data, and it is applied to underrepresented, few-shot slices. We apply Ex2 to a range of language understanding tasks and significantly improve over state-of-the-art methods on multiple few-shot learning benchmarks, including for relation extraction (FewRel) and intent classification + slot filling (SNIPS).

References in corpus (8)

• Prototypical Networks for Few-shot Learning
• Language Models are Few-Shot Learners
• The Effectiveness of Data Augmentation in Image Classification using Deep Learning
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Matching Networks for One Shot Learning
• Example-Based Named Entity Recognition
• Entity-Switched Datasets: An Approach to Auditing the In-Domain Robustness of Named Entity Recognition Models
• Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference

Cited by in corpus (8)

• A Survey on Data Augmentation for Text Classification
• Towards Zero-Label Language Learning
• Controllable Semantic Parsing via Retrieval Augmentation
• Semi-Supervised Few-Shot Intent Classification and Slot Filling
• Automated Annotation of Scientific Texts for ML-based Keyphrase Extraction and Validation
• ConvFiT: Conversational Fine-Tuning of Pretrained Language Models
• STraTA: Self-Training with Task Augmentation for Better Few-shot Learning
• Good-Enough Example Extrapolation