Does Data Augmentation Improve Generalization in NLP?
arXiv:2004.15012
Abstract
Neural models often exploit superficial features to achieve good performance, rather than deriving more general features. Overcoming this tendency is a central challenge in areas such as representation learning and ML fairness. Recent work has proposed using data augmentation, i.e., generating training examples where the superficial features fail, as a means of encouraging models to prefer the stronger features. We design a series of toy learning problems to test the hypothesis that data augmentation leads models to unlearn weaker heuristics, but not to learn stronger features in their place. We find partial support for this hypothesis: Data augmentation often hurts before it helps, and it is less effective when the preferred strong feature is much more difficult to extract than the competing weak feature.
References in corpus (9)
- Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
- Assessing BERT's Syntactic Abilities
- Frame-Semantic Parsing with Softmax-Margin Segmental RNNs and a Syntactic Scaffold
- Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
- Gender Bias in Neural Natural Language Processing
- Correlating neural and symbolic representations of language
- Gender Bias in Contextualized Word Embeddings
- Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process
- Certified Robustness to Adversarial Word Substitutions