Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics
arXiv:2110.01518
Abstract
Much of recent progress in NLU was shown to be due to models' learning dataset-specific heuristics. We conduct a case study of generalization in NLI (from MNLI to the adversarially constructed HANS dataset) in a range of BERT-based architectures (adapters, Siamese Transformers, HEX debiasing), as well as with subsampling the data and increasing the model size. We report 2 successful and 3 unsuccessful strategies, all providing insights into how Transformer-based models learn to generalize.
arXiv commentWorkshop on Insights from Negative Results (EMNLP 2021)
References in corpus (6)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- The information bottleneck method
- Enhanced LSTM for Natural Language Inference
- Well-Read Students Learn Better: On the Importance of Pre-training Compact Models
- Deep Variational Information Bottleneck
- Learning Robust Representations by Projecting Superficial Statistics Out