Improving Generalization by Incorporating Coverage in Natural Language Inference

Abstract

The task of natural language inference (NLI) is to identify the relation between the given premise and hypothesis. While recent NLI models achieve very high performance on individual datasets, they fail to generalize across similar datasets. This indicates that they are solving NLI datasets instead of the task itself. In order to improve generalization, we propose to extend the input representations with an abstract view of the relation between the hypothesis and the premise, i.e., how well the individual words, or word n-grams, of the hypothesis are covered by the premise. Our experiments show that the use of this information considerably improves generalization across different NLI datasets without requiring any external knowledge or additional data. Finally, we show that using the coverage information is not only beneficial for improving the performance across different datasets of the same task. The resulting generalization improves the performance across datasets that belong to similar but not the same tasks.

References in corpus (8)

• Bidirectional Attention Flow for Machine Comprehension
• Enhanced LSTM for Natural Language Inference
• The Natural Language Decathlon: Multitask Learning as Question Answering
• Neural Natural Language Inference Models Enhanced with External Knowledge
• Multi-Task Deep Neural Networks for Natural Language Understanding
• Learning and Evaluating General Linguistic Intelligence
• Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
• Non-entailed subsequences as a challenge for natural language inference

Cited by in corpus (3)

• Learning to Reason for Text Generation from Scientific Tables
• Improving Robustness by Augmenting Training Sentences with Predicate-Argument Structures
• The Curse of Performance Instability in Analysis Datasets: Consequences, Source, and Suggestions