Reasoning about Entailment with Neural Attention
arXiv:1509.06664
Abstract
While most approaches to automatically recognizing entailment relations have used classifiers employing hand engineered features derived from complex natural language processing pipelines, in practice their performance has been only slightly better than bag-of-word pair classifiers using only lexical similarity. The only attempt so far to build an end-to-end differentiable neural network for entailment failed to outperform such a simple similarity classifier. In this paper, we propose a neural model that reads two sentences to determine entailment using long short-term memory units. We extend this model with a word-by-word neural attention mechanism that encourages reasoning over entailments of pairs of words and phrases. Furthermore, we present a qualitative analysis of attention weights produced by this model, demonstrating such reasoning capabilities. On a large entailment dataset this model outperforms the previous best neural model and a classifier with engineered features by a substantial margin. It is the first generic end-to-end differentiable system that achieves state-of-the-art accuracy on a textual entailment dataset.
ICLR 2016 camera-ready, 9 pages, 10 figures (incl. subfigures)
References in corpus (5)
Cited by in corpus (126)
- Enhanced LSTM for Natural Language Inference
- Attention in Natural Language Processing
- GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
- From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification
- Attentive Pooling Networks
- Neural Natural Language Inference Models Enhanced with External Knowledge
- Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey
- Multimodal Residual Learning for Visual QA
- Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention
- Natural Language Inference over Interaction Space
- Long Short-Term Memory-Networks for Machine Reading
- Visual Entailment: A Novel Task for Fine-Grained Image Understanding
- On the Explainability of Natural Language Processing Deep Models
- Bilateral Multi-Perspective Matching for Natural Language Sentences
- Explicit Knowledge-based Reasoning for Visual Question Answering
- Order-Embeddings of Images and Language
- Learning Python Code Suggestion with a Sparse Pointer Network
- Learning Natural Language Inference with LSTM
- Dynamic Integration of Background Knowledge in Neural NLU Systems
- Enhancing Sentence Embedding with Generalized Pooling
- ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs
- Hyperbolic Entailment Cones for Learning Hierarchical Embeddings
- Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge
- Neural ranking models for document retrieval
- Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints
- Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes
- TARN: Temporal Attentive Relation Network for Few-Shot and Zero-Shot Action Recognition
- A Fast Unified Model for Parsing and Sentence Understanding
- Granger-causal Attentive Mixtures of Experts: Learning Important Features with Neural Networks
- Machine Reading Comprehension: a Literature Review
- Visualizing Attention in Transformer-Based Language Representation Models
- Learning to Attend via Word-Aspect Associative Fusion for Aspect-based Sentiment Analysis
- Sentence Pair Scoring: Towards Unified Framework for Text Comprehension
- Stress-Testing Neural Models of Natural Language Inference with Multiply-Quantified Sentences
- Multi-Cast Attention Networks for Retrieval-based Question Answering and Response Prediction
- Structure-Tags Improve Text Classification for Scholarly Document Quality Prediction
- Explaining Deep Neural Networks
- Textual Entailment with Structured Attentions and Composition
- DeepTransport: Learning Spatial-Temporal Dependency for Traffic Condition Forecasting
- Searching for Effective Neural Extractive Summarization: What Works and What's Next
- BERT Learns (and Teaches) Chemistry
- Call Attention to Rumors: Deep Attention Based Recurrent Neural Networks for Early Rumor Detection
- Reasoning with Sarcasm by Reading In-between
- Improving Natural Language Inference Using External Knowledge in the Science Questions Domain
- Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization
- Learning Visual Knowledge Memory Networks for Visual Question Answering
- Improving Multimodal Accuracy Through Modality Pre-training and Attention
- Automatic Judgment Prediction via Legal Reading Comprehension
- Generating Natural Language Inference Chains
- An Attentive Neural Architecture for Fine-grained Entity Type Classification
- Neural Networks for Modeling Source Code Edits
- Dual Attention Networks for Visual Reference Resolution in Visual Dialog
- Modelling Interaction of Sentence Pair with coupled-LSTMs
- Sequential Interpretability: Methods, Applications, and Future Direction for Understanding Deep Learning Models in the Context of Sequential Data
- Improving the Performance of Neural Machine Translation Involving Morphologically Rich Languages
- An Attention-Based Word-Level Interaction Model: Relation Detection for Knowledge Base Question Answering
- A Hierarchical Contextual Attention-based GRU Network for Sequential Recommendation
- Neural Architectures for Fine-grained Entity Type Classification
- Knowledge Enhanced Attention for Robust Natural Language Inference
- Multi-turn Inference Matching Network for Natural Language Inference
- Recurrent Neural Network Encoder with Attention for Community Question Answering
- Can Neural Networks Understand Logical Entailment?
- "Factual" or "Emotional": Stylized Image Captioning with Adaptive Learning and Attention
- Natural Language Inference from Multiple Premises
- Natural Language Generation with Neural Variational Models
- A Neural Architecture Mimicking Humans End-to-End for Natural Language Inference
- Breaking NLI Systems with Sentences that Require Simple Lexical Inferences
- ProtoryNet - Interpretable Text Classification Via Prototype Trajectories
- On the Effective Use of Pretraining for Natural Language Inference
- Are the Multilingual Models Better? Improving Czech Sentiment with Transformers
- Comparative Evaluation of Pretrained Transfer Learning Models on Automatic Short Answer Grading
- An Attention Mechanism for Answer Selection Using a Combined Global and Local View
- A Deep Neural Network for Chinese Zero Pronoun Resolution
- A Sequential Matching Framework for Multi-turn Response Selection in Retrieval-based Chatbots
- Set-to-Sequence Methods in Machine Learning: a Review
- A Study of the Plausibility of Attention between RNN Encoders in Natural Language Inference
- Satirical News Detection and Analysis using Attention Mechanism and Linguistic Features
- Detecting and Explaining Causes From Text For a Time Series Event
- AlignVE: Visual Entailment Recognition Based on Alignment Relations
- Threshold-Based Retrieval and Textual Entailment Detection on Legal Bar Exam Questions
- Path-Based Contextualization of Knowledge Graphs for Textual Entailment
- Story Cloze Ending Selection Baselines and Data Examination
- Syntax-based Attention Model for Natural Language Inference
- RAP-Net: Recurrent Attention Pooling Networks for Dialogue Response Selection
- Posing Fair Generalization Tasks for Natural Language Inference
- What If We Simply Swap the Two Text Fragments? A Straightforward yet Effective Way to Test the Robustness of Methods to Confounding Signals in Nature Language Inference Tasks
- Enhanced Word Representations for Bridging Anaphora Resolution
- Towards Neural Speaker Modeling in Multi-Party Conversation: The Task, Dataset, and Models
- MuFuRU: The Multi-Function Recurrent Unit
- Why and How to Pay Different Attention to Phrase Alignments of Different Intensities
- ASBERT: Siamese and Triplet network embedding for open question answering
- Detecting and Explaining Crisis
- Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks
- An Explainable CNN Approach for Medical Codes Prediction from Clinical Text
- Learning Word Sense Embeddings from Word Sense Definitions
- Combining Axiom Injection and Knowledge Base Completion for Efficient Natural Language Inference
- A Multi-Size Neural Network with Attention Mechanism for Answer Selection
- Automatic learner summary assessment for reading comprehension
- Simplifying Neural Machine Translation with Addition-Subtraction Twin-Gated Recurrent Networks
- Neural Article Pair Modeling for Wikipedia Sub-article Matching
- REGMAPR - Text Matching Made Easy
- AWE: Asymmetric Word Embedding for Textual Entailment
- ReDecode Framework for Iterative Improvement in Paraphrase Generation
- Understanding Roles and Entities: Datasets and Models for Natural Language Inference
- A Deterministic Algorithm for Bridging Anaphora Resolution
- Multi Resolution LSTM For Long Term Prediction In Neural Activity Video
- Toward Interpretability of Dual-Encoder Models for Dialogue Response Suggestions
- Constructing Explainable Opinion Graphs from Review
- LSTM Networks for Online Cross-Network Recommendations
- Sequence-to-Set Semantic Tagging: End-to-End Multi-label Prediction using Neural Attention for Complex Query Reformulation and Automated Text Categorization
- An Efficient Approach for Geo-Multimedia Cross-Modal Retrieval
- Contradiction Detection in Persian Text
- Retrieving and ranking short medical questions with two stages neural matching model
- Visuallly Grounded Generation of Entailments from Premises
- Understanding Deep Learning Performance through an Examination of Test Set Difficulty: A Psychometric Case Study
- Hierarchical RNN with Static Sentence-Level Attention for Text-Based Speaker Change Detection
- Deep Multimodal Learning: An Effective Method for Video Classification
- Logic, Language, and Calculus
- k-Nearest Neighbors by Means of Sequence to Sequence Deep Neural Networks and Memory Networks
- Multi-Perspective Inferrer: Reasoning Sentences Relationship from Holistic Perspective
- Two-Headed Monster And Crossed Co-Attention Networks
- Learning to Represent Bilingual Dictionaries
- Multi-level Head-wise Match and Aggregation in Transformer for Textual Sequence Matching
- Matching Questions and Answers in Dialogues from Online Forums
- Scaffolding Networks: Incremental Learning and Teaching Through Questioning
- Using Statistical and Semantic Models for Multi-Document Summarization