Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention

Abstract

In this paper, we proposed a sentence encoding-based model for recognizing text entailment. In our approach, the encoding of sentence is a two-stage process. Firstly, average pooling was used over word-level bidirectional LSTM (biLSTM) to generate a first-stage sentence representation. Secondly, attention mechanism was employed to replace average pooling on the same sentence for better representations. Instead of using target sentence to attend words in source sentence, we utilized the sentence's first-stage representation to attend words appeared in itself, which is called "Inner-Attention" in our paper . Experiments conducted on Stanford Natural Language Inference (SNLI) Corpus has proved the effectiveness of "Inner-Attention" mechanism. With less number of parameters, our model outperformed the existing best sentence encoding-based approach by a large margin.

References in corpus (7)

• Reasoning about Entailment with Neural Attention
• A large annotated corpus for learning natural language inference
• LSTM-based Deep Learning Models for Non-factoid Answer Selection
• Learning to Execute
• Order-Embeddings of Images and Language
• Natural Language Inference by Tree-Based Convolution and Heuristic Matching
• A Fast Unified Model for Parsing and Sentence Understanding

Cited by in corpus (57)

• Enhanced LSTM for Natural Language Inference
• Deep Learning Based Text Classification: A Comprehensive Review
• CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
• Neural Natural Language Inference Models Enhanced with External Knowledge
• Natural Language Inference over Interaction Space
• Bilateral Multi-Perspective Matching for Natural Language Sentences
• DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
• A Survey on Text Classification: From Shallow to Deep Learning
• Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling
• Distance-based Self-Attention Network for Natural Language Inference
• Enhancing Sentence Embedding with Generalized Pooling
• A Universal Representation Transformer Layer for Few-Shot Image Classification
• Dynamic Self-Attention : Computing Attention over Words Dynamically for Sentence Embedding
• Temporal Self-Attention Network for Medical Concept Embedding
• Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling
• Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference
• Extreme Multi-Label Legal Text Classification: A case study in EU Legislation
• Conditional Self-Attention for Query-based Summarization
• Generate, Delete and Rewrite: A Three-Stage Framework for Improving Persona Consistency of Dialogue Generation
• Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learning
• Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Tasks
• Multi-turn Inference Matching Network for Natural Language Inference
• Multimodal deep learning for short-term stock volatility prediction
• Leveraging Intra-User and Inter-User Representation Learning for Automated Hate Speech Detection
• Self-Attention Enhanced Selective Gate with Entity-Aware Embedding for Distantly Supervised Relation Extraction
• LadRa-Net: Locally-Aware Dynamic Re-read Attention Net for Sentence Semantic Matching
• Multi-task Learning for Universal Sentence Embeddings: A Thorough Evaluation using Transfer and Auxiliary Tasks
• Lifelong Learning of Hate Speech Classification on Social Media
• Ensemble Neural Relation Extraction with Adaptive Boosting
• Self-Attention Enhanced Patient Journey Understanding in Healthcare System
• Tensorized Self-Attention: Efficiently Modeling Pairwise and Global Dependencies Together
• Cell-aware Stacked LSTMs for Modeling Sentences
• AlignVE: Visual Entailment Recognition Based on Alignment Relations
• Skin disease diagnosis with deep learning: a review
• Path-Based Contextualization of Knowledge Graphs for Textual Entailment
• Threshold-Based Retrieval and Textual Entailment Detection on Legal Bar Exam Questions
• Learning Robust, Transferable Sentence Representations for Text Classification
• Context Attentive Document Ranking and Query Suggestion
• MHSAN: Multi-Head Self-Attention Network for Visual Semantic Embedding
• Improving Long-Tail Relation Extraction with Collaborating Relation-Augmented Attention
• Development of deep learning algorithms to categorize free-text notes pertaining to diabetes: convolution neural networks achieve higher accuracy than support vector machines
• What If We Simply Swap the Two Text Fragments? A Straightforward yet Effective Way to Test the Robustness of Methods to Confounding Signals in Nature Language Inference Tasks
• Refining Raw Sentence Representations for Textual Entailment Recognition via Attention
• Simplifying Neural Machine Translation with Addition-Subtraction Twin-Gated Recurrent Networks
• Neural Machine Translation: A Review and Survey
• Representation Learning for Words and Entities
• On Tree-Based Neural Sentence Modeling
• Understanding Roles and Entities: Datasets and Models for Natural Language Inference
• Pentagon at MEDIQA 2019: Multi-task Learning for Filtering and Re-ranking Answers using Language Inference and Question Entailment
• Tweets Can Tell: Activity Recognition using Hybrid Long Short-Term Memory Model
• Question-Aware Sentence Gating Networks for Question and Answering
• Eliminating Sentiment Bias for Aspect-Level Sentiment Classification with Unsupervised Opinion Extraction
• Incorporating Relevant Knowledge in Context Modeling and Response Generation
• BiteNet: Bidirectional Temporal Encoder Network to Predict Medical Outcomes
• SNU_IDS at SemEval-2018 Task 12: Sentence Encoder with Contextualized Vectors for Argument Reasoning Comprehension
• DGA-Net Dynamic Gaussian Attention Network for Sentence Semantic Matching
• Text Information Aggregation with Centrality Attention