Shortcut-Stacked Sentence Encoders for Multi-Domain Inference

Abstract

We present a simple sequential sentence encoder for multi-domain natural language inference. Our encoder is based on stacked bidirectional LSTM-RNNs with shortcut connections and fine-tuning of word embeddings. The overall supervised model uses the above encoder to encode two input sentences into two vectors, and then uses a classifier over the vector combination to label the relationship between these two sentences as that of entailment, contradiction, or neural. Our Shortcut-Stacked sentence encoders achieve strong improvements over existing encoders on matched and mismatched multi-domain natural language inference (top non-ensemble single-model result in the EMNLP RepEval 2017 Shared Task (Nangia et al., 2017)). Moreover, they achieve the new state-of-the-art encoding result on the original SNLI dataset (Bowman et al., 2015).

arXiv commentEMNLP 2017 RepEval Multi-NLI Shared Task (6 pages)

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Enhanced LSTM for Natural Language Inference
• A large annotated corpus for learning natural language inference
• Bilateral Multi-Perspective Matching for Natural Language Sentences
• A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks
• Natural Language Inference by Tree-Based Convolution and Heuristic Matching
• A Fast Unified Model for Parsing and Sentence Understanding

Cited by in corpus (17)

• Natural Language Inference over Interaction Space
• Visual Entailment: A Novel Task for Fine-Grained Image Understanding
• Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering
• Distance-based Self-Attention Network for Natural Language Inference
• Dynamic Self-Attention : Computing Attention over Words Dynamically for Sentence Embedding
• Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information
• Combining Fact Extraction and Verification with Neural Semantic Matching Networks
• Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms
• Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling
• Breaking NLI Systems with Sentences that Require Simple Lexical Inferences
• TextNAS: A Neural Architecture Search Space tailored for Text Representation
• What If We Simply Swap the Two Text Fragments? A Straightforward yet Effective Way to Test the Robustness of Methods to Confounding Signals in Nature Language Inference Tasks
• Combining Axiom Injection and Knowledge Base Completion for Efficient Natural Language Inference
• REGMAPR - Text Matching Made Easy
• Dynamic Compositionality in Recursive Neural Networks with Structure-aware Tag Representations
• Contextualized Embeddings based Convolutional Neural Networks for Duplicate Question Identification
• Teaching Syntax by Adversarial Distraction