A Fast Unified Model for Parsing and Sentence Understanding

Abstract

Tree-structured neural networks exploit valuable syntactic parse information as they interpret the meanings of sentences. However, they suffer from two key technical problems that make them slow and unwieldy for large-scale NLP tasks: they usually operate on parsed sentences and they do not directly support batched computation. We address these issues by introducing the Stack-augmented Parser-Interpreter Neural Network (SPINN), which combines parsing and interpretation within a single tree-sequence hybrid model by integrating tree-structured sentence interpretation into the linear sequential structure of a shift-reduce parser. Our model supports batched computation for a speedup of up to 25 times over other tree-structured models, and its integrated parser can operate on unparsed data with little loss in accuracy. We evaluate it on the Stanford NLI entailment task and show that it significantly outperforms other sentence-encoding models.

To appear at ACL 2016

References in corpus (4)

• Sequence to Sequence Learning with Neural Networks
• Transition-Based Dependency Parsing with Stack Long Short-Term Memory
• A Convolutional Neural Network for Modelling Sentences
• Long Short-Term Memory-Networks for Machine Reading

Cited by in corpus (22)

• Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention
• Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
• Neural Models for Information Retrieval
• Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
• Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling
• Shortcut-Stacked Sentence Encoders for Multi-Domain Inference
• Description Based Text Classification with Reinforcement Learning
• AMPNet: Asynchronous Model-Parallel Training for Dynamic Neural Networks
• SG-Net: Syntax-Guided Machine Reading Comprehension
• Neural Language Modeling by Jointly Learning Syntax and Lexicon
• Read + Verify: Machine Reading Comprehension with Unanswerable Questions
• Knowledge Enhanced Attention for Robust Natural Language Inference
• BERE: An accurate distantly supervised biomedical entity relation extraction network
• A Neural Architecture Mimicking Humans End-to-End for Natural Language Inference
• Dynamic-structured Semantic Propagation Network
• Syntax-based Attention Model for Natural Language Inference
• What If We Simply Swap the Two Text Fragments? A Straightforward yet Effective Way to Test the Robustness of Methods to Confounding Signals in Nature Language Inference Tasks
• Modelling Domain Relationships for Transfer Learning on Retrieval-based Question Answering Systems in E-commerce
• Multi-task Sentence Encoding Model for Semantic Retrieval in Question Answering Systems
• Multi-level Head-wise Match and Aggregation in Transformer for Textual Sequence Matching
• FastTrees: Parallel Latent Tree-Induction for Faster Sequence Encoding
• Neural-Network Guided Expression Transformation