Syntax-based Attention Model for Natural Language Inference
arXiv:1607.06556
Abstract
Introducing attentional mechanism in neural network is a powerful concept, and has achieved impressive results in many natural language processing tasks. However, most of the existing models impose attentional distribution on a flat topology, namely the entire input representation sequence. Clearly, any well-formed sentence has its accompanying syntactic tree structure, which is a much rich topology. Applying attention to such topology not only exploits the underlying syntax, but also makes attention more interpretable. In this paper, we explore this direction in the context of natural language inference. The results demonstrate its efficacy. We also perform extensive qualitative analysis, deriving insights and intuitions of why and how our model works.
Submitted to EMNLP 2016
References in corpus (9)
- Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
- Effective Approaches to Attention-based Neural Machine Translation
- Reasoning about Entailment with Neural Attention
- A large annotated corpus for learning natural language inference
- Long Short-Term Memory-Networks for Machine Reading
- Towards Universal Paraphrastic Sentence Embeddings
- Learning Natural Language Inference with LSTM
- Natural Language Inference by Tree-Based Convolution and Heuristic Matching
- A Fast Unified Model for Parsing and Sentence Understanding