Attention Boosted Sequential Inference Model
arXiv:1812.01840
Abstract
Attention mechanism has been proven effective on natural language processing. This paper proposes an attention boosted natural language inference model named aESIM by adding word attention and adaptive direction-oriented attention mechanisms to the traditional Bi-LSTM layer of natural language inference models, e.g. ESIM. This makes the inference model aESIM has the ability to effectively learn the representation of words and model the local subsentential inference between pairs of premise and hypothesis. The empirical studies on the SNLI, MultiNLI and Quora benchmarks manifest that aESIM is superior to the original ESIM model.
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Enhanced LSTM for Natural Language Inference
- A large annotated corpus for learning natural language inference
- Long Short-Term Memory-Networks for Machine Reading
- Bilateral Multi-Perspective Matching for Natural Language Sentences
- Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering
- Distance-based Self-Attention Network for Natural Language Inference
- A Decomposable Attention Model for Natural Language Inference
- Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information
- Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling