An Attention Mechanism for Answer Selection Using a Combined Global and Local View
arXiv:1707.01378
Abstract
We propose a new attention mechanism for neural based question answering, which depends on varying granularities of the input. Previous work focused on augmenting recurrent neural networks with simple attention mechanisms which are a function of the similarity between a question embedding and an answer embeddings across time. We extend this by making the attention mechanism dependent on a global embedding of the answer attained using a separate network. We evaluate our system on InsuranceQA, a large question answering dataset. Our model outperforms current state-of-the-art results on InsuranceQA. Further, we visualize which sections of text our attention mechanism focuses on, and explore its performance across different parameter settings.
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- Recurrent Neural Network Regularization
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- Reasoning about Entailment with Neural Attention
- Learning Longer Memory in Recurrent Neural Networks
- Applying Deep Learning to Answer Selection: A Study and An Open Task
- Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models