Gaussian Attention Model and Its Application to Knowledge Base Embedding and Question Answering

Abstract

We propose the Gaussian attention model for content-based neural memory access. With the proposed attention model, a neural network has the additional degree of freedom to control the focus of its attention from a laser sharp attention to a broad attention. It is applicable whenever we can assume that the distance in the latent space reflects some notion of semantics. We use the proposed attention model as a scoring function for the embedding of a knowledge base into a continuous vector space and then train a model that performs question answering about the entities in the knowledge base. The proposed attention model can handle both the propagation of uncertainty when following a series of relations and also the conjunction of conditions in a natural way. On a dataset of soccer players who participated in the FIFA World Cup 2014, we demonstrate that our model can handle both path queries and conjunctive queries well.

16 pages, 4 figures

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Sequence Transduction with Recurrent Neural Networks
• Large-scale Simple Question Answering with Memory Networks
• Gated Graph Sequence Neural Networks
• Word Representations via Gaussian Embedding
• Traversing Knowledge Graphs in Vector Space
• Compositional Vector Space Models for Knowledge Base Completion
• Open Question Answering with Weakly Supervised Embedding Models

Cited by in corpus (3)

• An Interpretable Reasoning Network for Multi-Relation Question Answering
• Question-Aware Memory Network for Multi-hop Question Answering in Human-Robot Interaction
• Learning to Order Sub-questions for Complex Question Answering