Towards Neural Network-based Reasoning

Abstract

We propose Neural Reasoner, a framework for neural network-based reasoning over natural language sentences. Given a question, Neural Reasoner can infer over multiple supporting facts and find an answer to the question in specific forms. Neural Reasoner has 1) a specific interaction-pooling mechanism, allowing it to examine multiple facts, and 2) a deep architecture, allowing it to model the complicated logical relations in reasoning tasks. Assuming no particular structure exists in the question and facts, Neural Reasoner is able to accommodate different types of reasoning and different forms of language expressions. Despite the model complexity, Neural Reasoner can still be trained effectively in an end-to-end manner. Our empirical studies show that Neural Reasoner can outperform existing neural reasoning systems with remarkable margins on two difficult artificial tasks (Positional Reasoning and Path Finding) proposed in [8]. For example, it improves the accuracy on Path Finding(10K) from 33.4% [6] to over 98%.

References in corpus (5)

• Sequence to Sequence Learning with Neural Networks
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• ADADELTA: An Adaptive Learning Rate Method
• Convolutional Neural Network Architectures for Matching Natural Language Sentences
• Ask Me Anything: Dynamic Memory Networks for Natural Language Processing

Cited by in corpus (21)

• End-To-End Memory Networks
• Learning Multiagent Communication with Backpropagation
• Adding Gradient Noise Improves Learning for Very Deep Networks
• End-to-End Differentiable Proving
• Query-Reduction Networks for Question Answering
• Neural Programmer: Inducing Latent Programs with Gradient Descent
• Visual Question Answering: A Survey of Methods and Datasets
• Joint Attention in Driver-Pedestrian Interaction: from Theory to Practice
• CFO: Conditional Focused Neural Question Answering with Large-scale Knowledge Bases
• A Parallel-Hierarchical Model for Machine Comprehension on Sparse Data
• A Deep Memory-based Architecture for Sequence-to-Sequence Learning
• Attention-Based Convolutional Neural Network for Machine Comprehension
• Reasoning over RDF Knowledge Bases using Deep Learning
• Neural Entity Reasoner for Global Consistency in NER
• Exploring Different Dimensions of Attention for Uncertainty Detection
• NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language
• Evolutionary Algorithm for Sinhala to English Translation
• Skip-Thought Memory Networks
• Exploiting Sentence Embedding for Medical Question Answering
• Multi-Relational Question Answering from Narratives: Machine Reading and Reasoning in Simulated Worlds
• Improved Dynamic Memory Network for Dialogue Act Classification with Adversarial Training