Can Neural Networks Understand Logical Entailment?
arXiv:1802.08535
Abstract
We introduce a new dataset of logical entailments for the purpose of measuring models' ability to capture and exploit the structure of logical expressions against an entailment prediction task. We use this task to compare a series of architectures which are ubiquitous in the sequence-processing literature, in addition to a new model class---PossibleWorldNets---which computes entailment as a "convolution over possible worlds". Results show that convolutional networks present the wrong inductive bias for this class of problems relative to LSTM RNNs, tree-structured neural networks outperform LSTM RNNs due to their enhanced ability to exploit the syntax of logic, and PossibleWorldNets outperform all benchmarks.
arXiv commentPublished at ICLR 2018 (main conference)
References in corpus (17)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Neural Machine Translation by Jointly Learning to Align and Translate
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- Character-Aware Neural Language Models
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- Reasoning about Entailment with Neural Attention
- The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
- A large annotated corpus for learning natural language inference
- Neural Programmer-Interpreters
- Neural GPUs Learn Algorithms
- Latent Predictor Networks for Code Generation
- Learning Natural Language Inference with LSTM
- ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs
- Learning to Discover Efficient Mathematical Identities
- Compositional Distributional Semantics with Long Short Term Memory
- Better Computer Go Player with Neural Network and Long-term Prediction
Cited by in corpus (5)
- Learning a SAT Solver from Single-Bit Supervision
- Analysing Mathematical Reasoning Abilities of Neural Models
- Learning Heuristics for Quantified Boolean Formulas through Deep Reinforcement Learning
- Automated proof synthesis for propositional logic with deep neural networks
- Teaching Temporal Logics to Neural Networks