Augmenting Neural Networks with First-order Logic
arXiv:1906.06298
Abstract
Today, the dominant paradigm for training neural networks involves minimizing task loss on a large dataset. Using world knowledge to inform a model, and yet retain the ability to perform end-to-end training remains an open question. In this paper, we present a novel framework for introducing declarative knowledge to neural network architectures in order to guide training and prediction. Our framework systematically compiles logical statements into computation graphs that augment a neural network without extra learnable parameters or manual redesign. We evaluate our modeling strategy on three tasks: machine comprehension, natural language inference, and text chunking. Our experiments show that knowledge-augmented networks can strongly improve over baselines, especially in low-data regimes.
arXiv commentAccepted in ACL 2019. Minor fixes in Fig 4; extra citation in related works; Typo fix in constraint N3 and its description
References in corpus (14)
- Distilling the Knowledge in a Neural Network
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- Effective Approaches to Attention-based Neural Machine Translation
- A large annotated corpus for learning natural language inference
- Learning to Generate Reviews and Discovering Sentiment
- Introduction to the CoNLL-2000 Shared Task: Chunking
- Neural Natural Language Inference Models Enhanced with External Knowledge
- Neural-Symbolic Learning and Reasoning: A Survey and Interpretation
- Harnessing Deep Neural Networks with Logic Rules
- SparseMAP: Differentiable Sparse Structured Inference
- Learning to Compose Words into Sentences with Reinforcement Learning
- Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge
- Deep Probabilistic Logic: A Unifying Framework for Indirect Supervision
- Backpropagating through Structured Argmax using a SPIGOT