Exploring End-to-End Differentiable Natural Logic Modeling

Abstract

We explore end-to-end trained differentiable models that integrate natural logic with neural networks, aiming to keep the backbone of natural language reasoning based on the natural logic formalism while introducing subsymbolic vector representations and neural components. The proposed model adapts module networks to model natural logic operations, which is enhanced with a memory component to model contextual information. Experiments show that the proposed framework can effectively model monotonicity-based reasoning, compared to the baseline neural network models without built-in inductive bias for monotonicity-based reasoning. Our proposed model shows to be robust when transferred from upward to downward inference. We perform further analyses on the performance of the proposed model on aggregation, showing the effectiveness of the proposed subcomponents on helping achieve better intermediate aggregation performance.

arXiv comment10 pages

References in corpus (8)

• Enhanced LSTM for Natural Language Inference
• ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
• Neural Natural Language Inference Models Enhanced with External Knowledge
• The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision
• Enhancing Sentence Embedding with Generalized Pooling
• Neural Module Networks for Reasoning over Text
• Neural Logic Machines
• Dynamic Self-Attention : Computing Attention over Words Dynamically for Sentence Embedding