Analyzing Differentiable Fuzzy Logic Operators

Abstract

The AI community is increasingly putting its attention towards combining symbolic and neural approaches, as it is often argued that the strengths and weaknesses of these approaches are complementary. One recent trend in the literature are weakly supervised learning techniques that employ operators from fuzzy logics. In particular, these use prior background knowledge described in such logics to help the training of a neural network from unlabeled and noisy data. By interpreting logical symbols using neural networks, this background knowledge can be added to regular loss functions, hence making reasoning a part of learning. We study, both formally and empirically, how a large collection of logical operators from the fuzzy logic literature behave in a differentiable learning setting. We find that many of these operators, including some of the most well-known, are highly unsuitable in this setting. A further finding concerns the treatment of implication in these fuzzy logics, and shows a strong imbalance between gradients driven by the antecedent and the consequent of the implication. Furthermore, we introduce a new family of fuzzy implications (called sigmoidal implications) to tackle this phenomenon. Finally, we empirically show that it is possible to use Differentiable Fuzzy Logics for semi-supervised learning, and compare how different operators behave in practice. We find that, to achieve the largest performance improvement over a supervised baseline, we have to resort to non-standard combinations of logical operators which perform well in learning, but no longer satisfy the usual logical laws.

47 pages, 18 figures. V2: Added analysis for existential quantification. Improved experiments and writing

References in corpus (11)

• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Loopy Belief Propagation for Approximate Inference: An Empirical Study
• Deep Learning: A Critical Appraisal
• Logic Tensor Networks
• Neural-Symbolic Learning and Reasoning: A Survey and Interpretation
• Towards Deep Symbolic Reinforcement Learning
• DeepProbLog: Neural Probabilistic Logic Programming
• Logical Neural Networks
• Semi-Supervised Learning using Differentiable Reasoning
• Constraint-Based Visual Generation
• Injecting Prior Knowledge for Transfer Learning into Reinforcement Learning Algorithms using Logic Tensor Networks

Cited by in corpus (5)

• Logic Tensor Networks
• GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection
• Zero-shot Multi-Domain Dialog State Tracking Using Descriptive Rules
• Fuzzy Generative Adversarial Networks
• Every time I fire a conversational designer, the performance of the dialog system goes down