A Semantic Loss Function for Deep Learning with Symbolic Knowledge

Abstract

This paper develops a novel methodology for using symbolic knowledge in deep learning. From first principles, we derive a semantic loss function that bridges between neural output vectors and logical constraints. This loss function captures how close the neural network is to satisfying the constraints on its output. An experimental evaluation shows that it effectively guides the learner to achieve (near-)state-of-the-art results on semi-supervised multi-class classification. Moreover, it significantly increases the ability of the neural network to predict structured objects, such as rankings and paths. These discrete concepts are tremendously difficult to learn, and benefit from a tight integration of deep learning and symbolic reasoning methods.

This version appears in the Proceedings of the 35th International Conference on Machine Learning (ICML 2018)

References in corpus (4)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
• Imposing Hard Constraints on Deep Networks: Promises and Limitations
• Adversarial Sets for Regularising Neural Link Predictors

Cited by in corpus (13)

• Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems
• Ontology Reasoning with Deep Neural Networks
• A Boxology of Design Patterns for Hybrid Learning and Reasoning Systems
• Multi-Label Classification Neural Networks with Hard Logical Constraints
• ROAD-R: The Autonomous Driving Dataset with Logical Requirements
• Inclusion of Domain-Knowledge into GNNs using Mode-Directed Inverse Entailment
• Relational Generalized Few-Shot Learning
• Faster and Safer Training by Embedding High-Level Knowledge into Deep Reinforcement Learning
• PK-GCN: Prior Knowledge Assisted Image Classification using Graph Convolution Networks
• Adversarial Constraint Learning for Structured Prediction
• SaaS: Speed as a Supervisor for Semi-supervised Learning
• Adma: A Flexible Loss Function for Neural Networks
• Semantic Loss Application to Entity Relation Recognition