A Review of Some Techniques for Inclusion of Domain-Knowledge into Deep Neural Networks

Abstract

We present a survey of ways in which existing scientific knowledge are included when constructing models with neural networks. The inclusion of domain-knowledge is of special interest not just to constructing scientific assistants, but also, many other areas that involve understanding data using human-machine collaboration. In many such instances, machine-based model construction may benefit significantly from being provided with human-knowledge of the domain encoded in a sufficiently precise form. This paper examines the inclusion of domain-knowledge by means of changes to: the input, the loss-function, and the architecture of deep networks. The categorisation is for ease of exposition: in practice we expect a combination of such changes will be employed. In each category, we describe techniques that have been shown to yield significant changes in the performance of deep neural networks.

16 pages; Accepted at Nature Scientific Reports. arXiv admin note: substantial text overlap with arXiv:2103.00180

References in corpus (16)

• Distilling the Knowledge in a Neural Network
• Knowledge Graph Convolutional Networks for Recommender Systems
• Neural-Symbolic Learning and Reasoning: A Survey and Interpretation
• Logic Tensor Networks: Deep Learning and Logical Reasoning from Data and Knowledge
• The Modern Mathematics of Deep Learning
• Estimating Node Importance in Knowledge Graphs Using Graph Neural Networks
• Logical Neural Networks
• DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning
• Incorporating Symbolic Domain Knowledge into Graph Neural Networks
• Inclusion of Domain-Knowledge into GNNs using Mode-Directed Inverse Entailment
• Beyond Graph Neural Networks with Lifted Relational Neural Networks
• "When they say weed causes depression, but it's your fav antidepressant": Knowledge-aware Attention Framework for Relationship Extraction
• NeuralLog: a Neural Logic Language
• "Who can help me?": Knowledge Infused Matching of Support Seekers and Support Providers during COVID-19 on Reddit
• DeepStochLog: Neural Stochastic Logic Programming
• MultiplexNet: Towards Fully Satisfied Logical Constraints in Neural Networks

Cited by in corpus (13)

• Deep Learning with Logical Constraints
• A Survey on Large Language Model-based Agents for Statistics and Data Science
• Towards responsible AI for education: Hybrid human-AI to confront the Elephant in the room
• RISnet: A Domain-Knowledge Driven Neural Network Architecture for RIS Optimization with Mutual Coupling and Partial CSI
• LAMBDA: A Large Model Based Data Agent
• Superpixel-based Knowledge Infusion in Deep Neural Networks for Image Classification
• Brain informed transfer learning for categorizing construction hazards
• Framework for Learning and Control in the Classical and Quantum Domains
• Explanatory Debiasing: Involving Domain Experts in the Data Generation Process to Mitigate Representation Bias in AI Systems
• Identifying type II quasars at intermediate redshift with few-shot learning photometric classification
• Beyond Traditional Neural Networks: Toward adding Reasoning and Learning Capabilities through Computational Logic Techniques
• NuGraph2 with Context-Aware Inputs: Physics-Inspired Improvements in Semantic Segmentation
• Domain Knowledge Based Temporal-Spatial Graph Convolution Network for ECG Recognition