A Review of Some Techniques for Inclusion of Domain-Knowledge into Deep Neural Networks

Abstract

We present a survey of ways in which existing scientific knowledge are included when constructing models with neural networks. The inclusion of domain-knowledge is of special interest not just to constructing scientific assistants, but also, many other areas that involve understanding data using human-machine collaboration. In many such instances, machine-based model construction may benefit significantly from being provided with human-knowledge of the domain encoded in a sufficiently precise form. This paper examines the inclusion of domain-knowledge by means of changes to: the input, the loss-function, and the architecture of deep networks. The categorisation is for ease of exposition: in practice we expect a combination of such changes will be employed. In each category, we describe techniques that have been shown to yield significant changes in the performance of deep neural networks.

arXiv comment16 pages; Accepted at Nature Scientific Reports. arXiv admin note: substantial text overlap with arXiv:2103.00180

References in corpus (22)

• Distilling the Knowledge in a Neural Network
• Artificial Intelligence: the global landscape of ethics guidelines
• Deep Visual Domain Adaptation: A Survey
• Knowledge Graph Convolutional Networks for Recommender Systems
• Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems
• Neural-Symbolic Learning and Reasoning: A Survey and Interpretation
• Logic Tensor Networks: Deep Learning and Logical Reasoning from Data and Knowledge
• The Modern Mathematics of Deep Learning
• Estimating Node Importance in Knowledge Graphs Using Graph Neural Networks
• Logical Neural Networks
• kLog: A Language for Logical and Relational Learning with Kernels
• DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning
• Incorporating Symbolic Domain Knowledge into Graph Neural Networks
• Propositionalization and Embeddings: Two Sides of the Same Coin
• Inclusion of Domain-Knowledge into GNNs using Mode-Directed Inverse Entailment
• Beyond Graph Neural Networks with Lifted Relational Neural Networks
• "When they say weed causes depression, but it's your fav antidepressant": Knowledge-aware Attention Framework for Relationship Extraction
• Knowledge-Based Distant Regularization in Learning Probabilistic Models
• NeuralLog: a Neural Logic Language
• "Who can help me?": Knowledge Infused Matching of Support Seekers and Support Providers during COVID-19 on Reddit
• MultiplexNet: Towards Fully Satisfied Logical Constraints in Neural Networks
• DeepStochLog: Neural Stochastic Logic Programming

Cited by in corpus (13)

• Deep Learning with Logical Constraints
• A Survey on Large Language Model-based Agents for Statistics and Data Science
• Towards responsible AI for education: Hybrid human-AI to confront the Elephant in the room
• RISnet: A Domain-Knowledge Driven Neural Network Architecture for RIS Optimization with Mutual Coupling and Partial CSI
• LAMBDA: A Large Model Based Data Agent
• Superpixel-based Knowledge Infusion in Deep Neural Networks for Image Classification
• Brain informed transfer learning for categorizing construction hazards
• Framework for Learning and Control in the Classical and Quantum Domains
• Explanatory Debiasing: Involving Domain Experts in the Data Generation Process to Mitigate Representation Bias in AI Systems
• Identifying type II quasars at intermediate redshift with few-shot learning photometric classification
• Beyond Traditional Neural Networks: Toward adding Reasoning and Learning Capabilities through Computational Logic Techniques
• NuGraph2 with Context-Aware Inputs: Physics-Inspired Improvements in Semantic Segmentation
• Domain Knowledge Based Temporal-Spatial Graph Convolution Network for ECG Recognition