Improving Deep Learning Models via Constraint-Based Domain Knowledge: a Brief Survey

Abstract

Deep Learning (DL) models proved themselves to perform extremely well on a wide variety of learning tasks, as they can learn useful patterns from large data sets. However, purely data-driven models might struggle when very difficult functions need to be learned or when there is not enough available training data. Fortunately, in many domains prior information can be retrieved and used to boost the performance of DL models. This paper presents a first survey of the approaches devised to integrate domain knowledge, expressed in the form of constraints, in DL learning models to improve their performance, in particular targeting deep neural networks. We identify five (non-mutually exclusive) categories that encompass the main approaches to inject domain knowledge: 1) acting on the features space, 2) modifications to the hypothesis space, 3) data augmentation, 4) regularization schemes, 5) constrained learning.

References in corpus (13)

• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• The Effectiveness of Data Augmentation in Image Classification using Deep Learning
• Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems
• SMILES Enumeration as Data Augmentation for Neural Network Modeling of Molecules
• Task-based End-to-end Model Learning in Stochastic Optimization
• Data augmentation instead of explicit regularization
• Imposing Hard Constraints on Deep Networks: Promises and Limitations
• Feature Engineering for Predictive Modeling using Reinforcement Learning
• The More You Know: Using Knowledge Graphs for Image Classification
• Combination of Domain Knowledge and Deep Learning for Sentiment Analysis
• Combining Learning and Optimization for Transprecision Computing
• Neural-encoding Human Experts' Domain Knowledge to Warm Start Reinforcement Learning
• Augmenting Neural Networks with First-order Logic

Cited by in corpus (9)

• Enhancing Discrete Choice Models with Representation Learning
• Injecting Knowledge in Data-driven Vehicle Trajectory Predictors
• A Lagrangian Dual-based Theory-guided Deep Neural Network
• Building Domain-Specific Machine Learning Workflows: A Conceptual Framework for the State-of-the-Practice
• CorrLoss: Integrating Co-Occurrence Domain Knowledge for Affect Recognition
• Guiding Video Prediction with Explicit Procedural Knowledge
• A Quantitative Perspective on Values of Domain Knowledge for Machine Learning
• Informative regularization for a multi-layer perceptron RR Lyrae classifier under data shift
• A Clustering-aided Ensemble Method for Predicting Ridesourcing Demand in Chicago