A Quantitative Perspective on Values of Domain Knowledge for Machine Learning

Abstract

With the exploding popularity of machine learning, domain knowledge in various forms has been playing a crucial role in improving the learning performance, especially when training data is limited. Nonetheless, there is little understanding of to what extent domain knowledge can affect a machine learning task from a quantitative perspective. To increase the transparency and rigorously explain the role of domain knowledge in machine learning, we study the problem of quantifying the values of domain knowledge in terms of its contribution to the learning performance in the context of informed machine learning. We propose a quantification method based on Shapley value that fairly attributes the overall learning performance improvement to different domain knowledge. We also present Monte-Carlo sampling to approximate the fair value of domain knowledge with a polynomial time complexity. We run experiments of injecting symbolic domain knowledge into semi-supervised learning tasks on both MNIST and CIFAR10 datasets, providing quantitative values of different symbolic knowledge and rigorously explaining how it affects the machine learning performance in terms of test accuracy.

References in corpus (17)

• Distilling the Knowledge in a Neural Network
• European Union regulations on algorithmic decision-making and a "right to explanation"
• Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems
• Model-Agnostic Interpretability of Machine Learning
• Interaction Networks for Learning about Objects, Relations and Physics
• Towards Robust Interpretability with Self-Explaining Neural Networks
• One-Shot Unsupervised Cross Domain Translation
• Reinforcement Learning from Imperfect Demonstrations
• Hybrid Knowledge Routed Modules for Large-scale Object Detection
• A Simple and Effective Model-Based Variable Importance Measure
• Neuron Shapley: Discovering the Responsible Neurons
• Bounding the Estimation Error of Sampling-based Shapley Value Approximation
• Improving Deep Learning Models via Constraint-Based Domain Knowledge: a Brief Survey
• APQ: Joint Search for Network Architecture, Pruning and Quantization Policy
• Explaining Deep Neural Networks
• Who's responsible? Jointly quantifying the contribution of the learning algorithm and training data
• ModuleNet: Knowledge-inherited Neural Architecture Search

Cited by in corpus (1)

• Self-explaining variational posterior distributions for Gaussian Process models