Knowledge-Based Distant Regularization in Learning Probabilistic Models

Abstract

Exploiting the appropriate inductive bias based on the knowledge of data is essential for achieving good performance in statistical machine learning. In practice, however, the domain knowledge of interest often provides information on the relationship of data attributes only distantly, which hinders direct utilization of such domain knowledge in popular regularization methods. In this paper, we propose the knowledge-based distant regularization framework, in which we utilize the distant information encoded in a knowledge graph for regularization of probabilistic model estimation. In particular, we propose to impose prior distributions on model parameters specified by knowledge graph embeddings. As an instance of the proposed framework, we present the factor analysis model with the knowledge-based distant regularization. We show the results of preliminary experiments on the improvement of the generalization capability of such model.

arXiv comment7 pages, 2 figures, presented in the Eighth International Workshop on Statistical Relational AI

References in corpus (6)

• A Review of Relational Machine Learning for Knowledge Graphs
• Learning with Structured Sparsity
• SSP: Semantic Space Projection for Knowledge Graph Embedding with Text Descriptions
• Fast Robust PCA on Graphs
• Learning the Structure of Generative Models without Labeled Data
• Supervised Feature Selection in Graphs with Path Coding Penalties and Network Flows

Cited by in corpus (2)

• A Review of Some Techniques for Inclusion of Domain-Knowledge into Deep Neural Networks
• Incorporating Domain Knowledge into Deep Neural Networks