KG-BERT: BERT for Knowledge Graph Completion

Abstract

Knowledge graphs are important resources for many artificial intelligence tasks but often suffer from incompleteness. In this work, we propose to use pre-trained language models for knowledge graph completion. We treat triples in knowledge graphs as textual sequences and propose a novel framework named Knowledge Graph Bidirectional Encoder Representations from Transformer (KG-BERT) to model these triples. Our method takes entity and relation descriptions of a triple as input and computes scoring function of the triple with the KG-BERT language model. Experimental results on multiple benchmark knowledge graphs show that our method can achieve state-of-the-art performance in triple classification, link prediction and relation prediction tasks.

References in corpus (6)

• Distributed Representations of Words and Phrases and their Compositionality
• A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network
• RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space
• ProjE: Embedding Projection for Knowledge Graph Completion
• SSP: Semantic Space Projection for Knowledge Graph Embedding with Text Descriptions
• DOLORES: Deep Contextualized Knowledge Graph Embeddings

Cited by in corpus (40)

• A Survey on Knowledge Graphs: Representation, Acquisition and Applications
• Unifying Large Language Models and Knowledge Graphs: A Roadmap
• Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion
• Drug Repurposing for COVID-19 via Knowledge Graph Completion
• A Comparative Study of Pretrained Language Models for Long Clinical Text
• A Review of Knowledge Graph Completion
• From Discrimination to Generation: Knowledge Graph Completion with Generative Transformer
• KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completion
• Ontology-enhanced Prompt-tuning for Few-shot Learning
• Relphormer: Relational Graph Transformer for Knowledge Graph Representations
• Enhancing Heterogeneous Knowledge Graph Completion with a Novel GAT-based Approach
• Knowledge Enhanced Pretrained Language Models: A Compreshensive Survey
• r-GAT: Relational Graph Attention Network for Multi-Relational Graphs
• Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model
• Trigger-GNN: A Trigger-Based Graph Neural Network for Nested Named Entity Recognition
• Triple Classification for Scholarly Knowledge Graph Completion
• Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study
• Combining pre-trained language models and structured knowledge
• Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning
• Progressive Distillation Based on Masked Generation Feature Method for Knowledge Graph Completion
• A Roadmap to Domain Knowledge Integration in Machine Learning
• On the Role of Conceptualization in Commonsense Knowledge Graph Construction
• Knowledge Graph Embedding with Atrous Convolution and Residual Learning
• Discovering Supply Chain Links with Augmented Intelligence
• Temporal Knowledge Graph Reasoning Triggered by Memories
• Relation Adversarial Network for Low Resource Knowledge Graph Completion
• Plumber: A Modular Framework to Create Information Extraction Pipelines
• Relation Clustering in Narrative Knowledge Graphs
• Knowledge Graph Refinement based on Triplet BERT-Networks
• Inductive Learning on Commonsense Knowledge Graph Completion
• Path-based knowledge reasoning with textual semantic information for medical knowledge graph completion
• Effective FAQ Retrieval and Question Matching With Unsupervised Knowledge Injection
• Relational World Knowledge Representation in Contextual Language Models: A Review
• Commonsense Knowledge Mining from Term Definitions
• OKGIT: Open Knowledge Graph Link Prediction with Implicit Types
• Technical Report of Team GraphMIRAcles in the WikiKG90M-LSC Track of OGB-LSC @ KDD Cup 2021
• Alleviating the Knowledge-Language Inconsistency: A Study for Deep Commonsense Knowledge
• Benchmarking Commonsense Knowledge Base Population with an Effective Evaluation Dataset
• Technological taxonomies for hypernym and hyponym retrieval in patent texts
• CEAR: Cross-Entity Aware Reranker for Knowledge Base Completion