CoKE: Contextualized Knowledge Graph Embedding

Abstract

Knowledge graph embedding, which projects symbolic entities and relations into continuous vector spaces, is gaining increasing attention. Previous methods allow a single static embedding for each entity or relation, ignoring their intrinsic contextual nature, i.e., entities and relations may appear in different graph contexts, and accordingly, exhibit different properties. This work presents Contextualized Knowledge Graph Embedding (CoKE), a novel paradigm that takes into account such contextual nature, and learns dynamic, flexible, and fully contextualized entity and relation embeddings. Two types of graph contexts are studied: edges and paths, both formulated as sequences of entities and relations. CoKE takes a sequence as input and uses a Transformer encoder to obtain contextualized representations. These representations are hence naturally adaptive to the input, capturing contextual meanings of entities and relations therein. Evaluation on a wide variety of public benchmarks verifies the superiority of CoKE in link prediction and path query answering. It performs consistently better than, or at least equally well as current state-of-the-art in almost every case, in particular offering an absolute improvement of 21.0% in H@10 on path query answering. Our code is available at \url{https://github.com/PaddlePaddle/Research/tree/master/KG/CoKE}.

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• A Review of Relational Machine Learning for Knowledge Graphs
• A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network
• RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space
• Hypernetwork Knowledge Graph Embeddings
• Probabilistic Logic Neural Networks for Reasoning
• Modeling Relational Data with Graph Convolutional Networks
• KBLRN : End-to-End Learning of Knowledge Base Representations with Latent, Relational, and Numerical Features

Cited by in corpus (12)

• A Survey on Knowledge Graphs: Representation, Acquisition and Applications
• Multi-hop Attention Graph Neural Network
• CoLAKE: Contextualized Language and Knowledge Embedding
• Adaptive Attentional Network for Few-Shot Knowledge Graph Completion
• Link Prediction on N-ary Relational Facts: A Graph-based Approach
• Knowledge Graph Embedding with Atrous Convolution and Residual Learning
• KACC: A Multi-task Benchmark for Knowledge Abstraction, Concretization and Completion
• Benchmark and Best Practices for Biomedical Knowledge Graph Embeddings
• KARL-Trans-NER: Knowledge Aware Representation Learning for Named Entity Recognition using Transformers
• OKGIT: Open Knowledge Graph Link Prediction with Implicit Types
• PPKE: Knowledge Representation Learning by Path-based Pre-training
• HittER: Hierarchical Transformers for Knowledge Graph Embeddings