Technical Report of Team GraphMIRAcles in the WikiKG90M-LSC Track of OGB-LSC @ KDD Cup 2021

Abstract

Link prediction in large-scale knowledge graphs has gained increasing attention recently. The OGB-LSC team presented OGB Large-Scale Challenge (OGB-LSC), a collection of three real-world datasets for advancing the state-of-the-art in large-scale graph machine learning. In this paper, we introduce the solution of our team GraphMIRAcles in the WikiKG90M-LSC track of OGB-LSC @ KDD Cup 2021. In the WikiKG90M-LSC track, the goal is to automatically predict missing links in WikiKG90M, a large scale knowledge graph extracted from Wikidata. To address this challenge, we propose a framework that integrates three components -- a basic model ComplEx-CMRC, a rule miner AMIE 3, and an inference model to predict missing links. Experiments demonstrate that our solution achieves an MRR of 0.9707 on the test dataset. Moreover, as the knowledge distillation in the inference model uses test tail candidates -- which are unavailable in practice -- we conduct ablation studies on knowledge distillation. Experiments demonstrate that our model without knowledge distillation achieves an MRR of 0.9533 on the full validation dataset.

References in corpus (6)

• Distilling the Knowledge in a Neural Network
• RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space
• Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning
• OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs
• Duality-Induced Regularizer for Tensor Factorization Based Knowledge Graph Completion
• Topology-Aware Correlations Between Relations for Inductive Link Prediction in Knowledge Graphs