Understanding the Performance of Knowledge Graph Embeddings in Drug Discovery

Abstract

Knowledge Graphs (KG) and associated Knowledge Graph Embedding (KGE) models have recently begun to be explored in the context of drug discovery and have the potential to assist in key challenges such as target identification. In the drug discovery domain, KGs can be employed as part of a process which can result in lab-based experiments being performed, or impact on other decisions, incurring significant time and financial costs and most importantly, ultimately influencing patient healthcare. For KGE models to have impact in this domain, a better understanding of not only of performance, but also the various factors which determine it, is required. In this study we investigate, over the course of many thousands of experiments, the predictive performance of five KGE models on two public drug discovery-oriented KGs. Our goal is not to focus on the best overall model or configuration, instead we take a deeper look at how performance can be affected by changes in the training setup, choice of hyperparameters, model parameter initialisation seed and different splits of the datasets. Our results highlight that these factors have significant impact on performance and can even affect the ranking of models. Indeed these factors should be reported along with model architectures to ensure complete reproducibility and fair comparisons of future work, and we argue this is critical for the acceptance of use, and impact of KGEs in a biomedical setting.

References in corpus (6)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Variational Graph Auto-Encoders
• RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space
• A Review of Biomedical Datasets Relating to Drug Discovery: A Knowledge Graph Perspective
• PyKEEN 1.0: A Python Library for Training and Evaluating Knowledge Graph Embeddings
• Neural Multi-Hop Reasoning With Logical Rules on Biomedical Knowledge Graphs

Cited by in corpus (5)

• Toward Degree Bias in Embedding-Based Knowledge Graph Completion
• BioBLP: A Modular Framework for Learning on Multimodal Biomedical Knowledge Graphs
• A Knowledge Graph-Enhanced Tensor Factorisation Model for Discovering Drug Targets
• Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study
• Improving Molecule Generation and Drug Discovery with a Knowledge-enhanced Generative Model