Benchmarking Network Embedding Models for Link Prediction: Are We Making Progress?

Abstract

Network embedding methods map a network's nodes to vectors in an embedding space, in such a way that these representations are useful for estimating some notion of similarity or proximity between pairs of nodes in the network. The quality of these node representations is then showcased through results of downstream prediction tasks. Commonly used benchmark tasks such as link prediction, however, present complex evaluation pipelines and an abundance of design choices. This, together with a lack of standardized evaluation setups can obscure the real progress in the field. In this paper, we aim to shed light on the state-of-the-art of network embedding methods for link prediction and show, using a consistent evaluation pipeline, that only thin progress has been made over the last years. The newly conducted benchmark that we present here, including 17 embedding methods, also shows that many approaches are outperformed even by simple heuristics. Finally, we argue that standardized evaluation tools can repair this situation and boost future progress in this field.

References in corpus (15)

• Efficient Estimation of Word Representations in Vector Space
• DeepWalk: Online Learning of Social Representations
• LINE: Large-scale Information Network Embedding
• Graph Embedding Techniques, Applications, and Performance: A Survey
• struc2vec: Learning Node Representations from Structural Identity
• Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches
• Representation Learning on Graphs: Methods and Applications
• Evaluating Link Prediction Methods
• VERSE: Versatile Graph Embeddings from Similarity Measures
• Stacking Models for Nearly Optimal Link Prediction in Complex Networks
• Watch Your Step: Learning Node Embeddings via Graph Attention
• Analysis of the Impact of Negative Sampling on Link Prediction in Knowledge Graphs
• Network Representation Learning: A Survey
• Conditional Network Embeddings
• Network Representation Learning: Consolidation and Renewed Bearing

Cited by in corpus (3)

• Progresses and Challenges in Link Prediction
• Source Localization of Graph Diffusion via Variational Autoencoders for Graph Inverse Problems
• EvalNE: A Framework for Evaluating Network Embeddings on Link Prediction