A Comparative Study for Unsupervised Network Representation Learning

Abstract

There has been appreciable progress in unsupervised network representation learning (UNRL) approaches over graphs recently with flexible random-walk approaches, new optimization objectives and deep architectures. However, there is no common ground for systematic comparison of embeddings to understand their behavior for different graphs and tasks. In this paper we theoretically group different approaches under a unifying framework and empirically investigate the effectiveness of different network representation methods. In particular, we argue that most of the UNRL approaches either explicitly or implicit model and exploit context information of a node. Consequently, we propose a framework that casts a variety of approaches -- random walk based, matrix factorization and deep learning based -- into a unified context-based optimization function. We systematically group the methods based on their similarities and differences. We study the differences among these methods in detail which we later use to explain their performance differences (on downstream tasks). We conduct a large-scale empirical study considering 9 popular and recent UNRL techniques and 11 real-world datasets with varying structural properties and two common tasks -- node classification and link prediction. We find that there is no single method that is a clear winner and that the choice of a suitable method is dictated by certain properties of the embedding methods, task and structural properties of the underlying graph. In addition we also report the common pitfalls in evaluation of UNRL methods and come up with suggestions for experimental design and interpretation of results.

Accepted for publication in IEEE TKDE

References in corpus (12)

• Distributed Representations of Words and Phrases and their Compositionality
• DeepWalk: Online Learning of Social Representations
• LINE: Large-scale Information Network Embedding
• Inductive Representation Learning on Large Graphs
• Graph Convolutional Neural Networks for Web-Scale Recommender Systems
• Graph Embedding Techniques, Applications, and Performance: A Survey
• Variational Graph Auto-Encoders
• Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec
• Pitfalls of Graph Neural Network Evaluation
• VERSE: Versatile Graph Embeddings from Similarity Measures
• Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Ranking
• Adversarially Regularized Graph Autoencoder for Graph Embedding

Cited by in corpus (11)

• MultiVERSE: a multiplex and multiplex-heterogeneous network embedding approach
• Network representation learning systematic review: ancestors and current development state
• GraphWorld: Fake Graphs Bring Real Insights for GNNs
• Link prediction in dynamic networks using random dot product graphs
• Explainable, Stable, and Scalable Graph Convolutional Networks for Learning Graph Representation
• Self-supervised edge features for improved Graph Neural Network training
• Predicting Feynman periods in ${\varphi }^4$-theory
• GraphScale: A Framework to Enable Machine Learning over Billion-node Graphs
• Graph-based State Representation for Deep Reinforcement Learning
• Free Energy Node Embedding via Generalized Skip-gram with Negative Sampling
• CitationIE: Leveraging the Citation Graph for Scientific Information Extraction