Network Representation Learning: Consolidation and Renewed Bearing

Abstract

Graphs are a natural abstraction for many problems where nodes represent entities and edges represent a relationship across entities. An important area of research that has emerged over the last decade is the use of graphs as a vehicle for non-linear dimensionality reduction in a manner akin to previous efforts based on manifold learning with uses for downstream database processing, machine learning and visualization. In this systematic yet comprehensive experimental survey, we benchmark several popular network representation learning methods operating on two key tasks: link prediction and node classification. We examine the performance of 12 unsupervised embedding methods on 15 datasets. To the best of our knowledge, the scale of our study -- both in terms of the number of methods and number of datasets -- is the largest to date. Our results reveal several key insights about work-to-date in this space. First, we find that certain baseline methods (task-specific heuristics, as well as classic manifold methods) that have often been dismissed or are not considered by previous efforts can compete on certain types of datasets if they are tuned appropriately. Second, we find that recent methods based on matrix factorization offer a small but relatively consistent advantage over alternative methods (e.g., random-walk based methods) from a qualitative standpoint. Specifically, we find that MNMF, a community preserving embedding method, is the most competitive method for the link prediction task. While NetMF is the most competitive baseline for node classification. Third, no single method completely outperforms other embedding methods on both node classification and link prediction tasks. We also present several drill-down analysis that reveals settings under which certain algorithms perform well (e.g., the role of neighborhood context on performance) -- guiding the end-user.

References in corpus (17)

• Emergence of scaling in random networks
• Distributed Representations of Words and Phrases and their Compositionality
• Inductive Representation Learning on Large Graphs
• Link Prediction in Complex Networks: A Survey
• Graph Embedding Techniques, Applications, and Performance: A Survey
• Predicting Missing Links via Local Information
• A Review of Relational Machine Learning for Knowledge Graphs
• Higher-order organization of complex networks
• struc2vec: Learning Node Representations from Structural Identity
• Variational Graph Auto-Encoders
• VERSE: Versatile Graph Embeddings from Similarity Measures
• Watch Your Step: Learning Node Embeddings via Graph Attention
• Anomaly Detection with Score functions based on Nearest Neighbor Graphs
• Diving into the shallows: a computational perspective on large-scale shallow learning
• MILE: A Multi-Level Framework for Scalable Graph Embedding
• PL-NMF: Parallel Locality-Optimized Non-negative Matrix Factorization
• Fusion Graph Convolutional Networks

Cited by in corpus (4)

• A Note on Over-Smoothing for Graph Neural Networks
• MILE: A Multi-Level Framework for Scalable Graph Embedding
• Benchmarking Network Embedding Models for Link Prediction: Are We Making Progress?
• CSNE: Conditional Signed Network Embedding