Neural graph embeddings as explicit low-rank matrix factorization for link prediction
arXiv:2011.09907 · doi:10.1016/j.patcog.2022.108977
Abstract
Learning good quality neural graph embeddings has long been achieved by minimizing the point-wise mutual information (PMI) for co-occurring nodes in simulated random walks. This design choice has been mostly popularized by the direct application of the highly-successful word embedding algorithm word2vec to predicting the formation of new links in social, co-citation, and biological networks. However, such a skeuomorphic design of graph embedding methods entails a truncation of information coming from pairs of nodes with low PMI. To circumvent this issue, we propose an improved approach to learning low-rank factorization embeddings that incorporate information from such unlikely pairs of nodes and show that it can improve the link prediction performance of baseline methods from 1.2% to 24.2%. Based on our results and observations we outline further steps that could improve the design of next graph embedding algorithms that are based on matrix factorization.
References in corpus (10)
- Emergence of scaling in random networks
- Efficient Estimation of Word Representations in Vector Space
- Statistical mechanics of complex networks
- Cooperative Game Theory Approaches for Network Partitioning
- Evolution of networks
- Spectral Networks and Locally Connected Networks on Graphs
- Diffusion-Convolutional Neural Networks
- GraphAIR: Graph Representation Learning with Neighborhood Aggregation and Interaction
- GLEE: Geometric Laplacian Eigenmap Embedding
- Word Embedding Algorithms as Generalized Low Rank Models and their Canonical Form