Variational Graph Normalized Auto-Encoders

Abstract

Link prediction is one of the key problems for graph-structured data. With the advancement of graph neural networks, graph autoencoders (GAEs) and variational graph autoencoders (VGAEs) have been proposed to learn graph embeddings in an unsupervised way. It has been shown that these methods are effective for link prediction tasks. However, they do not work well in link predictions when a node whose degree is zero (i.g., isolated node) is involved. We have found that GAEs/VGAEs make embeddings of isolated nodes close to zero regardless of their content features. In this paper, we propose a novel Variational Graph Normalized AutoEncoder (VGNAE) that utilize L2-normalization to derive better embeddings for isolated nodes. We show that our VGNAEs outperform the existing state-of-the-art models for link prediction tasks. The code is available at https://github.com/SeongJinAhn/VGNAE.

References in corpus (20)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Semi-Supervised Classification with Graph Convolutional Networks
• Spectral Networks and Locally Connected Networks on Graphs
• Simplifying Graph Convolutional Networks
• Variational Graph Auto-Encoders
• Pitfalls of Graph Neural Network Evaluation
• NormFace: L2 Hypersphere Embedding for Face Verification
• Predict then Propagate: Graph Neural Networks meet Personalized PageRank
• L2-constrained Softmax Loss for Discriminative Face Verification
• DeeperGCN: All You Need to Train Deeper GCNs
• PairNorm: Tackling Oversmoothing in GNNs
• How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision
• Adversarially Regularized Graph Autoencoder for Graph Embedding
• Graph InfoClust: Leveraging cluster-level node information for unsupervised graph representation learning
• Measuring Word Significance using Distributed Representations of Words
• Attention is Not Only a Weight: Analyzing Transformers with Vector Norms
• Deep Metric Learning with Spherical Embedding
• How much does a word weigh? Weighting word embeddings for word sense induction
• Graph Convolutional Networks using Heat Kernel for Semi-supervised Learning

Cited by in corpus (5)

• Reliable Node Similarity Matrix Guided Contrastive Graph Clustering
• A Review of Graph-Powered Data Quality Applications for IoT Monitoring Sensor Networks
• Independent Distribution Regularization for Private Graph Embedding
• CoCon: A Data Set on Combined Contextualized Research Artifact Use
• Contributions to Representation Learning with Graph Autoencoders and Applications to Music Recommendation