Self-Supervised Graph Learning with Proximity-based Views and Channel Contrast

Abstract

We consider graph representation learning in a self-supervised manner. Graph neural networks (GNNs) use neighborhood aggregation as a core component that results in feature smoothing among nodes in proximity. While successful in various prediction tasks, such a paradigm falls short of capturing nodes' similarities over a long distance, which proves to be important for high-quality learning. To tackle this problem, we strengthen the graph with two additional graph views, in which nodes are directly linked to those with the most similar features or local structures. Not restricted by connectivity in the original graph, the generated views allow the model to enhance its expressive power with new and complementary perspectives from which to look at the relationship between nodes. Following a contrastive learning approach, we propose a method that aims to maximize the agreement between representations across generated views and the original graph. We also propose a channel-level contrast approach that greatly reduces computation cost, compared to the commonly used node level contrast, which requires computation cost quadratic in the number of nodes. Extensive experiments on seven assortative graphs and four disassortative graphs demonstrate the effectiveness of our approach.

arXiv comment17 pages, 8 figures

References in corpus (18)

• Adam: A Method for Stochastic Optimization
• Scikit-learn: Machine Learning in Python
• The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Fast Graph Representation Learning with PyTorch Geometric
• Simplifying Graph Convolutional Networks
• Graph Contrastive Learning with Adaptive Augmentation
• struc2vec: Learning Node Representations from Structural Identity
• Variational Graph Auto-Encoders
• Deep Graph Contrastive Representation Learning
• Learning Structural Node Embeddings Via Diffusion Wavelets
• VERSE: Versatile Graph Embeddings from Similarity Measures
• InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization
• subgraph2vec: Learning Distributed Representations of Rooted Sub-graphs from Large Graphs
• Geom-GCN: Geometric Graph Convolutional Networks
• A Mutual Information Maximization Perspective of Language Representation Learning
• Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networks