Graph Self-Supervised Learning: A Survey

Abstract

Deep learning on graphs has attracted significant interests recently. However, most of the works have focused on (semi-) supervised learning, resulting in shortcomings including heavy label reliance, poor generalization, and weak robustness. To address these issues, self-supervised learning (SSL), which extracts informative knowledge through well-designed pretext tasks without relying on manual labels, has become a promising and trending learning paradigm for graph data. Different from SSL on other domains like computer vision and natural language processing, SSL on graphs has an exclusive background, design ideas, and taxonomies. Under the umbrella of graph self-supervised learning, we present a timely and comprehensive review of the existing approaches which employ SSL techniques for graph data. We construct a unified framework that mathematically formalizes the paradigm of graph SSL. According to the objectives of pretext tasks, we divide these approaches into four categories: generation-based, auxiliary property-based, contrast-based, and hybrid approaches. We further describe the applications of graph SSL across various research fields and summarize the commonly used datasets, evaluation benchmark, performance comparison and open-source codes of graph SSL. Finally, we discuss the remaining challenges and potential future directions in this research field.

26 pages, 9 figures, 9 tables

References in corpus (51)

• LINE: Large-scale Information Network Embedding
• Bootstrap your own latent: A new approach to self-supervised Learning
• Graph Contrastive Learning with Adaptive Augmentation
• Variational Graph Auto-Encoders
• Graph Contrastive Learning with Augmentations
• GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• Predicting multicellular function through multi-layer tissue networks
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning
• Deep Graph Contrastive Representation Learning
• Contrastive Multi-View Representation Learning on Graphs
• Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection
• Multi-view Graph Contrastive Representation Learning for Drug-Drug Interaction Prediction
• Graph-Bert: Only Attention is Needed for Learning Graph Representations
• HDMI: High-order Deep Multiplex Infomax
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• Graph Barlow Twins: A self-supervised representation learning framework for graphs
• How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision
• Self-supervised Learning on Graphs: Deep Insights and New Direction
• Graph Contrastive Learning Automated
• GPT-GNN: Generative Pre-Training of Graph Neural Networks
• Contrastive Learning of Structured World Models
• When Does Self-Supervision Help Graph Convolutional Networks?
• Self-supervised Consensus Representation Learning for Attributed Graph
• Graph-based, Self-Supervised Program Repair from Diagnostic Feedback
• Graph InfoClust: Leveraging cluster-level node information for unsupervised graph representation learning
• Contrastive Learning for Recommender System
• Self-Supervised Learning of Graph Neural Networks: A Unified Review
• Contrastive Learning with Stronger Augmentations
• Self-supervised Graph-level Representation Learning with Local and Global Structure
• Towards Robust Graph Contrastive Learning
• Self-supervised Training of Graph Convolutional Networks
• Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization
• GraphCL: Contrastive Self-Supervised Learning of Graph Representations
• SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism
• Pre-Training Graph Neural Networks for Generic Structural Feature Extraction
• Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning
• SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks
• CAGNN: Cluster-Aware Graph Neural Networks for Unsupervised Graph Representation Learning
• Spatio-Temporal Deep Graph Infomax
• Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning
• FedGL: Federated Graph Learning Framework with Global Self-Supervision
• Self-supervised Graph Neural Networks without explicit negative sampling
• Iterative Graph Self-Distillation
• Knowledge-aware Contrastive Molecular Graph Learning
• Contrastive Self-supervised Learning for Graph Classification
• Self-supervised edge features for improved Graph Neural Network training
• Calibrating and Improving Graph Contrastive Learning
• Hop-Count Based Self-Supervised Anomaly Detection on Attributed Networks
• Self-Supervised Deep Graph Embedding with High-Order Information Fusion for Community Discovery
• Maximizing Mutual Information Across Feature and Topology Views for Learning Graph Representations

Cited by in corpus (50)

• Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning
• Self-supervised learning methods and applications in medical imaging analysis: A survey
• Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection
• Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs
• Pre-training Molecular Graph Representation with 3D Geometry
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• Trustworthy Graph Neural Networks: Aspects, Methods and Trends
• Current and future directions in network biology
• Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning
• SelfCF: A Simple Framework for Self-supervised Collaborative Filtering
• Self-Supervised Anomaly Detection in Computer Vision and Beyond: A Survey and Outlook
• Augmentation-Free Graph Contrastive Learning of Invariant-Discriminative Representations
• Graph Communal Contrastive Learning
• Jointly Contrastive Representation Learning on Road Network and Trajectory
• Beyond Low-pass Filtering: Graph Convolutional Networks with Automatic Filtering
• HetGPT: Harnessing the Power of Prompt Tuning in Pre-Trained Heterogeneous Graph Neural Networks
• Combining Intra-Risk and Contagion Risk for Enterprise Bankruptcy Prediction Using Graph Neural Networks
• Graph Neural Networks: Methods, Applications, and Opportunities
• Graph Foundation Models: Concepts, Opportunities and Challenges
• Exploring Edge Disentanglement for Node Classification
• E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation
• Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentials
• X-GOAL: Multiplex Heterogeneous Graph Prototypical Contrastive Learning
• Anomaly Detection Based on Isolation Mechanisms: A Survey
• A Comprehensive Survey on Self-Supervised Learning for Recommendation
• RevGNN: Negative Sampling Enhanced Contrastive Graph Learning for Academic Reviewer Recommendation
• Towards Graph Self-Supervised Learning with Contrastive Adjusted Zooming
• A Network Science perspective of Graph Convolutional Networks: A survey
• Hierarchical Aggregations for High-Dimensional Multiplex Graph Embedding
• Reliable Node Similarity Matrix Guided Contrastive Graph Clustering
• Disambiguated Node Classification with Graph Neural Networks
• Semi-supervised Domain Adaptation on Graphs with Contrastive Learning and Minimax Entropy
• CVTGAD: Simplified Transformer with Cross-View Attention for Unsupervised Graph-level Anomaly Detection
• EasyDGL: Encode, Train and Interpret for Continuous-time Dynamic Graph Learning
• Unsupervised Multiplex Graph Learning with Complementary and Consistent Information
• A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective
• Voucher Abuse Detection with Prompt-based Fine-tuning on Graph Neural Networks
• Balancing Graph Embedding Smoothness in Self-Supervised Learning via Information-Theoretic Decomposition
• A Review on Self-Supervised Learning for Time Series Anomaly Detection: Recent Advances and Open Challenges
• Negative-Free Self-Supervised Gaussian Embedding of Graphs
• InfoNCE is a Free Lunch for Semantically guided Graph Contrastive Learning
• An Empirical Study of Graph Contrastive Learning
• Whole-Graph Representation Learning For the Classification of Signed Networks
• Str-GCL: Structural Commonsense Driven Graph Contrastive Learning
• Hierarchical Vector Quantized Graph Autoencoder with Annealing-Based Code Selection
• Evaluating Modules in Graph Contrastive Learning
• Dynamic Cluster Analysis to Detect and Track Novelty in Network Telescopes
• Learning Feature Encoder with Synthetic Anomalies for Weakly Supervised Graph Anomaly Detection
• Self-Supervised Contrastive Graph Clustering Network via Structural Information Fusion
• Gradient scarcity with Bilevel Optimization for Graph Learning