Graph Self-Supervised Learning: A Survey

Abstract

Deep learning on graphs has attracted significant interests recently. However, most of the works have focused on (semi-) supervised learning, resulting in shortcomings including heavy label reliance, poor generalization, and weak robustness. To address these issues, self-supervised learning (SSL), which extracts informative knowledge through well-designed pretext tasks without relying on manual labels, has become a promising and trending learning paradigm for graph data. Different from SSL on other domains like computer vision and natural language processing, SSL on graphs has an exclusive background, design ideas, and taxonomies. Under the umbrella of graph self-supervised learning, we present a timely and comprehensive review of the existing approaches which employ SSL techniques for graph data. We construct a unified framework that mathematically formalizes the paradigm of graph SSL. According to the objectives of pretext tasks, we divide these approaches into four categories: generation-based, auxiliary property-based, contrast-based, and hybrid approaches. We further describe the applications of graph SSL across various research fields and summarize the commonly used datasets, evaluation benchmark, performance comparison and open-source codes of graph SSL. Finally, we discuss the remaining challenges and potential future directions in this research field.

arXiv comment26 pages, 9 figures, 9 tables

References in corpus (77)

• Efficient Estimation of Word Representations in Vector Space
• Distributed Representations of Words and Phrases and their Compositionality
• Auto-Encoding Variational Bayes
• A Comprehensive Survey on Graph Neural Networks
• DeepWalk: Online Learning of Social Representations
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• LINE: Large-scale Information Network Embedding
• Inductive Representation Learning on Large Graphs
• Bootstrap your own latent: A new approach to self-supervised Learning
• A Survey on Knowledge Graphs: Representation, Acquisition and Applications
• Learning deep representations by mutual information estimation and maximization
• Graph Contrastive Learning with Adaptive Augmentation
• Self-supervised Learning: Generative or Contrastive
• Variational Graph Auto-Encoders
• Graph Contrastive Learning with Augmentations
• Hierarchical Graph Representation Learning with Differentiable Pooling
• GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• DropEdge: Towards Deep Graph Convolutional Networks on Node Classification
• Pitfalls of Graph Neural Network Evaluation
• Predicting multicellular function through multi-layer tissue networks
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning
• Deep Clustering for Unsupervised Learning of Visual Features
• Self-Supervised Graph Transformer on Large-Scale Molecular Data
• Deep Graph Contrastive Representation Learning
• Contrastive Multi-View Representation Learning on Graphs
• Constrained Graph Variational Autoencoders for Molecule Design
• On Mutual Information Maximization for Representation Learning
• Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection
• Generative Image Inpainting with Contextual Attention
• Multi-view Graph Contrastive Representation Learning for Drug-Drug Interaction Prediction
• Strategies for Pre-training Graph Neural Networks
• Diffusion Improves Graph Learning
• Graph-Bert: Only Attention is Needed for Learning Graph Representations
• HDMI: High-order Deep Multiplex Infomax
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• Graph Barlow Twins: A self-supervised representation learning framework for graphs
• How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision
• Self-supervised Learning on Graphs: Deep Insights and New Direction
• Graph Contrastive Learning Automated
• GPT-GNN: Generative Pre-Training of Graph Neural Networks
• Contrastive Learning of Structured World Models
• When Does Self-Supervision Help Graph Convolutional Networks?
• Heterogeneous Deep Graph Infomax
• Semi-Implicit Graph Variational Auto-Encoders
• Self-supervised Consensus Representation Learning for Attributed Graph
• Graph-based, Self-Supervised Program Repair from Diagnostic Feedback
• Graph InfoClust: Leveraging cluster-level node information for unsupervised graph representation learning
• Contrastive Learning for Recommender System
• Self-supervised Auxiliary Learning with Meta-paths for Heterogeneous Graphs
• Self-Supervised Learning of Graph Neural Networks: A Unified Review
• Contrastive Learning with Stronger Augmentations
• Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networks
• Towards Robust Graph Contrastive Learning
• Self-supervised Graph-level Representation Learning with Local and Global Structure
• Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization
• Self-supervised Training of Graph Convolutional Networks
• SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism
• GraphCL: Contrastive Self-Supervised Learning of Graph Representations
• CAGNN: Cluster-Aware Graph Neural Networks for Unsupervised Graph Representation Learning
• Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning
• Pre-Training Graph Neural Networks for Generic Structural Feature Extraction
• SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks
• Spatio-Temporal Deep Graph Infomax
• Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning
• FedGL: Federated Graph Learning Framework with Global Self-Supervision
• Knowledge-aware Contrastive Molecular Graph Learning
• Iterative Graph Self-Distillation
• Self-supervised Graph Neural Networks without explicit negative sampling
• Contrastive Self-supervised Learning for Graph Classification
• Self-supervised edge features for improved Graph Neural Network training
• Calibrating and Improving Graph Contrastive Learning
• Hop-Count Based Self-Supervised Anomaly Detection on Attributed Networks
• Self-Supervised Deep Graph Embedding with High-Order Information Fusion for Community Discovery
• Maximizing Mutual Information Across Feature and Topology Views for Learning Graph Representations

Cited by in corpus (50)

• Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning
• Self-supervised learning methods and applications in medical imaging analysis: A survey
• Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection
• Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs
• Pre-training Molecular Graph Representation with 3D Geometry
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• Trustworthy Graph Neural Networks: Aspects, Methods and Trends
• Current and future directions in network biology
• Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning
• SelfCF: A Simple Framework for Self-supervised Collaborative Filtering
• Self-Supervised Anomaly Detection in Computer Vision and Beyond: A Survey and Outlook
• Augmentation-Free Graph Contrastive Learning of Invariant-Discriminative Representations
• Graph Communal Contrastive Learning
• Jointly Contrastive Representation Learning on Road Network and Trajectory
• Beyond Low-pass Filtering: Graph Convolutional Networks with Automatic Filtering
• HetGPT: Harnessing the Power of Prompt Tuning in Pre-Trained Heterogeneous Graph Neural Networks
• Combining Intra-Risk and Contagion Risk for Enterprise Bankruptcy Prediction Using Graph Neural Networks
• Graph Neural Networks: Methods, Applications, and Opportunities
• Graph Foundation Models: Concepts, Opportunities and Challenges
• Exploring Edge Disentanglement for Node Classification
• E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation
• Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentials
• X-GOAL: Multiplex Heterogeneous Graph Prototypical Contrastive Learning
• Anomaly Detection Based on Isolation Mechanisms: A Survey
• A Comprehensive Survey on Self-Supervised Learning for Recommendation
• RevGNN: Negative Sampling Enhanced Contrastive Graph Learning for Academic Reviewer Recommendation
• Towards Graph Self-Supervised Learning with Contrastive Adjusted Zooming
• A Network Science perspective of Graph Convolutional Networks: A survey
• Hierarchical Aggregations for High-Dimensional Multiplex Graph Embedding
• Reliable Node Similarity Matrix Guided Contrastive Graph Clustering
• Disambiguated Node Classification with Graph Neural Networks
• Semi-supervised Domain Adaptation on Graphs with Contrastive Learning and Minimax Entropy
• CVTGAD: Simplified Transformer with Cross-View Attention for Unsupervised Graph-level Anomaly Detection
• EasyDGL: Encode, Train and Interpret for Continuous-time Dynamic Graph Learning
• Unsupervised Multiplex Graph Learning with Complementary and Consistent Information
• A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective
• Voucher Abuse Detection with Prompt-based Fine-tuning on Graph Neural Networks
• Balancing Graph Embedding Smoothness in Self-Supervised Learning via Information-Theoretic Decomposition
• A Review on Self-Supervised Learning for Time Series Anomaly Detection: Recent Advances and Open Challenges
• Negative-Free Self-Supervised Gaussian Embedding of Graphs
• InfoNCE is a Free Lunch for Semantically guided Graph Contrastive Learning
• An Empirical Study of Graph Contrastive Learning
• Whole-Graph Representation Learning For the Classification of Signed Networks
• Str-GCL: Structural Commonsense Driven Graph Contrastive Learning
• Hierarchical Vector Quantized Graph Autoencoder with Annealing-Based Code Selection
• Evaluating Modules in Graph Contrastive Learning
• Dynamic Cluster Analysis to Detect and Track Novelty in Network Telescopes
• Learning Feature Encoder with Synthetic Anomalies for Weakly Supervised Graph Anomaly Detection
• Self-Supervised Contrastive Graph Clustering Network via Structural Information Fusion
• Gradient scarcity with Bilevel Optimization for Graph Learning