Graph Self-Supervised Learning with Learnable Structural and Positional Encodings

Abstract

Traditional Graph Self-Supervised Learning (GSSL) struggles to capture complex structural properties well. This limitation stems from two main factors: (1) the inadequacy of conventional Graph Neural Networks (GNNs) in representing sophisticated topological features, and (2) the focus of self-supervised learning solely on final graph representations. To address these issues, we introduce \emph{GenHopNet}, a GNN framework that integrates a $k$-hop message-passing scheme, enhancing its ability to capture local structural information without explicit substructure extraction. We theoretically demonstrate that \emph{GenHopNet} surpasses the expressiveness of the classical Weisfeiler-Lehman (WL) test for graph isomorphism. Furthermore, we propose a structural- and positional-aware GSSL framework that incorporates topological information throughout the learning process. This approach enables the learning of representations that are both sensitive to graph topology and invariant to specific structural and feature augmentations. Comprehensive experiments on graph classification datasets, including those designed to test structural sensitivity, show that our method consistently outperforms the existing approaches and maintains computational efficiency. Our work significantly advances GSSL's capability in distinguishing graphs with similar local structures but different global topologies.

This paper is accepted by The World Wide Web Conference (WWW) 2025

References in corpus (46)

• A Comprehensive Survey on Graph Neural Networks
• Semi-Supervised Classification with Graph Convolutional Networks
• A Simple Framework for Contrastive Learning of Visual Representations
• Inductive Representation Learning on Large Graphs
• Graph Contrastive Learning with Adaptive Augmentation
• Learning Convolutional Neural Networks for Graphs
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• Contrastive Multi-View Representation Learning on Graphs
• Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting
• Representation Learning on Graphs: Methods and Applications
• Recipe for a General, Powerful, Scalable Graph Transformer
• InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization
• On Valid Optimal Assignment Kernels and Applications to Graph Classification
• COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning
• Measuring and Improving the Use of Graph Information in Graph Neural Networks
• Power Normalizations in Fine-grained Image, Few-shot Image and Graph Classification
• Structure-Aware Transformer for Graph Representation Learning
• Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks
• How Powerful are K-hop Message Passing Graph Neural Networks
• RetGK: Graph Kernels based on Return Probabilities of Random Walks
• Nested Graph Neural Networks
• InfoGCL: Information-Aware Graph Contrastive Learning
• Large-Scale Representation Learning on Graphs via Bootstrapping
• Graph Neural Networks with Learnable Structural and Positional Representations
• Contrastive Laplacian Eigenmaps
• Let Invariant Rationale Discovery Inspire Graph Contrastive Learning
• Expressive Power of Invariant and Equivariant Graph Neural Networks
• Graph-adaptive Rectified Linear Unit for Graph Neural Networks
• Building powerful and equivariant graph neural networks with structural message-passing
• REFINE: Random RangE FInder for Network Embedding
• Spectral Augmentation for Self-Supervised Learning on Graphs
• Graph Inductive Biases in Transformers without Message Passing
• Specformer: Spectral Graph Neural Networks Meet Transformers
• MetAug: Contrastive Learning via Meta Feature Augmentation
• Equivariant Subgraph Aggregation Networks
• MetaMask: Revisiting Dimensional Confounder for Self-Supervised Learning
• Fisher-Bures Adversary Graph Convolutional Networks
• Interventional Contrastive Learning with Meta Semantic Regularizer
• Understanding and Mitigating Hyperbolic Dimensional Collapse in Graph Contrastive Learning
• Reconstruction for Powerful Graph Representations
• Path Neural Networks: Expressive and Accurate Graph Neural Networks
• Message Passing Neural Networks for Traffic Forecasting
• Boosting the Cycle Counting Power of Graph Neural Networks with I${}^2$-GNNs
• Efficient Subgraph GNNs by Learning Effective Selection Policies
• Learnable Expansion of Graph Operators for Multi-Modal Feature Fusion