Graph Barlow Twins: A self-supervised representation learning framework for graphs

Abstract

The self-supervised learning (SSL) paradigm is an essential exploration area, which tries to eliminate the need for expensive data labeling. Despite the great success of SSL methods in computer vision and natural language processing, most of them employ contrastive learning objectives that require negative samples, which are hard to define. This becomes even more challenging in the case of graphs and is a bottleneck for achieving robust representations. To overcome such limitations, we propose a framework for self-supervised graph representation learning - Graph Barlow Twins, which utilizes a cross-correlation-based loss function instead of negative samples. Moreover, it does not rely on non-symmetric neural network architectures - in contrast to state-of-the-art self-supervised graph representation learning method BGRL. We show that our method achieves as competitive results as the best self-supervised methods and fully supervised ones while requiring fewer hyperparameters and substantially shorter computation time (ca. 30 times faster than BGRL).

References in corpus (9)

• Bootstrap your own latent: A new approach to self-supervised Learning
• Fast Graph Representation Learning with PyTorch Geometric
• Variational Graph Auto-Encoders
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• Predicting multicellular function through multi-layer tissue networks
• Deep Graph Contrastive Representation Learning
• Exploring Simple Siamese Representation Learning
• A Note on Connecting Barlow Twins with Negative-Sample-Free Contrastive Learning
• Self-supervised Graph Representation Learning via Bootstrapping

Cited by in corpus (11)

• Graph Self-Supervised Learning: A Survey
• Large-Scale Representation Learning on Graphs via Bootstrapping
• CARL-G: Clustering-Accelerated Representation Learning on Graphs
• A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective
• InfoNCE is a Free Lunch for Semantically guided Graph Contrastive Learning
• Negative-Free Self-Supervised Gaussian Embedding of Graphs
• Str-GCL: Structural Commonsense Driven Graph Contrastive Learning
• An Empirical Study of Graph Contrastive Learning
• SIGNL: A Label-Efficient Audio Deepfake Detection System via Spectral-Temporal Graph Non-Contrastive Learning
• Jointly Learnable Data Augmentations for Self-Supervised GNNs
• Data-Driven Self-Supervised Graph Representation Learning