Data-Driven Self-Supervised Graph Representation Learning

Abstract

Self-supervised graph representation learning (SSGRL) is a representation learning paradigm used to reduce or avoid manual labeling. An essential part of SSGRL is graph data augmentation. Existing methods usually rely on heuristics commonly identified through trial and error and are effective only within some application domains. Also, it is not clear why one heuristic is better than another. Moreover, recent studies have argued against some techniques (e.g., dropout: that can change the properties of molecular graphs or destroy relevant signals for graph-based document classification tasks). In this study, we propose a novel data-driven SSGRL approach that automatically learns a suitable graph augmentation from the signal encoded in the graph (i.e., the nodes' predictive feature and topological information). We propose two complementary approaches that produce learnable feature and topological augmentations. The former learns multi-view augmentation of node features, and the latter learns a high-order view of the topology. Moreover, the augmentations are jointly learned with the representation. Our approach is general that it can be applied to homogeneous and heterogeneous graphs. We perform extensive experiments on node classification (using nine homogeneous and heterogeneous datasets) and graph property prediction (using another eight datasets). The results show that the proposed method matches or outperforms the SOTA SSGRL baselines and performs similarly to semi-supervised methods. The anonymised source code is available at https://github.com/AhmedESamy/dsgrl/

References in corpus (21)

• Semi-Supervised Classification with Graph Convolutional Networks
• LINE: Large-scale Information Network Embedding
• Bootstrap your own latent: A new approach to self-supervised Learning
• Graph Contrastive Learning with Adaptive Augmentation
• Graph Contrastive Learning with Augmentations
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• Exploring Simple Siamese Representation Learning
• Deep Graph Contrastive Representation Learning
• Contrastive Multi-View Representation Learning on Graphs
• TUDataset: A collection of benchmark datasets for learning with graphs
• SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation
• VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning
• HDMI: High-order Deep Multiplex Infomax
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• Graph Barlow Twins: A self-supervised representation learning framework for graphs
• Meta-Path Guided Embedding for Similarity Search in Large-Scale Heterogeneous Information Networks
• Graph Contrastive Learning Automated
• Heterogeneous Information Network Embedding for Recommendation
• Self-Supervised Learning of Graph Neural Networks: A Unified Review
• Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning
• Self-supervised Graph Neural Networks without explicit negative sampling