Data-Driven Self-Supervised Graph Representation Learning

Abstract

Self-supervised graph representation learning (SSGRL) is a representation learning paradigm used to reduce or avoid manual labeling. An essential part of SSGRL is graph data augmentation. Existing methods usually rely on heuristics commonly identified through trial and error and are effective only within some application domains. Also, it is not clear why one heuristic is better than another. Moreover, recent studies have argued against some techniques (e.g., dropout: that can change the properties of molecular graphs or destroy relevant signals for graph-based document classification tasks). In this study, we propose a novel data-driven SSGRL approach that automatically learns a suitable graph augmentation from the signal encoded in the graph (i.e., the nodes' predictive feature and topological information). We propose two complementary approaches that produce learnable feature and topological augmentations. The former learns multi-view augmentation of node features, and the latter learns a high-order view of the topology. Moreover, the augmentations are jointly learned with the representation. Our approach is general that it can be applied to homogeneous and heterogeneous graphs. We perform extensive experiments on node classification (using nine homogeneous and heterogeneous datasets) and graph property prediction (using another eight datasets). The results show that the proposed method matches or outperforms the SOTA SSGRL baselines and performs similarly to semi-supervised methods. The anonymised source code is available at https://github.com/AhmedESamy/dsgrl/

References in corpus (34)

• DeepWalk: Online Learning of Social Representations
• Semi-Supervised Classification with Graph Convolutional Networks
• LINE: Large-scale Information Network Embedding
• Inductive Representation Learning on Large Graphs
• Bootstrap your own latent: A new approach to self-supervised Learning
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Graph Contrastive Learning with Adaptive Augmentation
• Momentum Contrast for Unsupervised Visual Representation Learning
• MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding
• ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
• Graph Contrastive Learning with Augmentations
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• Pitfalls of Graph Neural Network Evaluation
• Exploring Simple Siamese Representation Learning
• Deep Graph Contrastive Representation Learning
• Contrastive Multi-View Representation Learning on Graphs
• GraphSAINT: Graph Sampling Based Inductive Learning Method
• TUDataset: A collection of benchmark datasets for learning with graphs
• SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation
• VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning
• Diffusion Improves Graph Learning
• HDMI: High-order Deep Multiplex Infomax
• InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• Graph Barlow Twins: A self-supervised representation learning framework for graphs
• Meta-Path Guided Embedding for Similarity Search in Large-Scale Heterogeneous Information Networks
• A Fair Comparison of Graph Neural Networks for Graph Classification
• Graph Contrastive Learning Automated
• Heterogeneous Information Network Embedding for Recommendation
• Heterogeneous Deep Graph Infomax
• Self-Supervised Learning of Graph Neural Networks: A Unified Review
• Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning
• Unsupervised Attributed Multiplex Network Embedding
• Self-supervised Graph Neural Networks without explicit negative sampling