Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning

Abstract

Graph representation learning has attracted lots of attention recently. Existing graph neural networks fed with the complete graph data are not scalable due to limited computation and memory costs. Thus, it remains a great challenge to capture rich information in large-scale graph data. Besides, these methods mainly focus on supervised learning and highly depend on node label information, which is expensive to obtain in the real world. As to unsupervised network embedding approaches, they overemphasize node proximity instead, whose learned representations can hardly be used in downstream application tasks directly. In recent years, emerging self-supervised learning provides a potential solution to address the aforementioned problems. However, existing self-supervised works also operate on the complete graph data and are biased to fit either global or very local (1-hop neighborhood) graph structures in defining the mutual information based loss terms. In this paper, a novel self-supervised representation learning method via Subgraph Contrast, namely \textsc{Subg-Con}, is proposed by utilizing the strong correlation between central nodes and their sampled subgraphs to capture regional structure information. Instead of learning on the complete input graph data, with a novel data augmentation strategy, \textsc{Subg-Con} learns node representations through a contrastive loss defined based on subgraphs sampled from the original graph instead. Compared with existing graph representation learning approaches, \textsc{Subg-Con} has prominent performance advantages in weaker supervision requirements, model learning scalability, and parallelization. Extensive experiments verify both the effectiveness and the efficiency of our work compared with both classic and state-of-the-art graph representation learning approaches on multiple real-world large-scale benchmark datasets from different domains.

arXiv commentBest Student Paper Runner-up Award at ICDM 2020

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• Efficient Estimation of Word Representations in Vector Space
• Semi-Supervised Classification with Graph Convolutional Networks
• A Simple Framework for Contrastive Learning of Visual Representations
• BPR: Bayesian Personalized Ranking from Implicit Feedback
• Fast Graph Representation Learning with PyTorch Geometric
• Simplifying Graph Convolutional Networks
• Variational Graph Auto-Encoders
• FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling
• How Powerful are Graph Neural Networks?
• GraphSAINT: Graph Sampling Based Inductive Learning Method
• MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing
• LanczosNet: Multi-Scale Deep Graph Convolutional Networks
• GMNN: Graph Markov Neural Networks
• Graph-Bert: Only Attention is Needed for Learning Graph Representations
• A critical analysis of self-supervision, or what we can learn from a single image
• GResNet: Graph Residual Network for Reviving Deep GNNs from Suspended Animation

Cited by in corpus (8)

• Graph Self-Supervised Learning: A Survey
• Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• Contrastive Self-supervised Sequential Recommendation with Robust Augmentation
• RPT: Toward Transferable Model on Heterogeneous Researcher Data via Pre-Training
• Quantifying and Mitigating Privacy Risks of Contrastive Learning
• Few-Shot Electronic Health Record Coding through Graph Contrastive Learning
• Social Influence Prediction with Train and Test Time Augmentation for Graph Neural Networks