Self-supervised Graph Neural Networks without explicit negative sampling
arXiv:2103.14958
Abstract
Real world data is mostly unlabeled or only few instances are labeled. Manually labeling data is a very expensive and daunting task. This calls for unsupervised learning techniques that are powerful enough to achieve comparable results as semi-supervised/supervised techniques. Contrastive self-supervised learning has emerged as a powerful direction, in some cases outperforming supervised techniques. In this study, we propose, SelfGNN, a novel contrastive self-supervised graph neural network (GNN) without relying on explicit contrastive terms. We leverage Batch Normalization, which introduces implicit contrastive terms, without sacrificing performance. Furthermore, as data augmentation is key in contrastive learning, we introduce four feature augmentation (FA) techniques for graphs. Though graph topological augmentation (TA) is commonly used, our empirical findings show that FA perform as good as TA. Moreover, FA incurs no computational overhead, unlike TA, which often has O(N^3) time complexity, N-number of nodes. Our empirical evaluation on seven publicly available real-world data shows that, SelfGNN is powerful and leads to a performance comparable with SOTA supervised GNNs and always better than SOTA semi-supervised and unsupervised GNNs. The source code is available at https://github.com/zekarias-tilahun/SelfGNN.
arXiv commentAccepted in The International Workshop on Self-Supervised Learning for the Web (SSL'21) @ the Web Conference 2021 (WWW'21)
References in corpus (13)
- Semi-Supervised Classification with Graph Convolutional Networks
- Bootstrap your own latent: A new approach to self-supervised Learning
- Language Models are Few-Shot Learners
- Pitfalls of Graph Neural Network Evaluation
- FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling
- Exploring Simple Siamese Representation Learning
- Contrastive Multi-View Representation Learning on Graphs
- Strategies for Pre-training Graph Neural Networks
- Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks
- A simple yet effective baseline for non-attributed graph classification
- Understanding Self-supervised Learning with Dual Deep Networks
- BYOL works even without batch statistics
- Multi-Stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labels