SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation

Abstract

Graph contrastive learning (GCL) has emerged as a dominant technique for graph representation learning which maximizes the mutual information between paired graph augmentations that share the same semantics. Unfortunately, it is difficult to preserve semantics well during augmentations in view of the diverse nature of graph data. Currently, data augmentations in GCL that are designed to preserve semantics broadly fall into three unsatisfactory ways. First, the augmentations can be manually picked per dataset by trial-and-errors. Second, the augmentations can be selected via cumbersome search. Third, the augmentations can be obtained by introducing expensive domain-specific knowledge as guidance. All of these limit the efficiency and more general applicability of existing GCL methods. To circumvent these crucial issues, we propose a \underline{Sim}ple framework for \underline{GRA}ph \underline{C}ontrastive l\underline{E}arning, \textbf{SimGRACE} for brevity, which does not require data augmentations. Specifically, we take original graph as input and GNN model with its perturbed version as two encoders to obtain two correlated views for contrast. SimGRACE is inspired by the observation that graph data can preserve their semantics well during encoder perturbations while not requiring manual trial-and-errors, cumbersome search or expensive domain knowledge for augmentations selection. Also, we explain why SimGRACE can succeed. Furthermore, we devise adversarial training scheme, dubbed \textbf{AT-SimGRACE}, to enhance the robustness of graph contrastive learning and theoretically explain the reasons. Albeit simple, we show that SimGRACE can yield competitive or better performance compared with state-of-the-art methods in terms of generalizability, transferability and robustness, while enjoying unprecedented degree of flexibility and efficiency.

Accepted by The Web Conference 2022 (WWW 2022)

References in corpus (11)

• graph2vec: Learning Distributed Representations of Graphs
• Deep Graph Contrastive Representation Learning
• Contrastive Multi-View Representation Learning on Graphs
• TUDataset: A collection of benchmark datasets for learning with graphs
• Exploring Generalization in Deep Learning
• Co-learning: Learning from Noisy Labels with Self-supervision
• Graph Contrastive Learning Automated
• GPT-GNN: Generative Pre-Training of Graph Neural Networks
• Towards Robust Graph Contrastive Learning
• A Survey of Pretraining on Graphs: Taxonomy, Methods, and Applications
• Understanding Adversarial Robustness Through Loss Landscape Geometries

Cited by in corpus (15)

• A Comprehensive Survey on Deep Graph Representation Learning
• Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure
• Automated Spatio-Temporal Graph Contrastive Learning
• RecDCL: Dual Contrastive Learning for Recommendation
• All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretraining
• CLDG: Contrastive Learning on Dynamic Graphs
• GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learning
• Unsupervised Social Event Detection via Hybrid Graph Contrastive Learning and Reinforced Incremental Clustering
• Towards Robust Recommendation via Decision Boundary-aware Graph Contrastive Learning
• Graph Contrastive Learning with Cohesive Subgraph Awareness
• Endowing Pre-trained Graph Models with Provable Fairness
• A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective
• Handling Feature Heterogeneity with Learnable Graph Patches
• Generative Data Augmentation in Graph Contrastive Learning for Recommendation
• Data-Driven Self-Supervised Graph Representation Learning