Graph Contrastive Learning with Augmentations

Abstract

Generalizable, transferrable, and robust representation learning on graph-structured data remains a challenge for current graph neural networks (GNNs). Unlike what has been developed for convolutional neural networks (CNNs) for image data, self-supervised learning and pre-training are less explored for GNNs. In this paper, we propose a graph contrastive learning (GraphCL) framework for learning unsupervised representations of graph data. We first design four types of graph augmentations to incorporate various priors. We then systematically study the impact of various combinations of graph augmentations on multiple datasets, in four different settings: semi-supervised, unsupervised, and transfer learning as well as adversarial attacks. The results show that, even without tuning augmentation extents nor using sophisticated GNN architectures, our GraphCL framework can produce graph representations of similar or better generalizability, transferrability, and robustness compared to state-of-the-art methods. We also investigate the impact of parameterized graph augmentation extents and patterns, and observe further performance gains in preliminary experiments. Our codes are available at https://github.com/Shen-Lab/GraphCL.

Supplementary materials are available at https://yyou1996.github.io/files/neurips2020_graphcl_supplement.pdf. NeurIPS 2020

References in corpus (30)

• Semi-Supervised Classification with Graph Convolutional Networks
• A Simple Framework for Contrastive Learning of Visual Representations
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Learning deep representations by mutual information estimation and maximization
• Unsupervised Data Augmentation for Consistency Training
• struc2vec: Learning Node Representations from Structural Identity
• Adversarial Attacks on Neural Networks for Graph Data
• Variational Graph Auto-Encoders
• graph2vec: Learning Distributed Representations of Graphs
• Towards Deeper Graph Neural Networks
• Discriminative Embeddings of Latent Variable Models for Structured Data
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• Contrastive Multi-View Representation Learning on Graphs
• How Powerful are Graph Neural Networks?
• TUDataset: A collection of benchmark datasets for learning with graphs
• Benchmarking Graph Neural Networks
• Adversarial Attack on Graph Structured Data
• Graph-Bert: Only Attention is Needed for Learning Graph Representations
• InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization
• Self-supervised Learning on Graphs: Deep Insights and New Direction
• Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks
• Selfie: Self-supervised Pretraining for Image Embedding
• Revisiting Self-Supervised Visual Representation Learning
• Heterogeneous Deep Graph Infomax
• When Does Self-Supervision Help Graph Convolutional Networks?
• Are Powerful Graph Neural Nets Necessary? A Dissection on Graph Classification
• Batch Virtual Adversarial Training for Graph Convolutional Networks
• Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning
• Self-supervised Training of Graph Convolutional Networks

Cited by in corpus (57)

• Molecular Contrastive Learning of Representations via Graph Neural Networks
• Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning
• Graph Self-Supervised Learning: A Survey
• Knowledge Graph Contrastive Learning for Recommendation
• SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation
• Heterogeneous Graph Contrastive Learning for Recommendation
• Contrastive Meta Learning with Behavior Multiplicity for Recommendation
• A Review-aware Graph Contrastive Learning Framework for Recommendation
• Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure
• COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning
• Graph Rationalization with Environment-based Augmentations
• Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrast
• Automated Spatio-Temporal Graph Contrastive Learning
• KPGT: Knowledge-Guided Pre-training of Graph Transformer for Molecular Property Prediction
• GOOD-D: On Unsupervised Graph Out-Of-Distribution Detection
• Dual Space Graph Contrastive Learning
• Graph Communal Contrastive Learning
• SSLRec: A Self-Supervised Learning Framework for Recommendation
• Multi-level Contrastive Learning Framework for Sequential Recommendation
• Mask and Reason: Pre-Training Knowledge Graph Transformers for Complex Logical Queries
• Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentations
• Propagation Tree Is Not Deep: Adaptive Graph Contrastive Learning Approach for Rumor Detection
• Graph Contrastive Learning with Generative Adversarial Network
• Contrastive Graph Convolutional Networks for Hardware Trojan Detection in Third Party IP Cores
• KGNN: Harnessing Kernel-based Networks for Semi-supervised Graph Classification
• A Contrastive Variational Graph Auto-Encoder for Node Clustering
• X-GOAL: Multiplex Heterogeneous Graph Prototypical Contrastive Learning
• Unsupervised Social Event Detection via Hybrid Graph Contrastive Learning and Reinforced Incremental Clustering
• CLDG: Contrastive Learning on Dynamic Graphs
• Causal invariant geographic network representations with feature and structural distribution shifts
• Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph
• Towards Mitigating Dimensional Collapse of Representations in Collaborative Filtering
• RPT: Toward Transferable Model on Heterogeneous Researcher Data via Pre-Training
• Modeling Multiple Views via Implicitly Preserving Global Consistency and Local Complementarity
• Dismantling Complex Networks by a Neural Model Trained from Tiny Networks
• Ego-Vehicle Action Recognition based on Semi-Supervised Contrastive Learning
• Learning Robust Representation through Graph Adversarial Contrastive Learning
• LightKG: Efficient Knowledge-Aware Recommendations with Simplified GNN Architecture
• Heterogeneous Social Event Detection via Hyperbolic Graph Representations
• Robust Basket Recommendation via Noise-tolerated Graph Contrastive Learning
• CARL-G: Clustering-Accelerated Representation Learning on Graphs
• TinyKG: Memory-Efficient Training Framework for Knowledge Graph Neural Recommender Systems
• Graph Information Bottleneck for Remote Sensing Segmentation
• Self-supervised Representation Learning on Electronic Health Records with Graph Kernel Infomax
• Joint Data and Feature Augmentation for Self-Supervised Representation Learning on Point Clouds
• Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networks
• Towards Deviation-Robust Agent Navigation via Perturbation-Aware Contrastive Learning
• A Self-supervised Method for Entity Alignment
• VGA: Vision and Graph Fused Attention Network for Rumor Detection
• Exploring Global Information for Session-based Recommendation
• Deep Contrastive Multiview Network Embedding
• Self-supervised Representations and Node Embedding Graph Neural Networks for Accurate and Multi-scale Analysis of Materials
• PreGIP: Watermarking the Pretraining of Graph Neural Networks for Deep Intellectual Property Protection
• Towards Better Modeling with Missing Data: A Contrastive Learning-based Visual Analytics Perspective
• How to Use Graph Data in the Wild to Help Graph Anomaly Detection?
• Imbalanced Graph-Level Anomaly Detection via Counterfactual Augmentation and Feature Learning
• Data-Driven Self-Supervised Graph Representation Learning