Deep Graph Contrastive Representation Learning

Abstract

Graph representation learning nowadays becomes fundamental in analyzing graph-structured data. Inspired by recent success of contrastive methods, in this paper, we propose a novel framework for unsupervised graph representation learning by leveraging a contrastive objective at the node level. Specifically, we generate two graph views by corruption and learn node representations by maximizing the agreement of node representations in these two views. To provide diverse node contexts for the contrastive objective, we propose a hybrid scheme for generating graph views on both structure and attribute levels. Besides, we provide theoretical justification behind our motivation from two perspectives, mutual information and the classical triplet loss. We perform empirical experiments on both transductive and inductive learning tasks using a variety of real-world datasets. Experimental experiments demonstrate that despite its simplicity, our proposed method consistently outperforms existing state-of-the-art methods by large margins. Moreover, our unsupervised method even surpasses its supervised counterparts on transductive tasks, demonstrating its great potential in real-world applications.

Work in progress; updated experiments

References in corpus (15)

• Distributed Representations of Words and Phrases and their Compositionality
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• A Simple Framework for Contrastive Learning of Visual Representations
• Learning deep representations by mutual information estimation and maximization
• Fast Graph Representation Learning with PyTorch Geometric
• Simplifying Graph Convolutional Networks
• struc2vec: Learning Node Representations from Structural Identity
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Variational Graph Auto-Encoders
• DropEdge: Towards Deep Graph Convolutional Networks on Node Classification
• Predicting multicellular function through multi-layer tissue networks
• Contrastive Multiview Coding
• GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs
• Unsupervised Embedding Learning via Invariant and Spreading Instance Feature
• Colorization as a Proxy Task for Visual Understanding

Cited by in corpus (72)

• Graph Contrastive Learning with Adaptive Augmentation
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning
• Graph Self-Supervised Learning: A Survey
• Mining Latent Structures for Multimedia Recommendation
• A Comprehensive Survey on Deep Graph Representation Learning
• SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation
• Multi-level Cross-view Contrastive Learning for Knowledge-aware Recommender System
• Multi-view Graph Contrastive Representation Learning for Drug-Drug Interaction Prediction
• A Review-aware Graph Contrastive Learning Framework for Recommendation
• Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure
• Graph Barlow Twins: A self-supervised representation learning framework for graphs
• COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning
• Improving Knowledge-aware Recommendation with Multi-level Interactive Contrastive Learning
• Reconstruction Enhanced Multi-View Contrastive Learning for Anomaly Detection on Attributed Networks
• Augmentation-Free Graph Contrastive Learning of Invariant-Discriminative Representations
• GraphAIR: Graph Representation Learning with Neighborhood Aggregation and Interaction
• DIG: A Turnkey Library for Diving into Graph Deep Learning Research
• Graph Communal Contrastive Learning
• Jointly Contrastive Representation Learning on Road Network and Trajectory
• From Canonical Correlation Analysis to Self-supervised Graph Neural Networks
• Self-supervised Consensus Representation Learning for Attributed Graph
• Heterogeneous Graph Contrastive Learning with Meta-path Contexts and Adaptively Weighted Negative Samples
• InfoGCL: Information-Aware Graph Contrastive Learning
• Graph Neural Networks: Methods, Applications, and Opportunities
• Large-Scale Representation Learning on Graphs via Bootstrapping
• Towards Robust Graph Contrastive Learning
• A Contrastive Variational Graph Auto-Encoder for Node Clustering
• Modeling Network-level Traffic Flow Transitions on Sparse Data
• Relational Self-Supervised Learning on Graphs
• Gophormer: Ego-Graph Transformer for Node Classification
• Unsupervised Social Event Detection via Hybrid Graph Contrastive Learning and Reinforced Incremental Clustering
• CLDG: Contrastive Learning on Dynamic Graphs
• Incorporating Attributes and Multi-Scale Structures for Heterogeneous Graph Contrastive Learning
• Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning
• Large-scale graph representation learning with very deep GNNs and self-supervision
• Source Free Unsupervised Graph Domain Adaptation
• Reliable Node Similarity Matrix Guided Contrastive Graph Clustering
• Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Prediction
• Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning
• Knowledge-aware Contrastive Molecular Graph Learning
• Fairness-Aware Node Representation Learning
• Temporal Contrastive Graph Learning for Video Action Recognition and Retrieval
• Learning Robust Representation through Graph Adversarial Contrastive Learning
• CARL-G: Clustering-Accelerated Representation Learning on Graphs
• Independent Distribution Regularization for Private Graph Embedding
• DGTN: Dual-channel Graph Transition Network for Session-based Recommendation
• CSGDN: Contrastive Signed Graph Diffusion Network for Predicting Crop Gene-phenotype Associations
• Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networks
• Label Contrastive Coding based Graph Neural Network for Graph Classification
• Negative-Free Self-Supervised Gaussian Embedding of Graphs
• An Empirical Study of Graph Contrastive Learning
• Structure-Aware Hard Negative Mining for Heterogeneous Graph Contrastive Learning
• When Contrastive Learning Meets Active Learning: A Novel Graph Active Learning Paradigm with Self-Supervision
• Dynamic Graph Collaborative Filtering
• Deep Contrastive Multiview Network Embedding
• Ano-Graph: Learning Normal Scene Contextual Graphs to Detect Video Anomalies
• Simplification of Graph Convolutional Networks: A Matrix Factorization-based Perspective
• Contrastive Adaptive Propagation Graph Neural Networks for Efficient Graph Learning
• Attributed Graph Clustering with Multi-Scale Weight-Based Pairwise Coarsening and Contrastive Learning
• Inductive Subgraph Embedding for Link Prediction
• Hard Negative Sampling via Regularized Optimal Transport for Contrastive Representation Learning
• Jointly Learnable Data Augmentations for Self-Supervised GNNs
• A unified framework based on graph consensus term for multi-view learning
• Data-Driven Self-Supervised Graph Representation Learning
• Multi-hop Graph Convolutional Network with High-order Chebyshev Approximation for Text Reasoning
• GraphVICRegHSIC: Towards improved self-supervised representation learning for graphs with a hyrbid loss function
• CompanyKG: A Large-Scale Heterogeneous Graph for Company Similarity Quantification
• Locality Relationship Constrained Multi-view Clustering Framework
• Self-Supervised Graph Learning with Proximity-based Views and Channel Contrast
• Graph Contrastive Pre-training for Effective Theorem Reasoning
• Graph Partner Neural Networks for Semi-Supervised Learning on Graphs