Contrastive Multi-View Representation Learning on Graphs

Abstract

We introduce a self-supervised approach for learning node and graph level representations by contrasting structural views of graphs. We show that unlike visual representation learning, increasing the number of views to more than two or contrasting multi-scale encodings do not improve performance, and the best performance is achieved by contrasting encodings from first-order neighbors and a graph diffusion. We achieve new state-of-the-art results in self-supervised learning on 8 out of 8 node and graph classification benchmarks under the linear evaluation protocol. For example, on Cora (node) and Reddit-Binary (graph) classification benchmarks, we achieve 86.8% and 84.5% accuracy, which are 5.5% and 2.4% relative improvements over previous state-of-the-art. When compared to supervised baselines, our approach outperforms them in 4 out of 8 benchmarks. Source code is released at: https://github.com/kavehhassani/mvgrl

ICML 2020

References in corpus (13)

• A Simple Framework for Contrastive Learning of Visual Representations
• Inductive Representation Learning on Large Graphs
• struc2vec: Learning Node Representations from Structural Identity
• Representation Learning on Graphs with Jumping Knowledge Networks
• Variational Graph Auto-Encoders
• Contrastive Multiview Coding
• graph2vec: Learning Distributed Representations of Graphs
• VERSE: Versatile Graph Embeddings from Similarity Measures
• Graph networks as learnable physics engines for inference and control
• Subgraph Matching Kernels for Attributed Graphs
• Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
• Memory-Based Graph Networks
• Relational Graph Representation Learning for Open-Domain Question Answering

Cited by in corpus (79)

• Graph Contrastive Learning with Adaptive Augmentation
• Graph Contrastive Learning with Augmentations
• Contrastive Representation Learning: A Framework and Review
• Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning
• Graph Self-Supervised Learning: A Survey
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning
• SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation
• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• Multi-level Cross-view Contrastive Learning for Knowledge-aware Recommender System
• Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection
• A Review-aware Graph Contrastive Learning Framework for Recommendation
• Pre-training Molecular Graph Representation with 3D Geometry
• Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure
• Memory-Guided Multi-View Multi-Domain Fake News Detection
• COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning
• Improving Knowledge-aware Recommendation with Multi-level Interactive Contrastive Learning
• Reconstruction Enhanced Multi-View Contrastive Learning for Anomaly Detection on Attributed Networks
• Dual Space Graph Contrastive Learning
• DIG: A Turnkey Library for Diving into Graph Deep Learning Research
• Graph Communal Contrastive Learning
• Jointly Contrastive Representation Learning on Road Network and Trajectory
• CCGL: Contrastive Cascade Graph Learning
• Propagation Tree Is Not Deep: Adaptive Graph Contrastive Learning Approach for Rumor Detection
• Contrastive Knowledge Graph Error Detection
• Semi-Supervised Hierarchical Graph Classification
• Contrastive Learning with Hard Negative Samples
• Self-Supervised Hypergraph Convolutional Networks for Session-based Recommendation
• Large-Scale Representation Learning on Graphs via Bootstrapping
• Contrastive Multi-Level Graph Neural Networks for Session-based Recommendation
• Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization
• A Contrastive Variational Graph Auto-Encoder for Node Clustering
• X-GOAL: Multiplex Heterogeneous Graph Prototypical Contrastive Learning
• Improving Text-to-Image Synthesis Using Contrastive Learning
• A Self-supervised Riemannian GNN with Time Varying Curvature for Temporal Graph Learning
• GraphCrop: Subgraph Cropping for Graph Classification
• CLDG: Contrastive Learning on Dynamic Graphs
• Relational Self-Supervised Learning on Graphs
• Automated Self-Supervised Learning for Graphs
• Deep Contrastive Graph Learning with Clustering-Oriented Guidance
• Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning
• AnchorGAE: General Data Clustering via $O(n)$ Bipartite Graph Convolution
• Reliable Node Similarity Matrix Guided Contrastive Graph Clustering
• Counterfactual Learning on Graphs: A Survey
• Node Similarity Preserving Graph Convolutional Networks
• Self-supervised Graph Neural Networks without explicit negative sampling
• Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning
• Knowledge-aware Contrastive Molecular Graph Learning
• Iterative Graph Self-Distillation
• Socially-Aware Self-Supervised Tri-Training for Recommendation
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption
• Learning Robust Representation through Graph Adversarial Contrastive Learning
• Self-Supervised Graph Co-Training for Session-based Recommendation
• Improving Transferability of Representations via Augmentation-Aware Self-Supervision
• Robust Basket Recommendation via Noise-tolerated Graph Contrastive Learning
• CARL-G: Clustering-Accelerated Representation Learning on Graphs
• Position-Aware Subgraph Neural Networks with Data-Efficient Learning
• Generative Models as a Data Source for Multiview Representation Learning
• Self-supervised Graph Representation Learning via Bootstrapping
• CogDL: A Comprehensive Library for Graph Deep Learning
• Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networks
• Towards Deviation-Robust Agent Navigation via Perturbation-Aware Contrastive Learning
• Negative-Free Self-Supervised Gaussian Embedding of Graphs
• Evaluating Modules in Graph Contrastive Learning
• Deep Code Search with Naming-Agnostic Contrastive Multi-View Learning
• Contrastive Attraction and Contrastive Repulsion for Representation Learning
• When Contrastive Learning Meets Active Learning: A Novel Graph Active Learning Paradigm with Self-Supervision
• Leveraging Meta-path Contexts for Classification in Heterogeneous Information Networks
• Self-Supervised Longitudinal Neighbourhood Embedding
• Deep Contrastive Multiview Network Embedding
• AKE-GNN: Effective Graph Learning with Adaptive Knowledge Exchange
• CLOP: Video-and-Language Pre-Training with Knowledge Regularizations
• Graph Self-Supervised Learning with Learnable Structural and Positional Encodings
• Simplification of Graph Convolutional Networks: A Matrix Factorization-based Perspective
• Ano-Graph: Learning Normal Scene Contextual Graphs to Detect Video Anomalies
• Hard Negative Sampling via Regularized Optimal Transport for Contrastive Representation Learning
• Barlow Graph Auto-Encoder for Unsupervised Network Embedding
• Pairwise Half-graph Discrimination: A Simple Graph-level Self-supervised Strategy for Pre-training Graph Neural Networks
• CompanyKG: A Large-Scale Heterogeneous Graph for Company Similarity Quantification
• Data-Driven Self-Supervised Graph Representation Learning