Motif-Driven Contrastive Learning of Graph Representations

Abstract

Pre-training Graph Neural Networks (GNN) via self-supervised contrastive learning has recently drawn lots of attention. However, most existing works focus on node-level contrastive learning, which cannot capture global graph structure. The key challenge to conducting subgraph-level contrastive learning is to sample informative subgraphs that are semantically meaningful. To solve it, we propose to learn graph motifs, which are frequently-occurring subgraph patterns (e.g. functional groups of molecules), for better subgraph sampling. Our framework MotIf-driven Contrastive leaRning Of Graph representations (MICRO-Graph) can: 1) use GNNs to extract motifs from large graph datasets; 2) leverage learned motifs to sample informative subgraphs for contrastive learning of GNN. We formulate motif learning as a differentiable clustering problem, and adopt EM-clustering to group similar and significant subgraphs into several motifs. Guided by these learned motifs, a sampler is trained to generate more informative subgraphs, and these subgraphs are used to train GNNs through graph-to-subgraph contrastive learning. By pre-training on the ogbg-molhiv dataset with MICRO-Graph, the pre-trained GNN achieves 2.04% ROC-AUC average performance enhancement on various downstream benchmark datasets, which is significantly higher than other state-of-the-art self-supervised learning baselines.

References in corpus (17)

• Semi-Supervised Classification with Graph Convolutional Networks
• A Simple Framework for Contrastive Learning of Visual Representations
• Inductive Representation Learning on Large Graphs
• Neural Discrete Representation Learning
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Learning deep representations by mutual information estimation and maximization
• Momentum Contrast for Unsupervised Visual Representation Learning
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• Self-Supervised Graph Transformer on Large-Scale Molecular Data
• How Powerful are Graph Neural Networks?
• DeeperGCN: All You Need to Train Deeper GCNs
• Strategies for Pre-training Graph Neural Networks
• InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization
• Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances
• Unsupervised Inductive Graph-Level Representation Learning via Graph-Graph Proximity
• Pre-training Graph Neural Networks with Kernels

Cited by in corpus (7)

• Graph Neural Networks: Taxonomy, Advances and Trends
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• Graph Communal Contrastive Learning
• Motif-based Graph Self-Supervised Learning for Molecular Property Prediction
• Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentials
• An Empirical Study of Graph Contrastive Learning
• Pairwise Half-graph Discrimination: A Simple Graph-level Self-supervised Strategy for Pre-training Graph Neural Networks