Adversarial Graph Augmentation to Improve Graph Contrastive Learning

Abstract

Self-supervised learning of graph neural networks (GNN) is in great need because of the widespread label scarcity issue in real-world graph/network data. Graph contrastive learning (GCL), by training GNNs to maximize the correspondence between the representations of the same graph in its different augmented forms, may yield robust and transferable GNNs even without using labels. However, GNNs trained by traditional GCL often risk capturing redundant graph features and thus may be brittle and provide sub-par performance in downstream tasks. Here, we propose a novel principle, termed adversarial-GCL (AD-GCL), which enables GNNs to avoid capturing redundant information during the training by optimizing adversarial graph augmentation strategies used in GCL. We pair AD-GCL with theoretical explanations and design a practical instantiation based on trainable edge-dropping graph augmentation. We experimentally validate AD-GCL by comparing with the state-of-the-art GCL methods and achieve performance gains of up-to $14\%$ in unsupervised, $6\%$ in transfer, and $3\%$ in semi-supervised learning settings overall with 18 different benchmark datasets for the tasks of molecule property regression and classification, and social network classification.

arXiv commentAccepted to NeurIPS 2021

References in corpus (32)

• Scikit-learn: Machine Learning in Python
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Categorical Reparameterization with Gumbel-Softmax
• The information bottleneck method
• Learning deep representations by mutual information estimation and maximization
• Graph Contrastive Learning with Adaptive Augmentation
• struc2vec: Learning Node Representations from Structural Identity
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Variational Graph Auto-Encoders
• Contrastive Multiview Coding
• graph2vec: Learning Distributed Representations of Graphs
• Graph Self-Supervised Learning: A Survey
• Discriminative Embeddings of Latent Variable Models for Structured Data
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• A Survey on Graph Kernels
• Learning Structural Node Embeddings Via Diffusion Wavelets
• Deep Variational Information Bottleneck
• TUDataset: A collection of benchmark datasets for learning with graphs
• Benchmarking Graph Neural Networks
• Parameterized Explainer for Graph Neural Network
• Strategies for Pre-training Graph Neural Networks
• On Variational Bounds of Mutual Information
• InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization
• Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow
• Machine Learning on Graphs: A Model and Comprehensive Taxonomy
• Graph Contrastive Learning Automated
• Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning
• Graph isomorphisms in quasi-polynomial time
• Motif-Driven Contrastive Learning of Graph Representations
• Recognizing Predictive Substructures with Subgraph Information Bottleneck

Cited by in corpus (9)

• Graph Self-Supervised Learning: A Survey
• Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure
• COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning
• Modeling Multiple Views via Implicitly Preserving Global Consistency and Local Complementarity
• Graph Contrastive Learning with Cohesive Subgraph Awareness
• Improving the Robustness of Transformer-based Large Language Models with Dynamic Attention
• Towards Deviation-Robust Agent Navigation via Perturbation-Aware Contrastive Learning
• Handling Feature Heterogeneity with Learnable Graph Patches
• Data-Driven Self-Supervised Graph Representation Learning