Graph Contrastive Learning Automated

Abstract

Self-supervised learning on graph-structured data has drawn recent interest for learning generalizable, transferable and robust representations from unlabeled graphs. Among many, graph contrastive learning (GraphCL) has emerged with promising representation learning performance. Unfortunately, unlike its counterpart on image data, the effectiveness of GraphCL hinges on ad-hoc data augmentations, which have to be manually picked per dataset, by either rules of thumb or trial-and-errors, owing to the diverse nature of graph data. That significantly limits the more general applicability of GraphCL. Aiming to fill in this crucial gap, this paper proposes a unified bi-level optimization framework to automatically, adaptively and dynamically select data augmentations when performing GraphCL on specific graph data. The general framework, dubbed JOint Augmentation Optimization (JOAO), is instantiated as min-max optimization. The selections of augmentations made by JOAO are shown to be in general aligned with previous "best practices" observed from handcrafted tuning: yet now being automated, more flexible and versatile. Moreover, we propose a new augmentation-aware projection head mechanism, which will route output features through different projection heads corresponding to different augmentations chosen at each training step. Extensive experiments demonstrate that JOAO performs on par with or sometimes better than the state-of-the-art competitors including GraphCL, on multiple graph datasets of various scales and types, yet without resorting to any laborious dataset-specific tuning on augmentation selection. We release the code at https://github.com/Shen-Lab/GraphCL_Automated.

arXiv commentSupplementary materials are available at https://yyou1996.github.io/files/icml2021_graphcl_automated_supplement.pdf. ICML 2021

Cited by in corpus (18)

• Graph Self-Supervised Learning: A Survey
• SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation
• Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection
• Pre-training Molecular Graph Representation with 3D Geometry
• Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• 3D Infomax improves GNNs for Molecular Property Prediction
• Graph Transformer for Recommendation
• KPGT: Knowledge-Guided Pre-training of Graph Transformer for Molecular Property Prediction
• Propagation Tree Is Not Deep: Adaptive Graph Contrastive Learning Approach for Rumor Detection
• KGNN: Harnessing Kernel-based Networks for Semi-supervised Graph Classification
• Feature Overcorrelation in Deep Graph Neural Networks: A New Perspective
• Unsupervised Social Event Detection via Hybrid Graph Contrastive Learning and Reinforced Incremental Clustering
• Automated Self-Supervised Learning for Graphs
• Position-Aware Subgraph Neural Networks with Data-Efficient Learning
• Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networks
• Jointly Learnable Data Augmentations for Self-Supervised GNNs
• Data-Driven Self-Supervised Graph Representation Learning