Graph Contrastive Learning with Adaptive Augmentation

Abstract

Recently, contrastive learning (CL) has emerged as a successful method for unsupervised graph representation learning. Most graph CL methods first perform stochastic augmentation on the input graph to obtain two graph views and maximize the agreement of representations in the two views. Despite the prosperous development of graph CL methods, the design of graph augmentation schemes -- a crucial component in CL -- remains rarely explored. We argue that the data augmentation schemes should preserve intrinsic structures and attributes of graphs, which will force the model to learn representations that are insensitive to perturbation on unimportant nodes and edges. However, most existing methods adopt uniform data augmentation schemes, like uniformly dropping edges and uniformly shuffling features, leading to suboptimal performance. In this paper, we propose a novel graph contrastive representation learning method with adaptive augmentation that incorporates various priors for topological and semantic aspects of the graph. Specifically, on the topology level, we design augmentation schemes based on node centrality measures to highlight important connective structures. On the node attribute level, we corrupt node features by adding more noise to unimportant node features, to enforce the model to recognize underlying semantic information. We perform extensive experiments of node classification on a variety of real-world datasets. Experimental results demonstrate that our proposed method consistently outperforms existing state-of-the-art baselines and even surpasses some supervised counterparts, which validates the effectiveness of the proposed contrastive framework with adaptive augmentation.

arXiv commentAccepted to WWW 2021, authors' version. 12 pages, 3 figures, 5 tables. arXiv admin note: substantial text overlap with arXiv:2006.04131

References in corpus (30)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• FaceNet: A Unified Embedding for Face Recognition and Clustering
• DeepWalk: Online Learning of Social Representations
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Cooperative Game Theory Approaches for Network Partitioning
• Inductive Representation Learning on Large Graphs
• Learning deep representations by mutual information estimation and maximization
• Unsupervised Representation Learning by Predicting Image Rotations
• Fast Graph Representation Learning with PyTorch Geometric
• Simplifying Graph Convolutional Networks
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Variational Graph Auto-Encoders
• GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training
• Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec
• Learning Representations by Maximizing Mutual Information Across Views
• What Makes for Good Views for Contrastive Learning?
• Pitfalls of Graph Neural Network Evaluation
• Deep Graph Contrastive Representation Learning
• Contrastive Multi-View Representation Learning on Graphs
• Representation Learning on Graphs: Methods and Applications
• On Mutual Information Maximization for Representation Learning
• Diffusion Improves Graph Learning
• On Variational Bounds of Mutual Information
• What Should Not Be Contrastive in Contrastive Learning
• A Framework For Contrastive Self-Supervised Learning And Designing A New Approach
• On Mutual Information in Contrastive Learning for Visual Representations
• Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networks
• Hierarchical Graph Convolutional Networks for Semi-supervised Node Classification

Cited by in corpus (92)

• Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning
• Graph Self-Supervised Learning: A Survey
• Mining Latent Structures for Multimedia Recommendation
• A Comprehensive Survey on Deep Graph Representation Learning
• SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation
• Multi-level Cross-view Contrastive Learning for Knowledge-aware Recommender System
• Multi-Modal Self-Supervised Learning for Recommendation
• Contrastive Meta Learning with Behavior Multiplicity for Recommendation
• Debiased Contrastive Learning for Sequential Recommendation
• Graph Neural Networks: Taxonomy, Advances and Trends
• Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• Graph Barlow Twins: A self-supervised representation learning framework for graphs
• COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning
• Graph Transformer for Recommendation
• Graph Rationalization with Environment-based Augmentations
• Contrastive Self-supervised Sequential Recommendation with Robust Augmentation
• Multi-behavior Self-supervised Learning for Recommendation
• Reconstruction Enhanced Multi-View Contrastive Learning for Anomaly Detection on Attributed Networks
• Augmentation-Free Graph Contrastive Learning of Invariant-Discriminative Representations
• Automated Spatio-Temporal Graph Contrastive Learning
• Graph Masked Autoencoder for Sequential Recommendation
• Hyperspectral Image Classification With Contrastive Graph Convolutional Network
• GraphAIR: Graph Representation Learning with Neighborhood Aggregation and Interaction
• Dual Space Graph Contrastive Learning
• Meta-optimized Contrastive Learning for Sequential Recommendation
• Self-supervised Contrastive Attributed Graph Clustering
• Graph Communal Contrastive Learning
• SE-GSL: A General and Effective Graph Structure Learning Framework through Structural Entropy Optimization
• Jointly Contrastive Representation Learning on Road Network and Trajectory
• From Canonical Correlation Analysis to Self-supervised Graph Neural Networks
• Propagation Tree Is Not Deep: Adaptive Graph Contrastive Learning Approach for Rumor Detection
• Graph Contrastive Learning with Generative Adversarial Network
• Heterogeneous Graph Contrastive Learning with Meta-path Contexts and Adaptively Weighted Negative Samples
• Generating Counterfactual Hard Negative Samples for Graph Contrastive Learning
• Graph Neural Networks: Methods, Applications, and Opportunities
• Large-Scale Representation Learning on Graphs via Bootstrapping
• Towards Robust Graph Contrastive Learning
• A Contrastive Variational Graph Auto-Encoder for Node Clustering
• X-GOAL: Multiplex Heterogeneous Graph Prototypical Contrastive Learning
• Multi-channel learning for integrating structural hierarchies into context-dependent molecular representation
• Relational Self-Supervised Learning on Graphs
• CLDG: Contrastive Learning on Dynamic Graphs
• Automated Self-Supervised Learning for Graphs
• GraFN: Semi-Supervised Node Classification on Graph with Few Labels via Non-Parametric Distribution Assignment
• Inductive Graph Alignment Prompt: Bridging the Gap between Graph Pre-training and Inductive Fine-tuning From Spectral Perspective
• NCAGC: A Neighborhood Contrast Framework for Attributed Graph Clustering
• USER: Unsupervised Structural Entropy-based Robust Graph Neural Network
• Reliable Node Similarity Matrix Guided Contrastive Graph Clustering
• Code Recommendation for Open Source Software Developers
• Advancing Biomedicine with Graph Representation Learning: Recent Progress, Challenges, and Future Directions
• Graph Contrastive Learning with Cohesive Subgraph Awareness
• Fairness-Aware Node Representation Learning
• Temporal Graph Representation Learning with Adaptive Augmentation Contrastive
• CVTGAD: Simplified Transformer with Cross-View Attention for Unsupervised Graph-level Anomaly Detection
• CARL-G: Clustering-Accelerated Representation Learning on Graphs
• Endowing Pre-trained Graph Models with Provable Fairness
• Unsupervised Multiplex Graph Learning with Complementary and Consistent Information
• FairSample: Training Fair and Accurate Graph Convolutional Neural Networks Efficiently
• Noise-Resilient Unsupervised Graph Representation Learning via Multi-Hop Feature Quality Estimation
• Local Contrastive Feature learning for Tabular Data
• Position-Aware Subgraph Neural Networks with Data-Efficient Learning
• Debiased Graph Poisoning Attack via Contrastive Surrogate Objective
• A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective
• Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networks
• Adaptive Multi-layer Contrastive Graph Neural Networks
• Negative-Free Self-Supervised Gaussian Embedding of Graphs
• InfoNCE is a Free Lunch for Semantically guided Graph Contrastive Learning
• Multi-Level Graph Contrastive Learning
• Negative as Positive: Enhancing Out-of-distribution Generalization for Graph Contrastive Learning
• Hierarchical Vector Quantized Graph Autoencoder with Annealing-Based Code Selection
• EAGLE: Contrastive Learning for Efficient Graph Anomaly Detection
• Structure-Aware Hard Negative Mining for Heterogeneous Graph Contrastive Learning
• Evaluating Modules in Graph Contrastive Learning
• Multi-Sample based Contrastive Loss for Top-k Recommendation
• An Empirical Study of Graph Contrastive Learning
• Exploring Global Information for Session-based Recommendation
• When Contrastive Learning Meets Active Learning: A Novel Graph Active Learning Paradigm with Self-Supervision
• Deep Contrastive Multiview Network Embedding
• Graph Self-Supervised Learning with Learnable Structural and Positional Encodings
• Separated Contrastive Learning for Matching in Cross-domain Recommendation with Curriculum Scheduling
• Jointly Learnable Data Augmentations for Self-Supervised GNNs
• Inductive Subgraph Embedding for Link Prediction
• 10 Security and Privacy Problems in Large Foundation Models
• GLADMamba: Unsupervised Graph-Level Anomaly Detection Powered by Selective State Space Model
• Locality Relationship Constrained Multi-view Clustering Framework
• Self-Supervised Graph Learning with Proximity-based Views and Channel Contrast
• Clustering-Oriented Generative Attribute Graph Imputation
• Data-Driven Self-Supervised Graph Representation Learning
• Link prediction on multi-relational graphs from an influence propagation perspective
• Scalable Deep Metric Learning on Attributed Graphs
• A unified framework based on graph consensus term for multi-view learning