COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning

Abstract

Graph contrastive learning (GCL) improves graph representation learning, leading to SOTA on various downstream tasks. The graph augmentation step is a vital but scarcely studied step of GCL. In this paper, we show that the node embedding obtained via the graph augmentations is highly biased, somewhat limiting contrastive models from learning discriminative features for downstream tasks. Thus, instead of investigating graph augmentation in the input space, we alternatively propose to perform augmentations on the hidden features (feature augmentation). Inspired by so-called matrix sketching, we propose COSTA, a novel COvariance-preServing feaTure space Augmentation framework for GCL, which generates augmented features by maintaining a "good sketch" of original features. To highlight the superiority of feature augmentation with COSTA, we investigate a single-view setting (in addition to multi-view one) which conserves memory and computations. We show that the feature augmentation with COSTA achieves comparable/better results than graph augmentation based models.

This paper is accepted by the ACM KDD 2022

References in corpus (12)

• Learning Representations by Maximizing Mutual Information Across Views
• Deep Graph Contrastive Representation Learning
• Contrastive Multi-View Representation Learning on Graphs
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for Recommendation
• HRCF: Enhancing Collaborative Filtering via Hyperbolic Geometric Regularization
• Graph-based Semi-supervised Learning: A Comprehensive Review
• GraphCL: Contrastive Self-Supervised Learning of Graph Representations
• Contrastive Laplacian Eigenmaps
• Graph-adaptive Rectified Linear Unit for Graph Neural Networks
• REFINE: Random RangE FInder for Network Embedding
• Modeling Scale-free Graphs with Hyperbolic Geometry for Knowledge-aware Recommendation

Cited by in corpus (11)

• GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learning
• Bipartite Graph Convolutional Hashing for Effective and Efficient Top-N Search in Hamming Space
• Traffic Forecasting on New Roads Using Spatial Contrastive Pre-Training (SCPT)
• Understanding and Mitigating Hyperbolic Dimensional Collapse in Graph Contrastive Learning
• Noise-Resilient Unsupervised Graph Representation Learning via Multi-Hop Feature Quality Estimation
• A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective
• Negative-Free Self-Supervised Gaussian Embedding of Graphs
• InfoNCE is a Free Lunch for Semantically guided Graph Contrastive Learning
• Negative as Positive: Enhancing Out-of-distribution Generalization for Graph Contrastive Learning
• Momentum Contrastive Pre-training for Question Answering
• Graph Self-Supervised Learning with Learnable Structural and Positional Encodings