Automated Self-Supervised Learning for Graphs

Abstract

Graph self-supervised learning has gained increasing attention due to its capacity to learn expressive node representations. Many pretext tasks, or loss functions have been designed from distinct perspectives. However, we observe that different pretext tasks affect downstream tasks differently cross datasets, which suggests that searching pretext tasks is crucial for graph self-supervised learning. Different from existing works focusing on designing single pretext tasks, this work aims to investigate how to automatically leverage multiple pretext tasks effectively. Nevertheless, evaluating representations derived from multiple pretext tasks without direct access to ground truth labels makes this problem challenging. To address this obstacle, we make use of a key principle of many real-world graphs, i.e., homophily, or the principle that "like attracts like," as the guidance to effectively search various self-supervised pretext tasks. We provide theoretical understanding and empirical evidence to justify the flexibility of homophily in this search task. Then we propose the AutoSSL framework which can automatically search over combinations of various self-supervised tasks. By evaluating the framework on 7 real-world datasets, our experimental results show that AutoSSL can significantly boost the performance on downstream tasks including node clustering and node classification compared with training under individual tasks. Code is released at https://github.com/ChandlerBang/AutoSSL.

ICLR 2022

References in corpus (12)

• Semi-Supervised Classification with Graph Convolutional Networks
• Fast Graph Representation Learning with PyTorch Geometric
• Graph Contrastive Learning with Adaptive Augmentation
• Variational Graph Auto-Encoders
• Contrastive Multi-View Representation Learning on Graphs
• Geom-GCN: Geometric Graph Convolutional Networks
• Self-supervised Learning on Graphs: Deep Insights and New Direction
• Graph Contrastive Learning Automated
• When Does Self-Supervision Help Graph Convolutional Networks?
• Gender Bias in Contextualized Word Embeddings
• Loss Function Search for Face Recognition
• Elastic Graph Neural Networks

Cited by in corpus (5)

• Graph Neural Networks: Methods, Applications, and Opportunities
• Feature Overcorrelation in Deep Graph Neural Networks: A New Perspective
• Rethinking Sensors Modeling: Hierarchical Information Enhanced Traffic Forecasting
• CARL-G: Clustering-Accelerated Representation Learning on Graphs
• AutoLoss: Automated Loss Function Search in Recommendations