Self-supervised Auxiliary Learning for Graph Neural Networks via Meta-Learning
arXiv:2103.00771
Abstract
In recent years, graph neural networks (GNNs) have been widely adopted in the representation learning of graph-structured data and provided state-of-the-art performance in various applications such as link prediction, node classification, and recommendation. Motivated by recent advances of self-supervision for representation learning in natural language processing and computer vision, self-supervised learning has been recently studied to leverage unlabeled graph-structured data. However, employing self-supervision tasks as auxiliary tasks to assist a primary task has been less explored in the literature on graphs. In this paper, we propose a novel self-supervised auxiliary learning framework to effectively learn graph neural networks. Moreover, this work is the first study showing that a meta-path prediction is beneficial as a self-supervised auxiliary task for heterogeneous graphs. Our method is learning to learn a primary task with various auxiliary tasks to improve generalization performance. The proposed method identifies an effective combination of auxiliary tasks and automatically balances them to improve the primary task. Our methods can be applied to any graph neural network in a plug-in manner without manual labeling or additional data. Also, it can be extended to any other auxiliary tasks. Our experiments demonstrate that the proposed method consistently improves the performance of node classification and link prediction.
arXiv commentSubmitted to TPAMI, 14pages. arXiv admin note: substantial text overlap with arXiv:2007.08294
References in corpus (23)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Prototypical Networks for Few-shot Learning
- Inductive Representation Learning on Large Graphs
- Conditional Image Synthesis With Auxiliary Classifier GANs
- Graph Neural Networks: A Review of Methods and Applications
- Fast Graph Representation Learning with PyTorch Geometric
- Simplifying Graph Convolutional Networks
- Graph Convolutional Matrix Completion
- A Survey on Multi-Task Learning
- Interaction Networks for Learning about Objects, Relations and Physics
- Graph Transformer Networks
- Strategies for Pre-training Graph Neural Networks
- Fast Context Adaptation via Meta-Learning
- Self-supervised Learning on Graphs: Deep Insights and New Direction
- NSML: Meet the MLaaS platform with a real-world case study
- NSML: A Machine Learning Platform That Enables You to Focus on Your Models
- Learning to Propagate for Graph Meta-Learning
- Meta-Graph: Few Shot Link Prediction via Meta Learning
- Learning What and Where to Transfer
- Pre-Training Graph Neural Networks for Generic Structural Feature Extraction
- One-Shot Relational Learning for Knowledge Graphs
- Pre-training Graph Neural Networks with Kernels
- Meta Relational Learning for Few-Shot Link Prediction in Knowledge Graphs