A Meta-Learning Approach for Graph Representation Learning in Multi-Task Settings
arXiv:2012.06755
Abstract
Graph Neural Networks (GNNs) are a framework for graph representation learning, where a model learns to generate low dimensional node embeddings that encapsulate structural and feature-related information. GNNs are usually trained in an end-to-end fashion, leading to highly specialized node embeddings. However, generating node embeddings that can be used to perform multiple tasks (with performance comparable to single-task models) is an open problem. We propose a novel meta-learning strategy capable of producing multi-task node embeddings. Our method avoids the difficulties arising when learning to perform multiple tasks concurrently by, instead, learning to quickly (i.e. with a few steps of gradient descent) adapt to multiple tasks singularly. We show that the embeddings produced by our method can be used to perform multiple tasks with comparable or higher performance than classically trained models. Our method is model-agnostic and task-agnostic, thus applicable to a wide variety of multi-task domains.
Accepted at the NeurIPS Workshop on Meta-Learning (MetaLearn) 2020
References in corpus (17)
- Scikit-learn: Machine Learning in Python
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Fast Graph Representation Learning with PyTorch Geometric
- Few-Shot Learning with Graph Neural Networks
- Adversarial Attacks on Graph Neural Networks via Meta Learning
- TUDataset: A collection of benchmark datasets for learning with graphs
- Which Tasks Should Be Learned Together in Multi-task Learning?
- Machine Learning on Graphs: A Model and Comprehensive Taxonomy
- Learning to Propagate for Graph Meta-Learning
- Meta-GNN: On Few-shot Node Classification in Graph Meta-learning
- Edge-labeling Graph Neural Network for Few-shot Learning
- Meta-Graph: Few Shot Link Prediction via Meta Learning
- Torchmeta: A Meta-Learning library for PyTorch
- Neural Relational Inference with Fast Modular Meta-learning
- Meta-Learning GNN Initializations for Low-Resource Molecular Property Prediction
- Multitask Learning on Graph Neural Networks: Learning Multiple Graph Centrality Measures with a Unified Network