Meta-Learning GNN Initializations for Low-Resource Molecular Property Prediction

Abstract

Building in silico models to predict chemical properties and activities is a crucial step in drug discovery. However, limited labeled data often hinders the application of deep learning in this setting. Meanwhile advances in meta-learning have enabled state-of-the-art performances in few-shot learning benchmarks, naturally prompting the question: Can meta-learning improve deep learning performance in low-resource drug discovery projects? In this work, we assess the transferability of graph neural networks initializations learned by the Model-Agnostic Meta-Learning (MAML) algorithm - and its variants FO-MAML and ANIL - for chemical properties and activities tasks. Using the ChEMBL20 dataset to emulate low-resource settings, our benchmark shows that meta-initializations perform comparably to or outperform multi-task pre-training baselines on 16 out of 20 in-distribution tasks and on all out-of-distribution tasks, providing an average improvement in AUPRC of 11.2% and 26.9% respectively. Finally, we observe that meta-initializations consistently result in the best performing models across fine-tuning sets with $k\in \{16,32,64,128,256\}$ instances.

arXiv commentICML 2020 Workshop on Graph Representation Learning and Beyond (GRL+)

References in corpus (10)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Neural Message Passing for Quantum Chemistry
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• Theoretical Models of Learning to Learn
• Convolutional Networks on Graphs for Learning Molecular Fingerprints
• Massively Multitask Networks for Drug Discovery
• Meta-Learning: A Survey
• Strategies for Pre-training Graph Neural Networks
• Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples
• Modeling Industrial ADMET Data with Multitask Networks

Cited by in corpus (4)

• A Meta-Learning Approach for Graph Representation Learning in Multi-Task Settings
• Bayesian Graph Neural Networks for Molecular Property Prediction
• Improving Molecular Graph Neural Network Explainability with Orthonormalization and Induced Sparsity
• An In-depth Summary of Recent Artificial Intelligence Applications in Drug Design