Learn molecular representations from large-scale unlabeled molecules for drug discovery

Abstract

How to produce expressive molecular representations is a fundamental challenge in AI-driven drug discovery. Graph neural network (GNN) has emerged as a powerful technique for modeling molecular data. However, previous supervised approaches usually suffer from the scarcity of labeled data and have poor generalization capability. Here, we proposed a novel Molecular Pre-training Graph-based deep learning framework, named MPG, that leans molecular representations from large-scale unlabeled molecules. In MPG, we proposed a powerful MolGNet model and an effective self-supervised strategy for pre-training the model at both the node and graph-level. After pre-training on 11 million unlabeled molecules, we revealed that MolGNet can capture valuable chemistry insights to produce interpretable representation. The pre-trained MolGNet can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of drug discovery tasks, including molecular properties prediction, drug-drug interaction, and drug-target interaction, involving 13 benchmark datasets. Our work demonstrates that MPG is promising to become a novel approach in the drug discovery pipeline.

References in corpus (13)

• Semi-Supervised Classification with Graph Convolutional Networks
• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• A Simple Framework for Contrastive Learning of Visual Representations
• Inductive Representation Learning on Large Graphs
• Gaussian Error Linear Units (GELUs)
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• Towards Deeper Graph Neural Networks
• ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction
• Massively Multitask Networks for Drug Discovery
• Strategies for Pre-training Graph Neural Networks
• SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery
• InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization
• N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Molecules

Cited by in corpus (3)

• ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction
• Dual-view Molecule Pre-training
• BatmanNet: Bi-branch Masked Graph Transformer Autoencoder for Molecular Representation