Molecular Contrastive Learning of Representations via Graph Neural Networks

Abstract

Molecular Machine Learning (ML) bears promise for efficient molecule property prediction and drug discovery. However, labeled molecule data can be expensive and time-consuming to acquire. Due to the limited labeled data, it is a great challenge for supervised-learning ML models to generalize to the giant chemical space. In this work, we present MolCLR: Molecular Contrastive Learning of Representations via Graph Neural Networks (GNNs), a self-supervised learning framework that leverages large unlabeled data (~10M unique molecules). In MolCLR pre-training, we build molecule graphs and develop GNN encoders to learn differentiable representations. Three molecule graph augmentations are proposed: atom masking, bond deletion, and subgraph removal. A contrastive estimator maximizes the agreement of augmentations from the same molecule while minimizing the agreement of different molecules. Experiments show that our contrastive learning framework significantly improves the performance of GNNs on various molecular property benchmarks including both classification and regression tasks. Benefiting from pre-training on the large unlabeled database, MolCLR even achieves state-of-the-art on several challenging benchmarks after fine-tuning. Additionally, further investigations demonstrate that MolCLR learns to embed molecules into representations that can distinguish chemically reasonable molecular similarities.

References in corpus (7)

• Graph Contrastive Learning with Augmentations
• Graph Transformer Networks
• ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction
• Massively Multitask Networks for Drug Discovery
• Understanding molecular representations in machine learning: The role of uniqueness and target similarity
• Orbital Graph Convolutional Neural Network for Material Property Prediction
• Deep Reinforcement Learning Optimizes Graphene Nanopores for Efficient Desalination

Cited by in corpus (36)

• A Comprehensive Survey on Deep Graph Representation Learning
• polyBERT: A chemical language model to enable fully machine-driven ultrafast polymer informatics
• TransPolymer: a Transformer-based language model for polymer property predictions
• MechRetro is a chemical-mechanism-driven graph learning framework for interpretable retrosynthesis prediction and pathway planning
• Comprehensive evaluation of deep and graph learning on drug-drug interactions prediction
• Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrast
• Graph Neural Networks for Molecules
• GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text
• End-to-end Wind Turbine Wake Modelling with Deep Graph Representation Learning
• Artificial Intelligence in Material Engineering: A review on applications of AI in Material Engineering
• Graph Neural Networks Accelerated Molecular Dynamics
• Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentials
• Geometry-enhanced Pre-training on Interatomic Potentials
• Multi-channel learning for integrating structural hierarchies into context-dependent molecular representation
• Surrogate- and invariance-boosted contrastive learning for data-scarce applications in science
• MolSets: Molecular Graph Deep Sets Learning for Mixture Property Modeling
• Spiers Memorial Lecture: How to do impactful research in artificial intelligence for chemistry and materials science
• Discovering dynamic laws from observations: the case of self-propelled, interacting colloids
• Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems
• A Large Encoder-Decoder Family of Foundation Models For Chemical Language
• Counterfactual Learning on Graphs: A Survey
• Benchmarking Large Language Models for Polymer Property Predictions
• Beyond Independent Error Assumptions in Large GNN Atomistic Models
• Beyond Over-smoothing: Uncovering the Trainability Challenges in Deep Graph Neural Networks
• HD-Bind: Encoding of Molecular Structure with Low Precision, Hyperdimensional Binary Representations
• Contrastive Dual-Interaction Graph Neural Network for Molecular Property Prediction
• Chemical Language Model Linker: blending text and molecules with modular adapters
• A simple framework for contrastive learning phases of matter
• A Survey on Learning from Graphs with Heterophily: Recent Advances and Future Directions
• Multi-Level Fusion Graph Neural Network for Molecule Property Prediction
• Handling Feature Heterogeneity with Learnable Graph Patches
• PASCL: Supervised Contrastive Learning with Perturbative Augmentation for Particle Decay Reconstruction
• Pretraining a Foundation Model for Small-Molecule Natural Products
• Introduction to Graph Neural Networks for Machine Learning Engineers
• A Machine Learning Benchmarking Framework for Lipid Nanoparticle Transfection Efficiency Prediction
• Dual-Modality Representation Learning for Molecular Property Prediction