Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrast

Abstract

Deep learning has been a prevalence in computational chemistry and widely implemented in molecule property predictions. Recently, self-supervised learning (SSL), especially contrastive learning (CL), gathers growing attention for the potential to learn molecular representations that generalize to the gigantic chemical space. Unlike supervised learning, SSL can directly leverage large unlabeled data, which greatly reduces the effort to acquire molecular property labels through costly and time-consuming simulations or experiments. However, most molecular SSL methods borrow the insights from the machine learning community but neglect the unique cheminformatics (e.g., molecular fingerprints) and multi-level graphical structures (e.g., functional groups) of molecules. In this work, we propose iMolCLR: improvement of Molecular Contrastive Learning of Representations with graph neural networks (GNNs) in two aspects, (1) mitigating faulty negative contrastive instances via considering cheminformatics similarities between molecule pairs; (2) fragment-level contrasting between intra- and inter-molecule substructures decomposed from molecules. Experiments have shown that the proposed strategies significantly improve the performance of GNN models on various challenging molecular property predictions. In comparison to the previous CL framework, iMolCLR demonstrates an averaged 1.3% improvement of ROC-AUC on 7 classification benchmarks and an averaged 4.8% decrease of the error on 5 regression benchmarks. On most benchmarks, the generic GNN pre-trained by iMolCLR rivals or even surpasses supervised learning models with sophisticated architecture designs and engineered features. Further investigations demonstrate that representations learned through iMolCLR intrinsically embed scaffolds and functional groups that can reason molecule similarities.

arXiv comment30 pages, 4 figures, 4 tables. Manuscript in submission

References in corpus (22)

• Deep Learning in Neural Networks: An Overview
• A Comprehensive Survey on Graph Neural Networks
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Neural Message Passing for Quantum Chemistry
• Geometric deep learning: going beyond Euclidean data
• Automatic chemical design using a data-driven continuous representation of molecules
• SchNet - a deep learning architecture for molecules and materials
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Molecular Contrastive Learning of Representations via Graph Neural Networks
• Graph Contrastive Learning with Augmentations
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction
• Self-Supervised Graph Transformer on Large-Scale Molecular Data
• Directional Message Passing for Molecular Graphs
• Learning from Protein Structure with Geometric Vector Perceptrons
• Analyzing Learned Molecular Representations for Property Prediction
• Keeping it Simple: Language Models can learn Complex Molecular Distributions
• Pre-training Molecular Graph Representation with 3D Geometry
• Molecular representation learning with language models and domain-relevant auxiliary tasks
• Deep Reinforcement Learning Optimizes Graphene Nanopores for Efficient Desalination
• Dual-view Molecule Pre-training

Cited by in corpus (5)

• Graph Neural Networks for Molecules
• Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentials
• Beyond Independent Error Assumptions in Large GNN Atomistic Models
• Pretraining a Foundation Model for Small-Molecule Natural Products
• RetroMPA: A Molecular Property-Aware Auxiliary Framework for Enhancing Retrosynthesis Prediction