Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentials

Abstract

Recent advances in equivariant graph neural networks (GNNs) have made deep learning amenable to developing fast surrogate models to expensive ab initio quantum mechanics (QM) approaches for molecular potential predictions. However, building accurate and transferable potential models using GNNs remains challenging, as the data is greatly limited by the expensive computational costs and level of theory of QM methods, especially for large and complex molecular systems. In this work, we propose denoise pretraining on nonequilibrium molecular conformations to achieve more accurate and transferable GNN potential predictions. Specifically, atomic coordinates of sampled nonequilibrium conformations are perturbed by random noises and GNNs are pretrained to denoise the perturbed molecular conformations which recovers the original coordinates. Rigorous experiments on multiple benchmarks reveal that pretraining significantly improves the accuracy of neural potentials. Furthermore, we show that the proposed pretraining approach is model-agnostic, as it improves the performance of different invariant and equivariant GNNs. Notably, our models pretrained on small molecules demonstrate remarkable transferability, improving performance when fine-tuned on diverse molecular systems, including different elements, charged molecules, biomolecules, and larger systems. These results highlight the potential for leveraging denoise pretraining approaches to build more generalizable neural potentials for complex molecular systems.

arXiv commentPublished in Journal of Chemical Theory and Computation. 32 pages, 5 figures, 2 tables

References in corpus (26)

• Decoupled Weight Decay Regularization
• Gaussian Approximation Potentials: the accuracy of quantum mechanics, without the electrons
• On representing chemical environments
• SchNet - a deep learning architecture for molecules and materials
• ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
• E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
• Machine Learning of Accurate Energy-Conserving Molecular Force Fields
• PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments and Partial Charges
• Molecular Contrastive Learning of Representations via Graph Neural Networks
• Less is more: sampling chemical space with active learning
• Machine Learning Unifies the Modelling of Materials and Molecules
• Towards Exact Molecular Dynamics Simulations with Machine-Learned Force Fields
• ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• Graph Self-Supervised Learning: A Survey
• Symmetry-Adapted Machine-Learning for Tensorial Properties of Atomistic Systems
• Hierarchical modeling of molecular energies using a deep neural network
• Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations
• Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrast
• Graph Neural Networks Accelerated Molecular Dynamics
• TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentials
• High-Dimensional Potential Energy Surfaces for Molecular Simulations
• Pre-training via Denoising for Molecular Property Prediction
• Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matching
• Motif-Driven Contrastive Learning of Graph Representations
• Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamics

Cited by in corpus (5)

• DPA-2: a large atomic model as a multi-task learner
• Graph Neural Networks for Molecules
• Machine-learned molecular mechanics force field for the simulation of protein-ligand systems and beyond
• Geometry-enhanced Pre-training on Interatomic Potentials
• Molecular relaxation by reverse diffusion with time step prediction