Knowledge Distillation of a Protein Language Model Yields a Foundational Implicit Solvent Model

Abstract

Implicit solvent models (ISMs) promise to deliver the accuracy of explicit solvent simulations at a fraction of the computational cost. However, despite decades of development, their accuracy has remained insufficient for many critical applications, particularly for simulating protein folding and the behavior of intrinsically disordered proteins. Developing a transferable, data-driven ISM that overcomes the limitations of traditional analytical formulas remains a central challenge in computational chemistry. Here we address this challenge by introducing a novel strategy that distills the evolutionary information learned by a protein language model, ESM3, into a computationally efficient graph neural network (GNN). We show that this GNN potential, trained on effective energies from ESM3, is robust enough to drive stable, long-timescale molecular dynamics simulations. When combined with a standard electrostatics term, our hybrid model accurately reproduces protein folding free-energy landscapes and predicts the structural ensembles of intrinsically disordered proteins. This approach yields a single, unified model that is transferable across both folded and disordered protein states, resolving a long-standing limitation of conventional ISMs. By successfully distilling evolutionary knowledge into a physical potential, our work delivers a foundational implicit solvent model poised to accelerate the development of predictive, large-scale simulation tools.

References in corpus (38)

• Distilling the Knowledge in a Neural Network
• SchNet - a deep learning architecture for molecules and materials
• ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
• E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
• Rank-normalization, folding, and localization: An improved $\hat{R}$ for assessing convergence of MCMC
• PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments and Partial Charges
• Towards Exact Molecular Dynamics Simulations with Machine-Learned Force Fields
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• Directional Message Passing for Molecular Graphs
• MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields
• Equivariant message passing for the prediction of tensorial properties and molecular spectra
• SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
• Coarse Graining Molecular Dynamics with Graph Neural Networks
• Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations
• Cormorant: Covariant Molecular Neural Networks
• GemNet: Universal Directional Graph Neural Networks for Molecules
• Machine Learning Coarse-Grained Potentials of Protein Thermodynamics
• End-to-End Differentiable Molecular Mechanics Force Field Construction
• Machine Learning Implicit Solvation for Molecular Dynamics
• Flow-matching -- efficient coarse-graining of molecular dynamics without forces
• The need to implement FAIR principles in biomolecular simulations
• TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentials
• TorchMD-Net 2.0: Fast Neural Network Potentials for Molecular Simulations
• Adversarial-Residual-Coarse-Graining: Applying machine learning theory to systematic molecular coarse-graining
• Machine-learned molecular mechanics force field for the simulation of protein-ligand systems and beyond
• Contrastive Learning of Coarse-Grained Force Fields
• EspalomaCharge: Machine learning-enabled ultra-fast partial charge assignment
• Equivariant Graph Mechanics Networks with Constraints
• Implicit Solvent Approach Based on Generalised Born and Transferable Graph Neural Networks for Molecular Dynamics Simulations
• Spatial Attention Kinetic Networks with E(n)-Equivariance
• Accurate Molecular Dynamics Enabled by Efficient Physically-Constrained Machine Learning Approaches
• Extending the RANGE of Graph Neural Networks: Relaying Attention Nodes for Global Encoding
• Scaling Graph Neural Networks to Large Proteins
• Thermodynamically Informed Multimodal Learning of High-Dimensional Free Energy Models in Molecular Coarse Graining
• Learning data efficient coarse-grained molecular dynamics from forces and noise
• Egret-1: Pretrained Neural Network Potentials for Efficient and Accurate Bioorganic Simulation
• MD-LLM-1: A Large Language Model for Molecular Dynamics
• FlashSchNet: Fast and Accurate Coarse-Grained Neural Network Molecular Dynamics