DPA-2: a large atomic model as a multi-task learner

Abstract

The rapid advancements in artificial intelligence (AI) are catalyzing transformative changes in atomic modeling, simulation, and design. AI-driven potential energy models have demonstrated the capability to conduct large-scale, long-duration simulations with the accuracy of ab initio electronic structure methods. However, the model generation process remains a bottleneck for large-scale applications. We propose a shift towards a model-centric ecosystem, wherein a large atomic model (LAM), pre-trained across multiple disciplines, can be efficiently fine-tuned and distilled for various downstream tasks, thereby establishing a new framework for molecular modeling. In this study, we introduce the DPA-2 architecture as a prototype for LAMs. Pre-trained on a diverse array of chemical and materials systems using a multi-task approach, DPA-2 demonstrates superior generalization capabilities across multiple downstream tasks compared to the traditional single-task pre-training and fine-tuning methodologies. Our approach sets the stage for the development and broad application of LAMs in molecular and materials simulation research.

References in corpus (42)

• Gaussian Approximation Potentials: the accuracy of quantum mechanics, without the electrons
• On representing chemical environments
• Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
• Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics
• ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
• E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
• Moment Tensor Potentials: a class of systematically improvable interatomic potentials
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Machine Learning of Accurate Energy-Conserving Molecular Force Fields
• A Spectral Analysis Method for Automated Generation of Quantum-Accurate Interatomic Potentials
• A Universal Graph Deep Learning Interatomic Potential for the Periodic Table
• DP-GEN: A concurrent learning platform for the generation of reliable deep learning based potential energy models
• Less is more: sampling chemical space with active learning
• The Open Catalyst 2020 (OC20) Dataset and Community Challenges
• DeePMD-kit v2: A software package for Deep Potential models
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• Active Learning of Uniformly Accurate Inter-atomic Potentials for Materials Simulation
• SchNet: A continuous-filter convolutional neural network for modeling quantum interactions
• Towards Universal Neural Network Potential for Material Discovery Applicable to Arbitrary Combination of 45 Elements
• The Individual and Collective Effects of Exact Exchange and Dispersion Interactions on the Ab Initio Structure of Liquid Water
• WACSF - Weighted Atom-Centered Symmetry Functions as Descriptors in Machine Learning Potentials
• Embedded Atom Neural Network Potentials: Efficient and Accurate Machine Learning with a Physically Inspired Representation
• A foundation model for atomistic materials chemistry
• Deep Potentials for Materials Science
• Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules
• End-to-end Symmetry Preserving Inter-atomic Potential Energy Model for Finite and Extended Systems
• Systematically improvable optimized atomic basis sets for {\it ab inito} calculations
• Deep Potential generation scheme and simulation protocol for the Li10GeP2S12-type superionic conductors
• Unified Graph Neural Network Force-field for the Periodic Table
• EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations
• Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs
• Accurate Deep Potential model for the Al-Cu-Mg alloy in the full concentration space
• GemNet-OC: Developing Graph Neural Networks for Large and Diverse Molecular Simulation Datasets
• Rotation Invariant Graph Neural Networks using Spin Convolutions
• Universal interatomic potential for perovskite oxides
• Spherical Channels for Modeling Atomic Interactions
• Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentials
• From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction
• Towards Foundation Models for Materials Science: The Open MatSci ML Toolkit
• Highly Accurate Quantum Chemical Property Prediction with Uni-Mol+
• May the Force be with You: Unified Force-Centric Pre-Training for 3D Molecular Conformations
• Soft Phonon Modes and Diffuse Scattering in Pb(In1/2Nb1/2)O3-Pb(Mg1/3Nb2/3)O3-PbTiO3 Relaxor Ferroelectrics

Cited by in corpus (14)

• DeePMD-kit v3: A Multiple-Backend Framework for Machine Learning Potentials
• AI-driven inverse design of materials: Past, present and future
• The Evolution of Machine Learning Potentials for Molecules, Reactions and Materials
• Machine Learning - Driven Materials Discovery: Unlocking Next-Generation Functional Materials - A review
• Advances in modeling complex materials: The rise of neuroevolution potentials
• InvDesFlow: An AI-driven materials inverse design workflow to explore possible high-temperature superconductors
• InvDesFlow-AL: active learning-based workflow for inverse design of functional materials
• Universal machine learning interatomic potentials poised to supplant DFT in modeling general defects in metals and random alloys
• Pre-training, fine-tuning, and distillation (PFD): Automatically generating machine learning force fields from universal models
• Efficient moment tensor machine-learning interatomic potential for accurate description of defects in Ni-Al Alloys
• Crystal structure prediction with host-guided inpainting generation and foundation potentials
• All-in-one foundational models learning across quantum chemical levels
• Reinforcement Fine-Tuning for Materials Design
• Intrinsic structure of relaxor ferroelectrics from first principles