$\Delta$-model correction of Foundation Model based on the models own understanding

Abstract

Foundation models of interatomic potentials, so called universal potentials, may require fine-tuning or residual corrections when applied to specific subclasses of materials. In the present work, we demonstrate how such augmentation can be accomplished via $\Delta$-learning based on the representation already embedded in the universal potentials. The $\Delta$-model introduced is a Gaussian Process Regression (GPR) model and various types of aggregation (global, species-separated, and atomic) of the representation vector are discussed. Employing a specific universal potential, CHGNet [Deng et al., Nat. Mach. Intell. 5, 1031 (2023)], in a global structure optimization setting, we find that it correctly describes the energetics of the "8" Cu oxide, which is an ultra-thin oxide film on Cu(111). The universal potential model even predicts a more favorable structure compared to that discussed in recent DFT-based literature. Moving to sulfur adatom overlayers on Cu(111), Ag(111), and Au(111) the CHGNet model, however, requires corrections. We demonstrate that these are efficiently provided via the GPR-based $\Delta$-model formulated on the CHGNet's own internal atomic embedding representation. The need for corrections is tracked to the scarcity of metal-sulfur atomic environments in the materials project database that CHGNet is trained on leading to an overreliance on sulfur-sulfur atomic environments. Other universal potentials trained on the same data, MACE-MP0, SevenNet-0, and ORB-v2-only-MPtrj show similar behavior, but with varying degrees of error, demonstrating the general need for augmentation schemes for universal potential models.

arXiv comment10 pages, 9 figures

References in corpus (23)

• Gaussian Approximation Potentials: the accuracy of quantum mechanics, without the electrons
• On representing chemical environments
• Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
• E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
• PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments and Partial Charges
• The Open Catalyst 2020 (OC20) Dataset and Community Challenges
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalysts
• GPAW: An open Python package for electronic-structure calculations
• How to quantify energy landscapes of solids
• Efficient global structure optimization with a machine learned surrogate model
• Efficient Bayesian Inference of Atomistic Structure in Complex Functional Materials
• Cormorant: Covariant Molecular Neural Networks
• Experimental and theoretical study of oxygen adsorption structures on Ag(111)
• Accelerating CALYPSO Structure Prediction by Data-driven Learning of Potential Energy Surface
• Deep learning generative model for crystal structure prediction
• IrO2 Surface Complexions Identified Through Machine Learning and Surface Investigations
• Machine-learning accelerated geometry optimization in molecular simulation
• Orb: A Fast, Scalable Neural Network Potential
• Atomistic Structure Learning Algorithm with surrogate energy model relaxation
• Accelerating the prediction of inorganic surfaces with machine learning interatomic potentials
• Augmentation of Universal Potentials for Broad Applications
• Machine-learning enabled optimization of atomic structures using atoms with fractional existence

Cited by in corpus (4)

• Application-specific machine-learned interatomic potentials: exploring the trade-off between DFT convergence, MLIP expressivity, and computational cost
• RAFFLE: Active learning accelerated interface structure prediction
• Active Δ-learning with universal potentials for global structure optimization
• Gradient-based grand canonical optimization enabled by graph neural networks with fractional atomic existence