Lifelong Machine Learning Potentials

Abstract

Machine learning potentials (MLPs) trained on accurate quantum chemical data can retain the high accuracy, while inflicting little computational demands. On the downside, they need to be trained for each individual system. In recent years, a vast number of MLPs has been trained from scratch because learning additional data typically requires to train again on all data to not forget previously acquired knowledge. Additionally, most common structural descriptors of MLPs cannot represent efficiently a large number of different chemical elements. In this work, we tackle these problems by introducing element-embracing atom-centered symmetry functions (eeACSFs) which combine structural properties and element information from the periodic table. These eeACSFs are a key for our development of a lifelong machine learning potential (lMLP). Uncertainty quantification can be exploited to transgress a fixed, pre-trained MLP to arrive at a continuously adapting lMLP, because a predefined level of accuracy can be ensured. To extend the applicability of an lMLP to new systems, we apply continual learning strategies to enable autonomous and on-the-fly training on a continuous stream of new data. For the training of deep neural networks, we propose the continual resilient (CoRe) optimizer and incremental learning strategies relying on rehearsal of data, regularization of parameters, and the architecture of the model.

20 pages, 6 figures

References in corpus (9)

• Array Programming with NumPy
• ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
• Machine learning for molecular simulation
• Machine Learning Unifies the Modelling of Materials and Molecules
• How van der Waals interactions determine the unique properties of water
• Efficient and Accurate Machine-Learning Interpolation of Atomic Energies in Compositions with Many Species
• Training neural nets to learn reactive potential energy surfaces using interactive quantum chemistry in virtual reality
• Uncertainty estimation for molecular dynamics and sampling
• Quantum chemical roots of machine-learning molecular similarity descriptors

Cited by in corpus (11)

• Navigating chemical reaction space with a steering wheel
• SCINE -- Software for Chemical Interaction Networks
• FeNNol: an Efficient and Flexible Library for Building Force-field-enhanced Neural Network Potentials
• Automated Microsolvation for Minimum Energy Path Construction in Solution
• CoRe Optimizer: An All-in-One Solution for Machine Learning
• Machine Learning Enhanced Calculation of Quantum-Classical Binding Free Energies
• Hierarchical quantum embedding by machine learning for large molecular assemblies
• Apax: A Flexible and Performant Framework For The Development of Machine-Learned Interatomic Potentials
• Lifelong Machine Learning Potentials for Chemical Reaction Network Explorations
• Modal Backflow Neural Quantum States for Anharmonic Vibrational Calculations
• Solving intractable chemical problems by tensor decomposition