Lifelong Machine Learning Potentials

Abstract

Machine learning potentials (MLPs) trained on accurate quantum chemical data can retain the high accuracy, while inflicting little computational demands. On the downside, they need to be trained for each individual system. In recent years, a vast number of MLPs has been trained from scratch because learning additional data typically requires to train again on all data to not forget previously acquired knowledge. Additionally, most common structural descriptors of MLPs cannot represent efficiently a large number of different chemical elements. In this work, we tackle these problems by introducing element-embracing atom-centered symmetry functions (eeACSFs) which combine structural properties and element information from the periodic table. These eeACSFs are a key for our development of a lifelong machine learning potential (lMLP). Uncertainty quantification can be exploited to transgress a fixed, pre-trained MLP to arrive at a continuously adapting lMLP, because a predefined level of accuracy can be ensured. To extend the applicability of an lMLP to new systems, we apply continual learning strategies to enable autonomous and on-the-fly training on a continuous stream of new data. For the training of deep neural networks, we propose the continual resilient (CoRe) optimizer and incremental learning strategies relying on rehearsal of data, regularization of parameters, and the architecture of the model.

arXiv comment20 pages, 6 figures

References in corpus (40)

• Array Programming with NumPy
• Gaussian Approximation Potentials: the accuracy of quantum mechanics, without the electrons
• On representing chemical environments
• ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
• Perspective on density functional theory
• Moment Tensor Potentials: a class of systematically improvable interatomic potentials
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Progressive Neural Networks
• PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments and Partial Charges
• Continual Learning Through Synaptic Intelligence
• A Universal Graph Deep Learning Interatomic Potential for the Periodic Table
• A Performance and Cost Assessment of Machine Learning Interatomic Potentials
• Machine learning for molecular simulation
• Less is more: sampling chemical space with active learning
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• Machine Learning Unifies the Modelling of Materials and Molecules
• Active learning of linearly parametrized interatomic potentials
• A Fourth-Generation High-Dimensional Neural Network Potential with Accurate Electrostatics Including Non-local Charge Transfer
• Physics-inspired structural representations for molecules and materials
• How van der Waals interactions determine the unique properties of water
• Machine learning for electronically excited states of molecules
• Ab initio thermodynamics of liquid and solid water
• SpookyNet: Learning Force Fields with Electronic Degrees of Freedom and Nonlocal Effects
• Efficient and Accurate Machine-Learning Interpolation of Atomic Energies in Compositions with Many Species
• Alchemical and structural distribution based representation for improved QML
• Interatomic potentials for ionic systems with density functional accuracy based on charge densities obtained by a neural network
• WACSF - Weighted Atom-Centered Symmetry Functions as Descriptors in Machine Learning Potentials
• MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields
• De novo exploration and self-guided learning of potential-energy surfaces
• Machine learning enables long time scale molecular photodynamics simulations
• Perspective on integrating machine learning into computational chemistry and materials science
• Fast and Accurate Uncertainty Estimation in Chemical Machine Learning
• Neural Network Potentials for Chemistry: Concepts, Applications and Prospects
• Training neural nets to learn reactive potential energy surfaces using interactive quantum chemistry in virtual reality
• Uncertainty estimation for molecular dynamics and sampling
• Lifelong Learning of Spatiotemporal Representations with Dual-Memory Recurrent Self-Organization
• Insights into lithium manganese oxide-water interfaces using machine learning potentials
• Closing the gap between theory and experiment for lithium manganese oxide spinels using a high-dimensional neural network potential
• Predicting Oxidation and Spin States by High-Dimensional Neural Networks: Applications to Lithium Manganese Oxide Spinels
• Quantum chemical roots of machine-learning molecular similarity descriptors

Cited by in corpus (11)

• Navigating chemical reaction space with a steering wheel
• SCINE -- Software for Chemical Interaction Networks
• FeNNol: an Efficient and Flexible Library for Building Force-field-enhanced Neural Network Potentials
• Automated Microsolvation for Minimum Energy Path Construction in Solution
• CoRe Optimizer: An All-in-One Solution for Machine Learning
• Machine Learning Enhanced Calculation of Quantum-Classical Binding Free Energies
• Hierarchical quantum embedding by machine learning for large molecular assemblies
• Apax: A Flexible and Performant Framework For The Development of Machine-Learned Interatomic Potentials
• Lifelong Machine Learning Potentials for Chemical Reaction Network Explorations
• Modal Backflow Neural Quantum States for Anharmonic Vibrational Calculations
• Solving intractable chemical problems by tensor decomposition