Committee neural network potentials control generalization errors and enable active learning

Abstract

It is well known in the field of machine learning that committee models improve accuracy, provide generalization error estimates, and enable active learning strategies. In this work, we adapt these concepts to interatomic potentials based on artificial neural networks. Instead of a single model, multiple models that share the same atomic environment descriptors yield an average that outperforms its individual members as well as a measure of the generalization error in the form of the committee disagreement. We not only use this disagreement to identify the most relevant configurations to build up the model's training set in an active learning procedure, but also monitor and bias it during simulations to control the generalization error. This facilitates the adaptive development of committee neural network potentials and their training sets, while keeping the number of ab initio calculations to a minimum. To illustrate the benefits of this methodology, we apply it to the development of a committee model for water in the condensed phase. Starting from a single reference ab initio simulation, we use active learning to expand into new state points and to describe the quantum nature of the nuclei. The final model, trained on 814 reference calculations, yields excellent results under a range of conditions, from liquid water at ambient and elevated temperatures and pressures to different phases of ice, and the air-water interface - all including nuclear quantum effects. This approach to committee models will enable the systematic development of robust machine learning models for a broad range of systems.

arXiv commentThe following article has been submitted to the Journal of Chemical Physics

References in corpus (34)

• Canonical sampling through velocity-rescaling
• Gaussian Approximation Potentials: the accuracy of quantum mechanics, without the electrons
• Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
• Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics
• ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
• Moment Tensor Potentials: a class of systematically improvable interatomic potentials
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Machine Learning of Accurate Energy-Conserving Molecular Force Fields
• A Spectral Analysis Method for Automated Generation of Quantum-Accurate Interatomic Potentials
• PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments and Partial Charges
• Less is more: sampling chemical space with active learning
• Machine Learning Unifies the Modelling of Materials and Molecules
• Active learning of linearly parametrized interatomic potentials
• Active Learning of Uniformly Accurate Inter-atomic Potentials for Materials Simulation
• Machine Learning Molecular Dynamics for the Simulation of Infrared Spectra
• How van der Waals interactions determine the unique properties of water
• Nuclear quantum effects enter the mainstream
• Efficient stochastic thermostatting of path integral molecular dynamics
• Ab initio thermodynamics of liquid and solid water
• Efficient and Accurate Machine-Learning Interpolation of Atomic Energies in Compositions with Many Species
• Interatomic potentials for ionic systems with density functional accuracy based on charge densities obtained by a neural network
• How to remove the spurious resonances from ring polymer molecular dynamics
• Data-driven learning of total and local energies in elemental boron
• Fast and Accurate Uncertainty Estimation in Chemical Machine Learning
• Automated Fitting of Neural Network Potentials at Coupled Cluster Accuracy: Protonated Water Clusters as Testing Ground
• The Interplay of Structure and Dynamics in the Raman Spectrum of Liquid Water over the Full Frequency and Temperature Range
• Using Gaussian Process Regression to Simulate the Vibrational Raman Spectra of Molecular Crystals
• PiNN: A Python Library for Building Atomic Neural Networks of Molecules and Materials
• High Order Path Integrals Made Easy
• Automatically growing global reactive neural network potential energy surfaces: a trajectory free active learning strategy
• High-dimensional neural network potentials for solvation: The case of protonated water clusters in helium
• Converged Colored Noise Path Integral Molecular Dynamics Study of the Zundel Cation down to Ultra-low Temperatures at Coupled Cluster Accuracy
• Quantum Nature of the Hydrogen Bond from Ambient Conditions down to Ultra-low Temperatures
• Monte Carlo evaluation of the equilibrium isotope effects using the Takahashi-Imada factorization of the Feynman path integral

Cited by in corpus (42)

• The first-principles phase diagram of monolayer nanoconfined water
• Self-Consistent Determination of Long-Range Electrostatics in Neural Network Potentials
• Uncertainty estimation for molecular dynamics and sampling
• Strategies for the Construction of Machine-Learning Potentials for Accurate and Efficient Atomic-Scale Simulations
• Tutorial: How to Train a Neural Network Potential
• Ephemeral data derived potentials for random structure search
• Differentiable sampling of molecular geometries with uncertainty-based adversarial attacks
• A complete description of thermodynamic stabilities of molecular crystals
• Global optimization of atomistic structure enhanced by machine learning
• Fast Uncertainty Estimates in Deep Learning Interatomic Potentials
• Accurate large-scale simulations of siliceous zeolites by neural network potentials
• Deep Ensembles vs. Committees for Uncertainty Estimation in Neural-Network Force Fields: Comparison and Application to Active Learning
• Spectroscopy from Machine Learning by Accurately Representing the Atomic Polar Tensor
• Uncertainty-biased molecular dynamics for learning uniformly accurate interatomic potentials
• Molecular dynamics simulation with finite electric fields using Perturbed Neural Network Potentials
• Predicting Properties of Periodic Systems from Cluster Data: A Case Study of Liquid Water
• Developments and Further Applications of Ephemeral Data Derived Potentials
• Fully First-Principles Surface Spectroscopy with Machine Learning
• Modeling the Ga/As binary system across temperaturesand compositions from first principles
• Reducing the cost of neural network potential generation for reactive molecular systems
• Assessment of Neural Network Augmented Reynolds Averaged Navier Stokes Turbulence Model in Extrapolation Modes
• A machine learning potential for simulating infrared spectra of nanosilicate clusters
• Random sampling versus active learning algorithms for machine learning potentials of quantum liquid water
• Machine learning potentials for complex aqueous systems made simple
• The Wetting of H${}_2$O by CO${}_2$
• Gaussian Process Regression Adaptive Density-Guided Approach: Towards Calculations of Potential Energy Surfaces for Larger Molecules
• Efficient Generation of Stable Linear Machine-Learning Force Fields with Uncertainty-Aware Active Learning
• Efficient ensemble uncertainty estimation in Gaussian Processes Regression
• Efficient dataset generation for machine learning perovskite alloys
• On the Quality of Uncertainty Estimates from Neural Network Potential Ensembles
• Apax: A Flexible and Performant Framework For The Development of Machine-Learned Interatomic Potentials
• On the increase of the melting temperature of water confined in one-dimensional nano-cavities
• Atomistic Simulations of Oxide-Water Interfaces using Machine Learning Potentials
• Modeling Chemical Exfoliation of Non-van der Waals Chromium Sulfides by Machine Learning Interatomic Potentials and Monte Carlo Simulations
• Multi-head committees enable direct uncertainty prediction for atomistic foundation models
• Neural Network Interaction Potentials for para-Hydrogen with Flexible Molecules
• Free energy profiles for chemical reactions in solution from high-dimensional neural network potentials: The case of the Strecker synthesis
• Structure and thermodynamics of defects in Na-feldspar from a neural network potential
• Accurate Machine Learning Interatomic Potentials for Polyacene Molecular Crystals: Application to Single Molecule Host-Guest Systems
• Machine-Learned Potentials for Solvation Modeling
• Probing the Temporal Response of Liquid Water to a THz Pump Pulse Using Machine Learning-Accelerated Non-Equilibrium Molecular Dynamics
• Scalable Training of Neural Network Potentials for Complex Interfaces Through Data Augmentation