On the Quality of Uncertainty Estimates from Neural Network Potential Ensembles

Abstract

Neural network potentials (NNPs) combine the computational efficiency of classical interatomic potentials with the high accuracy and flexibility of the ab initio methods used to create the training set, but can also result in unphysical predictions when employed outside their training set distribution. Estimating the epistemic uncertainty of an NNP is required in active learning or on-the-fly generation of potentials. Inspired from their use in other machine-learning applications, NNP ensembles have been used for uncertainty prediction in several studies, with the caveat that ensembles do not provide a rigorous Bayesian estimate of the uncertainty. To test whether NNP ensembles provide accurate uncertainty estimates, we train such ensembles in four different case studies, and compare the predicted uncertainty with the errors on out-of-distribution validation sets. Our results indicate that NNP ensembles are often overconfident, underestimating the uncertainty of the model, and require to be calibrated for each system and architecture. We also provide evidence that Bayesian NNPs, obtained by sampling the posterior distribution of the model parameters using Monte-Carlo techniques, can provide better uncertainty estimates.

References in corpus (18)

• Quantum ESPRESSO: a modular and open-source software project for quantum simulations of materials
• Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties
• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics
• SchNet - a deep learning architecture for molecules and materials
• Interatomic potentials from first-principles calculations: the force-matching method
• DP-GEN: A concurrent learning platform for the generation of reliable deep learning based potential energy models
• A Performance and Cost Assessment of Machine Learning Interatomic Potentials
• Towards Exact Molecular Dynamics Simulations with Machine-Learned Force Fields
• Physics-inspired structural representations for molecules and materials
• Active Learning of Uniformly Accurate Inter-atomic Potentials for Materials Simulation
• Ab initio thermodynamics of liquid and solid water
• Interatomic potentials for ionic systems with density functional accuracy based on charge densities obtained by a neural network
• Committee neural network potentials control generalization errors and enable active learning
• Unsupervised machine learning in atomistic simulations, between predictions and understanding
• Deep Potential generation scheme and simulation protocol for the Li10GeP2S12-type superionic conductors
• Convolutional neural networks for atomistic systems
• Fast Neural Network Approach for Direct Covariant Forces Prediction in Complex Multi-Element Extended Systems