On the Quality of Uncertainty Estimates from Neural Network Potential Ensembles

Abstract

Neural network potentials (NNPs) combine the computational efficiency of classical interatomic potentials with the high accuracy and flexibility of the ab initio methods used to create the training set, but can also result in unphysical predictions when employed outside their training set distribution. Estimating the epistemic uncertainty of an NNP is required in active learning or on-the-fly generation of potentials. Inspired from their use in other machine-learning applications, NNP ensembles have been used for uncertainty prediction in several studies, with the caveat that ensembles do not provide a rigorous Bayesian estimate of the uncertainty. To test whether NNP ensembles provide accurate uncertainty estimates, we train such ensembles in four different case studies, and compare the predicted uncertainty with the errors on out-of-distribution validation sets. Our results indicate that NNP ensembles are often overconfident, underestimating the uncertainty of the model, and require to be calibrated for each system and architecture. We also provide evidence that Bayesian NNPs, obtained by sampling the posterior distribution of the model parameters using Monte-Carlo techniques, can provide better uncertainty estimates.

References in corpus (9)

• Quantum ESPRESSO: a modular and open-source software project for quantum simulations of materials
• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• DP-GEN: A concurrent learning platform for the generation of reliable deep learning based potential energy models
• A Performance and Cost Assessment of Machine Learning Interatomic Potentials
• Physics-inspired structural representations for molecules and materials
• Committee neural network potentials control generalization errors and enable active learning
• Unsupervised machine learning in atomistic simulations, between predictions and understanding
• Deep Potential generation scheme and simulation protocol for the Li10GeP2S12-type superionic conductors
• Fast Neural Network Approach for Direct Covariant Forces Prediction in Complex Multi-Element Extended Systems