Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensembles

Abstract

Neural networks (NNs) often assign high confidence to their predictions, even for points far out-of-distribution, making uncertainty quantification (UQ) a challenge. When they are employed to model interatomic potentials in materials systems, this problem leads to unphysical structures that disrupt simulations, or to biased statistics and dynamics that do not reflect the true physics. Differentiable UQ techniques can find new informative data and drive active learning loops for robust potentials. However, a variety of UQ techniques, including newly developed ones, exist for atomistic simulations and there are no clear guidelines for which are most effective or suitable for a given case. In this work, we examine multiple UQ schemes for improving the robustness of NN interatomic potentials (NNIPs) through active learning. In particular, we compare incumbent ensemble-based methods against strategies that use single, deterministic NNs: mean-variance estimation, deep evidential regression, and Gaussian mixture models. We explore three datasets ranging from in-domain interpolative learning to more extrapolative out-of-domain generalization challenges: rMD17, ammonia inversion, and bulk silica glass. Performance is measured across multiple metrics relating model error to uncertainty. Our experiments show that none of the methods consistently outperformed each other across the various metrics. Ensembling remained better at generalization and for NNIP robustness; MVE only proved effective for in-domain interpolation, while GMM was better out-of-domain; and evidential regression, despite its promise, was not the preferable alternative in any of the cases. More broadly, cost-effective, single deterministic models cannot yet consistently match or outperform ensembling for uncertainty quantification in NNIPs.

27 pages, 4 figures, Supporting Information (22 pages)

References in corpus (7)

• E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
• Phase transitions of hybrid perovskites simulated by machine-learning force fields trained on-the-fly with Bayesian inference
• How van der Waals interactions determine the unique properties of water
• Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations
• How to validate machine-learned interatomic potentials
• Differentiable sampling of molecular geometries with uncertainty-based adversarial attacks
• Fast Uncertainty Estimates in Deep Learning Interatomic Potentials

Cited by in corpus (12)

• Machine-learning-accelerated simulations to enable automatic surface reconstruction
• Accurate machine learning force fields via experimental and simulation data fusion
• Model-free quantification of completeness, uncertainties, and outliers in atomistic machine learning using information theory
• Uncertainty Quantification Metrics for Deep Regression
• Random sampling versus active learning algorithms for machine learning potentials of quantum liquid water
• Physics-Based Hybrid Machine Learning for Critical Heat Flux Prediction with Uncertainty Quantification
• Efficient ensemble uncertainty estimation in Gaussian Processes Regression
• Enhanced sampling of robust molecular datasets with uncertainty-based collective variables
• Flow Matching for Accelerated Simulation of Atomic Transport in Crystalline Materials
• Free energy profiles for chemical reactions in solution from high-dimensional neural network potentials: The case of the Strecker synthesis
• Quantifying uncertainty in machine learning on nuclear binding energy
• A Simple and Scalable Kernel Density Approach for Reliable Uncertainty Quantification in Atomistic Machine Learning