Deep Ensembles vs. Committees for Uncertainty Estimation in Neural-Network Force Fields: Comparison and Application to Active Learning

Abstract

A reliable uncertainty estimator is a key ingredient in the successful use of machine-learning force fields for predictive calculations. Important considerations are correlation with error, overhead during training and inference, and efficient workflows to systematically improve the force field. However, in the case of neural-network force fields, simple committees are often the only option considered due to their easy implementation. Here we present a generalization of the deep-ensemble design, based on multiheaded neural networks and a heteroscedastic loss, that can efficiently deal with uncertainties in both the energy and the forces. We compare uncertainty metrics based on deep ensembles, committees and bootstrap-aggregation ensembles using data for an ionic liquid and a perovskite surface. We demonstrate an adversarial approach to active learning to efficiently and progressively refine the force fields. That active learning workflow is realistically possible thanks to exceptionally fast training enabled by residual learning and a nonlinear learned optimizer.

References in corpus (27)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Neural Message Passing for Quantum Chemistry
• On representing chemical environments
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• A real-space grid implementation of the Projector Augmented Wave method
• E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
• Machine Learning of Accurate Energy-Conserving Molecular Force Fields
• Less is more: sampling chemical space with active learning
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Active Learning of Uniformly Accurate Inter-atomic Potentials for Materials Simulation
• SchNet: A continuous-filter convolutional neural network for modeling quantum interactions
• Committee neural network potentials control generalization errors and enable active learning
• Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations
• Fast and Accurate Uncertainty Estimation in Chemical Machine Learning
• Reinforced dynamics for enhanced sampling in large atomic and molecular systems
• Differentiable sampling of molecular geometries with uncertainty-based adversarial attacks
• Fast Uncertainty Estimates in Deep Learning Interatomic Potentials
• Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs
• Gauge invariance of thermal transport coefficients
• On the Limitations of Representing Functions on Sets
• Learning in High Dimension Always Amounts to Extrapolation
• Low Temperature Specific Heat of Doped SrTiO${}_3$: Doping Dependence of the Effective Mass and Kadowaki-Woods Scaling Violation
• Enhancing approximation abilities of neural networks by training derivatives
• VeLO: Training Versatile Learned Optimizers by Scaling Up
• Filtering a distribution simultaneously in real and Fourier space
• Equilibrium Aggregation: Encoding Sets via Optimization

Cited by in corpus (8)

• Machine-learning-accelerated simulations to enable automatic surface reconstruction
• Uncertainty-biased molecular dynamics for learning uniformly accurate interatomic potentials
• Uncertainty Quantification in Atomistic Simulations of Silicon using Interatomic Potentials
• Efficient ensemble uncertainty estimation in Gaussian Processes Regression
• Apax: A Flexible and Performant Framework For The Development of Machine-Learned Interatomic Potentials
• Replica exchange nested sampling
• Multi-head committees enable direct uncertainty prediction for atomistic foundation models
• Active learning potentials for first-principles phase diagrams using replica-exchange nested sampling