Fast and Accurate Uncertainty Estimation in Chemical Machine Learning

Abstract

We present a scheme to obtain an inexpensive and reliable estimate of the uncertainty associated with the predictions of a machine-learning model of atomic and molecular properties. The scheme is based on resampling, with multiple models being generated based on sub-sampling of the same training data. The accuracy of the uncertainty prediction can be benchmarked by maximum likelihood estimation, which can also be used to correct for correlations between resampled models, and to improve the performance of the uncertainty estimation by a cross-validation procedure. In the case of sparse Gaussian Process Regression models, this resampled estimator can be evaluated at negligible cost. We demonstrate the reliability of these estimates for the prediction of molecular energetics, and for the estimation of nuclear chemical shieldings in molecular crystals. Extension to estimate the uncertainty in energy differences, forces, or other correlated predictions is straightforward. This method can be easily applied to other machine learning schemes, and will be beneficial to make data-driven predictions more reliable, and to facilitate training-set optimization and active-learning strategies.

References in corpus (22)

• Quantum ESPRESSO: a modular and open-source software project for quantum simulations of materials
• Advanced capabilities for materials modelling with Quantum ESPRESSO
• Gaussian Approximation Potentials: the accuracy of quantum mechanics, without the electrons
• On representing chemical environments
• Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
• All-electron magnetic response with pseudopotentials: NMR chemical shifts
• A General-Purpose Machine Learning Framework for Predicting Properties of Inorganic Materials
• Comparing molecules and solids across structural and alchemical space
• Machine Learning Unifies the Modelling of Materials and Molecules
• Machine-learning based interatomic potential for amorphous carbon
• Machine learning a general purpose interatomic potential for silicon
• Machine Learning Energies of 2 M Elpasolite (ABC${}_2$D${}_6$) Crystals
• ANI-1: A data set of 20M off-equilibrium DFT calculations for organic molecules
• Symmetry-Adapted Machine-Learning for Tensorial Properties of Atomistic Systems
• Automatic Selection of Atomic Fingerprints and Reference Configurations for Machine-Learning Potentials
• Chemical Shifts in Molecular Solids by Machine Learning
• Growth Mechanism and Origin of High $s{p}^3$ Content in Tetrahedral Amorphous Carbon
• Efficient nonparametric $n$-body force fields from machine learning
• When Gaussian Process Meets Big Data: A Review of Scalable GPs
• Equation of state of fluid methane from first principles with machine learning potentials
• Enhancement of DFT-calculations at Petascale: Nuclear Magnetic Resonance, Hybrid Density Functional Theory and Car-Parrinello calculations
• Projector augmented-wave and all-electron calculations across the periodic table: a comparison of structural and energetic properties

Cited by in corpus (47)

• Physics-inspired structural representations for molecules and materials
• Active Learning of Uniformly Accurate Inter-atomic Potentials for Materials Simulation
• Committee neural network potentials control generalization errors and enable active learning
• Learning the electronic density of states in condensed matter
• Machine Learning for Quantum Matter
• Predicting molecular dipole moments by combining atomic partial charges and atomic dipoles
• Uncertainty estimation for molecular dynamics and sampling
• Using Gaussian Process Regression to Simulate the Vibrational Raman Spectra of Molecular Crystals
• Metrics for Benchmarking and Uncertainty Quantification: Quality, Applicability, and a Path to Best Practices for Machine Learning in Chemistry
• A complete description of thermodynamic stabilities of molecular crystals
• i-PI 3.0: a flexible and efficient framework for advanced atomistic simulations
• A Bayesian approach to NMR crystal structure determination
• Universal QM/MM Approaches for General Nanoscale Applications
• Bayesian Force Fields from Active Learning for Simulation of Inter-Dimensional Transformation of Stanene
• Putting Density Functional Theory to the Test in Machine-Learning-Accelerated Materials Discovery
• Deep Ensembles vs. Committees for Uncertainty Estimation in Neural-Network Force Fields: Comparison and Application to Active Learning
• Efficient implementation of atom-density representations
• Accurate machine learning force fields via experimental and simulation data fusion
• Lifelong Machine Learning Potentials
• Simulating solvation and acidity in complex mixtures with first-principles accuracy: the case of CH${}_3$SO${}_3$H and H${}_2$O${}_2$ in phenol
• Towards a mathematical framework to inform Neural Network modelling via Polynomial Regression
• Predicting the phase diagram of titanium dioxide with random search and pattern recognition
• Finite-temperature materials modeling from the quantum nuclei to the hot electrons regime
• The importance of nuclear quantum effects for NMR crystallography
• Prediction uncertainty validation for computational chemists
• Twin Neural Network Regression
• Modeling the Ga/As binary system across temperaturesand compositions from first principles
• chemtrain: Learning Deep Potential Models via Automatic Differentiation and Statistical Physics
• Calibration in Machine Learning Uncertainty Quantification: beyond consistency to target adaptivity
• Thermal conductivity of Li${}_3$PS${}_4$ solid electrolytes with ab initio accuracy
• Active Learning Approach to Optimization of Experimental Control
• Accelerated Bayesian Inference for Molecular Simulations using Local Gaussian Process Surrogate Models
• Operando Characterization and Molecular Simulations Reveal the Growth Kinetics of Graphene on Liquid Copper during Chemical Vapor Deposition
• A deep learning model for chemical shieldings in molecular organic solids including anisotropy
• Prediction rigidities for data-driven chemistry
• Uncertainty Quantification in Atomistic Simulations of Silicon using Interatomic Potentials
• Machine Learning Enhanced Calculation of Quantum-Classical Binding Free Energies
• Machine learning at the atomic-scale
• Electronic Descriptors for Supervised Spectroscopic Predictions
• Morphological evolution via surface diffusion learned by convolutional, recurrent neural networks: extrapolation and prediction uncertainty
• Multi-head committees enable direct uncertainty prediction for atomistic foundation models
• Lifelong Machine Learning Potentials for Chemical Reaction Network Explorations
• Ab initio machine learning in chemical compound space
• Heuristics and Uncertainty Quantification in Rational and Inverse Compound and Catalyst Design
• Surrogate-Based Black-Box Optimization Method for Costly Molecular Properties
• Accurate Machine Learning Interatomic Potentials for Polyacene Molecular Crystals: Application to Single Molecule Host-Guest Systems
• Learning atomic forces from uncertainty-calibrated adversarial attacks