The long road to calibrated prediction uncertainty in computational chemistry

Abstract

Uncertainty quantification (UQ) in computational chemistry (CC) is still in its infancy. Very few CC methods are designed to provide a confidence level on their predictions, and most users still rely improperly on the mean absolute error as an accuracy metric. The development of reliable uncertainty quantification methods is essential, notably for computational chemistry to be used confidently in industrial processes. A review of the CC-UQ literature shows that there is no common standard procedure to report nor validate prediction uncertainty. I consider here analysis tools using concepts (calibration and sharpness) developed in meteorology and machine learning for the validation of probabilistic forecasters. These tools are adapted to CC-UQ and applied to datasets of prediction uncertainties provided by composite methods, Bayesian Ensembles methods, machine learning and a posteriori statistical methods.

References in corpus (5)

• Putting Density Functional Theory to the Test in Machine-Learning-Accelerated Materials Discovery
• Quantifying Confidence in Density Functional Theory Predicted Magnetic Ground States
• Using the Gini coefficient to characterize the shape of computational chemistry error distributions
• From Prescriptive to Predictive: an Interdisciplinary Perspective on the Future of Computational Chemistry
• Revisiting "What Every Computer Scientist Should Know About Floating-point Arithmetic"

Cited by in corpus (4)

• Universal QM/MM Approaches for General Nanoscale Applications
• Uncertainty-biased molecular dynamics for learning uniformly accurate interatomic potentials
• Prediction uncertainty validation for computational chemists
• Calibration in Machine Learning Uncertainty Quantification: beyond consistency to target adaptivity