Prediction uncertainty validation for computational chemists
arXiv:2204.13477 · doi:10.1063/5.0109572
Abstract
Validation of prediction uncertainty (PU) is becoming an essential task for modern computational chemistry. Designed to quantify the reliability of predictions in meteorology, the calibration-sharpness (CS) framework is now widely used to optimize and validate uncertainty-aware machine learning (ML) methods. However, its application is not limited to ML and it can serve as a principled framework for any PU validation. The present article is intended as a step-by-step introduction to the concepts and techniques of PU validation in the CS framework, adapted to the specifics of computational chemistry. The presented methods range from elementary graphical checks to more sophisticated ones based on local calibration statistics. The concept of tightness, is introduced. The methods are illustrated on synthetic datasets and applied to uncertainty quantification data extracted from the computational chemistry literature.
Accepted for publication in J. Chem. Phys
References in corpus (12)
- Uncertainty estimation for molecular dynamics and sampling
- Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification
- Molecule-specific Uncertainty Quantification in Quantum Chemical Studies
- The long road to calibrated prediction uncertainty in computational chemistry
- Impact of non-normal error distributions on the benchmarking and ranking of Quantum Machine Learning models
- Evaluating Forecasts with scoringutils in R
- Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantification
- Using the Gini coefficient to characterize the shape of computational chemistry error distributions
- Parametric and Multivariate Uncertainty Calibration for Regression and Object Detection
- Confidence curves for UQ validation: probabilistic reference vs. oracle
- Heuristics and Uncertainty Quantification in Rational and Inverse Compound and Catalyst Design
- Robust and scalable uncertainty estimation with conformal prediction for machine-learned interatomic potentials