Reliable estimation of prediction uncertainty for physico-chemical property models

Abstract

The predictions of parameteric property models and their uncertainties are sensitive to systematic errors such as inconsistent reference data, parametric model assumptions, or inadequate computational methods. Here, we discuss the calibration of property models in the light of bootstrapping, a sampling method akin to Bayesian inference that can be employed for identifying systematic errors and for reliable estimation of the prediction uncertainty. We apply bootstrapping to assess a linear property model linking the 57Fe Moessbauer isomer shift to the contact electron density at the iron nucleus for a diverse set of 44 molecular iron compounds. The contact electron density is calculated with twelve density functionals across Jacob's ladder (PWLDA, BP86, BLYP, PW91, PBE, M06-L, TPSS, B3LYP, B3PW91, PBE0, M06, TPSSh). We provide systematic-error diagnostics and reliable, locally resolved uncertainties for isomer-shift predictions. Pure and hybrid density functionals yield average prediction uncertainties of 0.06-0.08 mm/s and 0.04-0.05 mm/s, respectively, the latter being close to the average experimental uncertainty of 0.02 mm/s. Furthermore, we show that both model parameters and prediction uncertainty depend significantly on the composition and number of reference data points. Accordingly, we suggest that rankings of density functionals based on performance measures (e.g., the coefficient of correlation, r2, or the root-mean-square error, RMSE) should not be inferred from a single data set. This study presents the first statistically rigorous calibration analysis for theoretical Moessbauer spectroscopy, which is of general applicability for physico-chemical property models and not restricted to isomer-shift predictions. We provide the statistically meaningful reference data set MIS39 and a new calibration of the isomer shift based on the PBE0 functional.

49 pages, 9 figures, 7 tables

References in corpus (6)

• Predicting Electronic Structure Properties of Transition Metal Complexes with Neural Networks
• Towards quantifying the role of exact exchange in predictions of transition metal complex properties
• Systematic Error Estimation for Chemical Reaction Energies
• A critical review of statistical calibration/prediction models handling data inconsistency and model inadequacy
• Where Does the Density Localize? Convergent Behavior for Global Hybrids, Range Separation, and DFT+U
• The parameters uncertainty inflation fallacy

Cited by in corpus (28)

• Exploration of Reaction Pathways and Chemical Transformation Networks
• The Exploration of Chemical Reaction Networks
• Uncertainty Quantification in Classical Molecular Dynamics
• Bayesian machine learning for quantum molecular dynamics
• Autonomous Reaction Network Exploration in Homogeneous and Heterogeneous Catalysis
• Molassembler: Molecular graph construction, modification and conformer generation for inorganic and organic molecules
• Semiempirical Molecular Orbital Models based on the Neglect of Diatomic Differential Overlap Approximation
• Gaussian Process-Based Refinement of Dispersion Corrections
• Mechanism Deduction from Noisy Chemical Reaction Networks
• The parameters uncertainty inflation fallacy
• Probabilistic performance estimators for computational chemistry methods: the empirical cumulative distribution function of absolute errors
• Self-Parametrizing System-Focused Atomistic Models
• Gaussian process model of 51-dimensional potential energy surface for protonated imidazole dimer
• The long road to calibrated prediction uncertainty in computational chemistry
• Statistical Analysis of Semiclassical Dispersion Corrections
• Molecule-specific Uncertainty Quantification in Quantum Chemical Studies
• Impact of non-normal error distributions on the benchmarking and ranking of Quantum Machine Learning models
• Probabilistic performance estimators for computational chemistry methods: Systematic Improvement Probability and Ranking Probability Matrix. I. Theory
• Prediction uncertainty validation for computational chemists
• Using the Gini coefficient to characterize the shape of computational chemistry error distributions
• Probabilistic performance estimators for computational chemistry methods: Systematic Improvement Probability and Ranking Probability Matrix. II. Applications
• Molecular relaxation by reverse diffusion with time step prediction
• Evaluating Scalable Uncertainty Estimation Methods for DNN-Based Molecular Property Prediction
• Striking the Right Balance of Encoding Electron Correlation in the Hamiltonian and the Wavefunction Ansatz
• Ultra-fast Spectroscopy for High-Throughput and Interactive Quantum Chemistry
• Automated Preparation of Nanoscopic Structures: Graph-Based Sequence Analysis, Mismatch Detection, and pH-Consistent Protonation with Uncertainty Estimates
• Heuristics and Uncertainty Quantification in Rational and Inverse Compound and Catalyst Design
• Data-driven Uncertainty Quantification for Systematic Coarse-grained Models