Probabilistic performance estimators for computational chemistry methods: Systematic Improvement Probability and Ranking Probability Matrix. II. Applications

Abstract

In the first part of this study (Paper I), we introduced the systematic improvement probability (SIP) as a tool to assess the level of improvement on absolute errors to be expected when switching between two computational chemistry methods. We developed also two indicators based on robust statistics to address the uncertainty of ranking in computational chemistry benchmarks: Pinv , the inversion probability between two values of a statistic, and Pr , the ranking probability matrix. In this second part, these indicators are applied to nine data sets extracted from the recent benchmarking literature. We illustrate also how the correlation between the error sets might contain useful information on the benchmark dataset quality, notably when experimental data are used as reference.

References in corpus (5)

• Machine Learning of Molecular Electronic Properties in Chemical Compound Space
• Reliable estimation of prediction uncertainty for physico-chemical property models
• Choosing a density functional for static molecular polarizabilities
• Probabilistic performance estimators for computational chemistry methods: the empirical cumulative distribution function of absolute errors
• Probabilistic performance estimators for computational chemistry methods: Systematic Improvement Probability and Ranking Probability Matrix. I. Theory

Cited by in corpus (4)

• Impact of non-normal error distributions on the benchmarking and ranking of Quantum Machine Learning models
• Probabilistic performance estimators for computational chemistry methods: Systematic Improvement Probability and Ranking Probability Matrix. I. Theory
• Using the Gini coefficient to characterize the shape of computational chemistry error distributions
• Acknowledging user requirements for accuracy in computational chemistry benchmarks