The Transferability Limits of Static Benchmarks

Abstract

Every practical method to solve the Schrödinger equation for interacting many-particle systems introduces approximations. Such methods are therefore plagued by systematic errors. For computational chemistry, it is decisive to quantify the specific error for some system under consideration. Traditionally, the primary resource for such an error assessment have been benchmarking results, usually taken from the literature. However, their transferability to a specific molecular system, and hence, the reliability of the traditional approach always remains uncertain to some degree. In this communication, we elaborate on the shortcomings of this traditional way of static benchmarking by exploiting statistical analyses at the example of one of the largest quantum chemical benchmark sets available. We demonstrate the uncertainty of error estimates in the light of the choice of reference data selected for a benchmark study. To alleviate the issues with static benchmarks, we advocate to rely instead on a rolling and system-focused approach for rigorously quantifying the uncertainty of a quantum chemical result.

arXiv comment16 pages, 3 figures

References in corpus (10)

• Climbing the Density Functional Ladder: Non-Empirical Meta-Generalized Gradient Approximation Designed for Molecules and Solids
• Nonlocal van der Waals density functional: The simpler the better
• New Benchmark Set of Transition-Metal Coordination Reactions for the Assessment of Density Functionals
• The Exploration of Chemical Reaction Networks
• Error-Controlled Exploration of Chemical Reaction Networks with Gaussian Processes
• Systematic Error Estimation for Chemical Reaction Energies
• Calculation of Ligand Dissociation Energies in Large Transition-Metal Complexes
• Gaussian Process-Based Refinement of Dispersion Corrections
• Error Assessment of Computational Models in Chemistry
• Molecule-specific Uncertainty Quantification in Quantum Chemical Studies