Black Box Probabilistic Numerics

Abstract

Probabilistic numerics casts numerical tasks, such the numerical solution of differential equations, as inference problems to be solved. One approach is to model the unknown quantity of interest as a random variable, and to constrain this variable using data generated during the course of a traditional numerical method. However, data may be nonlinearly related to the quantity of interest, rendering the proper conditioning of random variables difficult and limiting the range of numerical tasks that can be addressed. Instead, this paper proposes to construct probabilistic numerical methods based only on the final output from a traditional method. A convergent sequence of approximations to the quantity of interest constitute a dataset, from which the limiting quantity of interest can be extrapolated, in a probabilistic analogue of Richardson's deferred approach to the limit. This black box approach (1) massively expands the range of tasks to which probabilistic numerics can be applied, (2) inherits the features and performance of state-of-the-art numerical methods, and (3) enables provably higher orders of convergence to be achieved. Applications are presented for nonlinear ordinary and partial differential equations, as well as for eigenvalue problems-a setting for which no probabilistic numerical methods have yet been developed.

References in corpus (14)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Probabilistic ODE Solvers with Runge-Kutta Means
• Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature
• Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees
• Fast Automatic Bayesian Cubature Using Lattice Sampling
• Probabilistic Line Searches for Stochastic Optimization
• Implicit Probabilistic Integrators for ODEs
• Probabilistic Linear Multistep Methods
• Improving Quadrature for Constrained Integrands
• On the Bayesian Solution of Differential Equations
• Testing whether a Learning Procedure is Calibrated
• Probabilistic Iterative Methods for Linear Systems
• BayesCG As An Uncertainty Aware Version of CG
• A Fourier State Space Model for Bayesian ODE Filters