Error Bounds of the Invariant Statistics in Machine Learning of Ergodic Itô Diffusions
arXiv:2105.10102 · doi:10.1016/j.physd.2021.133022
Abstract
This paper studies the theoretical underpinnings of machine learning of ergodic Itô diffusions. The objective is to understand the convergence properties of the invariant statistics when the underlying system of stochastic differential equations (SDEs) is empirically estimated with a supervised regression framework. Using the perturbation theory of ergodic Markov chains and the linear response theory, we deduce a linear dependence of the errors of one-point and two-point invariant statistics on the error in the learning of the drift and diffusion coefficients. More importantly, our study shows that the usual -norm characterization of the learning generalization error is insufficient for achieving this linear dependence result. We find that sufficient conditions for such a linear dependence result are through learning algorithms that produce a uniformly Lipschitz and consistent estimator in the hypothesis space that retains certain characteristics of the drift coefficients, such as the usual linear growth condition that guarantees the existence of solutions of the underlying SDEs. We examine these conditions on two well-understood learning algorithms: the kernel-based spectral regression method and the shallow random neural networks with the ReLU activation function.
arXiv comment34 pages
References in corpus (19)
- User-friendly tail bounds for sums of random matrices
- Towards Exact Molecular Dynamics Simulations with Machine-Learned Force Fields
- Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data
- Consistency of spectral clustering
- A simple framework to justify linear response theory
- Nonparametric forecasting of low-dimensional dynamical systems
- A discrete approach to stochastic parametrization and dimensional reduction in nonlinear dynamics
- Learning dynamical systems from data: a simple cross-validation perspective
- Data-driven Discovery of Closure Models
- Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques
- Permutationally Invariant, Reproducing Kernel-Based Potential Energy Surfaces for Polyatomic Molecules: From Formaldehyde to Acetone
- Non-parametric Estimation of Stochastic Differential Equations with Sparse Gaussian Processes
- Nonparametric statistical inference for drift vector fields of multi-dimensional diffusions
- Linear response, or else
- Kernel-based Prediction of Non-Markovian Time Series
- Parametric Reduced Models for the Nonlinear Schrödinger Equation
- Modeling of Missing Dynamical Systems: Deriving Parametric Models using a Nonparametric Framework
- A Parameter Estimation Method Using Linear Response Statistics: Numerical Scheme
- A Parameter Estimation Method Using Linear Response Statistics